WO2020199315A1 - 一种光场相机的快速盲标定方法 - Google Patents

一种光场相机的快速盲标定方法 Download PDF

Info

Publication number
WO2020199315A1
WO2020199315A1 PCT/CN2019/087067 CN2019087067W WO2020199315A1 WO 2020199315 A1 WO2020199315 A1 WO 2020199315A1 CN 2019087067 W CN2019087067 W CN 2019087067W WO 2020199315 A1 WO2020199315 A1 WO 2020199315A1
Authority
WO
WIPO (PCT)
Prior art keywords
light field
field camera
calibration
image
depth
Prior art date
Application number
PCT/CN2019/087067
Other languages
English (en)
French (fr)
Inventor
金欣
孙绪福
戴琼海
Original Assignee
清华大学深圳研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学深圳研究生院 filed Critical 清华大学深圳研究生院
Publication of WO2020199315A1 publication Critical patent/WO2020199315A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera

Definitions

  • the invention relates to the field of computer vision and digital image processing, and in particular to a fast blind calibration method of a light field camera 2.0.
  • the biggest difference between a light field camera and a traditional camera is that a micro lens array is added to its structure, which enables the image sensor to record more effective information, including the direction information and position information of the scene. It is because of this structure that the light field camera has many special abilities, such as refocusing after taking a picture, changing the angle of view, and acquiring depth information. Because the light field camera has the characteristics of being able to capture the position and direction information of the light at the same time, it has received extensive attention in the aspects of virtual reality, holographic technology, and three-dimensional reconstruction. In order to obtain a better imaging effect, the light field camera has high requirements for the position of the microlens array, so an accurate light field camera calibration method is required.
  • Light field camera 1.0 refers to a light field camera whose distance from the image sensor to the micro lens array is equal to the focal length of the micro lens
  • light field camera 2.0 refers to a light field camera whose distance from the image sensor to the micro lens array is not equal to the focal length of the micro lens. Since the sub-image under each microlens of the light field camera 1.0 is the integral of each direction of a certain point in the scene, it only describes the angle information, so its sub-images can hardly show the texture information of the real scene; The sub-image of Field Camera 2.0 moderately reduces the angle information, while increasing the position information.
  • the existing light field camera calibration and construction methods are mainly based on the priori of the existing micro lens array structure parameters, using Gaussian formula to calculate the corresponding object distance and image distance and build the corresponding light field camera structure, and then use the image
  • the image recorded by the sensor fine-tunes the position of the microlens array and the object to make the image of the image sensor clear.
  • this method can realize the construction of a light field camera, it is often unable to obtain accurate structural parameters, resulting in a deviation between the actual position of the microlens array and the theoretical position, and the non-optimal imaging recorded by the image sensor.
  • accurate geometric parameters can obtain more accurate results. Therefore, it is of great significance to realize the rapid calibration of the light field camera 2.0 structure.
  • the main purpose of the present invention is to make up for the above-mentioned shortcomings of the prior art, and to propose a fast blind calibration method for a light field camera, which can obtain the geometric parameters of the light field camera 2.0 through a single shot and imaging, and enhance the data in the image processing process And improve the accuracy of functions such as refocusing, depth map acquisition, and viewing angle shift.
  • the present invention proposes a fast blind calibration method of a light field camera, which includes the following steps:
  • A1 Insert a microlens array between the main lens and the image sensor to build the initial light field camera 2.0 structure, and establish the light field camera 2.0 calibration mathematical model according to the imaging process;
  • A2 Use the built-up light field camera 2.0 structure to take pictures of the depth calibration board with more than three layers, and record the image of the calibration board collected by the image sensor, wherein the depth calibration board with more than three layers has three or more Different depths provide block size values at different depth positions;
  • step A3 Based on the overlapping feature of the microlens imaging part under the light-field camera 2.0 structure, combined with the calibration plate image collected in step A2, obtain the block size value at each depth position of the calibration plate image;
  • step A4 Substitute the block size values at different depth positions of the calibration plate image obtained in step A3 into the light field camera 2.0 calibration mathematical model established in step A1, and calculate the geometric parameters of the microlens array to achieve the Fast calibration of light field cameras.
  • step A1 when the initial light field camera 2.0 structure is built, the light emitted by the object in the shooting scene is refracted by the main lens and then imaged on the relay image surface between the main lens and the micro lens array
  • the microlens array performs secondary imaging on the image on the relay image plane and is recorded by the image sensor, wherein the light propagation process satisfies Gaussian imaging, and the corresponding relationship is:
  • u is the distance from the object to the main lens
  • v is the distance between the main lens and the relay image plane
  • F is the focal length of the main lens
  • a is the relay image plane
  • b is the distance from the microlens array to the image sensor
  • f is the focal length of the microlens array.
  • step A1 the light field image under the light field camera 2.0 structure is rendered, and the image collected by each microlens satisfies the relationship:
  • D is the size of the image collected by each microlens
  • p is the size of the non-overlapping area in the center of the image collected by each microlens
  • a is the distance from the relay image plane to the microlens array
  • B is the distance from the microlens array to the image sensor; establish a mathematical model based on the above relationship;
  • step A4 according to the block size values at different depth positions of the calibration plate image obtained in step A3, and the interval between the three different depths, the geometric parameters of the light field camera 2.0 structure are calculated from the following equations:
  • u is the distance from the object to the main lens
  • v is the distance between the main lens and the relay image plane
  • F is the focal length of the main lens
  • ⁇ u is the distance between depth 1 and depth i
  • ⁇ v is the interval between the relay image plane 1 and the relay image plane i
  • p i is the corresponding block size value at the depth i, where i is 2 or 3.
  • Step A3 includes: for the acquired calibration plate image, divide it into three parts according to the depth, firstly estimate the corresponding block size values at the three depth positions, and then upsample the image acquired by the microlens to A predetermined multiple, but by comparing the similar areas between the images collected by adjacent microlenses, the accurate block size value at the corresponding depth position is calculated.
  • the preliminary estimation is performed using the gradient value method, and the size of the block with the smoothest edge is obtained as a rough estimation value.
  • the predetermined multiple is 100 times.
  • the up-sampling is performed using bilinear interpolation.
  • the block similarity algorithm SSIM is used to compare the similar areas. According to the characteristic that the overlap area of adjacent microlens images is an accurate block size value, a more accurate block size can be obtained by comparing the similar areas between adjacent microlens images. value.
  • the calibration board is a stepped calibration board with more than three different depths.
  • the calibration board has a gradual texture structure.
  • the method further includes: using the geometric parameters to perform subsequent light field data processing, including refocusing, viewing angle conversion, and calculating a depth map.
  • step A1 a single shot is taken of the three-layer depth calibration board.
  • the fast blind calibration method of the light field camera of the present invention can accurately acquire the geometric parameters of the light field camera 2.0 structure under the condition of a single shot, and realize the fast calibration of the light field camera.
  • the present invention is not only simple to operate and quick to calibrate, but also can enhance the accuracy of data in the image processing process, and improve the accuracy of functions such as refocusing, depth map acquisition, and viewing angle conversion.
  • the present invention is not only suitable for the situation where some microlens structure parameters are known, but also for blind calibration of the light field without a priori conditions, the present invention can also obtain accurate geometric parameter information to calibrate the light field camera, which solves the problem of The problem of accurately constructing a light field camera with a lens array is highly versatile.
  • the accuracy of the calibration can be improved, thereby improving the accuracy of the geometric parameters.
  • using algorithms based on gradient and structural similarity we can achieve accurate acquisition of the size of the microlens block, further enhancing the reliability of calibration.
  • FIG. 1 is a flowchart of a method for quickly calibrating the structure of a light field camera 2.0 according to a preferred embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of the initial state of the rapid calibration of the light field camera 2.0 structure of the preferred embodiment of the present invention
  • FIG. 3 is a schematic diagram of the imaging and rendering process of the light field camera 2.0 structure of the preferred embodiment of the present invention.
  • FIG. 4 shows the imaging process of the light field camera 2.0 structure of the preferred embodiment of the present invention for objects of different depths (three different depths are listed here).
  • Fig. 5 is a schematic diagram of a calibration plate structure used for rapid calibration of the light field camera 2.0 structure of the preferred embodiment of the present invention.
  • a fast blind calibration method for a light field camera specifically includes the following steps:
  • A1 Insert the micro lens array 3 between the main lens 2 and the image sensor 4 to build the initial light field camera 2.0 structure, and establish the light field camera 2.0 calibration mathematical model according to the imaging process.
  • the light propagation process satisfies Gaussian imaging, and the corresponding relationship can be expressed as:
  • u is the distance between the object 1 and the main lens 2
  • v is the distance between the main lens 2 and the relay image plane 5
  • F is the focal length of the main lens 2
  • a is the middle Following the distance from the image plane 5 to the microlens array 3
  • b is the distance from the microlens array 3 to the image sensor 4
  • f is the focal length of the microlens array 3.
  • a 4088*3070 industrial camera is used for imaging, the microlens array 3 is closely arranged in a regular hexagon and the focal length is 1.33mm, and the main lens 2 is a 20mm fixed focus lens.
  • the light field camera is set up as shown in Figure 2, and the light from the focus plane in the scene is refractionated by the main lens and then focused on the relay image plane 5.
  • the microlens array 3 is placed about 1.6 mm in front of the plane of the image sensor 4.
  • the imaging process of the light field camera 2.0 structure To establish a mathematical model of the calibration process. It mainly includes the rendering theory of the light field camera 2.0 structure: since each microlens records part of the image at the relay image plane 5, the images recorded by adjacent microlenses will have overlapping areas, so the light field camera 2.0 When rendering the light field image under the structure, we need to select the appropriate size of each microlens image.
  • D is the size of the microlens
  • p is the size of the center non-overlapping area we selected.
  • 6 represents the microlens pattern, which is the collected light field data
  • 7 represents the rendered sub-aperture image
  • 8 represents the relay image.
  • ⁇ u is the interval between the depth 1 and the depth i
  • ⁇ v is the interval between the relay image plane 51 and the relay image plane 5i
  • p i is the corresponding block size value at the depth i, where i can take 2 or 3.
  • A2 Use the built-up light field camera 2.0 to take a single shot of the calibration board 10 through the calibration board 10 with a depth of more than three layers and record the calibration board image collected by the image sensor 4.
  • the calibration plate 10 in step A2 has more than three different depths, which can achieve accurate acquisition of block size values at different depth positions. As shown in Figure 5, by adopting a pattern with a gradient change, this change can increase the intensity difference when the block size is not selected properly, thereby enhancing the reliability of the calibration.
  • the calibration plate 10 has a gradual texture structure, which can facilitate us to find the microlens image size at different depths.
  • the calibration plate 10 with a gradual texture structure can enlarge the difference, so that it is easy to judge.
  • the calibration plate 10 is placed at an appropriate distance in front of the light field camera, so that the obtained calibration plate image can fill the field of view of the camera.
  • step A3 Based on the overlapping characteristics of the micro lens imaging part under the light field camera 2.0 structure, combined with the calibration plate image collected in step A2, the block size at each depth position of the calibration plate image is obtained.
  • Step A3 includes: for the light field image of the calibration plate 10 that has been taken, we need to process it separately according to its different depth images. First, the collected calibration plate image is divided into three parts according to the depth, and the corresponding block is obtained. The size value. Since the image is smoother when the microlens block is selected accurately, we use but not limited to the gradient value method to initially estimate the block size at the corresponding depth position.
  • the characteristic of the block size value is to obtain a more accurate block size value by comparing similar areas between adjacent microlens images. There are overlapping image parts between adjacent microlens images, but the position is somewhat different, and the position difference is exactly equal to the size of the block. We compare the adjacent microlens images to find the specific position of the similar area, and then calculate the distance between the initial position and the position of the similar area to obtain the precise block size at this depth.
  • SSIM block similarity algorithm
  • A4 Substitute the block size values at different depth positions of the calibration plate 10 obtained in step A3 into the light field camera 2.0 calibration model established in step A1, and calculate the geometric parameters of the microlens array 3 to achieve rapid calibration of the light field camera.
  • the calculated block size values at different depth positions and the distance difference between different depths in the scene are substituted into the established mathematical model of light field calibration to obtain the geometric parameters of the light field camera 2.0.
  • the acquired geometric parameters of the light field camera 2.0 can also be used in subsequent light field data processing, including refocusing, viewing angle conversion, calculating depth maps, etc., to achieve accurate image processing in subsequent image processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

一种光场相机的快速盲标定方法，包括：在主透镜与图像传感器之间***微透镜阵列以搭建初始的光场相机2.0结构，根据成像过程建立光场相机2.0标定数学模型；利用搭建的光场相机2.0结构对三层以上深度标定板进行拍摄，并记录图像传感器采集的标定板图像，该标定板具有三个或更多个不同的深度，提供不同深度位置处的块尺寸值；基于光场相机2.0结构下微透镜成像部分交叠的特性，结合采集到的标定板图像，获取标定板图像的每个深度位置处的块尺寸值；将块尺寸值代入到光场相机2.0标定数学模型，计算得到微透镜阵列的几何参数，实现对光场相机的快速标定。本发明能够在单次拍摄的条件下准确获取光场相机2.0结构的几何参数，实现对光场相机的快速标定。

Description

一种光场相机的快速盲标定方法 技术领域
本发明涉及计算机视觉与数字图像处理领域，尤其涉及光场相机2.0的一种快速盲标定方法。
背景技术
光场相机不同于传统相机的最大之处在于其结构中加入了微透镜阵列，因而使得图像传感器上能够记录更多的有效信息，包括场景的方向信息和位置信息等。正是由于这种结构，使光场相机具有许多特殊的本领，如拍照后重新聚焦、转换视角、获取深度信息等。由于光场相机具有能够同时捕获光线的位置与方向信息的特点，使得其在虚拟现实、全息技术、三维重建等方面受到广泛的关注。为了获得较好的成像效果，光场相机对于微透镜阵列的位置要求较高，因此需要精准的光场相机标定方法。
光场相机1.0是指图像传感器到微透镜阵列的距离等于微透镜焦距的光场相机，而光场相机2.0指的是图像传感器到微透镜阵列的距离不等于微透镜焦距的光场相机。由于光场相机1.0的每个微透镜下的子图像是场景中某个点的各个方向的积分，只是描述了角度信息，所以它的子图像几乎是看不出真实场景的纹理信息；相反光场相机2.0的子图像适度地减少了角度信息，同时增加了位置信息。
现有的光场相机标定及搭建方法主要是通过在已有微透镜阵列结构参数的先验基础上，利用高斯公式计算出对应的物距及像距并搭建相应光场相机结构，之后利用图像传感器记录的图像微调微透镜阵列和物体的位置来使得图像传感器所成像清晰。虽然这种方法能够实现对光场相机的搭建，但往往不能够获取准确的结构参数导致微透镜阵列实际位置与理论位置有偏移，图像传感器记录的非最佳成像。此外，在对光场采集到的图像进行处理的过程中，准确的几何参数能够获取更加精准的结果，因此实现对光场相机2.0结构的快速标定具有重要意义。
以上背景技术内容的公开仅用于辅助理解本发明的构思及技术方案，其并 不必然属于本专利申请的现有技术，在没有明确的证据表明上述内容在本专利申请的申请日已经公开的情况下，上述背景技术不应当用于评价本申请的新颖性和创造性。
发明内容
本发明的主要目的在于弥补现有技术的上述缺陷，提出一种光场相机的快速盲标定方法，能够通过单次拍摄成像实现对光场相机2.0结构几何参数的获取，增强图像处理过程中数据的准确性，并提升重聚焦、深度图获取、视角转变等功能的精确度。
为此，本发明提出一种光场相机的快速盲标定方法，包括以下步骤：
A1：在主透镜与图像传感器之间***微透镜阵列以搭建初始的光场相机2.0结构，根据成像过程建立光场相机2.0标定数学模型；
A2：利用搭建的所述光场相机2.0结构对三层以上深度标定板进行拍摄，并记录所述图像传感器采集的标定板图像，其中所述三层以上深度标定板具有三个或更多个不同的深度，提供不同深度位置处的块尺寸值；
A3：基于光场相机2.0结构下微透镜成像部分交叠的特性，结合步骤A2采集到的所述标定板图像，获取所述标定板图像的每个深度位置处的块尺寸值；
A4：将步骤A3得到的所述标定板图像的不同深度位置处的块尺寸值代入到步骤A1建立的所述光场相机2.0标定数学模型，计算得到所述微透镜阵列的几何参数，实现对光场相机的快速标定。
进一步地：
步骤A1中在搭建初始的光场相机2.0结构时，使得拍摄场景中的物体发出的光线经过所述主透镜折射后成像于所述主透镜与所述微透镜阵列之间的中继像面上，所述微透镜阵列对所述中继像面上的像进行二次成像并由所述图像传感器记录，其中该光线传播过程满足高斯成像，其对应关系为：
Figure PCTCN2019087067-appb-000001
其中，u是所述物体到所述主透镜的距离，v是所述主透镜与所述中继像面之间的距离，F是所述主透镜的焦距；a是所述中继像面到所述微透镜阵列的 距离，b是所述微透镜阵列到图像传感器的距离，f是所述微透镜阵列的焦距。
步骤A1中，对光场相机2.0结构下的光场图像进行渲染，每个微透镜所采集到的图像满足关系：
Figure PCTCN2019087067-appb-000002
其中，D是每个微透镜所采集到的图像的尺寸，p是每个微透镜所采集到的图像中心不重叠区域的尺寸，a是所述中继像面到所述微透镜阵列的距离，b是所述微透镜阵列到图像传感器的距离；根据以上的关系建立数学模型；
步骤A4中，根据步骤A3获得的所述标定板图像的不同深度位置处的块尺寸值，以及三个不同的深度之间的间隔，由如下方程组求取光场相机2.0结构的几何参数：
Figure PCTCN2019087067-appb-000003
其中u是所述物体到所述主透镜的距离，v是所述主透镜与所述中继像面之间的距离，F是所述主透镜的焦距，Δu是深度1与深度i的间隔，Δv是中继像面1与中继像面i的间隔，p i是深度i时对应的块尺寸值，其中i取2或3。
步骤A3包括:对于采集到的所述标定板图像，根据深度划分为三个部分，首先初步估算所述三个深度位置处对应的块尺寸值,接着将所述微透镜采集的图像上采样至预定倍数，然而通过比较相邻微透镜采集到的图像之间的相似区域，计算出对应深度位置处精确的块尺寸值。
利用梯度值的方法进行所述初步估算，求取边缘最圆滑的块尺寸大小作为粗略的估计值。
所述预定倍数为100倍。
利用双线性插值的方式进行所述上采样。
利用块相似性算法SSIM进行所述相似区域的比较，根据相邻微透镜图像重叠区域大小即为准确的块尺寸值的特性，通过比较相邻微透镜图像间的相似区域获取更精确的块尺寸值。
所述标定板为具有阶梯状的三个以上不同深度的标定板，优选地，所述标 定板上具有渐变的纹理结构。
所述方法还包括：利用所述几何参数进行后续的光场数据处理，包括重聚焦、视角转换、计算深度图。
步骤A1中对所述三层深度标定板进行单次拍摄。
本发明的有益效果包括：
本发明的光场相机的快速盲标定方法，能够在单次拍摄的条件下就实现对光场相机2.0结构的几何参数的准确获取，实现对光场相机的快速标定。本发明不仅操作简单、标定迅捷，而且能够增强图像处理过程中数据的准确性，以及提升重聚焦、深度图获取、视角转变等功能的精确度。本发明不仅适用于已知部分微透镜结构参数的情况，对于无先验条件情况下的光场盲标定，本发明同样能够获取准确的几何参数信息进行光场相机的标定，解决了对任意微透镜阵列准确搭建光场相机的问题，具有很强的通用性。优选方案中，通过使用带有渐变纹理的标定板，能够提升标定的准确性，从而提升几何参数的准确度。此外，利用基于梯度和结构相似性的算法，我们能够实现对微透镜块尺寸的准确获取，进一步增强标定的可靠性。
附图说明
图1是本发明优选实施例的光场相机2.0结构的快速标定方法的流程图；
图2是本发明优选实施例的光场相机2.0结构的快速标定的初始状态结构示意图；
图3是是本发明优选实施例的光场相机2.0结构成像与渲染的过程示意图。
图4是本发明优选实施例的光场相机2.0结构对于不同深度的物体的成像过程(此处列举了三个不同深度)。
图5是本发明优选实施例的光场相机2.0结构的快速标定所使用的标定板结构示意图。
具体实施方式
下面对照附图并结合优选的实施方式对本发明作进一步说明。在具体的实施方案中，可按下面方式操作。需注意的是，在下面的实施过程中的光场相机搭建的结构、微透镜阵列的参数都仅为列举说明，本发明所涵盖的范围不局限 于所列举的这些方法。
参见图1至图5，一种光场相机的快速盲标定方法，具体包括以下步骤：
A1：在主透镜2与图像传感器4之间***微透镜阵列3以搭建初始的光场相机2.0结构，根据成像过程建立光场相机2.0标定数学模型。
步骤A1中搭建初始的光场相机2.0结构时，使得拍摄场景中物体1发出的光线经过主透镜2折射后成像于中继像面5，微透镜阵列3对中继像面5处的像进行二次成像并被图像传感器4记录。如图2所示，该光线传播过程满足高斯成像，其对应关系可以表示为：
Figure PCTCN2019087067-appb-000004
其中，u是物体1到所述主透镜2的距离，v是所述主透镜2与所述中继像面5之间的距离，F是所述主透镜2的焦距；a是所述中继像面5到所述微透镜阵列3的距离，b是所述微透镜阵列3到图像传感器4的距离，f是所述微透镜阵列3的焦距。
在本实施例中，采用4088*3070的工业相机进行成像，微透镜阵列3采用正六边形紧密排布且焦距为1.33mm，主透镜2采用20mm定焦镜头。如图2所示搭建光场相机，场景中的聚焦平面发出的光线经过主镜头的折射后聚焦于中继像面5。将微透镜阵列3置于图像传感器4平面前1.6mm左右。
由于光场相机2.0结构的紧密性与精密性，其几何参数a，b，v无法通过常规测量方法得到，我们结合光场相机2.0结构的成像过程建立标定过程的数学模型。其中主要包括光场相机2.0结构的渲染理论：由于每个微透镜都对中继像面5处的部分像进行记录，相邻微透镜所记录图像会有重叠区域，因此在对光场相机2.0结构下的光场图像进行渲染时，我们需要选取每个微透镜图像的合适尺寸。
如图3所示，我们选取相邻的三个微透镜为例，每个微透镜所采集到的图像尺寸为D，而仅有中间尺寸为p的区域是不重叠区域，这两者之间满足关系：
Figure PCTCN2019087067-appb-000005
其中，D是微透镜的尺寸，p是我们所选取的中心不重叠区域尺寸。
图3中，6表示微透镜图案，为采集的光场数据，7表示渲染后子孔径图像，8表示中继图像。
为了获取光场相机2.0的几何参数，可以根据以上的关系建立数学模型，通过拍摄三个不同深度位置处的图像，如图4所示。在后续步骤中，在已知其不同深度处的块尺寸值的情况下，联立方程组，便可求取光场相机2.0的几何参数，其方程组可以表示为：
Figure PCTCN2019087067-appb-000006
其中Δu是深度1与深度i的间隔，Δv是中继像面51与中继像面5i的间隔，p i是深度i时对应的块尺寸值，其中i可取2或3。
A2：通过具有纹理的三层以上深度的标定板10，利用搭建的光场相机2.0对标定板10单次拍摄，并记录图像传感器4采集的标定板图像。
步骤A2中的标定板10具有三个以上不同的深度，能够实现对不同深度位置处块尺寸值的准确获取。如图5所示，通过采用带有梯度变化的图案，这种变化能够使得当块尺寸选取不当时增大强度差，从而增强标定的可靠性。
在本实施例中，优选采用具有阶梯状的三个以上不同深度的标定板10。更优选地，标定板10上具有渐变的纹理结构，能够方便我们寻找不同深度处的微透镜图像尺寸。当所选取的块尺寸不准确时，具有渐变纹理结构的标定板10能够放大差异，从而易于判断。如图5所示，将标定板10置于光场相机前适当距离处，使得获取到的标定板图像能填满相机的视场。
A3：基于光场相机2.0结构下微透镜成像部分交叠的特性，结合步骤A2采集到的标定板图像，获取标定板图像每个深度位置处的块尺寸大小。
步骤A3包括：对于已拍摄的标定板10的光场图像，我们需要根据其不同的深度图像分别进行处理，首先将采集到的标定板图像根据深度划分为三个部分，求取其对应的块尺寸值。由于当微透镜块选取准确时，图像更加光滑，我们利用但不限于梯度值的方法初步估算对应深度位置处的块尺寸大小。接着为了进一步提高块尺寸的精度，我们通过上采样的方式，将图像上采样100倍， 利用但不限于块相似性算法(SSIM)的方法，根据相邻微透镜图像重叠区域大小即为准确的块尺寸值的特性，通过比较相邻微透镜图像间的相似区域获取更精确的块尺寸值。相邻微透镜图像间有重叠的图像部分，但是位置有些差异，位置的差异正好等于块尺寸的大小。我们通过比较相邻微透镜图像，找到其相似区域的具***置，即可通过计算初始位置与相似区域位置的距离从而获取在该深度情况下精确的块尺寸大小。
在本实施例中，我们将步骤A2所采集到的标定板图像，首先根据深度的不同，将其划分为三个部分，对每一个区域分别计算最优的块尺寸值。我们利用但不限于梯度值的方法求取边缘最圆滑的块尺寸大小作为粗略的估计值，接着利用但不限于双线性插值的方式，将微透镜图像上采样至100倍，然后利用但不限于结构相似性的方法，寻找相邻微透镜图像的相似区域的间隔，计算得到精确的对应深度块尺寸值。
A4：将步骤A3得到的标定板10不同深度位置处的块尺寸值代入到步骤A1建立的光场相机2.0标定模型，计算得到微透镜阵列3的几何参数，实现对光场相机的快速标定。
在本实施例中，将计算得到的不同深度位置处的块尺寸值与不同深度在场景中的距离差代入到建立的光场标定的数学模型当中，求取光场相机2.0的几何参数。我们将标定板不同深度间的间隔以及对应深度情况下光场渲染图像块尺寸的大小分别代入到A1步骤所建立的标定数学模型中，通过求解方程组的方式获取该光场相机2.0的几何参数，从而实现对光场相机2.0结构的快速标定。
此外，获取的光场相机2.0的几何参数也可以运用到后续的光场数据处理当中，包括重聚焦、视角转换、计算深度图等，在后续的图像处理中实现对图像的准确处理。
以上内容是结合具体的/优选的实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，其还可以对这些已描述的实施例做出若干替代或变型，而这些替代或变型方式都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种光场相机的快速盲标定方法，其特征在于，包括以下步骤：
    A1：在主透镜与图像传感器之间***微透镜阵列以搭建初始的光场相机2.0结构，根据成像过程建立光场相机2.0标定数学模型；
    A2：利用搭建的所述光场相机2.0结构对三层以上深度标定板进行拍摄，并记录所述图像传感器采集的标定板图像，其中所述三层以上深度标定板具有三个或更多个不同的深度，提供不同深度位置处的块尺寸值；
    A3：基于光场相机2.0结构下微透镜成像部分交叠的特性，结合步骤A2采集到的所述标定板图像，获取所述标定板图像的每个深度位置处的块尺寸值；
    A4：将步骤A3得到的所述标定板图像的不同深度位置处的块尺寸值代入到步骤A1建立的所述光场相机2.0标定数学模型，计算得到所述微透镜阵列的几何参数，实现对光场相机的快速标定。
  2. 根据权利要求1所述的光场相机的快速盲标定方法，其特征在于，步骤A1中在搭建初始的光场相机2.0结构时，使得拍摄场景中的物体发出的光线经过所述主透镜折射后成像于所述主透镜与所述微透镜阵列之间的中继像面上，所述微透镜阵列对所述中继像面上的像进行二次成像并由所述图像传感器记录，其中该光线传播过程满足高斯成像，其对应关系为：
    Figure PCTCN2019087067-appb-100001
    其中，u是所述物体到所述主透镜的距离，v是所述主透镜与所述中继像面之间的距离，F是所述主透镜的焦距；a是所述中继像面到所述微透镜阵列的距离，b是所述微透镜阵列到图像传感器的距离，f是所述微透镜阵列的焦距。
  3. 根据权利要求1或2所述的光场相机的快速盲标定方法，其特征在于，
    步骤A1中，对光场相机2.0结构下的光场图像进行渲染，每个微透镜所采集到的图像满足关系：
    Figure PCTCN2019087067-appb-100002
    其中，D是每个微透镜所采集到的图像的尺寸，p是每个微透镜所采集到 的图像中心不重叠区域的尺寸，a是所述中继像面到所述微透镜阵列的距离，b是所述微透镜阵列到图像传感器的距离；根据以上的关系建立数学模型；
    步骤A4中，根据步骤A3获得的所述标定板图像的不同深度位置处的块尺寸值，以及三个不同的深度之间的间隔，由如下方程组求取光场相机2.0结构的几何参数：
    Figure PCTCN2019087067-appb-100003
    其中u是所述物体到所述主透镜的距离，v是所述主透镜与所述中继像面之间的距离，F是所述主透镜的焦距，Δu是深度1与深度i的间隔，Δv是中继像面1与中继像面i的间隔，p i是深度i时对应的块尺寸值，其中i取2或3。
  4. 根据权利要求1至3任一项所述的光场相机的快速盲标定方法，其特征在于，步骤A3包括:对于采集到的所述标定板图像，根据深度划分为三个部分，首先初步估算所述三个深度位置处对应的块尺寸值,接着将所述微透镜采集的图像上采样至预定倍数，然后通过比较相邻微透镜采集到的图像之间的相似区域，计算出对应深度位置处精确的块尺寸值。
  5. 根据权利要求4所述的光场相机的快速盲标定方法，其特征在于，利用梯度值的方法进行所述初步估算，求取边缘最圆滑的块尺寸大小作为粗略的估计值。
  6. 根据权利要求4或5所述的光场相机的快速标定方法，其特征在于，所述预定倍数为100倍。
  7. 根据权利要求6所述的光场相机的快速标定方法，其特征在于，利用双线性插值的方式进行所述上采样。
  8. 根据权利要求4至7任一项所述的光场相机的快速盲标定方法，其特征在于，利用块相似性算法SSIM进行所述相似区域的比较，根据相邻微透镜图像重叠区域大小即为准确的块尺寸值的特性，通过比较相邻微透镜图像间的相似区域获取更精确的块尺寸值。
  9. 根据权利要求1至8任一项所述的光场相机的快速盲标定方法，其特征在于，所述标定板为具有阶梯状的三个以上不同深度的标定板，优选地，所述 标定板上具有渐变的纹理结构。
  10. 根据权利要求1至9任一项所述的光场相机的快速盲标定方法，其特征在于，步骤A1中对所述三层深度标定板进行单次拍摄。
PCT/CN2019/087067 2019-04-01 2019-05-15 一种光场相机的快速盲标定方法 WO2020199315A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910257799.5 2019-04-01
CN201910257799.5A CN110009693B (zh) 2019-04-01 2019-04-01 一种光场相机的快速盲标定方法

Publications (1)

Publication Number Publication Date
WO2020199315A1 true WO2020199315A1 (zh) 2020-10-08

Family

ID=67169252

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/087067 WO2020199315A1 (zh) 2019-04-01 2019-05-15 一种光场相机的快速盲标定方法

Country Status (2)

Country Link
CN (1) CN110009693B (zh)
WO (1) WO2020199315A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110473261B (zh) * 2019-08-15 2022-04-19 中国科学院光电技术研究所 一种鲁棒的光场相机标定方法
CN111912606B (zh) * 2020-07-09 2022-05-20 河海大学 一种基于光学折射的相机物距标定方法
CN113115017B (zh) * 2021-03-05 2022-03-18 上海炬佑智能科技有限公司 3d成像模组参数检验方法以及3d成像装置
CN114326273B (zh) * 2022-03-16 2022-05-13 成都工业学院 一种用于光场扩展的投影机阵列定位装置
CN114666573A (zh) * 2022-03-23 2022-06-24 北京拙河科技有限公司 一种光场相机校准方法及***

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1256831A1 (en) * 2001-05-11 2002-11-13 MVTec Software GmbH Method and system for calibrating an image range device
CN103366356A (zh) * 2012-04-06 2013-10-23 北京三星通信技术研究有限公司 通用标定***和方法及其集成成像重构***和方法
CN104050662A (zh) * 2014-05-30 2014-09-17 清华大学深圳研究生院 一种用光场相机一次成像直接获取深度图的方法
CN106842178A (zh) * 2017-01-13 2017-06-13 清华大学深圳研究生院 一种光场距离估计方法与光场成像***

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8189089B1 (en) * 2009-01-20 2012-05-29 Adobe Systems Incorporated Methods and apparatus for reducing plenoptic camera artifacts
TWI508554B (zh) * 2013-05-21 2015-11-11 Univ Nat Taiwan 基於光場相機的影像對焦處理方法及其系統
CN104463949B (zh) * 2014-10-24 2018-02-06 郑州大学 一种基于光场数字重聚焦的快速三维重建方法及其***
CN105488810B (zh) * 2016-01-20 2018-06-29 东南大学 一种聚焦光场相机内外参数标定方法
CN106303228B (zh) * 2016-08-04 2019-09-13 深圳市未来媒体技术研究院 一种聚焦型光场相机的渲染方法和***
CN106373152B (zh) * 2016-09-18 2019-02-01 清华大学深圳研究生院 一种基于手持式光场相机的距离估计方法
CN108051183B (zh) * 2017-11-16 2019-06-28 上海理工大学 基于高斯光学的聚焦型光场相机参数标定方法
CN109325981B (zh) * 2018-09-13 2020-10-02 北京信息科技大学 基于聚焦像点的微透镜阵列型光场相机几何参数标定方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1256831A1 (en) * 2001-05-11 2002-11-13 MVTec Software GmbH Method and system for calibrating an image range device
CN103366356A (zh) * 2012-04-06 2013-10-23 北京三星通信技术研究有限公司 通用标定***和方法及其集成成像重构***和方法
CN104050662A (zh) * 2014-05-30 2014-09-17 清华大学深圳研究生院 一种用光场相机一次成像直接获取深度图的方法
CN106842178A (zh) * 2017-01-13 2017-06-13 清华大学深圳研究生院 一种光场距离估计方法与光场成像***

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
SUN JUNYANG;SUN JUN;XU CHUANLONG;ZHANG BIAO;WANG SHIMIN: "A Calibration Method of Focused Light Field Cameras Based on Light Field Images", ACTA OPTICA SINICA, vol. 37, no. 5, 30 May 2017 (2017-05-30), pages 184 - 194, XP055739919, DOI: 10.3788/aos201737.0515002 *
ZHANG CHUNPING, WANG QING: "Survey on Imaging Model and Calibration of Light Field Camera", CHINESE JOURNAL OF LASERS, vol. 43, no. 6, 30 June 2016 (2016-06-30), pages 270 - 281, XP055739935, DOI: 10.3788/CJL201643.0609004 *

Also Published As

Publication number Publication date
CN110009693B (zh) 2020-12-11
CN110009693A (zh) 2019-07-12

Similar Documents

Publication Publication Date Title
WO2020199315A1 (zh) 一种光场相机的快速盲标定方法
Zhang et al. Zoom to learn, learn to zoom
US11024046B2 (en) Systems and methods for depth estimation using generative models
CN107977997B (zh) 一种结合激光雷达三维点云数据的相机自标定方法
WO2019100933A1 (zh) 用于三维测量的方法、装置以及***
EP3516626B1 (en) Device and method for obtaining distance information from views
TWI510086B (zh) 數位重對焦方法
US8897545B2 (en) Apparatus and method for determining a confidence value of a disparity estimate
WO2016184099A1 (zh) 基于光场数据分布的深度估计方法
WO2018024006A1 (zh) 一种聚焦型光场相机的渲染方法和***
CN107545586B (zh) 基于光场极线平面图像局部的深度获取方法及***
JP5370606B2 (ja) 撮像装置、画像表示方法、及びプログラム
CN107809610A (zh) 摄像头参数集算出装置、摄像头参数集算出方法以及程序
CN108322726A (zh) 一种基于双摄像头的自动对焦方法
CN105659054A (zh) 摄像装置和相位差检测方法
US11950015B2 (en) Method for improved acquisition of images for photogrammetry
US9948909B2 (en) Apparatus and a method for modifying colors of a focal stack of a scene according to a color palette
JP4193292B2 (ja) 多眼式データ入力装置
CN107004274A (zh) 估计未聚焦全光数据的深度的方法和装置
JP6039301B2 (ja) 撮像装置、撮像システム、撮像装置の制御方法、プログラム、および、記憶媒体
TW201537949A (zh) 立體攝影系統及其方法
JP2020194454A (ja) 画像処理装置および画像処理方法、プログラム、並びに記憶媒体
CN110827343B (zh) 基于能量增强散焦响应的改进光场深度估计方法
Li et al. A Bayesian approach to uncertainty-based depth map super resolution
CN113804124A (zh) 一种基于神经网络模型的三维测量方法及***

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19923125

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19923125

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19923125

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22.03.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19923125

Country of ref document: EP

Kind code of ref document: A1