CN106296661A - A kind of demarcation preprocess method being applicable to light-field camera - Google Patents

Abstract

The present invention discloses a kind of demarcation preprocess method being applicable to light-field camera, and described method comprises the steps: S1: white Image semantic classification, obtains the subimage under each lenticule；S2: a given gridiron pattern image, by pretreated to gridiron pattern image and step S1 white image phase and, it is thus achieved that the subimage under appointment lenticule；S3: each subimage obtaining step S2 carries out Corner Detection.In white image procossing, determine lenticular center position first with halation method, then pass through the means lenticular center position of optimization such as linear fit, subimage focus point, surrounding lenticule central point.In Corner Detection, use the detection of Hough line, then calculate the position of angle point, then by a self-defining index continuous iteration optimization angle point, obtain more accurate coordinate position.

Description

A kind of demarcation preprocess method being applicable to light-field camera

Technical field

The invention belongs to camera calibration field, especially relate to a kind of pretreatment side in light-field camera calibration process Method.

Background technology

In recent years, camera family welcomes emerging member's hand-held light-field camera.From principle, it is overturned Traditional camera gather light mode, traditional camera to the directive smooth line integral of certain point in scene, and light-field camera (under What the light-field camera occurred in literary composition referred to is exactly hand-held light-field camera) to the light line integral of all directions of certain point in scene. Functionally, it changes the shooting style of existing traditional camera, and traditional camera is usually first to focus on and takes pictures, and light-field camera Can first take pictures refocusing.From application prospect, it can combine with burning the hotest virtual reality, augmented reality, Hardware supported is provided for them.

Although light-field camera has lot of advantages, but it has a demarcation side the most ripe unlike traditional camera Method.The method the most much demarcated about light-field camera is both for Lytro camera, such as at CVPR, (calculating regarded in 2013 Feel with pattern recognition meeting) on deliver based on 4D ligh field model, between real scene and the data of sensor record set up The scaling method of contact, Cho on ICCV proposed a kind of data preprocessing method from frequency domain angle analysis, afterwards the same year Bok also been proposed use line features localization.The above research is both for light-field camera 1.0 and launches, and light-field camera 1.0 refers to Sensor is to the distance of microlens array equal to the light-field camera of lenticule focal length, and light-field camera 2.0 refers to sensor and arrives The distance of microlens array is not equal to the light-field camera of lenticule focal length.Due to the son under each lenticule of light-field camera 1.0 Image is the integration of all directions of certain point in scene, only describes angle information, so its subimage is almost seen Do not go out the texture information of real scene；The subimage of contrary light-field camera 2.0 moderately decreases angle information, adds simultaneously Positional information.The difference of this principle, result in used some in scaling method proposed for light-field camera 1.0 Preprocessing means is not suitable for light-field camera 2.0, it is therefore necessary to propose light-field camera 2.0 a kind of practical and effective Demarcate preprocess method.

Light-field camera scaling method pretreatment mainly has two steps.The first step, dialogue image processes, and obtains micro- Lens centre point, and dialogue image carries out rotating and balance correction.The main flow means obtaining lenticule central point at present are light Dizzy method.Second step, Corner Detection.The image-forming principle of light-field camera 2.0 makes X-comers detection be possibly realized, but subgraph As containing angle information, making again it be different from traditional camera checkerboard angle point detection process.

Summary of the invention

The purpose of the present invention is to propose to a kind of demarcation preprocess method being applicable to light-field camera, solve for light-field camera In 1.0 scaling methods proposed, some used preprocessing means are not suitable for the problem of calibrating of light-field camera 2.0.

For reaching above-mentioned purpose, present invention employs techniques below scheme:

A kind of demarcation preprocess method being applicable to light-field camera, it is characterised in that comprise the steps:

S1: white Image semantic classification, obtains the subimage under each lenticule；

S2: a given gridiron pattern image, by pretreated to gridiron pattern image and step S1 white image phase and, it is thus achieved that refer to Determine the subimage under lenticule；

S3: each subimage obtaining step S2 carries out Corner Detection.

Preferably, step S1 comprises the steps:

S11: dialogue image carries out disk filtering；

S12: utilize the method asking for maximum in morphology to search out all of Local modulus maxima in white image, the most micro- Lens centre point；

S13: the lenticule central point finding step S12 does the linear fit of horizontal and vertical, corrects those not directly Central point on line；

S14: utilize the method for Morphological Reconstruction to reconstruct the subimage under each lenticule further, and ask for this subgraph The center of gravity of picture, goes to correct lenticular center position with focus point.

It is further preferred that step S14 comprises the steps:

S141: iris out the subimage under a lenticule first with a square boxes；Then morphology weight is used The method of structure reconstructs the subimage under lenticule；

S142: find the profile of the subimage that step S141 obtains, the profile got is estimated judgement, chooses rule Then such as following formula:

C_*=argmin_C∈I|C-π·Dist_centers|

Wherein I is the set of all profiles found；

Utilize optimum profiles C_*Calculate the position of centre of gravity of subimage, and go to revise lenticule center of gravity with this focus point Point；

S143: calculate the central point distance between each lenticule and surrounding lenticule, be designated as { D_i,j, i represents that this is micro- The serial number of mirror, j represents lenticular serial number around this lenticule；Calculate { D again_i,jStandard deviation Stdev_i,jIf, This standard deviation is more than threshold value thre1, then using this lenticule as candidate's exception lenticule；Remove { D_i,jIn }, maximum is Little value, calculates standard deviation Stdev ' again_i,jIf standard deviation is still greater than threshold value thre2, then this lenticule labelling For abnormal lenticule；

S144: calculate the average of lenticule central point aroundThis average is utilized to correct lenticule further Center position.

Preferably, step S3 comprises the steps:

S31: the subimage under each lenticule is done line detection；

S32: the straight line detected is classified, and ask for the intersection point of straight line as candidate angular；

S33: weed out angle point close together in candidate angular；

S34: diagonal angle point is optimized.

It is further preferred that step S31 comprises the steps:

S311: subimage does canny rim detection, removes lenticule edge so that the edge in subimage is only chess The edge of dish lattice；

S312: use Hough line to detect all straight lines in subimage.

It is further preferred that step S32 comprises the steps:

S321: according to the inclination angle of straight line, straight line is divided into two classes, is designated as { L_A,L_B, more each class straight line is carried out density Cluster, result is designated as { L_Ai},{L_Bj}.Each class straight line is obtained the average of straight line parameter, is designated as

S322: calculate two classes respectively from { L_AiAnd { L_BjStraight line between intersection point, computing formula is as follows:

$x=\frac{r_1{\mathrm{sin\θ}}_2-r_2{\mathrm{sin\θ}}_1}{\mathrm{\Δ}}$

$y=\frac{r_2{\mathrm{cos\θ}}_1-r_1{\mathrm{cos\θ}}_2}{\mathrm{\Δ}}$

Wherein Δ=cos θ₁sinθ₂-cosθ₂sinθ₁。

It is further preferred that the criterion that step S33 weeds out angle point close together in candidate angular is: if a friendship The distance of some a to intersection point b less than a threshold value, then all deletes intersection point as candidate using the two intersection point；If intersection point a arrives other The distance of intersection point is both greater than this threshold value, then a is deleted intersection point from candidate and remove, otherwise delete intersection point a, antinode b and carry out Same operation.

It is further preferred that step S34 comprises the steps:

S341: seek First-order Gradient along square frame, and First-order Gradient value is sued for peace, be designated asIt is referred to as First-order Gradient With, then calculate perimeter p eri of square frame_q；Wherein q represents square frame to be calculated, grad_iRepresent the i+1 along square frame Pixel and the difference of ith pixel；

S342: angle point is defined as:

$corner={\mathrm{argmax}}_q\frac{\underset{q}{\Σ}{\mathrm{grad}}_i}{{\mathrm{peri}}_q};$

S343: constantly iteration reduces square frame perimeter p eri_q, just can improve the degree of accuracy of angle point.

It is further preferred that described First-order Gradient and perimeter p eri with described square frame_qUnrelated, the most relevant with position, Comprise First-order Gradient and the maximum of the square frame of angle point, it may be assumed that

∑_dgrad_i=∑_agrad_i>∑_bgrad_i>∑_cgrad_i。

Advantages of the present invention and providing the benefit that:

1, a kind of method that lenticule center position optimizes is proposed；

2, the lenticule of breakage can be marked well, there is stronger robustness；

3, propose and do the Corner Detection of light-field camera 2.0 by the method for line detection；

4, the angle point that a kind of arithmetic operators optimization being similar to curl detects is proposed.

Accompanying drawing explanation

Fig. 1 is the basic flow sheet figure of the present invention；

Fig. 2 is the lenticule central point schematic diagram that the embodiment of the present invention utilizes subimage local maximum to mark；

Fig. 3 is the lenticule central point schematic diagram after the embodiment of the present invention utilizes linear fit to correct；

Fig. 4 is embodiment of the present invention lenticule central point schematic diagram after center of gravity optimizes；

Fig. 5 is that the embodiment of the present invention utilizes a square boxes to iris out the subimage schematic diagram under a lenticule；

Fig. 6 is that the embodiment of the present invention utilizes Morphological Reconstruction to isolate the subimage schematic diagram under single lenticule；

Fig. 7 is that embodiment of the present invention angle point optimizes schematic diagram.

Detailed description of the invention

Below in conjunction with detailed description of the invention and compare accompanying drawing the present invention is described in further detail.It is emphasized that That the description below is merely exemplary rather than in order to limit the scope of the present invention and application thereof.

Refer to Fig. 1.A kind of demarcation preprocess method being applicable to light-field camera that the present invention proposes, including walking as follows Rapid:

S1: white Image semantic classification, obtains the subimage under each lenticule.

First dialogue image carries out disk filtering, then utilizes the method asking for maximum in morphology to search out white image In all of Local modulus maxima, as shown in Figure 2, density bullet point is expressed as lenticular central point.As shown in Figure 2, position In same a line or with string central point the most point-blank, this is not met by linear array with microlens array.Therefore, Needing these central points by row with by arranging the fitting a straight line carrying out horizontal and vertical respectively, the result of matching is as shown in Figure 3. Can also calculate simultaneously and close on two lenticule central point approximate distance, be designated as Dist_centers。

Halation method requires that incident ray vertical incidence thin lens, bright spot are only the central point of thin lens.But remove The lenticule of close main lens optical axis, remaining lenticular incident ray is also unsatisfactory for this it is assumed that cause bright spot and micro- Lens centre point has the deviation of two or three pixels.Now go to estimate that lenticule central point is not the most appropriate by bright spot, it is necessary to Utilize other means position optimization to central point.Its step is as follows:

1) square boxes is utilized to iris out the subimage under a lenticule, as shown in Figure 5, due in this square frame Further comprises some information of other subimages, use the method for Morphological Reconstruction to reconstruct to the subgraph under lenticule herein Picture, as shown in Figure 6.

2) find the profile of subimage, the profile got is estimated judgement, selection rule such as following formula:

C_*=argmin_C∈I|C-π·Dist_centers| (1)

Wherein I is the set of all profiles found.Utilize optimum profiles C_*Calculate the position of centre of gravity of subimage, and use This focus point goes to revise lenticule focus point.

3) calculate the central point distance between each lenticule and surrounding lenticule, be designated as { D_i,j, i represents this lenticule Serial number, j represents lenticular serial number around this lenticule.Calculate { D again_i,jStandard deviation Stdev_i,jIf, should Standard deviation is more than threshold value thre1, then using this lenticule as candidate's exception lenticule.Remove { D_i,jMinimax in } Value, calculates standard deviation Stdev ' again_i,jIf standard deviation still greater than threshold value thre2, is then labeled as this lenticule Abnormal lenticule.

4) average of lenticule central point around is calculatedThis average is utilized to correct further lenticular Center position.

Eventually pass the central point after optimization as shown in Figure 4.

S2: a given gridiron pattern image, by pretreated to gridiron pattern image and step S1 white image phase and, it is thus achieved that refer to Determine the subimage under lenticule.

S3: each subimage obtaining step S2 carries out Corner Detection.

By white image processing step, the subimage under each lenticule can be obtained.A given gridiron pattern image, can With utilize gridiron pattern image and white image phase with, it is thus achieved that specify the subimage under lenticule.Then each subimage is carried out Process, including following link:

1) subimage is done canny rim detection, remove lenticule edge so that the edge in subimage is only gridiron pattern Edge.

2) use Hough line to detect all straight lines in subimage, then according to the inclination angle of straight line, straight line is divided It is two classes, is designated as { L_A,L_B, more each class straight line is carried out Density Clustering, result is designated as { L_Ai},{L_Bj}.To each class straight line Obtain the average of straight line parameter, be designated as

3) two classes are calculated respectively from { L_AiAnd { L_BjStraight line between intersection point, computing formula is as follows:

$x=\frac{r_1{\mathrm{sin\θ}}_2-r_2{\mathrm{sin\θ}}_1}{\mathrm{\Δ}}$

$y=\frac{r_2{\mathrm{cos\θ}}_1-r_1{\mathrm{cos\θ}}_2}{\mathrm{\Δ}}---\left(2\right)$

Wherein Δ=cos θ₁sinθ₂-cosθ₂sinθ₁.Then these intersection points are screened, it is judged that criterion is, if a friendship The distance of some a to intersection point b less than a threshold value, then all deletes intersection point as candidate using the two intersection point, if intersection point a arrives other The distance of intersection point is both greater than this threshold value, then a is deleted intersection point from candidate and remove, otherwise delete intersection point a, antinode b and carry out Same operation.

4) angle point optimization.As shown in Figure 7, seek First-order Gradient along square frame, and First-order Gradient value is sued for peace, be designated asBe referred to as First-order Gradient and, then calculate perimeter p eri of square frame_q.Wherein q represents square frame to be calculated, grad_i Represent the difference of i+1 the pixel along square frame and ith pixel.Understand:

∑_dgrad_i=∑_agrad_i>∑_bgrad_i>∑_cgrad_i

Above formula illustrates under certain condition, First-order Gradient and perimeter p eri with square frame_qUnrelated, the most relevant with position, bag The First-order Gradient of the square frame containing angle point and maximum.Angle point is defined as:

$corner={\mathrm{argmax}}_q\frac{\underset{q}{\Σ}{\mathrm{grad}}_i}{{\mathrm{peri}}_q}---\left(3\right)$

It follows that only need continuous iteration to reduce square frame perimeter p eri_qJust can improve the degree of accuracy of angle point.

Above content is to combine concrete/the most made for the present invention further description, it is impossible to Assert the present invention be embodied as be confined to these explanations.General technical staff of the technical field of the invention is come Saying, without departing from the inventive concept of the premise, these embodiments having described that can also be made some replacements or modification by it, And these substitute or variant all should be considered as belonging to protection scope of the present invention.

Claims (9)

1. the demarcation preprocess method being applicable to light-field camera, it is characterised in that comprise the steps:

S1: white Image semantic classification, obtains the subimage under each lenticule；

S2: a given gridiron pattern image, by pretreated to gridiron pattern image and step S1 white image phase and, it is thus achieved that specify micro- Subimage under lens；

S3: each subimage obtaining step S2 carries out Corner Detection.

The demarcation preprocess method being applicable to light-field camera the most according to claim 1, it is characterised in that step S1 includes Following steps:

S11: dialogue image carries out disk filtering；

S12: utilize the method asking for maximum in morphology to search out all of Local modulus maxima, i.e. lenticule in white image Central point；

S13: the lenticule central point finding step S12 does the linear fit of horizontal and vertical, corrects those not on straight line Central point；

S14: utilize the method for Morphological Reconstruction to reconstruct the subimage under each lenticule further, and ask for this subimage Center of gravity, goes to correct lenticular center position with focus point.

The demarcation preprocess method being applicable to light-field camera the most according to claim 2, it is characterised in that step S14 bag Include following steps:

S141: iris out the subimage under a lenticule first with a square boxes；Then Morphological Reconstruction is used Method reconstructs the subimage under lenticule；

S142: finding the profile of the subimage that step S141 obtains, the profile got is estimated judgement, selection rule is such as Following formula:

C_*=argmin_C∈I|C-π·Dist_centers|

Wherein I is the set of all profiles found；

Utilize optimum profiles C_*Calculate the position of centre of gravity of subimage, and go to revise lenticule focus point with this focus point；

S143: calculate the central point distance between each lenticule and surrounding lenticule, be designated as { D_i,j, i represents that this is lenticular Serial number, j represents lenticular serial number around this lenticule；Calculate { D again_i,jStandard deviation Stdev_i,jIf, this mark Quasi-difference is more than threshold value thre1, then using this lenticule as candidate's exception lenticule；Remove { D_i,jMaximin in }, Again calculate standard deviation Stdevi '_{I, j}If standard deviation still greater than threshold value thre2, is then labeled as this lenticule different Often lenticule；

S144: calculate the average of lenticule central point aroundUtilize this average correct further lenticular in Heart point position.

The demarcation preprocess method being applicable to light-field camera the most according to claim 1, it is characterised in that step S3 includes Following steps:

S31: the subimage under each lenticule is done line detection；

S32: the straight line detected is classified, and ask for the intersection point of straight line as candidate angular；

S33: weed out angle point close together in candidate angular；

S34: diagonal angle point is optimized.

The demarcation preprocess method being applicable to light-field camera the most according to claim 4, it is characterised in that step S31 bag Include following steps:

S311: subimage does canny rim detection, removes lenticule edge so that the edge in subimage is only gridiron pattern Edge；

S312: use Hough line to detect all straight lines in subimage.

The demarcation preprocess method being applicable to light-field camera the most according to claim 4, it is characterised in that step S32 bag Include following steps:

S321: according to the inclination angle of straight line, straight line is divided into two classes, is designated as { L_A,L_B, then it is poly-that each class straight line carries out density Class, result is designated as { L_Ai},{L_Bj}；Each class straight line is obtained the average of straight line parameter, is designated as

S322: calculate two classes respectively from { L_AI} and { L_BIntersection point between the straight line of j}, computing formula is as follows:

$x=\frac{r_1{\mathrm{sin\θ}}_2-r_2{\mathrm{sin\θ}}_1}{\mathrm{\Δ}}$

$y=\frac{r_2{\mathrm{cos\θ}}_1-r_1{\mathrm{cos\θ}}_2}{\mathrm{\Δ}}$

Wherein Δ=cos θ₁sinθ₂-cosθ₂sinθ₁。

The demarcation preprocess method being applicable to light-field camera the most according to claim 4, it is characterised in that step S33 is picked The criterion removing angle point close together in candidate angular is: if the distance of an intersection point a to intersection point b is less than a threshold value, Then the two intersection point is all deleted intersection point as candidate；If intersection point a is both greater than this threshold value to the distance of other intersection points, then A deletes intersection point from candidate and removes, and otherwise deletes intersection point a, antinode b and operates equally.

The demarcation preprocess method being applicable to light-field camera the most according to claim 4, it is characterised in that step S34 bag Include following steps:

S341: seek First-order Gradient along square frame, and First-order Gradient value is sued for peace, be designated asBe referred to as First-order Gradient and, so After calculate perimeter p eri of square frame_q；Wherein q represents square frame to be calculated, grad_iRepresent along square frame i+1 pixel and The difference of ith pixel；

S342: angle point is defined as:

$corner={\mathrm{argmax}}_q\frac{\underset{q}{\Σ}{\mathrm{grad}}_i}{{\mathrm{peri}}_q};$

S343: constantly iteration reduces square frame perimeter p eri_q, just can improve the degree of accuracy of angle point.

The demarcation preprocess method being applicable to light-field camera the most according to claim 8, it is characterised in that a described ladder Degree and perimeter p eri with described square frame_qUnrelated, the most relevant with position, comprise First-order Gradient and the maximum of the square frame of angle point, it may be assumed that

∑_dgrad_i=∑_agrad_i>∑_bgrad_i>∑_cgrad_i。