CN102521822B - Active light-emitting type target for automatic calibration based on machine vision and calibrating method thereof - Google Patents

Abstract

The invention relates to an active light-emitting type target calibrating method for automatic calibration based on machine vision. A plurality of small light emitting lamps arranged into a rectangular dot matrix are embedded on a target of a two-dimensional flat plate type, and all the small lamps are arranged equidistantly in a horizontal direction and a vertical direction, and colors of the four small light-emitting lamps on the four vertexes of the rectangular dot matrix are different; each small light-emitting lamp is controlled by an independent switch; and the operating modes of the target include a remote operating mode and a close operating mode: when the target is arranged at a close place, all the small light-emitting lamps on the target are opened; when the target is arranged at a remote place, the small light-emitting lamps with different colors at the four corners of the target are opened, and the residual small light-emitting lamps emit light every two columns. The calibrating method comprises the following steps of: shooting multiple groups of target images from any selected different positions by a camera, and respectively shooting one opening image and one closing image of the small light-emitting lamps at the same position; removing the background of each group of images by using a grey difference subtraction method; extracting a light emitting central point (calibrating point) of each small light-emitting lamp by using a grey gravity method; and directly substituting the obtained data into the camera calibration algorithm to calculate inner and outer parameters of the camera.

Description

A kind of active light-emitting type target and scaling method thereof for automatic calibration based on machine vision

Technical field:

The present invention relates to a kind of active light-emitting type target and scaling method thereof that can be used for automatic calibration based on machine vision, belong to computer vision field.

Background technology:

In machine vision, camera marking method mainly is divided into traditional scaling method and certainly demarcates two kinds at present.The former need to utilize the demarcation thing, and wherein the two-step approach development is comparatively ripe; Certainly demarcating does not need only to find the solution camera parameters by the relation between sequence of pictures with reference to demarcating thing, but this method is still having some deficits aspect stability and precision.

Yet use traditional scaling method to realize the automatic Calibration of camera parameters, the core key is automatically to identify calibration point.What use than multi-scheme at present is one the target of geometry in particular and color to be arranged.Utilize the design of geometric relationship in special shape and color relations to write the self-identifying algorithm, lines, circle and the array point etc. at the multiplex black and white of geometric configuration interval.But the common disadvantage of these schemes is to eliminate the interference of background environment, if occur the characteristic pattern similar to target or other in background with interfering factor, easily judges calibration point or the calibration point of failing to judge by accident.And it is complicated that this programmed algorithm is tending towards usually, and it is very high that some scheme is made requirement to target.Therefore a kind of cost of necessary design is lower, algorithm is simple and easy, be not subject to the scheme of external environmental interference.

Summary of the invention

The objective of the invention is to propose a kind of scaling method that can pass through the target of simple process implementation video camera automatic Calibration.And target is with low cost, and timing signal can filter the target background to be made when extracting calibration point without interruption, can automatically identify and extract calibration point.

Technical scheme of the present invention is: the active light-emitting type target and the scaling method thereof that are used for automatic calibration based on machine vision, adopt the target of two-dimensional flat plate formula, it is characterized in that being embedded with on target the luminous lampet that several are arranged in rectangular lattice, all lampets are equidistantly arranged with vertical direction in the horizontal direction, lampet on four summits of rectangle is with the property distinguished, its color is different, remaining lampet solid colour but not identical with lampet color on four jiaos; Every lampet is by independently switch control; The mode of operation of target is placed at a distance, two kinds nearby: when being placed on nearby, the lampet on target is all opened; When being placed at a distance, open with the lampet of the property distinguished on four jiaos of targets, all the other lampet interlacing are luminous every row.

Demarcating steps is: take many group target images at arbitrarily selected diverse location, same position is taken respectively each, the picture of lampet opening and closing; Use gray scale to do poor method processing of subtracting each other and respectively organize image removal background; The method of use grey scale centre of gravity extracts the luminescent center point of lampet in image; Set up the world coordinate system of unified direction, calculate the world coordinates of whole lampets; Directly bring result of calculation into Camera Calibration Algorithm at last, solve inner parameter and the external parameter of video camera, wherein inner parameter is focal length, the pixel ratio factor, level, vertical pixel unit length and principal point coordinate; External parameter is rotation and the displacement relation between world coordinate system and camera coordinate system.

The method of using target of the present invention to carry out the video camera automatic Calibration comprises,

Step 1: place the diverse location of target in camera coverage, take many group coloured images.Each position take respectively the lampet opening and closing picture each one.

Step 2: process every picture group sheet, obtain not having the only target picture of surplus lampet luminous point of background.On this figure, the gray-scale value of every is that the bright light picture deducts at the gray-scale value of this some the gray-scale value of picture at this point of turning off the light.

Step 3: the gained picture is carried out threshold segmentation, remove near the scattering aperture of lampet.

Step 4: for each luminous point, use the method for grey scale centre of gravity to calculate its luminous focus point.Suppose that the luminous light intensity of lampet is symmetrical, so the focus point of this moment is exactly the geometric center point of lampet.Use grey scale centre of gravity method formula:

X wherein, y is the point that image coordinate is fastened, and f (x, y) is the gray-scale value of this point, and x1, y1 are the focus point coordinates, calculate the central point of whole luminous points, obtain the whole calibration point coordinates under image coordinate system.

Step 5: utilize on target four jiaos of lampets to set up the unified world coordinate system of direction for the characteristics of different colours, the world coordinates of lampet can obtain by the size of measuring the actual range initial point.Calculate the world coordinates of whole lampets, obtain getting under world coordinate system total data.

The imaging model of video camera is usually to be approximately pin-hole model.Take a summit of target as initial point, target plane is O _wX _wY _wWorld coordinate system O is set up on the plane _wX _wY _wZ _wWith the video camera center O _cBe initial point, set up camera coordinate system O _cX _cY _cZ _cTake the left upper apex O of image as initial point, set up image coordinate system O ₀U ₀V ₀If P is any point on target, its coordinate under world coordinate system is P _w(x _w, y _w, z _w), getting coordinate under camera coordinate system is P _c(x _c, y _c, z _c), coordinate is P under image coordinate system ₀(x ₀, y ₀).The corresponding relation of world coordinate system and image coordinate system is:

$S\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right]=A\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=A\left[\begin{array}{cc}R& T\end{array}\right]\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& c& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& T\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]---\left(1\right)$

Wherein s is scale-up factor.Matrix [R T] is called external parameter, and wherein R and T represent respectively rotation matrix and the translation matrix of world coordinate system and camera coordinate system.Matrix A is called inner parameter, wherein (u ₀, v ₀) for the intersection point of optical axis and image coordinate system is the principal point coordinate, α and β are respectively the normalization focal length of video camera on x, y direction, c is the out of plumb factor between video camera x axle and y axle.

Step 6: the world coordinate system that step 5 is obtained, the calibration point coordinate data under image coordinate system are brought in the calibration algorithm of Zhang Zhengyou two-step approach, calculate the inside and outside parameter of video camera.

The present invention has designed a kind of target of active light-emitting type.Use this target timing signal, can make the poor background of removing by image, adopt the method that judges picture gray-scale value size to determine the calibration point of target, complete the automatic Calibration of video camera in the substitution calibration algorithm.

The invention has the beneficial effects as follows: the target of realizing the video camera automatic Calibration by simple method.Obtain the accurate inside and outside parameter of video camera, and the target that arranges is with low cost, timing signal can filter the target background to be made when extracting calibration point without interruption, can automatically identify and extract calibration point.

Description of drawings

Fig. 1 is video camera aperture perspective transform schematic diagram.

Fig. 2 is the schematic diagram of target.

Fig. 3 is the schematic diagram that target is worked a long way off.

Specific embodiments

Be embedded with the luminous lampet that several are arranged in rectangular lattice on the target of the present invention's design, for example ranks are distributed as 9x11.All lampets are equidistantly arranged with vertical direction in the horizontal direction, and the lampet on four summits of rectangle is with the property distinguished, and its color is different, all the other lampet solid colours but not identical with lampet color on four jiaos.Every lampet is by independently switch control, and the later stage can be by the switch of lampet in the programmed control array.

Concrete calibration process is as follows:

Step 1: target is placed on diverse location in camera coverage, takes many group coloured images.Each position take respectively the lampet opening and closing picture each one.During shooting, control lampet with master switch, in every picture group sheet, the position of target remains unchanged.

Step 2:, first colour picture is converted to the gray scale picture, then make in every group two pictures gray scales subtract each other.Program is: travel through whole pictures, in new picture, the gray-scale value of every is this two pictures absolute value that gray-scale value subtracts each other on respective point.This step has been removed identical background in two target pictures, the lampet of surplus array type only on new picture.

Step 3: new picture is carried out threshold segmentation: namely travel through picture, if the gray-scale value of coordinate points is less than given threshold values, with regard to zero setting; If more than or equal to, remain unchanged.This step has been removed near the scattering aperture lampet.

Step 4: for each luminous point, use the method for grey scale centre of gravity to calculate its luminous focus point.Suppose that the luminous light intensity of lampet is symmetrical, so the focus point of this moment is exactly the geometric center point of lampet.Use grey scale centre of gravity method formula:

X wherein, y are the points on the coordinate system of image, and f (x, y) is the gray-scale value of this point, and x1, y1 are the focus point coordinates.Calculate the central point of whole luminous points, obtain the coordinate of the whole calibration points (luminous point) under image coordinate system.

Step 5: calculate calibration point coordinate under world coordinate system, be divided into for two steps: 1. set up unified world coordinate system.The color of four jiaos of LED lamps in the judgement original image, if the color of four jiaos of lampets is red, yellow, and green, white, every pictures unification is take red LED lamp as initial point, red, green LED lamp place straight line is the x axle, red, yellow LED lamp place straight line is the y axle, with the orientation determination z axle of the right-hand rule, set up world coordinate system.2. calculate each LED lamp and get coordinate in world coordinate system.The LED lamp coordinate is all 0 on the z direction, is physical size size apart from initial point in the value of x, y direction.This step obtains whole calibration points (luminous point) coordinate under world coordinate system.

Step 6: bring in the calibration algorithm of Zhang Zhengyou two-step approach getting data under world coordinate system, image coordinate system, calculate the parameter of video camera.

The imaging model of video camera is usually to be approximately pin-hole model.Take a summit of target as initial point, target plane is O _wX _wY _wWorld coordinate system O is set up on the plane _wX _wY _wZ _wWith the video camera center O _cBe initial point, set up camera coordinate system O _cX _cY _cZ _cTake the left upper apex O of image as initial point, set up image coordinate system O ₀U ₀V ₀If P is any point on target, its coordinate under world coordinate system is P _w(x _w, y _w, z _w), getting coordinate under camera coordinate system is P _c(x _c, y _c, z _c), coordinate is P under image coordinate system ₀(x ₀, y ₀).The corresponding relation of world coordinate system and image coordinate system is:

$S\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right]=A\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=A\left[\begin{array}{cc}R& T\end{array}\right]\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& c& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& T\end{array}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]---\left(1\right)$

Wherein s is scale-up factor.Matrix [R T] is called external parameter, and wherein R and T represent respectively rotation matrix and the translation matrix of world coordinate system and camera coordinate system.Matrix A is called inner parameter, wherein (u ₀, v ₀) for the intersection point of optical axis and image coordinate system is the principal point coordinate, α and β are respectively the normalization focal length of video camera on x, y direction, c is the out of plumb factor between video camera x axle and y axle.

Concrete calibration algorithm is as follows:

Homography matrix H is:

$H=A\left[\begin{array}{cc}R& T\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& c& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& T\end{array}\right]={\left[\begin{array}{ccc}{h_1}^T& {h_2}^T& {h_3}^T\end{array}\right]}^T---\left(2\right)$

If one is demarcated under luminous point alive boundary coordinate system to such an extent that coordinate be M=[x _w, y _w, 1] ^I, under the image coordinate system of actual measurement, coordinate is m=[u, v, 1] ^T, the image coordinate that re-projection calculates is m '=[u ', v ', 1] ^T, have:

$m^{\′}=\frac{1}{{h_3}^{T}M}\left[\begin{array}{c}{h_1}^{T}M\\ {h_2}^{T}M\end{array}\right]---\left(3\right)$

Suppose the error Normal Distribution between m and m ', the estimation of homography matrix H should make the likelihood function as (4) formula reach maximum.

$\frac{1}{\mathrm{\Π}\sqrt{2\mathrm{\π}}\mathrm{\σ}}{e}^{-\frac{1}{2}\mathrm{\Σ}\frac{{\left|\right|m-m^{\′}\left|\right|}^2}{{\mathrm{\σ}}^2}}---\left(4\right)$

If think that the σ value all equates for whole reference point, the estimation of H changed into non-linear minimization problem:

min∑||m-m′|| ² (5)

Perpendicular quadrature by between (1) formula and r1 and r3 can get:

h ₁ ^TA ^-TA ^-1h ₂＝0 (6)

h ₁ ^TA ^-TA ^-1h ₁＝h ₂ ^TA ^-TA ^-1h ₂ (7)

Order

$B=A^{-T}{A}^{-1}=\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{21}& B_{22}& B_{23}\\ B_{31}& B_{32}& B_{33}\end{array}\right]=$

$\left[\begin{array}{ccc}\frac{1}{{\mathrm{\α}}^2}& -\frac{c}{{\mathrm{\α}}^2\mathrm{\β}}& \frac{{\mathrm{cv}}_{\mathrm{\α}}-u_{\mathrm{\α\β}}}{{\mathrm{\α}}^2\mathrm{\β}}\\ -\frac{c}{{\mathrm{\α}}^2\mathrm{\β}}& \frac{c^2}{{\mathrm{\α}}^2{\mathrm{\β}}^2}+\frac{1}{{\mathrm{\β}}^2}& -\frac{c(c{v}_{\mathrm{\α}}-u_{\mathrm{\α\β}})}{{\mathrm{\α}}^2{\mathrm{\β}}^2}-\frac{v_{\mathrm{\α}}}{{\mathrm{\β}}^2}\\ \frac{{\mathrm{cv}}_{\mathrm{\α}}-u_{\mathrm{\α\β}}}{{\mathrm{\α}}^2\mathrm{\β}}& -\frac{c({\mathrm{cv}}_{\mathrm{\α}}-u_{\mathrm{\α\β}})}{{\mathrm{\α}}^2{\mathrm{\β}}^2}-\frac{v_{\mathrm{\α}}}{{\mathrm{\β}}^2}& \frac{{({\mathrm{cv}}_{\mathrm{\α}}-u_{\mathrm{\α\β}})}^2}{{\mathrm{\α}}^2{\mathrm{\β}}^2}+\frac{{v_{\mathrm{\α}}}^2}{{\mathrm{\β}}^2}+1\end{array}\right]---\left(8\right)$

If vectorial b is:

B＝[B ₁₁ B ₁₂ B ₂₂ B ₁₃ B ₂₃ B ₃₃] (9)

Have:

h _i ^TB _h ⁱ＝v _ij ^Tb (10)

Wherein

vij＝[h _i1h _j1h _i1h _j2+h _i2h _j1h _i2h _j2h _i3h _j1+h _i1h _j3h _i1h _j3+h _i3h _j2h _i3h _j3] ^T (11)

h _i＝[h _i1h _i2h _i3] ^T (12)

Simultaneous (8) and (12) can get:

$\left[\begin{array}{c}{v_{12}}^T\\ {(v_{11}-v_{12})}^T\end{array}\right]*b=0---\left(13\right)$

Take m width picture, obtain m as the equation of (13) formula, simultaneous gets:

Vb＝0 (14)

After solving b, the inner parameter matrix A is:

$\left\{\begin{array}{c}{v}_0=(B_{12}{B}_{13}-B_{11}{B}_{23})/(B_{11}{B}_{12}-{B_{12}}^2)\\ \mathrm{\λ}=B_{33}-[{B_{13}}^2+v_0(B_{12}{B}_{13}-B_{11}{B}_{23})]/B_{11}\\ \mathrm{\α}=\sqrt{\mathrm{\λ}/B_{11}}\\ \begin{array}{c}\mathrm{\β}=\sqrt{\frac{{\mathrm{\λB}}_{11}}{B_{11}{B}_{12}}-{B_{12}}^2}\\ u_0=-B_{13}{\mathrm{\α}}^2/\mathrm{\λ}\end{array}\end{array}\right.---\left(15\right)$

After finding the solution A, external parameter is:

$\left\{\begin{array}{c}{r}_1=\mathrm{\λ}{A}^{-1}{h}_1\\ r_2=\mathrm{\λ}{A}^{-1}{h}_2\\ r_3=r_1\×r_2\\ T=\mathrm{\λ}{A}^{-1}{h}_2\\ \mathrm{\λ}=\frac{1}{\left|\right|A^{-1}{h}_1\left|\right|}=\frac{1}{\left|\right|A^{-1}{h}_2\left|\right|}\end{array}\right.---\left(16\right).$

Claims (1)

1. active light-emitting type target scaling method that is used for automatic calibration based on machine vision, adopt the target of two-dimensional flat plate formula, it is characterized in that being embedded with on target the luminous lampet that several are arranged in rectangular lattice, all lampets are equidistantly arranged with vertical direction in the horizontal direction, four lampet colors on four summits of rectangle are different, remaining lampet solid colour but not identical with lampet color on four jiaos; Every lampet is by independently switch control; The mode of operation of target is placed at a distance, two kinds nearby: when being placed on nearby, the lampet on target is all opened; When being placed at a distance, open with the lampet of the property distinguished on four jiaos of targets, all the other lampet interlacing are luminous every row;

Demarcating steps is: video camera is taken many group target images at arbitrarily selected diverse location, and same position is taken respectively each, the picture of lampet opening and closing; Use gray scale to do poor method processing of subtracting each other and respectively organize image removal background; The method of use grey scale centre of gravity extracts the luminescent center point of lampet in image; Set up the world coordinate system of unified direction, calculate the world coordinates of whole lampets; With the direct substitution Camera Calibration Algorithm of result of calculation, solve inner parameter and the external parameter of video camera at last, wherein inner parameter is focal length, the pixel ratio factor, level, vertical pixel unit length and principal point coordinate; External parameter is rotation and the displacement relation between world coordinate system and camera coordinate system;

Step 1: place the diverse location of target in camera coverage, take many group coloured images; Every picture group sheet be take respectively in each position the lampet opening and closing picture each one;

Step 2: process every picture group sheet, obtain not having the only target picture of surplus lampet luminous point of background; On this figure, the gray-scale value of every is that the bright light picture deducts at the gray-scale value of this some the gray-scale value of picture at this point of turning off the light;

Step 3: the gained picture is carried out Threshold segmentation, remove near the scattering aperture of lampet;

Step 4: for each luminous point, use the method for grey scale centre of gravity to calculate its luminous focus point; Suppose that the luminous light intensity of lampet is symmetrical, so the focus point of this moment is exactly the geometric center point of lampet; Use grey scale centre of gravity method formula:

X wherein, y is the point that image coordinate is fastened, and f (x, y) is the gray-scale value of this point, and x1, y1 are the focus point coordinates; Calculate the central point of whole luminous points, obtain the whole calibration point coordinates under image coordinate system;

Step 5: utilize on target four jiaos of lampets to set up the unified world coordinate system of direction for the characteristics of different colours, the world coordinates of lampet obtains by the size of measuring the actual range initial point; Calculate the world coordinates of whole lampets, obtain the total data under world coordinate system;

The imaging of video camera is approximately pin-hole model: take a summit of target as initial point, target plane is O _wX _wY _wWorld coordinate system O is set up on the plane _wX _wY _wZ _wWith the video camera center O _cBe initial point, set up camera coordinate system O _cX _cY _cZ _cTake the left upper apex O of image as initial point, set up image coordinate system OU ₀V ₀If P is any point on target, its coordinate under world coordinate system is P _w(x _w, y _w, z _w), the coordinate under camera coordinate system is P _c(x _c, y _c, z _c), coordinate is P under image coordinate system ₀(x ₀, y ₀); The corresponding relation of world coordinate system and image coordinate system is:

$S\left[\begin{array}{c}{x}_0\\ y_0\\ 1\end{array}\right]=A\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\end{array}\right]=A\left[\mathrm{RT}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& c& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\mathrm{RT}\right]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]$ (1)

Wherein S is scale-up factor; Matrix [R T] is called external parameter, and wherein R and T represent respectively rotation matrix and the translation matrix of world coordinate system and camera coordinate system; Matrix A is called inner parameter, wherein (u ₀, v ₀) for the intersection point of optical axis and image coordinate system is the principal point coordinate, α and β are respectively the normalization focal length of video camera on x, y direction, c is the out of plumb factor between video camera x axle and y axle;

Step 6: the world coordinate system that the employing step 5 obtains, the inside and outside parameter that the calibration point coordinate data under image coordinate system calculates video camera.

Images