Object projection method and system
Specification object projection method and systems technology field
The present invention relates to a kind of object projection method and with the target projection system of this method, more specifically to a kind of object projection method with verification process and with the target projection system of this method.Background technology
In using video camera as the system for obtaining information, it is crucial that available information is correctly extracted from image.What the judgement to target identification correctness was relied primarily on is the shape of target, and the information such as color and size, what the matching recognizing target two video cameras was relied primarily on is exactly that epipolar-line constraint and Ordinal Consistency are constrained.In the system using binocular camera, the foundation of objective is realized using principle of parallax.The shortcoming of this system is the identification of target, and matching and the verification of correctness set up are more difficult.The content of the invention
The technical problem to be solved in the present invention is, for the three-dimensional coordinate identification of the target projection system of prior art, matching and the defect for the verification of correctness more difficulty set up, the affine transformation relationship that a kind of utilization twin camera is set up between target and single camera is provided, by the identification of target, matching and the information rebuild are focused in projected image, verify that three-dimensional coordinate is recognized according to projected image, the object projection method and the target projection system with this method of matching and the correctness set up.
The technical solution adopted for the present invention to solve the technical problems is:A kind of object projection method is constructed, wherein, including:
Obtain matching double points corresponding three-dimensional coordinate of the object in the first video camera and the second video camera;Obtain the affine projection matrix of the 3rd video camera;
Verification process:
The object is calculated on the 3rd video camera according to the corresponding three-dimensional coordinate of the matching double points of the affine projection matrix and the object of the 3rd video camera in the first video camera and the second video camera
Theoretical barycentric coodinates;
Theoretical projected size of the object on the 3rd video camera is calculated according to the corresponding three-dimensional coordinate of the matching double points;
Obtain the actual center gravity coordinate that the object is projected on the 3rd video camera;
Obtain actual projected size of the object on the 3rd video camera;
Judge whether the corresponding three-dimensional coordinate of the matching double points is correct jointly according to the comparative result of the comparative result and the theoretical projected size and the actual projected size of the theoretical barycentric coodinates and the actual center gravity coordinate.
Object projection method of the present invention, wherein, the corresponding three-dimensional coordinate of the matching double points is set up process by calibration process and three-dimensional coordinate and obtained:
Calibration process:Using first video camera and the calibrating parameters of second video camera, obtain first video camera and second Camera calibration into the first correction matrix of first video camera of parallel vidicon and the second correction matrix of second video camera;
Three-dimensional coordinate sets up process:Matching double points of the object in first video camera and second video camera are obtained by the target identification matching to first video camera and second video camera;The corresponding three-dimensional coordinate of the matching double points is calculated using first correction matrix, second correction matrix and the matching double points.
Object projection method of the present invention, wherein, the calibrating parameters include inner parameter and outside match parameter, the inner parameter is obtained by stereo calibration plate scaling method, the outside match parameter matches the coordinate data of first video camera and second camera acquisition using permanent order, then carries out the coordinate data of permanent order matching first video camera and second camera acquisition described in Nonlinear least squares optimization to obtain using Li Wenbeige-Marquart algorithm.
The object projection methods of the present invention of a " 0, wherein, the inner parameter includes the α of A 0γ ν0
0 0 1 ax = fl dx, ay=fl dy, wherein/be focal length of camera, dx, ^ are horizontally oriented the pixel spacing with vertical direction respectively, be video camera photocentre, be the out of plumb factor of u axles and V axles;Radial distortion parameter kl, k2;Tangential distortion parameter pl, p2;The outside match parameter is joined including spin matrix
With translation matrix parameter
Object projection method of the present invention, wherein, the affine projection matrix is prepared by the following:The two-dimensional coordinate that the corresponding three-dimensional coordinate of the matching double points and matching double points correspondence of at least 6 groups diverse locations of collection are projected on the 3rd video camera;
The two-dimensional coordinate projected according to the corresponding three-dimensional coordinate of the matching double points of at least 6 groups diverse locations and matching double points correspondence on the 3rd video camera, affine projection matrix is calculated using linear least square.
Object projection method of the present invention, wherein, the two-dimensional coordinate that the corresponding three-dimensional coordinate of the matching double points and matching double points correspondence of described at least 6 groups diverse locations of collection are projected on the 3rd video camera is prepared by the following:
It is coordinate system to set the 3rd video camera, and the two-dimensional coordinate of at least 6 groups diverse locations is taken from the coordinate system;
When the object move to two-dimensional coordinate that the matching double points project on the 3rd video camera with it is described take the two-dimensional coordinate of at least 6 groups diverse locations to overlap from the coordinate system when, obtain the corresponding three-dimensional coordinate of the matching double points of at least 6 groups diverse locations corresponding with the two-dimensional coordinate of at least 6 groups diverse locations.
A kind of target projection system, including processing unit, imaging unit are constructed, wherein, in addition to authentication unit, the imaging system includes the first video camera and the second video camera, and the authentication unit includes authentication module and the 3rd video camera:
The processing unit:For obtaining the corresponding three-dimensional coordinate of matching double points of the object in the first video camera and the second video camera;With the affine projection matrix for obtaining the 3rd video camera;
The authentication module:Theoretical barycentric coodinates of the object on the 3rd video camera are calculated for the corresponding three-dimensional coordinate of matching double points according to the affine projection matrix and the object of the 3rd video camera in the first video camera and the second video camera;
Theoretical projected size of the object on the 3rd video camera is calculated according to the corresponding three-dimensional coordinate of the matching double points;
Obtain the actual center gravity coordinate that the object is projected on the 3rd video camera;
Obtain actual projected size of the object on the 3rd video camera;
According to the comparative result and the theoretical projected size of the theoretical barycentric coodinates and the actual center gravity coordinate with 222 comparative results for stating actual projected size judge whether the corresponding three-dimensional coordinate of the matching double points correct jointly. 333
Target projection system of the present invention, wherein, the processing unit includes demarcating module and three-dimensional coordinate sets up module:
The demarcating module:For the calibrating parameters using first video camera and second video camera, obtain first video camera and second Camera calibration into the first correction matrix of first video camera of parallel vidicon and the second correction matrix of second video camera;
The three-dimensional coordinate sets up module:For obtaining matching double points of the object in first video camera and second video camera by the target identification matching to first video camera and second video camera;The corresponding three-dimensional coordinate of the matching double points is calculated using first correction matrix, second correction matrix and the matching double points.
Target projection system of the present invention, wherein, the calibrating parameters include inner parameter and outside match parameter, the inner parameter is obtained by stereo calibration plate scaling method, the outside match parameter matches the coordinate data of first video camera and second camera acquisition using permanent order, then carries out the coordinate data of permanent order matching first video camera and second camera acquisition described in Nonlinear least squares optimization to obtain using Li Wenbeige-Marquart algorithm.
The target projection systems of the present invention of a " 0, wherein, the inner parameter includes a of A 0y v0
0 0 1 ax = fl dx, ay=fl dy, wherein/be focal length of camera, dx, be horizontally oriented pixel spacing with vertical direction respectively, be video camera photocentre, be the out of plumb factor of u axles and V axles;Radial distortion parameter kl, k2;Tangential distortion parameter pl, p2;The outside match parameter is joined including spin matrix
With translation matrix parameter
Target projection system of the present invention, wherein, the affine projection matrix is prepared by the following:
The two-dimensional coordinate that the corresponding three-dimensional coordinate of the matching double points and matching double points correspondence of at least 6 groups diverse locations of collection are projected on the 3rd video camera;
The two-dimensional coordinate projected according to the corresponding three-dimensional coordinate of the matching double points of at least 6 groups diverse locations and matching double points correspondence on the 3rd video camera, affine projection matrix is calculated using linear least square.
Target projection system of the present invention, wherein, the two-dimensional coordinate that the corresponding three-dimensional coordinate of the matching double points and matching double points correspondence of described at least 6 groups diverse locations of collection are projected on the 3rd video camera is prepared by the following:
It is coordinate system to set the 3rd video camera, and the two-dimensional coordinate of at least 6 groups diverse locations is taken from the coordinate system;
When the object move to two-dimensional coordinate that the matching double points project on the 3rd video camera with it is described take the two-dimensional coordinate of at least 6 groups diverse locations to overlap from the coordinate system when, obtain the corresponding three-dimensional coordinate of the matching double points of at least 6 groups diverse locations corresponding with the two-dimensional coordinate of at least 6 groups diverse locations.
Target projection system of the present invention, wherein, first video camera and second video camera are provided with infra-red transmitting filter.
Target projection system of the present invention, wherein, the authentication unit includes the 3rd target projection system of the present invention described at least two, wherein, the imaging unit still further comprises at least one imaging video camera.
Implement the object projection method and the target projection system with this method of the present invention, have the advantages that:Increase the 3rd video camera with authentication module on the basis of original target projection system, utilize the mapping between the three-dimensional coordinate and two-dimensional coordinate of foundation, the accuracy of the identification of checking three-dimensional coordinate, matching and the result set up.
The calibrating parameters obtained in advance are so that target projection process is simpler accurate;The acquisition of inner parameter and outside match parameter is so that the first correction matrix of the first video camera and the second correction matrix of the second video camera are more accurate, and the collimation of the first video camera and the second video camera is more efficient;The calculating of affine projection matrix allows to accurately calculate the first projected size on the 3rd video camera;And affine projection matrix can be with
Obtained by a variety of methods;First video camera and second video camera are provided with infra-red transmitting filter, eliminate the interference of visible ray, it is more accurate that three-dimensional coordinate is set up;Using the checking of the multiple-camera with authentication module, the identification for judging twin camera, matching and the correctness set up are further increased;The authentication unit can be used on multiple-camera(More than or equal to three)Target projection imaging system in.Brief description of the drawings
Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:
Fig. 1 is the structural representation of the preferred embodiment of target projection system of the present invention;
Fig. 2A is the inner parameter and the demarcation image of outside match parameter of a secondary calibrating camera of the preferred embodiment of target projection system of the present invention;
Fig. 2 B are the inner parameter of another secondary calibrating camera of the preferred embodiment of target projection system of the present invention and the demarcation image of outside match parameter;
Fig. 3 is two-dimensional coordinate arrangement and the coordinate value figure of the match point of 8 groups of diverse locations of the calculating affine projection matrix of the preferred embodiment of target projection system of the present invention.Embodiment
With reference to diagram, the preferred embodiments of the present invention are described in detail.
As shown in Figure 1, in the object projection method of the present invention, process and verification process are set up including calibration process, three-dimensional coordinate, the first video camera 21 and the calibrating parameters of the second video camera 22 that wherein three-dimensional coordinate is utilized during setting up are prepared by the following:
The calibrating parameters of video camera include inner parameter and outside match parameter.
Calibration of camera:
Intrinsic parameters of the camera, is changeless for focusing formula camera chain.Inner parameter includes:
A-
ax=fl dx, a=fl dy, wherein " being focal length of camera, dx, being respectively
The pixel spacing of 001 horizontal directions and vertical direction, it is the photocentre of video camera, is the out of plumb factor of U axles and V axles;Radial distortion parameter k l, k2;Tangential distortion parameter p l, p2.
Rice demarcates the inner parameter of all video cameras with stereo calibration plate scaling method.Demarcation is with image as schemed
Shown in 2A, 2B.All camera acquisitions obtain the barycentric coodinates of all calibrating blocks imagings using the connectivity analysis of connected domain to after uncalibrated image.Then these data messages are utilized, intrinsic parameters of the camera is demarcated according to flat board scaling method.Scaling method is current common method, can have different scaling methods to realize.
Outside match parameter demarcation:
Relative position relation information between the mainly video camera of video camera outside, including
Spin matrix parameter
Utilize the above-mentioned middle barycentric coodinates for gathering the first video camera 21 and the imaging of the calibrating block of the second video camera 22.The coordinate data of two camera acquisitions is matched using permanent order.Then the non-linear minimum coordinate data for optimizing permanent order first video camera 21 of matching and the collection of the second video camera 22 is carried out using Li Wenbeige-Marquart (Levenberg-Marquardt) algorithm, obtains final camera calibration parameter.Final argument includes the outside match parameter between the inner parameter and two video cameras of the first video camera 21 and the respective video camera of the second video camera 22.This step implementation method it is not unique.
Calibration process:Using the first video camera 21 and the calibrating parameters of the second video camera 22, by two camera acquisition image rectifications into parallel vidicon;This step implementation method is not unique.
A. the first correction matrix of the first video camera 21 is calculated:Set demarcation virtual point under the coordinate system of the first video camera 21=(Q 0 2000) ,
According to formula:
Calculate virtual point coordinates under the coordinate system of the first video camera 21(According to formula:
R、 +R
p 17 oi +p【
Column vector in the first correction matrix is calculated respectively.The first of the first video camera 21 is obtained according to (2) formulaPositive matrices: 2
B. the second correction matrix of the second video camera 22 is calculated:Set demarcation virtual point under the coordinate system of the second video camera 22=(Q 0 2000)
According to formula:
Pn=RPu+T (4) calculate virtual point coordinates under the coordinate system of the second video camera 22.According to formula:
Column vector in the second correction matrix is calculated respectively.The second correction matrix of the second video camera 22 is obtained according to (5) formula:
Η、
(6) 。
Three-dimensional coordinate sets up process:By being imaged to the-camera 21 and second;22 target identification matching obtains the corresponding matching double points of object:
Target identification thing is used as using circular reflective object.The support vector machine classifier crossed by circular reflective object sample training is respectively adopted in the first video camera 21 and the second video camera 22 object is identified, and calculates the matching double points (barycentric coodinates pair that object is imaged in the first video camera 21 and the second video camera 22)With then realize the correct matching of target using epipolar-line constraint and Ordinal Consistency constraint.
The corresponding three-dimensional coordinate of the matching double points is calculated using first correction matrix, second correction matrix and the matching double points;
A. pixel point coordinates distortion correction, while by matching double points P.Be changed into orthoscopic image coordinate and one.Coordinate is corrected used here as iterative calculation.
B. correction image coordinate is calculated according to formula:
Po
C. target three-dimensional coordinate is set up according to formula:
Disparity = Pl[l] - 1], it is image that it, which is neutralized,xCoordinate(9)
Β ξ+ζ Zc=1.0/Disparity of B Shui;Xc=B Shui pQ[l] /Disparity; (10) Yc=Zc * 70[1]/1.0;This step implementation method it is not unique.
Verification process:Three-dimensional coordinate sets up process and has obtained the corresponding three-dimensional coordinate P of the matching double points;According to the affine projection matrix and object of the 3rd video camera 32 in the first video camera 21 and the second shooting
Matching double points P in machine 22.Three-dimensional coordinate P corresponding with Pi counts the matching double points P.With the coordinate on the 3rd video camera 32
/ / (12);
Theoretical barycentric coodinates of the object on the 3rd video camera 32 are calculated according to the coordinate Q on the video camera 32 of the matching double points the 3rd of the object:This method is known target identification thing size, is here the diameter D of circular target thing, the 3rd video camera
32 focal length f and object three-dimensional coordinate P, then calculates the projection radius of circle of circle object according to formula:
Radius = Dx f I Pz(13) theoretical projected size of the object on the 3rd video camera 32 is calculated according to coordinate of the matching double points on the 3rd video camera 32:
Drawn on the image of the 3rd video camera 32 centered on theoretical barycentric coodinates, Radius is the filled circles of radius;
Obtain the actual center gravity coordinate that the object is projected on the 3rd video camera 32;
Actual projected size of the object on the 3rd video camera 32 is obtained according to coordinate of the matching double points on the 3rd video camera 32:
Whether the difference of the actual center gravity coordinate projected according to the theoretical barycentric coodinates and the object on the 3rd video camera 32 is less than the first setting value and whether the difference of the theoretical projected size and the actual projected size judges the matching double points P jointly less than the second setting value.It is whether correct with corresponding three-dimensional coordinate.The identification, matching and foundation that the corresponding three-dimensional coordinate of the matching double points is then thought when judging and being both less than setting value for two kinds are correct.
As shown in Figure 3(Fig. 3 uses 640*480 resolutions, and coordinate makes corresponding modification if other resolutions), the affine projection matrix utilized in verification process is prepared by the following:
At least 6 groups of collection(It is 8 groups in Fig. 3)The matching double points of diverse location and corresponding three-dimensional coordinate Q and the matching double points P.The two-dimensional coordinate correspondingly projected on the 3rd video camera 32:
It is coordinate system to set the 3rd video camera 32, takes the two-dimensional coordinate of 8 groups of diverse locations in the coordinate system;The matching double points P is moved to when the object.During with the two-dimensional coordinate projected on the 3rd video camera 32 with being overlapped with the 3rd video camera 32 for the two-dimensional coordinate of 8 groups of diverse locations of coordinate system, the matching double points Po and corresponding three-dimensional coordinate ^ of 8 groups of diverse locations corresponding with the two-dimensional coordinate of 8 groups of diverse locations are obtained
Set up equation formulations:
Ax= b (15)
Wherein affine projection matrix parameter is wherein formula(14) it can be abbreviated as
¾o ¾i a12Formula(15) ;Using linear least square calculating parameter X:
x= {ATAyl ATb ( 16) .Gather the matching double points P of at least 6 groups diverse locations.Three-dimensional coordinate corresponding with Pi and the matching double points P.The method of the two-dimensional coordinate correspondingly projected on the 3rd video camera 32 can also be set up at least 6 groups corresponding three-dimensional coordinates and two-dimensional coordinate to realize by correct match cognization.
It is used as the preferred embodiments of the present invention, the realization of the present invention at least needs three video cameras, and keep the synchronism of video camera shooting image, the first video camera 21 and the second video camera 22 set up for three-dimensional coordinate are the video camera for being provided with infra-red transmitting filter so that pass through the first video camera 21 and the second video camera 22
The three-dimensional coordinate that carries out of image of shooting set up and can effectively exclude the interference of visible ray, it is more accurate that three-dimensional coordinate is set up, and this set can be selected according to actual needs.
As the preferred embodiments of the present invention, authentication unit 3 includes at least two the 3rd video cameras 32 for being used to verify the three-dimensional coordinate set up.Using the checking of the multiple-camera with authentication module, the identification for judging twin camera, matching and the correctness set up are further increased.It can be needed to select the quantity using checking video camera according to available accuracy.
As the preferred embodiments of the present invention, imaging unit 2 can be multiple-camera(More than or equal to three)Target projection imaging system, can add judge multiple-camera identification, matching and set up correctness.
It the foregoing is only embodiments of the invention; it is not intended to limit the scope of the invention; the equivalent structure transformation that every utilization description of the invention and accompanying drawing content are made; or other related technical fields are directly or indirectly used in, it is included within the scope of the present invention.