CN113470118B - Target size estimation system based on trinocular vision - Google Patents

Target size estimation system based on trinocular vision Download PDF

Info

Publication number
CN113470118B
CN113470118B CN202110800603.XA CN202110800603A CN113470118B CN 113470118 B CN113470118 B CN 113470118B CN 202110800603 A CN202110800603 A CN 202110800603A CN 113470118 B CN113470118 B CN 113470118B
Authority
CN
China
Prior art keywords
ptz
machine
parameter
representing
ball
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110800603.XA
Other languages
Chinese (zh)
Other versions
CN113470118A (en
Inventor
姜柯
李爱华
蔡艳平
王涛
苏延召
韩德帅
冯国彦
李庆辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rocket Force University of Engineering of PLA
Original Assignee
Rocket Force University of Engineering of PLA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rocket Force University of Engineering of PLA filed Critical Rocket Force University of Engineering of PLA
Priority to CN202110800603.XA priority Critical patent/CN113470118B/en
Publication of CN113470118A publication Critical patent/CN113470118A/en
Application granted granted Critical
Publication of CN113470118B publication Critical patent/CN113470118B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a target size estimation system based on three-eye vision, which utilizes a gun camera as a guiding function of a main camera, utilizes a Levenberg-Marquardt optimization algorithm to realize the optimal solution of focal length and PT angle of a PTZ (point to point) spherical camera, and further utilizes a direct method to realize the size estimation of a moving target.

Description

Target size estimation system based on trinocular vision
Technical Field
The invention relates to a target size estimation system, in particular to a target size estimation system based on three-eye vision.
Background
In important and important parts of military or civil facilities, such as airport floors, oil reservoirs, chemical plants and the like, people have higher requirements on the performance of intelligent video monitoring.
In the prior art, a binocular vision method is generally adopted, a binocular vision system realized by combining a gun camera or an omnidirectional camera with a PTZ (pulse-to-zoom) ball machine is adopted, a gun camera or an omnidirectional camera is firstly used for detecting a moving target, and then the PTZ ball machine is used for realizing tracking and amplifying snapshot.
The binocular vision system realized by combining a gun camera or an omnidirectional camera with a PTZ (pulse-to-zoom) dome camera is used for detecting a moving target firstly, and then tracking and amplifying snapshot are realized by using the gun camera or the omnidirectional camera, but in the aspect of size estimation, the system has the following two defects:
two cameras with overlarge focal length (the PTZ dome camera can enable the focal length to be far larger than that of the static camera during zooming and tracking), larger errors can be generated during three-dimensional correction, and the effective size of the corrected images can be greatly different, namely, the images of the PTZ dome camera can only correspond to a smaller area of a gun camera image, so that the target detail is seriously lost, and the size estimation precision is lower;
the measurement accuracy of binocular vision is greatly affected by the focal length and the base line. Under the condition that the base line is unchanged, the depth estimation precision is higher as the focal length is larger, and the obtained three-dimensional coordinate information of the space point is more accurate. However, the focal length of both the camera and the omnidirectional camera is very small, typically below 8mm, which is detrimental to the acquisition of target sizes at a distance from the camera.
In summary, the existing binocular vision-based target size estimation system has the problem that the estimation result is inaccurate.
Disclosure of Invention
The invention aims to provide a target size estimation system based on three-eye vision, which is used for solving the problem that the target size estimation system based on binocular vision in the prior art has inaccurate estimation results.
In order to realize the tasks, the invention adopts the following technical scheme:
the system comprises a three-eye vision module, a camera calibration module, a PTZ control parameter calculation module, a PTZ parameter optimization module and a target size estimation module;
the triple vision module comprises a gun camera 3 and two PTZ ball machines 2, wherein the gun camera 3 and the PTZ ball machines 2 are arranged on the same horizontal line, and the gun camera 3 is positioned in the middle of the two PTZ ball machines 2; the optical axis of the PTZ ball machine 2 in the zero position is the same as the direction of the gun machine 3;
the triple vision module is used for acquiring images containing the same moving object and obtaining a gun camera image, a first PTZ (point to Z) ball camera image and a second PTZ ball camera image;
the camera calibration moduleThe method is used for calibrating internal and external parameters of the gun camera and the two PTZ ball cameras to obtain internal parameters of the gun camera, external parameters of the gun camera and internal parameters of the first PTZ ball cameraFirst PTZ ball machine initial rotation matrix R 0 First PTZ spherical translation vector t_l= (t) x _l,t y _l,t z L), second PTZ balloon machine internal parameterInitial rotation matrix R of second PTZ ball machine 0 R and a second PTZ balloon machine translation vector t_r= (t) x _r,t y _r,t z _r)；
The PTZ control parameter calculation module is used for obtaining a first PTZ ball machine rotation matrix R by adopting the formula I PT L and a second PTZ ball machine rotation matrix R PT _r；
Wherein θ is P L represents the value of the P-direction rotation angle of the first PTZ ball machine, θ T L represents the value of the T-direction rotation angle of the first PTZ ball machine, θ P R represents the value of the P-direction rotation angle of the second PTZ ball machine, θ T R represents the value of the T-direction rotation angle of the second PTZ ball machine;
the PTZ parameter optimization module is used for utilizing an LM optimization algorithm to perform a rotation matrix R of the first PTZ ball machine PT L, second PTZ ball machine rotation matrix R PT Optimizing R, the first PTZ ball machine internal parameter K_l and the second PTZ ball machine internal parameter K_r to obtain an optimized rotation matrix R 'of the first PTZ ball machine' PT L, rotation matrix R 'of optimized second PTZ ball machine' PT R, an optimized first PTZ balloon internal parameter K '_l, and an optimized second PTZ balloon internal parameter K' _r;
the target size estimation module stores a first computer program which, when executed by a processor, performs the steps of:
a, respectively obtaining coordinates of a starting point A and an ending point B of the same moving object on a first PTZ spherical machine image and a second PTZ spherical machine image on a two-dimensional image to obtain a first object starting point (u l _A,v l A), a first target termination point (u l _B,v l B), a second target start point (u r _A,v r A) and a second target termination point (u) r _B,v r _B)；
Step b, obtaining the true distance d of the moving object by adopting the II AB Is the value of (1):
wherein X is W_A Representing the real coordinates of the space of the starting point A, X W_B Representing the real coordinates of the termination point B in space; a is that 1 Representing a first parameter matrix of the starting point, B 1 A second parameter matrix representing a starting point; a is that 2 Representing a first parameter matrix of termination points, B 2 A second parameter matrix representing an end point;
wherein m is i L represents the first PTZ machine parameter, m i R represents a second PTZ balloon machine parameter, i=11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34;
the first PTZ machine parameter and the second PTZ machine parameter are obtained using equation III.
Further, when the camera calibration module performs internal and external parameter calibration on the gun camera and the two PTZ ball machines, a Zhang Zhengyou camera calibration method is adopted.
Further, the PTZ parameter optimization module stores a second computer program for obtaining the rotation matrix R 'of the PTZ ball machine after optimization' PT K and the optimized PTZ balloon machine internal parameters K' _k, k=l, r, said second computer program, when executed by the processor, implements the steps of:
step 1, obtaining a basic matrix F_k by adopting a formula IV:
F_k＝K_l -T R 0 _kR PT _kS(t 0 )_kK_r -1 IV (IV)
Wherein S (t) 0 ) K represents a translation parameter matrix,
t x k represents the value of the translation vector of the PTZ sphere machine on the x-axis, t y K represents the value of the translation vector of the PTZ sphere machine on the y-axis, t z K represents the value of the translation vector of the PTZ sphere machine in the z-axis;
step 2, adopting LM algorithm iteration to solve the V to obtain the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, value theta of T-direction rotation angle of PTZ ball machine after optimization T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ′_k；
Wherein e l_n Coordinate value e representing the nth point in the first PTZ dome camera image r_n Coordinate values representing an nth point in the second PTZ dome camera image, n=1,..n, N is a positive integer, e l_n And e r_n Coordinate values for matching corresponding points; (j) 1 Element 1 of the representation vector j, (j) 2 Representing the 2 nd element of vector j; Φ (·) represents the parameter set to be optimized Φ (·) =Φ (f) x _k,f y _k,θ P _k,θ T _k)；
Step 3, according to the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, value theta of T-direction rotation angle of PTZ ball machine after optimization T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ' k, obtaining the rotation matrix R ' of the PTZ ball machine after optimization ' PT K and the optimized PTZ balloon machine internal parameter K' _k.
Compared with the prior art, the invention has the following technical effects:
1. according to the target size estimation system based on three-eye vision, provided by the invention, the camera is used as a guiding function of a main camera, and the optimization solution of the focal length and the PT angle of the PTZ spherical camera is realized by using a Levenberg-Marquardt optimization algorithm, so that the size estimation of a moving target is realized by using a direct method;
2. the target size estimation system based on three-eye vision provided by the invention utilizes the PT rotation angle and focal length parameter optimization solving method realized by the LM algorithm. A time weighted PT control parameter least square fitting algorithm is provided, and the control precision of the PTZ ball machine is further improved.
Drawings
FIG. 1 is a schematic diagram of a three-dimensional vision module according to the present invention;
FIG. 2 is a schematic view of a wheel distance estimation provided in one embodiment of the present invention, wherein FIG. 2 (a) is a first PTZ machine image and FIG. 2 (b) is a second PTZ machine image;
fig. 3 is a schematic diagram of height estimation provided in an embodiment of the present invention, where fig. 3 (a) is a first PTZ dome camera image and fig. 3 (b) is a second PTZ dome camera image.
The reference numerals in the figures represent: 1-guide rail, 2-PTZ ball machine, 3-rifle bolt.
Detailed Description
The invention will now be described in detail with reference to the drawings and examples. So that those skilled in the art may better understand the present invention. It is to be expressly noted that in the description below, detailed descriptions of known functions and designs are omitted here as perhaps obscuring the present invention.
The definition or concept of the present invention is described below:
gun machine: the rifle bolt is one of the monitoring camera types. The gun camera is cuboid in appearance, and the front is provided with a C/CS lens interface.
PTZ ball machine: the Pan-Tilt-Zoom dome camera is abbreviated as Pan/Tilt/Zoom in security monitoring application, and represents a spherical monitoring camera for omnibearing (left-right/up-down) movement of a tripod head and Zoom control of a lens.
Example 1
The embodiment discloses a target size estimation system based on three-eye vision, which comprises a three-eye vision module, a camera calibration module, a PTZ control parameter calculation module, a PTZ parameter optimization module and a target size estimation module;
the triple vision module comprises a gun camera 3 and two PTZ ball machines 2, wherein the gun camera 3 and the PTZ ball machines 2 are arranged on the same horizontal line, and the gun camera 3 is positioned in the middle of the two PTZ ball machines 2; the optical axis of the PTZ ball machine 2 in the zero position is the same as the direction of the gun machine 3;
in this embodiment, as shown in fig. 1, an industrial guide rail 1, which is 1.5 m long and manufactured by taiwan upper silver company, is used as a mounting platform, and the guide rail may be fixed to a bracket or a wall. Four positions A, C, D are arranged on the guide rail 1, wherein two PTZ ball machines 2 are fixedly arranged at C, D positions and are respectively positioned at two ends of the guide rail 1. The point A is positioned in the center of the guide rail 1 and is used for installing the gun stock 3. Wherein the point A is the position of the bolt when the system works normally.
The optical axes of the two PTZ ball machines 2 in the zero position are oriented towards the gun machine 3, and the angle difference of the optical axes in the horizontal and vertical directions is made to be zero as much as possible. The coverage area of the PTZ spherical machine 2 is set to be in the P (horizontal) direction (-90 degrees) and in the T (vertical) direction (-20-90 degrees), so that the 180-degree area right in front of the system can be covered.
In the invention, the point A gun camera 3+C or the point D PTZ ball camera 2 can realize the purpose of utilizing the characteristic of fixing gun camera parameters, guiding the ball camera to complete active PTZ tracking, and realizing the error correction of PT rotation angle and focal length parameters of the PTZ ball camera through matching point information;
the point C PTZ baller + point D PTZ baller may constitute a binocular PTZ vision system for estimating target size.
In the embodiment, the gun camera 3 adopts a sea-health DS-2CD854F-E, is a 300-ten-thousand-pixel network gun type camera, adopts a 2.8mm wide-angle lens, works at 1920 multiplied by 1080 resolution, and can directly output network video data;
the PTZ ball machine 2 adopts Sony EVI-D90P, the model PTZ ball machine has a rotation function of pitching-20 degrees to 90 degrees and horizontal-170 degrees to 170 degrees, 28 times of optical zooming and adopts VISCA control protocol and analog signal output;
in this embodiment, the video analysis terminal and the local area network of the computer are used to obtain video data, so as to complete the functions of monitoring, tracking control, size estimation and display output of the target, and the analysis terminal uses a GPU graphics acceleration card (Hua Shuo GTX560Ti DCII) to simultaneously complete the functions of GPU acceleration calculation and dual screen display driving. The device is provided with two displays, one is used for monitoring the video of the gun camera in real time, and the other is used for displaying the operation interfaces of the real-time monitoring of the two PTZ ball machines.
The triple vision module is used for acquiring images containing the same moving object and obtaining a gun camera image, a first PTZ (point to Z) ball camera image and a second PTZ ball camera image;
the camera calibration module is used for calibrating internal and external parameters of the gun camera and the two PTZ ball cameras to obtain internal parameters of the gun camera, external parameters of the gun camera and internal parameters of the first PTZ ball cameraFirst PTZ ball machine initial rotation matrix R 0 First PTZ spherical translation vector t_l= (t) x _l,t y _l,t z L), second PTZ balloon machine internal parameterInitial rotation matrix R of second PTZ ball machine 0 R and a second PTZ balloon machine translation vector t_r= (t) x _r,t y _r,t z _r)；
In this embodiment, the camera calibration module performs an internal and external parameter calibration on the gun camera and the two PTZ cameras by using a Zhang Zhengyou camera calibration method.
In the present invention, f x_l Representing the x-direction pixel focal length of the first PTZ sphere machine in pixels, f y_l Representing the focal length of the y-direction pixel of the first PTZ camera in pixels, c x_1 Representing the offset of the first PTZ sphere machine on the x-axis relative to the center of the projection plane coordinates, c y_1 Representing the offset of the first PTZ sphere machine on the y-axis relative to the projected plane coordinate center, (t) x _l,t y _l,t z L) represents the value of the translation vector in x, y, z axis; similarly, f x_r Representing the x-direction pixel focal length of the first PTZ sphere machine in pixels, f y_r Representing the focal length of the y-direction pixel of the first PTZ camera in pixels, c x_2 Representing the offset of the second PTZ sphere machine on the x-axis relative to the center of the projection plane coordinates, c y_ 2 represents the offset of the second PTZ sphere machine on the y-axis with respect to the center of the projected plane coordinates, (t) x _r,t y _r,t z R) represents the value of the translation vector in the x, y, z axes.
The PTZ control parameter calculation module is used for obtaining a first PTZ ball machine rotation matrix R by adopting the formula I PT L and a second PTZ ball machine rotation matrix R PT _r；
Wherein θ is P L represents the value of the P-direction rotation angle of the first PTZ ball machine, θ T L represents the value of the T-direction rotation angle of the first PTZ ball machine, θ P R represents the value of the P-direction rotation angle of the second PTZ ball machine, θ T R represents the value of the T-direction rotation angle of the second PTZ ball machine;
the PTZ parameter optimization module is used for utilizing an LM optimization algorithm to perform a rotation matrix R of the first PTZ ball machine PT L, second PTZ ball machine rotation matrix R PT R, first PTZ ball machine internal parameter K_l and second PTZ ball machineOptimizing the internal parameter K_r to obtain a rotation matrix R 'of the optimized first PTZ ball machine' PT L, rotation matrix R 'of optimized second PTZ ball machine' PT R, an optimized first PTZ balloon internal parameter K '_l, and an optimized second PTZ balloon internal parameter K' _r;
in the embodiment, an LM optimization algorithm is adopted for iterative solution to obtain an optimized rotation matrix R 'of the first PTZ ball machine' PT L, rotation matrix R 'of optimized second PTZ ball machine' PT R, an optimized first PTZ balloon internal parameter K '_l, and an optimized second PTZ balloon internal parameter K' _r.
The PTZ parameter optimization module stores a second computer program for obtaining the rotation matrix R 'of the PTZ ball machine after optimization' PT K and the optimized PTZ balloon internal parameters K' _k, k=l, r.
In this embodiment, steps 1-3 are performed with two values calculated for each step, one being the parameter of the first PTZ machine and one being the parameter of the second PTZ machine. For example, in step 1, two basic matrices are obtained, one is the basic matrix f_l of the first PTZ machine and the other is the basic matrix f_r of the second PTZ machine.
In this embodiment, parameters of a gun camera and an initial rotation matrix are known, only the rotation angle and focal lengths fx and fy of the PTZ dome camera need to be optimized, and the initial solutions of the four parameters to be optimized can directly use corresponding values of control parameters to define a search starting point. Therefore, the method adopts the LM algorithm to directly optimize and solve the four parameters, and has higher precision and faster solving speed. First, several factors that affect the basis matrix between the bolt face and the PTZ dome:
(1) The relative relation between the PTZ spherical machine coordinate system and the gun machine coordinate system, namely the rotation matrix R0 and the translation vector t0, are all known quantities obtained through calibration;
(2) Rotation R of PTZ ball machine around self optical center due to PT movement PT The method consists of a variable sum to be solved, and an initial solution of the variable sum can be obtained by PT control parameters;
(3) The focal length change caused by the Z parameter control of the PTZ ball machine is expressed by fx and fy, and the initial solution of the focal length change can be obtained by the Z control parameter.
The second computer program when executed by the processor performs the steps of:
step 1, obtaining a basic matrix F_k by adopting a formula IV:
wherein S (t) 0 ) K represents a translation parameter matrix,
t x k represents the value of the translation vector of the PTZ sphere machine on the x-axis, t y K represents the value of the translation vector of the PTZ sphere machine on the y-axis, t z K represents the value of the translation vector of the PTZ sphere machine in the z-axis;
step 2, adopting LM algorithm iteration to solve the V to obtain the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, value theta of T-direction rotation angle of PTZ ball machine after optimization T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ′_k；
Wherein e l_n Coordinate value e representing the nth point in the first PTZ dome camera image r_n Coordinate values representing an nth point in the second PTZ dome camera image, n=1,..n, N is a positive integer, e l_n And e r_n Coordinate values for matching corresponding points; (j) 1 Element 1 of the representation vector j, (j) 2 Representing the 2 nd element of vector j; Φ (·) represents the parameter set to be optimized Φ (·) =Φ (f) x _k,f y _k,θ P _k,θ T _k)；
Step 3, according to the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, optimizedValue theta of T-direction rotation angle of PTZ ball machine T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ' k, obtaining the rotation matrix R ' of the PTZ ball machine after optimization ' PT K and the optimized PTZ balloon machine internal parameter K' _k.
The angle value corresponding to the control parameter PT and the fx, fy values corresponding to the Z control parameter are used herein as initial values for the LM algorithm iterative search.
In addition, in order to improve the control precision of the PTZ spherical machine during active tracking, the invention provides a time weighted PT control parameter least square fitting algorithm. Because of uncertainty of mechanical errors and difficulty in measuring the rotating angle of the PTZ ball machine, the control error of the PT angle is not calibrated in advance in a calibration stage, and the least square error fitting is performed online after the accurate angle value is obtained by using the optimization method introduced in the previous section.
The invention adopts the mathematical model as shown in the formula (6.59) between PT control parameters and angles:
the target size estimation module stores a first computer program which, when executed by a processor, performs the steps of:
a, respectively obtaining coordinates of a starting point A and an ending point B of the same moving object on a first PTZ spherical machine image and a second PTZ spherical machine image on a two-dimensional image to obtain a first object starting point (u l _A,v l A), a first target termination point (u l _B,v l B), a second target start point (u r _A,v r A) and a second target termination point (u) r _B,v r _B)；
Step b, obtaining the real distance of the moving object by adopting the IISeparation d AB Is the value of (1):
wherein X is W_A Representing the real coordinates of the space of the starting point A, X W_B Representing the real coordinates of the termination point B in space; a is that 1 Representing a first parameter matrix of the starting point, B 1 A second parameter matrix representing a starting point; a is that 2 Representing a first parameter matrix of termination points, B 2 A second parameter matrix representing an end point;
wherein m is i L represents the first PTZ machine parameter, m i R represents a second PTZ balloon machine parameter, i=11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34;
the first PTZ machine parameter and the second PTZ machine parameter are obtained using equation III.
In this embodiment, after obtaining the precise focal length parameters and rotation angle information of two PTZ cameras, the projection relationship between the spatial coordinates and the pixel coordinates of the PTZ cameras can be obtained as shown in formula VI:
where P_k is the projection matrix (P_k includes P_l and P_r), where R 0 And k and t_k are the initial rotation matrix and translation vector (R 0 K includes R 0 L and R 0 R, and the same applies,t_k includes t_l and t_r), K ' _k (K ' _k includes K ' _l and K ' _r) and R '. PT _k(R′ PT K includes R' PT L and R' PT R) is the accurate PTZ ball machine internal and external parameters obtained after the PTZ parameters are optimized.
By rewriting it in the form of elements, projection expressions VII (6.65) and VIII (6.66) of the two PTZ ballers can be obtained, where all elements of the projection matrix are known.
Combined VII (6.65) and VIII (6.66) of formula (6.66), and elimination of s l Sum s r Can be obtained about X W 、Y W And Z W For example, formula IX (6.67), formula X (6.68).
The equation set has four equations, three unknowns, and can be solved by using a least square method. The two formulas are modified to be of formula XI (6.69).
Taking a car as an example as shown in fig. 2, the distance between the wheels of the car can be obtained by selecting the corresponding points (A1-A2, B1-B2) between the centers of the two wheels in fig. 2 (a) and 2 (B), respectively, and the height is calculated by selecting the corresponding points (C1-C2, D1-D2) between the head and the foot of the pedestrian as shown in fig. 3. Table 1 shows the experimental results and errors of the direct method for estimating the dimensions.
TABLE 1 estimation results (cm) by direct method
In order to correct the parameter error of the PTZ ball machine, the invention utilizes the PT rotation angle and focal length parameter optimization solving method realized by the LM algorithm. A time weighted PT control parameter least square fitting algorithm is provided, and the control precision of the PTZ ball machine is further improved. A target size estimation method is provided, and experimental comparison and error analysis are performed. The result shows that the three-vision system provided by the invention can effectively track the moving target and realize the size estimation of the target under the condition of less manual operation.
From the above description of embodiments, it will be clear to a person skilled in the art that the present invention may be implemented by means of software plus necessary general purpose hardware, but of course also by means of hardware, the former being in many cases a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a readable storage medium, such as a floppy disk, a hard disk, or an optical disk of a computer, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of the embodiments of the present invention.

Claims (3)

1. The target size estimation system based on the three-eye vision is characterized by comprising a three-eye vision module, a camera calibration module, a PTZ control parameter calculation module, a PTZ parameter optimization module and a target size estimation module;
the triple vision module comprises a gun camera (3) and two PTZ ball machines (2), wherein the gun camera (3) and the PTZ ball machines (2) are arranged on the same horizontal line, and the gun camera (3) is positioned at the middle of the two PTZ ball machines (2); the optical axis of the PTZ ball machine (2) in the zero position is the same as the direction of the gun machine (3);
the triple vision module is used for acquiring images containing the same moving object and obtaining a gun camera image, a first PTZ (point to Z) ball camera image and a second PTZ ball camera image;
the camera calibration module is used for calibrating internal and external parameters of the gun camera and the two PTZ ball cameras to obtain internal parameters of the gun camera, external parameters of the gun camera and internal parameters of the first PTZ ball cameraFirst PTZ ball machine initial rotation matrix R 0 First PTZ spherical translation vector t_l= (t) x _l,t y _l,t z L), second PTZ balloon machine internal parameterInitial rotation matrix R of second PTZ ball machine 0 R and a second PTZ balloon machine translation vector t_r= (t) x _r,t y _r,t z R); wherein f x_1 Representing the focal length of the first PTZ balloon machine in the x-direction, f y_1 Representing the focal length of the first PTZ balloon machine in the y-direction, f x_2 Representing the focal length of the second PTZ balloon machine in the x-direction, f y_2 Representing the focal length of the first PTZ balloon machine in the y-direction, c x_1 Representing the offset of the first PTZ sphere machine on the x-axis relative to the center of the projection plane coordinates, c y_1 Representing the offset of the first PTZ sphere machine on the y-axis relative to the center of the projection plane coordinates, c x_2 Representing the offset of the second PTZ sphere machine on the x-axis relative to the center of the projection plane coordinates, c y_2 Representing the offset of the second PTZ sphere machine on the y-axis relative to the center of the projection plane coordinates;
the PTZ control parameter calculation module is used for obtaining a first PTZ ball machine rotation matrix R by adopting the formula I PT L and a second PTZ ball machine rotation matrix R PT _r；
Wherein θ is P L represents the value of the P-direction rotation angle of the first PTZ ball machine, θ T L represents the value of the T-direction rotation angle of the first PTZ ball machine, θ P R represents the value of the P-direction rotation angle of the second PTZ ball machine, θ T R represents the value of the T-direction rotation angle of the second PTZ ball machine;
the PTZ parameter optimization module is used for utilizing an LM optimization algorithm to perform a rotation matrix R of the first PTZ ball machine PT L, second PTZ ball machine rotation matrix R PT Optimizing R, the first PTZ ball machine internal parameter K_l and the second PTZ ball machine internal parameter K_r to obtain an optimized rotation matrix R 'of the first PTZ ball machine' PT L, rotation matrix R 'of optimized second PTZ ball machine' PT R, an optimized first PTZ balloon internal parameter K '_l, and an optimized second PTZ balloon internal parameter K' _r;
the target size estimation module stores a first computer program which, when executed by a processor, performs the steps of:
a, respectively obtaining coordinates of a starting point A and an ending point B of the same moving object on a first PTZ spherical machine image and a second PTZ spherical machine image on a two-dimensional image to obtain a first object starting point (u l _A,v l A), a first target termination point (u l _B,v l B), a second target start point (u r _A,v r A) and a second target termination point (u) r _B,v r _B)；
Step b, obtaining the true distance d of the moving object by adopting the II AB Is the value of (1):
wherein X is W_A Representing the real coordinates of the space of the starting point A, X W_B Representing the real coordinates of the termination point B in space; a is that 1 Representing a first parameter matrix of the starting point, B 1 A second parameter matrix representing a starting point; a is that 2 Representing a first parameter matrix of termination points, B 2 A second parameter matrix representing an end point;
wherein m is i L represents the first PTZ machine parameter, m i R represents a second PTZ balloon machine parameter, i=11, 12, 13, 14, 21, 22, 23, 24, 31, 32, 33, 34;
the first PTZ machine parameter and the second PTZ machine parameter are obtained using equation III.
2. The binocular vision based target size estimation system of claim 1, wherein the camera calibration module performs an internal and external parameter calibration of the bolt face and the two PTZ cameras using a Zhang Zhengyou camera calibration method.
3. The three-dimensional vision-based target size estimation system according to claim 2, wherein the PTZ parameter optimization module stores a second computer program for obtaining the rotation matrix R 'of the optimized PTZ dome camera' PT K and the optimized PTZ balloon machine internal parameters K' _k, k=l, r, said second computer program, when executed by the processor, implements the steps of:
step 1, obtaining a basic matrix F_k by adopting a formula IV:
F_k＝K_l -T R 0 _kR PT _kS(t 0 )_kK_r -1 IV (IV)
Wherein S (t) 0 ) K represents a translation parameter matrix,wherein t is x K represents the value of the translation vector of the PTZ sphere machine on the x-axis, t y K represents the value of the translation vector of the PTZ sphere machine on the y-axis, t z K represents the value of the translation vector of the PTZ sphere machine in the z-axis;
step 2, adopting LM algorithm iteration to solve the V to obtain the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, value theta of T-direction rotation angle of PTZ ball machine after optimization T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ′_k；
Wherein e l_n Coordinate value e representing the nth point in the first PTZ dome camera image r_n Coordinate values representing an nth point in the second PTZ dome camera image, n=1,..n, N is a positive integer, e l_n And e r_n Coordinate values for matching corresponding points; (j) 1 Element 1 of the representation vector j, (j) 2 Representing the 2 nd element of vector j; Φ (·) represents the parameter set to be optimized Φ (·) =Φ (f) x _k,f y _k,θ P _k,θ T _k)；
Step 3, according to the value theta of the P-direction rotation angle of the optimized PTZ ball machine P ' k, value theta of T-direction rotation angle of PTZ ball machine after optimization T 'k' and the optimized ball machine x is inward reference f x 'k' and optimized ball machine y inward parameter f y ' k, obtaining the rotation matrix R ' of the PTZ ball machine after optimization ' PT K and the optimized PTZ balloon machine internal parameter K' _k.
CN202110800603.XA 2021-07-15 2021-07-15 Target size estimation system based on trinocular vision Active CN113470118B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110800603.XA CN113470118B (en) 2021-07-15 2021-07-15 Target size estimation system based on trinocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110800603.XA CN113470118B (en) 2021-07-15 2021-07-15 Target size estimation system based on trinocular vision

Publications (2)

Publication Number Publication Date
CN113470118A CN113470118A (en) 2021-10-01
CN113470118B true CN113470118B (en) 2023-12-05

Family

ID=77880436

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110800603.XA Active CN113470118B (en) 2021-07-15 2021-07-15 Target size estimation system based on trinocular vision

Country Status (1)

Country Link
CN (1) CN113470118B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113949814B (en) * 2021-11-09 2024-01-26 重庆紫光华山智安科技有限公司 Gun-ball linkage snapshot method, device, equipment and medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015163830A1 (en) * 2014-04-22 2015-10-29 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi Target localization and size estimation via multiple model learning in visual tracking
CN106981083A (en) * 2017-03-22 2017-07-25 大连理工大学 The substep scaling method of Binocular Stereo Vision System camera parameters
CN110148169A (en) * 2019-03-19 2019-08-20 长安大学 A kind of vehicle target 3 D information obtaining method based on PTZ holder camera
CN110415278A (en) * 2019-07-30 2019-11-05 中国人民解放***箭军工程大学 The ptz camera that moves linearly assists principal and subordinate's tracking of binocular PTZ vision system
CN110430400A (en) * 2019-08-12 2019-11-08 中国人民解放***箭军工程大学 A kind of ground level method for detecting area of the movable video camera of binocular
CN112819904A (en) * 2021-03-15 2021-05-18 亮风台（上海）信息科技有限公司 Method and equipment for calibrating ptz camera
CN112946646A (en) * 2021-01-29 2021-06-11 西安电子科技大学 Satellite target attitude and size estimation method based on ISAR image interpretation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015163830A1 (en) * 2014-04-22 2015-10-29 Aselsan Elektronik Sanayi Ve Ticaret Anonim Sirketi Target localization and size estimation via multiple model learning in visual tracking
CN106981083A (en) * 2017-03-22 2017-07-25 大连理工大学 The substep scaling method of Binocular Stereo Vision System camera parameters
CN110148169A (en) * 2019-03-19 2019-08-20 长安大学 A kind of vehicle target 3 D information obtaining method based on PTZ holder camera
CN110415278A (en) * 2019-07-30 2019-11-05 中国人民解放***箭军工程大学 The ptz camera that moves linearly assists principal and subordinate's tracking of binocular PTZ vision system
CN110430400A (en) * 2019-08-12 2019-11-08 中国人民解放***箭军工程大学 A kind of ground level method for detecting area of the movable video camera of binocular
CN112946646A (en) * 2021-01-29 2021-06-11 西安电子科技大学 Satellite target attitude and size estimation method based on ISAR image interpretation
CN112819904A (en) * 2021-03-15 2021-05-18 亮风台（上海）信息科技有限公司 Method and equipment for calibrating ptz camera

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Motion and inertial parameter estimation of non-cooperative target on orbit using stereo vision;DongmingGe et al;《Advances in Space Research》;20200915;1-10 *
基于改进的3维ICP匹配的单目视觉里程计;袁梦等;《机器人》;20180131;第40卷(第1期);56-63 *

Also Published As

Publication number Publication date
CN113470118A (en) 2021-10-01

Similar Documents

Publication Publication Date Title
CN106643699B (en) Space positioning device and positioning method in virtual reality system
US8139111B2 (en) Height measurement in a perspective image
US8699005B2 (en) Indoor surveying apparatus
US20060227211A1 (en) Method and apparatus for measuring position and orientation
Zhang et al. A robust and rapid camera calibration method by one captured image
CN108810473B (en) Method and system for realizing GPS mapping camera picture coordinate on mobile platform
US20150116691A1 (en) Indoor surveying apparatus and method
Luhmann Precision potential of photogrammetric 6DOF pose estimation with a single camera
CN113487683B (en) Target tracking system based on trinocular vision
CN113345028B (en) Method and equipment for determining target coordinate transformation information
KR20210049581A (en) Apparatus for acquisition distance for all directions of vehicle
CN113534737B (en) PTZ (Pan/Tilt/zoom) dome camera control parameter acquisition system based on multi-view vision
CN112288825A (en) Camera calibration method and device, electronic equipment, storage medium and road side equipment
KR20130121290A (en) Georeferencing method of indoor omni-directional images acquired by rotating line camera
Ding et al. A robust detection method of control points for calibration and measurement with defocused images
CN108153417B (en) Picture compensation method and head-mounted display device adopting same
CN113470118B (en) Target size estimation system based on trinocular vision
Satoh et al. A head tracking method using bird's-eye view camera and gyroscope
CN113340272B (en) Ground target real-time positioning method based on micro-group of unmanned aerial vehicle
CN111105467A (en) Image calibration method and device and electronic equipment
CN111220118B (en) Laser range finder based on visual inertial navigation system and range finding method
CN113538596B (en) Moving target tracking system based on trinocular vision
CN113034615B (en) Equipment calibration method and related device for multi-source data fusion
JP3221384B2 (en) 3D coordinate measuring device
CN115334247A (en) Camera module calibration method, visual positioning method and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant