CN109427077A - It is a kind of for demarcating the processor of automobile viewing system - Google Patents
It is a kind of for demarcating the processor of automobile viewing system Download PDFInfo
- Publication number
- CN109427077A CN109427077A CN201710757152.XA CN201710757152A CN109427077A CN 109427077 A CN109427077 A CN 109427077A CN 201710757152 A CN201710757152 A CN 201710757152A CN 109427077 A CN109427077 A CN 109427077A
- Authority
- CN
- China
- Prior art keywords
- camera
- vertex
- coordinate
- vehicle
- automobile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The present invention relates to a kind of for demarcating the processor of automobile viewing system, the automobile viewing system includes multiple fish-eye cameras, the processor is for executing following computer program steps, to obtain the spin matrix Rt of each camera: when the vehicle is stationary, it obtains camera under camera coordinate system and calculates the normal vector z of equilateral triangle under camera coordinate system according to triangle side length and space geometry relationship to observation vector v1, v2, the v3 on three vertex of known marker；When automobile does advance linear motion along the X-axis of vehicle axis system, the x-axis unit vector of camera coordinate system is calculated；Calculate spin matrix Rt:Rt=(x, x × z, z) * (X, X × Z, Z) of camera‑1.Compared with prior art, the process that processor of the present invention uses takes full advantage of potential prior information, scaling scheme is succinct, and required calibration space is smaller, stated accuracy is high, anti-interference stronger, and the requirement to operator's technical ability is significantly reduced to the degree that driver oneself can do.
Description
Technical field
The present invention relates to a kind of processors with computer program, are for demarcating automobile and looking around more particularly, to one kind
The processor of system.
Background technique
With the increase of car ownership, the increasingly complexity of traffic environment parks have become the difficulty of driver safely
Topic.In order to reduce potential security risk in the process of parking, various auxiliary parking systems come into being, such as backsight reversing camera
The systems such as head, reversing radar, these systems mitigate driver to a certain extent and park burden, but there are still not intuitively with
And the defect of blind area cannot be completely eliminated.360 degree of viewing systems are real by the region that multiple cameras cover 360 degree of vehicle body surrounding
Vision dead zone is now eliminated, while 360 degree of viewing systems drive seamless spliced be presented at width vertical view picture of vehicle body circumferential images
The person of sailing, therefore show more intuitive.
Although value accounting of 360 degree of viewing systems in whole vehicle be not high, occupied when factory calibration
An important ring, the reason is that current scaling scheme also needs a large amount of manpower to participate in, which increase viewing systems to whole
The cost accounting of vehicle.It is not big enough to be worth accounting/cost accounting, will affect popularizing for viewing system.Mesh is described below
The factory calibration scheme of preceding mainstream:
Due to that can have installation deviation when same money vehicle can have assembling deviation and the installation of vehicle body surrounding camera, if
360 degree of viewing system image mosaics are carried out using one group of preset parameter, then these deviations will affect image mosaic effect, therefore
Offline calibration must be carried out to eliminate the influence of these deviations.There are the following problems for conventional plant scaling scheme: (1) to calibration ring
Border is more demanding, for example needs to stop vehicle onto fixed position, and required precision is very high, and which increase the technology of worker operation hardly possiblies
Degree, occupies more human time's costs；(2) the method is difficult to apply to the calibration of the shop 4S, is more difficult for driver personal institute
With indirectly increasing potential purchase vehicle cost.(3) the automatic Calibration scheme of a set of safe and convenient facilitates BEV product in the past
Market is filled after dress marketplace trend, is transitioned into stock market from developing market.
In fact, ginseng includes camera position parameter T:(t outside six of camerax,ty,tz) and rotation parameter R:(rx,
ry,rz), wherein R:(rx,ry,rz) of equal value with the spin matrix Rt of 3*3.Their property is different.Divide from measurement difficulty,
tx,ty,tzMeasurement difficulty it is smaller, an ordinary people is also that can measure with common tool, rx,ry,rzMeasurement
Difficulty is larger, and ordinary people can not be capable of measuring substantially.Divide from the requirement to precision, tx,tyError influence it is smaller, looking around
Performance on figure is subtle translation；tzError influence secondly, the performance on panoramic view is the variation of scale；rx,ry,
rzError influence it is very big, it will lead to the torsional deformation of image, not received by human eye.Height based on human eye is suitable
Ying Xing, tx,ty,tzSmall error be that will not influence final display effect and the guarantee to safety, it is unique to require
It is exactly the t of four camerasx,tyIncremental error must be consistent, rzThe amount of multiplying error must be consistent.
Currently based on the on-line proving scheme of lane detection there are lane line length and width are inaccurate, adjacent lane line is not right
Together, left and right camera calibration is difficult and calibration process (it is generally necessary on a highway) the disadvantages of there are security risks.
Summary of the invention
It is an object of the present invention to overcome the above-mentioned drawbacks of the prior art and provide a kind of environmental requirement is low, people
The power processor for being used to demarcate automobile viewing system at low cost.
The purpose of the present invention can be achieved through the following technical solutions:
It is a kind of for demarcating the processor of automobile viewing system, the automobile viewing system includes multiple flakes camera shootings
Head, the processor is for executing following computer program steps, to obtain the spin matrix Rt of each camera:
When the vehicle is stationary, it calculates according to the pixel coordinate of three vertex in shooting image and is imaged under camera coordinate system
Observation vector v1, v2, the v3 of head to three vertex of known marker, three vertex connecting lines composition of the marker etc.
Side triangle and be bonded with level ground, solve camera under camera coordinate system to three vertex observed range s1,
S2, s3 make the distance between three vertex meet d12=d23=d31=d, wherein d is the side length of equilateral triangle, is then obtained
The coordinate on three vertex under camera coordinate system, so that the normal vector of equilateral triangle be calculated, wherein the sight
Preferred method for solving of the ranging from s1, s2, s3 are as follows: assign initial value to s1, s2, s3, determine three vertex under camera coordinate system
Position p1, p2, p3, with d12、d23、d31Energy function E, E=(d are constructed with the error sum of squares of d12–d)2+(d23–d)2+
(d31–d)2, the convergency value of s1, s2 and s3 are obtained using iterative algorithm, to keep convergency value correct as a result, according to observed range
After the convergency value of s1, s2 and s3 obtain the normal vector of equilateral triangle, calculates and image grease head highness t under vehicle axis systemz, by tzWith
The priori measurement height of camera compares, if comparison result is greater than tolerance interval, assigns initial value to s1, s2 and s3 again and changes
In generation, calculates convergency value, until comparison result is acceptable；
When automobile does advance linear motion along the X-axis of vehicle axis system, marker under camera coordinate system is obtained respectively
Coordinate of a certain vertex in two different moments calculates the x-axis unit vector of camera coordinate system according to two coordinate differences；
Calculate spin matrix Rt:Rt=(x, x × z, z) * (X, X × Z, Z) of camera-1, wherein X indicates vehicle coordinate
It is X-axis unit vector, Z indicates that the unit vector of the Z axis of vehicle axis system, x, z are list corresponding with X, Z under camera coordinate system
Bit vector, x are actually parallel with the normal vector of equilateral triangle.
After obtaining spin matrix Rt, the processor is also used to execute following computer program steps, to be taken the photograph
As the horizontal position parameter t of headxAnd ty:
When turning center point C (0, c, 0) of the automobile under vehicle coordinate is turned with radii fixus, marker vertex is obtained
Different moments coordinate under camera coordinate system acquires multi-group data, and is converted into vehicle according to current calibrating parameters and sits
Coordinate under mark system is kept not if c is the known quantity that precision is met the requirements according to the distance of same vertices to turning center point
The principle column equation of change calculates the horizontal position parameter t of cameraxAnd tyError dxAnd dy, by dxAnd dyRespectively with camera
Horizontal position parameter priori value be added, obtain horizontal position parameter txAnd ty；If c is that unknown quantity or precision are unsatisfactory for requiring
Known quantity, then by same vertex shooting three times, a circle is determined using line-of-sight course, by the x-axis coordinate value in the center of circle with 0 it
Between gap as horizontal position parameter txError ex, by exIt is added, obtains with the priori value of the horizontal position parameter of camera
Horizontal position parameter tx, that is, the accurate solution to Y direction is abandoned, but retain and really solve to X-direction refinement, Y direction is then stayed
It is solved to subsequent overlapping region image registration techniques.
Preferably, automobile is turned with minimum turning radius, because turning radius can directly obtain in this case；
The multi-group data of acquisition are as follows: same vertex at least three different moments coordinate or at least two difference vertex at least two not
Coordinate in the same time.
Obtain the horizontal position parameter t of each cameraxAnd tyAfterwards, the processor is also used to execute following computer
Program step, to update the horizontal position parameter t of each cameraxAnd ty, retain publicly-owned error, eliminate privately owned error:
When vehicle stationary, the image that two camera i and j take the same marker vertex P is obtained, is utilized respectively
The respective calibrating parameters of the camera sought calculate coordinate P of the vertex P under vehicle axis systemiAnd Pj, change automobile position
Or marker vertex, at least two groups coordinate data is acquired, coordinate data is substituted into following formula, simultaneous calculates two cameras i and j
Privately owned error biAnd bj:
Pi–Pj=bi-bj
Privately owned error is added with the priori value of the horizontal position parameter of corresponding camera respectively, obtains updated level
Location parameter txAnd ty。
Compared with prior art, the invention has the following advantages that
(1) program in calibration process environment and technical requirements it is low, vehicle need not reach specific position, commonly horizontally
The image in face, fish-eye camera acquisition is simple, and the equilateral triangle for demarcating marker is easy arrangement.
(2) since staking-out work is accidental sex work for the shop 4S, employee is less likely to be difficult shape by professional training
At the experience handicraft that skill comes from practice, factory calibration scheme requires too high for 4S salesman's work.And the side that this patent program requires
Case is easy to operate, without stringent positioning requirements, is conducive to the shop the 4S maintenance of BEV viewing system, or even driver is facilitated to carry out certainly
Main operation, it is highly-safe, be conducive to fill market after allowing BEV viewing system to fill marketplace trend in the past, move towards storage from developing market
Market.
(3) use equilateral triangle pel scheme, a series of its advantages are utilized: first vertex of a triangle and side it
Between not stringent sequence distinguish, convenient for detection；Secondly the Environment identification degree of triangle pattern is higher, in general place seldom
There is the triangle pattern that can produce interference, detects generation of the cost of triangle than detecting other figures using image algorithm
Valence is lower, and error rate is lower；Again, the stability of triangular structure reduces the complexity of plane iterative algorithm, reduces calculation
Method converges to the risk of error result, avoids algorithmic statement to nonplanar possibility；Finally, the production of equilateral triangle
(drafting) process is simple and can guarantee accurate.
(4) potential prior information is taken full advantage of, it is unwise to the parameter of human eye sensitivity and to human eye that differentiation treats those
The parameter of sense, scaling scheme is succinct, and required calibration space is smaller, stated accuracy is high, anti-interference stronger, wants to operator's technical ability
Seek the degree that driver oneself can do that is significantly reduced to.
Detailed description of the invention
Fig. 1 is marker layout drawing in 4 process of embodiment (1)；
Fig. 2 is normal viewing angle of 4 camera of embodiment to triangle marker vertex；
Fig. 3 is the aerial prospective figure for the triangle observed in 4 camera coordinate system of embodiment；
When Fig. 4 is that 4 vehicle of embodiment is turned with minimum turning radius, movement of the vertex of a triangle relative to turning center
Relationship；
Fig. 5 is that 4 marker of embodiment is located at the layout drawing in overlapping region；
Fig. 6 is the demarcation flow figure of embodiment 4.
Specific embodiment
The present invention is described in detail with specific embodiment below in conjunction with the accompanying drawings.The present embodiment is with technical solution of the present invention
Premised on implemented, the detailed implementation method and specific operation process are given, but protection scope of the present invention is not limited to
Following embodiments.
Embodiment 1
A kind of for demarcating the processor of automobile viewing system, automobile viewing system includes multiple fish-eye cameras, processing
Device is for executing following computer program steps, to obtain the spin matrix Rt of each camera:
When the vehicle is stationary, it calculates according to the pixel coordinate of three vertex in shooting image and is imaged under camera coordinate system
Observation vector v1, v2, the v3 of head to three vertex of known marker, three vertex connecting lines composition equilateral triangle of marker
Shape and be bonded with level ground, solve camera under camera coordinate system to three vertex observed range s1, s2, s3,
The distance between three vertex are made to meet d12=d23=d31=d, wherein d is the side length of equilateral triangle, then obtains camera
The coordinate on three vertex under coordinate system, so that the normal vector z of equilateral triangle be calculated, wherein observed range s1,
The preferred method for solving of s2, s3 are as follows: assign initial value to s1, s2, s3, determine position p1 of three vertex under camera coordinate system,
P2, p3, with d12、d23、d31Energy function E, E=(d are constructed with the error sum of squares of d12–d)2+(d23–d)2+(d31–d)2, adopt
The convergency value of s1, s2 and s3 are obtained with iterative algorithm, to keep convergency value correct as a result, according to observed range s1, s2 and s3
After convergency value obtains the normal vector z of equilateral triangle, calculates and image grease head highness t under vehicle axis systemz(generally in vehicle axis system
In, the direction of Z axis is that straight down, X-direction is directed towards front, and Y direction is directed to front-right), by tzWith camera shooting
The priori measurement height of head compares, if comparison result is greater than tolerance interval, assigns initial value and iteration meter to s1, s2 and s3 again
Convergency value is calculated, until comparison result is acceptable；
When automobile does advance linear motion along the X-axis of vehicle axis system, marker under camera coordinate system is obtained respectively
Coordinate of a certain vertex in two different moments calculates the x-axis unit vector of camera coordinate system according to two coordinate differences；
Calculate spin matrix Rt:Rt=(x, x × z, z) * (X, X × Z, Z) of camera-1, wherein X indicates world coordinates
It is X-axis unit vector, Z indicates the unit vector of the Z axis of vehicle axis system.
After obtaining spin matrix Rt, processor is also used to execute following computer program steps, to obtain camera
Horizontal position parameter txAnd ty:
When turning center point C (0, c, 0) of the automobile under vehicle coordinate is turned with minimum turning radius, marker is obtained
Different moments coordinate of the vertex under camera coordinate system, acquisition multi-group data (can be same vertex at least three different moments
Coordinate or at least two difference vertex at least two different moments coordinate), and be transformed into vehicle according to current calibrating parameters
Under coordinate system, the principle column equation remained unchanged according to the distance of same vertices to turning center point calculates the level of camera
Location parameter txAnd tyError dxAnd dy, by dxAnd dyIt is added, obtains with the priori value of the horizontal position parameter of camera respectively
Horizontal position parameter txAnd ty。
Obtain the horizontal position parameter t of each cameraxAnd tyAfterwards, processor is also used to execute following computer program step
Suddenly, to update the horizontal position parameter t of each cameraxAnd ty, retain publicly-owned error, eliminate privately owned error:
When vehicle stationary, the image that two camera i and j take the same marker vertex P is obtained, is utilized respectively
The respective calibrating parameters of the camera sought calculate coordinate P of the vertex P under vehicle axis systemiAnd Pj, change automobile position
Or marker vertex, at least two groups coordinate data is acquired, coordinate data is substituted into following formula, simultaneous calculates two cameras i and j
Privately owned error biAnd bj:
Pi–Pj=bi-bj
Privately owned error is added with the priori value of the horizontal position parameter of corresponding camera respectively, obtains updated level
Location parameter txAnd ty。
In turning center point C (0, c, 0), turning radius c is known quantity or unknown quantity, when c is unknown quantity, by same
One vertex is shot three times, is determined a circle using line-of-sight course, round radius is obtained, as turning radius c.
The processor of the present embodiment can be the combination, general of the video frequency collection card and industrial personal computer that are integrated in fish-eye camera
Processor, digital signal processor, specific integrated circuit, one or more combinations in microprocessor.
Embodiment 2
A kind of automobile viewing system caliberating device, including processor described in embodiment 1 and the mark for being arranged in level ground
Know object, marker is the equilateral triangle dough sheet or equilateral triangle frame to fit with level ground.
It is core of the invention thinking using equilateral triangle as element figure, this has broken current most marks
Determine the habit thinking demarcated in scheme using lines or square.Equilateral triangle has some special advantages: firstly,
User can make an absolute accurate equilateral triangle of three isometric line segments, then be cut into plank, or on ground
Face corresponding region is painted certain color and is all easy to, and other any shapes do not have this condition (user do not need in order to
It demarcates the viewing system of oneself and extra purchase accessory, but ruler and a rope can be done it yourself with one).Its
Secondary, the not stringent sequence requirement in three vertex of triangle and three sides, three vertex can uniquely determine a plane, this
A property makes that effective solution can be rapidly converged in the iterative process of algorithm to avoid many ambiguities.It is more than more than three points
Pel, more problems can be faced in optimization process, for example converge to locally optimal solution, or converge to one it is nonplanar
As a result.
To sum up, the either calibration of the shop 4S or user oneself calibration, method is the same, if look for one it is flat and
Texture region few as far as possible, spreads and (draws) equilateral triangle of several pieces of fixed side lengths, triangle on position and angle all
There is no strict requirements, can be arbitrary.Then it keeps straight on and turns according to certain requirement, so that it may complete calibration.
Note: during actual implementation, the shape for being not intended to limit calibration object has to be triangle.Because being mentioned in this patent
To normal vector computational algorithm in, not being distinctly claimed target is equilateral triangle, as long as three vertex can explicitly be distinguished
Distance each other can.If specific triangle is then decomposited from the figure again using other figures
Method described in this patent still can be used in subconstiuent.In this case, it should it is applied to be equally considered as this patent
Within protection scope.
Embodiment 3
A kind of automobile viewing system with caliberating device described in embodiment 2, including caliberating device and the camera shooting of multiple flakes
Head.Fish-eye camera can be to be mounted on the four ultra-wide angle fish-eye cameras of vehicle body all around, and when work, four cameras are same
When acquire vehicle's surroundings image, by image processing unit distort reduction → visual angle conversion → image mosaic → image enhancement,
Ultimately form 360 degree of seamless panorama top views of a width vehicle's surroundings.While showing panorama sketch, it can also show any
The single-view of one side, and scale line is cooperated to be accurately located position and the distance of barrier.Staking-out work formally goes up road in vehicle
It carries out before, as shown in Figure 1, it is shown that the top view of certain disposing way of marker triangle, triangle are equally arranged in
Vehicle body guarantees that visually placement position and angle are not strict with all around and as close as possible to camera.
Embodiment 4
The present embodiment is viewing system scaling method matched with Examples 1 to 3, first to fish-eye camera before calibration
Carry out non-distortion rectification.After non-distortion rectification, triangle can be detected by straight-line detection, and with lines come cocked hat
Vertex, it is accurately more than being positioned to angle point.
It is demarcation flow below:
(1) stationary vehicle, by four triangle pendulum in the position close to camera
Four triangles are put by the vertical view angle of field approximate location of Fig. 1, its position and angle are not wanted strictly
It asks, as long as guaranteeing three vertex of triangle all completely in the picture.Fig. 2 is normal viewing angle, and Fig. 3 is camera coordinate system
In the aerial prospective figure of triangle observed.In Fig. 3, origin O corresponds to the position of camera, P1, P2, and P3 corresponds to triangle
Three vertex, plane corresponding to triangle is exactly ground.V1, v2, v3 correspond to camera to the observation vector on three vertex, are
It is known, it is calculated by the location of pixels of three vertex on the image.S1, S2, S3 correspond to camera to three vertex away from
From being unknown.D12, d23, d31 correspond to three side lengths of triangle.
Enabling the coordinate of triangle on the image is (Pix1, Pix2, Pix3), can be calculated in camera coordinate system by it
Observation vector (v1, v2, v3) (known) of the camera to three vertex.Enabling (p1, p2, p3) is that triangular apex is sat in camera
The position observed in mark system, (s1, s2, s3) are observed range (unknown).(p1, p2, p3) is unknown, but as long as being aware of
(s1, s2, s3), so that it may calculate (p1, p2, p3).These three observed ranges (s1, s2, s3) are solved below, make (p1, p2, p3)
The distance between these three vertex meet: d12=d23=d31=d. wherein d be equilateral triangle side length.
Three equations can be listed according to condition above, but this equation group is difficult to find out analytic solutions, it can be with
With d12,d23,d31Energy function E=(the d of (s1, s2, s3) is constructed with the error sum of squares of d12–d)2+(d23–d)2+(d31–d
)2=f (s1, s2, s3), then iteratively solves.
This energy theorem be similar to spring energy storage formula (if p1, the line between p2, p3 is imagined as cannot be curved
If bent spring), it is possible to directly energy theorem above is not iterated, a kind of more intuitive base is provided here
In the iterative scheme of physical model:
In the camera coordinate system, tri- vectors of v1, v2, v3 are known and fixed.Initial value is assigned to s1, s2, s3, just only
One has determined p1, p2, p3 and their distance d12,d23,d31, general d under original state12≠d23≠d31≠ d. P1, P2,
The complete rings on three brackets v1, v2, v3 are imagined on tri- vertex P3, P1, P2, and the connecting line between P3 is imagined as spring,
Spring cannot be bent, and can only be stretched, and when the length that it is under relaxed state is d, pulling force be generated when being greater than d, when being less than d
Repulsion is generated, slides ring on bracket under the action of pulling force or repulsion.PiPoint is to PjThe power that point generates is expressed as fij,fijGreatly
It is repulsion when 0, is pulling force when less than 0, it is evident that fij=fji。
Assuming that the power that P1 point and P2 point generate P3 point is f respectively13And f23, them can be calculated and project to bracket V3
Power on direction is respectively F13And F23, comprehensive function power F3=F that P3 point is currently subject to13+F23Similarly calculate the current institute of each point
Resultant force F1, F2, the F3 being subject to, they are along holder orientation.The learning rate that a very little is added to this power, makes
Small displacement occurs under force for P1, P2, P3, and iteration continues in this way until d under convergence, convergence state12=d23=d31
=d.
It notices on a complicated pel, such as square, we, which understand problems faced, has: algorithm may converge to
One it is non-planar on or algorithm although converge in a plane, but have more Local Extremum, more a variety of convergences knots
Fruit makes subsequent processing become very troublesome.The convergence result of algorithm based on equilateral triangle pel is centainly in one and puts down
On face, and three kinds of different local minimum positions only at most can be converged to, in our actual tests, the algorithm is almost total
Correct result can be converged to.
Assuming that converge to mistake as a result, it is following can use camera shooting grease head highness prior information result is excluded.
Because being aware of three vertex to the observed range (s1, s2, s3) of camera, plane where triangle has just been uniquely determined, and
The normal vector z of this plane, the normal vector z for then passing through the plane carry out the height t of calculating observation positionz(tzIt is the height of camera
Vertical height of the observation point O to plane where triangle in degree and figure).If this is highly measured with the priori of camera
Height is not obviously inconsistent, then can repel this and be iterated as a result, reselecting initial value.Generally going through this step can be obtained by
Unique correct solution.Then it is understood that spin matrix Rt maps the Z axis unit vector Z=(0,0,1) in world coordinate system
Triangulation method vector z into camera coordinate system, wherein z and Z is known, we have obtained the first of spin matrix Rt
A expression formula: Rt*Z=z.
Determine that the X-axis unit vector (1,0,0) in world coordinate system, Y-axis unit vector (0,1,0) are right respectively separately below
Answer the vector x and y in camera coordinate system.
(2) steering wheel is ajusted, and vehicle is allowed to keep straight on
It allows vehicle to keep straight on a small distance, if can not enough keep straight on back and forth, but should ensure that triangle primitives always right
In the camera view answered.It is assumed that detecting that the ground location of triangle is T at the t0 moment0(P10,P20,P30), at the t1 moment
Detecting (tracking) and arriving the position of triangle is T1(P11,P21,P31), corresponding position in the camera coordinate system is t0(p10,
p20,p30) and t1(p11,p21,p31).Because vehicle is straight trip, pass through P10-P11Obtained vector VP is obviously and vehicle coordinate
The X-axis of system is parallel.It is corresponding, in the camera coordinate system, enable p10-p11=vp, then it is understood that spin matrix Rt reflects VP
It is mapped to vp.By VP to be become to the X-axis unit vector X (1,0,0) of world coordinate system, vp is become to image multiplied by a coefficient
Vector x in head coordinate system, it is of equal value: Rt*X=x.Use (p20,P21), (p30,P31) to do the effect of above-mentioned processing be the same
, therefore the multiple points that can be will be observed that integrate, and keep result more accurate.
So far the Z axis in vehicle axis system and the corresponding correspondence in the camera coordinate system of X-axis positive direction has been obtained
Vector z and x, it is assumed that spin matrix Rt then has:
Rt* (X, X × Z, Z)=(x, x × z, z)
Rt=(x, x × z, z) * (X, X × Z, Z)-1
Wherein X × Z refers to the apposition of vector X and Z, and x × z is the apposition of x and z, this can guarantee that calculated result is
Spin matrix, rather than rotate the compound matrice with reflection transformation.So far, spin matrix Rt and camera has been obtained in we
Height tz, there remains two parameter txAnd tyUsually the two parameters have directly just been able to satisfy using priori value when dispatching from the factory
Precision needs, but still need to demarcate in rear dress.In the state of vehicle straight trip, it is impossible to calibrate txAnd ty
Two parameters.If it is desired to further calibrating the two parameters, needs to know the turning radius of vehicle, then pass through maximum radius
A nonlinear transformation condition is created in turning.Certainly if it is not known that vehicle minimum turning radius also has no relations, t after allxWith
tyPriori measurement had been provided with certain precision, t can be utilized in step (4)xAnd tyPriori measurement, then
Registration by four cameras in overlapping region, to minimize measurement error, Lai Tigao txAnd tyPrecision, it is ensured that four figure
As being matched in overlapping region.
(3) vehicle is turned with minimum turning radius
If the minimum turning radius of our known vehicles, and turned with minimum turning radius, it can front and back
Small size traveling repeatedly.Assuming that the coordinate of turning center C is (0, c, 0), divide A, two kinds of situations of B below.
A. the precision of minimum turning radius c is enough
When Fig. 4 shows that vehicle is turned with minimum turning radius, the vertex of a triangle observed is relative to turning center
Movement relation.P is any one vertex of the triangle detected, and P (t0) and P (t1) are that it is corresponding at t0 the and t1 moment
Ground location.(since we have calculated that spin matrix Rt, and the height t of cameraz, for txAnd ty, we temporarily adopt
With the measured value of priori, complete mapping equation is thus obtained, so the point on image is mapped to the coordinate on ground
It is feasible)
Since c is known and accurate, then the distance of P (t0) to C should be equal to the distance that P (t1) arrives C, it will be assumed that P
(t0) coordinate is (Px 0,Py 0, 0), the coordinate of P (t1) is (Px 1,Py 1, 0), and current outer ginseng txAnd tyThere is error d respectivelyxWith
dy, then have:
((Px 0+dx)-0)2+((Py 0+dy)-c)2=((Px 1+dx)-0)2+((Py 1+dy)-c)2
To above formula abbreviation, quadratic term can about fall, the result is that dxAnd dyAn expression formula.Such a expression formula, there are two
D can be solvedxAnd dy, and there are three vertex for our triangle, and can select during vehicle turning more
Point forms an oversaturated expression formula, solves the smallest solution of error sum of squares with least square method.The d that will be solvedxAnd dyAdd
To original txAnd tyOn, so that it may obtain accurate result.
So far, each camera has been calculated separately and has joined T (t outside sixx,ty,tz) and R:(rx,ry,rz)
B. minimum turning radius c is unknown or precision is inadequate
T at this timeyIt is unable to get accurate solution, but txIt is possible.But algorithm is different, needs to vertex of a triangle
Continuous tracking is done, the position of the point of three sequences is at least obtained.Since vehicle is to turn, three points can not be straight at one
On line, thus may determine that a unique circle.By the different observation positions on different vertex come superposition calculation, the center of circle can be made
Precision it is higher.The gap of this center location and vehicle theory turning center (0, c) is exactly txAnd tyError current exWith
ey.Since the precision of c is inadequate, the parameter of X-direction: t can only be updated herex=tx+ex.
(4) four triangles are put into overlapping region by overlapping region image registration
Regardless of step 3 is to be executed in the form of A or executed in the form of B, step 4 here be may serve to further
Improve txAnd tyPrecision.Four cameras, label are respectively 1,2,3,4, and each camera has a pair of of location parameter txAnd ty
Have to be optimized, is 8 parameters to be optimized, t in total1(tx 1,ty 1),t2(tx 2,ty 2),t3(tx 3,ty 3),t4(tx 4,ty 4), enable T1,
T2,T2,T4It is their exact value, ei=ti–TiIt is current error.By eiIt is decomposed into publicly-owned error ciWith privately owned error bi, ei
=ci+bi, publicly-owned error is consistent to four cameras, c1=c2=c3=c4=c, and the privately owned error of four cameras is mutually only
It is vertical:
tx 1=Tx 1+cx+bx 1；ty 1=Ty 1+cy+by 1；
tx 2=Tx 2+cx+bx 2；ty 2=Ty 1+cy+by 2；
tx 3=Tx 3+cx+bx 3；ty 3=Ty 1+cy+by 3；
tx 4=Tx 4+cx+bx 4；ty 4=Ty 1+cy+by 4；
It, can be by their privately owned error b using the image registration algorithm of overlapping regioni(bx i,by i) remove, thus by this
8 random parameters (8 freedom degrees) are converted into two random parameter c (cx,cy) (two freedom degrees), and theoretically, this is remaining
Publicly-owned error the calibrating parameters t of first four camera that is centainly less than of variancei(tx i,ty i) variance average value.Process
It is as follows:
According to shown in Fig. 5, four triangles are placed in respectively in four overlapping regions, make each triangle can be by phase
Two adjacent cameras observe that the position and angle to triangle are all without exact requirements simultaneously.Due to current calibrating parameters
In addition to txAnd tyHave outside a small amount of error, other parameters are all that accurately, the triangle on image is projected in ground coordinate
Afterwards, apparent deformation is not had, is only slightly translated, therefore the same triangle that different cameras observe can be accurate
Matching.
Assuming that the ground coordinate that some vertex P of any one triangle is observed in two adjacent camera i and j
Respectively PiAnd Pj, then have: Pi–Pj=ei–ej=(c+bi)-(c+bj)=bi-bj
Can obtain any number of points by the position of changable triangle, thus expression above can have it is enough
It is a, they are got up side by side, b is calculated by least square method1,b2,b3,b4Optimal solution, then updated calibration ginseng
Number:
tx 1-bx 1=Tx 1+cx；ty 1-by 1=Ty 1+cy；
tx 2-bx 2=Tx 2+cx；ty 2-by 2=Ty 2+cy；
tx 3-bx 3=Tx 3+cx；ty 3-by 3=Ty 3+cy；
tx 4-bx 4=Tx 4+cx；ty 4-by 4=Ty 4+cy；
New parameter has subtracted the privately owned error of each camera on the basis of original, only remains publicly-owned error, to four
Camera is identical, has generally so far met the requirements.
As long as the software interactive process being designed correctly, entire calibration process can accomplish without any letup.Calibration process is substantially
On can summarize are as follows: (1) as Fig. 1 puts triangle, vehicle slightly front and back straight trip starts to adopt figure, waits software feedback success (2)
Vehicle is turned with maximum turning radius, is slightly travelled, and starts to adopt figure, software feedback success (3) stationary vehicle is waited, by three
Angular pel resets the overlapping region being placed in such as Fig. 5, starts to adopt figure, waits software feedback success, then terminates.
Claims (10)
1. a kind of for demarcating the processor of automobile viewing system, the automobile viewing system includes multiple fish-eye cameras,
It is characterized in that, the processor is used to execute following computer program steps, to obtain the spin matrix of each camera
Rt:
When the vehicle is stationary, obtain camera coordinate system under camera to three vertex of known marker observation vector v1,
V2, v3, three vertex connecting lines of the marker constitute equilateral triangle and are bonded with level ground, according to triangle edges
Long and space geometry relationship calculates the normal vector of equilateral triangle under camera coordinate system；
When automobile does advance linear motion along the X-axis of vehicle axis system, it is a certain that marker under camera coordinate system is obtained respectively
Coordinate of the vertex in two different moments calculates the X-axis unit vector of vehicle axis system in camera coordinate according to two coordinate differences
Correspondence x-axis unit vector under system；
Calculate spin matrix Rt:Rt=(x, x × z, z) * (X, X × Z, Z) of camera-1, wherein X, Z indicate vehicle axis system X
Axis unit vector, Z axis unit vector, x, z indicate the corresponding form under camera coordinate system.
2. a kind of automobile viewing system caliberating device according to claim 1, which is characterized in that when the vehicle is stationary, take the photograph
As under head coordinate system camera to three vertex observation vector v1, v2, v3 according to three vertex in shooting image
Pixel coordinate calculates.
3. a kind of automobile viewing system caliberating device according to claim 1, which is characterized in that when the vehicle is stationary, take the photograph
The normal vector z calculation method of plane as where marker under head coordinate system are as follows: camera is described under solution camera coordinate system
Observed range s1, s2, the s3 on three vertex, make the distance between three vertex meet d12=d23=d31=d, wherein d is known
Equilateral triangle side length, the coordinate on three vertex under camera coordinate system is then obtained, to be calculated equilateral
The normal vector z of triangle.
4. a kind of automobile viewing system caliberating device according to claim 3, which is characterized in that the observed range
S1, s2, s3 method for solving are as follows: assign initial value to s1, s2, s3, determine position p1, p2 of three vertex under camera coordinate system,
P3, with d12、d23、d31Energy function E, E=(d are constructed with the error sum of squares of d12–d)2+(d23–d)2+(d31–d)2, use
Iterative algorithm obtains the convergency value of s1, s2 and s3.
5. a kind of automobile viewing system caliberating device according to claim 4, which is characterized in that according to observed range s1,
After the convergency value of s2 and s3 obtains the normal vector z of equilateral triangle, calculates and image grease head highness t under vehicle axis systemz, by tzWith take the photograph
Compare as the priori of head measures height, if comparison result is greater than tolerance interval, initial value and iteration are assigned to s1, s2 and s3 again
Calculate convergency value.
6. a kind of automobile viewing system caliberating device according to claim 1, which is characterized in that obtain camera spin moment
After battle array Rt, the processor is also used to execute following computer program steps, to obtain the horizontal position parameter of camera
txAnd ty:
When turning center point C (0, c, 0) of the automobile under vehicle coordinate is with fixed semidiameter turn, obtains marker vertex and exist
Different moments coordinate under camera coordinate system acquires multi-group data, and is converted into vehicle coordinate according to current calibrating parameters
Coordinate under system is kept if c is the known quantity that precision meets sets requirement according to the distance of same vertices to turning center point
Constant principle column equation calculates the horizontal position parameter t of cameraxAnd tyError dxAnd dy, by dxAnd dyRespectively with camera shooting
The priori value of the horizontal position parameter of head is added, and obtains horizontal position parameter txAnd ty。
7. a kind of automobile viewing system caliberating device according to claim 6, which is characterized in that automobile is with minimum turning half
Diameter turning.
8. a kind of automobile viewing system caliberating device according to claim 6, which is characterized in that the multi-group data of acquisition
Are as follows: same vertex at least three different moments coordinate or at least two difference vertex at least two different moments coordinate.
9. a kind of automobile viewing system caliberating device according to claim 6, which is characterized in that obtain the water of each camera
Flat location parameter txAnd tyAfterwards, the processor is also used to execute following computer program steps, to update each camera
Horizontal position parameter txAnd ty:
When vehicle stationary, the image that two camera i and j take the same marker vertex P is obtained, is utilized respectively and has asked
The respective calibrating parameters of the camera taken calculate coordinate P of the vertex P under vehicle axis systemiAnd Pj, change automobile position or mark
Know object vertex, acquire at least two groups coordinate data, coordinate data is substituted into following formula, simultaneous calculates the privately owned of two cameras i and j
Error biAnd bj:
Pi–Pj=bi-bj
Privately owned error is added with the priori value of the horizontal position parameter of corresponding camera respectively, obtains updated horizontal position
Parameter txAnd ty。
10. a kind of automobile viewing system caliberating device according to claim 6, which is characterized in that if c is unknown quantity or essence
Degree is unsatisfactory for desired known quantity, then by three times, a circle being determined using line-of-sight course, by the x in the center of circle to the shooting of same vertex
Gap between axial coordinate value and 0 is as horizontal position parameter txError ex, by exWith the elder generation of the horizontal position parameter of camera
Value addition is tested, horizontal position parameter t is obtainedx。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710757152.XA CN109427077B (en) | 2017-08-29 | 2017-08-29 | Processor for calibrating automobile all-round looking system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710757152.XA CN109427077B (en) | 2017-08-29 | 2017-08-29 | Processor for calibrating automobile all-round looking system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109427077A true CN109427077A (en) | 2019-03-05 |
| CN109427077B CN109427077B (en) | 2021-10-12 |
Family
ID=65502030
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710757152.XA Active CN109427077B (en) | 2017-08-29 | 2017-08-29 | Processor for calibrating automobile all-round looking system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109427077B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910311A (en) * | 2019-10-30 | 2020-03-24 | 同济大学 | Automatic splicing method for multi-channel panoramic camera based on two-dimensional code |
| CN114463439A (en) * | 2022-01-18 | 2022-05-10 | 襄阳达安汽车检测中心有限公司 | Vehicle-mounted camera correction method and device based on image calibration technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5675407A (en) * | 1995-03-02 | 1997-10-07 | Zheng Jason Geng | Color ranging method for high speed low-cost three dimensional surface profile measurement |
| JP2011226931A (en) * | 2010-04-20 | 2011-11-10 | Fujitsu Ten Ltd | Calibration method |
| KR20130115717A (en) * | 2012-04-13 | 2013-10-22 | 주식회사 이미지넥스트 | Vehicle installed camera extrinsic parameter estimation method and apparatus, and calibration plate using thereof |
| CN103617606A (en) * | 2013-11-26 | 2014-03-05 | 中科院微电子研究所昆山分所 | Vehicle multi-angle panorama generating method for aided driving |
| CN204595936U (en) * | 2015-04-22 | 2015-08-26 | 寅家电子科技（上海）有限公司 | A kind of calibration mark thing for vehicle-mounted camera parameter calibration |
| CN105691293A (en) * | 2014-11-27 | 2016-06-22 | 德尔福电子(苏州)有限公司 | Automatic control system and method for automobile steering lamps |
-
2017
- 2017-08-29 CN CN201710757152.XA patent/CN109427077B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5675407A (en) * | 1995-03-02 | 1997-10-07 | Zheng Jason Geng | Color ranging method for high speed low-cost three dimensional surface profile measurement |
| JP2011226931A (en) * | 2010-04-20 | 2011-11-10 | Fujitsu Ten Ltd | Calibration method |
| KR20130115717A (en) * | 2012-04-13 | 2013-10-22 | 주식회사 이미지넥스트 | Vehicle installed camera extrinsic parameter estimation method and apparatus, and calibration plate using thereof |
| CN103617606A (en) * | 2013-11-26 | 2014-03-05 | 中科院微电子研究所昆山分所 | Vehicle multi-angle panorama generating method for aided driving |
| CN105691293A (en) * | 2014-11-27 | 2016-06-22 | 德尔福电子(苏州)有限公司 | Automatic control system and method for automobile steering lamps |
| CN204595936U (en) * | 2015-04-22 | 2015-08-26 | 寅家电子科技（上海）有限公司 | A kind of calibration mark thing for vehicle-mounted camera parameter calibration |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110910311A (en) * | 2019-10-30 | 2020-03-24 | 同济大学 | Automatic splicing method for multi-channel panoramic camera based on two-dimensional code |
| CN110910311B (en) * | 2019-10-30 | 2023-09-26 | 同济大学 | Automatic splicing method of multi-path looking-around camera based on two-dimension code |
| CN114463439A (en) * | 2022-01-18 | 2022-05-10 | 襄阳达安汽车检测中心有限公司 | Vehicle-mounted camera correction method and device based on image calibration technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109427077B (en) | 2021-10-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103759669B (en) | A kind of monocular vision measuring method of heavy parts | |
| CN102927908B (en) | Robot eye-on-hand system structured light plane parameter calibration device and method | |
| US9858639B2 (en) | Imaging surface modeling for camera modeling and virtual view synthesis | |
| CN103759670B (en) | A kind of object dimensional information getting method based on numeral up short | |
| Scaramuzza et al. | Extrinsic self calibration of a camera and a 3d laser range finder from natural scenes | |
| CN111750820B (en) | Image positioning method and system | |
| Oliveira et al. | Multimodal inverse perspective mapping | |
| KR20170015227A (en) | Drive-by calibration from static targets | |
| CN101216937B (en) | Parameter calibration method for moving containers on ports | |
| CN106204625A (en) | A kind of variable focal length flexibility pose vision measuring method | |
| KR101583663B1 (en) | Method for generating calibration indicator of camera for vehicle | |
| CN102254318A (en) | Method for measuring speed through vehicle road traffic videos based on image perspective projection transformation | |
| CN112561841A (en) | Point cloud data fusion method and device for laser radar and camera | |
| CN102650886A (en) | Vision system based on active panoramic vision sensor for robot | |
| CN103065323A (en) | Subsection space aligning method based on homography transformational matrix | |
| CN106157322B (en) | A kind of camera installation site scaling method based on plane mirror | |
| CN106500596A (en) | The measuring method of structure light panorama measuring system | |
| CN103258329A (en) | Camera calibration method based on one-dimensional feature of balls | |
| CN110009682A (en) | A kind of object recognition and detection method based on monocular vision | |
| CN105526906B (en) | Wide-angle dynamic high precision laser angular measurement method | |
| CN109724586B (en) | Spacecraft relative pose measurement method integrating depth map and point cloud | |
| CN108469254A (en) | A kind of more visual measuring system overall calibration methods of big visual field being suitable for looking up and overlooking pose | |
| CN104422425A (en) | Irregular-outline object space attitude dynamic measuring method | |
| CN104482921A (en) | Water surface target measuring method | |
| CN109255819A (en) | Kinect scaling method and device based on plane mirror |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |