CN102360499A - Multi-lane line tracking method based on Kalman filter bank - Google Patents

Abstract

The invention discloses a multi-lane line tracking method based on a Kalman filer bank, which belongs to the technical field of processing of digital images. The method comprises the following steps of: outputting the optimal estimation value of a lane line by using a Kalman filter model on the basis of performing lane line detection on an acquired vehicle front road image by establishing the Kalman filter model tracked by the lane line; and when the same lane line is detected through continuous 25 road images, determining that the optimal estimation of a Kalman filter is effective, and using the optimal estimation value output by the Kalman filter as a detection result. By the method, the detection result of the lane line in a lane departure warning system is more stable, so that the phenomena of inaccurate lane line detection or detection omission caused by vehicle body vibration, illumination variation, road surface damages and the like are greatly reduced; and according to set quantity of filters, a corresponding lane line number can be tracked simultaneously, a region of interest (ROI) of the lane line can be relatively unitarily tracked, and the detection accuracy and the disturbance resistance of the lane line are greatly improved.

Description

A kind of Multi-lane Lines method for tracing based on group of Kalman filters

Technical field

The invention belongs to the digital image processing techniques field, relate generally to the method for following the trail of simultaneously to detected many lane lines in the road video image.

Background technology

(Lane Departure Warning System LDWS), is that a kind of mode driver assistance person through warning reduces the system of automobile because of the deviation occurrence of traffic accident in the lane departure warning system.Socioeconomic continuous progress; And the develop rapidly of The present computer technology; New technologies such as multi-sensor technology and Control on Communication theory have been merged; Make the popularization and application of LDWS become possibility, thereby improve the security of car steering to greatest extent, social economy's loss is reduced to bottom line.

At present the implementation of the LDWS method that all is based on computer vision realizes, at first in the road video image of gathering, discerns lane line, judges whether to send according to the relation of vehicle location and lane line again and departs from early warning.Because lane line is whether vehicle is departed from the reference of judging, so the most important condition that LDWS realizes is exactly to detect lane line accurately through the method for Digital Image Processing.Yet vehicle is in the process of moving because vehicle body shake, light variation, road surface dirt and lane line breakage or the like situation all can cause same lane line target detection unstable result in the road video image that collects; Thereby cause a certain frame or the complete target deviation of a few two field picture or lose, it is inaccurate to influence the LDWS early warning.

Therefore, after the method that adopts Digital Image Processing detects lane line, must adopt the method for target tracking, detected lane line carries out sample training as sample, and prediction optimal estimation value is accomplished the tracking of lane line.The optimal estimation value of output replaces observed reading and carries out early warning at last, will make the LDWS duty more steady more, greatly reduces because of lane line and detects error to the LDWS Effect on Performance.Therefore the present invention proposes and adopt group of Kalman filters simultaneously detected many lane lines to be followed the trail of, can improve the lane line accuracy of detection greatly.

The implementation method of at present lane line being carried out target tracking has following several kinds:

1, will comprise fully detected lane line at the coordinate of interior quadrilateral area as sample information input card Thalmann filter; Thereby predict the zone that comprises lane line in next width of cloth sequence image; It is defined as area-of-interest (Region of Interest; ROI), to accelerate lane line detection speed and accuracy, utilized single Kalman filter that the area-of-interest that comprises lane line is followed the trail of.Though this method has been accelerated detection speed, when in the ROI that predicts the outcome, can not detecting lane line, system is with regard to cisco unity malfunction.This method sees document for details: Zheng Banggui, Tian Ping Xiang, Duan Jianmin. based on the Research on Lane Detection Based on Hough Transform of Kalman prediction and contrary projection. and computer engineering and design .2009,30 (6).

2, to the prediction of area-of-interest wicket.According to the lane line that has detected; Define the part that a zonule has only comprised a lane line; Thereby the zonule that this zonule coordinate is comprised this lane line as the sample information input card Thalmann filter prediction next one; This method is the improvement to 1 method; Area-of-interest is narrowed down to certain part that only comprises a lane line from comprising all lane lines, improved the lane line accuracy of detection, but still can not avoid LDWS can not normally detect the problem of lane line fully; This method sees document for details: surplus spissatus, and Zhang Weigong. based on the lane line recognition and tracking of dynamic area-of-interest. industrial instrument and the 5th phase of automation equipment .2009.

3, adopt other algorithm filters to come lane line is followed the trail of,, set up the dynamic vision window that follow the tracks of in the track as adopting extended Kalman filter technology and image processing techniques; Extract the method for lane boundary again, see document for details: Chen Ying, Han Chongzhao. merge based on the track of EKF and follow the tracks of. highway communication science and technology .Vol.12; No.12, Nov.2004. also have the Screening Treatment algorithm that adopts a kind of effective ROI of traffic lane line that combines based on contrast and big Tianjin law, and effective ROI has been carried out the adaptive threshold image segmentation; Through effective screening to ROI, can reduce influences such as discontinuous place of traffic lane line or breakage, see document for details: Guan Xin; Jia Xin; Gao Zhenhai. area-of-interest selection and adaptive threshold are cut apart in the lane detection. highway communication science and technology .Vol.26, No.6, Jun.2009.

Above-mentioned various tracing algorithm all is to adopt single wave filter that lane line ROI is followed the trail of, thereby then in ROI, discerns the tracking that lane line is accomplished lane line indirectly.Along with steady development of economy; The high grade pavement track is more and more wideer, and the image quality of image capture device is also increasingly high simultaneously, will inevitably cause piece image many lane lines to occur; Adopt above-mentioned tracing algorithm to be difficult to each bar lane line that accurate trace detection arrives this moment; Under this background, the present invention adopts group of Kalman filters directly detected many lane line targets to be followed the trail of, and no longer need follow the trail of lane line ROI zone.

Summary of the invention

The present invention proposes a kind of Multi-lane Lines method for tracing based on group of Kalman filters, and this method need not preestablish the lane line area-of-interest, can directly follow the trail of detected Multi-lane Lines.This method can improve the accuracy that lane line detects greatly, does not influence the requirement of LDWS for real-time simultaneously.

Describe content of the present invention for ease, at first some terms are defined.

Definition 1: Kalman filtering.(Kalman Filter KF) is the recursive data Processing Algorithm of estimating the linear dynamic system state to Kalman filtering, and through to the dynamically updating of sample training, constantly NextState and measurement data are described in prediction.Kalman filter principle and implementation procedure see document for details: He Zishu. modern digital signal Processing and application thereof. and publishing house of Tsing-Hua University. chapter 7: Kalman filtering.

Definition 2: Hough transformation.Be the Hough conversion, utilize the duality between straight line plane space and Hough planar point, line, the straight line that promptly intersects in the corresponding Hough of the some plane of conllinear in the image straight line space; Conversely, the Hough Plane intersects all has the point of conllinear corresponding with it in image space in all straight lines of same point.Utilizing Hough transformation can the linear feature search problem be converted into Hough plane maximum value search problem, is one of the most widely used algorithm in the extraction of straight line field.Principle and implementation procedure see document for details: Paul Gonzales. Digital Image Processing. and the Electronic Industry Press. the 10th chapter: image segmentation.

A kind of Multi-lane Lines method for tracing based on group of Kalman filters may further comprise the steps:

Step 1: in the vehicle ' process, adopt Vehicular video collecting device collection vehicle road ahead image; And, carry out lane line and detect the road image data input computing machine that collects.

Use the Hough change detection to go out lane line, represent detected lane line, i.e. (ρ with lateral distance ρ and azimuth angle theta _i, θ _i) i=1,2,3L representes detected lane line on the road image.Use detailed algorithm that Hough conversion carrying out lane line detects referring to document: Liu sighs. based on the vehicle ' ancillary technique research of monocular vision. and the master thesis .2009.09.17. of University of Electronic Science and Technology

Step 2: set up the kalman filter models that lane line is followed the trail of.

Step 2-1: definition lane line state;

Because the motion of vehicle in two continuous frames video image acquisition process can be considered uniform motion, representes with state vector X (k) for the state of detected lane line in each frame road image:

X(k)＝[ρ(k)，θ(k)，u(k)，ω(k)] ^T

Wherein

expression vehicle is at state k; It is t radial velocity constantly;

representes vehicle at state k, i.e. t angular velocity constantly.

Step 2-2: the state equation of Kalman filter in the definition lane line tracing process;

Because system satisfies following formula by state X (k) to NextState X (k+1):

$\left\{\begin{array}{c}\mathrm{\ρ}(k+1)=\mathrm{\ρ}\left(k\right)+u\left(k\right)+w_1\left(k\right)\\ \mathrm{\θ}(k+1)=\mathrm{\θ}\left(k\right)+\mathrm{\ω}\left(k\right)+w_2\left(k\right)\\ u(k+1)=u\left(k\right)+w_3\left(k\right)\\ \mathrm{\ω}(k+1)=\mathrm{\ω}\left(k\right)+w_4\left(k\right)\end{array}\right.$

The system noise of set condition k is: W (k)=[w ₁(k), w ₂(k), w ₃(k), w ₄(k)] ^T, so the state equation of Kalman filter is expressed as:

X(k)＝AX(k-1)+W(k)

Wherein $A=\left[\begin{array}{cccc}1& 0& 1& 0\\ 0& 1& 0& 1\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right].$

Step 2-3: the observation equation of definition lane line tracing process;

Because observed reading is the result of actual condition value and noise effect, promptly observed reading can be represented as follows:

$Z\left(k\right)=\left[\begin{array}{c}{z}_1\left(k\right)\\ z_2\left(k\right)\end{array}\right]=\left[\begin{array}{c}\mathrm{\ρ}\left(k\right)+v_1\left(k\right)\\ \mathrm{\θ}\left(k\right)+v_2\left(k\right)\end{array}\right]$

Therefore obtaining observation equation is:

Z(k)＝HX(k)+V(k)

Wherein measure matrix $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right],$ V (k) is observation noise and V (k)=[v ₁(k) v ₂(k)] ^T

Step 2-4: definition Kalman filter iteration renewal equation:

X(k|k)＝X(k|k-1)+Kg(k)(Z(k)-HX(k|k-1))

Wherein:

Kg(k)＝P(k|k-1)H ^T/(HP(k|k-1)H ^T+R)

P(k|k)＝(I-Kg(k)H)P(k|k-1)

X (k|k) is the optimization estimated value of the k state of Kalman filter output, and X (k|k-1) is the result of the k state that utilizes the k-1 status predication and go out, and Kg (k) is Kalman filter gain under the state k, and Z (k) is the observed reading of state k, and H measures matrix.

Step 3: system initialization.

Step 3-1: set condition vector initial value;

Can know that by the Kalman filter ultimate principle as long as original state satisfies F [X (0)]=E [X (0|0)], resulting estimated value is no inclined to one side, so the initialization system original state is null vector, promptly

X(0)＝[0000] ^T

Step 3-2: initialization system noise initial value;

System noise W (k) is a zero-mean stochastic process vector, and system noise is a statistical independent under the different state constantly, and its correlation matrix satisfies following formula:

$E\left[W\left(n\right)W^H\left(k\right)\right]=Q\left(n\right)\mathrm{\&delta;}(n-k)=\left\{\begin{array}{cc}Q\left(n\right),& k=\mathrm{<figure-callout\; id="0"\; label="n"\; filenames="HDA0000072582010000021.png,HDA0000072582010000022.png"\; state="\{\{state\}\}">n</figure-callout>}\\ 0,& k\&NotEqual;n\end{array}\right.$

Suppose also be statistical independent between the different conditions noise constantly together, so matrix Q (k) is a diagonal matrix.Because system noise is because vehicle mobility can cause that in tracing process, influence not quite, for convenience of calculation, rule of thumb data are supposed w ₁(k), w ₂(k), w ₃(k), w ₄(k) independent same distribution, and satisfy

Therefore initialization system noise initial value is:

$Q\left(k\right)=E\left[W\left(k\right)W^H\left(k\right)\right]=\left[\begin{array}{cccc}0.05& 0& 0& 0\\ 0& 0.05& 0& 0\\ 0& 0& 0.05& 0\\ 0& 0& 0& 0.05\end{array}\right]$

Step 3-3: set the observation noise initial value;

Rule of thumb data can know that observation noise is bigger than the system noise influence, and v ₁(k), v ₂(k) satisfy independent same distribution, set

I=1,2, so set the observation noise initial value be:

$R\left(k\right)=E\left[V\left(k\right)V^T\left(k\right)\right]=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$

Step 3-4: set the observation noise autocorrelation value;

In order to follow the trail of rapid convergence, the observation noise autocorrelation value is taken as 100, and it is following to set initial value:

$P\left(0\right)=100\&times;\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]$

Step 4: adopt 20 groups of Kalman filter, all adopt in the step 3 initial value of setting, accomplish the interative computation of a Kalman filter, export the optimal estimation value (ρ of one group of lane line according to the Kalman filter iteration renewal equation of step 2-4 definition _i, θ _i) i=1,2,3, L, 20.

Step 5: single filter is enabled control.

Step 5-1: a detected lane line in the road image is defined as (ρ θ), obtains by following formula respectively that (ρ is θ) to (ρ _i, θ _i) i=1,2,3, L, 20 distance B _i

D _i＝|ρ-ρ _i|+|θ-θ _i|*W _image

W wherein _ImageThe pixel wide of expression road image.

Step 5-2: obtain minor increment D _Min=min (D _i), and what remember last iteration is D apart from minimum value _Min'.

Step 5-3: every group of corresponding enable counter of wave filter is designated as C _i, and satisfy following formula

Promptly when minor increment was less than or equal to the minor increment of last iteration, count value added 1, otherwise count value subtracts 1.

Step 5-4: lane line is followed the trail of; Setting effective sample input number is 25, promptly when continuous 25 width of cloth road images detect same lane line, thinks that the optimal estimation of Kalman filter is effective; Otherwise the sample number that praises is not enough, and does not follow the trail of.Therefore with C _iThrough valve [0,25], promptly handle C by following formula _i:

$C_i=\left\{\begin{array}{cc}25& C_i\&GreaterEqual;25\\ C_i& C_i<25\end{array}\right.$

Step 5-5: the E that enables that obtains each wave filter _iRepresent as follows:

$E_i=\left\{\begin{array}{cc}1& C_i=25\\ 0& C_i<25\end{array}\right.,i=\mathrm{1,2,3},L,20$

Step 6: with the optimal estimation value of Kalman filter output as testing result.

Enable bit E with bank of filters _i=1 optimal estimation value replaces detected value as testing result; Can detect inaccurate or omission thereby greatly reduce the lane line that causes because of reasons such as vehicle body shake, illumination variation, road surface breakages so that the testing result of lane line is more stable in the lane departure warning system.

Innovation part of the present invention is:

1, the tracking of lane line has been adopted the notion of bank of filters; Number according to the wave filter of setting; Can follow the trail of corresponding lane line number simultaneously, the single relatively tracking to the lane line area-of-interest has improved precision and antijamming capability that lane line detects greatly.

2, add the enable bit control of single filter in the bank of filters, do not had the pairing wave filter of detected lane line therefore to be in off position, promoted the arithmetic speed of tracing algorithm greatly.

Description of drawings

Fig. 1 is a schematic flow sheet of the present invention.

Fig. 2 is that bank of filters output valve and detected value matching times are counted (through the C behind [0,25] valve _i) figure, through (a), (b), C (c) _iValue can know that detect two lane lines in the road image this moment.

Fig. 3 is the interior ρ detected value of t=30s, optimal estimation value, a predicted value curve map in the test video.

Embodiment

The present invention adopts above step, utilizes the Video And Image Tool Box among the Matlab R2009 (a), has realized whole algorithm.Choose one section road image in certain high regime and do test, adopted group of Kalman filters to follow the trail of, lane line detects accuracy and improves 60%, and every width of cloth road image time of on average following the trail of is only expended 0.08s, has obtained very desirable testing result.

Need to prove: following parameters does not influence the generality of this patent.

1, the number of bank of filters median filter shows the bar number that can follow the trail of maximum lane lines simultaneously, and this algorithm is set at 20, but can increase and decrease according to actual conditions.

2, Kalman filter enables to judge, the effective count value of sample shows the basic demand of output optimal estimation value to sample continuous effective property, and this algorithm value is 25, but can increase and decrease according to actual needs.

Claims (1)

1. Multi-lane Lines method for tracing based on group of Kalman filters may further comprise the steps:

Step 1: in the vehicle ' process, adopt Vehicular video collecting device collection vehicle road ahead image; And, carry out lane line and detect the road image data input computing machine that collects;

Use the Hough change detection to go out lane line, represent detected lane line, i.e. (ρ with lateral distance ρ and azimuth angle theta _i, θ _i) i=1,2,3L representes detected lane line on the road image;

Step 2: set up the kalman filter models that lane line is followed the trail of;

Step 2-1: definition lane line state;

Because the motion of vehicle in two continuous frames video image acquisition process can be considered uniform motion, representes with state vector X (k) for the state of detected lane line in each frame road image:

X(k)＝[ρ(k)，θ(k)，u(k)，ω(k)] ^T

Wherein expression vehicle is at state k; It is t radial velocity constantly; representes vehicle at state k, i.e. t angular velocity constantly;

Step 2-2: the state equation of Kalman filter in the definition lane line tracing process;

Because system satisfies following formula by state X (k) to NextState X (k+1):

$\left\{\begin{array}{c}\mathrm{\&rho;}(k+1)=\mathrm{\&rho;}\left(k\right)+u\left(k\right)+w_1\left(k\right)\\ \mathrm{\&theta;}(k+1)=\mathrm{\&theta;}\left(k\right)+\mathrm{\&omega;}\left(k\right)+w_2\left(k\right)\\ u(k+1)=u\left(k\right)+w_3\left(k\right)\\ \mathrm{\&omega;}(k+1)=\mathrm{\&omega;}\left(k\right)+w_4\left(k\right)\end{array}\right.$

The system noise of set condition k is: W (k)=[w ₁(k), w ₂(k), w ₃(k), w ₄(k)] ^T, so the state equation of Kalman filter is expressed as:

X(k)＝AX(k-1)+W(k)

Wherein $A=\left[\begin{array}{cccc}1& 0& 1& 0\\ 0& 1& 0& 1\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right].$

Step 2-3: the observation equation of definition lane line tracing process;

Because observed reading is the result of actual condition value and noise effect, promptly observed reading can be represented as follows:

$Z\left(k\right)=\left[\begin{array}{c}{z}_1\left(k\right)\\ z_2\left(k\right)\end{array}\right]=\left[\begin{array}{c}\mathrm{\&rho;}\left(k\right)+v_1\left(k\right)\\ \mathrm{\&theta;}\left(k\right)+v_2\left(k\right)\end{array}\right]$

Therefore obtaining observation equation is:

Z(k)＝HX(k)+V(k)

Wherein measure matrix $H=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\end{array}\right],$ V (k) is observation noise and V (k)=[v ₁(k) v ₂(k)] ^T

Step 2-4: definition Kalman filter iteration renewal equation:

X(k|k)＝X(k|k-1)+Kg(k)(Z(k)-HX(k|k-1))

Wherein:

Kg(k)＝P(k|k-1)H ^T/(HP(k|k-1)H ^T+R)

P(k|k)＝(I-Kg(k)H)P(k|k-1)

X (k|k) is the optimization estimated value of the k state of Kalman filter output, and X (k|k-1) is the result of the k state that utilizes the k-1 status predication and go out, and Kg (k) is Kalman filter gain under the state k, and Z (k) is the observed reading of state k, and H measures matrix;

Step 3: system initialization;

Step 3-1: set condition vector initial value;

Can know that by the Kalman filter ultimate principle as long as original state satisfies E [X (0)]=E [X (0|0)], resulting estimated value is no inclined to one side, so the initialization system original state is null vector, promptly

X(0)＝[0000] ^T

Step 3-2: initialization system noise initial value;

System noise W (k) is a zero-mean stochastic process vector, and system noise is a statistical independent under the different state constantly, and its correlation matrix satisfies following formula:

$E\left[W\left(n\right)W^H\left(k\right)\right]=Q\left(n\right)\mathrm{\&delta;}(n-k)=\left\{\begin{array}{cc}Q\left(n\right),& k=n\\ 0,& k\&NotEqual;n\end{array}\right.$

Suppose also be statistical independent between the different conditions noise constantly together, so matrix Q (k) is a diagonal matrix; Because system noise is because vehicle mobility can cause that in tracing process, influence not quite, for convenience of calculation, rule of thumb data are supposed w ₁(k), w ₂(k), w ₃(k), w ₄(k) independent same distribution, and satisfy

I=1,2,3,4, so initialization system noise initial value is:

$Q\left(k\right)=E\left[W\left(k\right)W^H\left(k\right)\right]=\left[\begin{array}{cccc}0.05& 0& 0& 0\\ 0& 0.05& 0& 0\\ 0& 0& 0.05& 0\\ 0& 0& 0& 0.05\end{array}\right]$

Step 3-3: set the observation noise initial value;

Rule of thumb data can know that observation noise is bigger than the system noise influence, and v ₁(k), v ₂(k) satisfy independent same distribution, set

I=1,2, so set the observation noise initial value be:

$R\left(k\right)=E\left[V\left(k\right)V^T\left(k\right)\right]=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$

Step 3-4: set the observation noise autocorrelation value;

In order to follow the trail of rapid convergence, the observation noise autocorrelation value is taken as 100, and it is following to set initial value:

$P\left(0\right)=100\&times;\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]$

Step 4: adopt 20 groups of Kalman filter, all adopt in the step 3 initial value of setting, accomplish the interative computation of a Kalman filter, export the optimal estimation value (ρ of one group of lane line according to the Kalman filter iteration renewal equation of step 2-4 definition _i, θ _i) i=1,2,3, L, 20;

Step 5: single filter is enabled control;

Step 5-1: a detected lane line in the road image is defined as (ρ θ), obtains by following formula respectively that (ρ is θ) to (ρ _i, θ _i) i=1,2,3, L, 20 distance B _i

D _i＝|ρ-ρ _i|+|θ-θ _i|*W _image

W wherein _ImaeThe pixel wide of expression road image;

Step 5-2: obtain minor increment D _Min=min (D _i), and remember last iteration apart from minimum value do _Dmin';

Step 5-3: every group of corresponding enable counter of wave filter is designated as C _i, and satisfy following formula

Promptly when minor increment was less than or equal to the minor increment of last iteration, count value added 1, otherwise count value subtracts 1;

Step 5-4: lane line is followed the trail of; Setting effective sample input number is 25, promptly when continuous 25 width of cloth road images detect same lane line, thinks that the optimal estimation of Kalman filter is effective; Otherwise the sample number that praises is not enough, and does not follow the trail of.Therefore with C _iThrough valve [0,25], promptly handle C by following formula _i:

$C_i=\left\{\begin{array}{cc}25& C_i\&GreaterEqual;25\\ C_i& C_i<25\end{array}\right.$

Step 5-5: the E that enables that obtains each wave filter _iRepresent as follows:

$E_i=\left\{\begin{array}{cc}1& C_i=25\\ 0& C_i<25\end{array}\right.,i=\mathrm{1,2,3},L,20$

Step 6: with the optimal estimation value of Kalman filter output as testing result.

Images