CN103978978A - Inversion projection transformation based lane keeping method - Google Patents

Abstract

The invention relates to an inversion projection transformation based lane keeping method. The inversion projection transformation based lane keeping method mainly comprises the following steps of performing inversion projection transformation on shot perspective drawings according to intrinsic parameters of a camera and converting into vertical views; detecting lane lines and extracting lane central lines based on the vertical views and computing and setting distance between corresponding endpoints of area central lines; judging whether an automobile has lane departure or not due to compare between the distance and threshold values. The inversion projection transformation based lane keeping method has the advantages of overcoming a traditional lane keeping system disadvantage that the lane lines are not benefited to judge due to direct extraction of the vertical views, accurately judging whether the automobile has lane departure or not and achieving lane keeping.

Description

Based on the lane keeping method of contrary projective transformation

Technical field

The present invention relates to traffic safety ancillary technique, relate in particular to a kind of lane keeping method based on contrary projective transformation.

Background technology

Along with social development, automobile has become the indispensable vehicle of modern society, and it has not only promoted the progress of human culture, also in the life that changes at leisure people.The universal urban roads safety that makes of automobile becomes a focus attracting people's attention.Recent years, automobile enters the paces of family to be accelerated gradually, and current Chinese motor vehicles for civilian use recoverable amount has exceeded 100,000,000, exceedes Germany, is only second to the U.S..Meanwhile, Chinese population is numerous, and each big city population density is very high, Che Yuche, and the contradiction of people and car is also sharp-pointed especially, Frequent Accidents on road, the quantity that traffic accident occurs is much higher than developed country, also higher than other developing countries.Learn according to Public Security Department and State Statistics Bureau's statistics, China is annual because traffic accident direct economic loss reaches billions of units, but also is ascendant trend year by year.Along with the development of computer technology, image processing techniques and computer vision technique are so more and more that to be applied to vehicle electric field.Intelligent driving also becomes a current popular topic.Lane Keeping System can allow automobile produce while keeping travelling when tired not run-off-road at chaufeur, or automobile travels in master control remains on lane mark, improves the comfort level of driving.

It is low that camera has cost, and high integration and be easy to the advantages such as maintenance is widely used, especially in Lane Keeping System in automobile active safety system.Show according to an investigation of doing in Europe, 39% unexpected traffic accident is at run-off-road unintentionally and produce.Along with the development of intelligent transportation, travelling of automobile is more and more intelligent, and safety when this can not only increase running car also can improve the comfort level of navigating mate.When driver's operation is unskilled or when fatigue driving, automobile generation deviation can cause serious traffic accident.And Lane Keeping System can not only keep automobile normally to travel in lane mark, can also allow when needed automobile independently travel, improve the traveling comfort of chaufeur.

Summary of the invention

The technical problem to be solved in the present invention is for defect of the prior art, and a kind of lane keeping method based on contrary projective transformation is provided.

The technical solution adopted for the present invention to solve the technical problems is: a kind of based on intrinsic parameters of the camera the lane keeping method against projective transformation, comprise the following steps:

(1) to being arranged on camera calibration, and calculate camera inner parameter; Camera inner parameter comprises focal distance f c, principal point coordinate cc, inclination factor alpha_c, distortion factor kc;

(2) obtain the road ahead information that calibrated camera is taken, coloured image is converted into gray level image, and gray level image is carried out to denoising;

(3) according to contrary projective transformation principle, utilize camera inner parameter, the area-of-interest A of photographic images is reduced to birds-eye view; Described area-of-interest A is the rectangular area of vehicle front road 10m × 5m.

(4) in birds-eye view, utilize hough change detection identification lane mark, and calculate the end points coordinate of track line of centers;

(5) in birds-eye view, calculate lane mark line of centers and area-of-interest A line of centers end-point distances B1 and the B2 in area-of-interest A, and judge threshold value C B1 and B2 and that whether exceed setting;

(6) if B1 and B2 and exceed the threshold value C of setting, handle to turn to actuating unit to turn to adjust and make B1 with B2's and lower than the threshold value of setting;

(7) if distance does not exceed the threshold value C of setting, the detection that enters next frame image.

Press such scheme, photographic images carried out to the method that contrary projective transformation is converted to birds-eye view and comprise:

3.1) projective transformation:

Be located at the coordinate of spatial point P in the camera reference system of putting centered by camera

${\mathrm{XX}}_c=\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right],$

Make x=X _c/ Z _c; Y=Y _c/ Z _c; r ²=x ²+ y ²;

Projective transformation is in the following ways:

$x_d=\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\end{array}\right]=(1+\mathrm{kc}\left(1\right)r^2+\mathrm{kc}\left(2\right)r^4+\mathrm{kc}\left(5\right)r^6)x_n+\mathrm{dx};$

Wherein, dx is tangential distortion vector:

$\mathrm{dx}=\left[\begin{array}{c}2\mathrm{kc}\left(3\right)\mathrm{xy}+\mathrm{kc}\left(4\right)(r^2+2x^2)\\ \mathrm{kc}\left(3\right)(r^2+2y^2)+2\mathrm{kc}\left(4\right)\mathrm{xy}\end{array}\right];$

3.2) be pixel coordinate by the coordinate transformation after projection:

If pixel coordinate X_pixel is

$X\_\mathrm{pixel}=\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right]$

Pixel coordinate X_pixel and normalized coordinate vector X _dconversion formula as follows:

$\left[\begin{array}{c}{y}_p\\ y_p\\ 1\end{array}\right]=\mathrm{KK}\begin{array}{c}\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\\ 1\end{array}\right]\end{array}$

Wherein KK is that world coordinates is tied to camera Conversion Matrix of Coordinate

$\mathrm{KK}=\left[\begin{array}{ccc}\mathrm{fc}\left(1\right)& \mathrm{alph}{a}_c*\mathrm{fc}\left(1\right)& \mathrm{cc}\left(1\right)\\ 0& \mathrm{fc}\left(2\right)& \mathrm{cc}\left(2\right)\\ 0& 0& 1\end{array}\right].$

The beneficial effect that the present invention produces is: the inventive method has overcome in traditional vehicle lane keeping system directly extracts the shortcoming that lane mark in transparent view is unfavorable for judgement, can judge more accurately whether automobile deviation occurs, and then realize lane keeping.

Brief description of the drawings

Below in conjunction with drawings and Examples, the invention will be further described, in accompanying drawing:

Fig. 1 is the method flow diagram of the embodiment of the present invention;

Fig. 2 is transparent view and the area-of-interest of pick up camera actual photographed in the embodiment of the present invention;

Fig. 3 is the lane information figure obtaining through contrary projective transformation in the embodiment of the present invention;

Fig. 4 is the lane mark information of utilizing hough conversion to extract in the embodiment of the present invention;

Fig. 5 is the information of track line of centers and region of interest centers line in the embodiment of the present invention;

Fig. 6 is that in the embodiment of the present invention, schematic diagram is not departed from track;

Fig. 7 is deviation schematic diagram in the embodiment of the present invention.

Detailed description of the invention

In order to make object of the present invention, technical scheme and advantage clearer, below in conjunction with embodiment, the present invention is further elaborated.Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not intended to limit the present invention.

As shown in Figure 1, a kind of lane keeping method based on contrary projective transformation, comprises the following steps:

1) utilize Camera Calibration Toolbox for Matlab software to carry out camera calibration, and calculate camera inner parameter, inner parameter comprises focal distance f c (2x1 vector), principal point coordinate cc (2x1 vector), inclination factor alpha_c (scalar), distortion factor kc (5x1 vector).

2) utilize calibrated camera to take road ahead information, this information is transparent view, as shown in Figure 2; Coloured image is converted into gray level image;

3) gray level image is processed, kept basic style characteristic, and remove noise;

According to contrary projective transformation principle, utilize camera inner parameter to reduce to the area-of-interest A of photographic images, wherein area-of-interest A is the rectangular area of vehicle dead ahead road 10m × 5m; Need in world coordinate system, measure the actual coordinate # of area-of-interest point P

${\mathrm{XX}}_c=\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]$

If x _nfor normalized image projection:

$x_n=\left[\begin{array}{c}{X}_c/Z_c\\ Y_c/Z_c\end{array}\right]=\left[\begin{array}{c}x\\ y\end{array}\right]$

Make r ²=x ²+ y ².

Consider cam lens distortion, new normalized point coordinate x _dbe defined as follows:

$x_d=\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\end{array}\right]=(1+\mathrm{kc}\left(1\right)r^2+\mathrm{kc}\left(2\right)r^4+\mathrm{kc}\left(5\right)r^6)x_n+\mathrm{dx}$

Wherein dx is tangential distortion vector:

$\mathrm{dx}=\left[\begin{array}{c}2\mathrm{kc}\left(3\right)\mathrm{xy}+\mathrm{kc}\left(4\right)(r^2+2x^2)\\ \mathrm{kc}\left(3\right)(r^2+2y^2)+2\mathrm{kc}\left(4\right)\mathrm{xy}\end{array}\right]$

If pixel coordinate X_pixel is

$X\_\mathrm{pixel}=\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right]$

Pixel coordinate X_pixel and normalized coordinate vector X _drelation as follows:

$\left[\begin{array}{c}{y}_p\\ y_p\\ 1\end{array}\right]=\mathrm{KK}\begin{array}{c}\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\\ 1\end{array}\right]\end{array}$

Wherein KK is that world coordinates is tied to camera Conversion Matrix of Coordinate

$\mathrm{KK}=\left[\begin{array}{ccc}\mathrm{fc}\left(1\right)& \mathrm{alph}{a}_c*\mathrm{fc}\left(1\right)& \mathrm{cc}\left(1\right)\\ 0& \mathrm{fc}\left(2\right)& \mathrm{cc}\left(2\right)\\ 0& 0& 1\end{array}\right]$

Utilize above formula the point P in world coordinate system can be reverted to the corresponding point (x in contrary projective transformation image _p, y _p), go back original image as shown in Figure 3.

4) as shown in Figure 4, utilize hough transfer pair birds-eye view to carry out detection and the extraction of lane mark, calculate the end points coordinate of track line of centers;

5) area-of-interest is parallel to the line of centers end points coordinate (as shown in Figure 5) of travel direction, then calculates distance B1 and B2 between track line of centers and the corresponding end points of region of interest centers line;

5.1) utilize hough change detection against the lane mark in image after projective transformation, and the end points coordinate of lane mark is obtained, and calculate the end points coordinate of track line of centers by the method for calculating mean value;

5.2) calculate the line of centers end points coordinate that area-of-interest A is parallel to travel direction.

5.3) calculate distance B1 and the B2 between track line of centers and the corresponding end points of region of interest centers line.

5.4) minimum value of B1+B2 while showing by experiment that track is actual and depart from, as threshold value C;

5.5) by judging that actual pixels is apart from the relation between B1+B2 and threshold value C, judge whether automobile has the risk departing from, and then make subsequent operation.

If B1+B2<C, there is not deviation in automobile, carries out the processing of next frame image, as shown in Figure 6; If B1+B2>C, judges automobile generation deviation, need operation to turn to actuating unit to carry out steering operation, as shown in Figure 7.

6) if automobile has the risk departing from, control and turn to actuating unit to carry out the control of automobile steering, continue the relation between monitoring B1+B2 and threshold value C;

If the risk that automobile does not depart from, or turning to distance B1+B2 after actuating unit running to be less than threshold value C, enter the processing of next frame image.

Should be understood that, for those of ordinary skills, can be improved according to the above description or convert, and all these improvement and conversion all should belong to the protection domain of claims of the present invention.

Claims (2)

1. the lane keeping method against projective transformation based on intrinsic parameters of the camera, is characterized in that, comprises the following steps:

(1) to being arranged on camera calibration, and calculate camera inner parameter; Camera inner parameter comprises focal distance f c, principal point coordinate cc, inclination factor alpha_c, distortion factor kc;

(2) obtain the road ahead information that calibrated camera is taken, coloured image is converted into gray level image, and gray level image is carried out to denoising;

(3) according to contrary projective transformation principle, utilize camera inner parameter, the area-of-interest A of photographic images is reduced to birds-eye view; Described area-of-interest A is the rectangular area of vehicle dead ahead road 10m × 5m;

(4) in birds-eye view, utilize hough change detection identification lane mark, and calculate the end points coordinate of track line of centers;

(5) in birds-eye view, calculate lane mark line of centers and area-of-interest A line of centers end-point distances B1 and the B2 in area-of-interest A, and judge threshold value C B1 and B2 and that whether exceed setting;

(6) if B1 and B2 and exceed the threshold value C of setting, handle to turn to actuating unit to turn to adjust and make B1 with B2's and lower than the threshold value of setting

(7) if distance does not exceed the threshold value C of setting, the detection that enters next frame image.

2. lane keeping method according to claim 1, is characterized in that, photographic images is carried out to the method that contrary projective transformation is converted to birds-eye view and comprise:

3.1) projective transformation:

Be located at the coordinate of spatial point P in the camera reference system of putting centered by camera

${\mathrm{XX}}_c=\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right],$

Make x=X _c/ Z _c; Y=Y _c/ Z _c; r ²=x ²+ y ²;

Projective transformation is in the following ways:

$x_d=\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\end{array}\right]=(1+\mathrm{kc}\left(1\right)r^2+\mathrm{kc}\left(2\right)r^4+\mathrm{kc}\left(5\right)r^6)x_n+\mathrm{dx};$

Wherein, dx is tangential distortion vector:

$\mathrm{dx}=\left[\begin{array}{c}2\mathrm{kc}\left(3\right)\mathrm{xy}+\mathrm{kc}\left(4\right)(r^2+2x^2)\\ \mathrm{kc}\left(3\right)(r^2+2y^2)+2\mathrm{kc}\left(4\right)\mathrm{xy}\end{array}\right];$

3.2) be pixel coordinate by the coordinate transformation after projection:

If pixel coordinate X_pixel is

$X\_\mathrm{pixel}=\left[\begin{array}{c}{x}_p\\ y_p\end{array}\right];$

Pixel coordinate X_pixel and normalized coordinate vector X _dconversion formula as follows:

$\left[\begin{array}{c}{y}_p\\ y_p\\ 1\end{array}\right]=\mathrm{KK}\begin{array}{c}\left[\begin{array}{c}{x}_d\left(1\right)\\ x_d\left(2\right)\\ 1\end{array}\right];\end{array}$

Wherein KK is that world coordinates is tied to camera Conversion Matrix of Coordinate

$\mathrm{KK}=\left[\begin{array}{ccc}\mathrm{fc}\left(1\right)& \mathrm{alpha}\_c*\mathrm{fc}\left(1\right)& \mathrm{cc}\left(1\right)\\ 0& \mathrm{fc}\left(2\right)& \mathrm{cc}\left(2\right)\\ 0& 0& 1\end{array}\right].$