CN107705577B - Real-time detection method and system for calibrating illegal lane change of vehicle based on lane line - Google Patents

Abstract

The invention relates to a real-time detection method for calibrating violation lane change of a vehicle based on a lane line, which specifically comprises the following steps: step 1) calibrating a lane line L, and calculating and acquiring a detection rectangular area D and an image I; step 2) detecting a vehicle set A ═ A of the image I in the detection rectangular region D in the step 1) based on the deep convolutional neural network₁,A₂,...,A_i}; step 3) screening out a vehicle set B which is intersected with the lane line L according to the vehicle set A obtained in the step 2) { B }₁,B₂,...,B_iAnd (c) the step of (c) in which,

matching with the tracking vehicle list TL again, and updating the tracking vehicle list TL ═ T₁,T₂,...,T_j}; step 4) judging the tracked vehicles T in the updated tracked vehicle list TL_jWhether the vehicle is a violation lane-changing vehicle or not; if tracking vehicle T_jMarking out the tracking vehicle T for the illegal lane changing vehicle_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

Description

Real-time detection method and system for calibrating illegal lane change of vehicle based on lane line

Technical Field

The invention belongs to the technical field of intelligent traffic systems and image recognition, and particularly relates to a real-time detection method and system for calibrating vehicle lane change violations based on lane lines.

Background

With the progress and development of society, urban motor vehicles are increasing day by day, the number of motor vehicles is rising continuously, so that traffic accidents are more serious, and the economic loss and the number of death people caused by road traffic accidents are rising continuously in recent years. The control of traffic accidents is becoming a problem more and more important to traffic management departments, and the leading cause of traffic accidents is car illegal activities. As a common traffic violation, lane change violations not only cause traffic congestion, but even possibly cause serious traffic accidents, in order to reduce the occurrence rate of traffic accidents, traffic management departments continuously promote an intelligent traffic management system, and a monitoring video-based lane change violating detection technology in the intelligent traffic management system is more critical, so that the research of monitoring video-based vehicle lane change violating detection is necessary.

The existing method for detecting the violation lane change of the vehicle mainly comprises two types: one is based on spatial distance, such as laser detection method, infrared detection method and ultrasonic detection method, and the methods have the problems of expensive equipment, small space coverage area, mutual interference among equipment, incapability of processing shielding conditions and the like; the other type is based on computer vision technology, and the method has the advantages of simple installation and maintenance, high visibility, high detection accuracy and the like.

At present, a method for detecting vehicles changing lanes violating regulations based on a computer vision technology extracts a moving target set, finds out a moving vehicle set and position information, and finally judges whether the discrete degree of a moving track of each vehicle exceeds a set threshold value so as to judge whether the vehicles change lanes violating regulations. The existing method has good detection effect when fewer vehicles are on the road surface and the vehicles move faster; however, it cannot cope with a lane change violation when the vehicle moves slowly or the number of road vehicles is large. In addition, the existing method is easily influenced by factors such as illumination, shading, shadow, video jitter and the like; when the number of vehicles is large, the calculation amount of the tracking mode of each vehicle is too large, and the real-time requirement is difficult to meet. Therefore, aiming at detecting vehicles changing the lanes illegally, a real-time detection method for detecting the lanes illegally changed, which has the advantages of wide application range, high accuracy and high detection speed and can meet the real-time requirement, is urgently needed.

Disclosure of Invention

The invention aims to solve the defects of the existing real-time detection method for the illegal lane change of the vehicle, and provides the real-time detection method for the illegal lane change of the vehicle based on lane line calibration. In addition, the method can realize the rapid and accurate identification of the violation lane-changing vehicle in the video.

In order to achieve the aim, the invention provides a lane change violation real-time detection method based on lane line calibration, which specifically comprises the following steps:

step 1) calibrating a lane line L, and calculating and acquiring a detection rectangular area D and an image I;

step 2) detecting a vehicle set A ═ A of the image I in the detection rectangular region D in the step 1) based on the deep convolutional neural network₁,A₂,...,A_i}；

Step 3) screening out a vehicle set B which is intersected with the lane line L according to the vehicle set A obtained in the step 2) { B }₁,B₂,...,B_iAnd (c) the step of (c) in which,

matching with the tracking vehicle list TL again, and updating the tracking vehicle list TL ═ T₁,T₂,...,T_j}；

Step 4) judging the tracked vehicles T in the updated tracked vehicle list TL_jWhether the vehicle is a violation lane-changing vehicle or not; if tracking vehicle T_jMarking out the tracking vehicle T for the illegal lane changing vehicle_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

The step 1) specifically comprises the following steps:

step 1-1) calibrating lane line L ═ { L ═₁,L₂,...,L_j}, each lane line L_jThe positions of (a) include: starting point pb_j＝{x_bj,y_bjAnd end point pe_j＝{x_ej,y_ejIn which x_bj,y_bjAbscissa and ordinate, x, of origin, respectively_ej,y_ejRespectively as the horizontal and vertical coordinates of the end point;

step 1-2) calculates a detection area rectangle D ═ x, y, w, h } according to the lane line position of step 1-1), wherein,

x＝min(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)

y＝min(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)

w＝max(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)-x

h＝max(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)-y

the step 2) specifically comprises the following steps:

normalizing the detection area D in the image I to 576 multiplied by 576 as an input image according to the detection area D and the image I in the step 1), and detecting a vehicle set A ═ { A } in the image I based on a deep convolutional neural network₁,A₂,...,A_i}，A_iPosition of (D) is denoted as PA_i＝{x_i,y_i,w_i,h_iThe detection effect is shown in FIG. 2 (b);

wherein, this convolutional neural network includes: 7 convolutional layers, 4 downsampling layers, 2 full-link layers and one output layer; scanning boundaries in the convolutional layers are automatically filled with 0, and neurons are activated by utilizing a Leaky-ReLu function; and maximum pooling is adopted in the down-sampling layers.

The convolution kernel size of convolution layer C1 is 9 × 9, 32 convolution kernels, the step size is 2, and the generated feature map size is 288 × 288; the window size of the downsampling layer S1 is 4 multiplied by 4, the step size is 4, and the size of the generated feature map is 72 multiplied by 72; the convolution layer C2 has convolution kernel size of 1 × 1, 4 convolution kernels and step size of 1, and the generated feature map size is 72 × 72; the convolution layer C3 has convolution kernel size of 3 × 3, 8 convolution kernels and step length of 1, and the generated feature map size is 72 × 72; the window size of the downsampling layer S2 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 36 multiplied by 36; the convolution layer C4 has convolution kernel size of 1 × 1, 8 convolution kernels and step size of 1, and the generated feature map size is 36 × 36; the convolution layer C5 has convolution kernel size of 3 × 3, 16 convolution kernels and step size of 1, and the generated feature map size is 36 × 36; the window size of the downsampling layer S3 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 18 multiplied by 18; the convolution layer C6 has convolution kernel size of 3 × 3, 32 convolution kernels and step size of 1, and the generated feature map size is 18 × 18; the window size of the downsampling layer S4 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 9 multiplied by 9; the convolution layer C7 has convolution kernel size of 1 × 1, 64 convolution kernels and step size of 1, and the generated feature map size is 9 × 9; the full-junction layer F1 is composed of 256 neurons, and the neurons are activated by using a Relu function; the full connection layer F2 is composed of 4096 neurons, and the neurons are activated by using a Leaky-ReLu function; the output layer consists of 891 neurons, which are activated using the Relu function.

The step 3) specifically comprises the following steps:

step 3-1) the set of vehicles a ═ { a) obtained according to step 2)₁,A₂,...,A_iScreening out a vehicle set B which is intersected with the lane line L from the lane line L₁,B₂,...,B_iAnd (c) the step of (c) in which,

step 3-2) traversing the tracking vehicle list TL ═ T according to the vehicle set B obtained in the step 3-1)₁,T₂,...,T_j}, selecting the contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; calculating the Hog characteristic Hog_jAnd hog_kThen calculate the remaining chord distances Cos_jk(ii) a Reselecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd a corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching each other and proceeding to step 3-3); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatching vehicles and entering the step 3-4);

step 3-3) vehicle C obtained according to step 3-2)_mUpdating tracked vehicle T_jIs the current vehicle position, i.e. TP_j＝CP_m(ii) a Updating tracked vehicles to current vehicle pictures, i.e. TM_j＝CM_m(ii) a Number of consecutive lost times TT_j0, the horizontal directed distance TD of the tracking point from the lane line L_jUpdated to the horizontal directed distance, TD, of the current tracking point from the lane line L_j＝CD_m(ii) a At this time C_mMatched, current vehicle is matched flag CF_i＝true；

Step 3-4) updating the tracked vehicle T_jNumber of continuous lost times TT_j', wherein TT_j’＝TT_j+1, if TT_j' if the number of consecutive lost tracking is more than 3, the vehicle T is deleted from the tracked vehicle list_j；

Step 3-5) traversing the vehicle set B, and enabling the unmatched vehicles B_u＝{BP_u,BM_u,BF_u,BQ_u,BD_uAs a new tracked vehicle T_u＝{TP_u,TM_u,TT_u,TOD_u,TD_uAdds to the tracked vehicle list TL, updates the vehicle tracking list TL, wherein,

TP_u＝BP_u

TM_u＝BM_u

TT_u＝0

TOD_u＝BD_u

TD_u＝BD_u。

the step 3-1) specifically comprises the following steps:

step 3-1-1) obtaining the lane line L and the vehicle A according to the step 1) and the step 2)_iPosition rectangle PA_iCalculating the lane line L_jAnd a vehicle A_iPosition rectangle PA_iIntersection of four sides and lane line L_jThe intersections with the left, right, upper and lower edges of the rectangle are pl_ij＝{x_lij,y_lij}、pr_ij＝{x_rij,y_rij}、pu_ij＝{x_uij,y_uij}、pd_ij＝{x_dij,y_dij}；

Wherein, { x_lij,y_lijIs the horizontal and vertical coordinates of the left edge intersection point, { x }_rij,y_rijAre right side edge intersection points respectivelyOrdinate, { x_uij,y_uijThe horizontal and vertical coordinates of the intersection point of the upper side edge are respectively, and the horizontal and vertical coordinates of the intersection point of the lower side edge are respectively.

Step 3-1-2) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the left side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+w_i

y_i+h_i/2＜y_lij＜y_i+h_i

step 3-1-3) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the right side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i＜x_uij＜x_i+3w_i/4

y_i+h_i/2＜y_rij＜y_i+h_i

step 3-1-4) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is positioned at the upper side and the lower side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+3w_i/4

x_i+w_i/4＜x_dij＜x_i+3w_i/4

the step 3-2) specifically comprises the following steps:

step 3-2-1) initializing vehicle information according to the vehicle set B obtained in step 3-1), wherein the vehicle information comprises the following steps: vehicle position BP_i＝{x_i,y_i,w_i,h_iGet the vehicle picture BM_iWhether the current vehicle is matched or not is marked BF_iFalse, tracking point BQ_i＝(x_i+w_i/2,y_i+4h_i/5) and tracking point BQ_iWith each lane line L_jIs a horizontal directed distance Bd_ijI.e. vehicles B_i＝{BP_i,BM_i,BF_i,BQ_i,BD_iIn which, BD_i＝{Bd_i1,Bd_i2,...,Bd_ij}；

Step 3-2-2) traversing and tracking vehicle list TL ═ T₁,T₂,...,T_j}，T_j＝{TP_j,TM_j,TT_j,TOD_j,TD_j}，T_jE.g. TL, wherein TP_jIndicating the position of the vehicle, TM_jShows a picture of a vehicle, TT_jIndicating number of consecutive tracking losses, TOD_jIndicating the horizontal directional distance, TD, of the starting tracking point from the lane line L_jRepresenting the horizontal directed distance between the current tracking point and the lane line L; TOD_j＝{TOd_j1,TOd_j2,...,TOd_jl}，TD_j＝{Td_j1,Td_j2,...,Td_jl}; computing tracked vehicle T_jVehicle position TP_jWith each unmatched vehicle position BP in the vehicle set B_iCoincidence degree U of_ij：

U_ij＝S_cij/S_zij

Wherein S is_zijIs BP_iAnd TP_jUnion of rectangular areas, S_cijIs BP_iAnd TP_jIntersection of rectangular areas;

selecting a contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; if C is empty, the vehicle T does not exist and is tracked in the vehicle set B_jMatching vehicles and entering the step 3-4); if C is not empty, entering step 3-2-3);

step 3-2-3) the set of vehicles C ═ { C) obtained according to step 3-2-2)₁,C₂,...,C_mWill track vehicle T_jVehicle picture TM_jAnd a picture CM of each vehicle in the vehicle set C_kNormalizing to 64 × 64 size, graying, and calculating the Hog characteristic Hog_jAnd hog_k(ii) a Wherein, the meterIn the calculation, the size of the window is 64 multiplied by 64, the size of the block is 16 multiplied by 16, the size of the block sliding increment is 8 multiplied by 8, the size of the cell is 8 multiplied by 8, and the number of gradient histograms in each cell unit is 9;

step 3-2-4) Hog-characteristic Hog obtained according to step 3-2-3)_jAnd hog_kCalculating the remaining chord distances Cos_jk；

Step 3-2-5) cosine distance Cos obtained according to step 3-2-4)_jkSelecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching each other and proceeding to step 3-3); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatched vehicles and proceed to step 3-4).

The step 4) specifically comprises the following steps:

step 4-1) traversing the updated tracked vehicle list TL obtained according to the step 3-5), traversing the updated tracked vehicle list TL, and calculating T_jStarting tracking point of (1) and lane line L_jHorizontal directed distance TOd'_jjAnd the current tracking point and the lane line L_jIs horizontal directed distance Td'_jjOf TOd'_jj*Td′_jjLess than 0, it means that the vehicles are successively located on the lane line L_jTwo sides, T_jA lane-changing vehicle violating regulations;

step 4-2) obtaining the violation lane-changing vehicle T according to the step 4-1)_jThen, mark out the tracking vehicle T_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

A real-time detection system for calibrating vehicle lane change violations based on lane lines is an intelligent traffic management system and comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that the steps of the detection method are realized when the processor executes the program.

The invention has the advantages that:

the method is based on the deep convolutional neural network to detect the vehicle set in the image, and compared with a method for extracting the vehicle set by utilizing the motion characteristics, the method is higher in accuracy and wider in application range. In addition, the method screens out the vehicles intersected with the lane line and then carries out matching judgment, does not need to record and judge the motion track of each vehicle, has high detection speed and can meet the requirement of real-time identification.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a flow chart of a method for detecting lane change violation of a vehicle in real time based on lane line calibration according to the present invention;

fig. 2(a) is a schematic diagram of the effect of setting a lane line in step 1) of the method for detecting the violation lane change of a vehicle based on lane line calibration in the embodiment of the invention;

fig. 2(b) is a schematic diagram of the effect of detecting a vehicle set in step 2) of the method for detecting the violation lane change of a vehicle based on lane line calibration in the embodiment of the invention;

fig. 2(c) is a schematic diagram of the effect of the vehicle set which is screened out and intersects with the lane line in step 3) of the lane violation lane change real-time detection method based on lane line calibration in the embodiment of the present invention;

fig. 2(d) is a schematic diagram of the effect of marking and tracking a violation lane-changing vehicle set in a vehicle list by step 5) of the method for detecting the violation lane-changing of the vehicle based on lane line calibration in the embodiment of the invention;

fig. 2(e) is a schematic diagram of the effect of the violation vehicle driving away state in step 5) of the method for detecting the violation lane change in real time based on lane line calibration in the embodiment of the invention;

fig. 3 is a network configuration diagram of a vehicle detection model in the embodiment of the present invention.

Detailed Description

As shown in fig. 1, the invention provides a lane change violation real-time detection method based on lane line calibration, which specifically comprises the following steps:

step 1) calibrating a lane line L, and calculating and acquiring a detection rectangular area D and an image I;

step 2) detecting a vehicle set A ═ A of the image I in the detection rectangular region D in the step 1) based on the deep convolutional neural network₁,A₂,...,A_i}；

Step 3) screening out a vehicle set B which is intersected with the lane line L according to the vehicle set A obtained in the step 2) { B }₁,B₂,...,B_iAnd (c) the step of (c) in which,

matching with the tracking vehicle list TL again, and updating the tracking vehicle list TL ═ T₁,T₂,...,T_j}；

Step 4) judging the tracked vehicles T in the updated tracked vehicle list TL_jWhether the vehicle is a violation lane-changing vehicle or not; if tracking vehicle T_jMarking out the tracking vehicle T for the illegal lane changing vehicle_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

The step 1) specifically comprises the following steps:

step 1-1) calibrating lane line L ═ { L ═₁,L₂,...,L_j}, each lane line L_jThe positions of (a) include: starting point pb_j＝{x_bj,y_bjAnd end point pe_j＝{x_ej,y_ejAs shown in fig. 2 (a); wherein x is_bj,y_bjAbscissa and ordinate, x, of origin, respectively_ej,y_ejRespectively as the horizontal and vertical coordinates of the end point;

step 1-2) calculates a detection area rectangle D ═ x, y, w, h } according to the lane line position of step 1-1), wherein,

x＝min(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)

y＝min(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)

w＝max(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)-x

h＝max(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)-y

the step 2) specifically comprises the following steps:

normalizing the detection area D in the image I to 576 multiplied by 576 as an input image according to the detection area D and the image I in the step 1), and detecting a vehicle set A ═ { A } in the image I based on a deep convolutional neural network₁,A₂,...,A_i}，A_iPosition of (D) is denoted as PA_i＝{x_i,y_i,w_i,h_iThe detection effect is shown in FIG. 2 (b);

as shown in fig. 3, the convolutional neural network includes: 7 convolutional layers, 4 downsampling layers, 2 full-link layers and one output layer; scanning boundaries in the convolutional layers are automatically filled with 0, and neurons are activated by utilizing a Leaky-ReLu function; and maximum pooling is adopted in the down-sampling layers.

The convolution kernel size of convolution layer C1 is 9 × 9, 32 convolution kernels, the step size is 2, and the generated feature map size is 288 × 288; the window size of the downsampling layer S1 is 4 multiplied by 4, the step size is 4, and the size of the generated feature map is 72 multiplied by 72; the convolution layer C2 has convolution kernel size of 1 × 1, 4 convolution kernels and step size of 1, and the generated feature map size is 72 × 72; the convolution layer C3 has convolution kernel size of 3 × 3, 8 convolution kernels and step length of 1, and the generated feature map size is 72 × 72; the window size of the downsampling layer S2 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 36 multiplied by 36; the convolution layer C4 has convolution kernel size of 1 × 1, 8 convolution kernels and step size of 1, and the generated feature map size is 36 × 36; the convolution layer C5 has convolution kernel size of 3 × 3, 16 convolution kernels and step size of 1, and the generated feature map size is 36 × 36; the window size of the downsampling layer S3 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 18 multiplied by 18; the convolution layer C6 has convolution kernel size of 3 × 3, 32 convolution kernels and step size of 1, and the generated feature map size is 18 × 18; the window size of the downsampling layer S4 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 9 multiplied by 9; the convolution layer C7 has convolution kernel size of 1 × 1, 64 convolution kernels and step size of 1, and the generated feature map size is 9 × 9; the full-junction layer F1 is composed of 256 neurons, and the neurons are activated by using a Relu function; the full connection layer F2 is composed of 4096 neurons, and the neurons are activated by using a Leaky-ReLu function; the output layer consists of 891 neurons, which are activated using the Relu function.

The step 3) specifically comprises the following steps:

step 3-1) the set of vehicles a ═ { a) obtained according to step 2)₁,A₂,...,A_iScreening out a vehicle set B which is intersected with the lane line L from the lane line L₁,B₂,...,B_iAnd (c) the step of (c) in which,

step 3-2) traversing the tracking vehicle list TL ═ T according to the vehicle set B obtained in the step 3-1)₁,T₂,...,T_j}, selecting the contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; calculating the Hog characteristic Hog_jAnd hog_kThen calculate the remaining chord distances Cos_jk(ii) a Reselecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd a corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching each other and proceeding to step 3-3); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatching vehicles and entering the step 3-4);

step 3-3) vehicle C obtained according to step 3-2)_mUpdating tracked vehicle T_jIs the current vehicle position, i.e. TP_j＝CP_m(ii) a Updating tracked vehicles to current vehicle pictures, i.e. TM_j＝CM_m(ii) a Continuous heelNumber of times of losing TT_j0, the horizontal directed distance TD of the tracking point from the lane line L_jUpdated to the horizontal directed distance, TD, of the current tracking point from the lane line L_j＝CD_m(ii) a At this time C_mMatched, current vehicle is matched flag CF_i＝true；

Step 3-4) updating the tracked vehicle T_jNumber of continuous lost times TT_j', wherein TT_j’＝TT_j+1, if TT_j' if the number of consecutive lost tracking is more than 3, the vehicle T is deleted from the tracked vehicle list_j；

Step 3-5) traversing the vehicle set B, and enabling the unmatched vehicles B_u＝{BP_u,BM_u,BF_u,BQ_u,BD_uAs a new tracked vehicle T_u＝{TP_u,TM_u,TT_u,TOD_u,TD_uAdds to the tracked vehicle list TL, updates the vehicle tracking list TL, wherein,

TP_u＝BP_u

TM_u＝BM_u

TT_u＝0

TOD_u＝BD_u

TD_u＝BD_u。

the step 3-1) specifically comprises the following steps:

step 3-1-1) obtaining the lane line L according to the step 1) and the step 2)_jAnd vehicle A_iPosition rectangle PA_iCalculating the lane line L_jAnd a vehicle A_iPosition rectangle PA_iIntersection of four sides and lane line L_jThe intersections with the left, right, upper and lower edges of the rectangle are pl_ij＝{x_lij,y_lij}、pr_ij＝{x_rij,y_rij}、pu_ij＝{x_uij,y_uij}、pd_ij＝{x_dij,y_dij}；

Wherein, { x_lij,y_lijIs the horizontal and vertical coordinates of the left edge intersection point, { x }_rij,y_rijThe horizontal and vertical coordinates of the right edge intersection point, { x } respectively_uij,y_uijThe horizontal and vertical coordinates of the intersection point of the upper side edge are respectively, and the horizontal and vertical coordinates of the intersection point of the lower side edge are respectively.

Step 3-1-2) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the left side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+w_i

y_i+h_i/2＜y_lij＜y_i+h_i

step 3-1-3) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the right side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i＜x_uij＜x_i+3w_i/4

y_i+h_i/2＜y_rij＜y_i+h_i

step 3-1-4) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is positioned at the upper side and the lower side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+3w_i/4

x_i+w_i/4＜x_dij＜x_i+3w_i/4。

the step 3-2) specifically comprises the following steps:

step 3-2-1) initializing vehicle information according to the vehicle set B obtained in step 3-1), wherein the vehicle information comprises the following steps: vehicle position BP_i＝{x_i,y_i,w_i,h_iGet the vehicle picture BM_iWhether the current vehicle is matched or not is marked BF_iFalse, tracking point BQ_i＝(x_i+w_i/2,y_i+4h_i/5) and tracking point BQ_iWith each lane line L_jHorizontal directed distance ofBd_ijI.e. vehicles B_i＝{BP_i,BM_i,BF_i,BQ_i,BD_iIn which, BD_i＝{Bd_i1,Bd_i2,...,Bd_il}；

Step 3-2-2) traversing and tracking vehicle list TL ═ T₁,T₂,...,T_j}，T_j＝{TP_j,TM_j,TT_j,TOD_j,TD_j}，T_jE.g. TL, wherein TP_jIndicating the position of the vehicle, TM_jShows a picture of a vehicle, TT_jIndicating number of consecutive tracking losses, TOD_jIndicating the horizontal directional distance, TD, of the starting tracking point from the lane line L_jRepresenting the horizontal directed distance between the current tracking point and the lane line L; TOD_j＝{TOd_j1,TOd_j2,...,TOd_jl}，TD_j＝{Td_j1,Td_j2,...,Td_jl}; computing tracked vehicle T_jVehicle position TP_jWith each unmatched vehicle position BP in the vehicle set B_iCoincidence degree U of_ij：

U_ij＝S_cij/S_zij

Wherein S is_zijIs BP_iAnd TP_jUnion of rectangular areas, S_cijIs BP_iAnd TP_jIntersection of rectangular areas;

selecting a contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; if C is empty, the vehicle T does not exist and is tracked in the vehicle set B_jMatching vehicles and entering the step 3-4); if C is not empty, entering step 3-2-3);

step 3-2-3) the set of vehicles C ═ { C) obtained according to step 3-2-2)₁,C₂,...,C_mWill track vehicle T_jVehicle picture TM_jAnd a picture CM of each vehicle in the vehicle set C_kNormalizing to 64 × 64 size, graying, and calculating the Hog characteristic Hog_jAnd hog_k(ii) a Wherein, in the calculation, the window size is 64 × 64, the block size is 16 × 16, the block sliding increment size is 8 × 8, the cell size is 8 × 8, and the number of gradient histograms in each cell unit is 9;

step 3-2-4) Hog-characteristic Hog obtained according to step 3-2-3)_jAnd hog_kCalculating the remaining chord distances Cos_jk；

Step 3-2-5) cosine distance Cos obtained according to step 3-2-4)_jkSelecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching with each other, and proceeding to step 3-3), the vehicle detection effect after matching is as shown in fig. 2 (c); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatched vehicles and step 3-4) are carried out, and the detection effect of the matching failure vehicles is shown in figure 2 (d).

The step 4) specifically comprises the following steps:

step 4-1) traversing the updated tracked vehicle list TL and calculating T_jStarting tracking point of (1) and lane line L_jHorizontal directed distance TOd'_jjAnd the current tracking point and the lane line L_jIs horizontal directed distance Td'_jjOf TOd'_jj*Td′_jjLess than 0, it means that the vehicles are successively located on the lane line L_jTwo sides, T_jA lane-changing vehicle violating regulations;

step 4-2) obtaining the violation lane-changing vehicle T according to the step 4-1)_jThen, mark out the tracking vehicle T_jAs shown in fig. 2(e), and removes the lane-change violation vehicle T from the updated tracked vehicle list TL_jThe information of (1).

Fig. 2(a) -2(e) are schematic diagrams illustrating the effect of detecting a lane-violation vehicle by using the method in the embodiment of the present invention.

A real-time detection system for calibrating vehicle lane change violations based on lane lines is an intelligent traffic management system and comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that the steps of the detection method are realized when the processor executes the program.

Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and are not limited. Although the present invention has been described in detail with reference to the embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (8)

1. A real-time detection method for calibrating vehicle lane change violations based on lane lines specifically comprises the following steps:

step 1) calibrating a lane line L, and calculating and acquiring a detection rectangular area D and an image I;

step 2) detecting a vehicle set A ═ A of the image I in the detection rectangular region D in the step 1) based on the deep convolutional neural network₁,A₂,...,A_i}；

Step 3) screening out a vehicle set B which is intersected with the lane line L according to the vehicle set A obtained in the step 2) { B }₁,B₂,...,B_iAnd (c) the step of (c) in which,

matching with the tracking vehicle list TL again, and updating the tracking vehicle list TL ═ T₁,T₂,...,T_j}；

Step 4) judging the tracked vehicles T in the updated tracked vehicle list TL_jWhether the vehicle is a violation lane-changing vehicle or not; if tracking vehicle T_jMarking out the tracking vehicle T for the illegal lane changing vehicle_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

2. The detection method according to claim 1, wherein the step 1) specifically comprises:

step 1-1) calibrating lane line L ═ { L ═₁,L₂,...,L_j}, each lane line L_jThe positions of (a) include: starting point pb_j＝{x_bj,y_bjAnd end point pe_j＝{x_ej,y_ej}; wherein x is_bj,y_bjAbscissa and ordinate, x, of origin, respectively_ej,y_ejRespectively as the horizontal and vertical coordinates of the end point;

step 1-2) calculates a detection area rectangle D ═ x, y, w, h } according to the lane line position of step 1-1), wherein,

x＝min(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)

y＝min(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)

w＝max(x_b1,x_e1,x_b2,x_e2,...,x_bj,x_ej)-x

h＝max(y_b1,y_e1,y_b2,y_e2,...,y_bj,y_ej)-y。

3. the detection method according to claim 2, wherein step 2) specifically comprises:

normalizing the detection rectangular area D in the image I to 576 multiplied by 576 as an input image according to the detection rectangular area D and the image I in the step 1), and detecting a vehicle set A ═ { A } in the image I based on a deep convolutional neural network₁,A₂,...,A_i}，A_iPosition of (D) is denoted as PA_i＝{x_i,y_i,w_i,h_i}；x_i,y_i,w_i,h_iRespectively corresponding to a vehicle rectangle A_iThe horizontal coordinate of the lower left corner, the vertical coordinate of the lower left corner, the width and the height;

wherein, this convolutional neural network includes: 7 convolutional layers, 4 downsampling layers, 2 full-link layers and one output layer; scanning boundaries in the convolutional layers are automatically filled with 0, and neurons are activated by utilizing a Leaky-ReLu function; maximum pooling is adopted in the down-sampling layers;

the convolution kernel size of convolution layer C1 is 9 × 9, 32 convolution kernels, the step size is 2, and the generated feature map size is 288 × 288; the window size of the downsampling layer S1 is 4 multiplied by 4, the step size is 4, and the size of the generated feature map is 72 multiplied by 72;

the convolution layer C2 has convolution kernel size of 1 × 1, 4 convolution kernels and step size of 1, and the generated feature map size is 72 × 72;

the convolution layer C3 has convolution kernel size of 3 × 3, 8 convolution kernels and step length of 1, and the generated feature map size is 72 × 72; the window size of the downsampling layer S2 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 36 multiplied by 36;

the convolution layer C4 has convolution kernel size of 1 × 1, 8 convolution kernels and step size of 1, and the generated feature map size is 36 × 36;

the convolution layer C5 has convolution kernel size of 3 × 3, 16 convolution kernels and step size of 1, and the generated feature map size is 36 × 36; the window size of the downsampling layer S3 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 18 multiplied by 18;

the convolution layer C6 has convolution kernel size of 3 × 3, 32 convolution kernels and step size of 1, and the generated feature map size is 18 × 18; the window size of the downsampling layer S4 is 2 multiplied by 2, the step size is 2, and the size of the generated feature map is 9 multiplied by 9;

the convolution layer C7 has convolution kernel size of 1 × 1, 64 convolution kernels and step size of 1, and the generated feature map size is 9 × 9;

the full-junction layer F1 is composed of 256 neurons, and the neurons are activated by using a Relu function; the full connection layer F2 is composed of 4096 neurons, and the neurons are activated by using a Leaky-ReLu function; the output layer consists of 891 neurons, which are activated using the Relu function.

4. The detection method according to claim 1, wherein the step 3) specifically comprises:

step 3-1) the set of vehicles a ═ { a) obtained according to step 2)₁,A₂,...,A_iScreening out a vehicle set B which is intersected with the lane line L from the lane line L₁,B₂,...,B_iAnd (c) the step of (c) in which,

step 3-2) traversing the tracking vehicle list TL ═ T according to the vehicle set B obtained in the step 3-1)₁,T₂,...,T_j}, selecting the contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; wherein, U_ij＝S_cij/S_zij(ii) a Wherein S is_zijIs BP_iAnd TP_jUnion of rectangular areas; s_cijIs BP_iAnd TP_jIntersection of rectangular areas; BP (Back propagation) of_iFor tracking vehicles B_iThe vehicle position of (a); calculating the Hog characteristic Hog_jAnd hog_kThen calculate the remaining chord distances Cos_jk(ii) a Reselecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd a corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching each other and proceeding to step 3-3); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatching vehicles and entering the step 3-4);

step 3-3) vehicle C obtained according to step 3-2)_mUpdating tracked vehicle T_jIs the current vehicle position, i.e. TP_j＝CP_m(ii) a Updating tracked vehicles to current vehicle pictures, i.e. TM_j＝CM_m；TP_jFor tracking vehicles T_jThe vehicle position of (a); CP (CP)_mIs the current vehicle position; TM_jFor tracking vehicles T_jThe vehicle picture of (1); CM (compact message processor)_mThe current vehicle picture is obtained; number of consecutive lost times TT_j0, the horizontal directed distance TD of the tracking point from the lane line L_jUpdated to the horizontal directed distance, TD, of the current tracking point from the lane line L_j＝CD_m(ii) a At this time C_mMatched, current vehicle is matched flag CF_i＝true；

Step 3-4) updating the tracked vehicle T_jNumber of continuous lost times TT_j', wherein TT_j’＝TT_j+1, if TT_j' if the number of consecutive lost tracking is more than 3, the vehicle T is deleted from the tracked vehicle list_j；

Step 3-5) traversing the vehicle set B, and enabling the unmatched vehicles B_u＝{BP_u,BM_u,BF_u,BQ_u,BD_uAs a new tracked vehicle T_u＝{TP_u,TM_u,TT_u,TOD_u,TD_uAdds to the tracked vehicle list TL, updates the vehicle tracking list TL, wherein,

wherein, TP_uNew tracked vehicle positions; TM_uA new tracked vehicle picture is obtained; TOD_uThe horizontal directed distance between the initial tracking point and the lane line L; TD_uThe horizontal directed distance between the current tracking point and the lane line L; tracking number of consecutive lost tracking TT of new tracked vehicle_u＝0；BP_uUnmatched vehicle positions; BM_uThe picture is an unmatched vehicle picture; BF (BF) generator_uA matched or not flag for an unmatched vehicle; BQ_uUnmatched vehicle tracking points; BD_uThe horizontal directional distance from the point to the lane line L is tracked for unmatched vehicles.

5. The detection method according to claim 4, wherein the step 3-1) specifically comprises:

step 3-1-1) obtaining the lane line L and the vehicle A according to the step 1) and the step 2)_iPosition rectangle PA_iCalculating the lane line L_jAnd a vehicle A_iPosition rectangle PA_iIntersection of four sides and lane line L_jThe intersections with the left, right, upper and lower edges of the rectangle are pl_ij＝{x_lij,y_lij}、pr_ij＝{x_rij,y_rij}、pu_ij＝{x_uij,y_uij}、pd_ij＝{x_dij,y_dij}；

Wherein, { x_lij,y_lijIs the horizontal and vertical coordinates of the left edge intersection point, { x }_rij,y_rijThe horizontal and vertical coordinates of the right edge intersection point, { x } respectively_uij,y_uijRespectively are the horizontal and vertical coordinates of the intersection point of the upper side edge and the horizontal and vertical coordinates of the intersection point of the lower side edge;

step 3-1-2) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the left side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+w_i

y_i+h_i/2＜y_lij＜y_i+h_i；

wherein x is_i,y_i,w_i,h_iRespectively corresponding to a vehicle rectangle A_iThe horizontal coordinate of the lower left corner, the vertical coordinate of the lower left corner, the width and the height;

step 3-1-3) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is on the upper side and the right side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used for_iAdding the vehicle into the vehicle set B;

x_i＜x_uij＜x_i+3w_i/4

y_i+h_i/2＜y_rij＜y_i+h_i；

step 3-1-4) according to the intersection point of the lane line and the rectangle obtained in the step 3-1-1), if the intersection point is positioned at the upper side and the lower side of the rectangle and meets the following preset conditions, the vehicle rectangle A is used_iAdding the vehicle into the vehicle set B;

x_i+w_i/4＜x_uij＜x_i+3w_i/4

x_i+w_i/4＜x_dij＜x_i+3w_i/4。

6. the detection method according to claim 4, wherein the step 3-2) specifically comprises:

step 3-2-1) according to the stepsInitializing vehicle information by the vehicle set B obtained in the step 3-1), wherein the vehicle information comprises: vehicle position BP_i＝{x_i,y_i,w_i,h_iGet the vehicle picture BM_iWhether the current vehicle is matched or not is marked BF_iFalse, tracking point BQ_i＝(x_i+w_i/2,y_i+4h_i/5) and tracking point BQ_iWith each lane line L_jIs a horizontal directed distance Bd_ijI.e. tracking vehicle B_i＝{BP_i,BM_i,BF_i,BQ_i,BD_iIn which, BD_i＝{Bd_i1,Bd_i2,...,Bd_il}; wherein x is_i,y_i,w_i,h_iRespectively corresponding to a vehicle rectangle B_iThe horizontal coordinate of the lower left corner, the vertical coordinate of the lower left corner, the width and the height;

step 3-2-2) traversing and tracking vehicle list TL ═ T₁,T₂,...,T_j}，T_j＝{TP_j,TM_j,TT_j,TOD_j,TD_j}，T_jE.g. TL, wherein TP_jIndicating the position of the vehicle, TM_jShows a picture of a vehicle, TT_jIndicating number of consecutive tracking losses, TOD_jIndicating the horizontal directional distance, TD, of the starting tracking point from the lane line L_jRepresenting the horizontal directed distance between the current tracking point and the lane line L; TOD_j＝{TOd_j1,TOd_j2,...,TOd_jl}，TD_j＝{Td_j1,Td_j2,...,Td_jl}; computing tracked vehicle T_jVehicle position TP_jWith each unmatched vehicle position BP in the vehicle set B_iCoincidence degree U of_ij：

U_ij＝S_cij/S_zij

Wherein S is_zijIs BP_iAnd TP_jUnion of rectangular areas, S_cijIs BP_iAnd TP_jIntersection of rectangular areas;

selecting a contact ratio U_ijA set of vehicles C > 0.2,

C＝{C₁,C₂,...,C_m}; if C is empty, the vehicle T does not exist and is tracked in the vehicle set B_jMatching vehicles and entering the step 3-4); if C is not empty, entering step 3-2-3);

step 3-2-3) the set of vehicles C ═ { C) obtained according to step 3-2-2)₁,C₂,...,C_mWill track vehicle T_jVehicle picture TM_jAnd a picture CM of each vehicle in the vehicle set C_kNormalizing to 64 × 64 size, graying, and calculating the Hog characteristic Hog_jAnd hog_k(ii) a Wherein, in the calculation, the window size is 64 × 64, the block size is 16 × 16, the block sliding increment size is 8 × 8, the cell size is 8 × 8, and the number of gradient histograms in each cell unit is 9;

step 3-2-4) Hog-characteristic Hog obtained according to step 3-2-3)_jAnd hog_kCalculating the remaining chord distances Cos_jk；

Step 3-2-5) cosine distance Cos obtained according to step 3-2-4)_jkSelecting a tracked vehicle T_jThe largest cosine distance Cos from the vehicle set C_jmAnd corresponding vehicle C_mIf Cos_jmIf greater than 0.75, then T_jAnd C_mMatching each other and proceeding to step 3-3); if Cos_jmLess than or equal to 0.75, the vehicle T is not existed and tracked in the vehicle set C_jMatched vehicles and proceed to step 3-4).

7. The detection method according to claim 1, wherein the step 4) specifically comprises:

step 4-1) traversing the updated tracked vehicle list TL and calculating T_jStarting tracking point of (1) and lane line L_jHorizontal directed distance TOd'_jjAnd the current tracking point and the lane line L_jIs horizontal directed distance Td'_jjOf TOd'_jj*Td′_jjLess than 0, it means that the vehicles are successively located on the lane line L_jTwo sides, T_jA lane-changing vehicle violating regulations;

step 4-2) according toStep 4-1) obtaining violation lane-changing vehicle T_jThen, mark out the tracking vehicle T_jAnd deleting the violation lane-change vehicle T from the updated tracked vehicle list TL_jThe information of (1).

8. A real-time detection system for calibrating vehicle lane change violations based on lane marking, the detection system is an intelligent traffic management system, and comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and the processor is characterized in that when executing the program, the steps of the real-time detection method of one of claims 1 to 7 are realized.

Images