CN106529415A - Characteristic and model combined road detection method - Google Patents

Abstract

The invention discloses a characteristic and model combined road detection method. The method comprises the following steps that S1) a camera collects lane image information in real time; S2) a road segmentation method based on a multilayer neural network is used to implement road segmentation, and a road area is searched for; S3) a road boundary is searched according to a segmentation result of the road area and a non-road area; S4) boundary points of the road are extracted, a condition probability density spreading algorithm is used to estimate positions of disappeared points, and the positions of disappeared points are tracked; and S5) the positions of disappeared points are fused to a heuristic fitting algorithm in a boundary fitting process of a secondary curve road to fit a road model. The road detection method is highly robust and highly real-time in complex environments, and has certain theoretical and practical application values.

Description

The Approach for road detection of binding characteristic and model

Technical field

Present invention relates particularly to the Approach for road detection of a kind of binding characteristic and model.

Background technology

Cognitive calculating of audio visual information is the cross discipline of information science, life science and combing science, its developing water The flat comprehensive strength for reflecting national information service and related industry, unmanned technology is audio visual information processing basic theory Integrated, and the national audio visual field of information processing of the key technologies such as the cognitive related brain-computer interface of research, audio visual The embodiment of holistic approach strength.Lane detection technology wherein based on computer vision, is pilotless automobile intelligent navigation system One of core technology of system.

Road in reality can be divided into structuring and unstructured road, and current unstructured road road environment is more , easily by odjective causes such as weather, the interference of illumination variation, accurate, real-time non-structural Road Detection is still for complicated, road surface characteristic It is a challenging problem, in the research of existing unstructured road detection algorithm, the Road Detection of feature based Algorithm robustness is stronger, but there is a problem of that algorithm complex is too high, adaptive capacity to environment is not enough；Road Detection based on model Algorithm real-time preferably, but still be present poor to the robustness of shade, illumination variation.

The content of the invention

The technical problem to be solved in the present invention is to provide the Approach for road detection of a kind of binding characteristic and model.

The Approach for road detection of binding characteristic and model, comprises the following steps：

S1：By camera Real-time Collection carriageway image information；

S2：Lane segmentation is carried out using the lane segmentation method based on multilayer neural network, wherein road area is found out；

S3：Road boundary is found out according to the segmentation result of road area and non-rice habitats region；

S4：The boundary point of road is extracted, use condition probability density propagation algorithm estimating vanishing point position, to road Road end point position is tracked；

S5：Calculated using the heuristic fitting during by end point Co-factor propagation to conic section road link edge fitting Method is fitted road model.

Further, the lane segmentation method based on multilayer neural network is as follows：

S2-1：Image is transformed into into HSV color spaces from rgb color space, by road area and non-rice habitats area sample In each pixel tone and saturation degree be supplied to as multilayer neural network input vector as the characteristic vector of pixel Input layer, the output result of each input layer are equal to the respective components in input feature value；

S2-2：According to network connection, hidden layer node is weighted summation to each input, and the scalar for obtaining is referred to as net sharp It is living, i.e.,：

Wherein, i for input layer index, d be input layer number, j for hidden layer node index, ω_ijFor defeated Enter node layer i to the weights of hidden layer node j, ω_j0For additional feature value x₀=1 with additional weights ω₀Product；

Each hidden layer node is using net activation as input, and excites an output component, and this component is net activation Nonlinear function, it is as follows from nonlinear function：

S2-3：Each output node layer will receive the output signal of hidden layer node, calculate its net activation：

Wherein, indexes of the j for hidden layer node, η_HFor node in hidden layer, k is the index for exporting node layer, ω_kjFor defeated Enter node layer j to the weights of hidden layer node k；

Output node layer excites output layer component in a similar fashion：

The process that classification judgement is carried out using multilayer neural network can be described as calculating output node layer and respectively exporting dividing The process of amount；

S2-4：Using the method for piecemeal segmentation, and use is optimized based on the court verdict correction strategy for being subordinate to probability, It is specific as follows：

(1) square of N*N is divided the image into into, four angular zones of each square are referred to as angle point, it is assumed that any one picture Vegetarian refreshments x belongs to road area R or non-rice habitats region NR, i.e. x ∈ { R, NR }, then angle point C ∈ { R, NR }, and angle point C belong to Road area R's or non-rice habitats region NR is subordinate to probability and can be defined as：

Wherein, N is pixel number in angle point region；

(2) according to four angle points in piecemeal type identification result and be subordinate to probability, piecemeal can be divided into three classes：

If four angle points of certain piecemeal belong to road area R, the piecemeal falls within road area R, and is subordinate to probability For：

If four angle points of certain piecemeal belong to non-rice habitats region NR, the piecemeal falls within non-rice habitats region NR, and is subordinate to Belonging to probability is：

If in four angle points of certain piecemeal, existing to belong to road area R, also there is the angle point for belonging to non-rice habitats region NR, then The piecemeal belongs to Mixed Zone MIX, and is subordinate to the average for being subordinate to probability that probability is that each angle point belongs to judgement classification；

(3) assume that the piecemeal for belonging to Mixed Zone MIX only appears in the piecemeal and non-rice habitats region for belonging to road area R Between NR；Piecemeal is from left to right scanned with behavior main sequence, if it is fast to run into mixing, its left and right piecemeal attribute whether phase is tested Together, if there is the piecemeal for belonging to Mixed Zone MIX in its left and right piecemeal, test piecemeal is moved again to the left or to the right；If should The left and right piecemeal attribute of block is identical, belongs to road area R, or belongs to non-rice habitats region NR, then tests the person in servitude of the piecemeal Category probability, if the probability that is subordinate to of the piecemeal is less than a given threshold value, its attribute is corrected for its left and right block's attribute phase Together；If the probability that is subordinate to of the piecemeal is higher than certain given threshold value, its attribute is not changed.

Further, to carry out extracting method as follows for the boundary point of road：

S4-1：Coordinate system is set up to image, generally with the image upper left corner as origin, is x-axis direction to the right, is y-axis downwards Direction, with y-axis as principal direction, from left to right scan image all pixels point；

S4-2：For every a line pixel sequence, find out Road sub-line section, if including a plurality of candidate roads sub-line section in one-row pixels point, merges the sub-line that wherein distance is closer to Section；

S4-3：The right boundary coordinate of each road line segment is recorded, as candidate roads boundary point；

S4-4：Scanning element is moved to next line, until all pixels spot scan is finished, all candidate roads sides is recorded Boundary's point set.

Further, by end point Co-factor propagation in road boundary fit procedure heuristic fitting algorithm is concrete such as Under：

S5-1：For the n-th frame in image sequence, in image, the observed value of the position of road end point Zn is defined as n-th The intersection point of road boundary the fitting result LRn and LRn of frame；

S5-2：By observed value Zn of the position of n-th frame road end point, counted using day sword probability density propagation algorithm Calculate the prior probability of the position of the (n+1)th frame road end point；

S5-3：The (n+1)th frame end point position is predicted the outcome according to step S5-2, and the time to the (n+1)th two field picture Road boundary point set is selected to carry out initial fitting using least square method to the left bounding lines LRn+1 and LRn+1 of road, fitting During, the weight of road end point is more than other road boundary point weights；

S5-4：The distance between the road boundary point of all extractions and the road boundary equation for fitting is calculated, if certain Point and road boundary equation distance are more than the average of distance between all road boundary points and absorbing boundary equation, and the shop will be excluded, And according to the coordinate of the point, find the candidate roads sub-line section is expert at by the point, if can find out a strip line segment boundary point with The distance of road boundary equation is close to, then it is candidate roads boundary point to replace the boundary point, otherwise, excludes the point；

S5-5：A most young waiter in a wineshop or an inn is reused using new road boundary point set and the (n+1)th predicting the outcome for frame end point position Multiplication is fitted the left bounding lines LRn+1 and LRn+1 of road, and calculates the intersection point of LRn+1 and LRn+1 as the (n+1)th frame road Observed value Zn+1 of end point；

S5-6：Use condition probability density propagation algorithm calculates posterior probability.

The invention has the beneficial effects as follows：

(1) the lane segmentation method based on multilayer neural network is improved, by multilayer neural network to road and non-road The tone of each pixel of road area sample, saturation degree are learnt, and are reached come new pixel of classifying using the network after study The antagonism shade of boosting algorithm and the purpose of illumination variation robustness；The method for employing piecemeal classification simultaneously suppresses camera to adopt The random noise that collection process is produced, and devise based on the court verdict correction strategy for being subordinate to probability, mitigate roadway area to reach The purpose that domain is affected with chaff interference in non-rice habitats region；

(2) road end point is tracked and is predicted using conditional probability density propagation algorithm, and by road end point Predict the outcome and incorporate road boundary fit procedure, compensate for not making full use of based on the traditional algorithm of single-frame images process and regard Frequency flow between each frame correlation shortcoming, having reached strengthens the purpose of algorithm stability and robustness.

Specific embodiment

The present invention is further elaborated for specific examples below, but not as a limitation of the invention.

Test 1 method of the present invention actual processing effect and traditional algorithm effect is compared.

First, the quantitative determination Performance Evaluation of Pixel-level

Using the quantitative evaluating method of Pixel-level, final detection result is divided into into four classes, all categories are as shown in table 1 below.

The quantitative evaluating method definition of 1 Pixel-level of table

In table 1, it is non-rice habitats region that TN is actual value, and algorithm detected value is also the pixel number in non-rice habitats region；FN It is road area for actual value, algorithm detected value is the pixel number in non-rice habitats region；It is non-rice habitats region that FP is actual value, Pixel number of the algorithm detected value for road area；It is road area that TP is actual value, and algorithm detected value is also road area Pixel number.

On the basis of differentiating to each pixel, the statistic property index of quantitative analysis method is as shown in table 2 below.

The quantitative evaluating method performance indications of 2 Pixel-level of table

Each performance indications that algorithm proposed by the present invention is counted in different scenes with existing Density Estimator algorithm On contrast it is as shown in table 3.

3 each performance indications comparative analysis of table

Relative to traditional algorithm, the algorithm of the present invention each performance indications in several scenes have all been lifted, accuracy of detection Up to 978%, detection quality is up to 94.3%, and recall rate is up to 97.5%.

2nd, the performance evaluation of disappearance point estimation

The video sequence complete to one carries out quantitative analysis using the testing result of distinct methods, as a result as shown in table 3.

The performance quantitative analysis of 3 disappearance point estimation of table

As shown in upper table, in average behavior of the detection quality with accuracy of detection, using the heuristic road of disappearance point estimation Road approximating method is obtained in that 3% or so lifting, and recall rate also has a certain upgrade.

Claims (4)

1. the Approach for road detection of binding characteristic and model, it is characterised in that comprise the following steps：

S1：By camera Real-time Collection carriageway image information；

S2：Lane segmentation is carried out using the lane segmentation method based on multilayer neural network, wherein road area is found out；

S3：Road boundary is found out according to the segmentation result of road area and non-rice habitats region；

S4：The boundary point of road is extracted, use condition probability density propagation algorithm estimating vanishing point position disappears to road Lose point position to be tracked；

S5：Intended using the heuristic fitting algorithm during by end point Co-factor propagation to conic section road link edge fitting Close road model.

2. Approach for road detection according to claim 1, it is characterised in that the road based on multilayer neural network point Segmentation method is as follows：

S2-1：Image is transformed into into HSV color spaces from rgb color space, road area is every with non-rice habitats area sample The tone of one pixel and saturation degree are supplied to input as the characteristic vector of pixel as multilayer neural network input vector Layer, the output result of each input layer are equal to the respective components in input feature value；

S2-2：According to network connection, hidden layer node is weighted summation to each input, and the scalar for obtaining is referred to as net activation, I.e.：

${\mathrm{net}}_j=\underset{i=1}{\overset{d}{\mathrm{\Σ}}}{x}_i{\mathrm{\ω}}_{ji}+{\mathrm{\ω}}_{j0}=\underset{i=0}{\overset{d}{\mathrm{\Σ}}}{x}_i{\mathrm{\ω}}_{ji}={\mathrm{\ω}}_j^{i}x,$

Wherein, i for input layer index, d be input layer number, j for hidden layer node index, ω_ijFor input layer Weights of the node i to hidden layer node j, ω_j0For additional feature value x₀=1 with additional weights ω₀Product；

Each hidden layer node is using net activation as input, and excites an output component, and this component is the non-thread of net activation Property function, it is as follows from nonlinear function：

$y_j=f\left({\mathrm{net}}_j\right)=\tanh x_j=\frac{e^{x_j}-e^{-x_j}}{e^{x_j}+e^{-x_j}};$

S2-3：Each output node layer will receive the output signal of hidden layer node, calculate its net activation：

${\mathrm{net}}_k=\underset{j=1}{\overset{{\mathrm{\η}}_H}{\Σ}}{y}_j{\mathrm{\ω}}_{kj}+{\mathrm{\ω}}_{k0}=\underset{j=0}{\overset{{\mathrm{\η}}_H}{\Σ}}{y}_j{\mathrm{\ω}}_{kj}={\mathrm{\ω}}_k^{j}y,$

Wherein, indexes of the j for hidden layer node, η_HFor node in hidden layer, k is the index for exporting node layer, ω_kjFor input layer Weights of the node j to hidden layer node k；

Output node layer excites output layer component in a similar fashion：

$z_k=f\left({\mathrm{net}}_k\right)=\tanh y_k=\frac{e^{y_k}-e^{-y_k}}{e^{y_k}+e^{-y_k}};$

The process that classification judgement is carried out using multilayer neural network can be described as calculating each output component of output node layer Process；

S2-4：The method split using piecemeal, and using being optimized based on the court verdict correction strategy for being subordinate to probability, specifically It is as follows：

(1) square of N*N is divided the image into into, four angular zones of each square are referred to as angle point, it is assumed that any one pixel x Road area R or non-rice habitats region NR, i.e. x ∈ { R, NR } is belonged to, then angle point C ∈ { R, NR }, and angle point C belong to roadway area Domain R's or non-rice habitats region NR is subordinate to probability and can be defined as：

$p(C\∈R)=\underset{i=1}{\overset{N}{\Σ}}if(x_i\∈R)/N$

$p\left(C\∈NR\right)=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}if\left(x_i\∈NR\right)/N$

Wherein, N is pixel number in angle point region；

(2) according to four angle points in piecemeal type identification result and be subordinate to probability, piecemeal can be divided into three classes：

If four angle points of certain piecemeal belong to road area R, the piecemeal falls within road area R, and is subordinate to probability and is：

$p(block\∈R)=\underset{i=1}{\overset{4}{\Σ}}p(C_i\∈R)/4;$

If four angle points of certain piecemeal belong to non-rice habitats region NR, the piecemeal falls within non-rice habitats region NR, and is subordinate to general Rate is：

$p(block\∈NR)=\underset{i=1}{\overset{4}{\Σ}}p(C_i\∈NR)/4;$

If in four angle points of certain piecemeal, existing to belong to road area R, also there is the angle point for belonging to non-rice habitats region NR, then this point Block belongs to Mixed Zone MIX, and is subordinate to the average for being subordinate to probability that probability is that each angle point belongs to judgement classification；

(3) assume the piecemeal for belonging to Mixed Zone MIX only appear in belong to the piecemeal of road area R and non-rice habitats region NR it Between；Piecemeal is from left to right scanned with behavior main sequence, if it is fast to run into mixing, whether identical, such as if testing its left and right piecemeal attribute Really there is the piecemeal for belonging to Mixed Zone MIX in its left and right piecemeal, then test piecemeal and move to the left or to the right again；If a left side for the block Right piecemeal attribute is identical, belongs to road area R, or belongs to non-rice habitats region NR, then that tests the piecemeal is subordinate to probability, If the probability that is subordinate to of the piecemeal is less than a given threshold value, it is identical that its attribute is corrected for its left and right block's attribute；If The probability that is subordinate to of the piecemeal is higher than certain given threshold value, then do not change its attribute.

3. Approach for road detection according to claim 1, it is characterised in that the boundary point of road carries out extracting method such as Under：

S4-1：Coordinate system is set up to image, generally with the image upper left corner as origin, is x-axis direction to the right, be downwards y-axis direction, With y-axis as principal direction, from left to right scan image all pixels point；

S4-2：For every a line pixel sequence, find out Sub-line section, if including a plurality of candidate roads sub-line section in one-row pixels point, merges the sub-line section that wherein distance is closer to；

S4-3：The right boundary coordinate of each road line segment is recorded, as candidate roads boundary point；

S4-4：Scanning element is moved to next line, until all pixels spot scan is finished, all candidate roads boundary points is recorded Collection.

4. Approach for road detection according to claim 1, it is characterised in that end point Co-factor propagation to road boundary is intended Heuristic fitting algorithm during conjunction is specific as follows：

S5-1：For the n-th frame in image sequence, in image, the observed value of the position of road end point Zn is defined as n-th frame The intersection point of road boundary fitting result LRn and LRn；

S5-2：By observed value Zn of the position of n-th frame road end point, is calculated using day sword probability density propagation algorithm The prior probability of the position of n+1 frame road end points；

S5-3：The (n+1)th frame end point position is predicted the outcome according to step S5-2, and the candidate road to the (n+1)th two field picture Roadside circle point set carries out initial fitting, the process of fitting using least square method to the left bounding lines LRn+1 and LRn+1 of road In, the weight of road end point is more than other road boundary point weights；

S5-4：Calculate the distance between the road boundary point of all extractions and the road boundary equation for fitting, if certain point with Average of the road boundary equation distance more than distance between all road boundary points and absorbing boundary equation, the shop will be excluded, and root According to the coordinate of the point, the candidate roads sub-line section is expert at by the point is found, if a strip line segment boundary point and road can be found out The distance of absorbing boundary equation is close to, then it is candidate roads boundary point to replace the boundary point, otherwise, excludes the point；

S5-5：Least square method is reused using new road boundary point set and the (n+1)th predicting the outcome for frame end point position The left bounding lines LRn+1 and LRn+1 of road is fitted, and calculates the intersection point of LRn+1 and LRn+1 and disappeared as the (n+1)th frame road Observed value Zn+1 of point；

S5-6：Use condition probability density propagation algorithm calculates posterior probability.