CN106778528A - A kind of method for detecting fatigue driving based on gaussian pyramid feature - Google Patents

Abstract

The invention discloses a kind of method for detecting fatigue driving based on gaussian pyramid feature, comprise the following steps：S1, the driver's driving image to shooting are pre-processed；S2, the pretreated view data is carried out down-sampled, obtain multi-resolution Gaussian pyramid feature；S3, the feature is carried out to be matched with the property data base for prestoring, judge whether driver is fatigue driving.Method for detecting fatigue driving based on gaussian pyramid feature of the invention uses gaussian pyramid characteristic analysis method, and algorithm complex is low compared to existing technology, computational efficiency is high, recognition accuracy is high.

Description

Fatigue driving detection method based on Gaussian pyramid characteristics

Technical Field

The invention relates to the field of image recognition, in particular to a fatigue driving detection method based on Gaussian pyramid characteristics.

Background

With the development of social economy, automobiles become the core strength of transporting goods and traffic in people's daily life, and unfortunately, with the increase of the usage amount of automobiles, the number of people who die and are seriously injured due to traffic accidents is increased year by year. Therefore, the problem of road traffic safety is increasingly attracting attention, and improving the safety performance of automobiles and reducing road traffic accidents are social problems which are worthy of attention and important problems which need to be solved urgently through scientific and technological means. Active safe driving is closely related to the behavior and habits of the person. The driving action is monitored and recognized, the safety of the driving action can be analyzed in advance, and unsafe actions are reminded to avoid accidents. In the driving process, the actions of hands and feet of the operated automobile and the head of a driver are the direct expressions of driving behaviors, such as looking up an instrument panel and observing a rearview mirror; however, some actions irrelevant to driving, such as looking at the gear of the transmission, watching the outside of a side window for a long time, operating vehicle-mounted electronic equipment for a long time, driving a mobile phone in the driving process, countersunk head and picking up objects, also exist, and the actions generate great hidden dangers for driving safety. Therefore, developing a fast motion recognition algorithm which sacrifices recognition rate a little under the condition of guaranteeing the real-time performance of the algorithm as much as possible becomes another idea of motion recognition research. Motion is important information for detecting the segmentation of objects and background in a scene, and such methods first assume the periodicity of pedestrian motion, according to which the pedestrian is separated from the background. Human motion recognition is a challenging subject in the field of computer vision, and differences in the appearances of different people, unstable backgrounds, moving cameras and scene lighting changes all add difficulty to motion recognition. Under conventional visual sensor conditions, human motion can be represented by continuous changes between adjacent frames in a video sequence. The analysis methods are mainly classified into three categories: the first category is spatio-temporal based feature analysis such as MHI (Motion Histogram), SI (Silhouette Histogram contour Histogram), and optical flow analysis and spatio-temporal feature volumes, etc.; the second type models the trunk and the four limbs of the human body, then detects the human body in each frame of image in the video sequence and obtains model parameters to describe the action change; the third category uses image statistics to perform statistical identification on the bottom layer information of video frames, for example, in the research of Ke Yan, color segmentation is performed on images in a video sequence, and then motion identification is performed by using statistical information of the same color blocks in consecutive images.

However, the methods of the first class of spatio-temporal feature analysis have low computational complexity and are easy to implement, but are sensitive to image noise; the second type of modeling human body trunk and limbs has good motion description accuracy, but large calculation amount and lower real-time performance; the third type uses image statistical method to perform statistical identification on the bottom information of video frame, which can not meet the requirements of real-time and easy realization. In conclusion, the traditional recognition algorithm has high operation complexity and low efficiency; the recognition accuracy is low.

Disclosure of Invention

The fatigue driving detection method based on the Gaussian pyramid features is capable of reducing algorithm complexity, high in calculation efficiency and high in recognition accuracy.

In order to achieve the purpose of the invention, the invention adopts the technical scheme that:

a fatigue driving detection method based on Gaussian pyramid characteristics comprises the following steps:

s1, preprocessing the shot driving image of the driver;

s2, performing down-sampling on the preprocessed image data to obtain multi-resolution Gaussian pyramid characteristics;

and S3, matching the characteristics with a pre-stored characteristic database, and judging whether the driver is fatigue driving.

Further, the step S1 includes performing optical flow, gaussian smoothing, and normalization calculation on the image once.

Further, the step S1 specifically includes performing optical flow calculation on a current frame image f (I, j, t) and a previous frame image f (I, j, t-1) of the original video sequence to obtain an X-direction velocity { U (X, Y), (X, Y) ∈ I } represented by U, a Y-direction velocity { V (X, Y), (X, Y) ∈ I } represented by V, and four features obtained in the direction of the separation velocity are respectively that U (U, j, t) is right₊Leftward U_-Downward V₊And upward V_-And gaussian smoothing is performed on all features:

the above formula is performed and normalized:

and (3) calculating the similarity S (i, j) of the optical flow characteristics obtained from the video sequence:

wherein,andrespectively representing the calculation results of the corresponding features of the pixel points in the two groups of optical flow feature sequences with similarity to be compared, c representsAndfour directions after Gaussian smoothing and normalization, i, j are respectively frame numbers in corresponding sequences;

performing convolution kernel of the unit matrix on the S (i, j) to obtain:

further, the step S2 includes,

s21, performing multi-level down-sampling on the video sequence to form a video sequence pyramid with L levels, and performing multi-level down-sampling on an initialized video sequence f with a given resolution of M × N₀(i, j, t), each layer f_l(i, j, t) is calculated recursively by the following formula:

wherein f is_l(i, j, t) represents that each frame of image f (i, j) is at the t th frame of pyramid level L (L is more than or equal to 0 and less than or equal to L), r (m, n) is a Gaussian filter,

wherein,

r(m,n)＝r(m)r(n)

r(0)＝a,r(1)＝r(-1)＝1/4,r(2)＝r(-2)＝1/4-2/a；

s22, starting from the level with the lowest resolution, calculating the motion characteristic sequence f of the l-th layer_lHierarchically computing the test samples and each sample in the training setSelecting K candidates according to K neighbors according to the result of the similarity; f is then calculated at level l-1_l-1(i, j, t) corresponds to the similarity between the motion feature and the K candidates previously selected in the l layer, and the result of K neighbors is used for comparison of the l-2 layer with higher resolution until the l with the highest resolution is equal to 0 layer.

Further, the characteristic database stored in advance stores abnormal driving behavior characteristics.

Compared with the prior art, the invention has the beneficial effects

The fatigue driving detection method based on the Gaussian pyramid characteristics adopts a Gaussian pyramid characteristic analysis method, and compared with the prior art, the fatigue driving detection method based on the Gaussian pyramid characteristics is low in algorithm complexity, high in calculation efficiency and high in identification accuracy.

Drawings

Fig. 1 is a flowchart of a fatigue driving detection method based on gaussian pyramid characteristics according to the present invention.

FIG. 2 is a graphical representation of the results of the test in one embodiment.

Detailed Description

The present invention will be described in further detail with reference to specific embodiments. It should be understood that the scope of the above-described subject matter is not limited to the following examples, and any techniques implemented based on the disclosure of the present invention are within the scope of the present invention.

Example 1:

a fatigue driving detection method based on Gaussian pyramid characteristics comprises the following steps:

s1, preprocessing the shot driving image of the driver;

s2, performing down-sampling on the preprocessed image data to obtain multi-resolution Gaussian pyramid characteristics;

and S3, matching the characteristics with a pre-stored characteristic database, and judging whether the driver is fatigue driving.

The fatigue driving detection method based on the Gaussian pyramid characteristics adopts a Gaussian pyramid characteristic analysis method, and compared with the prior art, the fatigue driving detection method based on the Gaussian pyramid characteristics is low in algorithm complexity, high in calculation efficiency and high in identification accuracy.

In a specific embodiment, the step S1 includes performing optical flow, gaussian smoothing, and normalization calculation on the image once.

In one embodiment, the step S1 specifically includes performing optical flow calculation on a current frame image f (I, j, t) and a previous frame image f (I, j, t-1) of the original video sequence to obtain an X-direction velocity { U (X, Y), (X, Y) ∈ I } represented by U, a Y-direction velocity { V (X, Y), (X, Y) ∈ I } represented by V, and four features obtained from the direction of the separation velocity are respectively U (X, j, t) to the right₊Leftward U_-Downward V₊And upward V_-And gaussian smoothing is performed on all features:

the above formula is performed and normalized:

and (3) calculating the similarity S (i, j) of the optical flow characteristics obtained from the video sequence:

wherein,andrespectively representing the calculation results of the corresponding features of the pixel points in the two groups of optical flow feature sequences with similarity to be compared, c representsAndfour directions after Gaussian smoothing and normalization, i, j are respectively frame numbers in corresponding sequences;

performing convolution kernel of the unit matrix on the S (i, j) to obtain:

in one embodiment, the step S2 includes,

s21, performing multi-level down-sampling on the video sequence to form a video sequence pyramid with L levels, and performing multi-level down-sampling on an initialized video sequence f with a given resolution of M × N₀(i, j, t), each layer f_l(i, j, t) is calculated recursively by the following formula:

wherein f is_l(i, j, t) represents that each frame of image f (i, j) is at the t th frame of pyramid level L (L is more than or equal to 0 and less than or equal to L), r (m, n) is a Gaussian filter,

wherein,

r(m,n)＝r(m)r(n)

r(0)＝a,r(1)＝r(-1)＝1/4,r(2)＝r(-2)＝1/4-2/a；

s22, starting from the level with the lowest resolution, calculating the motion characteristic sequence f of the l-th layer_lCalculating the similarity of the test sample and each sample in the training set in a hierarchy, and selecting K candidates according to K neighbors; f is then calculated at level l-1_l-1(i, j, t) corresponds to the similarity between the motion feature and the K candidates previously selected in the l layer, and the result of K neighbors is used for comparison of the l-2 layer with higher resolution until the l with the highest resolution is equal to 0 layer.

In order to identify driving actions in real Time, identify and alarm actions irrelevant to driving, overcome the inherent difficulty of driving action identification and more accurately identify abnormal driving behaviors, the method provides an action characteristic pyramid characteristic based on optical flow, and realizes a rapid action identification algorithm constrained by coarse-to-fine DTW (dynamic Time warping): the method comprises the steps of firstly carrying out optical flow, Gaussian smoothing and normalization calculation on an image in sequence, then carrying out down-sampling on optical flow sequence data generated by an action video sequence for multiple times to form a multi-resolution Gaussian pyramid feature, and finally carrying out classification and identification on actions on the basis of a plurality of layers of pyramid features by taking improvement of overall identification speed as a target. And (3) on the basis of the identification of the action identification research common database, discussing the integral framework of action characteristic extraction, action identification and algorithm, and finally applying the integral framework to the actual problem of driving action identification to judge abnormal driving behaviors. The DTW algorithm provided by the invention can automatically register the two sequences and restore the registration path without considering the inspiration of action in the calculation, and a more accurate detection result is obtained under the real-time requirement. The method comprises the following steps: and searching a path in the similarity matrix, so that points on the path are all calculated by the most similar frame between the two actions, saving coordinates and the direction of the previous point in the path calculation process, and returning according to the original path and removing the points on the boundary to obtain a real matching path after one time of DTW calculation is finished.

Compared with the prior art, the method greatly improves the speed of similarity calculation, and simultaneously keeps that the recognition rate is not greatly reduced due to higher similarity of two adjacent layers of characteristic sequences of the pyramid. The number of selected layers and the K value of K neighbor are proper, so that the calculated amount can be greatly reduced under the condition that the recognition rate is basically unchanged, and the abnormal driving action can be reasonably and efficiently recognized; specifically, the algorithm solves the problem that the starting point and the end point of the sequence are not easy to determine in the traditional DTW method, and can automatically align the starting point and the end point of the action in the calculation process, so that the recognition rate is improved, and meanwhile, the method for preprocessing the optical flow characteristics reduces the influence of noise on the recognition result and improves the recognition rate.

In one embodiment, the pre-stored characteristic database stores abnormal driving behavior characteristics.

Common undesirable driving behaviors include: (1) watching non-driving directions for a long time, such as watching scenes outside windows on two sides; (2) looking up the gear lever during gear shifting; (3) answering the mobile phone; (4) operating the electronic device for a long time or viewing the meter for a long time, and the like.

One embodiment of the present invention provides a sample, which identifies and detects seven poor driving behaviors, including: the method comprises the steps of head burying, object picking, head turning and rear viewing, left rearview mirror viewing, center rearview mirror viewing, right rearview mirror viewing, mobile phone receiving and head lowering for gear viewing.

And (3) carrying out sample library training on the seven behaviors in advance, and storing the trained samples to form a characteristic database. And then, calculating the similarity of the behaviors in actual driving by using the algorithm of the invention, and selecting the behavior corresponding to the corresponding sequence with the maximum similarity as the recognition result. Fig. 2 shows the results obtained after the 126 action sequence tests of the present example, wherein 120 actions are completely correctly recognized, which accounts for 95.2%, and 6 false recognition are: a countersunk pick identified as looking at the left rearview mirror; turning the head and then seeing the head, and recognizing the head as a rearview mirror in watching; a head-down viewing gear is identified as viewing the right rearview mirror; the rear-view mirror is recognized as a head turning rear-view mirror when the user looks at the right rear-view mirror; receiving the mobile phone, and recognizing as watching the right rearview mirror; the head is turned to look back, and the head is identified as looking at the right rearview mirror. The above false action is analyzed, the actions before and after the false identification are related to each other, and when the action amplitude is too small, the optical flow characteristics are relatively close to each other and can be identified by the false, so the reliability of the method of the invention is relatively high, and the algorithm can realize the identification within 3 seconds on a commonly configured CPU (a 2.5GHz dual-core CPU using a memory 2G in the embodiment).

After the abnormal driving action is recognized, the voice reminding is carried out on the driver to remind the driver to stop the dangerous driving action.

While the present invention has been described in detail with reference to the embodiments shown in the drawings, the present invention is not limited to the above embodiments, and various modifications or alterations can be made by those skilled in the art without departing from the spirit and scope of the claims of the present application.

Claims (5)

1. A fatigue driving detection method based on Gaussian pyramid characteristics is characterized by comprising the following steps:

s1, preprocessing the shot driving image of the driver;

s2, performing down-sampling on the preprocessed image data to obtain multi-resolution Gaussian pyramid characteristics;

and S3, matching the characteristics with a pre-stored characteristic database, and judging whether the driver is fatigue driving.

2. The fatigue driving detecting method based on gaussian pyramid feature of claim 1, wherein said step S1 comprises performing optical flow, gaussian smoothing and normalization calculation once on said image.

3. The method of claim 2, wherein the step S1 specifically comprises performing optical flow calculation on a current frame image f (I, j, t) and a previous frame image f (I, j, t-1) of the original video sequence to obtain an X-direction velocity { U (X, Y), (X, Y) ∈ I } represented by U, a Y-direction velocity { V (X, Y), (X, Y) ∈ I } represented by V, and four separation velocity directions are respectively U (X, Y) to the right₊Leftward U_-Downward V₊And upward V_-And gaussian smoothing is performed on all features:

$\left\{\begin{array}{c}{\hat{U}}_{+}(x,y)=U_{+}(x,y)\⊗g(i,j,0,\mathrm{\δ})\\ {\hat{U}}_{-}(x,y)=U_{-}(x,y)\⊗g(i,j,0,\mathrm{\δ})\\ {\hat{V}}_{+}(x,y)=V_{+}(x,y)\⊗g(i,j,0,\mathrm{\δ})\\ {\hat{V}}_{-}(x,y)=V_{-}(x,y)\⊗g(i,j,0,\mathrm{\δ})\end{array}\right.,$

the above formula is performed and normalized:

$\left\{\begin{array}{cc}{\hat{\hat{U}}}_{+}={\hat{U}}_{+}/\left|{\hat{U}}_{+}\right|,& {\hat{\hat{U}}}_{-}={\hat{U}}_{-}/\left|{\hat{U}}_{-}\right|\\ {\hat{\hat{V}}}_{+}={\hat{V}}_{+}/\left|{\hat{V}}_{+}\right|,& {\hat{\hat{V}}}_{-}={\hat{V}}_{-}/\left|{\hat{V}}_{-}\right|\end{array}\right.,$

and (3) calculating the similarity S (i, j) of the optical flow characteristics obtained from the video sequence:

$S(i,j)=\underset{c=1}{\overset{4}{\Σ}}\underset{x,y\∈I}{\Σ}{a}_c^i(x,y)\·b_c^j(x,y)$

wherein,andrespectively representing the calculation results of the corresponding features of the pixel points in the two groups of optical flow feature sequences with similarity to be compared, c representsAndfour directions after Gaussian smoothing and normalization, i, j are respectively frame numbers in corresponding sequences;

performing convolution kernel of the unit matrix on the S (i, j) to obtain:

$S^T(i,j)=S(i,j)\⊗I\left(T\right).$

4. the fatigue driving detecting method based on Gaussian pyramid characteristics as claimed in claim 1, wherein the step S2 includes,

s21, performing multi-level down-sampling on the video sequence to form a video sequence pyramid with L levels, and performing multi-level down-sampling on an initialized video sequence f with a given resolution of M × N₀(i, j, t), each layer f_l(i, j, t) is calculated recursively by the following formula:

$f_l(i,j,t)=\underset{m=-2}{\overset{2}{\Σ}}\underset{n=-2}{\overset{2}{\Σ}}r(m,n)f_{l-1}(2i+m,2j+n,t),$

wherein f is_l(i, j, t) represents that each frame of image f (i, j) is at the t th frame of pyramid level L (L is more than or equal to 0 and less than or equal to L), r (m, n) is a Gaussian filter,

wherein,

r(m,n)＝r(m)r(n)

r(0)＝a,r(1)＝r(-1)＝1/4,r(2)＝r(-2)＝1/4-2/a；

s22, starting from the level with the lowest resolution, calculating the motion characteristic sequence f of the l-th layer_lCalculating the similarity of the test sample and each sample in the training set in a hierarchy, and selecting K candidates according to K neighbors; f is then calculated at level l-1_l-1(i, j, t) corresponds to the similarity between the motion feature and the K candidates previously selected in the l layer, and the result of K neighbors is used for comparison of the l-2 layer with higher resolution until the l with the highest resolution is equal to 0 layer.

5. The fatigue driving detection method based on the gaussian pyramid feature of claim 1, wherein the pre-stored feature database stores therein abnormal driving behavior features.