CN110008834A - A kind of the steering wheel intervention detection and statistical method of view-based access control model - Google Patents
A kind of the steering wheel intervention detection and statistical method of view-based access control model Download PDFInfo
- Publication number
- CN110008834A CN110008834A CN201910150734.0A CN201910150734A CN110008834A CN 110008834 A CN110008834 A CN 110008834A CN 201910150734 A CN201910150734 A CN 201910150734A CN 110008834 A CN110008834 A CN 110008834A
- Authority
- CN
- China
- Prior art keywords
- intervention
- steering wheel
- frame
- network structure
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
Steering wheel intervention detection and statistical method the invention discloses a kind of view-based access control model, comprising: building detects comprising steering wheel and intervene the Design Integrative Network Structure of Attribute Recognition, and utilizes the sample image training Design Integrative Network Structure；Using the Design Integrative Network Structure, using single-frame images to be detected as the input of Design Integrative Network Structure, using steering wheel as detection target, the intervention attribute information exported according to Design Integrative Network Structure, judge whether steering wheel is intervened on current single-frame images, obtains intervening judging result；Using the intervention judging result in Statistics of Density method processing preset time, the beginning and ending time point and intervene duration that steering wheel is intervened are obtained.The hardware device that this method is relied on is few, is easy to implement, and can eliminate the erroneous detection in detection process and missing inspection, and the accuracy of testing result is high, and can be counted to testing result in order to more intuitively embody.
Description
Technical field
The invention belongs to steering wheels to intervene detection field, and in particular to a kind of steering wheel intervention detection of view-based access control model and system
Meter method.
Background technique
Hand detection and gesture identification are important and grinding with good prospect of human-computer interaction and robot application
Study carefully problem, has important application, in recent years, nothing in sports field, security field, traffic safety driving field
People drive in match the detection method intervened of steering wheel more and more attention has been paid to.
In unmanned match, driver is to intervention number, each time duration of intervening of vehicle steering wheel
Evaluate two important indicators of Unmanned Systems' superiority and inferiority degree.The direct embodiment of driver's intervention steering wheel is hand and steering wheel
It touches, therefore steering wheel intervention detection can be converted into hand and the touching of steering wheel detects.Most of steering wheel touching inspections
Examining system is all detected (such as pressure sensor) using sensor, such as Authorization Notice No. is the special of 105143015 B of CN
Sharp document, then the detection method used is sentences in sensor arrangement to steering wheel according to the unlike signal that sensor generates
Whether the hand of disconnected driver touches steering wheel, and it is auxiliary that this steering wheel touching detection technique is widely used in various Senior Officers
In auxiliary system (ADAS).In fact, touching signal is outer except through special sensor acquisition, can also be received by video image
Collection, with the logic of similar human eye to determine whether touching.Using the method for vision, we only need a camera, without in side
The sensor that various complexity are arranged on disk greatlies simplify steering wheel and cab space design, and low in cost.
Summary of the invention
Steering wheel intervention detection and statistical method the purpose of the present invention is to provide a kind of view-based access control model, this method institute according to
Bad hardware device is few, is easy to implement, and can eliminate the erroneous detection in detection process and missing inspection, and the accuracy of testing result is high, and
Testing result can be counted in order to more intuitively embody.
To achieve the above object, the technical solution used in the present invention are as follows:
A kind of the steering wheel intervention detection and statistical method of view-based access control model, the steering wheel intervention detection of the view-based access control model
With statistical method, comprising:
Building detects comprising steering wheel and intervenes the Design Integrative Network Structure of Attribute Recognition, and utilizes sample image training institute
State Design Integrative Network Structure；
Using the Design Integrative Network Structure, using single-frame images to be detected as the input of Design Integrative Network Structure, with
Steering wheel judges on current single-frame images as detection target according to the intervention attribute information that Design Integrative Network Structure exports
Whether steering wheel is intervened, and obtains intervening judging result；
Using the intervention judging result in Statistics of Density method processing preset time, that steering wheel is intervened is obtained
Only time point and intervention duration.
Preferably, the building detects comprising steering wheel and intervenes the Design Integrative Network Structure of Attribute Recognition, comprising:
Infrastructure network is constructed, which includes 9 convolutional layers and 5 maximum pond layers；
Several candidate windows are arranged in each candidate region on the last layer characteristic pattern of the infrastructure network,
Include the coordinate information of the circumscribed rectangle of steering wheel in each candidate window, whether have object judgement information, target category probability letter
Breath and the intervention attribute information form Design Integrative Network Structure.
Preferably, described utilize the sample image training Design Integrative Network Structure, comprising:
Driving video is obtained, 1 frame is extracted at interval of N frame from the driving video and is saved；
Be labeled to obtain sample image to the image saved, marked content include: the circumscribed rectangle of steering wheel coordinate,
Whether target category and steering wheel are intervened；
Sample image random division is taken to obtain test set and training set；
Using the sample image training Design Integrative Network Structure in training set, until the sample image in test set is surveyed
It tries the Design Integrative Network Structure and reaches preset condition.
Preferably, the infrastructure network since input layer I, successively passes through convolutional layer C1, maximum pond layer
M1, convolutional layer C2, maximum pond layer M2, convolutional layer C3, maximum pond layer M3, convolutional layer C4, maximum pond layer M4, convolutional layer
C5, maximum pond layer M5, convolutional layer C6, convolutional layer C7, convolutional layer C8, convolutional layer C9.
Preferably, the loss function for intervening attribute information calculates, comprising: li=(yp_i-pi)2；Wherein, liIt indicates
Loss function；yp_iIndicate the output valve for intervening attribute information；piIndicate true value；
The gradient for intervening attribute information calculates, comprising:Wherein, δ indicates ladder
Degree；yp_iIndicate the output valve for intervening attribute information, piIndicate true value.
Preferably, the intervention judging result using in Statistics of Density method processing preset time, comprising:
Setting fusion width is N frame, and a frame f is specified in the intervention judging resultt, count frame ftPreceding N frame, rear N frame
And including frame ftTotalframes inside is 2N+1, and counting the intervention frame number in totalframes in intervention states is nt, then calculate
Obtaining the intervention density within the scope of 2N+1 frame is dt, and
If dt>=0.5, then current 2N+1 frame range is defined as intervention states；If dt< 0.5, then current 2N+
1 frame range is defined as non-intervention states.
Preferably, described obtain the beginning and ending time point and intervene duration that steering wheel is intervened, comprising: in the period
dt-1To dt+TIt is interior:
If dt-1< 0.5, and dt>=0.5, then it represents that dtCorresponding time point is the start time point of this intervention states；
If dt+T-1>=0.5, and dt+T< 0.5, and dtTo dt+T-1Between the intervention density that judges be more than or equal to 0.5, then
Indicate dt+TCorresponding time point is the termination time point of this intervention states；
And obtain a length of T when the intervention of this intervention states；Wherein, T indicates the period；dtIndicate current 2N+1 frame range
Interior intervention density；dt-1It indicates relative to dtPrevious 2N+1 frame within the scope of intervention density；dt+TIt indicates relative to dtBy
Intervention density within the scope of the later 2N+1 frame of T duration；dt+T-1It indicates relative to dt+TPrevious 2N+1 frame within the scope of intervention
Density.
Preferably, the value of the N is 5~10.
The steering wheel intervention detection of view-based access control model provided by the invention and statistical method devise a target detection and category
The property integrated end-to-end deep learning network of identification mission, the detection of hand and steering wheel is integrated into a network, is avoided
The real-time detection of hand touching steering wheel is carried out dependent on multiple networks or tactful mode, simplification detection method improves detection efficiency；
And the video sequence based on timing is intervened into Statistics of Density algorithm and is detected applied to steering wheel intervention, eliminate the mistake in detection process
It examines and rising, stopping time point and intervene time duration for generation is intervened in missing inspection, estimation.
Detailed description of the invention
Fig. 1 is that the present invention is based on a kind of embodiment frame diagrams of the steering wheel intervention detection of vision and statistical method；
Fig. 2 is that the present invention is based on a kind of embodiment flow charts of the steering wheel intervention detection of vision and statistical method；
Fig. 3 is that steering wheel intervention of the present invention detects application scenarios schematic diagram；
Fig. 4 is the schematic diagram after image labeling of the present invention；
Fig. 5 is a kind of embodiment schematic diagram of Design Integrative Network Structure of the present invention；
Fig. 6 is that steering wheel intervenes detection effect figure in single-frame images of the present invention；
Fig. 7 is the intervention timing diagram that the present invention ideally intervenes detection output；
Fig. 8 is the intervention timing diagram for intervening detection output under virtual condition of the present invention；
Fig. 9 is that the present invention intervenes density calculating schematic diagram；
Figure 10 is the intervention density map that the intervention timing diagram in Fig. 8 of the present invention obtains after intervening density and calculating；
Figure 11 is another intervention density map of the invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that the described embodiment is only a part of the embodiment of the present invention, instead of all the embodiments.Based on this
Embodiment in invention, every other reality obtained by those of ordinary skill in the art without making creative efforts
Example is applied, shall fall within the protection scope of the present invention.
Unless otherwise defined, all technical and scientific terms used herein and belong to technical field of the invention
The normally understood meaning of technical staff is identical.Term as used herein in the specification of the present invention is intended merely to description tool
The purpose of the embodiment of body is not to be to limit the present invention.
As shown in Figure 1, present embodiments providing the steering wheel intervention detection and statistical method of a kind of view-based access control model, this method
Mainly include two stages:
First stage is the steering wheel intervention detection based on deep learning, is input, using steering wheel as inspection with single-frame images
Target is surveyed, the target frame comprising steering wheel is detected using algorithm of target detection, it is right on End features figure to be multiplexed the target frame
The feature vector answered judges whether steering wheel is intervened on the frame image；
Second stage is the intervention Statistics of Density based on image sequence, is eliminated in the first stage accidentally using Statistics of Density algorithm
Inspection and the interference of missing inspection bring, thus the beginning and ending time point intervened every time and intervention duration.
As shown in Fig. 2, the steering wheel intervention detection of the view-based access control model of the present embodiment and statistical method are broadly divided into following 4
Step.
S1, monitoring system is built in driver's cabin, collect application scenarios video image.
In obtaining driving video, application scenarios video image mainly consists of two parts: first is that unmanned ratio is better than
True driver's monitor video in journey；Second is that monitoring view using the driver that more private cars imitate unmanned heat
Frequently.Why sample data set includes two parts, is the monitoring view because true unmanned match monitor video resource is limited
Angle, environment inside car are excessively single, and a large amount of period steering wheels lead to serious imbalanced training sets all in no intervention states.It is private
The monitor video of family's vehicle simulation mainly makes up the deficiency of real games scene, increases sample diversity.
Firstly, adjusting multiple monitoring visual angles, more vehicles being used to increase angles and environmental diversity；Secondly, private car drives
When the steering wheel most of the time be in by intervention states (steering wheel is not intervened when fraction of time such as dead ship condition), with this
The problem of reducing imbalanced training sets；Again, driver specially imitates various vehicle driving postures and driving habit in simulation process, such as double
Disk, singlehanded large face machine, palm scratching plate, finger hook disk etc. are held, increases the diversity of intervention states sample with this；Finally, mould
Intend various extraneous natural environments, including when passing through tunnel dim illumination, fine day when heavy exposure etc..Application Scenarios-Example is such as
Shown in Fig. 3.
Comprehensive driving video is obtained using scene abundant, in order to the test and training in later period.
S2, image is manually marked, every image labeling content includes: that (such as rectangle is left for the coordinate of the circumscribed rectangle of steering wheel
Coordinate upper, at bottom righthand side angle), whether steering wheel by hand touching (such as 0 indicate to be intervened, 1 indicates not intervened)
Video is disassembled frame by frame, and extracts 1 frame every N frame and saves as sample, N=25 is taken in the present embodiment, and further
It removes wherein because fuzzy, dark, transition such as exposes at the invalid picture caused by reasons, remaining picture is labeled.
It under normal circumstances, at most only include a steering wheel in every picture, the content of mark is that steering wheel target is circumscribed
Whether coordinate (xmin, ymin, xmax, ymax), target category c and steering wheel at the upper left of rectangle, bottom righthand side angle are intervened
Indicate i.Due to only having steering wheel one kind target, therefore c is always 0 (classification id is since 0)；Intervene in flag bit, is indicated using 0
Intervened, 1 indicates not intervened.
When training, target frame coordinate is converted into center point coordinate and the high form (x, y, w, h) of width, and respectively divided by original
For the wide height of figure to normalize, finally formed label form is [c, x, y, w, h, i], picture finally mark after the completion of schematic diagram such as
Shown in Fig. 4, under normal circumstances, label is directly to be labeled on picture with colour, shows label in the present embodiment for clarity, therefore
Label is separated with picture, makes label for labelling above picture.
It includes 45798 pictures that valid data collection is ultimately formed in the present embodiment, wherein selecting 5798 at random as survey
Examination collection, remaining 40000 are used as training set.
S3, building detect comprising steering wheel and intervene the Design Integrative Network Structure of Attribute Recognition, and are instructed using sample image
Practice the Design Integrative Network Structure, obtains the network knot that whether can be touched by hand compared with steering wheel in the every frame image of accurate judgement
Structure.
The main purpose in this stage is to judge whether steering wheel is intervened in single-frame images.It is real from the point of view of technological layer
Matter is two classification problems, has obtained very high accuracy rate currently based on the image classification problem of deep learning, but net of classifying
Network requires to input most of pixel that subject goal in picture occupies entire image, because the principle of sorter network is by several
The feature that entire image is extracted after secondary convolution, pondization operation is classified, if subject goal only occupies entire image seldom one
Partial pixel, feature are easy to lose or flooded by background during the extraction process, it is difficult to obtain ideal classification accuracy.
Unlike usual classification task, the class object of the present embodiment is not jobbie but two kinds of object (hands
And steering wheel) three-dimensional space position relationship, this spatial relation on 2d appearance form multiplicity, this is described
The useful pixel accounting of kind positional relationship is few, and subtlety, which is easy to generate, to be obscured.
In view of the particularity of application scenarios, the present embodiment does not use the sorter network of standard, but combining target detection
Thought detects intervention relationship as an attribute of each target, and with target frame position coordinates, objective degrees of confidence, mesh
The information such as mark classification export simultaneously.Design in this way there are two benefit, one is object detection task be more concerned with it is important in image
Clarification of objective extract, intervene relationship judgement also only in accordance with target frame instruction local feature rather than entire image, so as to
To reduce background interference, Detection accuracy is improved；The second is using the relationship of intervention as an attribute of each target, rather than will
Cascade sort network judges again after local feature is taken off, and sufficiently realizes feature multiplexing in this way, avoid the design of complex network with
Training, while detection process being made to have more real-time.
Common-denominator target has 2 classes, respectively hand and steering wheel in the detection.Detect whether that the intuitive thinking intervened is to examine respectively
The position coordinates of hand and steering wheel are measured, then judge whether there is intervention according to position coordinates, at this moment detection-phase can be deposited
Essence is reduced to the conventional object detection task of hand and steering wheel, however infers that three-dimensional space position relationship is extremely by two-dimensional coordinate
Difficult, therefore scheme is difficult to carry out.Another thinking is, directly detect hand whether intervene steering wheel or steering wheel whether by
Hand intervention goes reasoning to intervene relationship without using artificial rule here, and is handed over to deep neural network and learns simultaneously reasoning automatically.
The present embodiment is further simplified as a detection direction disk using second of thinking, is used as one for whether it is intervened
Attribute exports together with object detection information.
S31, as shown in figure 5, building comprising steering wheel detect and intervene Attribute Recognition Design Integrative Network Structure, comprising:
Infrastructure network is constructed, which includes 9 convolutional layers and 5 maximum pond layers.Specifically, base
Plinth network structure successively passes through convolutional layer C1, maximum pond layer M1, convolutional layer C2, maximum pond layer M2 since input layer I,
Convolutional layer C3, maximum pond layer M3, convolutional layer C4, maximum pond layer M4, convolutional layer C5, maximum pond layer M5, convolutional layer C6, volume
Lamination C7, convolutional layer C8, convolutional layer C9.
The network parameter of convolutional layer uses the form of [k_size, k_size, channels, stride] to indicate, wherein k_
Size is convolution kernel size, and channels is output feature port number, and stride is step-length；The network parameter of pond layer uses
[k_size, k_size, stride] is indicated, wherein k_size is Chi Huahe size, and stride is step-length；The input parameter of each layer
Using [resolution, resolution, channel], wherein resolution is the resolution ratio of image, and channel is logical
Road number.Specifically, infrastructure network is as shown in table 1.
1 infrastructure network of table
| Network layer type | Network parameter | Input parameter |
| Convolutional layer C1 | [3,3,16,1] | [416,416,3] |
| Maximum pond layer M1 | [2,2,2] | [416,416,3] |
| Convolutional layer C2 | [3,3,32,1] | [208,208,16] |
| Maximum pond layer M2 | [2,2,2] | [208,208,16] |
| Convolutional layer C3 | [3,3,64,1] | [104,104,32] |
| Maximum pond layer M3 | [2,2,2] | [104,104,32] |
| Convolutional layer C4 | [3,3,128,1] | [52,52,64] |
| Maximum pond layer M4 | [2,2,2] | [52,52,64] |
| Convolutional layer C5 | [3,3,256,1] | [26,26,128] |
| Maximum pond layer M5 | [2,2,2] | [26,26,128] |
| Convolutional layer C6 | [3,3,512,1] | [13,13,256] |
| Convolutional layer C7 | [3,3,1024,1] | [13,13,256] |
| Convolutional layer C8 | [3,3,1024,1] | [13,13,256] |
| Convolutional layer C9 | [1,1,35,1] | [13,13,256] |
In the tiny-yolov2 algorithm of target detection of standard, each anchor is set on the last layer characteristic pattern of convolution
Whether set 5 anchor box, each anchor box has target to sentence comprising 4 target frame coordinate x, y, w, h information, 1
Determining the other possibility probabilistic information P_c of information P_o, every type, (due to only having steering wheel one kind target here, therefore only 1 is believed
Breath), therefore the feature port number of the target detection the last layer characteristic pattern of standard is 5* (4+1+1)=30.
However, the steering wheel intervention of the present embodiment has detected mostly one and has done unlike the algorithm of target detection of standard
Pre- attribute information P_i.Each candidate region of the present embodiment on the last layer characteristic pattern of the infrastructure network
(anchor) 5 candidate windows (anchor box) are set, and the coordinate comprising the circumscribed rectangle of steering wheel is believed in each candidate window
Whether breath (x, y, w, h information) has object judgement information (P_o), target category probabilistic information (P_c) and the intervention attribute
Information (P_i) forms Design Integrative Network Structure.
Further, will intervene after attribute information P_i is placed in target category probabilistic information P_c, for describing mesh in the frame
Target attribute, therefore the parameter of the last layer convolutional layer C9 of the Design Integrative Network Structure of the present embodiment output is [13,13,35],
I.e. the feature port number of the network the last layer characteristic pattern is 5* (4+1+1+1)=35.
In this way, just form steering wheel intervention detection and intervene Attribute Recognition Design Integrative Network Structure, using one
Change network structure, using single-frame images to be detected as the input of Design Integrative Network Structure, using steering wheel as detection target, root
According to the intervention attribute information that Design Integrative Network Structure exports, judge whether steering wheel is intervened on current single-frame images
It obtains intervening judging result.
Since the attribute of the present embodiment only accounts for whether steering wheel is intervened this, the network structure and standard
The only more intervention attribute positions of target detection network structure, in fact, attribute position can extend, it might even be possible to according to need
It asks and extends to tens of, Shuo Baiwei.Common Attribute Recognition system generally uses target detection network and the cascade side of sorter network
Formula detects the coordinate of targets of interest in image using algorithm of target detection first, is then taken off according to the coordinate and is wrapped in original image
The regional area is finally input in a sorter network by the regional area containing the target, exports objective attribute target attribute.
And target detection and Attribute Recognition Design Integrative Network Structure that the present embodiment proposes, it will test task and Attribute Recognition
Task is integrated into a network, has sufficiently been multiplexed detection network feature information, avoids time-consuming serious, the consumption memory of cascade network
The problems such as serious.The network structure that the present embodiment proposes still keeps the characteristic of end-to-end training, reduces cascade network substep
Trained trouble.The target detection and Attribute Recognition integrated network of the present embodiment design are detected as with being applied to steering wheel intervention
Example is described in detail, but this thought can extend in the application of other intelligent visions, such as target following, target positioning etc.
Deng.
S32, loss function design.
Standard target detection output (x, y, w, h, P_o, P_c) loss function design and gradient calculation with it is existing
It is consistent in technology, such as described in yolov2 paper, no longer repeated herein.It is further illustrated below by specific formula
That adds in the Design Integrative Network Structure of the present embodiment intervenes the loss function design and calculating of attribute information output P_i.
It can see from sample labeling form, the output of P_i is in fact carrying out simple two classification, using linear
It returns to classify, its loss is described using mean square error, loss function calculates, comprising: loss function li=(yp_i-pi)2；
Wherein, liIndicate loss function；yp_iIndicate the output valve for intervening attribute information, piTrue value is indicated, to intervene attribute information
True value.
When gradient calculates, in order to mitigate intervention states and non-intervention states imbalanced training sets problem, less sample is produced
Raw gradient value is multiplied by a coefficient, to increase it to the modified influence power of network weight.Since intervention states sample is less, such as
The markup information of the pre- flag bit of dried fruit is yp_i=0, i.e., steering wheel is in by intervention states in the picture, we are by calculating
Multiplied by 1.1, then gradient calculates to design includes: gradientWherein, δ indicates gradient；yp_iIt indicates
Intervene the output valve of attribute information, piTrue value is indicated, for the true value for intervening attribute information.
The training and test of S33, Design Integrative Network Structure.
In training process, all pictures are input in network by reisize to 416x416 resolution ratio, total the number of iterations
Max_batches=45000, each batch size are 64, and weight decay factor λ=0.0005, backpropagation uses momentum
Method, factor of momentum v=0.9.Learning rate damped manner is piecewise constant decaying, and initial learning rate is learning_rate=
0.0001, it is promoted after iteration 100 times to 0.001, hereafter every iteration 10000 times, learning rate is multiplied by a decay factor 0.1.
Middle training set size is 40000 in the present embodiment, and test set size is 5798.After 40000 training, surveying
It is 99% that steering wheel target detection accuracy of the mean (Average Precision, AP) is measured on examination collection, intervenes Detection accuracy
90%, detecting speed on GTX1070 is 6ms/f.Detection effect schematic diagram as shown in fig. 6, wherein normal expression do not intervene,
I.e. steering wheel is not touched by hand, and automobile is in normal automatic Pilot state；Unnormal indicates to intervene, i.e., steering wheel is touched by hand
It touches, automobile is in pilot steering state.
Calculate whether judgement is current is in intervention states according to the loss function of intervention attribute information: when intervention attribute information
Output valve yp_i> 0.5 indicates not intervene, and output valve indicates that not intervene confidence level higher closer to 1；When intervention attribute
The output valve y of informationp_i≤ 0.5 indicates to intervene, and output valve indicates that the confidence level intervened is higher closer to 0.
S4, the intervention judging result in preset time is handled using Statistics of Density method, obtains steering wheel and is intervened
Beginning and ending time point and intervene duration.
It is successfully identifying after each frame image obtained in driving video, in order to further increase the accuracy of identification,
Simultaneously in order to intuitively be presented in video whether have intervention states, the present embodiment further finds steering wheel in time series
The start time point intervened terminates time point, and the duration intervened every time is calculated with this and counts the intervention of whole section of video
Number.
Ideal recognition result that whether steering wheel is intervened as shown in fig. 7, intervening Detection accuracy is 100%, at this moment when
Sequence analysis is just very simple, need to only judge whether intervention states occur to count on easily and intervene starting point and intervene duration.So
And intervention detection in practice is often as shown in figure 8, being constantly present mistake since steering wheel detects and intervenes detection in present condition
Poor (missing inspection, erroneous detection), even if accuracy rate reaches 99%, average 100 frames can also generate an erroneous detection, calculate to table in time series
Now an erroneous detection will be generated for average every 4s.Obviously directly rely on intervene detection network output as a result, it is difficult to correct parsing
Intervene beginning and ending time point and intervenes duration.
To solve the above-mentioned problems, sentenced in the present embodiment using the intervention in Statistics of Density method processing preset time
Break as a result, as shown in figure 9, specifically including:
Setting fusion width is N frame, and a frame f is specified in intervening judging resultt, count frame ftPreceding N frame, rear N frame and
Including frame ftTotalframes inside is 2N+1, and counting the intervention frame number in totalframes in intervention states is nt, then it is calculated
Intervention density within the scope of 2N+1 frame is dt, and
If dt>=0.5, then current 2N+1 frame range is defined as intervention states；If dt< 0.5, then current 2N+
1 frame range is defined as non-intervention states.
As shown in Figure 10, the intervention density map obtain after intervention density calculates for Fig. 8, it can be seen that detection
Although there are a large amount of missing inspections and erroneous detections for the intervention timing of network output, after intervening density and calculating, remains to accurate statistics and do
Pre- number.The intervention states presented in figure and the boundary of non-intervention states are obvious, can be obtained by the intervention density map more clear
The case where whether clear accurate steering wheel is intervened.
At the judgement in the above described manner within the scope of each 2N+1 frame whether intervention states after, steering wheel can be obtained and done
Pre- beginning and ending time point and intervention duration specifically includes as shown in figure 11: in period dt-1To dt+TIt is interior:
If dt-1< 0.5, and dt>=0.5, then it represents that dtCorresponding time point is the start time point of this intervention states；
If dt+T-1>=0.5, and dt+T< 0.5, and dtTo dt+T-1Between the intervention density that judges be more than or equal to 0.5, then
Indicate dt+TCorresponding time point is the termination time point of this intervention states；
And obtain a length of T when the intervention of this intervention states；Wherein, T indicates period and T > 0；dtIndicate current 2N+1
Intervention density within the scope of frame；dt-1It indicates relative to dtPrevious 2N+1 frame within the scope of intervention density；dt+TIndicate opposite
In dtThe intervention density within the scope of 2N+1 frame after T duration；dt+T-1It indicates relative to dt+TPrevious 2N+1 frame range
Interior intervention density.
It is noted that taking N frame before and after sequence due to intervening Statistics of Density intervenes testing result, when the start-stop intervened
Between put all can than really put delay N frame.Between 5~10, that is, between 0.2s to 0.4s, which exists usual N value
It is acceptable in the statistics of the present embodiment.
Each technical characteristic of embodiment described above can be combined arbitrarily, for simplicity of description, not to above-mentioned reality
It applies all possible combination of each technical characteristic in example to be all described, as long as however, the combination of these technical characteristics is not present
Contradiction all should be considered as described in this specification.
The embodiments described above only express several embodiments of the present invention, and the description thereof is more specific and detailed, but simultaneously
It cannot therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
It says, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to protection of the invention
Range.Therefore, the scope of protection of the patent of the invention shall be subject to the appended claims.
Claims (8)
1. a kind of steering wheel intervention detection of view-based access control model and statistical method, which is characterized in that the direction of the view-based access control model
Disk intervention detection and statistical method, comprising:
Building detects comprising steering wheel and intervenes the Design Integrative Network Structure of Attribute Recognition, and utilizes sample image training described one
Body network structure；
Using the Design Integrative Network Structure, using single-frame images to be detected as the input of Design Integrative Network Structure, with direction
Disk judges direction on current single-frame images according to the intervention attribute information that Design Integrative Network Structure exports as detection target
Whether disk is intervened, and obtains intervening judging result；
Using the intervention judging result in Statistics of Density method processing preset time, when obtaining the start-stop that steering wheel is intervened
Between put and intervene duration.
2. the steering wheel intervention detection of view-based access control model as described in claim 1 and statistical method, which is characterized in that the building
The Design Integrative Network Structure of Attribute Recognition is detected and intervened comprising steering wheel, comprising:
Infrastructure network is constructed, which includes 9 convolutional layers and 5 maximum pond layers；
Several candidate windows are arranged in each candidate region on the last layer characteristic pattern of the infrastructure network, each
In candidate window comprising the circumscribed rectangle of steering wheel coordinate information, whether have object judgement information, target category probabilistic information with
And the intervention attribute information, form Design Integrative Network Structure.
3. the steering wheel intervention detection of view-based access control model as claimed in claim 2 and statistical method, which is characterized in that the utilization
The sample image training Design Integrative Network Structure, comprising:
Driving video is obtained, 1 frame is extracted at interval of N frame from the driving video and is saved；
The image saved is labeled to obtain sample image, marked content includes: the coordinate of the circumscribed rectangle of steering wheel, target
Whether classification and steering wheel are intervened；
Sample image random division is taken to obtain test set and training set；
Using the sample image training Design Integrative Network Structure in training set, until the sample image test in test set should
Design Integrative Network Structure reaches preset condition.
4. the steering wheel intervention detection of view-based access control model as claimed in claim 2 and statistical method, which is characterized in that the base
Plinth network structure successively passes through convolutional layer C1, maximum pond layer M1, convolutional layer C2, maximum pond layer M2 since input layer I,
Convolutional layer C3, maximum pond layer M3, convolutional layer C4, maximum pond layer M4, convolutional layer C5, maximum pond layer M5, convolutional layer C6, volume
Lamination C7, convolutional layer C8, convolutional layer C9.
5. the steering wheel intervention detection of view-based access control model as described in claim 1 and statistical method, which is characterized in that the intervention
The loss function of attribute information calculates, comprising: li=(yp_i-pi)2；Wherein, liIndicate loss function；yp_iIt indicates to intervene attribute
The output valve of information；piIndicate true value；
The gradient for intervening attribute information calculates, comprising:Wherein, δ indicates gradient；yp_i
Indicate the output valve for intervening attribute information, piIndicate true value.
6. the steering wheel intervention detection of view-based access control model as described in claim 1 and statistical method, which is characterized in that the use
Statistics of Density method handles the intervention judging result in preset time, comprising:
Setting fusion width is N frame, and a frame f is specified in the intervention judging resultt, count frame ftPreceding N frame, rear N frame and
Including frame ftTotalframes inside is 2N+1, and counting the intervention frame number in totalframes in intervention states is nt, then it is calculated
Intervention density within the scope of 2N+1 frame is dt, and
If dt>=0.5, then current 2N+1 frame range is defined as intervention states；If dt< 0.5, then current 2N+1 frame
Range is defined as non-intervention states.
7. the steering wheel intervention detection of view-based access control model as claimed in claim 6 and statistical method, which is characterized in that described to obtain
The beginning and ending time point and intervene duration that steering wheel is intervened, comprising: in period dt-1To dt+TIt is interior:
If dt-1< 0.5, and dt>=0.5, then it represents that dtCorresponding time point is the start time point of this intervention states；
If dt+T-1>=0.5, and dt+T< 0.5, and dtTo dt+T-1Between the intervention density that judges be more than or equal to 0.5, then it represents that
dt+TCorresponding time point is the termination time point of this intervention states；
And obtain a length of T when the intervention of this intervention states；Wherein, T indicates the period；dtIt indicates within the scope of current 2N+1 frame
Intervene density；dt-1It indicates relative to dtPrevious 2N+1 frame within the scope of intervention density；dt+TIt indicates relative to dtWhen by T
Intervention density within the scope of long later 2N+1 frame；dt+T-1It indicates relative to dt+TPrevious 2N+1 frame within the scope of intervention it is close
Degree.
8. the steering wheel intervention detection of view-based access control model as claimed in claim 7 and statistical method, which is characterized in that the N's
Value is 5~10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910150734.0A CN110008834B (en) | 2019-02-28 | 2019-02-28 | Steering wheel intervention detection and statistics method based on vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910150734.0A CN110008834B (en) | 2019-02-28 | 2019-02-28 | Steering wheel intervention detection and statistics method based on vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008834A true CN110008834A (en) | 2019-07-12 |
| CN110008834B CN110008834B (en) | 2021-04-06 |
Family
ID=67166379
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910150734.0A Active CN110008834B (en) | 2019-02-28 | 2019-02-28 | Steering wheel intervention detection and statistics method based on vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008834B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310841A (en) * | 2020-02-24 | 2020-06-19 | 中南大学湘雅医院 | Medical image classification method, apparatus, device, computer device and storage medium |
| CN114360321A (en) * | 2021-11-09 | 2022-04-15 | 易显智能科技有限责任公司 | Hand action sensing system, training system and training method for motor vehicle driver |
| CN118107605A (en) * | 2024-04-30 | 2024-05-31 | 润芯微科技(江苏)有限公司 | Vehicle control method and system based on steering wheel gesture interaction |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| CN102547139A (en) * | 2010-12-30 | 2012-07-04 | 北京新岸线网络技术有限公司 | Method for splitting news video program, and method and system for cataloging news videos |
| CN104078039A (en) * | 2013-03-27 | 2014-10-01 | 广东工业大学 | Voice recognition system of domestic service robot on basis of hidden Markov model |
| US20140292692A1 (en) * | 2013-03-27 | 2014-10-02 | Honda Motor Co., Ltd. | Input apparatus, input method, and input program |
| CN104092988A (en) * | 2014-07-10 | 2014-10-08 | 深圳市中控生物识别技术有限公司 | Method, device and system for managing passenger flow in public place |
| CN104207791A (en) * | 2014-08-26 | 2014-12-17 | 江南大学 | Fatigue driving detection method |
| CN104228845A (en) * | 2013-06-13 | 2014-12-24 | 福特全球技术公司 | Hand/steering wheel contact detection using observer |
| CN105488957A (en) * | 2015-12-15 | 2016-04-13 | 小米科技有限责任公司 | Fatigue driving detection method and apparatus |
| CN105513354A (en) * | 2015-12-22 | 2016-04-20 | 电子科技大学 | Video-based urban road traffic jam detecting system |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
| CN106845344A (en) * | 2016-12-15 | 2017-06-13 | 重庆凯泽科技股份有限公司 | Demographics' method and device |
| CN107274678A (en) * | 2017-08-14 | 2017-10-20 | 河北工业大学 | A kind of night vehicle flowrate and model recognizing method based on Kinect |
| CN107479044A (en) * | 2017-08-23 | 2017-12-15 | 西安电子工程研究所 | Based on an adaptive track initiation method of mark density real-time statistics |
| CN107944341A (en) * | 2017-10-27 | 2018-04-20 | 荆门程远电子科技有限公司 | Driver based on traffic monitoring image does not fasten the safety belt automatic checkout system |
| CN108399388A (en) * | 2018-02-28 | 2018-08-14 | 福州大学 | A kind of middle-high density crowd quantity statistics method |
| CN108647617A (en) * | 2018-05-02 | 2018-10-12 | 深圳市唯特视科技有限公司 | A kind of positioning of driver's hand and grasping analysis method based on convolutional neural networks |
| CN109151501A (en) * | 2018-10-09 | 2019-01-04 | 北京周同科技有限公司 | A kind of video key frame extracting method, device, terminal device and storage medium |
-
2019
- 2019-02-28 CN CN201910150734.0A patent/CN110008834B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102547139A (en) * | 2010-12-30 | 2012-07-04 | 北京新岸线网络技术有限公司 | Method for splitting news video program, and method and system for cataloging news videos |
| CN102324016A (en) * | 2011-05-27 | 2012-01-18 | 郝红卫 | Statistical method for high-density crowd flow |
| CN102263937A (en) * | 2011-07-26 | 2011-11-30 | 华南理工大学 | Driver's driving behavior monitoring device and monitoring method based on video detection |
| CN102289660A (en) * | 2011-07-26 | 2011-12-21 | 华南理工大学 | Method for detecting illegal driving behavior based on hand gesture tracking |
| CN104078039A (en) * | 2013-03-27 | 2014-10-01 | 广东工业大学 | Voice recognition system of domestic service robot on basis of hidden Markov model |
| US20140292692A1 (en) * | 2013-03-27 | 2014-10-02 | Honda Motor Co., Ltd. | Input apparatus, input method, and input program |
| CN104228845A (en) * | 2013-06-13 | 2014-12-24 | 福特全球技术公司 | Hand/steering wheel contact detection using observer |
| CN104092988A (en) * | 2014-07-10 | 2014-10-08 | 深圳市中控生物识别技术有限公司 | Method, device and system for managing passenger flow in public place |
| CN104207791A (en) * | 2014-08-26 | 2014-12-17 | 江南大学 | Fatigue driving detection method |
| CN105488957A (en) * | 2015-12-15 | 2016-04-13 | 小米科技有限责任公司 | Fatigue driving detection method and apparatus |
| CN105513354A (en) * | 2015-12-22 | 2016-04-20 | 电子科技大学 | Video-based urban road traffic jam detecting system |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
| CN106845344A (en) * | 2016-12-15 | 2017-06-13 | 重庆凯泽科技股份有限公司 | Demographics' method and device |
| CN107274678A (en) * | 2017-08-14 | 2017-10-20 | 河北工业大学 | A kind of night vehicle flowrate and model recognizing method based on Kinect |
| CN107479044A (en) * | 2017-08-23 | 2017-12-15 | 西安电子工程研究所 | Based on an adaptive track initiation method of mark density real-time statistics |
| CN107944341A (en) * | 2017-10-27 | 2018-04-20 | 荆门程远电子科技有限公司 | Driver based on traffic monitoring image does not fasten the safety belt automatic checkout system |
| CN108399388A (en) * | 2018-02-28 | 2018-08-14 | 福州大学 | A kind of middle-high density crowd quantity statistics method |
| CN108647617A (en) * | 2018-05-02 | 2018-10-12 | 深圳市唯特视科技有限公司 | A kind of positioning of driver's hand and grasping analysis method based on convolutional neural networks |
| CN109151501A (en) * | 2018-10-09 | 2019-01-04 | 北京周同科技有限公司 | A kind of video key frame extracting method, device, terminal device and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310841A (en) * | 2020-02-24 | 2020-06-19 | 中南大学湘雅医院 | Medical image classification method, apparatus, device, computer device and storage medium |
| CN111310841B (en) * | 2020-02-24 | 2023-06-20 | 中南大学湘雅医院 | Medical image classification method, medical image classification device, medical image classification apparatus, medical image classification computer device, and medical image classification storage medium |
| CN114360321A (en) * | 2021-11-09 | 2022-04-15 | 易显智能科技有限责任公司 | Hand action sensing system, training system and training method for motor vehicle driver |
| CN118107605A (en) * | 2024-04-30 | 2024-05-31 | 润芯微科技(江苏)有限公司 | Vehicle control method and system based on steering wheel gesture interaction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008834B (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105069472B (en) | A kind of vehicle checking method adaptive based on convolutional neural networks | |
| CN105512720B (en) | A kind of public transit vehicle passenger flow statistics method and system | |
| CN109447033A (en) | Vehicle front obstacle detection method based on YOLO | |
| CN109508715A (en) | A kind of License Plate and recognition methods based on deep learning | |
| CN103886308B (en) | A kind of pedestrian detection method of use converging channels feature and soft cascade grader | |
| CN110097109A (en) | A kind of road environment obstacle detection system and method based on deep learning | |
| CN105335966B (en) | Multiscale morphology image division method based on local homogeney index | |
| CN110018524A (en) | A kind of X-ray safety check contraband recognition methods of view-based access control model-attribute | |
| CN109800629A (en) | A kind of Remote Sensing Target detection method based on convolutional neural networks | |
| CN104123549B (en) | Eye positioning method for real-time monitoring of fatigue driving | |
| CN109033950A (en) | Vehicle based on multiple features fusion cascade deep model, which is disobeyed, stops detection method | |
| CN106682586A (en) | Method for real-time lane line detection based on vision under complex lighting conditions | |
| CN103049751A (en) | Improved weighting region matching high-altitude video pedestrian recognizing method | |
| CN106203274A (en) | Pedestrian's real-time detecting system and method in a kind of video monitoring | |
| CN110008834A (en) | A kind of the steering wheel intervention detection and statistical method of view-based access control model | |
| CN106355602A (en) | Multi-target locating and tracking video monitoring method | |
| CN109766936A (en) | Image change detection method based on information transmitting and attention mechanism | |
| CN111553201A (en) | Traffic light detection method based on YOLOv3 optimization algorithm | |
| CN109190444A (en) | A kind of implementation method of the lane in which the drivers should pay fees vehicle feature recognition system based on video | |
| CN109543632A (en) | A kind of deep layer network pedestrian detection method based on the guidance of shallow-layer Fusion Features | |
| CN110009648A (en) | Trackside image Method of Vehicle Segmentation based on depth Fusion Features convolutional neural networks | |
| CN106128121A (en) | Vehicle queue length fast algorithm of detecting based on Local Features Analysis | |
| CN106127812A (en) | A kind of passenger flow statistical method of non-gate area, passenger station based on video monitoring | |
| CN109948690A (en) | A kind of high-speed rail scene perception method based on deep learning and structural information | |
| CN106373146A (en) | Target tracking method based on fuzzy learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |