CN109886150A - A kind of driving behavior recognition methods based on Kinect video camera - Google Patents
A kind of driving behavior recognition methods based on Kinect video camera Download PDFInfo
- Publication number
- CN109886150A CN109886150A CN201910088268.8A CN201910088268A CN109886150A CN 109886150 A CN109886150 A CN 109886150A CN 201910088268 A CN201910088268 A CN 201910088268A CN 109886150 A CN109886150 A CN 109886150A
- Authority
- CN
- China
- Prior art keywords
- driver
- driving behavior
- skeleton
- recognition methods
- video camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
Abstract
The driving behavior recognition methods based on Kinect video camera that the present invention provides a kind of, comprising: skeleton tracking is carried out to foreground target driver using the video frame of Kinect video camera acquisition, and positions the skeleton data of driver；According to the skeleton data of driver, obtains multiple body joint point coordinate values of the driver in each frame of video frame and it is normalized；Then feature extraction is carried out to body joint point coordinate value, generates multiple single frames skeleton character vectors；Multiple single frames skeleton character vectors are finally sent into preset deep learning model and identify that judgement is criterion behavior or unlawful practice.Compared with the prior art, the present invention can realize the identification of driver's operation behavior according to multiple body joint point coordinates of driver's skeleton and corresponding skeleton character vector, reach the prevention effect of alarm or early warning, and it when being extracted based on Kinect video camera to bone information, is not influenced by light and precision is higher, more reliable.
Description
Technical field
The present invention relates to computer vision field and field of video image processing, more particularly to a kind of Kinect that is based on to image
The driving behavior recognition methods of machine.
Background technique
Currently, the unlawful practice of harbour driver repeatedly directly results in harbour major accident, for example, one of driver is disobeyed
Rule behavior is often possible to that the entire production line is caused to stop production, and will cause equipment damage and casualties when serious.It is found by investigation,
The unlawful practice of driver is concentrated mainly on, and operator is absent minded, in driving procedure occur play mobile phone, receive calls or
Both hands are detached from the violation driving behavior of operating stick for a long time.In view of this, the current driving behavior to driver is known in real time
Other and early warning has realistic price and important function to the security reliability for promoting harbour service.
In the prior art, though harbour there are monitoring system come supervise driver according to prescribed form carry out safe driving,
But it also needs manually to monitor.However this also brings along another question, that is, absent minded, the appearance of monitoring personnel
Monitor video when violation driving behavior cannot be monitored personnel when capturing in time, still cannot effectively prevent the hair of accident
Raw, it is even more impossible to prejudge to dangerous driving behavior, it is therefore desirable to a kind of driver's operation behavior identification without human intervention
Method.
On the other hand, Activity recognition is related to computer identification and the fields such as pattern-recognition, is one important and have challenge again
The project of property.Activity recognition is based on video or image, extracts the characteristic information of effective expression behavior, and further to feature into
Row identification is to obtain the concrete behavior of human body.Traditional recognition methods is that feature extraction is carried out to picture to identify,
Such as histogram of gradients (HOG), light stream histogram (HOF), scale invariant feature variation (SIFT), but often by complicated ring
The influence in border, such as illumination interference.It can be seen from the above, existing Activity recognition is primarily present both sides problem: first is that how
It chooses strong feature human body behavior is described, second is that how to model dynamic time, neither loss is effectively believed
Breath can also maximally utilise information simultaneously and be identified.
In view of this, a kind of driving behavior recognition methods for driver how is designed, to overcome in the prior art
Drawbacks described above or deficiency, be that related technical personnel need a project solving in the industry.
Summary of the invention
Driving behavior for prior art drawbacks described above existing when identifying and monitoring, the present invention provides a kind of bases
In the driving behavior recognition methods of Kinect video camera.
According to one aspect of the present invention, a kind of driving behavior recognition methods based on Kinect video camera is provided, is wrapped
Include following steps:
A) skeleton tracking is carried out to foreground target driver using the video frame of Kinect video camera acquisition, and positions driving
The skeleton data of member；
B) according to the skeleton data of the driver, multiple joints of the driver in each frame of the video frame are obtained
Point coordinate value is simultaneously normalized it, wherein the internal relations between the body joint point coordinate value are for characterizing described drive
The driving behavior for the person of sailing；
C) feature extraction is carried out to the body joint point coordinate value, generates multiple single frames skeleton character vectors；And
D) the multiple single frames skeleton character vector is sent into preset deep learning model to identify, is tied according to identification
Fruit judges that current driving condition is standard driving behavior or violation driving behavior.
In an embodiment wherein, the deep learning model includes normal data and violation data, the normal data pair
The standard driving behavior of driver described in Ying Yu, the violation data correspond to answering the call, playing mobile phone or both hands for the driver
It is detached from the violation driving behavior of operating stick for a long time.
In an embodiment wherein, after step c), this method further include: according to the frame sequential of the video frame, according to
Secondary input single frames skeleton character vector to the deep learning model is trained, and updates the deep learning model.
In an embodiment wherein, it is complete that the skeleton data of the driver is that the Kinect video camera navigates to driver
25 artis and corresponding body joint point coordinate value of body.
In an embodiment wherein, for only having the driver of sitting posture, the single frames skeleton character vector is constant by stablizing
Several body joint point coordinates and these artis between constituted angle composition.
In an embodiment wherein, after step d), this method further include: one preset time interval of delay returns
Above-mentioned steps a), and recycle the driving behavior identification process for executing above-mentioned steps a) to above-mentioned steps d).
In an embodiment wherein, above-mentioned steps d) further include: if recognition result is violation driving behavior, issue acousto-optic
Alarm or mark indication signal.
Using the driving behavior recognition methods of the invention based on Kinect video camera, adopted first with Kinect video camera
The video frame of collection carries out skeleton tracking to foreground target driver, and positions the skeleton data of driver；Then according to driver
Skeleton data, obtain multiple body joint point coordinate values of the driver in each frame of video frame and it be normalized place
Reason；Then feature extraction is carried out to body joint point coordinate value, generates multiple single frames skeleton character vectors；Finally by multiple single frames bones
Feature vector is sent into preset deep learning model and is identified, judges that current driving condition is that standard drives according to recognition result
Behavior or violation driving behavior.Compared with the prior art, the present invention can according to multiple body joint point coordinates of driver's skeleton and
Corresponding skeleton character vector realizes the identification of driver's operation behavior, reaches the prevention effect of alarm or early warning, and base
When Kinect video camera extracts bone information, is not influenced by light and precision is higher, more reliable.
Detailed description of the invention
Reader is after having read a specific embodiment of the invention referring to attached drawing, it will more clearly understands of the invention
Various aspects.Wherein,
Fig. 1 shows the flow diagram of the driving behavior recognition methods of the invention based on Kinect video camera.
25 joints of the extracted human body of Kinect video camera used in driving behavior recognition methods Fig. 2 shows Fig. 1
The schematic diagram of point.
Specific embodiment
In order to keep techniques disclosed in this application content more detailed with it is complete, can refer to attached drawing and of the invention following
Various specific embodiments, identical label represents the same or similar component in attached drawing.However, those skilled in the art
It should be appreciated that embodiment provided hereinafter is not intended to limit the invention covered range.In addition, attached drawing is used only for
It is schematically illustrated, and is drawn not according to its full size.
With reference to the accompanying drawings, the specific embodiment of various aspects of the present invention is described in further detail.
Fig. 1 shows the flow diagram of the driving behavior recognition methods of the invention based on Kinect video camera.Fig. 2 shows figures
The schematic diagram of extracted 25 artis of human body of Kinect video camera used in 1 driving behavior recognition methods.
Referring to Figures 1 and 2, in this embodiment, the driving behavior recognition methods of the invention based on Kinect video camera
Including step S101~S107.Compared with the prior art, the present invention by above-mentioned steps S101, step S103, step S105 and
Step S107 realizes that driver operates according to multiple body joint point coordinates of driver's skeleton and corresponding skeleton character vector
The identification of behavior, reaches the prevention effect of alarm or early warning, and when being extracted based on Kinect video camera to bone information, not by
The influence of light and precision is higher, more reliable.
In step s101, skeleton is carried out to foreground target driver using the video frame of Kinect video camera acquisition to chase after
Track, and position the skeleton data of driver.Preferably, the skeleton data of driver is that navigate to driver complete for Kinect video camera
25 artis and corresponding body joint point coordinate value of body.As shown in Fig. 2, the position of 25 artis of skeleton and meaning point
Not are as follows:
The head Head-, Neck- neck, SpineShoulder- backbone shoulder, SpineMid- backbone center,
SpineBase- backbone bottom, the left shoulder of ShoulderLeft-, the left elbow of ElbowLeft-, the left wrist of WristLeft-, HandLeft-
Left hand, the left finger tip of HandTipLeft-, the left thumb of ThumbLeft-, the left stern of HipLeft-, the right shoulder of ShoulderRight-,
The right elbow of ElbowRight-, the right wrist of WristRight-, the HandRight- right hand, the right finger tip of HandTipRight-,
The right thumb of ThumbRight-, the right stern of HipRight-, the left knee of KneeLeft-, AnkleLeft- left ankle, FootLeft- left foot,
The right knee of KneeRight-, AnkleRight- right ankle, FootRight- right crus of diaphragm.
In step s 103, according to the skeleton data of driver, the multiple of the driver in each frame of video frame are obtained
Body joint point coordinate value is simultaneously normalized it, and wherein the internal relations between body joint point coordinate value are for characterizing described drive
The driving behavior for the person of sailing.
In step s105, feature extraction is carried out to body joint point coordinate value, generates multiple single frames skeleton character vectors.One
In a little embodiments, for only having the driver of sitting posture, single frames skeleton character vector by stablize constant several body joint point coordinates with
And the angle composition constituted between these artis.For example, since the driving behavior of driver is sitting posture, then it is main to extract
The joint information of the upper part of the body of skeleton profile, such as SpineBase (backbone bottom), Head (head), ElbowLeft are (left
Elbow), WristLeft (left wrist), ElbowRight (right elbow), WristRight (right wrist) several artis coordinate.
In step s 107, multiple single frames skeleton character vectors are sent into preset deep learning model to identify, root
Judge that current driving condition is standard driving behavior or violation driving behavior according to recognition result.Preferably, the deep learning model
Including normal data and violation data, which corresponds to the standard driving behavior of driver, which corresponds to
Driver's answers the call, plays the violation driving behavior that mobile phone or both hands are detached from operating stick for a long time.
According to some embodiments, after above-mentioned steps S105, which further includes according to video frame
Frame sequential, sequentially input single frames skeleton character vector to deep learning model and be trained, and update deep learning model.
According to some embodiments, after above-mentioned steps S107, which further includes that delay one is default
Time interval, return above-mentioned steps S101, and recycle execute above-mentioned steps S101 to above-mentioned steps S107 driving behavior know
Other process.In addition, in step s 107, if recognition result is violation driving behavior, issuing sound-light alarm or mark instruction
Signal.
For the accuracy rate for examining driving behavior recognition methods of the invention, inventor is using Kinect video camera acquisition view
It frequently, altogether include 20 frames, totally 3200 driving behavior sequences, wherein randomly select 2400 driving behavior sequences and establish depth
Model is practised, remaining 800 driving behavior sequences are tested.Emulation experiment statistics indicate that, the recognition accuracy of standard gestures
It is 100%, the recognition accuracy for playing the unlawful practice of mobile phone is 98.0%, and the recognition accuracy for the unlawful practice answered the call is
98.0%, the recognition accuracy that both hands are detached from the unlawful practice of operating stick for a long time is 96.0%.
Using the driving behavior recognition methods of the invention based on Kinect video camera, adopted first with Kinect video camera
The video frame of collection carries out skeleton tracking to foreground target driver, and positions the skeleton data of driver；Then according to driver
Skeleton data, obtain multiple body joint point coordinate values of the driver in each frame of video frame and it be normalized place
Reason；Then feature extraction is carried out to body joint point coordinate value, generates multiple single frames skeleton character vectors；Finally by multiple single frames bones
Feature vector is sent into preset deep learning model and is identified, judges that current driving condition is that standard drives according to recognition result
Behavior or violation driving behavior.Compared with the prior art, the present invention can according to multiple body joint point coordinates of driver's skeleton and
Corresponding skeleton character vector realizes the identification of driver's operation behavior, reaches the prevention effect of alarm or early warning, and base
When Kinect video camera extracts bone information, is not influenced by light and precision is higher, more reliable.
Above, a specific embodiment of the invention is described with reference to the accompanying drawings.But those skilled in the art
It is understood that without departing from the spirit and scope of the present invention, can also make to a specific embodiment of the invention each
Kind change and replacement.These changes and replacement are all fallen within the scope of the invention as defined in the claims.
Claims (7)
1. a kind of driving behavior recognition methods based on Kinect video camera, which is characterized in that the driving behavior recognition methods packet
Include following steps:
A) skeleton tracking is carried out to foreground target driver using the video frame of Kinect video camera acquisition, and positions driver's
Skeleton data；
B) according to the skeleton data of the driver, the multiple artis for obtaining the driver in each frame of the video frame are sat
Scale value is simultaneously normalized it, wherein the internal relations between the body joint point coordinate value are for characterizing the driver
Driving behavior；
C) feature extraction is carried out to the body joint point coordinate value, generates multiple single frames skeleton character vectors；And
D) the multiple single frames skeleton character vector is sent into preset deep learning model to identify, is sentenced according to recognition result
The current driving condition that breaks is standard driving behavior or violation driving behavior.
2. driving behavior recognition methods according to claim 1, which is characterized in that the deep learning model includes standard
Data and violation data, the normal data correspond to the standard driving behavior of the driver, which corresponds to described
Driver's answers the call, plays the violation driving behavior that mobile phone or both hands are detached from operating stick for a long time.
3. driving behavior recognition methods according to claim 1, which is characterized in that after step c), the driving behavior
Recognition methods further include:
According to the frame sequential of the video frame, sequentially inputs single frames skeleton character vector to the deep learning model and instructed
Practice, and updates the deep learning model.
4. driving behavior recognition methods according to claim 1, which is characterized in that the skeleton data of the driver is institute
State 25 artis and corresponding body joint point coordinate value that Kinect video camera navigates to driver's whole body.
5. driving behavior recognition methods according to claim 1, which is characterized in that for only having the driver of sitting posture, institute
Single frames skeleton character vector is stated by stablizing the angle group constituted between constant several body joint point coordinates and these artis
At.
6. driving behavior recognition methods according to claim 1, which is characterized in that after step d), the driving behavior
Recognition methods further include:
Postpone a preset time interval, return to above-mentioned steps a), and recycles and execute above-mentioned steps a) driving to above-mentioned steps d)
Sail Activity recognition process.
7. driving behavior recognition methods according to claim 6, which is characterized in that above-mentioned steps d) further include:
If recognition result is violation driving behavior, sound-light alarm or mark indication signal are issued.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910088268.8A CN109886150A (en) | 2019-01-29 | 2019-01-29 | A kind of driving behavior recognition methods based on Kinect video camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910088268.8A CN109886150A (en) | 2019-01-29 | 2019-01-29 | A kind of driving behavior recognition methods based on Kinect video camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109886150A true CN109886150A (en) | 2019-06-14 |
Family
ID=66927275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910088268.8A Pending CN109886150A (en) | 2019-01-29 | 2019-01-29 | A kind of driving behavior recognition methods based on Kinect video camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886150A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110490143A (en) * | 2019-08-21 | 2019-11-22 | 西安工程大学 | A kind of action identification method of adaptive model |
| CN112046662A (en) * | 2020-08-13 | 2020-12-08 | 哈尔滨工业大学（深圳） | Walking-replacing following robot and walking-replacing following method thereof |
| CN112307846A (en) * | 2019-08-01 | 2021-02-02 | 北京新联铁集团股份有限公司 | Analysis method for violation of crew service |
| CN112446352A (en) * | 2020-12-14 | 2021-03-05 | 深圳地平线机器人科技有限公司 | Behavior recognition method, behavior recognition device, behavior recognition medium, and electronic device |
| WO2022142786A1 (en) * | 2020-12-30 | 2022-07-07 | 中兴通讯股份有限公司 | Driving behavior recognition method, and device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469073A (en) * | 2015-12-16 | 2016-04-06 | 安徽创世科技有限公司 | Kinect-based call making and answering monitoring method of driver |
| CN107657236A (en) * | 2017-09-29 | 2018-02-02 | 厦门知晓物联技术服务有限公司 | Vehicle security drive method for early warning and vehicle-mounted early warning system |
| CN108446678A (en) * | 2018-05-07 | 2018-08-24 | 同济大学 | A kind of dangerous driving behavior recognition methods based on skeleton character |
| CN108830215A (en) * | 2018-06-14 | 2018-11-16 | 南京理工大学 | Hazardous act recognition methods based on personnel's framework information |
| CN108921044A (en) * | 2018-06-11 | 2018-11-30 | 大连大学 | Driver's decision feature extracting method based on depth convolutional neural networks |
-
2019
- 2019-01-29 CN CN201910088268.8A patent/CN109886150A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469073A (en) * | 2015-12-16 | 2016-04-06 | 安徽创世科技有限公司 | Kinect-based call making and answering monitoring method of driver |
| CN107657236A (en) * | 2017-09-29 | 2018-02-02 | 厦门知晓物联技术服务有限公司 | Vehicle security drive method for early warning and vehicle-mounted early warning system |
| CN108446678A (en) * | 2018-05-07 | 2018-08-24 | 同济大学 | A kind of dangerous driving behavior recognition methods based on skeleton character |
| CN108921044A (en) * | 2018-06-11 | 2018-11-30 | 大连大学 | Driver's decision feature extracting method based on depth convolutional neural networks |
| CN108830215A (en) * | 2018-06-14 | 2018-11-16 | 南京理工大学 | Hazardous act recognition methods based on personnel's framework information |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112307846A (en) * | 2019-08-01 | 2021-02-02 | 北京新联铁集团股份有限公司 | Analysis method for violation of crew service |
| CN110490143A (en) * | 2019-08-21 | 2019-11-22 | 西安工程大学 | A kind of action identification method of adaptive model |
| CN112046662A (en) * | 2020-08-13 | 2020-12-08 | 哈尔滨工业大学（深圳） | Walking-replacing following robot and walking-replacing following method thereof |
| CN112046662B (en) * | 2020-08-13 | 2023-01-17 | 哈尔滨工业大学（深圳） | Walking-replacing following robot and walking-replacing following method thereof |
| CN112446352A (en) * | 2020-12-14 | 2021-03-05 | 深圳地平线机器人科技有限公司 | Behavior recognition method, behavior recognition device, behavior recognition medium, and electronic device |
| WO2022142786A1 (en) * | 2020-12-30 | 2022-07-07 | 中兴通讯股份有限公司 | Driving behavior recognition method, and device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886150A (en) | A kind of driving behavior recognition methods based on Kinect video camera | |
| EP2718902B1 (en) | Generation of avatar reflecting player appearance | |
| US8210945B2 (en) | System and method for physically interactive board games | |
| JP4860749B2 (en) | Apparatus, system, and method for determining compatibility with positioning instruction in person in image | |
| CN104346816B (en) | Depth determining method and device and electronic equipment | |
| JP4692526B2 (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| KR101660157B1 (en) | Rehabilitation system based on gaze tracking | |
| US20120077163A1 (en) | 3d monocular visual tracking therapy system for the rehabilitation of human upper limbs | |
| CN109840478B (en) | Action evaluation method and device, mobile terminal and readable storage medium | |
| JP2017134712A (en) | Hand-wash monitoring system | |
| CN110458100A (en) | Based on target detection and the identification of the table tennis drop point of tracking and methods of marking and system | |
| KR20090084035A (en) | A real time motion recognizing method | |
| CN111222379A (en) | Hand detection method and device | |
| JP7496648B2 (en) | Surgery content evaluation system, surgery content evaluation method, and computer program | |
| CN109063661A (en) | Gait analysis method and device | |
| JP2009000410A (en) | Image processor and image processing method | |
| CN117637166A (en) | Hand rehabilitation evaluation method and system based on real-time tracking of joint points | |
| US11676357B2 (en) | Modification of projected structured light based on identified points within captured image | |
| KR20120048334A (en) | The video game console | |
| CN106580270A (en) | Facial nerve/muscle rehabilitation monitoring and guidance device and system | |
| CN115553779A (en) | Emotion recognition method and device, electronic equipment and storage medium | |
| CN113051973A (en) | Method and device for posture correction and electronic equipment | |
| CN115761873A (en) | Shoulder rehabilitation movement duration evaluation method based on gesture and posture comprehensive visual recognition | |
| CN113100755B (en) | Limb rehabilitation training and evaluating system based on visual tracking control | |
| CN115690895A (en) | Human skeleton point detection-based multi-person motion detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190614 |