CN111368696A - Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation - Google Patents
Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation Download PDFInfo
- Publication number
- CN111368696A CN111368696A CN202010129691.0A CN202010129691A CN111368696A CN 111368696 A CN111368696 A CN 111368696A CN 202010129691 A CN202010129691 A CN 202010129691A CN 111368696 A CN111368696 A CN 111368696A
- Authority
- CN
- China
- Prior art keywords
- driving behavior
- frame
- key points
- detecting
- sensitive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for detecting illegal driving behaviors of dangerous chemical transport vehicles based on visual coordination, which comprises the steps of firstly making a data set and carrying out image enhancement and normalization processing; marking dangerous driving behavior sensitive articles, and training a data set aiming at the sensitive objects through a target recognition neural network; reading each frame of the video, detecting a sensitive object through a trained network model, and then detecting key points of human body postures of the frames; detecting the human body posture to obtain the coordinate positions of the key points of the left hand and the right hand in the frame image, judging the position information of the key points and the sensitive object identification frame, and identifying corresponding illegal driving behaviors; and early warning is carried out when the continuous time window of the illegal driving behavior exceeds a set threshold value, and the video clip is reserved. The invention can adapt to real-time detection and early warning of dangerous driving behaviors of the monitored video of the driver, and is convenient to find and remind the non-standard driving behaviors in time, thereby reducing the occurrence of traffic accidents, having various detection types and good identification effect.
Description
Technical Field
The invention relates to the field of image processing and traffic safety, in particular to a dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation.
Background
With the rapid development of social economy, the market scale of chemical industry products is getting bigger and bigger, and the products become the foundation of national economy. According to research data, the scale of the market of dangerous chemicals in China in 2018 reaches 1.4 trillion, and the scale is continuously expanded in the future. Two red-headed documents are issued continuously by Jiangsu province according to the spirit of 'consensus center, State Council about promoting the development of reform in the field of safety production' (2016 No. [ 2016 ] 32) issued in 2016, 12, 18. And the two files require forced installation of an anti-collision system for two passengers and one danger simultaneously. In 2017, 10 and 27 days, a notice of an intelligent active safety prevention and control technology of key operation vehicles is formally issued by a transportation hall in Jiangsu province in 27 days (Soviu transportation (2017) 97). The huge market scale of hazardous chemicals poses challenges to the production, storage and transportation of hazardous chemicals, and the production and storage of hazardous chemicals form a mature system by introducing foreign standardized management. Management and monitoring of the transport of hazardous chemicals is somewhat difficult compared to production and storage. Firstly, the transportation scale of hazardous chemicals is huge, and a large number of vehicles are needed to perform distributed transportation at the same time, so that centralized management is difficult. Due to the long transport distance, the complex transport roads and the uncertainty of the transport lines, the real-time dynamics of the transport vehicles are difficult to get a timely feedback. In 2017, the number of dangerous chemical tank trucks in China exceeds 21 ten thousand, and the total tonnage reaches 422 ten thousand tons. The total amount of dangerous chemicals transported by China per year exceeds 10 hundred million tons, the dangerous chemicals transported by roads account for more than three times of the total amount of the dangerous chemicals transported by roads, and the scale of the dangerous chemicals transported by the roads is still in an ascending trend. Secondly, the chemical properties of the hazardous chemical substances are extremely unstable, and the hazardous chemical substances have the characteristics of flammability, explosiveness, easy corrosion and the like, and if the hazardous chemical substances are rubbed, heated or impacted in the transportation process, the hazardous chemical substances can cause the rupture, the combustion and even the explosion of the vehicle tank body, cause the casualties and the property loss of people, and cause immeasurable damage to the environment and the ecology.
In the existing technology for detecting the violation of the driver, the following types of video detection technologies are adopted by different video behavior characteristic detection technologies: (1) based on a multilayer convolutional neural network: the method comprises the steps of lifting an ISA deep network model feature extraction method formed based on an Independent Subspace Analysis (ISA) model and a neural network theory, and carrying out classification and identification on human behaviors by combining a data preprocessing method and other methods. (2) Based on a background subtraction method: and the method proposes that a background subtraction algorithm and a deep learning technology are fused, and finally verification of an algorithm model is performed on a test data set, so that the neural network technology can be used for interactive behavior recognition, and the accuracy of the behavior recognition is ensured. (3) Based on SVM vector strong classifier: li Qinghui et al compress the optical flow sequence into an ordered optical flow graph through a Rank support vector machine algorithm, and finally fuse the C3D descriptor and the VGG descriptor of the double-flow network and input the fused result into a linear SVM for behavior recognition. (4) Based on RGB two-dimensional behavior recognition: the method has the advantages that various parts of a person are coded by different colors, continuous changes of human body actions in the video are identified by local connection and a greedy algorithm, the method utilizing local characteristics is applied to data acquisition with high difficulty, great success is achieved, and the method is also helpful for acquiring behavior information of a driver. (5) The clustering-based multitask identification method comprises the following steps: liu A A et al propose a hierarchical clustering multi-task learning (HC-MTL) method, which realizes behavior recognition by strengthening the characteristics of specific behaviors through an objective function.
The common video behavior detection technical methods are effective ideas for traffic video big data processing, but have no behavior characteristic specific processing for video monitoring of drivers of dangerous chemical transport vehicles, cannot distinguish dangerous driving behaviors of the drivers, cannot distinguish sensitive articles, and cannot comprehensively identify the dangerous driving behaviors of the drivers.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a method and a system for detecting illegal driving behaviors of a dangerous chemical transport vehicle based on visual coordination, which can be used for identifying and early warning the illegal driving behaviors of a dangerous chemical transport vehicle driver such as breaking away from a steering wheel by two hands, taking a call and a phone call, drinking water and the like, and are high in detection efficiency and good in visualization effect.
The technical scheme is as follows: the invention relates to a dangerous chemical transport vehicle illegal driving behavior detection method based on visual coordination, which comprises the following steps of:
(1) making a data set and carrying out image enhancement and image normalization processing;
(2) marking dangerous driving behavior sensitive articles, and training a data set aiming at the sensitive articles through a target recognition neural network; the sensitive articles comprise a steering wheel, a telephone, a water cup and smoke;
(3) reading each frame of image of the video, detecting a sensitive object through a trained network model, and detecting key points of human body postures of the frame images;
(4) obtaining the coordinate positions of the left and right hand key points in the frame image according to the pixel coordinates of the elbow key points and the wrist key points, judging the position information of the left and right hand key points and the sensitive object identification frame, and identifying corresponding illegal driving behaviors; if the frame coordinates of the key points of both hands are outside the area of the steering wheel, the hands are considered to be separated from the steering wheel; if the frame coordinates of the left and/or right-hand key points are in other sensitive article detection areas, determining the type of the illegal driving behavior according to the detection type of the sensitive articles;
(5) and early warning is carried out when the continuous time window of the illegal driving behavior exceeds a set threshold value, and a corresponding video clip is reserved.
Preferably, the step (1) comprises:
(11) carry out image extraction to the nonstandard driving behavior in the dangerous chemicals transport vechicle driver surveillance video, the nonstandard driving behavior characteristic includes: separating the two hands from the steering wheel, making and receiving calls, drinking water and smoking;
(12) and carrying out contrast enhancement and normalization processing on the image data set.
Preferably, the neural network used in the step (2) is a modified yolov3-medium network, and is characterized in that a gradient generated by L1 regularization is added to a gradient of a BN (batch normalization) layer, a clipping rate is set to be 0.8 for clipping, gamma parameters of all BN layers are extracted into a list and sorted from small to large, a clipping threshold value is a value of 0.8 quantiles in the list, and an offset β is merged into BN in a next convolutional layer for calculation.
Preferably, the step (3) includes:
(31) reading each frame in the video stream and marking as origin _ img, firstly sending a frame image into a network model to detect a sensitive article, and obtaining a BBOX frame starting point and a confidence coefficient of the frame image;
(32) sending origin _ img into a posture detection framework, and detecting the position of a human body;
(33) after each human body obtains the detection frame, independently detecting all the limb nodes, carrying out top-down joint point augmentation connection on a single ROI area, and obtaining key point coordinate information;
(34) rendering origin _ img according to the sensitive article detection window and the posture detection information to obtain result _ img.
Preferably, in the step (4), the finger part key points are obtained by extending the vector direction by one half through the elbow key point and wrist key point pixel coordinates.
Preferably, the early warning judgment rule in the step (5) is as follows: and recording the frame time for detecting the illegal driving behavior for the first time, and performing early warning prompt and segment retention when the continuous detection time exceeds 1 second and the detected frame number exceeds 60 percent of the total frame number.
Based on the same inventive concept, the invention provides a dangerous chemical transport vehicle illegal driving behavior detection system based on visual coordination, which comprises:
the preprocessing module is used for making a data set and carrying out image enhancement and image normalization processing;
the sensitive article detection training module is used for marking the sensitive articles of dangerous driving behaviors and training the data set aiming at the sensitive articles through a neural network; the sensitive articles comprise a steering wheel, a telephone, a water cup and smoke;
the object and posture detection module is used for reading each frame of image of the video, detecting a sensitive object through the trained network model, and detecting key points of human posture of the frame image;
the illegal driving behavior recognition module is used for obtaining the coordinate positions of the left and right hand key points in the frame image according to the pixel coordinates of the elbow key points and the wrist key points, judging the position information of the left and right hand key points and the sensitive object recognition frame and recognizing the corresponding illegal driving behavior; if the frame coordinates of the key points of both hands are outside the area of the steering wheel, the hands are considered to be separated from the steering wheel; if the frame coordinates of the left and/or right-hand key points are in other sensitive article detection areas, determining the type of the illegal driving behavior according to the detection type of the sensitive articles;
and the early warning module is used for early warning when the continuous time window of the illegal driving behavior exceeds a set threshold value and reserving the corresponding video clip.
Based on the same inventive concept, the system for detecting the illegal driving behavior of the hazardous chemical substance transport vehicle based on the visual coordination comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the computer program realizes the illegal driving behavior detection method of the hazardous chemical substance transport vehicle based on the visual coordination when being loaded to the processor.
Has the advantages that: compared with the prior art, the invention has the beneficial effects that: 1. compared with the prior art, the target recognition and posture detection cooperative detection algorithm solves the problems of poor detection efficiency, low detection precision and single detection means of the traditional driver behavior detection system; 2. the detection of articles and postures in a complex environment can be handled, the method is suitable for the running environment of dangerous chemical transport vehicles in different time periods of daytime color images and nighttime infrared gray images, and has the advantages of high applicability, strong practicability and good application value.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of the present invention;
FIG. 2 is a diagram of the yolov3-medium network for identifying sensitive objects in infrared night vision video in accordance with an embodiment of the present invention;
FIG. 3 is a diagram of the view of a yolov3-medium network for identifying objects sensitive to the viewing angle of a daytime driver in the embodiment of the present invention;
FIG. 4 is a driver attitude detection diagram in an embodiment of the present invention;
fig. 5 is a visual fusion detection diagram of illegal driving behaviors of the hazardous chemical substance transport vehicle in the embodiment of the invention.
Detailed Description
The invention will be further explained with reference to the drawings.
As shown in fig. 1, the method for detecting illegal driving behavior of hazardous chemical substance transport vehicles based on visual coordination disclosed in the embodiment of the invention comprises the following steps:
(1) preprocessing for image enhancement and image normalization processing of production data set
The data set source of the embodiment is monitoring videos of dangerous goods drivers in Huai' an city, and the video data is used for monitoring the driving of a plurality of drivers in a week. The monitoring video is night gray level images acquired by the infrared camera and daytime driving video data acquired by the high-definition camera. And screening the collected videos for the non-standard driving behaviors manually, intercepting the non-standard driving behaviors (the characteristics of the non-standard driving behaviors comprise that two hands are separated from a steering wheel, a call is received and made, water drinking and smoking) of a driver in the driving process, and sampling 5 pieces of the same non-standard driving behaviors in different postures. And numbering the intercepted picture data sets and storing the picture data sets in the same directory.
Because the environment in the vehicle is noisy, the imaging quality of the infrared camera at night is poor, and the image needs to be preprocessed, so that the detection precision of sensitive articles is improved. The processing steps mainly comprise contrast enhancement and image normalization processing. The method for enhancing data from the initial gray level image data in the screenshot comprises the following steps: each channel pixel is multiplied by one number, so that the integral pixels of the image are enlarged, the contrast is improved, the outline characteristics are enhanced, and the training and the detection of the image are facilitated. The pixel values of the image after image enhancement can be expressed by the following formula.
G(X,Y)＝N*F(x,y)
In the formula: (X, Y) denotes the original image pixel, N denotes the multiple, F (X, Y) is the gray value in each pixel, and G (X, Y) denotes the pixel value after the contrast enhancement process.
And (3) carrying out image normalization processing on the data set picture subjected to image enhancement, carrying out dispersion normalization processing on the data set according to 416 x 416px, and carrying out linear transformation on the original data to map a result value between [0-1 ].
y＝(x-MinValue)/(MaxValue-MinValue)
In the formula, x and y are pixel point values before and after conversion, respectively, and MaxValue and MinValue are maximum pixel point value and minimum pixel point value in the sample, respectively.
(2) And marking dangerous driving behavior sensitive articles, and training a data set aiming at sensitive objects such as a steering wheel, a telephone, a water cup, smoke and the like through a target recognition neural network. The dataset labeling tool of this step uses lambllmg, labeled in the format of VOC 2007. Firstly, a folder to be processed is led into a labeling tool, sensitive articles such as a mobile phone, a water cup and cigarettes are set to be label values, then the sensitive articles are labeled by using a rectangular frame, and then the corresponding label values are selected. And storing the marked data in the same file directory. The marked data set comprises a picture set and an xml file set, and the content recorded in the xml file comprises a file name, a storage path, an image size, an image, a label value and the relative position of the marked image in the original image. 10000 marked VOC format data set images are used as input, the scale of the input image is 416 x 3, the sample number selected by batch training is set to be 64, the learning rate learning _ rate is set to be 0.001, and the activation function is Leaky ReLU.
The target recognition neural network adopts an improved yolov3-medium network, adjusts a yolov3-medium model, removes some unimportant neurons in the network to reduce the calculated amount and the weight quantity, firstly regularizes L1 to generate a sparse weight matrix, and adds the obtained gradient into the gradient of a BN (batch normalization) layer, uses a scaling factor gamma in the BN layer as an importance factor, sorts the weight parameter gamma according to the L1 absolute value of each neuron from small to large, sets the weight parameter gamma lower than a threshold value 0.8 to be 0, then calculates the convolution calculation result obtained by the BN layer and the combined weight parameter β (offset), and improves the calculation speed of the model.
1000 iterative training were performed on the samples. And after training is finished, measuring the average precision (class prediction accuracy) mAP @0.5 of the target detection mean value and GloU (the size coincidence degree of a prediction box and an actual box).
(3) Reading each frame of image of the video, detecting a sensitive object through a trained network model, and detecting key points of human body postures of the frame images. Reading the total frame number of the video to be detected through CV2(Open Source Computer Vision Library2), setting a starting frame and an ending frame of a video segment containing the illegal driving behavior, and performing global initialization operation. Reading each frame read by CV2 as origin _ img, passing the pixel matrix as input to yolov3-medium model network by normalization, and running forward pass to obtain the predicted bounding box list as the output of the network. And filtering the prediction frame with the low confidence score, and saving the image containing the final bounding box. If sensitive articles such as a steering wheel, a telephone, a water cup and smoke are detected, marking is carried out and confidence is given, and the effect graphs of the detected sensitive articles are shown in fig. 2 and fig. 3.
And simultaneously sending the origin _ img frames into a posture detection frame, firstly detecting the position of the human body, and then carrying out posture estimation. After each human body obtains a prediction frame, all the limb nodes are independently detected, the joint point augmentation connection from top to bottom is carried out on the single ROI area, and the coordinate information of key points is obtained (the specific implementation of posture detection can be seen in Fang H, Xie S, Tai Y, et al. The same video stream BBOX is normalized to solve the problem of color jitter of consecutive frames of video, and each frame of initial image origin _ img is rendered by combining the sensitive article detection window and the attitude detection information. The image of the human body posture estimation detection is shown in fig. 4.
(4) And performing joint detection on target detection and human body posture estimation, extending the vector direction by half by means of the pixel coordinates of the elbow key point and the wrist key point to obtain key points of the finger part, and further calculating and extracting the coordinate positions of the left finger and the right finger in the frame image. Judging whether the key points of the fingers of the two hands are overlapped with the sensitive article identification area or not so as to detect illegal driving behaviors such as playing mobile phones, smoking, drinking and the like in the driving process; the dangerous driving behavior that the hands are separated from the steering wheel possibly occurring in the driving process of a driver is judged by judging whether the positions of the key points of the fingers are overlapped with the identification area of the steering wheel.
(5) And setting a threshold, and if the dangerous driving behavior of the driver is detected to exceed the threshold in continuous time, carrying out voice broadcast to remind the driver that the driver is in the illegal driving state, please standardize the driving behavior, reserve video nodes and provide basis for the management and departure of the public transportation platform. If the frame time of the illegal driving behavior detected for the first time is recorded, when the continuous detection time exceeds 1 second and the detected frame number exceeds 60 percent of the total frame number, early warning prompt and fragment retention are carried out. Text in the dangerous driving behavior video is visualized through CV2, the behavior state of the driver in the video is displayed, and the detection effect is shown in FIG. 5.
Experiments prove that the method for detecting the illegal driving behavior of the dangerous chemical transport vehicle based on the visual coordination can be widely and effectively applied to the monitoring video in the dangerous chemical transport vehicle, can detect and early warn dangerous driving behaviors in real time in the video, and is convenient for finding and reminding the irregular driving behaviors in time, so that traffic accidents are reduced, the detection types are multiple, and the identification effect is good. Based on the same inventive concept, the invention discloses a dangerous chemical transport vehicle illegal driving behavior detection system based on visual cooperation, which comprises a preprocessing module, a detection module and a control module, wherein the preprocessing module is used for making a data set and carrying out image enhancement and image normalization processing; the sensitive article detection training module is used for marking the sensitive articles of dangerous driving behaviors and training the data set aiming at the sensitive articles through a neural network; the sensitive articles comprise a steering wheel, a telephone, a water cup and smoke; the object and posture detection module is used for reading each frame of image of the video, detecting a sensitive object through the trained network model, and detecting key points of human posture of the frame image; the illegal driving behavior recognition module is used for obtaining the coordinate positions of the left and right hand key points in the frame image according to the pixel coordinates of the elbow key points and the wrist key points, judging the position information of the left and right hand key points and the sensitive object recognition frame and recognizing the corresponding illegal driving behavior; if the frame coordinates of the key points of both hands are outside the area of the steering wheel, the hands are considered to be separated from the steering wheel; if the frame coordinates of the left and/or right-hand key points are in other sensitive article detection areas, determining the type of the illegal driving behavior according to the detection type of the sensitive articles; and the early warning module is used for early warning when the continuous time window of the illegal driving behavior exceeds a set threshold value and reserving the corresponding video clip. For details, reference is made to the above method embodiments, which are not described herein again.
Based on the same inventive concept, the hazardous chemical substance transport vehicle illegal driving behavior detection system based on visual coordination disclosed by the embodiment of the invention comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the computer program realizes the hazardous chemical substance transport vehicle illegal driving behavior detection method based on visual coordination when being loaded to the processor.
Claims (8)
1. A dangerous chemical transport vehicle illegal driving behavior detection method based on visual coordination is characterized by comprising the following steps:
(1) making a data set and carrying out image enhancement and image normalization processing;
(2) marking dangerous driving behavior sensitive articles, and training a data set aiming at the sensitive articles through a target recognition neural network; the sensitive articles comprise a steering wheel, a telephone, a water cup and smoke;
(3) reading each frame of image of the video, detecting a sensitive object through a trained network model, and detecting key points of human body postures of the frame images;
(4) obtaining the coordinate positions of the left and right hand key points in the frame image according to the pixel coordinates of the elbow key points and the wrist key points, judging the position information of the left and right hand key points and the sensitive object identification frame, and identifying corresponding illegal driving behaviors; if the frame coordinates of the key points of both hands are outside the area of the steering wheel, the hands are considered to be separated from the steering wheel; if the frame coordinates of the left and/or right-hand key points are in other sensitive article detection areas, determining the type of the illegal driving behavior according to the detection type of the sensitive articles;
(5) and early warning is carried out when the continuous time window of the illegal driving behavior exceeds a set threshold value, and a corresponding video clip is reserved.
2. The method for detecting the illegal driving behavior of the hazardous chemical substance transport vehicle based on the visual coordination according to claim 1, wherein the step (1) comprises the following steps:
(11) carry out image extraction to the nonstandard driving behavior in the dangerous chemicals transport vechicle driver surveillance video, the nonstandard driving behavior characteristic includes: separating the two hands from the steering wheel, making and receiving calls, drinking water and smoking;
(12) and carrying out contrast enhancement and normalization processing on the image data set.
3. The method for detecting the illegal driving behavior of the dangerous chemical transport vehicle based on the visual coordination as claimed in claim 1, wherein the neural network adopted in the step (2) is an improved yolov3-medium network, and is characterized in that the gradient generated by L1 regularization is added into the gradient of a BN layer, the cutting rate is set to be 0.8 for cutting, the gamma parameters of all BN layers are extracted into a list and are sorted from small to large, the cutting threshold value is a value of 0.8 quantile in the list, and the offset β is combined into the BN in the next convolutional layer for calculation.
4. The method for detecting the illegal driving behavior of the hazardous chemical substance transport vehicle based on the visual coordination according to claim 1, wherein the step (3) comprises the following steps:
(31) reading each frame in the video stream and marking as origin _ img, firstly sending a frame image into a network model to detect a sensitive article, and obtaining a BBOX frame starting point and a confidence coefficient of the frame image;
(32) sending origin _ img into a posture detection framework, and detecting the position of a human body;
(33) after each human body obtains the detection frame, independently detecting all the limb nodes, carrying out top-down joint point augmentation connection on a single ROI area, and obtaining key point coordinate information;
(34) rendering origin _ img according to the sensitive article detection window and the posture detection information to obtain result _ img.
5. The method for detecting the illegal driving behavior of the hazardous chemical substance transport vehicle based on the visual coordination as claimed in claim 1, wherein in the step (4), the vector direction of the elbow key point and the wrist key point is extended by one half through pixel coordinates of the two key points to obtain key points of the finger part.
6. The method for detecting the illegal driving behavior of the hazardous chemical substance transport vehicle based on the visual coordination as claimed in claim 1, wherein the early warning judgment rule in the step (5) is as follows: and recording the frame time for detecting the illegal driving behavior for the first time, and performing early warning prompt and segment retention when the continuous detection time exceeds 1 second and the detected frame number exceeds 60 percent of the total frame number.
7. The utility model provides a dangerous chemicals transport vechicle driving behavior detection system violating regulations based on vision is cooperative, its characterized in that includes:
the preprocessing module is used for making a data set and carrying out image enhancement and image normalization processing;
the sensitive article detection training module is used for marking the sensitive articles of dangerous driving behaviors and training the data set aiming at the sensitive articles through a neural network; the sensitive articles comprise a steering wheel, a telephone, a water cup and smoke;
the object and posture detection module is used for reading each frame of image of the video, detecting a sensitive object through the trained network model, and detecting key points of human posture of the frame image;
the illegal driving behavior recognition module is used for obtaining the coordinate positions of the left and right hand key points in the frame image according to the pixel coordinates of the elbow key points and the wrist key points, judging the position information of the left and right hand key points and the sensitive object recognition frame and recognizing the corresponding illegal driving behavior; if the frame coordinates of the key points of both hands are outside the area of the steering wheel, the hands are considered to be separated from the steering wheel; if the frame coordinates of the left and/or right-hand key points are in other sensitive article detection areas, determining the type of the illegal driving behavior according to the detection type of the sensitive articles;
and the early warning module is used for early warning when the continuous time window of the illegal driving behavior exceeds a set threshold value and reserving the corresponding video clip.
8. A system for detecting illegal driving behavior of a hazardous chemical substance transport vehicle based on visual coordination, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the computer program is loaded into the processor to realize the illegal driving behavior detection method of the hazardous chemical substance transport vehicle based on visual coordination according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010129691.0A CN111368696A (en) | 2020-02-28 | 2020-02-28 | Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010129691.0A CN111368696A (en) | 2020-02-28 | 2020-02-28 | Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111368696A true CN111368696A (en) | 2020-07-03 |
Family
ID=71212638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010129691.0A Pending CN111368696A (en) | 2020-02-28 | 2020-02-28 | Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111368696A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111985874A (en) * | 2020-08-21 | 2020-11-24 | 重庆电子工程职业学院 | Dangerous goods transportation management system and method |
| CN112149511A (en) * | 2020-08-27 | 2020-12-29 | 深圳市点创科技有限公司 | Method, terminal and device for detecting violation of driver based on neural network |
| CN112232273A (en) * | 2020-11-02 | 2021-01-15 | 上海翰声信息技术有限公司 | Early warning method and system based on machine learning identification image |
| CN112395978A (en) * | 2020-11-17 | 2021-02-23 | 平安科技（深圳）有限公司 | Behavior detection method and device and computer readable storage medium |
| CN112818913A (en) * | 2021-02-24 | 2021-05-18 | 西南石油大学 | Real-time smoking calling identification method |
| CN112818839A (en) * | 2021-01-29 | 2021-05-18 | 北京市商汤科技开发有限公司 | Method, device, equipment and medium for identifying violation behaviors of driver |
| CN112990069A (en) * | 2021-03-31 | 2021-06-18 | 新疆爱华盈通信息技术有限公司 | Abnormal driving behavior detection method, device, terminal and medium |
| CN113052071A (en) * | 2021-03-25 | 2021-06-29 | 淮阴工学院 | Method and system for rapidly detecting distraction behavior of driver of hazardous chemical substance transport vehicle |
| CN113065474A (en) * | 2021-04-07 | 2021-07-02 | 泰豪软件股份有限公司 | Behavior recognition method and device and computer equipment |
| CN113111866A (en) * | 2021-06-15 | 2021-07-13 | 深圳市图元科技有限公司 | Intelligent monitoring management system and method based on video analysis |
| CN113255509A (en) * | 2021-05-20 | 2021-08-13 | 福州大学 | Building site dangerous behavior monitoring method based on Yolov3 and OpenPose |
| CN113435402A (en) * | 2021-07-14 | 2021-09-24 | 深圳市比一比网络科技有限公司 | Method and system for detecting non-civilized behavior of train compartment |
| CN113449656A (en) * | 2021-07-01 | 2021-09-28 | 淮阴工学院 | Driver state identification method based on improved convolutional neural network |
| CN113470080A (en) * | 2021-07-20 | 2021-10-01 | 浙江大华技术股份有限公司 | Illegal behavior identification method |
| CN115147817A (en) * | 2022-06-17 | 2022-10-04 | 淮阴工学院 | Posture-guided driver distraction behavior recognition method of instance-aware network |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530730A (en) * | 2016-11-02 | 2017-03-22 | 重庆中科云丛科技有限公司 | Traffic violation detection method and system |
| CN108062536A (en) * | 2017-12-29 | 2018-05-22 | 纳恩博（北京）科技有限公司 | A kind of detection method and device, computer storage media |
| WO2018124672A1 (en) * | 2016-12-28 | 2018-07-05 | Samsung Electronics Co., Ltd. | Apparatus for detecting anomaly and operating method for the same |
| CN108510491A (en) * | 2018-04-04 | 2018-09-07 | 深圳市未来媒体技术研究院 | Blur the filter method of skeleton critical point detection result under background |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| CN109543651A (en) * | 2018-12-06 | 2019-03-29 | 长安大学 | A kind of driver's dangerous driving behavior detection method |
| CN109558865A (en) * | 2019-01-22 | 2019-04-02 | 郭道宁 | A kind of abnormal state detection method to the special caregiver of need based on human body key point |
| CN110443172A (en) * | 2019-07-25 | 2019-11-12 | 北京科技大学 | A kind of object detection method and system based on super-resolution and model compression |
| CN110618635A (en) * | 2019-10-08 | 2019-12-27 | 中兴飞流信息科技有限公司 | Train cab operation specification monitoring system based on AI technology |
-
2020
- 2020-02-28 CN CN202010129691.0A patent/CN111368696A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530730A (en) * | 2016-11-02 | 2017-03-22 | 重庆中科云丛科技有限公司 | Traffic violation detection method and system |
| WO2018124672A1 (en) * | 2016-12-28 | 2018-07-05 | Samsung Electronics Co., Ltd. | Apparatus for detecting anomaly and operating method for the same |
| CN108062536A (en) * | 2017-12-29 | 2018-05-22 | 纳恩博（北京）科技有限公司 | A kind of detection method and device, computer storage media |
| CN108510491A (en) * | 2018-04-04 | 2018-09-07 | 深圳市未来媒体技术研究院 | Blur the filter method of skeleton critical point detection result under background |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| CN109543651A (en) * | 2018-12-06 | 2019-03-29 | 长安大学 | A kind of driver's dangerous driving behavior detection method |
| CN109558865A (en) * | 2019-01-22 | 2019-04-02 | 郭道宁 | A kind of abnormal state detection method to the special caregiver of need based on human body key point |
| CN110443172A (en) * | 2019-07-25 | 2019-11-12 | 北京科技大学 | A kind of object detection method and system based on super-resolution and model compression |
| CN110618635A (en) * | 2019-10-08 | 2019-12-27 | 中兴飞流信息科技有限公司 | Train cab operation specification monitoring system based on AI technology |
Non-Patent Citations (4)
| Title |
|---|
| _从前从前_: "YOLOV3剪枝源码阅读---模型部署加速", 《HTTPS://WWW.JIANSHU.COM/P/D2D0D230EB74?CLICKTIME=1582269342》 * |
| PRADEEP.T.R 等: "Abnormal Behavior Detection In Intelligent Transport System for Intelligent Driving", 《INTERNATIONAL JOURNAL OF SOFTWARE & HARDWARE RESEARCH IN ENGINEERING》 * |
| 呼布钦: "基于深度学习的驾驶员头部及姿态识别和分类方法研究", 《中国优秀硕士学位论文全文数据库 工程科技II辑》 * |
| 熊群芳 等: "基于深度学习的驾驶员打电话行为检测方法", 《控制与信息技术》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111985874A (en) * | 2020-08-21 | 2020-11-24 | 重庆电子工程职业学院 | Dangerous goods transportation management system and method |
| CN112149511A (en) * | 2020-08-27 | 2020-12-29 | 深圳市点创科技有限公司 | Method, terminal and device for detecting violation of driver based on neural network |
| CN112232273A (en) * | 2020-11-02 | 2021-01-15 | 上海翰声信息技术有限公司 | Early warning method and system based on machine learning identification image |
| WO2021208735A1 (en) * | 2020-11-17 | 2021-10-21 | 平安科技（深圳）有限公司 | Behavior detection method, apparatus, and computer-readable storage medium |
| CN112395978A (en) * | 2020-11-17 | 2021-02-23 | 平安科技（深圳）有限公司 | Behavior detection method and device and computer readable storage medium |
| CN112395978B (en) * | 2020-11-17 | 2024-05-03 | 平安科技（深圳）有限公司 | Behavior detection method, behavior detection device and computer readable storage medium |
| CN112818839A (en) * | 2021-01-29 | 2021-05-18 | 北京市商汤科技开发有限公司 | Method, device, equipment and medium for identifying violation behaviors of driver |
| CN112818913A (en) * | 2021-02-24 | 2021-05-18 | 西南石油大学 | Real-time smoking calling identification method |
| CN112818913B (en) * | 2021-02-24 | 2023-04-07 | 西南石油大学 | Real-time smoking calling identification method |
| CN113052071A (en) * | 2021-03-25 | 2021-06-29 | 淮阴工学院 | Method and system for rapidly detecting distraction behavior of driver of hazardous chemical substance transport vehicle |
| CN112990069A (en) * | 2021-03-31 | 2021-06-18 | 新疆爱华盈通信息技术有限公司 | Abnormal driving behavior detection method, device, terminal and medium |
| CN113065474A (en) * | 2021-04-07 | 2021-07-02 | 泰豪软件股份有限公司 | Behavior recognition method and device and computer equipment |
| CN113255509A (en) * | 2021-05-20 | 2021-08-13 | 福州大学 | Building site dangerous behavior monitoring method based on Yolov3 and OpenPose |
| CN113111866B (en) * | 2021-06-15 | 2021-10-26 | 深圳市图元科技有限公司 | Intelligent monitoring management system and method based on video analysis |
| CN113111866A (en) * | 2021-06-15 | 2021-07-13 | 深圳市图元科技有限公司 | Intelligent monitoring management system and method based on video analysis |
| CN113449656A (en) * | 2021-07-01 | 2021-09-28 | 淮阴工学院 | Driver state identification method based on improved convolutional neural network |
| CN113435402A (en) * | 2021-07-14 | 2021-09-24 | 深圳市比一比网络科技有限公司 | Method and system for detecting non-civilized behavior of train compartment |
| CN113470080A (en) * | 2021-07-20 | 2021-10-01 | 浙江大华技术股份有限公司 | Illegal behavior identification method |
| CN113470080B (en) * | 2021-07-20 | 2024-05-14 | 浙江大华技术股份有限公司 | Illegal behavior recognition method |
| CN115147817A (en) * | 2022-06-17 | 2022-10-04 | 淮阴工学院 | Posture-guided driver distraction behavior recognition method of instance-aware network |
| CN115147817B (en) * | 2022-06-17 | 2023-06-20 | 淮阴工学院 | Driver distraction behavior recognition method of instance perception network guided by gestures |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368696A (en) | Dangerous chemical transport vehicle illegal driving behavior detection method and system based on visual cooperation | |
| CN107992819B (en) | Method and device for determining vehicle attribute structural features | |
| CN108875803B (en) | Hazardous chemical substance transport vehicle detection and identification method based on video image | |
| CN111695514A (en) | Vehicle detection method in foggy days based on deep learning | |
| CN108960074B (en) | Small-size pedestrian target detection method based on deep learning | |
| KR102105954B1 (en) | System and method for accident risk detection | |
| CN111523415A (en) | Image-based two-passenger one-dangerous vehicle detection method and device | |
| CN112613434A (en) | Road target detection method, device and storage medium | |
| CN113052071B (en) | Method and system for rapidly detecting distraction behavior of driver of hazardous chemical substance transport vehicle | |
| CN114972316A (en) | Battery case end surface defect real-time detection method based on improved YOLOv5 | |
| Varun et al. | A road traffic signal recognition system based on template matching employing tree classifier | |
| Mallios et al. | Vehicle damage severity estimation for insurance operations using in-the-wild mobile images | |
| CN116935361A (en) | Deep learning-based driver distraction behavior detection method | |
| CN115131826B (en) | Article detection and identification method, and network model training method and device | |
| CN115359306B (en) | Intelligent identification method and system for high-definition images of railway freight inspection | |
| He et al. | Video-based Machine Learning System for Commodity Classification. | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| TW201816745A (en) | Multifunctional intelligent driving auxiliary recording method and system improves the safety of vehicle during driving | |
| Tayo et al. | Vehicle license plate recognition using edge detection and neural network | |
| CN112633287A (en) | Text recognition method and device for mine multi-source heterogeneous image-text information | |
| Prakash-Borah et al. | Real-Time Helmet Detection and Number Plate Extraction Using Computer Vision | |
| Loungani et al. | Vision Based Vehicle-Pedestrian Detection and Warning System | |
| Ramakrishnan et al. | Autonomous Vehicle Image Classification using Deep Learning | |
| Khumaidi et al. | Vehicle Safe Distance Detection System Based On Image Processing As Accident Prevention With Faster R-CNN Method | |
| Jeon et al. | Dynamic Framerate SlowFast Network for Improving Autonomous Driving Performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |
|
| RJ01 | Rejection of invention patent application after publication |