CN111199647B - Monitoring video detection method for continuous lane changing and illegal turning of road vehicles - Google Patents
Monitoring video detection method for continuous lane changing and illegal turning of road vehicles Download PDFInfo
- Publication number
- CN111199647B CN111199647B CN201811365651.5A CN201811365651A CN111199647B CN 111199647 B CN111199647 B CN 111199647B CN 201811365651 A CN201811365651 A CN 201811365651A CN 111199647 B CN111199647 B CN 111199647B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- lane
- continuous
- video
- road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a monitoring video detection method for continuous lane changing and illegal turning of road vehicles, wherein a camera acquires video images of a road, and whether urban road vehicles have continuous lane changing and illegal turning is judged by processing the video images through a computer; the method has high detection precision, detects the vehicle based on the deep learning method, matches and tracks the vehicle, adopts the artificial neural network mode to carry out nonlinear fitting on the lane line, has wide application range, can detect not only a straight lane, but also complex conditions such as a curve and the like, has strong real-time performance, and can detect the illegal behavior in the video monitoring vehicle in real time.
Description
Technical Field
The invention belongs to the technical field of human induction equipment, and particularly relates to a monitoring video detection method for continuous lane changing and illegal turning of road vehicles.
Background
The existing detection of bad traffic behaviors such as continuous lane changing and illegal turning around of vehicles mainly comprises two types:
1. distance-based detection methods, such as ultrasonic detection, laser detection, which are expensive and sensitive to obstruction;
2. and video-based detection methods such as background modeling, frame difference method, optical flow method and the like.
The traditional detection method based on video, such as background modeling, frame difference method, optical flow method and the like, has a good detection effect on a straight road, however, due to the limitation of the installation angle of a monitoring camera in real life and the complex road shape, the detection of continuous lane change and illegal turning around of vehicles under the condition of road curvature is a difficult point of current research.
Therefore, it is necessary to invent a surveillance video detection method for continuous lane change and illegal turning of road vehicles.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method can detect whether an abnormal stopping event occurs in a monitored area or not in real time, and can also detect the current position of an abnormal vehicle.
The technical scheme adopted by the invention is as follows: the system comprises a camera and a computer, wherein the camera acquires a video image of a road, and the computer processes the video image to judge whether the urban road vehicle has continuous lane change and illegal turning;
the monitoring video detection method for the continuous lane change and illegal turning around of the road vehicle comprises the following steps:
s10, detecting and identifying a vehicle through a target detection method based on deep learning;
s20, vehicle matching and tracking are carried out on the basis of overlapping degree (IoU) of vehicle surrounding frames (bounding boxes) of two continuous frames of the video;
s30, fitting the curved lane line by adopting an artificial neural network method;
s40, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and detecting whether the vehicle has a continuous lane change phenomenon or not;
s50, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and detecting whether the vehicle breaks rules and turns around.
Further, in the step S10, a specific calculation process for detecting and identifying the vehicle through a real-time detection method based on deep learning is as follows:
s11, carrying out sample model training through urban road monitoring video data and network vehicle marking data;
s12, reading real-time urban road vehicle video data, and detecting an object through a model;
and S13, reading the detection frame, identifying the object, storing the data with the return value of the vehicle, and outputting the vehicle surrounding frame of the video frame.
Further, in the step S20, the specific process of vehicle matching and tracking is as follows:
s21, extracting overlapping degree (interaction-over-Unit, IoU) of vehicle surrounding frames (bounding box) of two continuous frames of a video, and taking IoU as a measurement standard to obtain a prediction range (bounding box) in output so as to calculate the correlation degree between the measured reality and the prediction, wherein the higher the correlation degree is, the higher the value is, the IoU calculation method is as follows:
IoU＝(Area of Overlap)/(Area of Union)；
s22, IoU of the current frame and the previous frame are calculated, if the current frame and the previous frame are larger than a certain set threshold value, the ID of the vehicle of the previous frame is assigned to the vehicle of the current frame, and the current position information of the vehicle is stored;
and S23, if the obtained IoU is less than or equal to the threshold value, giving a new ID to the current frame vehicle and simultaneously storing the current position information of the vehicle.
Further, in the step S30, a specific calculation process of the curved lane line fitting is as follows:
s31, reading the video frame and extracting the coordinates of the curved lane line;
s32, fitting the curve lane line by adopting an artificial neural network method, wherein the artificial neural network comprises an input layer, a 3-layer hidden layer and an output layer, the number of neurons in the hidden layer is 8, the mean square error is used as a loss function, and the learning rate is 0.035.
Further, in the step S40, the specific step of detecting whether the vehicle violates the continuous lane change includes:
s41, judging a lane line section where the vehicle is located based on vehicle position information and fitted curved lane line information obtained by vehicle matching and tracking, and marking as R1;
s42, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the lane position of the current vehicle is judged by associating the lane information with the vehicle position information;
s43, setting a specified frame number range, and if the lane in which the vehicle is located is continuously changed, for example, from R1 to R2, and is continuously changed to R3, the vehicle is considered to be in violation of lane change continuously.
Further, in the step S50, the specific process of detecting whether the vehicle has the illegal turning behavior is as follows:
s51, judging a lane line section where the vehicle is located based on vehicle position information and fitted curved lane line information obtained by vehicle matching and tracking, and marking as R1;
s52, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the lane position of the current vehicle is judged by associating the lane information with the vehicle position information;
s53, marking an area which can not turn around;
and S54, setting a specified frame number range, and in the non-turning-round area, if the lane where the vehicle is located is changed and the position of the vehicle is opposite to the direction, determining that the vehicle turns around illegally.
The technical scheme provided by the invention has the beneficial effects that: the method comprises the steps of collecting video images of roads through a camera, and processing the video images through a computer to judge whether urban road vehicles have continuous lane change and illegal turning;
1. the detection precision is high, the road is subjected to nonlinear fitting by adopting an artificial neural network mode, the position of any vehicle can be determined between two lanes, and the vehicle condition can be accurately judged on the basis of lane information and vehicle coordinates without difficulty;
2. the method has wide application range, and not only can detect straight lanes, but also can detect complex conditions such as curves and the like;
3. the method is strong in real-time performance, and can detect violation behaviors in the video monitoring vehicle in real time.
Description of the drawings:
FIG. 1 is a schematic flow chart of an embodiment of a surveillance video detection method for continuous lane change and illegal turning of road vehicles according to the present invention;
FIG. 2 is a schematic diagram of a neural network according to the present invention;
FIG. 3 is a schematic diagram showing the coincidence of a road curve fitted by the neural network algorithm and an actual two-dimensional spatial position;
fig. 4 is a black-and-white screenshot of the effect of vehicle matching tracking according to the overlapping degree of vehicle package frames of two consecutive frames.
Detailed Description
In order to more fully understand the technical contents of the present invention, the technical solutions of the present invention will be further described and illustrated below with reference to the accompanying drawings and specific embodiments, but not limited thereto.
Referring to fig. 1 to 4, the surveillance video detection method for continuous lane change and illegal turning of road vehicles comprises a camera and a computer, wherein the camera collects video images of roads, and the computer processes the video images to judge whether urban road vehicles have continuous lane change and illegal turning.
As shown in fig. 1, in the embodiment of the present invention, the method for detecting a surveillance video of a road vehicle changing lane continuously and turning around illegally includes the following steps:
s10, detecting and identifying a vehicle through a target detection method based on deep learning;
s20, vehicle matching and tracking are carried out on the basis of overlapping degree (IoU) of vehicle surrounding frames (bounding boxes) of two continuous frames of the video;
s30, fitting the curved lane line by adopting an artificial neural network method;
s40, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and detecting whether the vehicle has a continuous lane change phenomenon or not;
s50, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and detecting whether the vehicle breaks rules and turns around.
The concrete method of the steps is as follows:
wherein, step S10, the vehicle is detected and identified by the target detection method based on deep learning;
specifically, the vehicle detection and identification process is as follows:
s11, performing sample model training through urban road monitoring video data and network vehicle marking data, and using 159257 pictures as training data;
s12, reading real-time urban road vehicle video data, and detecting an object through the model;
and S13, reading the detection frame, identifying the object, saving the data with the return value of the vehicle, and outputting the vehicle surrounding frame of the video frame.
Wherein, in step S20, the overlapping degree (interaction) of the surrounding frames of the vehicle based on two consecutive frames of the video
over-Union, IoU) to perform vehicle matching tracking;
the method comprises the following specific steps:
s21, extracting overlapping degree (interaction-over-unity, IoU) of surrounding frames (bounding box) of two consecutive frames of video, and taking IoU as a measurement standard, obtaining a prediction range (bounding box) in the output, thereby calculating the correlation between the measured reality and the prediction, wherein the higher the correlation, the higher the value, the IoU calculation method is as follows:
IoU＝(Area of Overlap)/(Area of Union)；
s22, calculating IoU of the current frame and the previous frame, if the current frame and the previous frame are larger than a certain set threshold value, assigning the ID of the vehicle of the previous frame to the vehicle of the current frame, and storing the current position information of the vehicle;
s23, if the obtained IoU is less than or equal to the threshold, giving a new ID to the current frame vehicle and simultaneously storing the current position information of the vehicle.
In step S30, the curved lane line is fitted by using an artificial neural network method, and the specific calculation process of the curved lane line fitting is as follows:
s31, reading the video frame and extracting the coordinates of the curved lane line;
s32, fitting the curve lane line by adopting an artificial neural network method, wherein the artificial neural network comprises an input layer, a 3-layer hidden layer and an output layer, the number of neurons in the hidden layer is 8, the mean square error is used as a loss function, and the learning rate is 0.035.
Wherein, step S40, the relation between the vehicle tracking track and the curve lane line is analyzed in real time, thereby detecting whether the vehicle has continuous lane changing behavior,
specifically, the steps of detecting whether the vehicle has the continuous lane change behavior are as follows:
s41, judging a lane line section where the vehicle is located based on the vehicle position information and the fitted curved lane line information obtained by matching and tracking the vehicle, and marking the lane line section as R1;
s42, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the current lane position of the vehicle is judged by correlating the lane information with the vehicle position information;
and S43, setting the vehicle in the specified frame number range, and if the lane in which the vehicle is positioned is continuously changed, for example, from R1 to R2 and is continuously changed to R3, determining that the vehicle breaks the violation and continuously changing the lane.
The result is shown in fig. 3, and it can be seen from the figure that the road curve fitted by using the neural network algorithm coincides with the actual two-dimensional spatial position, and it can be seen that the accuracy is good.
Step S50, the relationship between the vehicle tracking trajectory and the curved lane line is analyzed in real time, so as to detect whether the vehicle has a violation of turning around.
Specifically, in step S50, the step of detecting whether the vehicle has the illegal turning behavior is as follows:
s51, judging a lane line section where the vehicle is located based on the vehicle position information and the fitted curved lane line information obtained by matching and tracking the vehicle, and marking the lane line section as R1;
s52, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the current lane position of the vehicle is judged by correlating the lane information with the vehicle position information;
s53, marking the non-reversible area;
and S54, setting a specified frame number range, in the non-U-turn region, if the lane where the vehicle is located is changed and the vehicle position is carried out in the opposite direction, determining that the vehicle turns around illegally, and carrying out black-and-white screenshot on the effect of vehicle matching tracking for the overlapping degree of vehicle package frames of two continuous frames as shown in FIG. 4.
The invention collects the video image of the road through the camera, and then judges whether the urban road vehicle has continuous lane change and illegal turning through the computer processing video image, and has the following advantages;
1. the detection precision is high, the road is subjected to nonlinear fitting by adopting an artificial neural network mode, the position of any vehicle can be determined between two lanes, and the vehicle condition can be accurately judged on the basis of lane information and vehicle coordinates without difficulty;
2. the method has wide application range, and not only can detect straight lanes, but also can detect complex conditions such as curves and the like;
3. the method is strong in real-time performance, and can detect violation behaviors in the video monitoring vehicle in real time.
The above description is only a preferred embodiment of the present patent, and not intended to limit the scope of the present patent, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the specification and the drawings, and which are directly or indirectly applied to other related technical fields, belong to the scope of the present patent protection.
Claims (6)
1. A monitoring video detection method for continuous lane changing and illegal turning of road vehicles is characterized by comprising the following steps: the system comprises a camera and a computer, wherein the camera acquires a video image of a road, and the computer processes the video image to judge whether the urban road vehicle has continuous lane change and illegal turning; the monitoring video detection method for the continuous lane change and illegal turning around of the road vehicle comprises the following steps:
s10, detecting and identifying a vehicle through a target detection method based on deep learning;
s20, vehicle matching and tracking are carried out on the basis of overlapping degree (interaction-overlapping unit, IoU) of vehicle surrounding frames (bounding boxes) of two continuous frames of the video;
s30, fitting the curved lane line by adopting an artificial neural network method;
s40, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and judging the lane line section where the vehicle is located so as to detect whether the vehicle has a continuous lane change phenomenon or not;
s50, analyzing the relation between the vehicle tracking track and the curved lane line in real time, and detecting whether the vehicle breaks rules and turns around.
2. The method for detecting the monitoring video of the road vehicle continuous lane change and the illegal turning around according to claim 1, characterized in that: in the step S10, a specific calculation process for detecting and identifying the vehicle by the real-time detection method based on deep learning is as follows:
s11, carrying out sample model training through urban road monitoring video data and network vehicle marking data;
s12, reading real-time urban road vehicle video data, and detecting an object through a model;
and S13, reading the detection frame, identifying the object, storing the data with the return value of the vehicle, and outputting the vehicle surrounding frame of the video frame.
3. The method for detecting the monitoring video of the road vehicle continuous lane change and the illegal turning around according to claim 1, characterized in that: in the step S20, the specific process of vehicle matching and tracking is as follows:
s21, extracting overlapping degree (interaction-overlapping unit, IoU) of vehicle surrounding frames (bounding box) of two continuous frames of a video, and taking IoU as a measurement standard, obtaining a prediction range (bounding box) in output so as to calculate the correlation degree between measurement reality and prediction, wherein the higher the correlation degree is, the higher the value is, the IoU calculation method is as follows: IoU ═ Area of Overlap)/(Area of Union;
s22, IoU of the current frame and the previous frame are calculated, if the current frame and the previous frame are larger than a certain set threshold value, the ID of the vehicle of the previous frame is assigned to the vehicle of the current frame, and the current position information of the vehicle is stored;
and S23, if the obtained IoU is less than or equal to the threshold value, giving a new ID to the current frame vehicle and simultaneously storing the current position information of the vehicle.
4. The method for detecting the monitoring video of the road vehicle continuous lane change and the illegal turning around according to claim 1, characterized in that: in the step S30, a specific calculation process of the curved lane line fitting is as follows:
s31, reading the video frame and extracting the coordinates of the curved lane line;
s32, fitting the curve lane line by adopting an artificial neural network method, wherein the artificial neural network comprises an input layer, a 3-layer hidden layer and an output layer, the number of neurons in the hidden layer is 8, the mean square error is used as a loss function, and the learning rate is 0.035.
5. The method for detecting the monitoring video of the road vehicle continuous lane change and the illegal turning around according to claim 1, characterized in that: in the step S40, the specific steps of detecting whether the vehicle violates the continuous lane change are as follows:
s41, judging a lane line section where the vehicle is located based on vehicle position information and fitted curved lane line information obtained by vehicle matching and tracking, and marking as R1;
s42, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the lane position of the current vehicle is judged by associating the lane information with the vehicle position information;
s43, setting a specified frame number range, and if the lane in which the vehicle is located is continuously changed, for example, from R1 to R2, and is continuously changed to R3, the vehicle is considered to be in violation of lane change continuously.
6. The method for detecting the monitoring video of the road vehicle continuous lane change and the illegal turning around according to claim 1, characterized in that: in the step S50, the specific process of detecting whether the vehicle has the illegal turning behavior is as follows:
s51, judging a lane line section where the vehicle is located based on vehicle position information and fitted curved lane line information obtained by vehicle matching and tracking, and marking as R1;
s52, if the position information of the vehicle is in R1 and is not changed continuously, the vehicle does not change the lane, and the lane position of the current vehicle is judged by associating the lane information with the vehicle position information;
s53, marking an area which can not turn around;
and S54, setting a specified frame number range, and in the non-turning-round area, if the lane where the vehicle is located is changed and the position of the vehicle is opposite to the direction, determining that the vehicle turns around illegally.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811365651.5A CN111199647B (en) | 2018-11-16 | 2018-11-16 | Monitoring video detection method for continuous lane changing and illegal turning of road vehicles |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811365651.5A CN111199647B (en) | 2018-11-16 | 2018-11-16 | Monitoring video detection method for continuous lane changing and illegal turning of road vehicles |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111199647A CN111199647A (en) | 2020-05-26 |
| CN111199647B true CN111199647B (en) | 2021-10-01 |
Family
ID=70747317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811365651.5A Active CN111199647B (en) | 2018-11-16 | 2018-11-16 | Monitoring video detection method for continuous lane changing and illegal turning of road vehicles |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111199647B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968378A (en) * | 2020-07-07 | 2020-11-20 | 浙江大华技术股份有限公司 | Motor vehicle red light running snapshot method and device, computer equipment and storage medium |
| CN112668428A (en) * | 2020-12-21 | 2021-04-16 | 北京百度网讯科技有限公司 | Vehicle lane change detection method, roadside device, cloud control platform and program product |
| CN112906566B (en) * | 2021-02-18 | 2024-02-13 | 天津大学 | Depth of field judging method for offending and staying of descending person in operation scene of suspension arm vehicle |
| CN112991749B (en) * | 2021-05-13 | 2021-07-20 | 腾讯科技（深圳）有限公司 | Vehicle violation identification method and device, medium and electronic equipment |
| CN113569666B (en) * | 2021-07-09 | 2023-12-15 | 东风汽车集团股份有限公司 | Method for detecting continuous illegal lane change of vehicle and computer equipment |
| CN113487878A (en) * | 2021-07-12 | 2021-10-08 | 重庆长安新能源汽车科技有限公司 | Motor vehicle illegal line pressing running detection method and system |
| CN116612642B (en) * | 2023-07-19 | 2023-10-17 | 长沙海信智能***研究院有限公司 | Vehicle continuous lane change detection method and electronic equipment |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622884B (en) * | 2012-03-22 | 2015-04-29 | 杭州电子科技大学 | Vehicle illegal turning behavior detection method based on tracking |
| CN102819952B (en) * | 2012-06-29 | 2014-04-16 | 浙江大学 | Method for detecting illegal lane change of vehicle based on video detection technique |
| CN106886755A (en) * | 2017-01-19 | 2017-06-23 | 北京航空航天大学 | A kind of intersection vehicles system for detecting regulation violation based on Traffic Sign Recognition |
| JP6763344B2 (en) * | 2017-04-12 | 2020-09-30 | トヨタ自動車株式会社 | Lane change support device |
| CN106981202A (en) * | 2017-05-22 | 2017-07-25 | 中原智慧城市设计研究院有限公司 | A kind of vehicle based on track model lane change detection method back and forth |
| CN108090456B (en) * | 2017-12-27 | 2020-06-19 | 北京初速度科技有限公司 | Training method for recognizing lane line model, and lane line recognition method and device |
-
2018
- 2018-11-16 CN CN201811365651.5A patent/CN111199647B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111199647A (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111199647B (en) | Monitoring video detection method for continuous lane changing and illegal turning of road vehicles | |
| US10522040B2 (en) | Real-time video analytics for traffic conflict detection and quantification | |
| KR100969995B1 (en) | System of traffic conflict decision for signalized intersections using image processing technique | |
| CN111753797B (en) | Vehicle speed measuring method based on video analysis | |
| Zhangyu et al. | A camera and LiDAR data fusion method for railway object detection | |
| Song et al. | Vehicle behavior analysis using target motion trajectories | |
| CN110379168B (en) | Traffic vehicle information acquisition method based on Mask R-CNN | |
| CN103456024B (en) | A kind of moving target gets over line determination methods | |
| CN111626275B (en) | Abnormal parking detection method based on intelligent video analysis | |
| CN110688922A (en) | Deep learning-based traffic jam detection system and detection method | |
| CN110929676A (en) | Deep learning-based real-time detection method for illegal turning around | |
| CN107221175A (en) | A kind of pedestrian is intended to detection method and system | |
| Abdel-Aty et al. | Advances and applications of computer vision techniques in vehicle trajectory generation and surrogate traffic safety indicators | |
| Im et al. | Bigdata analytics on CCTV images for collecting traffic information | |
| CN117115752A (en) | Expressway video monitoring method and system | |
| KR102616571B1 (en) | System and method for providing road traffic information based on image analysis using artificial intelligence | |
| CN114693722B (en) | Vehicle driving behavior detection method, detection device and detection equipment | |
| Laureshyn et al. | Automated video analysis as a tool for analysing road user behaviour | |
| Lu et al. | Monocular multi-kernel based lane marking detection | |
| Gao et al. | Automatic vehicle license plate recognition with color component texture detection and template matching | |
| CN112329671B (en) | Pedestrian running behavior detection method based on deep learning and related components | |
| Khuc | Computer vision based structural identification framework for bridge health mornitoring | |
| Dong | Investigation of computer vision concepts and methods for structural health monitoring and identification applications | |
| Wardha et al. | Development of automated technique for vehicle speed estimation and tracking in video stream | |
| Hnoohom et al. | The video-based safety methodology for pedestrian crosswalk safety measured: The case of Thammasat University, Thailand |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |