CN112289022A - Black smoke vehicle detection judgment and system based on space-time background comparison - Google Patents
Black smoke vehicle detection judgment and system based on space-time background comparison Download PDFInfo
- Publication number
- CN112289022A CN112289022A CN202011053114.4A CN202011053114A CN112289022A CN 112289022 A CN112289022 A CN 112289022A CN 202011053114 A CN202011053114 A CN 202011053114A CN 112289022 A CN112289022 A CN 112289022A
- Authority
- CN
- China
- Prior art keywords
- black smoke
- vehicle
- detection
- video stream
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of monitoring, analyzing and counting of illegal vehicles, and discloses a black smoke vehicle detection judgment and system based on space-time background comparison, wherein a video stream carrying traffic road condition images is acquired from an externally connected industrial camera; extracting key frames from the acquired video stream; carrying out common vehicle detection and black smoke vehicle detection on the key frame by using a deep learning algorithm, and respectively storing a detection screenshot and a 5-second video containing the black smoke vehicle; carrying out license plate recognition, Ringelman black smoke coefficient judgment and illegal road number judgment on the black smoke vehicle; and finishing the storage and analysis of the detection result, and outputting and storing the results of the black cigarette car video, screenshot, information and other data with the detection result for later analysis and inspection. The method effectively solves the problem that the black smoke vehicle is difficult to detect, and provides a method for extracting the key frame for accelerating the problem of low real-time detection speed, so that the accuracy and the real-time performance are obviously improved.
Description
Technical Field
The invention belongs to the technical field of monitoring, analyzing and counting of illegal vehicles, and particularly relates to a black smoke vehicle detection judgment and system based on space-time background comparison.
Background
In recent years, with the continuous development of national economy, the living standard of residents is continuously improved, and automobiles become essential vehicles in human life and production. However, along with the popularization of automobiles, the pollution of automobile exhaust is also more serious. According to the annual report of environmental management of motor vehicles in China issued by the environmental protection department in 2017, the number of motor vehicles kept in the country in 2016 reaches 2.95 hundred million vehicles, which is 8.1 percent higher than that in 2015, and the automobiles have the dominant position. In 2016, the preliminary accounting of pollutants discharged by motor vehicles in China is 4472.5 ten thousand tons, which is reduced by 1.3 percent compared with that in 2015. Wherein, 3419.3 million tons of carbon monoxide (CO), 422.0 million tons of Hydrocarbon (HC), 577.8 million tons of nitrogen oxide (NOx) and 53.4 million tons of Particulate Matters (PM). Automobiles are the most significant contributors to the overall amount of pollutant emissions, with emissions of CO and HC exceeding 80%, and NOx and PM exceeding 90%. These facts indicate that monitoring and control of the motor vehicle emitting the polluted gas are urgent. However, the black smoke vehicle supervision mode adopted in most cities in China mainly depends on manual supervision. According to investigation, a large amount of manpower needs to be hired in a manual supervision mode, each video needs to be specially equipped with at least 1 person for viewing, but the eyes of people are visual fatigue along with long-time use, and sometimes the eyes of people cannot see the black smoke car, so that a large amount of missed detection and false detection are caused.
On the other hand, computer science has been greatly developed in image recognition, object detection, and the like. From the nineties of the last century, scientists are no longer limited to the three-dimensional shape of an object when studying the object in an image, but begin to consider the characteristics of the object in the image, obtain the characteristic vector of the image by abstracting the image, and then compare and judge the characteristic vector with a sample set. After 2000, the recognition field is better developed, and people design various image features such as SIFT, HOG, LBP and the like, which have better robustness than image edges, corners and the like. Meanwhile, various powerful classifiers have appeared in the development of machine learning methods, and by applying these techniques, the accuracy of object detection and image recognition is becoming higher and higher, and the detection efficiency is also becoming faster and faster. Therefore, more and more environmental protection people are looking at intelligent monitoring. The intelligent monitoring has obvious advantages, firstly, the investment of manpower can be reduced, and the consumption of a large amount of manpower and financial resources is reduced; secondly, intelligent monitoring can not produce "fatigue", as long as the outage, the computer can keep efficient work all the time. In addition, an intelligent monitoring system is established, so that the full automation of the black smoke vehicle from discovery to submission of relevant evidence to relevant departments can be realized, and the possibility of information input errors possibly occurring in manual supervision is reduced.
Through the above analysis, the problems and defects of the prior art are as follows: the existing black smoke vehicle detection field has low accuracy, low instantaneity and low functionality.
The difficulty in solving the above problems and defects is: many conventional methods are not ideal for black smoke car determination because black smoke has the characteristics of indefinite shape, uneven motion, and non-unique color. The existing methods comprise color histogram comparison, detection based on a connected domain, a shape coding method and the like, although a plurality of improved methods improve the detection effect of the black smoke vehicle, most of the existing methods still perform detection judgment on a single frame image, and an algorithm with high accuracy rate capable of being practically applied is still lacked.
The existing black smoke vehicle detection algorithm has very limited functions, and many algorithms can only judge whether a black smoke vehicle exists in a current video and cannot indicate the position of the black smoke vehicle. The existing supervision personnel need not only the recognition of the black smoke vehicle, but also the position of the black smoke vehicle, the license plate number of the black smoke vehicle, the lane position, the black smoke degree judgment and the like. These additional requirements require algorithms capable of handling multiple tasks simultaneously, but this will certainly increase the amount of computation, and the requirement of the number of functions and the requirement of the operation speed become a dilemma.
Correspondingly, the increase of functions brings huge calculation amount, and one algorithm can complete black smoke detection in real time, but the license plate recognition can seriously slow down the running speed of the whole program. If auxiliary functions such as black smoke level determination and lane determination are required, the program runs very slowly and stops working due to memory problems.
The significance of solving the problems and the defects is as follows: the urban traffic can be seriously affected by the missed detection of the black smoke car, and the urban air is polluted. The defect that traditional black cigarette car detected has been overcome to this patent, utilizes "space-time background contrast" method, and the rate of accuracy that the black cigarette car detected is showing to be improved, has reduced the possibility that the black cigarette car missed checking, and this is favorable to traffic supervisory personnel can be real-time accurate locking vehicle violating the regulations. The multifunctionality that black cigarette car detected can provide more vehicle information violating the regulations, has removed the trouble of artifical judgement from, has greatly improved supervision efficiency, really becomes supervisory personnel's the helping hand of success. The real-time performance is always an important index of a black smoke vehicle detection algorithm, the real-time performance of the multifunctional black smoke vehicle detection system is a challenge due to the fact that huge calculated amount exists, the multi-thread processing mode is adopted, a large number of calculation links are needed in parallel processing, the memory pressure is relieved, and the multifunctional black smoke vehicle detection system with the real-time performance is achieved. This supervision efficiency that will effectively promote the car violating the regulations provides an effectual solution for black cigarette car monitoring field.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a black smoke vehicle detection judgment and system based on space-time background comparison.
The invention is realized in such a way, the black smoke vehicle detection and judgment method based on space-time background comparison reads the video stream of the black smoke vehicle detection and judgment method based on space-time background comparison, and transmits the externally collected road video stream to a local computer of the same address segment in a local area network form for real-time processing through an industrial camera which is erected in advance; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the detection result for rechecking analysis.
Further, the black smoke vehicle detection and judgment method based on space-time background comparison comprises the following steps:
firstly, reading video stream in real time, transmitting the externally acquired road video stream to a local computer of the same address segment in a local area network form by an externally erected industrial camera, and finishing the real-time acquisition of the video stream.
Secondly, extracting key frames from the acquired video stream for inputting the key frames into a deep learning detection algorithm to detect the black smoke cars of the video stream;
thirdly, carrying out black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a black smoke vehicle video with a detection result, a target screenshot, a license plate number, a Ringelmann black smoke coefficient and a lane where the black smoke vehicle video is located;
and fourthly, storing and analyzing the detection result, naming the video with the black smoke vehicle, the target screenshot, the license plate number, the Ringelmann black smoke coefficient and the result of the lane with the black smoke vehicle by using the timestamp, outputting and storing the result for later analysis and inspection.
Further, the third step of performing black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a black smoke vehicle video with a detection result, a target screenshot, a license plate number, a ringer-Mannheim black smoke coefficient and a lane where the black smoke vehicle video is located comprises the following steps:
(1) inputting a key frame picture, and under the condition of keeping the aspect ratio unchanged, adjusting the image into a 3-channel RGB image with the size of 416 x 416;
(2) the network is operated. The convolution layer of YOLOv3 divides the input image into S-S grids, predicts the size, position and confidence of the target of the boundary frame, generates the final vehicle detection data frame through the inhibition of non-maximum value, and returns;
(3) acquiring areas where black smoke possibly exists, and calculating areas where black smoke possibly exists in the front of and behind the vehicle according to the acquired vehicle detection frame; specifically, the length of the black smoke frame is consistent with that of the corresponding vehicle, and the width is 3/4 corresponding to the width of the vehicle;
(4) comparing the black smoke area with the background area, and respectively calculating the mean value and the variance of the pixel values of the black smoke area and the background area; if the pixel mean value of the black smoke region is reduced by more than 10% compared with the background, the formula
Considering that the black smoke exists, dividing the black smoke into 6 grades of the Ringelmann coefficient according to the degree of mean value reduction, see formula
(5) Detecting a license plate, wherein if the front area and the rear area of the vehicle are judged to be areas with the Ringelmann coefficient larger than 0, the current vehicle is considered to be a black smoke vehicle, and if the current vehicle is the black smoke vehicle, the area of the vehicle is input into a license plate detection algorithm;
(6) and (2) lane detection, if the vehicle is judged to be a black smoke vehicle, performing lane detection according to coordinates of the vehicle, reading an image under the current video stream, manually drawing a lane line position, calculating a lane line binary linear function slope and intercept, and calculating the central coordinates (x, y) of the black smoke vehicle according to a detection frame position (top, left, right, bottom), wherein the formula is shown in the specification
And
the y value is taken in to calculate the x corresponding to each lane linei(i is 0, 1, 2, … …) and determines that the vehicle is in the second lane, see equation
Further, the algorithm for inputting the vehicle area into the license plate detection specifically comprises: firstly, obtaining a binary image by using a u-net image segmentation algorithm; then, using Opencv to carry out edge detection to obtain the coordinates of the license plate area, and correcting the license plate graph; and finally, carrying out end-to-end identification on the license plate multiple labels by using a convolutional neural network.
It is a further object of the invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of: the reading of the video stream is to transmit the externally collected road video stream to a local computer of the same address field in a local area network mode for real-time processing through an industrial camera erected in advance; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the stored detection result for rechecking analysis.
It is another object of the present invention to provide a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of: the reading of the video stream is to transmit the externally collected road video stream to a local computer of the same address field in a local area network mode for real-time processing through an industrial camera erected in advance; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the stored detection result for rechecking analysis.
Another object of the present invention is to provide a black smoke vehicle detection and determination system based on space-time background comparison, which implements the black smoke vehicle detection and determination method based on space-time background comparison, wherein the black smoke vehicle detection and determination system based on space-time background comparison includes:
the video stream acquisition module is used for acquiring a video stream carrying traffic road condition images by an externally connected industrial camera;
the key frame extraction module is used for extracting key frames from the acquired video stream;
the vehicle detection module is used for carrying out common vehicle detection and black smoke vehicle detection on the key frames by adopting a deep learning algorithm, and respectively storing a detection screenshot and a 5-second video containing the black smoke vehicle;
the vehicle information identification module is used for carrying out license plate identification, Ringelmann black smoke coefficient judgment and illegal road number judgment on the black smoke vehicle;
and the detection result analysis module is used for finishing the storage and analysis of the detection result and outputting and storing the black cigarette car video, screenshot and information data result with the detection result.
Another object of the present invention is to provide an intelligent traffic control system, wherein the intelligent traffic control system is equipped with the black smoke vehicle detection and determination system based on space-time background comparison.
The invention also aims to provide an intelligent analysis system for intelligent traffic, which is equipped with the black smoke vehicle detection and judgment system based on space-time background comparison.
The invention also provides a rapid detection system for intelligent traffic, which is equipped with the black smoke vehicle detection and determination system based on space-time background comparison.
By combining all the technical schemes, the invention has the advantages and positive effects that: the black smoke vehicle detection system has the advantages that the black smoke vehicle detection system is a multifunctional system, most of the existing black smoke vehicle detection systems can only complete the recognition of black smoke vehicles, but in the actual traffic management, managers also need to perform license plate recognition, lane judgment, black smoke degree judgment, illegal road section judgment and the like on the black smoke vehicles, so that the existing black smoke vehicle detection system is difficult to completely meet the requirements of supervisors. On the basis of traditional black smoke vehicle detection, more practical functions such as license plate detection, lane judgment, Ringelmann coefficient judgment and the like are added, the improvement can more comprehensively meet the requirements of traffic supervisors, and the improvement is favorable for improving the traffic supervision efficiency.
The invention has the advantage of key frame extraction in the aspect of selecting video streams, the frame rate of an original video can reach the speed of 25 frames/second, but the motion of a target is continuous and slow. The similarity between adjacent frames is very high, if each frame is detected and stored, a large number of repeated detections will occur, which results in great reduction of the real-time performance of the detection, and therefore the selection of the key frame is a preprocessing operation before the target detection. The invention adopts an equal-interval sampling method, and samples one frame every k frames on the assumption that the frame rate of the video stream is k frames/second, thereby reducing repeated calculation and meeting the real-time requirement. Because the running speed of the vehicle under the camera is limited, the method is simple and efficient, the function of capturing the black smoke vehicle can be realized, and the condition of missed detection cannot be caused.
The invention has the advantage of robustness to complex environments in the aspect of black smoke vehicle detection. The detection performance of common black smoke vehicle detection algorithms can be greatly reduced in complex scenes (such as rainy days, wetlands, shadow roads and pothole roads), because the algorithms only consider the characteristics of the black smoke vehicle of the current frame, and the methods are easily interfered by information from the roads. The invention provides a space-time background-based comparison method, which is characterized in that a vehicle-free and black smoke-free area in a video historical frame is used as a background template, a current area to be detected is compared with the background template, if the ratio of pixel mean values exceeds a threshold value, the current area is judged to be a black smoke area, and if not, the current area is a normal area. The method greatly increases the robustness of the black smoke vehicle detection algorithm, and experiments show that under most road conditions, the black smoke vehicle detection accuracy of the algorithm is far higher than that of the existing algorithm, the false detection rate is lower than that of the existing algorithm, and the working difficulty of traffic supervision personnel in later stage inspection of the black smoke vehicle is effectively reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained from the drawings without creative efforts.
Fig. 1 is a flowchart of a black smoke vehicle detection and determination method based on space-time background comparison according to an embodiment of the present invention.
FIG. 2 is a schematic structural diagram of a black smoke vehicle detection and determination system based on space-time background comparison according to an embodiment of the present invention;
in fig. 2: 1. a video stream acquisition module; 2. a key frame extraction module; 3. a vehicle detection module; 4. a vehicle information identification module; 5. and the detection result is analyzed by a module.
Fig. 3 is a flow chart of an implementation of the black smoke vehicle detection and determination method based on space-time background comparison according to the embodiment of the present invention.
Fig. 4 is a schematic diagram of a detection result of a video stream obtained by using the deep learning algorithm YOLOv3 according to an embodiment of the present invention.
Fig. 5 is a schematic view of a vehicle information folder corresponding to each black smoke vehicle and named in a time stamp manner according to an embodiment of the present invention.
Fig. 6 is specific contents in a folder provided by an embodiment of the present invention, including a black smoke car screenshot, a black smoke car 5 second video, and a black smoke car information experiment.
Fig. 7 is a schematic diagram of black smoke vehicle information provided by an embodiment of the present invention, which is composed of a black smoke vehicle license plate number, a lingeman blackness coefficient, and a lane number.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Aiming at the problems in the prior art, the invention provides a black smoke vehicle detection judgment and system based on space-time background comparison, and the invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the method for detecting and determining a black smoke vehicle based on space-time background comparison provided by the invention comprises the following steps:
s101: and reading the video stream in real time, and transmitting the externally acquired road video stream to a local computer of the same address segment in a local area network form by an externally erected industrial camera to finish the real-time acquisition of the video stream.
S102: and extracting key frames from the acquired video stream for inputting into a deep learning detection algorithm to detect the black smoke car of the video stream.
S103: and carrying out black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a black smoke vehicle video with a detection result, a target screenshot, a license plate number, a Ringelmann black smoke coefficient, a lane where the black smoke vehicle video is located and the like.
S104: and (4) storing and analyzing the detection result, naming the results of the video with the black smoke vehicle, the target screenshot, the license plate number, the Ringelmann black smoke coefficient, the lane where the black smoke vehicle is located and the like by using the timestamp, outputting and storing the results for later analysis and inspection.
A person skilled in the art can also use other steps to implement the method for detecting and determining a black smoke car based on space-time background comparison provided by the present invention, and the method for detecting and determining a black smoke car based on space-time background comparison provided by the present invention shown in fig. 1 is only a specific example.
As shown in fig. 2, the black smoke vehicle detection and determination system based on space-time background comparison provided by the present invention includes:
the video stream acquisition module 1 is used for acquiring a video stream carrying traffic road condition images by an externally connected industrial camera;
a key frame extraction module 2, configured to extract a key frame from the acquired video stream;
the vehicle detection module 3 is used for performing common vehicle detection and black smoke vehicle detection on the key frames by adopting a deep learning algorithm, and respectively storing a detection screenshot and a 5-second video containing the black smoke vehicle;
the vehicle information identification module 4 is used for carrying out license plate identification, Ringelmann black smoke coefficient judgment and illegal road number judgment on the black smoke vehicle;
and the detection result analysis module 5 is used for finishing the storage and analysis of the detection result and outputting and storing the results of the black cigarette car video, screenshot, information and other data with the detection result.
The technical solution of the present invention is further described below with reference to the accompanying drawings.
As shown in fig. 3, the method for detecting and determining a black smoke vehicle based on space-time background comparison provided by the present invention specifically includes the following steps:
step one, reading the video stream in real time, and transmitting the externally acquired road video stream to a local computer of the same address segment in a local area network form by an externally erected industrial camera to finish the real-time acquisition of the video stream.
And step two, extracting key frames from the acquired video stream for inputting the key frames into a deep learning detection algorithm to detect the black smoke car of the video stream. The key frame is extracted by adopting an equal-interval sampling method, and if the frame rate of the video stream is k frames/second, one frame is sampled every k frames, so that repeated calculation is reduced, and the real-time requirement is met.
And step three, performing black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a video with the black smoke vehicle, a target screenshot, a license plate number, a Ringelmann black smoke coefficient, a lane where the black smoke vehicle is located and the like. The specific algorithm steps are as follows:
3a) and inputting the key frame picture, and adjusting the image into a 3-channel RGB image with the size of 416 x 416 under the condition of keeping the aspect ratio unchanged.
3b) The network is operated. The convolution layer of YOLOv3 divides the input image into S x S meshes, and predicts the bounding box size, position, and confidence of the target. And generating a final vehicle detection data frame through non-maximum value suppression, and returning.
3c) There is a region where black smoke may exist. And calculating areas where black smoke is likely to appear in the front and the back of the vehicle according to the obtained vehicle detection frame. Specifically, the length of the soot frame is kept consistent with the corresponding vehicle, and the width is 3/4 corresponding to the width of the vehicle.
3d) The black smoke area is compared to the background area. And respectively calculating the pixel value mean value and the variance of the black smoke area and the background area. If the pixel mean value of the black smoke region is reduced by more than 10% from the background (see formula 1), the black smoke is considered to be present, and the black smoke is classified into 6 levels of the lingermann coefficient according to the degree of reduction of the mean value, see formula (2).
3e) And (5) detecting a license plate. If the front and rear areas of the vehicle are judged to be areas with the Ringelmann coefficient larger than 0, the current vehicle is considered to be a black smoke vehicle, and if the current vehicle is the black smoke vehicle, the vehicle area is input into a license plate detection algorithm. The specific algorithm can be divided into the following steps: firstly, obtaining a binary image by using a u-net image segmentation algorithm; then, using Opencv to carry out edge detection to obtain the coordinates of the license plate area, and correcting the license plate graph; and finally, carrying out end-to-end identification on the license plate multiple labels by using a convolutional neural network.
3f) And detecting a lane. And if the vehicle is judged to be a black smoke vehicle, performing lane detection according to the coordinates of the vehicle. Reading an image flowing down from a current video, manually drawing the position of a lane line, calculating the slope and intercept of a binary linear function of the lane line, calculating the central coordinates (x and y) of the black smoke car according to the position (top, left, right, bottom) of a detection frame, and substituting y value to calculate the x corresponding to each lane line according to the formulas (3) and (4)i(i ═ 0, 1, 2, … …), and the vehicle is determined to be in the lane number ii, as shown in equation (5):
and step four, storing and analyzing the detection result, naming the results of the video with the black smoke car, the target screenshot, the license plate number, the Ringelmann black smoke coefficient, the lane where the black smoke car is located and the like by using the timestamp, outputting and storing the results for later analysis and inspection.
The technical effects of the invention are further explained by combining simulation experiments as follows:
1. simulation conditions
The simulation experiment of the invention is completed under the windows configuration of Intel (R) core (R) i7-7700CPU @3.60GHz3.60GHz CPU, memory 16G and display card GeForceGTX1080 Ti.
2. Content of simulation experiment
(1) Fig. 4 is a schematic diagram of the black smoke detection result of the video stream obtained by using the deep learning algorithm YOLOv3 according to the embodiment of the present invention.
(2) Fig. 5 is a schematic view of a vehicle information folder corresponding to each black smoke vehicle and named in a time stamp manner according to an embodiment of the present invention.
(3) Fig. 6 is specific contents in a folder provided by an embodiment of the present invention, including a black smoke car screenshot, a black smoke car 5 second video, and a black smoke car information experiment.
(4) Fig. 7 is a schematic diagram of black smoke vehicle information provided by an embodiment of the present invention, which is composed of a black smoke vehicle license plate number, a lingeman blackness coefficient, and a lane number.
The data of the simulation experiment is video stream data acquired in real time by a Haokwev snapshot camera iDS-TCV900, wherein the size of each frame of the image is 2560 × 1440, and the frame rate is 25 frames/second.
3. Simulation experiment results and analysis thereof
Fig. 4 shows the result that the video stream contains the black smoke vehicle, and it can be seen from the figure that the algorithm can effectively detect the black smoke region, and the algorithm can also effectively detect the black smoke region in rainy days, shadow ground and pothole road surfaces. Fig. 5 is a schematic diagram of a vehicle information folder corresponding to each black smoke vehicle and named in a timestamp mode, and the mode can store the accurate time of the black smoke vehicle, replace the inefficient work of calling and monitoring human eyes for troubleshooting, and improve the troubleshooting efficiency. The specific contents in the folder of fig. 6 include a screenshot of a black smoke car, a 5-second video of the black smoke car, and an information experiment of the black smoke car. Fig. 7 is a schematic diagram of black smoke vehicle information contained in the folder, which is composed of the number of black smoke vehicles, the ringelman blackness coefficient, and the number of lanes. The blackness level of the black smoke vehicle can be accurately judged, and subsequent screening work is facilitated.
It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided on a carrier medium such as a disk, CD-or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier, for example. The apparatus and its modules of the present invention may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of hardware circuits and software, e.g., firmware.
The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A black smoke vehicle detection and judgment method based on space-time background comparison is characterized in that the reading of video streams of the black smoke vehicle detection and judgment method based on space-time background comparison is realized by transmitting externally acquired road video streams to a local computer of the same address segment in a local area network mode through an industrial camera erected in advance for real-time processing; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the detection result for rechecking analysis.
2. The method for detecting and determining black smoke vehicles based on space-time background comparison as claimed in claim 1, wherein the method for detecting and determining black smoke vehicles based on space-time background comparison comprises the following steps:
firstly, reading a video stream in real time, and transmitting an externally acquired road video stream to a local computer of the same address segment in a local area network form by an externally erected industrial camera to finish the real-time acquisition of the video stream;
secondly, extracting key frames from the acquired video stream for inputting the key frames into a deep learning detection algorithm to detect the black smoke cars of the video stream;
thirdly, carrying out black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a black smoke vehicle video with a detection result, a target screenshot, a license plate number, a Ringelmann black smoke coefficient and a lane where the black smoke vehicle video is located;
and fourthly, storing and analyzing the detection result, naming the video with the black smoke vehicle, the target screenshot, the license plate number, the Ringelmann black smoke coefficient and the result of the lane with the black smoke vehicle by using the timestamp, outputting and storing the result for later analysis and inspection.
3. The black smoke vehicle detection and judgment method based on space-time background comparison as claimed in claim 1, wherein the third step of performing black smoke vehicle detection on the key frame by using a deep learning algorithm to obtain a black smoke vehicle video with a detection result, a target screenshot, a license plate number, a Ringelmann black smoke coefficient and a lane where the black smoke vehicle video is located comprises:
(1) inputting a key frame picture, and under the condition of keeping the aspect ratio unchanged, adjusting the image into a 3-channel RGB image with the size of 416 x 416;
(2) operating the network, dividing the input image into S-S grids by the convolution layer of YOLOv3, predicting the size, position and confidence coefficient of the target of the boundary frame, generating a final vehicle detection data frame through non-maximum value inhibition, and returning;
(3) acquiring areas where black smoke possibly exists, and calculating areas where black smoke possibly exists in the front of and behind the vehicle according to the acquired vehicle detection frame; specifically, the length of the black smoke frame is consistent with that of the corresponding vehicle, and the width is 3/4 corresponding to the width of the vehicle;
(4) black smoke areaComparing the area with the background area, and respectively calculating the mean value and the variance of the pixel values of the black smoke area and the background area; if the pixel mean value of the black smoke region is reduced by more than 10% compared with the background, the formula
Considering that the black smoke exists, dividing the black smoke into 6 grades of the Ringelmann coefficient according to the degree of mean value reduction, see formula
(5) Detecting a license plate, wherein if the front area and the rear area of the vehicle are judged to be areas with the Ringelmann coefficient larger than 0, the current vehicle is considered to be a black smoke vehicle, and if the current vehicle is the black smoke vehicle, the area of the vehicle is input into a license plate detection algorithm;
(6) and (2) lane detection, if the vehicle is judged to be a black smoke vehicle, performing lane detection according to coordinates of the vehicle, reading an image under the current video stream, manually drawing a lane line position, calculating a lane line binary linear function slope and intercept, and calculating the central coordinates (x, y) of the black smoke vehicle according to a detection frame position (top, left, right, bottom), wherein the formula is shown in the specification
And
substituting x value to calculate x corresponding to each lane linei(i is 0, 1, 2, … …) and determines that the vehicle is in the second lane, see equation
4. The method for detecting and determining the black smoke vehicle based on the spatiotemporal background comparison as claimed in claim 3, wherein the step of inputting the vehicle region into the license plate detection algorithm specifically comprises: firstly, obtaining a binary image by using a u-net image segmentation algorithm; then, using Opencv to carry out edge detection to obtain the coordinates of the license plate area, and correcting the license plate graph; and finally, carrying out end-to-end identification on the license plate multiple labels by using a convolutional neural network.
5. A computer device, characterized in that the computer device comprises a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to carry out the steps of: the reading of the video stream is to transmit the externally collected road video stream to a local computer of the same address field in a local area network mode for real-time processing through an industrial camera erected in advance; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the stored detection result for rechecking analysis.
6. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of: the reading of the video stream is to transmit the externally collected road video stream to a local computer of the same address field in a local area network mode for real-time processing through an industrial camera erected in advance; extracting key frames and adopting an equal-interval sampling method; detecting video streams, namely detecting black smoke vehicles by adopting a deep learning algorithm YOLOv3, and outputting videos with detection results, target screenshots, license plate numbers, Ringelmann black smoke coefficients and lanes where the black smoke vehicles are located; and outputting and storing the stored detection result for rechecking analysis.
7. A black smoke vehicle detection and judgment system based on space-time background comparison for implementing the black smoke vehicle detection and judgment method based on space-time background comparison according to any one of claims 1 to 4, wherein the black smoke vehicle detection and judgment system based on space-time background comparison comprises:
the video stream acquisition module is used for acquiring a video stream carrying traffic road condition images by an externally connected industrial camera;
the key frame extraction module is used for extracting key frames from the acquired video stream;
the vehicle detection module is used for carrying out common vehicle detection and black smoke vehicle detection on the key frames by adopting a deep learning algorithm, and respectively storing a detection screenshot and a 5-second video containing the black smoke vehicle;
the vehicle information identification module is used for carrying out license plate identification, Ringelmann black smoke coefficient judgment and illegal road number judgment on the black smoke vehicle;
and the detection result analysis module is used for finishing the storage and analysis of the detection result and outputting and storing the black cigarette car video, screenshot and information data result with the detection result.
8. An intelligent traffic control system, wherein the intelligent traffic control system is equipped with the black smoke vehicle detection and determination system based on space-time background comparison according to claim 7.
9. An intelligent analysis system for intelligent traffic, characterized in that the intelligent analysis system for intelligent traffic is equipped with the black smoke vehicle detection and determination system based on space-time background comparison of claim 7.
10. A rapid detection system for intelligent traffic, characterized in that the rapid detection system for intelligent traffic is equipped with the black smoke vehicle detection and determination system based on space-time background comparison of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011053114.4A CN112289022B (en) | 2020-09-29 | 2020-09-29 | Black smoke vehicle detection and judgment method and system based on space-time background comparison |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011053114.4A CN112289022B (en) | 2020-09-29 | 2020-09-29 | Black smoke vehicle detection and judgment method and system based on space-time background comparison |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112289022A true CN112289022A (en) | 2021-01-29 |
| CN112289022B CN112289022B (en) | 2022-02-25 |
Family
ID=74421544
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011053114.4A Active CN112289022B (en) | 2020-09-29 | 2020-09-29 | Black smoke vehicle detection and judgment method and system based on space-time background comparison |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112289022B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113111737A (en) * | 2021-03-26 | 2021-07-13 | 杭州鸿泉物联网技术股份有限公司 | Black smoke blackness detection method and device based on deep learning |
| CN113657305A (en) * | 2021-08-20 | 2021-11-16 | 深圳技术大学 | Video-based intelligent detection method for blackness level of black smoke vehicle and Ringelmann |
| CN117315596A (en) * | 2023-09-07 | 2023-12-29 | 重庆云网科技股份有限公司 | Deep learning-based motor vehicle black smoke detection and identification method |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030072471A1 (en) * | 2001-10-17 | 2003-04-17 | Hitachi, Ltd. | Lane recognition system |
| JP2004078303A (en) * | 2002-08-09 | 2004-03-11 | Mitsubishi Heavy Ind Ltd | Traffic composite monitoring apparatus |
| CA2448365A1 (en) * | 2002-11-12 | 2004-05-12 | Alstom Australia Limited | Detection and identification of vehicles with excess particulates in exhaust gases |
| JP2004184333A (en) * | 2002-12-05 | 2004-07-02 | Denso Corp | Distance measuring apparatus |
| CN106056100A (en) * | 2016-06-28 | 2016-10-26 | 重庆邮电大学 | Vehicle auxiliary positioning method based on lane detection and object tracking |
| CN106781523A (en) * | 2017-01-18 | 2017-05-31 | 安徽庆宇光电科技有限公司 | Black smoke car capturing system based on stereoscopic monitoring mode |
| CN106951821A (en) * | 2016-12-27 | 2017-07-14 | 湘潭大学 | A kind of black smoke car intelligent monitoring recognition methods based on image processing techniques |
| CN207516342U (en) * | 2018-02-12 | 2018-06-19 | 武汉蓓宇环保科技有限公司 | Horizontal motor-vehicle tail-gas remote sense monitoring system |
| CN109064495A (en) * | 2018-09-19 | 2018-12-21 | 东南大学 | A kind of bridge floor vehicle space time information acquisition methods based on Faster R-CNN and video technique |
| CN109255350A (en) * | 2018-08-29 | 2019-01-22 | 南京邮电大学 | A kind of new energy detection method of license plate based on video monitoring |
| CN109300314A (en) * | 2018-11-20 | 2019-02-01 | 北京朗观视觉科技有限公司 | Black smoke vehicle real-time monitoring grasp shoot device and method |
| CN110178167A (en) * | 2018-06-27 | 2019-08-27 | 潍坊学院 | Crossing video frequency identifying method violating the regulations based on video camera collaboration relay |
| CN110660222A (en) * | 2019-11-01 | 2020-01-07 | 河北工业大学 | Intelligent environment-friendly electronic snapshot system for black smoke vehicle on road |
| CN111145556A (en) * | 2019-12-24 | 2020-05-12 | 湖南轩源环保科技有限公司 | Black smoke vehicle license plate recognition method, system, equipment and readable storage medium |
| CN111241343A (en) * | 2020-01-07 | 2020-06-05 | 西安电子科技大学 | Road information monitoring and analyzing detection method and intelligent traffic control system |
-
2020
- 2020-09-29 CN CN202011053114.4A patent/CN112289022B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030072471A1 (en) * | 2001-10-17 | 2003-04-17 | Hitachi, Ltd. | Lane recognition system |
| JP2004078303A (en) * | 2002-08-09 | 2004-03-11 | Mitsubishi Heavy Ind Ltd | Traffic composite monitoring apparatus |
| CA2448365A1 (en) * | 2002-11-12 | 2004-05-12 | Alstom Australia Limited | Detection and identification of vehicles with excess particulates in exhaust gases |
| JP2004184333A (en) * | 2002-12-05 | 2004-07-02 | Denso Corp | Distance measuring apparatus |
| CN106056100A (en) * | 2016-06-28 | 2016-10-26 | 重庆邮电大学 | Vehicle auxiliary positioning method based on lane detection and object tracking |
| CN106951821A (en) * | 2016-12-27 | 2017-07-14 | 湘潭大学 | A kind of black smoke car intelligent monitoring recognition methods based on image processing techniques |
| CN106781523A (en) * | 2017-01-18 | 2017-05-31 | 安徽庆宇光电科技有限公司 | Black smoke car capturing system based on stereoscopic monitoring mode |
| CN207516342U (en) * | 2018-02-12 | 2018-06-19 | 武汉蓓宇环保科技有限公司 | Horizontal motor-vehicle tail-gas remote sense monitoring system |
| CN110178167A (en) * | 2018-06-27 | 2019-08-27 | 潍坊学院 | Crossing video frequency identifying method violating the regulations based on video camera collaboration relay |
| CN109255350A (en) * | 2018-08-29 | 2019-01-22 | 南京邮电大学 | A kind of new energy detection method of license plate based on video monitoring |
| CN109064495A (en) * | 2018-09-19 | 2018-12-21 | 东南大学 | A kind of bridge floor vehicle space time information acquisition methods based on Faster R-CNN and video technique |
| CN109300314A (en) * | 2018-11-20 | 2019-02-01 | 北京朗观视觉科技有限公司 | Black smoke vehicle real-time monitoring grasp shoot device and method |
| CN110660222A (en) * | 2019-11-01 | 2020-01-07 | 河北工业大学 | Intelligent environment-friendly electronic snapshot system for black smoke vehicle on road |
| CN111145556A (en) * | 2019-12-24 | 2020-05-12 | 湖南轩源环保科技有限公司 | Black smoke vehicle license plate recognition method, system, equipment and readable storage medium |
| CN111241343A (en) * | 2020-01-07 | 2020-06-05 | 西安电子科技大学 | Road information monitoring and analyzing detection method and intelligent traffic control system |
Non-Patent Citations (4)
| Title |
|---|
| HUANJIE TAO 等: "Smoky Vehicle Detection Based on Range Filtering on Three Orthogonal Planes and Motion Orientation Histogram", 《IEEEACESS》 * |
| 丁媚英: "佛山:全天候禁行黑烟车", 《环境》 * |
| 于兴智等: "工业烟气黑度监测***工程技术研究", 《真空电子技术》 * |
| 包信宗 等: "基于 DSP及ANN的烟气林格曼黑度监测研究", 《PROCEEDINGS OF THE 8TH WORLD CONGRESS ON INTELLIGENT CONTROL AND AUTOMATION》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113111737A (en) * | 2021-03-26 | 2021-07-13 | 杭州鸿泉物联网技术股份有限公司 | Black smoke blackness detection method and device based on deep learning |
| CN113657305A (en) * | 2021-08-20 | 2021-11-16 | 深圳技术大学 | Video-based intelligent detection method for blackness level of black smoke vehicle and Ringelmann |
| CN113657305B (en) * | 2021-08-20 | 2023-08-04 | 深圳技术大学 | Video-based intelligent detection method for black smoke vehicle and ringeman blackness level |
| CN117315596A (en) * | 2023-09-07 | 2023-12-29 | 重庆云网科技股份有限公司 | Deep learning-based motor vehicle black smoke detection and identification method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112289022B (en) | 2022-02-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112289022B (en) | Black smoke vehicle detection and judgment method and system based on space-time background comparison | |
| CN111368687B (en) | Sidewalk vehicle illegal parking detection method based on target detection and semantic segmentation | |
| CN109829403B (en) | Vehicle anti-collision early warning method and system based on deep learning | |
| CN111080620B (en) | Road disease detection method based on deep learning | |
| CN103279756B (en) | Vehicle detection based on integrated classifier analyzes system and determination method thereof | |
| He et al. | Obstacle detection of rail transit based on deep learning | |
| CN108509954A (en) | A kind of more car plate dynamic identifying methods of real-time traffic scene | |
| CN106934374B (en) | Method and system for identifying traffic signboard in haze scene | |
| CN111209923A (en) | Deep learning technology-based muck truck cover or uncover identification method | |
| CN113537211B (en) | Asymmetric IOU-based deep learning license plate frame positioning method | |
| CN112597928B (en) | Event detection method and related device | |
| CN112990004A (en) | Black smoke vehicle detection method based on optical flow method and deep learning convolutional neural network | |
| CN111523419A (en) | Video detection method and device for motor vehicle exhaust emission | |
| CN106650752A (en) | Vehicle body color recognition method | |
| CN113657305B (en) | Video-based intelligent detection method for black smoke vehicle and ringeman blackness level | |
| Tao et al. | A three-stage framework for smoky vehicle detection in traffic surveillance videos | |
| CN114842285A (en) | Roadside berth number identification method and device | |
| CN110765900A (en) | DSSD-based automatic illegal building detection method and system | |
| CN115187886A (en) | Vehicle violation detection method and device and electronic equipment | |
| CN115512315A (en) | Non-motor vehicle child riding detection method, electronic device and storage medium | |
| CN112016534B (en) | Neural network training method for vehicle parking violation detection, detection method and device | |
| CN114820676A (en) | Equipment running state identification method and device | |
| CN109145732B (en) | Black smoke vehicle detection method based on Gabor projection | |
| CN114005097A (en) | Train operation environment real-time detection method and system based on image semantic segmentation | |
| Li et al. | An Image Data Augmentation Algorithm Based on YOLOv5s-DA for Pavement Distress Detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |