CN115457780B - Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set - Google Patents
Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set Download PDFInfo
- Publication number
- CN115457780B CN115457780B CN202211082389.XA CN202211082389A CN115457780B CN 115457780 B CN115457780 B CN 115457780B CN 202211082389 A CN202211082389 A CN 202211082389A CN 115457780 B CN115457780 B CN 115457780B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- calculating
- priori knowledge
- knowledge set
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000001514 detection method Methods 0.000 claims abstract description 59
- 230000033001 locomotion Effects 0.000 claims abstract description 21
- 238000006073 displacement reaction Methods 0.000 claims abstract description 19
- 238000004364 calculation method Methods 0.000 claims abstract description 15
- 238000012937 correction Methods 0.000 claims abstract description 14
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000003384 imaging method Methods 0.000 claims abstract description 13
- 238000003708 edge detection Methods 0.000 claims abstract description 11
- 239000011159 matrix material Substances 0.000 claims description 19
- 238000005259 measurement Methods 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 6
- 229910003460 diamond Inorganic materials 0.000 claims description 5
- 239000010432 diamond Substances 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000013138 pruning Methods 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims 1
- 230000009466 transformation Effects 0.000 claims 1
- 238000013519 translation Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 206010034701 Peroneal nerve palsy Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/065—Traffic control systems for road vehicles by counting the vehicles in a section of the road or in a parking area, i.e. comparing incoming count with outgoing count
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/052—Detecting movement of traffic to be counted or controlled with provision for determining speed or overspeed
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an automatic vehicle flow and velocity measuring and calculating method based on a priori knowledge set, which comprises the following steps: step S1: acquiring a video frame by using a road monitoring camera, and inputting a target detection model to obtain a detection frame and a vehicle type of a vehicle; s2: performing multi-target tracking on all vehicles based on the detection frame, and distributing unique IDs for each vehicle; s3: performing edge detection on the video frame, and constructing a priori knowledge set based on the vehicle target detection and tracking results; s4: based on a priori knowledge set, the automatic correction of camera parameters is realized by utilizing an evolutionary algorithm or a method for directly modeling an imaging principle; s5: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the vehicle movement time by using the video fixed frame rate, and calculating the vehicle speed. The method provided by the invention can construct a priori knowledge set for automatic correction of the camera parameters, thereby realizing automatic calculation of the vehicle flow and the flow velocity.
Description
Technical Field
The invention relates to the field of intelligent traffic and image recognition, in particular to a vehicle flow and velocity automatic measuring and calculating method and system based on a priori knowledge set.
Background
The traffic flow refers to the number of vehicles passing through a certain section, a short section or a certain road section within a certain time range, and under the background of an intelligent traffic system, traffic signal lamp control, urban road planning, navigation based on real-time flow and the like can be performed based on flow space-time information, so that the method is particularly important for all-round all-weather real-time flow monitoring of a traffic area. For a specific vehicle, the flow rate refers to the distance travelled by the vehicle within a certain time, and the flow rate is taken as a physical quantity representing microscopic attributes of the vehicle, and plays an important role in traffic supervision, such as road abnormal condition detection, illegal overspeed evidence collection, congestion road section detection and the like.
The traditional method for calculating the statistical flow rate is mostly based on hardware equipment which is arranged in advance, including coils, radars, door frames and the like, and the technologies are relatively mature and popular. However, radar speed measurement is expensive and is easy to be reversely detected by the electronic dog; the induction coil speed measurement can only be used for detecting a fixed road section, has a certain damage to the road surface, is difficult to be laid in a large range, and has large data fluctuation; laser speed measurement accuracy is higher but data repeatability is poor and the price is high.
In recent years, along with the continuous development of computer vision technology and the increase of the number of road monitoring cameras, a method for directly counting traffic and measuring and calculating speed by analyzing monitoring video data is increasingly receiving attention. The method for realizing flow and flow velocity measurement based on the data of the monitoring camera is free from other sensors, and has low cost, but the current method mostly needs to manually calibrate the parameters of the camera in advance, and once the position of the camera changes or the angle deviates, the camera needs to be recalibrated, so that the automation is difficult to realize.
Disclosure of Invention
In order to solve the technical problems, the invention provides a vehicle flow rate automatic measuring and calculating method and system based on a priori knowledge set.
The technical scheme of the invention is as follows: an automatic vehicle flow and velocity measuring and calculating method based on a priori knowledge set comprises the following steps:
Step S1: acquiring continuous video frame data by using a road monitoring camera, and inputting a pre-trained target detection model to obtain detection frames and vehicle types of all vehicles in a scene in each frame;
Step S2: utilizing a tracking algorithm and an IOU matching rule of continuous frames to carry out multi-target tracking on all vehicles in the video frame based on the detection frame, and distributing unique ID (identity) for each vehicle;
Step S3: performing edge detection on the video frame, and constructing a priori knowledge set based on vehicle target detection and tracking results, wherein the priori knowledge set comprises: image coordinates of vanishing points in the scene; image coordinates of the lane lines, line segments representing real distances of widths of adjacent lanes, and image coordinates and real sizes of the dynamic targets;
step S4: based on the priori knowledge set, selecting the priori knowledge suitable for the current road traffic scene, and realizing automatic correction of camera parameters by using an evolutionary algorithm or a method for directly modeling an imaging principle;
step S5: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow indexes in a specific area and a specific time period; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the vehicle movement time by using the video fixed frame rate, and calculating the vehicle instantaneous speed and the average speed based on the displacement and the movement time.
Compared with the prior art, the invention has the following advantages:
The invention discloses a vehicle flow and velocity automatic measuring and calculating method based on a priori knowledge set, which is used for constructing a priori knowledge set for automatic correction of camera parameters based on road monitoring camera data, wherein the application scene of the priori knowledge set is wide, the limitation that manual intervention is required for current camera calibration is solved, and the information acquisition, transmission and calculation cost is reduced; based on a target detection algorithm and a multi-target tracking result, the automatic measurement and calculation of flow and flow velocity parameters are realized by combining the mapping relation from the corrected image coordinates to the real world coordinates.
Drawings
FIG. 1 is a flow chart of a method for automatically measuring and calculating traffic flow and velocity based on a priori knowledge set in an embodiment of the invention;
FIG. 2A is a schematic view of a first vanishing point according to an embodiment of the invention;
FIG. 2B is a schematic view of a second vanishing point according to an embodiment of the invention;
fig. 3 is a schematic diagram of a calculation principle of a rotation matrix and a translation matrix of a camera according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of a method for automatically measuring and calculating the flow rate of traffic flow based on a priori knowledge set in an embodiment of the invention;
fig. 5 is a block diagram of a vehicle flow rate automatic measurement system based on a priori knowledge set in an embodiment of the invention.
Detailed Description
The invention provides a vehicle flow and velocity automatic measuring and calculating method based on a priori knowledge set, which constructs the priori knowledge set for automatic correction of camera parameters, realizes automatic measurement and calculation of the vehicle flow and velocity according to the priori knowledge set, does not need manual intervention for calibration, and has higher robustness, real-time performance and accuracy.
The present invention will be further described in detail below with reference to the accompanying drawings by way of specific embodiments in order to make the objects, technical solutions and advantages of the present invention more apparent.
Example 1
As shown in fig. 1, the method for automatically measuring and calculating the flow rate of the vehicle flow based on the priori knowledge set provided by the embodiment of the invention comprises the following steps:
Step S1: acquiring continuous video frame data by using a road monitoring camera, and inputting a pre-trained target detection model to obtain detection frames and vehicle types of all vehicles in a scene in each frame;
step S2: utilizing a tracking algorithm and an IOU matching rule of continuous frames to carry out multi-target tracking on all vehicles in the video frame based on the detection frame, and distributing unique ID (identity) for each vehicle;
Step S3: and performing edge detection on the video frame, and constructing a priori knowledge set based on the vehicle target detection and tracking result, wherein the priori knowledge set comprises: image coordinates of vanishing points in the scene; image coordinates of the lane lines, line segments representing real distances of widths of adjacent lanes, and image coordinates and real sizes of the dynamic targets;
Step S4: based on the priori knowledge set, selecting priori knowledge suitable for the current road traffic scene, and realizing automatic correction of camera parameters by using an evolutionary algorithm or a method for directly modeling an imaging principle;
Step S5: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow indexes in a specific area and a specific time period; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the movement time of the vehicle by using the video fixed frame rate, and calculating the instantaneous speed and the average speed of the vehicle based on the displacement and the movement time.
In one embodiment, step S1 described above: the method comprises the steps of obtaining continuous video frame data by using a road monitoring camera, inputting a pre-trained target detection model, and obtaining detection frames and vehicle types of all vehicles in a scene in each frame, wherein the detection frames and the vehicle types specifically comprise:
The road monitoring camera is used for acquiring continuous video frame data and inputting a trained target detection model, YOLOv is adopted in the embodiment of the invention, the continuous video frame data is processed to obtain all vehicle detection frames in each frame, so that the image coordinates of the vehicle position are acquired, and meanwhile, the types corresponding to the vehicle, such as a car, a truck and the like, are output.
In one embodiment, step S2 above: utilizing a tracking algorithm and an IOU matching rule of continuous frames to carry out multi-target tracking on all vehicles in a video frame based on a detection frame, and distributing unique ID (identity) for each vehicle, wherein the method specifically comprises the following steps:
According to the embodiment of the invention, deepSort tracking algorithm is adopted to calculate the difference between the predicted detection frame and the predicted feature vector and the actual detection frame and the actual feature vector, and the corresponding relation of the same object detection frame among different frames is calculated. Because the scene of the invention is simpler, the automobile movement mode is more single, the calculation of the cross ratio of different detection frames between adjacent frames is added on the basis of DeepSort, the effect of multi-target detection can be assisted and improved, and each vehicle is allocated with a unique ID for identifying the identity, thereby realizing multi-target tracking.
In one embodiment, step S3 described above: performing edge detection on the video frame, and constructing a priori knowledge set based on the vehicle target detection and tracking result, wherein the priori knowledge set comprises: image coordinates of vanishing points in the scene; the image coordinates of the lane lines and the line segments representing the real distance between the widths of the adjacent lanes, and the image coordinates and the real size of the dynamic object specifically comprise:
1. constructing image coordinates of vanishing points in a scene, including the steps of:
Step S301: based on the detection and tracking results of the vehicle target, a short-time motion track of the vehicle is obtained, hough transform (Hough transform) based on parallel coordinates (parallel coordinates) is used for the motion track, the motion track is mapped into diamond space (diamond space) from an image space, the image coordinate of a first vanishing point is determined by calculating the position with the maximum intersection point, and the first vanishing point is consistent with the direction of a lane line;
as shown in fig. 2A, each lane line intersects at a first vanishing point (direction indicated by black arrow) outside the image;
Step S302: any one frame of image frames in the continuous video frames of the current camera is taken, the edges of the image frames are obtained through a Canny edge detection algorithm, the edges of the background, the edges consistent with the direction of the first vanishing point and the edges in the vertical direction are filtered and deleted, the position coordinates of the most intersection points of the rest edges in the diamond space, namely the second vanishing point image coordinates, are calculated by adopting the same method as that in the step S301, and the second vanishing point is consistent with the vertical direction of the lane line;
As shown in fig. 2B, the second vanishing point is located at the intersection of the line segments in the direction perpendicular to the lane line (as indicated by the gray arrow).
2. Constructing image coordinates of lane lines and line segments representing real distances of widths of adjacent lanes, comprising the following steps:
step S311: any frame of image frames in the vehicle video frames is taken, the edges of the image frames are obtained through a Canny edge detection algorithm, background edges are filtered, polygonal fitting is carried out on profile data, profiles which can be fitted by convex quadrilaterals are extracted, and a lane line candidate set is generated;
step S312: the lane line candidate set is screened according to pruning conditions, and the lane line set is obtained, wherein the pruning conditions comprise but are not limited to:
e) The size of the candidate lane line pixel region is within a preset range, for example, the pixel region is within a (50, 180) range;
f) The contour center of the candidate lane line is not overlapped with other objects;
g) The contour direction of the candidate lane line and the angle of the longitudinal axis of the image frame are not smaller than a preset angle, for example, the angle of the contour direction of the candidate lane line and the angle of the longitudinal axis of the image frame are not smaller than 15 degrees;
h) The inverse value of the Euclidean distance between the average RGB value of the candidate lane line and white (255 ) is larger than a preset threshold value, and the larger the inverse value is, the higher the probability that the candidate lane line is a real lane line is;
Step S313: any point on the obtained lane line is taken, the connection line between the point and the second vanishing point is obtained, the image coordinates of the intersection point of the two adjacent lane lines are determined according to the connection line, and the pixel distance of the two adjacent intersection points is calculated to obtain the real distance of the width of the two adjacent lanes; taking a connecting line between intersection point coordinates of two adjacent lane lines as a line segment for representing the real distance of the width of the adjacent lane, and adding a priori knowledge set;
Step S314: step S313 is repeated, and a plurality of sets of lane line image coordinates and pixel distances representing adjacent intersection points of the lane widths are calculated to form image coordinates of the lane lines in the prior knowledge set and a plurality of line segments representing real distances of the adjacent lane widths.
3. The method for constructing the image coordinates and the real size of the dynamic target comprises the following steps:
Step S321: based on the edges of the image frames in step S302, edges in the vertical direction of the vehicle are obtained;
step S322: and combining the detection frame of the vehicle, obtaining the edge of the vehicle in the vertical direction within the range of the detection frame, calculating the pixel length, and determining the real length, width and height of the vehicle according to the detected type of the vehicle.
In one embodiment, step S4 above: based on the priori knowledge set, selecting the priori knowledge suitable for the current road traffic scene, and realizing automatic correction of camera parameters by using an evolutionary algorithm or a method for directly modeling an imaging principle, wherein the method specifically comprises the following steps:
based on the priori knowledge set, the priori knowledge applicable to the current road traffic scene is selected and divided into the following two cases:
a) If the priori knowledge set contains the image coordinates of two vanishing points and any line segment with a known distance, the camera parameter matrix P can be obtained by a method of directly modeling an imaging principle based on the assumption that a main point is positioned in the center of an image;
As shown in fig. 3, the internal parameters and the external parameters of the camera are calculated based on the assumption that the principal point is located at the center of the image, respectively.
Firstly, the focal length f of the camera is calculated according to the following formula, namely an internal reference:
Wherein V 1、V2 is the projection point of the vanishing point on the imaging plane in two orthogonal directions, O c is the center of the camera, O i is the projection point of the center of the camera on the imaging plane, and V i is the foot drop of the straight line passing through O i on the straight line V iV2.
Considering that two vanishing points are in the directions of two orthogonal axes of a world coordinate system and take O w as a center, all parallel lines intersect at one vanishing point, a vector relation is established, as shown in the following formula:
Zc′＝Xc′×Yc′
secondly, calculating a rotation matrix R and a translation matrix T of a camera coordinate system, namely, external parameters:
For the two end points (P 1,P2) of the line segment with known length obtained by detection, the coordinate of the line segment on the plane of the camera is P 1m,P2m. Vector Translation such that P 1m coincides with P 1px gives/>Connecting O cP2e, namely intersecting the plane of the camera with P 2px Translating past lines to Q, and calculating to obtain a translation matrix/>
And multiplying the rotation matrix R and the translation matrix T to obtain a parameter matrix P of the camera.
B) If the priori knowledge set contains the image coordinates of two vanishing points and any line segment with a known distance, the camera parameter matrix P can be obtained by a method of directly modeling an imaging principle based on the assumption that a main point is positioned in the center of an image;
If the priori knowledge set contains a plurality of groups of static or dynamic image coordinates and line segments representing real distances, the evolutionary algorithm can be utilized to directly carry out optimization solution on the camera parameter matrix P, and the method comprises the following steps:
Step S41: taking a plurality of known image coordinates and line segments representing real distances from the priori knowledge set as the priori knowledge;
Step S42: initializing a parameter matrix P of the camera, and carrying out mapping calculation from image coordinates to world coordinates on the line segments in the step S41 through the P, so as to obtain the length of the line segments under the world coordinate system through calculation;
Step S43: optimizing P by taking the sum of the difference between the length of all line segments in the prior knowledge set mapped to the world coordinate system and the real length in the prior knowledge set as a target:
Wherein N is the number of line segments in the priori knowledge set, P k,Qk is the coordinates of the end point of the kth line segment in the world coordinate system, and P k,qk is reversely mapped to the coordinates in the world coordinate system through the parameter matrix P respectively;
For the corresponding image coordinates of P k,Qk in the image frame,/>
And (3) carrying out optimization operation by adopting a distribution estimation algorithm (EDA) until the sum of errors is smaller than a threshold value, and stopping optimization, wherein a final camera parameter matrix P is obtained.
In one embodiment, the step S5 is as follows: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow indexes in a specific area and a specific time period; calculating the displacement of the vehicle by using the corrected camera parameters, calculating the movement time of the vehicle by using the video fixed frame rate, and calculating the instantaneous speed and the average speed of the vehicle based on the displacement and the movement time, wherein the method specifically comprises the following steps:
step S51: dividing a statistical region according to the driving direction, the lane and a specific interval; determining a time span of statistical flow, e.g., seconds, minutes, hours, days, months, years, etc.;
step S52: based on the statistical region and the time span, counting a flow result;
step S53: calculating a time interval between two or more consecutive frames according to the known video frame rate;
Step S54: based on the image coordinates of the vehicle detection frame and according to the camera parameters, converting the image coordinates into real world coordinates, and calculating the vehicle displacement;
Step S55: based on the time interval and the vehicle displacement, the instantaneous speed of the vehicle between two consecutive frames or the average speed between several frames is calculated.
Fig. 4 shows a flow chart of a method for automatically measuring and calculating the flow rate of the vehicle flow based on a priori knowledge set.
The invention discloses a vehicle flow and velocity automatic measuring and calculating method based on a priori knowledge set, which is used for constructing a priori knowledge set for automatic correction of camera parameters based on road monitoring camera data, wherein the application scene of the priori knowledge set is wide, the limitation that manual intervention is required for current camera calibration is solved, and the information acquisition, transmission and calculation cost is reduced; based on a target detection algorithm and a multi-target tracking result, the automatic measurement and calculation of flow and flow velocity parameters are realized by combining the mapping relation from the corrected image coordinates to the real world coordinates.
Example two
As shown in fig. 5, an embodiment of the present invention provides an automatic vehicle flow and velocity measurement system based on a priori knowledge set, which includes the following modules:
the target detection module is used for acquiring continuous video frame data by using the road monitoring camera, inputting a pre-trained target detection model, and obtaining detection frames and vehicle types of all vehicles in each frame;
The multi-target tracking module is used for carrying out multi-target tracking on all vehicles in the video frame based on the detection frame by utilizing a tracking algorithm and an IOU matching rule of the continuous frame, and distributing an unique ID (identity) for each vehicle;
And a prior knowledge set constructing module, which is used for carrying out edge detection on the video frame and constructing a prior knowledge set based on the vehicle target detection and tracking result, wherein the prior knowledge set comprises: image coordinates of vanishing points in the scene; image coordinates of the lane lines, line segments representing real distances of widths of adjacent lanes, and image coordinates and real sizes of the dynamic targets;
The camera parameter correction module is used for selecting priori knowledge applicable to the current road traffic scene based on a priori knowledge set, and realizing automatic correction of camera parameters by utilizing an evolutionary algorithm or a method for directly modeling an imaging principle;
The vehicle flow and flow velocity statistics module is used for counting vehicle flow indexes in a specific area and a specific time period based on vehicle target detection and multi-target tracking results; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the movement time of the vehicle by using the video fixed frame rate, and calculating the instantaneous speed and the average speed of the vehicle based on the displacement and the movement time.
The above examples are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalents and modifications that do not depart from the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (7)
1. An automatic vehicle flow and velocity measuring and calculating method based on a priori knowledge set is characterized by comprising the following steps:
Step S1: acquiring continuous video frame data by using a road monitoring camera, and inputting a pre-trained target detection model to obtain detection frames and vehicle types of all vehicles in each frame;
Step S2: utilizing a tracking algorithm and an IOU matching rule of continuous frames to carry out multi-target tracking on all vehicles in the video frame based on the detection frame, and distributing unique ID (identity) for each vehicle;
Step S3: performing edge detection on the video frame, and constructing a priori knowledge set based on vehicle target detection and tracking results, wherein the priori knowledge set comprises: image coordinates of vanishing points in the scene; image coordinates of the lane lines, line segments representing real distances of widths of adjacent lanes, and image coordinates and real sizes of the dynamic targets;
step S4: based on the priori knowledge set, selecting the priori knowledge suitable for the current road traffic scene, and realizing automatic correction of camera parameters by using an evolutionary algorithm or a method for directly modeling an imaging principle;
step S5: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow indexes in a specific area and a specific time period; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the vehicle movement time by using the video fixed frame rate, and calculating the vehicle instantaneous speed and the average speed based on the displacement and the movement time.
2. The method for automatically measuring and calculating the vehicle flow rate based on the a priori knowledge set according to claim 1, wherein the constructing the image coordinates of the vanishing point in the scene in the step S3 specifically includes:
step S301: based on the detection and tracking results of the vehicle target, obtaining a short-time motion track of the vehicle, mapping the motion track into a diamond space from an image space by using Hough transformation based on parallel coordinates, and determining a first vanishing point image coordinate by calculating the position with the most intersection point, wherein the first vanishing point is consistent with the direction of a lane line;
Step S302: any image frame in the continuous video frames of the current camera is taken, the edge of the image frame is obtained through a Canny edge detection algorithm, the background edge, the edge consistent with the direction of the first vanishing point and the edge in the vertical direction are filtered and deleted, the same method as that in the step S301 is adopted, the position coordinate with the largest intersection point of the rest edge in the diamond space, namely the image coordinate of the second vanishing point, is calculated, and the second vanishing point is consistent with the vertical direction of the lane line.
3. The method for automatically measuring and calculating the traffic flow and the flow rate based on the priori knowledge set according to claim 2, wherein the step S3 is to construct the image coordinates of the lane lines and the line segments representing the actual distance between the widths of the adjacent lanes, and specifically comprises the steps of:
Step S311: any one image frame in the vehicle video frames is taken, the edges of the image frames are obtained through a Canny edge detection algorithm, background edges are filtered, polygonal fitting is carried out on profile data, profiles which can be fitted by convex quadrilaterals are extracted, and a lane line candidate set is generated;
Step S312: screening the lane line candidate set according to pruning conditions to obtain a lane line set, wherein the pruning conditions comprise but are not limited to:
a) The size of the candidate lane line pixel area is within a preset range;
b) The contour center of the candidate lane line is not overlapped with other objects;
c) The angle between the contour direction of the candidate lane line and the longitudinal axis of the image frame is not smaller than a preset angle;
d) The inverse value of the average RGB value of the candidate lane line and the white Euclidean distance is larger than a preset threshold value;
Step S313: any point on the obtained lane line is taken, the connection line between the point and the second vanishing point is obtained, the image coordinates of the intersection point of the two adjacent lane lines are determined according to the connection line, and the pixel distance of the two adjacent intersection points is calculated to obtain the real distance of the width of the two adjacent lanes; taking a connecting line between intersection point coordinates of two adjacent lane lines as a line segment for representing the real distance of the width of the adjacent lane, and adding the line segment into the priori knowledge set;
Step S314: and repeating the step S313, calculating to obtain a plurality of groups of lane line image coordinates and pixel distances of adjacent intersection points representing the width of the lane, and forming the image coordinates of the lane lines in the prior knowledge set and a plurality of line segments representing the actual distances of the width of the adjacent lane.
4. The method for automatically measuring and calculating the vehicle flow rate based on the priori knowledge set according to claim 3, wherein the step S3 of constructing the image coordinates and the real size of the dynamic object specifically includes:
step S321: obtaining edges in the vertical direction of the vehicle based on the edges of the image frames in the step S302;
Step S322: and combining the detection frame of the vehicle to obtain the edge in the vertical direction of the vehicle within the range of the detection frame, calculating the pixel length, and determining the real length, width and height of the vehicle according to the detected type of the vehicle.
5. The method for automatic measurement and calculation of vehicle flow rate based on a priori knowledge set according to claim 4, wherein said step S4: based on the priori knowledge set, the priori knowledge suitable for the current road traffic scene is selected, and the automatic correction of the camera parameters is realized by using an evolutionary algorithm or a method for directly modeling an imaging principle, which comprises the following steps:
based on the prior knowledge set, prior knowledge applicable to the current road traffic scene is selected:
if the prior knowledge set contains the image coordinates of two vanishing points and any line segment with a known distance, the camera parameter matrix P can be obtained by calculation based on the assumption that the main point is positioned in the center of the image by a method of directly modeling an imaging principle;
if the priori knowledge set contains a plurality of groups of static or dynamic image coordinates and line segments representing real distances, the evolutionary algorithm can be utilized to directly carry out optimization solution on the camera parameter matrix P, and the method comprises the following steps:
Step S41: taking a plurality of known image coordinates and line segments representing real distances from the priori knowledge set as priori knowledge;
Step S42: initializing a parameter matrix P of the camera, and carrying out mapping calculation from image coordinates to world coordinates on the line segments in the step S41 through the P, so as to obtain the length of the line segments under the world coordinate system through calculation;
step S43: optimizing P by taking the sum of the difference value of the length of all line segments mapped to the world coordinate system in the prior knowledge set and the real length in the prior knowledge set as a target:
Wherein N is the number of line segments in the priori knowledge set, P k,Qk is the coordinates of the end point of the kth line segment in the world coordinate system, and P k,qk is reversely mapped to the coordinates in the world coordinate system through the parameter matrix P respectively;
For the corresponding image coordinates of P k,Qk in the image frame,/>
And (3) carrying out optimization operation by adopting a distribution estimation algorithm until the sum of errors is smaller than a threshold value, and stopping optimization, so as to obtain a final camera parameter matrix P.
6. The method for automatic measurement and calculation of vehicle flow rate based on a priori knowledge set according to claim 5, wherein said step S5: based on the vehicle target detection and multi-target tracking results, counting the vehicle flow indexes in a specific area and a specific time period; calculating the displacement of the vehicle by using the corrected camera parameters, calculating the vehicle movement time by using the video fixed frame rate, and calculating the vehicle instantaneous speed and the average speed based on the displacement and the movement time, wherein the method specifically comprises the following steps:
Step S51: dividing a statistical region according to the driving direction, the lane and a specific interval; determining a time span of the statistical flow;
step S52: based on the statistical region and the time span, counting a flow result;
step S53: calculating a time interval between two or more consecutive frames according to the known video frame rate;
Step S54: based on the image coordinates of the vehicle detection frame and according to the camera parameters, converting the image coordinates into real world coordinates, and calculating the vehicle displacement;
step S55: based on the time interval and the vehicle displacement, calculating the instantaneous speed of the vehicle between two continuous frames or the average speed between a plurality of frames.
7. An automatic vehicle flow and velocity measuring and calculating system based on a priori knowledge set is characterized by comprising the following modules:
the target detection module is used for acquiring continuous video frame data by using the road monitoring camera, inputting a pre-trained target detection model, and obtaining detection frames and vehicle types of all vehicles in each frame;
The multi-target tracking module is used for carrying out multi-target tracking on all vehicles in the video frame based on the detection frame by utilizing a tracking algorithm and an IOU matching rule of the continuous frame, and distributing an unique ID (identity) of each vehicle;
And a prior knowledge set constructing module, configured to perform edge detection on the video frame, and construct a prior knowledge set based on a vehicle target detection and tracking result, where the prior knowledge set includes: image coordinates of vanishing points in the scene; image coordinates of the lane lines, line segments representing real distances of widths of adjacent lanes, and image coordinates and real sizes of the dynamic targets;
The camera parameter correction module is used for selecting priori knowledge applicable to the current road traffic scene based on the priori knowledge set, and realizing automatic correction of camera parameters by using an evolutionary algorithm or a method for directly modeling an imaging principle;
The vehicle flow and flow velocity statistics module is used for counting vehicle flow indexes in a specific area and a specific time period based on vehicle target detection and multi-target tracking results; and calculating the displacement of the vehicle by using the corrected camera parameters, calculating the vehicle movement time by using the video fixed frame rate, and calculating the vehicle instantaneous speed and the average speed based on the displacement and the movement time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211082389.XA CN115457780B (en) | 2022-09-06 | 2022-09-06 | Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211082389.XA CN115457780B (en) | 2022-09-06 | 2022-09-06 | Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115457780A CN115457780A (en) | 2022-12-09 |
| CN115457780B true CN115457780B (en) | 2024-06-14 |
Family
ID=84302761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211082389.XA Active CN115457780B (en) | 2022-09-06 | 2022-09-06 | Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115457780B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117953688A (en) * | 2024-02-18 | 2024-04-30 | 广东云达智能物联科技有限公司 | Smart city traffic flow monitoring system based on Internet of things |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110033479A (en) * | 2019-04-15 | 2019-07-19 | 四川九洲视讯科技有限责任公司 | Traffic flow parameter real-time detection method based on Traffic Surveillance Video |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5404306A (en) * | 1994-04-20 | 1995-04-04 | Rockwell International Corporation | Vehicular traffic monitoring system |
| CN110532896B (en) * | 2019-08-06 | 2022-04-08 | 北京航空航天大学 | Road vehicle detection method based on fusion of road side millimeter wave radar and machine vision |
| CN111145545B (en) * | 2019-12-25 | 2021-05-28 | 西安交通大学 | Road traffic behavior unmanned aerial vehicle monitoring system and method based on deep learning |
| US11682297B2 (en) * | 2020-03-31 | 2023-06-20 | Flir Systems Trading Belgium Bvba | Real-time scene mapping to GPS coordinates in traffic sensing or monitoring systems and methods |
| KR102283483B1 (en) * | 2021-02-19 | 2021-07-29 | 주식회사 싸인텔레콤 | Image analysis system for calculating traffic volume |
-
2022
- 2022-09-06 CN CN202211082389.XA patent/CN115457780B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110033479A (en) * | 2019-04-15 | 2019-07-19 | 四川九洲视讯科技有限责任公司 | Traffic flow parameter real-time detection method based on Traffic Surveillance Video |
Non-Patent Citations (1)
| Title |
|---|
| 基于YOLOv3的多车道车流量统计及车辆跟踪方法;汪辉;高尚兵;周君;周建;张莉雯;;国外电子测量技术;20200215(第02期);47-51 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115457780A (en) | 2022-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111563442B (en) | Slam method and system for fusing point cloud and camera image data based on laser radar | |
| CN107738612B (en) | Automatic parking space detection and identification system based on panoramic vision auxiliary system | |
| CN111666805B (en) | Class marking system for autopilot | |
| CN110569704A (en) | Multi-strategy self-adaptive lane line detection method based on stereoscopic vision | |
| CN104766058A (en) | Method and device for obtaining lane line | |
| CN110197173B (en) | Road edge detection method based on binocular vision | |
| CN110379168A (en) | A kind of vehicular traffic information acquisition method based on Mask R-CNN | |
| Dawson et al. | An energy minimization approach to automatic traffic camera calibration | |
| CN116503818A (en) | Multi-lane vehicle speed detection method and system | |
| CN115457780B (en) | Vehicle flow and velocity automatic measuring and calculating method and system based on priori knowledge set | |
| US20220245831A1 (en) | Speed estimation systems and methods without camera calibration | |
| CN117197019A (en) | Vehicle three-dimensional point cloud image fusion method and system | |
| CN111353481A (en) | Road obstacle identification method based on laser point cloud and video image | |
| CN116901089A (en) | Multi-angle vision distance robot control method and system | |
| CN116385994A (en) | Three-dimensional road route extraction method and related equipment | |
| CN116311903A (en) | Method for evaluating road running index based on video analysis | |
| CN116794650A (en) | Millimeter wave radar and camera data fusion target detection method and device | |
| CN115984772A (en) | Road ponding detection method and terminal based on video monitoring | |
| CN113091693B (en) | Monocular vision long-range distance measurement method based on image super-resolution technology | |
| CN115984768A (en) | Multi-target pedestrian real-time detection positioning method based on fixed monocular camera | |
| CN115018926A (en) | Method, device and equipment for determining pitch angle of vehicle-mounted camera and storage medium | |
| CN114820931A (en) | Virtual reality-based CIM (common information model) visual real-time imaging method for smart city | |
| CN116917936A (en) | External parameter calibration method and device for binocular camera | |
| CN113052118A (en) | Method, system, device, processor and storage medium for realizing scene change video analysis and detection based on high-speed dome camera | |
| CN113160299A (en) | Vehicle video speed measurement method based on Kalman filtering and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |