CN107315994B - Clustering method based on Spectral Clustering space trajectory - Google Patents
Clustering method based on Spectral Clustering space trajectory Download PDFInfo
- Publication number
- CN107315994B CN107315994B CN201710334850.9A CN201710334850A CN107315994B CN 107315994 B CN107315994 B CN 107315994B CN 201710334850 A CN201710334850 A CN 201710334850A CN 107315994 B CN107315994 B CN 107315994B
- Authority
- CN
- China
- Prior art keywords
- track
- coordinate system
- image
- axis
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a Clustering algorithm based on Spectral Clustering space tracks, which comprises the steps of collecting video images of a road by using a camera, extracting characteristic points of all moving targets in each frame of image in the video images by adopting an ORB algorithm, and tracking the characteristic points by using a KLT tracking algorithm based on bidirectional weighted invertibility constraint to obtain a plurality of moving tracks of all the moving targets and coordinate values of each track point on each moving track in an image coordinate system; constructing similar matrixes for the n motion tracks of all the moving targets by using rigid motion constraint to perform spectral clustering to obtain different types of motion tracks; performing inter-class combination on the motion tracks of different classes to obtain inter-class combined motion tracks; the method provided by the invention is not influenced and limited by various environments in engineering application, is easy to realize, and can effectively and accurately detect the vehicle in real time, thereby having wide application prospect.
Description
Technical Field
The invention belongs to the technical field of video detection, and particularly relates to a Clustering method based on a Spectral Clustering space track.
Background
Along with the rapid development of economy and the improvement of society, the living standard of people is improved, the number of motor vehicles is remarkably increased, and in contrast, the traffic capacity of roads is obviously reduced, and a series of problems of traffic jam, road blockage and the like are caused. The traffic flow of the road is detected and counted, and the information is sent to a supervision department, so that effective measures can be made, the traffic is relieved, and the purpose of controlling the traffic is achieved. Meanwhile, long-term traffic flow statistics provides important basis for design, maintenance and the like of urban roads in the future.
Vehicle detection and traffic flow statistics based on traffic scenes are more and more concerned due to the advantages of real-time detection performance, low cost, easiness in installation and use and the like. However, the current commonly used vehicle detection and traffic flow statistics software is limited by the traffic flow size, the scene complexity and the like, so that higher accuracy cannot be obtained, and in an actual scene, an expected effect cannot be achieved.
Disclosure of Invention
In view of the above problems or disadvantages in the prior art, it is an object of the present invention to provide a clustering method based on spectrum clustering spatial trajectories.
In order to achieve the purpose, the invention adopts the following technical scheme:
the Clustering method based on the Spectral Clustering space trajectory comprises the following steps:
setting the number of motion tracks of all the moving targets as n, wherein each motion track is provided with r continuous track points; wherein n is a natural number greater than or equal to 1, and r is a natural number greater than or equal to 1;
the image coordinate system takes any angle of each frame of image in the video image as an origin, the horizontal direction of each frame of image as a u-axis and the vertical direction of each frame of image as a v-axis;
the method comprises the following steps:
step 21, establishing a world coordinate system by taking the direction parallel to the road lane markings as a Y axis, the direction perpendicular to the road lane markings as an X axis, the X axis and the Y axis are both parallel to the road, the intersection point of the X axis and the Y axis is taken as an origin O, and the direction of the shortest distance between the camera and the road is taken as a Z axis;
step 22, selecting two optional motion tracks from the multiple motion tracks of all the motion targets, respectively recording the two motion tracks as M and N, and the height h of the track MMIs in the vertical direction of the horizontal plane, hMHas a value range of 1-3m, hMThe value interval of (a) is 0.1 m; height h of track NNIs in the horizontal direction of the horizontal plane, hNHas a value range of 0-4m, hNThe value interval of (a) is 0.01 m;
obtaining Δ D by the formula (1)1Dough making:
in the formula (1), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i)N(Pi) Representing trace points P on the trace M at the same momentiWith the track point P on the track NiDistance in the world coordinate system;
obtaining Δ D by the formula (2)2Dough making:
in the formula (2), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i+1)M(Pi) Representing points P on the track MiAnd Pi+1Distance in the world coordinate system; n (P)i+1)N(Pi) Representing points P on the track NiAnd Pi+1Distance in the world coordinate system;
tracing point P is determined by formula (3)iCoordinate values (u) in the image coordinate systemi,vi) Conversion into coordinate values (x) in world coordinate systemi,yi,zi):
Pi=C3×4 -1λipi(3)
In the formula (3), pi=[ui,vi,1]T,Pi=[xi,yi,zi,1]T,λiIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
step 23, with hM、hNAs two axes construct a plane as the zero plane, Δ D1Flour,. DELTA.D2The surfaces are two longitudinal planes vertical to the zero plane respectively;
ΔD1sum of surface and Δ D2The surfaces form two intersecting lines with the zero plane respectively as Δ D1Intersection line sum Δ D2Intersecting lines; let Delta D1Intersection line sum Δ D2Distance between intersecting lines is Δ Diff12And Δ D1Intersection line sum Δ D2The angle between the intersecting lines is theta, delta D1The slope of the line of intersection is k1,ΔD2The slope of the line of intersection is k2;
Step 24, obtaining an element w in the similarity matrix A through the formula (4)oq:
In formula (4), q is 1, 2.. n; n ═ 1,2,. n; delta D1IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time1A value; delta D2IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time2A value; h isMI1,1.1,1.2, 3, in m; h isNJ0,0.01,0.02, 3.99,4, in m;
step 25, repeating the steps 22 to 24 until every two of the N tracks are taken as a track M and a track N to obtain an nxn similar matrix A, and executing the step 26;
step 26, arranging the eigenvalues of the similarity matrix A in descending order, the sum of all eigenvalues being Sn;
The minimum k value is selected so thatThe clustering number of the n motion tracks is k, and step 27 is executed; wherein S iskK is the sum of the first k characteristic values, and k is a natural number which is more than or equal to 1;
and 27, constructing an n multiplied by K dimensional feature vector space by using the feature vectors corresponding to the first K feature values, clustering the n multiplied by K dimensional feature vector space by using a K-means algorithm, and clustering the n motion tracks into K categories of motion tracks.
Further, still include:
the method comprises the following steps:
step 31, selecting two categories from the k categories of motion tracks clustered in the step 2, and respectively marking as CaAnd CbLet class CaMiddle track paHas a minimum back projection velocity of vaClass CbMiddle track pbHas a minimum back projection velocity of vb(ii) a Wherein v isa≥0,vb≥0;
obtaining the height H of the feature point of the category to be merged by the formula (5)p:
In the formula (5), v is the speed of the moving object, and v ═ min (v)a,vb);vpThe minimum back projection speed of the category to be merged; hcThe height of the camera in the world coordinate system;
obtaining coordinate values of the feature points of the categories to be merged in a world coordinate system through an equation (6):
P′=C3×4 -1λip′ (6)
in formula (6), p '═ u'i,v′i,1]T;P′=[X′i,Y′i,Z′i,1]T;u′i,v′iThe coordinate values of the feature points of the categories to be merged in the image coordinate system; x'i,Y′′,Z′iThe coordinate values of the characteristic points of the categories to be merged in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
obtaining coordinate values of the feature points of the reference category in a world coordinate system by the following formula (7):
P″=C3×4 -1λip″ (7)
in formula (7), p ″ - [ u ″ ]i,v″i,1]T;P″=[X″i,Y″i,0,1]T;u″i,v″iCoordinate values of the characteristic points of the reference category in an image coordinate system; x ″)i,Y″i0 is a coordinate value of the characteristic point of the reference category in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
step 33, obtaining absolute distances Δ X, Δ Y, Δ Z between the feature points of the to-be-merged category and the feature points of the reference category in the world coordinate system by using formula (8):
step 34, if Δ X ═ X ', Δ Y ═ Y ', Δ Z ═ Z ', the to-be-merged class and the reference class are merged into one class; otherwise, go to step 35;
and 35, repeating the steps 31 to 34 until the motion tracks of the k categories are all used as the merged category and the category to be merged to obtain the motion track after the inter-category merging.
Further, an inverse matrix C of the perspective projection matrix of the camera is obtained by equation (9)-1 3×4:
C3×4 -1={K[R3×3|t3×1]}-1(9)
In the formula (9), K represents an intrinsic parameter matrix of the camera, R3×3Representing a rotation matrix between the camera coordinate system and the world coordinate system, t3×1Representing a translation matrix between a camera coordinate system and a world coordinate system;
the camera coordinate system is the optical center O of the cameraCIs the origin of a coordinate system, XCIn line with the u-axis direction of the image coordinate system, YCIn line with the direction of the v-axis of the image coordinate system, ZCThe axis being perpendicular to the plane formed by the image coordinate system, and ZCThe intersection of the axis with the image plane is called the principal point of the camera.
Further, the image coordinate system takes the upper left corner of each frame of image in the video image as an origin, the horizontal direction of each frame of image as a u-axis, and the vertical direction of each frame of image as a v-axis.
Compared with the prior art, the invention has the following technical effects:
the method provided by the invention is not influenced and limited by various environments in engineering application, is easy to realize, and can effectively and accurately detect the vehicle in real time, thereby having wide application prospect.
Drawings
FIG. 1 is a frame image of a video image in example 1;
FIG. 2 is a schematic view of an image coordinate system in example 1;
FIG. 3 is a result diagram of feature point extraction of a moving object in example 1;
FIG. 4 shows the result of tracking the movement locus of the vehicle in embodiment 1;
FIG. 5 is a schematic view of a world coordinate system in example 1;
FIG. 6(a) is a graph showing Δ D plotted for track No. 2 and track No. 3 in example 11Sum of surface and Δ D2Kneading; FIG. 6(b) is Δ D1Intersection line sum Δ D2Intersecting lines;
FIG. 7(a) is a graph showing Δ D plotted for track No. 0 and track No. 2 in example 11Sum of surface and Δ D2Kneading; FIG. 7(b) is Δ D1Intersection line sum Δ D2Intersecting lines;
FIG. 8 shows 4 motion profiles selected in example 1;
FIG. 9 is a graph showing a clustering result of a part of the motion trajectories in example 1;
FIG. 10 shows the relationship between the camera imaging model and three coordinate systems.
Detailed Description
The invention is further illustrated by the figures and examples.
Example 1
The embodiment provides a Clustering method based on a Spectral Clustering space track, which comprises the following steps:
wherein the ORB algorithm is from Ruble E., Rabaud V., Konolige K., Bradski G.ORB: and effective alternative to SIFT or SURF [ J ]. Proc.of IEEE Conf.on computer Vision,2011: 2564-.
KLT tracking algorithm from Song Lin, Cheng Yin Mei, Liu nan, etc. navigation of KLT visual tracking algorithm [ J ] infrared and laser engineering using multiple constrained drones, 2013.42(10): 2828-.
Setting the number of motion tracks of all the moving targets as n, wherein each motion track is provided with r continuous track points; wherein n is a natural number greater than or equal to 1, and r is a natural number greater than or equal to 1;
the image coordinate system takes any angle of each frame of image in the video image as an origin, the horizontal direction of each frame of image as a u-axis and the vertical direction of each frame of image as a v-axis;
the image coordinate system uses the lower left corner of each frame of image in the video image as the origin, the horizontal direction of each frame of image as the u-axis, and the vertical direction of each frame of image as the v-axis, as shown in fig. 2.
The traffic videos adopted in this embodiment are all 720 × 288 grayscale images, as shown in fig. 1, one of the frames of images is shown, fig. 3 is a feature point extracted by applying an ORB algorithm to a moving object in the image of fig. 1, fig. 4 is a plurality of moving tracks in the image of fig. 1, each track has a corresponding number, and 33 tracks in total, where the tracks 0 to 1 are tracks of the same vehicle, the tracks 2 to 9 are tracks of the same vehicle, the tracks 10 to 16 and 19 to 32 are tracks of the same vehicle, and the tracks 17 to 18 are tracks of the same vehicle.
the method comprises the following steps:
step 21, as shown in fig. 5, establishing a world coordinate system by taking the direction parallel to the road lane markings as a Y-axis, the direction perpendicular to the road lane markings as an X-axis, the X-axis and the Y-axis both parallel to the road, the intersection point of the X-axis and the Y-axis as an origin O, and the direction of the shortest distance between the camera and the road as a Z-axis;
step 22, the clustering method of this embodiment adopts the following principle: the rigid body has two characteristics in the motion process: 1. the connecting lines between any two points on the rigid body are parallel and equal in the process of the translation of the rigid body; 2. the position vector between any elements on the rigid body is different, and the position vector is different from the position vector to the position vector, but the displacement, the speed and the acceleration of each element are completely the same.
Selecting two motion tracks from multiple motion tracks of all motion targets, respectively recording as M and N, setting r continuous track points on each track, and height h of the track MMIs in the vertical direction of the horizontal plane, hMHas a value range of 1-3m, hMThe value interval of (a) is 0.1 m; height h of track NNIs in the horizontal direction of the horizontal plane, hNHas a value range of 0-4m, hNThe value interval of (a) is 0.01 m;
obtaining Δ D by the formula (1)1Dough making:
in the formula (1), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i)N(Pi) Representing trace points P on the trace M at the same momentiWith the track point P on the track NiDistance in the world coordinate system;
obtaining Δ D by the formula (2)2Dough making:
in the formula (2), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i+1)M(Pi) Representing points P on the track MiAnd Pi+1Distance in the world coordinate system; n (P)i+1)N(Pi) Representing points P on the track NiAnd Pi+1Distance in the world coordinate system;
tracing point P is determined by formula (3)iCoordinate values (u) in the image coordinate systemi,vi) Conversion into coordinate values (x) in world coordinate systemi,yi,zi):
P=C3×4 -1λip (3)
In the formula (3), p ═ ui,vi,1]T,P=[Xi,Yi,Zi,1]T,λiIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
step 23, with hM、hNAs two axes construct a plane as the zero plane, Δ D1、ΔD2Two longitudinal planes perpendicular to the zero plane, respectively, as Δ D1Sum of surface and Δ D2Kneading;
ΔD1sum of surface and Δ D2The surfaces form two intersecting lines with the zero plane respectively as Δ D1Intersection line sum Δ D2Intersecting lines; let Delta D1Intersection line sum Δ D2Distance between intersecting lines is Δ Diff12And Δ D1Intersection line sum Δ D2The angle between the intersecting lines is theta, delta D1The slope of the line of intersection is k1,ΔD2The slope of the line of intersection is k2;
Step 24, obtaining an element w in the similarity matrix A through the formula (4)oq:
In formula (4), q is 1, 2.. n; n ═ 1,2,. n; delta D1IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time1A value; delta D2IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time2A value; h isMI1,1.1,1.2, 3, in m; h isNJ0,0.01,0.02, 3.99,4, in m;
step 25, repeating the steps 22 to 24 until every two of the N tracks are taken as a track M and a track N to obtain an nxn similar matrix A, and executing the step 26;
step 26, arranging the eigenvalues of the similarity matrix A in descending order, the sum of all eigenvalues being Sn;
The minimum k value is selected so thatThe clustering number of the n motion tracks is k, and step 27 is executed; wherein S iskK is the sum of the first k characteristic values, and k is a natural number which is more than or equal to 1;
in this example, 95%;
step 27, constructing an n × K-dimensional feature vector space by using feature vectors corresponding to the first K feature values, clustering the n × K-dimensional feature vector space by using a K-means algorithm, and clustering n motion tracks into K categories of motion tracks;
in this embodiment, as shown in fig. 6(a), Δ D plotted for the No. 2 track and the No. 3 track1Sum of surface and Δ D2FIG. 6(b) shows Δ D1Intersection line sum Δ D2The intersecting lines, Δ D of track No. 2 and track No. 3, can be seen from FIGS. 6(a) and 6(b)1Intersection line sum Δ D2Height difference Δ Diff between intersecting lines12Is very small, and the included angle theta between two straight lines is very small, within the threshold value range, belonging to the motion trail of the same vehicle, so that the two straight lines belong to a classRespectively;
Δ D plotted for track No. 0 and track No. 2 as shown in FIG. 7(a)1Sum of surface and Δ D2FIG. 7(b) Δ D1Intersection line sum Δ D2The intersecting lines, Δ D of the track No. 0 and the track No. 2 can be seen from FIGS. 7(a) and 7(b)1Intersection line sum Δ D2Height difference Δ Diff between intersecting lines12The included angle theta between the two intersecting lines exceeds the threshold range, and does not belong to the motion trail of the same vehicle, so that the included angle theta belongs to two categories;
FIG. 8 is a graph of 4 motion trajectories taken from FIG. 4, respectively labeled as motion trajectories 0, 1,2, and 3;
table 1 shows the Δ Diff calculated by pairwise comparison of the traces obtained from the 4 traces in FIG. 812And theta results;
TABLE 1
The clustering result graph of this embodiment is a clustering result graph of a part of motion trajectories as shown in fig. 9.
Example 2
In this embodiment, on the basis of embodiment 1, in order to improve the accuracy of clustering, the method further includes:
and 3, performing inter-class combination on the k classes of motion tracks obtained in the step 2 to obtain inter-class combined motion tracks.
The method comprises the following steps:
step 31, selecting two categories from the k categories of motion tracks clustered in the step 2, and respectively marking as CaAnd CbLet class CaMiddle track paHas a minimum back projection velocity of vaClass CbMiddle track pbHas a minimum back projection velocity of vb;
obtaining the height H of the feature point of the category to be merged by the formula (5)p:
In the formula (5), v is the speed of the moving object, and v ═ min (v)a,vb);vpThe minimum back projection speed of the category to be merged; hcThe height of the camera in the world coordinate system;
obtaining coordinate values of the feature points of the categories to be merged in a world coordinate system through an equation (6):
P′=C3×4 -1λip′ (6)
in formula (6), p '═ u'i,v′i,1]T;P′=[X′i,Y′i,Z′i,1]T;u′i,v′iThe coordinate values of the feature points of the categories to be merged in the image coordinate system; x'i,Y′i,Z′iSummarizing the characteristic points of the categories to be merged in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
in this example, C3×4 -1={K[R3×3|t3×1]}-1K denotes the intrinsic parameter matrix of the camera for which the parameters are a fixed 3 × 3 matrix, R3×3Representing a rotation matrix between the camera coordinate system and the world coordinate system, t3×1Representing a translation matrix between a camera coordinate system and a world coordinate system;
the camera coordinate system is the optical center O of the cameraCIs the origin of a coordinate system, XCIn line with the u-axis direction of the image coordinate system, YCIn line with the direction of the v-axis of the image coordinate system, ZCThe axis being perpendicular to the plane formed by the image coordinate system, and ZCThe intersection of the axis with the image plane is called the principal point of the camera. As shown in fig. 10.
Obtaining coordinate values of the feature points of the reference category in a world coordinate system by the following formula (7):
P″=C3×4 -1λip″ (7)
in formula (7), p ″ - [ u ″ ]i,v″i,1]T;P″=[X″i,Y″i,0,1]T;u″i,v″iCoordinate values of the characteristic points of the reference category in an image coordinate system; x ″)i,Y″i0 is a coordinate value of the characteristic point of the reference category in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
in this example, C3×4 -1={K[R3×3|t3×1]}-1K denotes the intrinsic parameter matrix of the camera for which the parameters are a fixed 3 × 3 matrix, R3×3Representing a rotation matrix between the camera coordinate system and the world coordinate system, t3×1Representing a translation matrix between a camera coordinate system and a world coordinate system;
the camera coordinate system is the optical center O of the cameraCIs the origin of a coordinate system, XC、YCThe axis being parallel to the two-dimensional image plane, XCIn line with the u-axis direction of the image coordinate system, YCIn line with the direction of the v-axis of the image coordinate system, ZCThe axis being perpendicular to the plane formed by the image coordinate system, and ZCThe intersection of the axis with the image plane is called the principal point of the camera. As shown in fig. 10.
Step 33, obtaining absolute distances Δ X, Δ Y, Δ Z between the feature points of the to-be-merged category and the feature points of the reference category in the world coordinate system by using formula (8):
step 34, if Δ X ═ X ', Δ Y ═ Y ', Δ Z ═ Z ', the to-be-merged class and the reference class are merged into one class; otherwise, go to step 35;
and 35, repeating the steps 31 to 34 until the motion tracks of the k categories are all used as the merged category and the category to be merged to obtain the motion track after the inter-category merging.
Claims (4)
1. The Clustering method based on the Spectral Clustering space trajectory is characterized by comprising the following steps of:
step 1, carrying out video image acquisition on a road by using a camera to obtain a plurality of motion tracks of all moving targets in each frame of image in a video image and a coordinate value of each track point on each motion track in an image coordinate system;
setting the number of motion tracks of all the moving targets as n, wherein each motion track is provided with r continuous track points; wherein n is a natural number greater than or equal to 1, and r is a natural number greater than or equal to 1;
the image coordinate system takes any angle of each frame of image in the video image as an origin, the horizontal direction of each frame of image as a u-axis and the vertical direction of each frame of image as a v-axis;
step 2, constructing similar matrixes for the n motion tracks of all the moving targets by using rigid motion constraint to perform spectral clustering to obtain different types of motion tracks;
the method comprises the following steps:
step 21, establishing a world coordinate system by taking the direction parallel to the road lane markings as a Y axis, the direction perpendicular to the road lane markings as an X axis, the X axis and the Y axis are both parallel to the road, the intersection point of the X axis and the Y axis is taken as an origin O, and the direction of the shortest distance between the camera and the road is taken as a Z axis;
step 22, selecting two optional motion tracks from the multiple motion tracks of all the motion targets, respectively recording the two motion tracks as M and N, and the height h of the track MMIn the direction ofVertical direction of the horizontal plane, hMHas a value range of 1-3m, hMThe value interval of (a) is 0.1 m; height h of track NNIs in the horizontal direction of the horizontal plane, hNHas a value range of 0-4m, hNThe value interval of (a) is 0.01 m;
obtaining Δ D by the formula (1)1Dough making:
in the formula (1), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i)N(Pi) Representing trace points P on the trace M at the same momentiWith the track point P on the track NiDistance in the world coordinate system;
obtaining Δ D by the formula (2)2Dough making:
in the formula (2), N (P)i) Represents the ith trace point on trace N, M (P)i) Represents the ith trace point on trace M, M (P)i+1) Represents the i +1 th track point on the track M, N (P)i+1) Represents the i +1 th track point on the track N, i is 1, 2. M (P)i+1)M(Pi) Representing points P on the track MiAnd Pi+1Distance in the world coordinate system; n (P)i+1)N(Pi) Representing points P on the track NiAnd Pi+1Distance in the world coordinate system;
tracing point P is determined by formula (3)iCoordinate values (u) in the image coordinate systemi,vi) Conversion into coordinate values (x) in world coordinate systemi,yi,zi):
Pi=C3×4 -1λipi(3)
In the formula (3), pi=[ui,vi,1]T,Pi=[xi,yi,zi,1]T,λiIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
step 23, with hM、hNAs two axes construct a plane as the zero plane, Δ D1Flour,. DELTA.D2The surfaces are two longitudinal planes vertical to the zero plane respectively;
ΔD1sum of surface and Δ D2The surfaces form two intersecting lines with the zero plane respectively as Δ D1Intersection line sum Δ D2Intersecting lines; let Delta D1Intersection line sum Δ D2Distance between intersecting lines is Δ Diff12And Δ D1Intersection line sum Δ D2The angle between the intersecting lines is theta, delta D1The slope of the line of intersection is k1,ΔD2The slope of the line of intersection is k2;
Step 24, obtaining an element w in the similarity matrix A through the formula (4)oq:
In formula (4), q is 1, 2.. n; n ═ 1,2,. n; delta D1IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time1A value; delta D2IJIndicates when the height of the track M is hMIThe height of the track N is hNJΔ D of time2A value; h isMI1,1.1,1.2, 3, in m; h isNJ0,0.01,0.02, 3.99,4, in m;
step 25, repeating the steps 22 to 24 until every two of the N tracks are taken as a track M and a track N to obtain an nxn similar matrix A, and executing the step 26;
step 26, arranging the eigenvalues of the similarity matrix A in descending order, the sum of all eigenvalues being Sn;
Choose the bestSmall k value such thatThe clustering number of the n motion tracks is k, and step 27 is executed; wherein S iskThe sum of the first k characteristic values is obtained, k is a natural number which is more than or equal to 1, and is a ratio threshold value of the sum of the first k characteristic values and the sum of the n characteristic values;
and 27, constructing an n multiplied by K dimensional feature vector space by using the feature vectors corresponding to the first K feature values, clustering the n multiplied by K dimensional feature vector space by using a K-means algorithm, and clustering the n motion tracks into K categories of motion tracks.
2. The Spectral Clustering spatial trajectory-based Clustering method of claim 1, further comprising:
step 3, performing inter-class combination on the k classes of motion tracks obtained in the step 2 to obtain inter-class combined motion tracks;
the method comprises the following steps:
step 31, selecting two categories from the k categories of motion tracks clustered in the step 2, and respectively marking as CaAnd CbLet class CaMiddle track paHas a minimum back projection velocity of vaClass CbMiddle track pbHas a minimum back projection velocity of vb(ii) a Wherein v isa≥0,vb≥0;
Step 32, if vaLess than vbThen C will beaAs a reference category, track paAs a feature point of the reference category, CbAs category to merge, track pbAs feature points of the categories to be merged; if v isbLess than vaThen C will bebAs a reference category, track pbAs a feature point of the reference category, CaAs category to merge, track paAs feature points of the categories to be merged;
obtaining the height H of the feature point of the category to be merged by the formula (5)p:
In the formula (5), v is the speed of the moving object, and v ═ min (v)a,vb);vpThe minimum back projection speed of the category to be merged; hcThe height of the camera in the world coordinate system;
obtaining coordinate values of the feature points of the categories to be merged in a world coordinate system through an equation (6):
P′=C3×4 -1λip′ (6)
in formula (6), p '═ u'i,v′i,1]T;P′=[X′i,Y′i,Z′i,1]T;ui′,v′iThe coordinate values of the feature points of the categories to be merged in the image coordinate system; x'i,Y′i,Z′iThe coordinate values of the characteristic points of the categories to be merged in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
obtaining coordinate values of the feature points of the reference category in a world coordinate system by the following formula (7):
P″=C3×4 -1λip″ (7)
in formula (7), p ″ - [ u ″ ]i,v″i,1]T;P″=[X″i,Y″i,0,1]T;ui″,v″iCoordinate values of the characteristic points of the reference category in an image coordinate system; x ″)i,Y″i0 is a coordinate value of the characteristic point of the reference category in a world coordinate system; lambda [ alpha ]iIs a scale factor, 0 is more than or equal to lambdai≤1;C-1 3×4An inverse matrix representing a perspective projection matrix of the camera;
step 33, obtaining absolute distances Δ X, Δ Y, Δ Z between the feature points of the to-be-merged category and the feature points of the reference category in the world coordinate system by using formula (8):
step 34, if Δ X ═ X ', Δ Y ═ Y ', Δ Z ═ Z ', the to-be-merged class and the reference class are merged into one class; otherwise, go to step 35;
and 35, repeating the steps 31 to 34 until the motion tracks of the k categories are all used as the merged category and the category to be merged to obtain the motion track after the inter-category merging.
3. The Spectral Clustering spatial trajectory-based Clustering method according to claim 1 or 2, wherein the inverse matrix C of the perspective projection matrix of the camera is obtained by equation (9)-1 3×4:
C3×4 -1={K[R3×3|t3×1]}-1(9)
In the formula (9), K represents an intrinsic parameter matrix of the camera, R3×3Representing a rotation matrix between the camera coordinate system and the world coordinate system, t3×1Representing a translation matrix between a camera coordinate system and a world coordinate system;
the camera coordinate system is the optical center O of the cameraCIs the origin of a coordinate system, XCIn line with the u-axis direction of the image coordinate system, YCIn line with the direction of the v-axis of the image coordinate system, ZCThe axis being perpendicular to the plane formed by the image coordinate system, and ZCThe intersection of the axis with the image plane is called the principal point of the camera.
4. The Spectral Clustering spatial trajectory-based Clustering method according to claim 1 or 2, wherein the image coordinate system uses the upper left corner of each frame of image in the video image as an origin, the horizontal direction of each frame of image as a u-axis, and the vertical direction of each frame of image as a v-axis.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710334850.9A CN107315994B (en) | 2017-05-12 | 2017-05-12 | Clustering method based on Spectral Clustering space trajectory |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710334850.9A CN107315994B (en) | 2017-05-12 | 2017-05-12 | Clustering method based on Spectral Clustering space trajectory |
Publications (2)
Publication Number | Publication Date |
---|---|
CN107315994A CN107315994A (en) | 2017-11-03 |
CN107315994B true CN107315994B (en) | 2020-08-18 |
Family
ID=60181424
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710334850.9A Active CN107315994B (en) | 2017-05-12 | 2017-05-12 | Clustering method based on Spectral Clustering space trajectory |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107315994B (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108922172B (en) * | 2018-06-19 | 2021-03-05 | 上海理工大学 | Road congestion monitoring system based on vehicle characteristic matrix sequence change analysis |
CN109737976A (en) * | 2019-01-07 | 2019-05-10 | 上海极奥网络科技有限公司 | Map road section and lane line automatic Generation |
CN110189363B (en) * | 2019-05-30 | 2023-05-05 | 南京林业大学 | Airport scene moving target low-visual-angle video speed measuring method |
CN112634320A (en) * | 2019-09-24 | 2021-04-09 | 成都通甲优博科技有限责任公司 | Method and system for identifying object motion direction at intersection |
CN111311010B (en) * | 2020-02-22 | 2023-07-28 | 中国平安财产保险股份有限公司 | Vehicle risk prediction method, device, electronic equipment and readable storage medium |
CN112686941B (en) * | 2020-12-24 | 2023-09-19 | 北京英泰智科技股份有限公司 | Method and device for recognizing rationality of movement track of vehicle and electronic equipment |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102855638A (en) * | 2012-08-13 | 2013-01-02 | 苏州大学 | Detection method for abnormal behavior of vehicle based on spectrum clustering |
CN103456192A (en) * | 2013-09-01 | 2013-12-18 | 中国民航大学 | Terminal area prevailing traffic flow recognizing method based on track spectral clusters |
CN104504897A (en) * | 2014-09-28 | 2015-04-08 | 北京工业大学 | Intersection traffic flow characteristic analysis and vehicle moving prediction method based on trajectory data |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130136298A1 (en) * | 2011-11-29 | 2013-05-30 | General Electric Company | System and method for tracking and recognizing people |
-
2017
- 2017-05-12 CN CN201710334850.9A patent/CN107315994B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102855638A (en) * | 2012-08-13 | 2013-01-02 | 苏州大学 | Detection method for abnormal behavior of vehicle based on spectrum clustering |
CN103456192A (en) * | 2013-09-01 | 2013-12-18 | 中国民航大学 | Terminal area prevailing traffic flow recognizing method based on track spectral clusters |
CN104504897A (en) * | 2014-09-28 | 2015-04-08 | 北京工业大学 | Intersection traffic flow characteristic analysis and vehicle moving prediction method based on trajectory data |
Non-Patent Citations (4)
Title |
---|
"Similarity based vehicle trajectory clustering and anomaly detection";Zhouyu Fu et al.;《IEEE International Conference on Image Processing 2005》;20051114;第1-4页 * |
"Traffic flow characteristic analysis at intersections from multi-layer spectral clustering of motion patterns using raw vehicle trajectory";Le Xin et al.;《2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC)》;20111007;第513-519页 * |
"交通监控中运动目标轨迹的距离计算和聚类";李明之 等;《计算机工程与设计》;20120630;第33卷(第6期);第2417-2427页 * |
"基于视频车辆运动轨迹场的交通事件检测方法";李倩丽 等;《电视技术》;20151231;第39卷(第13期);第50-52页 * |
Also Published As
Publication number | Publication date |
---|---|
CN107315994A (en) | 2017-11-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107315994B (en) | Clustering method based on Spectral Clustering space trajectory | |
Gurghian et al. | Deeplanes: End-to-end lane position estimation using deep neural networksa | |
Sidla et al. | Pedestrian detection and tracking for counting applications in crowded situations | |
Zhang et al. | Joint human detection and head pose estimation via multistream networks for RGB-D videos | |
Zhu et al. | Object tracking in structured environments for video surveillance applications | |
Pang et al. | A novel method for resolving vehicle occlusion in a monocular traffic-image sequence | |
CN104899590B (en) | A kind of unmanned plane sensation target follower method and system | |
Perrollaz et al. | Probabilistic representation of the uncertainty of stereo-vision and application to obstacle detection | |
CN104378582A (en) | Intelligent video analysis system and method based on PTZ video camera cruising | |
CN102447835A (en) | Non-blind-area multi-target cooperative tracking method and system | |
CN104794737A (en) | Depth-information-aided particle filter tracking method | |
CN106570471B (en) | Dimension self-adaption multi-pose Face tracking based on compression track algorithm | |
CN113516853B (en) | Multi-lane traffic flow detection method for complex monitoring scene | |
CN105740804A (en) | Automatic vehicle tracking and driving method based on image processing | |
CN107480646A (en) | A kind of Vehicular video abnormal motion detection method based on binocular vision | |
CN112862858A (en) | Multi-target tracking method based on scene motion information | |
CN106327528A (en) | Moving object tracking method and operation method of unmanned aerial vehicle | |
Afonso et al. | Automatic estimation of multiple motion fields from video sequences using a region matching based approach | |
Cheng et al. | Structure-aware network for lane marker extraction with dynamic vision sensor | |
Liu et al. | Effective road lane detection and tracking method using line segment detector | |
CN114648557A (en) | Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle | |
Wang et al. | A new algorithm for robust pedestrian tracking based on manifold learning and feature selection | |
CN111260696A (en) | Method for edge-end-oriented pedestrian tracking and accurate people counting | |
Cheng et al. | Semantic segmentation for pedestrian detection from motion in temporal domain | |
Fangfang et al. | Real-time lane detection for intelligent vehicles based on monocular vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20171103 Assignee: Jiangsu Baisheng Engineering Consulting Co.,Ltd. Assignor: CHANG'AN University Contract record no.: X2022980013572 Denomination of invention: Clustering Method Based on Spectral Clustering Spatial Trajectory Granted publication date: 20200818 License type: Common License Record date: 20220831 |
|
EE01 | Entry into force of recordation of patent licensing contract |