CN108230356A - A kind of motor vehicles tracking based on perspective correction - Google Patents

A kind of motor vehicles tracking based on perspective correction Download PDF

Info

Publication number
CN108230356A
CN108230356A CN201710597612.7A CN201710597612A CN108230356A CN 108230356 A CN108230356 A CN 108230356A CN 201710597612 A CN201710597612 A CN 201710597612A CN 108230356 A CN108230356 A CN 108230356A
Authority
CN
China
Prior art keywords
tracking
image
perspective
vanishing point
motor vehicles
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710597612.7A
Other languages
Chinese (zh)
Inventor
柳林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Gongshang University
Original Assignee
Zhejiang Gongshang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Gongshang University filed Critical Zhejiang Gongshang University
Priority to CN201710597612.7A priority Critical patent/CN108230356A/en
Publication of CN108230356A publication Critical patent/CN108230356A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G06T5/80
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • G06T7/536Depth or shape recovery from perspective effects, e.g. by using vanishing points
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30232Surveillance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30248Vehicle exterior or interior
    • G06T2207/30252Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256Lane; Road marking

Abstract

Motor vehicles tracking in a kind of video of the invention.It is characterized in that being modeled using mixed Gauss model (GMM) road pavement, the image that road surface only includes identifier marking is obtained;Then the vanishing point information of lane line and zebra stripes is obtained using pavement marker graticule information, Affine Reconstruction matrix and the metric reconstruction matrix of video camera is asked for so as to obtain the perspective parameter of video camera using these vanishing point information;Using Affine Reconstruction matrix and metric reconstruction matrix, perspective correction can be carried out to tracking figure, motor vehicles tracking is carried out using meanshift methods on the image after perspective is corrected.Fore-aft vehicle size is basically identical in video pictures after perspective correction, has very big help to the accuracy for improving tracking, is conducive to further analyze the track of vehicle and behavior.

Description

A kind of motor vehicles tracking based on perspective correction
Technical field
This technology invention is related to computer image processing technology, the mainly correction of video image perspective, video object detection With motor vehicles tracking technique.
Background technology
On the one hand current big and medium-sized cities traffic increasingly congestion is primarily due to motor vehicles and increases significantly, and path resource It does not catch up with, on the other hand also in that part vehicle driver does not observe traffic laws, rule arbitrarily travel, so as to cause traffic order Sequence is chaotic, and traffic accident takes place frequently.Mainly there is main behavior of driving against traffic regulations on present road:Make a dash across the red light, random change lane, Ride line traveling, do not travel by traffic lights and (keep straight on when such as left-hand rotation lamp is bright), these acts of violating regulations have the duration it is short, The features such as place is not known, evidence is difficult to save from damage usually can not carry out on-site law-enforcing by traffic police, be also not suitable for using the line of induction Circle etc. is triggered, and can only carry out intelligent recognition and video recording to illegal activities by the way of video triggering.Accordingly for The act of violating regulations evidence obtaining of this non-at-scene law enforcement, it is necessary to there are front and rear 3 high-definition pictures (car plate can be seen clearly) of breaking rules and regulations to make For evidence, entire evidence obtaining activity will follow the non-at-scene law enforcement evidence obtaining specification of motor vehicle of Ministry of Public Security's formulation[1]
Above road, the direction of travel of vehicle hangs down substantially with image forming plane in camera pedestal vacancy in most of scenes Directly, the motor vehicles image that more goes is smaller.Many times the distance of vehicle tracking is long, can reach in practice 50 To 100 meters, due to the influence of perspective effect, vehicle image size during tracking initiation position is several times as much as tracking end position Vehicle image size in this way produces the accuracy of tracking very big influence.This influence is mainly reflected in two aspects, Perspective effect causes vehicle image to deform first, and different position shape vehicle of the same race occurs on the image, in the picture Performance is also different so that vehicle detection is not accurate enough;Secondly perspective effect causes vehicle image gradually to reduce so that calculates Obtained motion vector is not accurate enough.So how to eliminate the influence of perspective effect so that the tracking box of vehicle can be with vehicle Position and change size automatically, have great significance for the tracking of track of vehicle.
In existing literature method, the step of eliminating perspective effect is not all taken in the tracking for video object, and To prolonged target following, the change in size of tracking target is mostly adapted to using the method for active mapping window parameter.Such as It is the affine matrix [1] that front and rear frame is calculated using optical flow method in TLD methods, the big of tracking target is calculated according to affine matrix It is small;And in particle filter method, average distance and target size according to particle in motion target area to target's center's point Relationship establishes the mathematical model of tracking window size to adapt to the size [2] of tracking target;In Meanshift trackings In, document [3] proposes a kind of core window modification method of ± 10% increment, and this method needs to calculate present frame Central Plains respectively It is right to select maximum Bhattacharyya coefficients institute for the Bhattacharyya coefficients of beginning core window and ± 10% core window The window answered corresponds to the optimum size of target for best core window width.
Tracking window adaptive approach employed in more than tracking, can solve the problems, such as under certain condition, but It is for a long time, during the tracking of large scale transformation, easily with losing target.The method used in the present invention can track At the very start eliminate vehicle perspective influence, tracking target save in the video sequence it is in the same size, so as to effectively improve with The success rate and accuracy of track.
Bibliography
[1] Liu Kuo, Ning Yi, robust vehicle tracking algorithm [J] calculating based on TLD under the conditions of profound pine low resolution forever Machine is applied and software, Vol.33, No.12:265-269,2016.
[2] Peng Qing are gorgeous, Zhao Xunjie, the research of old wave particle filter and tracking window size self-adapting regulation method [J] infrared techniques, vol.34, No.10:568-572,2012.
[3] Sun Yu autumns, Hu Wei, Mean-Shift track algorithms [J] the Changjiang University journal of Li Xiong bandwidth self-adaptions (from section's version), vol.14, No.1:5-12,2017.
[4] is worn refined, On-line Estimation [J] computer engineering and application of the Qiu Wei against the deviation time under perspective projection .Vol.43.No.21:235-237,2009.
[5] is permitted to know big tilt license plate image method for quickly correcting research [J] computer applications for waiting based on vanishing point Study Vol.25, No.8:2405-2406,2008.
[6] .Lazaros Grammatikopoulos, etc.Automatic estimation of vehicle speed from uncalibrated video sequences[C].Proceeding of international Symposium on modern technologies, education and professional practice in Geodesy and related fields.sofia, 03-04november, 2005.pp:332-338.
Invention content
The technical problems to be solved by the invention are to provide the new technology that motor vehicles track in a kind of video.
The vanishing point position of image is calculated using the method for computer vision by means of the traffic marking on road surface by the present invention It puts, perspective correction is carried out to video pictures according to vanishing point data, vehicle tracking is carried out on the image after correction.After correction Fore-aft vehicle size is basically identical in video pictures, has very big help to the accuracy for improving tracking, is conducive to vehicle Track and behavior further analyze.
It compared with prior art, can an advantage of the invention is that it provides a kind of method of new motor vehicles tracking The complexity of vehicle tracking is reduced by the processing of front-end image, improves the accuracy of tracking.
Description of the drawings
Fig. 1 is commonsense method vehicle tracking example (from top to bottom, from left to right frame number is respectively 1,35,62 and 94)
Fig. 2 is the pavement image schematic diagram being calculated by GMM model
Fig. 3 calculates schematic diagram for vanishing point
Attached drawing 4 is the image and target following result after perspective correction
Specific embodiment
In system of the present invention, above urban road, the direction of travel of vehicle is basic in camera pedestal vacancy It is vertical with image forming plane.Since the distance of vehicle tracking is long, can reach in practice 30 to 50 meters, in image Perspective effect drastically influence last result.Perspective effect causes vehicle image to deform first, and shape vehicle of the same race is being schemed As upper different position appearance, performance in the picture is also different so that the vehicle location using detection is not accurate enough; Secondly perspective effect causes vehicle image gradually to reduce so that the motion vector that Mean Shift methods are calculated is not accurate enough Really.As shown in Figure 1, according to certain ratio, tracking box artificially reduces step by step again, but still will appear tracking box (tracking box of the taxi in the lower left corner is gradually reducing the situation of loss in such as attached drawing 1, due to the influence of side vehicle, also gradually Gradually deviate from vehicle in itself).So how to eliminate the influence of perspective effect, there is important meaning for the tracking of track of vehicle Justice.
1. basic principle
Perspective correction (or inverse perspective mapping) is applied relatively more in traffic engineering, is such as used for estimating automotive run-off-road Time [4], license plate image is corrected[5], calculate car speed[6]Deng.But perspective house of correction is adopted in these documents Method is different, and the model of the perspective correction employed in document [4] is given in advance, and in practical application mistake Cheng Zhong, this way are undesirable;Document [5] is basic come license plate image when calculating perspective matrix, but being imaged using vanishing point position It is parallel with camera lens, with the imaging pattern different from the present invention;Document [6] using calculate vanishing point position come into Row perspective correction, but a kind of its consideration camera situation vertical with track plane.Due to road conditions ratio in practice of construction More complicated, camera installation site cannot be guaranteed right over road, therefore also must for camera and the inclined situation of track plane It must consider.Practical road is monitored herein, and video camera is carried out using the various labels and graticule of real road Self calibration, be obtained by calculation perspective correction linear model, it is detected again after being corrected to video image and with Track.
2. mathematical model
2.1 camera model
The height that video camera is placed during due to actual monitoring is greater than pavement-height and height of car, does not consider height of car Influence for imaging, it is possible to which assuming that imaging at this moment is a planar imaging, i.e., the height on road surface is 0.So road surface On a point P (X, Y, 1)TWith it image corresponding points p (u, v, 1)TBetween relationship can use a homography square Battle array H is described:
HP=p;H=K [r1, r2, t] and (1)
This homography relationship between space plane and its projected image is independently of scene layout, and H is one non- Singularity matrix, therefore road surface can be restored from projected image in the case where differing a scale factor meaning in correction algorithm The real geometry of image.Homography matrix H can be further broken into:
In above formula, E is similitude transformation matrix, the zoom scale in corresponding x-axis and y-axis direction.HpFor Affine Reconstruction matrix, He For metric reconstruction matrix.By the transformation of Affine Reconstruction matrix and metric reconstruction matrix, the shape of original image can be restored.
The 2.2 camera parameter acquiring methods based on vanishing point
Pavement image can be few for doing the information demarcated to camera, the most it is apparent that the mark mark of the image on road surface Line here demarcates camera with the zebra stripes information on road surface.Zebra stripes are by a series of sizes are identical, direction is parallel Rectangular white square composition, by camera imaging principle it is found that these vertical parallel lines pool two on imaging plane Vanishing point FuAnd Fu, as shown in Figure 1.Camera is demarcated here with the information of two vanishing points.
Here Affine Reconstruction matrix HpBy passing through vanishing point (Fu, Fv) the line L that goes outTo calculate:
L=[I1 I2 1]T=Fu×Fv (3)
Metric reconstruction matrix HeIt can then be obtained by following formula:
The solution of above formula is a pair of of conjugate complex number with α, β in formula (2) correspond to its real and imaginary parts respectively.F is phase in formula The focal length of machine can also be calculated by vanishing point information[8]
3. pavement image extracts
It, many times can not be simple since the road surface many places of monitoring are in the bigger region of busier, the magnitude of traffic flow Extract the image on road surface in ground.Therefore the present invention is modeled first using GMM (mixed Gauss model) road pavement image, is run The interference of road vehicles can be excluded after a period of time, obtains pure pavement image.Such as the road surface in attached drawing 1, warp It crosses after the calculating of 6100 frames of GMM, the image on obtained road surface is as shown in Figure 2:
4. vanishing point calculates
As shown in Figure 3, the calculating of vanishing point is calculated by the position of zebra stripes and lane line.The position of wherein vertical vanishing point It puts and is obtained by the intersection point of lane line, when reality calculates, the center line of pick-up diatom is used as calculation basis, and by multiple intersection points Least square fitting obtains.And horizontal vanishing point is obtained by the edge calculations of zebra stripes, when reality calculates, by the side of zebra stripes Edge angle point is fitted to obtain edge line, then calculates the intersection point of edge line so as to obtain practical vanishing point position.
5. perspective correction and vehicle tracking
After acquiring perspective transformation matrix according to vanishing point position, the perspective of image is corrected according to the inverse transformation by formula (1) Come carry out.Picture quality after being corrected due to perspective has very big decline, so the discovery and detection of vehicle are still in artwork As upper progress, but the tracking of vehicle carries out on the image after correction.To the correcting image of image shown in attached drawing 1 with Track is as shown in Figure 4.
Attached drawing 1 and attached drawing 4 are compared as can be seen that after perspective correction, the visual angle of image becomes from vertical view from upper To lower head-up, the size of target vehicle in the picture is all consistent substantially, and it is corrected in original inclined zebra stripes region Also it is parallel with image level side afterwards.So can be consistent before and after the size of tracking box, can ensure to track so well Accuracy.But the picture quality after correcting has dropped much compared to original image, so the image after correction is only conducive to Tracking object and be unfavorable for detect object.While it is noted that in order to reduce calculation amount, the perspective correction in figure only exists Calculated before system work it is primary, behind the perspective corrections of all frames obtained by computation of table lookup.

Claims (6)

1. the motor vehicles tracking in a kind of video, which is characterized in that extracted including pavement marker graticule, vanishing point information is asked It takes, camera parameter is asked for, several parts such as perspective image correction and motor vehicles tracking.
2. according to the pavement strip extracting method that right 1 requires, mainly built using mixed Gauss model (GMM) road pavement Mould obtains the image that road surface only includes identifier marking.
3. according to the extraction vanishing point information approach that right 1 requires, mainly using Hough transform detection of straight lines, straight line is then utilized Intersection point obtains vanishing point position.There are two vanishing points, one be lane line vanishing point, one be zebra stripes vanishing point.
4. according to the camera parameter acquiring method that right 1 requires, the parameter of camera is mainly obtained according to Affine Reconstruction matrix.
5. it is corrected according to the perspective that right 1 requires, it, can be to original by the transformation of Affine Reconstruction matrix and metric reconstruction matrix Pavement image carries out perspective correction.
6. being tracked according to the motor vehicles that right 1 requires, mainly carried out using Meanshift methods.
CN201710597612.7A 2017-07-17 2017-07-17 A kind of motor vehicles tracking based on perspective correction Pending CN108230356A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710597612.7A CN108230356A (en) 2017-07-17 2017-07-17 A kind of motor vehicles tracking based on perspective correction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710597612.7A CN108230356A (en) 2017-07-17 2017-07-17 A kind of motor vehicles tracking based on perspective correction

Publications (1)

Publication Number Publication Date
CN108230356A true CN108230356A (en) 2018-06-29

Family

ID=62654125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710597612.7A Pending CN108230356A (en) 2017-07-17 2017-07-17 A kind of motor vehicles tracking based on perspective correction

Country Status (1)

Country Link
CN (1) CN108230356A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109492609A (en) * 2018-11-27 2019-03-19 上海芯仑光电科技有限公司 It is a kind of detect lane line method and vehicle and calculate equipment
CN109785463A (en) * 2019-01-25 2019-05-21 芜湖智久机器人有限公司 A kind of carriage fault detection system and method based on speed
CN111275740A (en) * 2020-01-19 2020-06-12 武汉大学 Satellite video target tracking method based on high-resolution twin network
CN115321322A (en) * 2022-08-22 2022-11-11 日立楼宇技术(广州)有限公司 Control method, device and equipment for elevator car door and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101458871A (en) * 2008-12-25 2009-06-17 北京中星微电子有限公司 Intelligent traffic analysis system and application system thereof
CN102254318A (en) * 2011-04-08 2011-11-23 上海交通大学 Method for measuring speed through vehicle road traffic videos based on image perspective projection transformation
CN102682467A (en) * 2011-03-15 2012-09-19 云南大学 Plane- and straight-based three-dimensional reconstruction method
CN102768804A (en) * 2012-07-30 2012-11-07 江苏物联网研究发展中心 Video-based traffic information acquisition method
CN103383733A (en) * 2013-05-16 2013-11-06 浙江智尔信息技术有限公司 Lane video detection method based on half-machine study
CN103700261A (en) * 2014-01-03 2014-04-02 河海大学常州校区 Video-based road traffic flow feature parameter monitoring and traffic comprehensive information service system
CN104599249A (en) * 2015-01-13 2015-05-06 重庆大学 Cable rope bearing bridge deck vehicle load distribution real-time detection method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101458871A (en) * 2008-12-25 2009-06-17 北京中星微电子有限公司 Intelligent traffic analysis system and application system thereof
CN102682467A (en) * 2011-03-15 2012-09-19 云南大学 Plane- and straight-based three-dimensional reconstruction method
CN102254318A (en) * 2011-04-08 2011-11-23 上海交通大学 Method for measuring speed through vehicle road traffic videos based on image perspective projection transformation
CN102768804A (en) * 2012-07-30 2012-11-07 江苏物联网研究发展中心 Video-based traffic information acquisition method
CN103383733A (en) * 2013-05-16 2013-11-06 浙江智尔信息技术有限公司 Lane video detection method based on half-machine study
CN103700261A (en) * 2014-01-03 2014-04-02 河海大学常州校区 Video-based road traffic flow feature parameter monitoring and traffic comprehensive information service system
CN104599249A (en) * 2015-01-13 2015-05-06 重庆大学 Cable rope bearing bridge deck vehicle load distribution real-time detection method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
SAYANAN SIVARAMAN等: "Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking,and Behavior Analysis", 《IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS》 *
张代明: "基于消失点的鲁棒逆透视变换", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
赵谦等: "《智能视频图像处理技术与应用》", 30 September 2016, 西安电子科技大学出版社 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109492609A (en) * 2018-11-27 2019-03-19 上海芯仑光电科技有限公司 It is a kind of detect lane line method and vehicle and calculate equipment
CN109785463A (en) * 2019-01-25 2019-05-21 芜湖智久机器人有限公司 A kind of carriage fault detection system and method based on speed
CN111275740A (en) * 2020-01-19 2020-06-12 武汉大学 Satellite video target tracking method based on high-resolution twin network
CN111275740B (en) * 2020-01-19 2021-10-22 武汉大学 Satellite video target tracking method based on high-resolution twin network
CN115321322A (en) * 2022-08-22 2022-11-11 日立楼宇技术(广州)有限公司 Control method, device and equipment for elevator car door and storage medium

Similar Documents

Publication Publication Date Title
US11170466B2 (en) Dense structure from motion
US11854272B2 (en) Hazard detection from a camera in a scene with moving shadows
CN108647638B (en) Vehicle position detection method and device
Jung et al. A lane departure warning system based on a linear-parabolic lane model
EP1796043B1 (en) Object detection
CN108091142A (en) For vehicle illegal activities Tracking Recognition under highway large scene and the method captured automatically
CN108230356A (en) A kind of motor vehicles tracking based on perspective correction
CN107688764B (en) Method and device for detecting vehicle violation
CN105678287B (en) A kind of method for detecting lane lines based on ridge measurement
CN110298216A (en) Vehicle deviation warning method based on lane line gradient image adaptive threshold fuzziness
JP2007234019A (en) Vehicle image area specifying device and method for it
Lin et al. Lane departure and front collision warning using a single camera
US10902271B2 (en) Recognizing a raised object on the basis of perspective images
Liu et al. Development of a vision-based driver assistance system with lane departure warning and forward collision warning functions
CN103204104B (en) Monitored control system and method are driven in a kind of full visual angle of vehicle
CN109635737A (en) Automobile navigation localization method is assisted based on pavement marker line visual identity
CN108021849B (en) Pedestrian early warning method and device
CN109791607A (en) It is detected from a series of images of video camera by homography matrix and identifying object
CN112950725A (en) Monitoring camera parameter calibration method and device
CN112172797B (en) Parking control method, device, equipment and storage medium
KR101405085B1 (en) Device and method for video analysis
CN111797810A (en) Method for acquiring forward-looking preview area of driver in driving process
CN103679121A (en) Method and system for detecting roadside using visual difference image
JP3747863B2 (en) In-vehicle distance measuring device
Irshad et al. Real-time lane departure warning system on a lower resource platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180629