CN112208523A - Method for estimating minimum collision time of vehicle collision early warning system - Google Patents

Method for estimating minimum collision time of vehicle collision early warning system Download PDF

Info

Publication number
CN112208523A
CN112208523A CN202011075866.0A CN202011075866A CN112208523A CN 112208523 A CN112208523 A CN 112208523A CN 202011075866 A CN202011075866 A CN 202011075866A CN 112208523 A CN112208523 A CN 112208523A
Authority
CN
China
Prior art keywords
obstacle
image data
millimeter wave
wave radar
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011075866.0A
Other languages
Chinese (zh)
Inventor
高利
王钧政
赵亚男
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202011075866.0A priority Critical patent/CN112208523A/en
Publication of CN112208523A publication Critical patent/CN112208523A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/095Predicting travel path or likelihood of collision
    • B60W30/0956Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00Input parameters relating to infrastructure
    • B60W2552/50Barriers

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)

Abstract

The method for estimating the minimum collision time of the vehicle collision early warning system comprises the steps of calibrating a camera by a Zhang-friend calibration method, calibrating millimeter wave radar data and camera data to be in the same time dimension according to delay difference elimination parameters, and calibrating millimeter wave radar data and camera image data to be in the same space dimension through reverse perspective transformation; correcting the track of the obstacle according to a lane linear second-order equation carried out on the image data of the camera; inputting video image data into an obstacle model to detect obstacles, and fusing millimeter wave radar data and the video image data into the same frame of obstacle image; and carrying out obstacle matching on the current obstacle image and the previous obstacle image to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle. The future state of the obstacle can be estimated, and the method is suitable for vehicles with higher requirements on safety and relatively complex traffic environments.

Description

Method for estimating minimum collision time of vehicle collision early warning system
Technical Field
The disclosure belongs to the technical field of automobile anti-collision safety, and particularly relates to a method for estimating minimum collision time of a vehicle collision early warning system, which is suitable for an automatic driving vehicle and an active driving safety system.
Background
Influenced by the technical development of the automatic driving vehicle, the field develops rapidly in recent years, a Sta (See-Think-Act) framework model of perception-decision-control is gradually formed, and a plurality of new effective methods are formed in many aspects and links.
However, the following limitations are often found in terms of collision warning:
firstly, the prediction capability for the future is insufficient; generally, the current collision warning is to predict an event which definitely may cause a collision or an obstacle driving track to realize collision warning, and characteristics such as vehicle speed change and direction change are not considered.
Second, there is less response to different vehicle types and driver behavior. The driver is more sensitive to different vehicles and driver behaviors, and collision avoidance is executed aiming at the type of the obstacle, and the collision avoidance is rarely processed.
Finally, in the sensing method of the past automatic driving technology, only different classified obstacles are matched with different speeds and volumes to be estimated, and the motion characteristics of the trajectories of the obstacles are not estimated.
Disclosure of Invention
In view of the above, the present disclosure provides a method for estimating a minimum collision time of a vehicle collision warning system, which is capable of estimating a future state of an obstacle and is suitable for a vehicle with a higher requirement on safety and a relatively complex traffic environment.
According to an aspect of the present disclosure, there is provided a method for minimum collision time estimation of a vehicle collision warning system, the method comprising:
calibrating the camera by a live-friend calibration method, calibrating millimeter wave radar data and video image data acquired by the camera to the same time dimension according to delay difference elimination parameters, and calibrating the millimeter wave radar data and the environmental video image data to the same space coordinate through reverse perspective transformation;
carrying out track correction on the barrier according to a lane line type second-order equation carried out on the video image data; inputting video image data into an obstacle detection model to identify and detect obstacles, and fusing millimeter wave radar data and the video image data into the same frame of obstacle image;
and matching the obstacle of the current frame with the obstacle of the previous frame to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle.
The method for estimating the minimum collision time of the vehicle collision early warning system comprises the steps of calibrating a camera by a Zhang-friend calibration method, calibrating millimeter wave radar data and video image data acquired by the camera to the same time dimension according to delay difference elimination parameters, and calibrating the millimeter wave radar data and the environment video image data to the same space coordinate through reverse perspective transformation; carrying out track correction on the barrier according to a lane line type second-order equation carried out on the video image data; inputting video image data into an obstacle detection model to identify and detect obstacles, and fusing millimeter wave radar data and the video image data into the same frame of obstacle image; and matching the obstacle of the current frame with the obstacle of the previous frame to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle. The future state of the obstacle can be estimated, and the method is suitable for vehicles with higher requirements on safety and relatively complex traffic environments.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
FIG. 1 illustrates a block diagram of a vehicle collision warning system minimum collision time estimation system according to an embodiment of the present disclosure;
FIG. 2 illustrates a flow chart of a method for minimum time to collision estimation for a vehicle collision warning system according to an embodiment of the present disclosure;
fig. 3 shows an obstacle location profile for a vehicle collision warning system in multiple time slices according to an embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
Fig. 1 shows a block diagram of a minimum collision time estimation system of a vehicle collision warning system according to an embodiment of the present disclosure.
The method for estimating the minimum collision time of the vehicle collision early warning system can be used for estimating the travelable area of automatic driving or unmanned driving. As shown in FIG. 1, the system for carrying out the operation of the method can comprise a video acquisition device, a millimeter wave radar and an on-board computing unit. The video acquisition equipment and the millimeter wave radar form an environment sensing layer, and the relative delay error eliminating parameters of the millimeter wave radar and the video sensor are acquired by calibrating the internal parameters and the distortion parameters of the video sensor by using the focus of the video sensor as the origin of coordinates and taking the positive direction of the millimeter wave radar as the positive direction and by using a Zhangfriend calibration method; and according to the delay error elimination parameter, the system of the millimeter wave radar and the video sensor in time space coordinates is realized. The vehicle-mounted computing unit and the decision analysis algorithm form a decision analysis layer, the output of the environment sensing layer is collected, target obstacle recognition, lane line recognition and obstacle clustering analysis are respectively carried out, data of the data are fused into a unified space and time coordinate system, target tracking and driving behavior characteristic binding are realized, fusion model safety estimation is carried out, the probability distribution of obstacle positions in a close time period is judged according to the probability distribution, the minimum collision time estimation of a vehicle collision early warning system is output based on the minimum collision time estimation, and early warning information output is completed.
And tracking, prejudging and superposing probability distribution of possible positions of the obstacles in the coordinate system by constructing a time-space dynamic coordinate system, and then finishing collision early warning estimation. Compared with the traditional method, the estimation and prejudgment capacity of the movement locus of the obstacle is enhanced, and the possible collision condition can be judged earlier; the tracking learning of the movement characteristics (such as driving behavior characteristics) of the obstacles is introduced, and different obstacles and different drivers can be distinguished and processed.
FIG. 2 shows a flowchart of a method for minimum time to collision estimation for a vehicle collision warning system according to an embodiment of the present disclosure. The method may be used for an autonomous vehicle, active driving safety system, the method may include:
step S1: calibrating the camera by a live-friend calibration method, calibrating the millimeter wave radar data and the video image data acquired by the camera to the same time dimension according to the delay difference elimination parameter, and calibrating the millimeter wave radar data and the environment video image data to the same space dimension through reverse perspective transformation.
The millimeter wave radar and the camera are generally installed at the front end of the vehicle, the horizontal forward direction faces the driving direction of the vehicle, and the installation position of the camera is relatively high and is close to the back. And the yaw angle is centered, and the pitching angle of the camera is adjusted to avoid a blind area of the camera in the visual field as much as possible. The camera focus is used as the space origin of a moving coordinate system, the relative coordinates of the millimeter wave radar are recorded, the pitch angle and the yaw angle are calibrated, and the internal parameters and the distortion parameters of the camera are acquired by a Zhangyingyou calibration method.
And (3) finding out the recording time corresponding to the millimeter wave radar data by caching the millimeter wave radar frame data and recording the event timestamp and marking the corresponding time of the note piece in the video data, and subtracting to obtain the delay difference between the millimeter wave radar and the camera, thereby eliminating the parameter for the delay difference.
The millimeter wave radar is connected with the vehicle-mounted computer through a Can bus, the camera Can be connected with the vehicle-mounted computer through RJ45 or CSI, millimeter wave data and video image data collected by the camera Can be input into the vehicle-mounted computer (Jetson Tx2 is selected for a test vehicle) so as to process the millimeter wave data and the video image data collected by the camera, the millimeter wave radar data and the video image data collected by the camera Can be calibrated to the same time dimension according to delay difference elimination parameters, and the millimeter wave radar data and the environment video image data are calibrated to the same space coordinate through reverse perspective transformation.
Step S2: carrying out track correction on the barrier according to a lane line type second-order equation carried out on the video image data; and inputting the video image data into the obstacle detection model to identify and detect the obstacle, and fusing the millimeter wave radar data and the video image data into the same frame of obstacle image.
And performing Kalman filtering on the millimeter wave radar data to improve the obstacle identification precision and realize the initial tracking of the target obstacle.
In the normal operation process, the vehicle-mounted computing unit periodically acquires data of the millimeter wave radar and the camera according to the frequency. Compared with the millimeter wave radar, the camera has larger delay, so that the millimeter wave radar data information is cached firstly, and after the latest video frame of the camera is obtained, the corresponding millimeter wave radar data is back-checked by eliminating the parameters through delay difference.
Collecting the video information by making a lane line edge curve to obtain a second-order equation parameter, Ax, of the road curve under the depression angle2+By2And + Cxy + Dx + Ey + F is 0 to correct the trajectory of the obstacle within the target range. Inputting video information of a camera into a trained obstacle neural network for obstacle identification, projecting the video information into a space coordinate system through inverse perspective transformation, and pre-marking the video information as newly added obstacles based on different obstacle typesBasic speed and gear shift flexibility of the obstacle. And adding the millimeter wave radar result and the video acquisition result into the same space coordinate system, overlapping the millimeter wave radar result and the video acquisition result with the previous frame of obstacle information set, storing and outputting the obstacle information set for caching.
Step S3: and carrying out obstacle matching on the current obstacle image and the previous obstacle image to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle.
And marking the information of the current frame obstacle image and the previous array obstacle image on the dynamic space coordinate system, and performing target obstacle set matching to obtain an obstacle information set. Each obstacle has information such as a space position, a speed, an area image and the like at the current moment, and target tracking matching is carried out on an obstacle information set.
The information characteristic of each obstacle is a multi-dimensional vector comprising shape, size, color, texture characteristics, etc. in addition to position and motion characteristic information. When the motion circular matching is carried out, namely the target of the previous frame is matched with the target of the next frame in the environment model, the matching degree weight of the position of the pre-judged obstacle is higher as the position of the pre-judged obstacle is closer, and the matching degree weight of the position of the pre-judged obstacle deviating from the obstacle is lower. For example, the value of the matching degree M of the target obstacle o of the previous frame and the target obstacle p of the next frame may be expressed as:
Figure BDA0002716756480000061
wherein s ismaxThe maximum moving distance range (the position range matched with the target obstacle identification) between the previous frame and the next frame is identified and estimated according to the relative speed of the target obstacle and the type of the target obstacle, and when the maximum moving distance range exceeds the maximum moving distance range, the target obstacle disappears and an alarm is output.
Figure BDA0002716756480000064
Is the distance between the next frame position of the target obstacle p of the next frame and the estimated position. Matching of single parameterThe calculation function is as follows:
Figure BDA0002716756480000062
simply speaking, the position change of the target obstacle is obtained after two rounds of matching are carried out in the plane model, so that more accurate posture and direction are obtained. The position change of the target obstacle is:
Figure BDA0002716756480000063
the tracking speed and the tracking speed direction of the target obstacle can be obtained, the tracking effect of the target obstacle can be further enhanced compared with that of a millimeter wave radar, and the millimeter wave radar can be used for data supplement when losing data. Meanwhile, the cache records the variable speed and variable direction probability of each tracked target obstacle.
Fig. 3 shows an obstacle location profile for a vehicle collision warning system in multiple time slices according to an embodiment of the present disclosure.
The Z axis of the moving coordinate system is replaced by time, as shown in the following figure 3, on each time slice, the position of each obstacle which possibly exists can be estimated based on the current speed, the variable speed probability and the variable direction probability of the target obstacle, a group of quantized and overlappable probability values can be obtained, and the limitations that the multi-target tracking is predicted to be crossed, the behavior of a driver is not estimated and the like in the traditional estimation method can be avoided.
If the correction of the vehicle body posture and the minimum safe distance are not added, the method comprises
Figure BDA0002716756480000071
t is constant positive number, each quadratic equation is solved to obtain collision time
Figure BDA0002716756480000072
And solving the two groups of solutions of the formula (5) and the formula (6), saving the solved t value without considering a negative number solution or a no solution condition, and determining that the t value is the minimum collision time to ensure that the t value is in the safe driving range of the vehicle. Therefore, based on the required safety probability parameter, traversing the possible minimum collision time of the adjacent short time in the current vehicle driving direction, and triggering the alarm when the minimum collision time is less than the threshold value of the safety driving range of the vehicle.
The method for estimating the minimum collision time of the vehicle collision early warning system comprises the steps of calibrating a camera by a Zhang-friend calibration method, calibrating millimeter wave radar data and video image data acquired by the camera to the same time dimension according to delay difference elimination parameters, and calibrating the millimeter wave radar data and the environment video image data to the same space coordinate through reverse perspective transformation; carrying out track correction on the barrier according to a lane line type second-order equation carried out on the video image data; inputting video image data into an obstacle detection model to identify and detect obstacles, and fusing millimeter wave radar data and the video image data into the same frame of obstacle image; and matching the obstacle of the current frame with the obstacle of the previous frame to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle. The future state of the obstacle can be estimated, and the method is suitable for vehicles with higher requirements on safety and relatively complex traffic environments.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (1)

1. A method for vehicle collision warning system minimum time to collision estimation, the method comprising:
calibrating the camera by a live-friend calibration method, calibrating millimeter wave radar data and video image data acquired by the camera to the same time dimension according to delay difference elimination parameters, and calibrating the millimeter wave radar data and the environmental video image data to the same space dimension through reverse perspective transformation;
carrying out track correction on the barrier according to a lane line type second-order equation carried out on the video image data; inputting video image data into an obstacle detection model to identify and detect obstacles, and fusing millimeter wave radar data and the video image data into the same frame of obstacle image;
and matching the obstacle of the current frame with the obstacle of the previous frame to evaluate the position of the obstacle, and obtaining the minimum collision time of the vehicle collision early warning system based on the distance between the position of the obstacle and the vehicle.
CN202011075866.0A 2020-10-10 2020-10-10 Method for estimating minimum collision time of vehicle collision early warning system Pending CN112208523A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011075866.0A CN112208523A (en) 2020-10-10 2020-10-10 Method for estimating minimum collision time of vehicle collision early warning system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011075866.0A CN112208523A (en) 2020-10-10 2020-10-10 Method for estimating minimum collision time of vehicle collision early warning system

Publications (1)

Publication Number Publication Date
CN112208523A true CN112208523A (en) 2021-01-12

Family

ID=74052945

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011075866.0A Pending CN112208523A (en) 2020-10-10 2020-10-10 Method for estimating minimum collision time of vehicle collision early warning system

Country Status (1)

Country Link
CN (1) CN112208523A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190073903A1 (en) * 2017-09-07 2019-03-07 Denso Corporation Collision avoidance apparatus
CN110239535A (en) * 2019-07-03 2019-09-17 国唐汽车有限公司 A kind of bend active collision avoidance control method based on Multi-sensor Fusion
CN110371112A (en) * 2019-07-06 2019-10-25 深圳数翔科技有限公司 A kind of intelligent barrier avoiding system and method for automatic driving vehicle
CN111060904A (en) * 2019-12-25 2020-04-24 中国汽车技术研究中心有限公司 Blind area monitoring method based on millimeter wave and vision fusion perception
CN111708016A (en) * 2020-08-03 2020-09-25 北京理工大学 Vehicle front collision early warning method with integration of millimeter wave radar and laser radar

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190073903A1 (en) * 2017-09-07 2019-03-07 Denso Corporation Collision avoidance apparatus
CN110239535A (en) * 2019-07-03 2019-09-17 国唐汽车有限公司 A kind of bend active collision avoidance control method based on Multi-sensor Fusion
CN110371112A (en) * 2019-07-06 2019-10-25 深圳数翔科技有限公司 A kind of intelligent barrier avoiding system and method for automatic driving vehicle
CN111060904A (en) * 2019-12-25 2020-04-24 中国汽车技术研究中心有限公司 Blind area monitoring method based on millimeter wave and vision fusion perception
CN111708016A (en) * 2020-08-03 2020-09-25 北京理工大学 Vehicle front collision early warning method with integration of millimeter wave radar and laser radar

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
党宏社等: "汽车防碰撞报警与制动距离的确定 ", 《长安大学学报(自然科学版)》 *
王超等: "基于单目视觉的车辆碰撞模型 ", 《南京理工大学学报》 *

Similar Documents

Publication Publication Date Title
CN110596694B (en) Complex environment radar multi-target tracking and road driving environment prediction method
CN109212521B (en) Target tracking method based on fusion of forward-looking camera and millimeter wave radar
CN107972662B (en) Vehicle forward collision early warning method based on deep learning
CN111025250B (en) On-line calibration method for vehicle-mounted millimeter wave radar
CN107985189B (en) Early warning method for lane changing depth of driver in high-speed driving environment
CN110542885A (en) Millimeter wave radar target tracking method in complex traffic environment
CN112285714A (en) Obstacle speed fusion method and device based on multiple sensors
CN112526521B (en) Multi-target tracking method for automobile millimeter wave anti-collision radar
CN114660330A (en) Method for determining uncertainty estimate of estimated velocity
US20240045052A1 (en) Method, apparatus and radar system for tracking objects
CN114495064A (en) Monocular depth estimation-based vehicle surrounding obstacle early warning method
US11753002B2 (en) Vehicular control system
CN115993597A (en) Visual radar perception fusion method and terminal equipment
CN112180361A (en) Vehicle-mounted radar target tracking method based on dynamic federal filtering
CN114518574A (en) Kurtosis-based pruning for sensor fusion systems
CN113848545A (en) Fusion target detection and tracking method based on vision and millimeter wave radar
CN111959515A (en) Forward target selection method, device and system based on visual detection
Hruschka et al. Uncertainty-adaptive, risk based motion planning in automated driving
CN116385997A (en) Vehicle-mounted obstacle accurate sensing method, system and storage medium
CN112208523A (en) Method for estimating minimum collision time of vehicle collision early warning system
CN112162550B (en) Three-dimensional target tracking method for active safety collision avoidance of automobile
US20230103178A1 (en) Systems and methods for onboard analysis of sensor data for sensor fusion
CN115082562A (en) External parameter calibration method, device, equipment, server and vehicle-mounted computing equipment
Nguyen et al. Optimized grid-based environment perception in advanced driver assistance systems
CN114802207A (en) Intelligent parking system obstacle information fusion method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210112

WD01 Invention patent application deemed withdrawn after publication