CN108944668B - Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input - Google Patents
Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input Download PDFInfo
- Publication number
- CN108944668B CN108944668B CN201810561025.7A CN201810561025A CN108944668B CN 108944668 B CN108944668 B CN 108944668B CN 201810561025 A CN201810561025 A CN 201810561025A CN 108944668 B CN108944668 B CN 108944668B
- Authority
- CN
- China
- Prior art keywords
- early warning
- vehicle
- lane
- speed
- threshold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Mechanical Engineering (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
A driving assistance early warning method based on vehicle-mounted 360-degree all-around input adopts a driving assistance early warning system, and the driving assistance early warning system comprises the following steps: the all-round-looking fisheye camera is used for acquiring image information around the vehicle; the vehicle-mounted sensor unit is used for acquiring vehicle signals; the control module is connected with the all-round fisheye camera, connected with the vehicle-mounted sensor unit, and used for receiving vehicle signals and/or image information, analyzing and processing the vehicle signals and/or the image information and outputting early warning signals; the early warning reminding module receives the early warning signal; the driving assistance early warning method comprises the following steps: the control module is used for judging the enabling of the working mode according to the vehicle signal collected by the vehicle-mounted sensor unit; calculating the pre-entry time TTZ1 of the door opening early warning area, the pre-entry time TTZ2 of the lane change pre-collision area, the vehicle following time HDT and the pre-line pressing time TLC; and executing a corresponding early warning strategy by the early warning reminding module according to the set early warning threshold value.
Description
Technical Field
The invention relates to the technical field of vehicles, in particular to an auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input.
Background
Intellectualization is one of the important trends in the development of the automobile industry nowadays, and the application of a vision system in the field of vehicle active safety is increasingly wide. The 360-degree look-around system is one of the existing advanced automobile auxiliary safety systems, and the system can provide auxiliary display of the conditions around the automobile for a driver under the low-speed working condition and provide visual assistance (such as parking and the like) for the low-speed operation of the driver, and becomes a standard configuration of a plurality of mass-produced automobile types. However, for commercial vehicles (heavy-duty vehicles, buses and the like), the vehicle is larger in size, the blind area is larger, the steering is more difficult, and the application of the all-round system can effectively reduce the range of the blind area of the vehicle, so that the driver can be helped to better know the surrounding environment of the vehicle. Therefore, the look-around system has important application value and market prospect for commercial vehicles.
Most of the existing mass production systems only provide a plane schematic diagram of the vehicle surrounding environment at a top view angle, the system view range is limited, most of the systems can only assist in displaying an area within 2-3 meters around the vehicle, and non-road plane part objects are seriously deformed in a mapping manner; the small-part stereoscopic all-around auxiliary system can truly assist the surrounding scene of the real vehicle by using the rendering capability of the GPU, and dynamically updates the scene content and the view angle range according to the operation of the driver vehicle. However, the application of the existing look-around system is only limited to the better vehicle operation of drivers under the scene working conditions such as parking, narrow roads and the like through scene splicing and projection transformation under the low-speed working condition, and an active safety function based on the input of the look-around system is not available.
Although the scene distortion of the all-round looking system is obvious, the view field range of the all-round looking system is wide, and with the wide application of a high-definition image technology, the effective detection range of the all-round looking system is also enhanced, so that the all-round looking system has certain active safety function application ductility. The invention provides an advanced auxiliary driving system under the full-speed working condition based on look-around input, and integrates the functions of lane departure, lane change and vehicle following distance early warning under a high-speed scene, pedestrian anti-collision, door opening prompting, auxiliary scene display and the like under a low-speed scene.
Disclosure of Invention
The invention aims to integrate driving assistance functions including car following safety, pedestrian protection, lane assistance, parking assistance and the like into a panoramic system through video intelligent analysis and processing, improve the active/auxiliary safety performance of related application of a 360-degree panoramic system in a mode of combining warning voice and picture early warning with scene auxiliary display, and provide an auxiliary driving early warning method based on vehicle-mounted 360-degree panoramic input.
The technical scheme adopted by the invention for solving the technical problems is as follows: a driving assistance early warning method based on vehicle-mounted 360-degree all-around input adopts a driving assistance early warning system, and the driving assistance early warning system comprises the following steps: the all-round-looking fisheye camera is used for acquiring image information around the vehicle; the vehicle-mounted sensor unit is used for acquiring vehicle signals; the control module is connected with the all-round fisheye camera, connected with the vehicle-mounted sensor unit, and used for receiving vehicle signals and/or image information, analyzing and processing the vehicle signals and/or the image information and outputting early warning signals; the early warning reminding module receives the early warning signal; the driving assistance early warning method comprises the following steps: the control module is used for judging the enabling of the working mode according to the vehicle signal collected by the vehicle-mounted sensor unit; calculating the pre-entry time TTZ1 of the door opening early warning area, the pre-entry time TTZ2 of the lane change pre-collision area, the vehicle following time HDT and the pre-line pressing time TLC; according to the set early warning threshold value, the early warning reminding module executes a corresponding early warning strategy;
the vehicle-mounted sensor unit includes: the system comprises a vehicle speed sensor, a gear signal sensor and a steering sensor, wherein the vehicle speed sensor, the gear signal sensor and the steering sensor are all connected with a control module;
the working modes comprise: a parking and door opening prompting early warning mode, a high-speed vehicle following distance too close early warning mode, a high-speed lane changing collision early warning mode and a high-speed lane departure early warning mode;
the control module collects vehicle signals according to the vehicle-mounted sensor unit to carry out working mode enabling judgment, and the method comprises the following steps: according to the signals collected by the vehicle speed sensor, the gear signal sensor and the steering sensor,
if the vehicle speed =0, stopping the vehicle, and enabling a parking and door opening prompt early warning mode;
if the vehicle speed v > v1, enabling the high-speed early warning function, wherein v1 is a high-speed function starting threshold;
judging the gear of the vehicle:
if the vehicle is in the D gear, enabling the high-speed following distance in the early warning mode; at the same time, the user can select the desired position,
if the vehicle has the intention of steering, enabling a high-speed lane-changing collision early warning mode;
if the vehicle does not have an intention to turn, the highway lane departure warning mode is enabled.
Further, the parking and door opening prompt early warning mode refers to that: when the vehicle stops, detecting and tracking a dynamic target input by a right circular viewing fisheye and in an open door early warning area, calculating the pre-entry time TTZ1 between the dynamic target and the open door early warning area of the vehicle, and sending a corresponding collision early warning signal when detecting that the vehicle has an intention to open the door according to a set early warning threshold;
the dynamic target detection comprises the following steps: inputting a right look-around fisheye into a door opening early warning area, generating sparse angular point features, calculating motion information of the angular point features, and updating angular point feature positions; selecting a characteristic point set moving towards the vehicle to generate a dynamic target region of interest;
the dynamic target tracking comprises the following steps: in the generated dynamic target region of interest, ACF characteristics are calculated, sliding window searching with limited scale and height-width ratio is carried out on a characteristic map, and the range of a door opening early warning area and the size of a detection window are adjusted according to fisheye camera installation parameters; filtering the feature position of the target corner, tracking the corner features in the candidate region which passes the target verification, and updating the feature position of the corner;
calculating the door opening early warning area pre-entry time TTZ 1: the door opening early warning area pre-entry time TTZ1 is the time required for a vehicle in an adjacent lane to travel according to the current speed and enter a door opening early warning area or a lane change pre-collision area, and the calculation formula is as follows:
TTZ1= D0/vel；
and D0 is the distance from the dynamic target to the door opening early warning area, and vel is the speed of the vehicle relative to the dynamic target.
The early warning strategy comprises the following steps:
if D0 is larger than the first threshold D1, no early warning is given;
if D1 is less than a second threshold value D2, triggering an early warning signal;
and if the second threshold value D2 is larger than the D0 and smaller than the first threshold value D1, judging whether to perform early warning according to TTZ1, triggering an early warning signal when the TTZ is smaller than a set threshold value T0, and otherwise not performing early warning.
Further, the high-speed lane change collision early warning mode comprises the following steps: under a high-speed working condition, detecting and tracking a dynamic target in a lane changing pre-collision area in the input of a rear-view fisheye camera, calculating the time TTZ2 for the dynamic target to enter the lane changing pre-collision area, and sending a corresponding collision early warning signal when detecting that a lane changing intention exists according to a set early warning threshold;
the dynamic target detection is as follows: inputting a lane-changing pre-collision area by a back-view fisheye camera to generate sparse angular point features, calculating motion information of the angular point features, and updating the positions of the angular point features; selecting a characteristic point set moving towards the vehicle to generate a dynamic target region of interest;
the dynamic target tracking comprises the following steps: in the generated dynamic target region of interest, ACF characteristics are calculated, sliding window searching with limited scale and aspect ratio is carried out on a characteristic map, a lane changing pre-collision area is changed, and the size of a detection window is adjusted according to fisheye camera installation parameters; filtering the feature position of the target corner, tracking the corner features in the candidate region which passes the target verification, and updating the feature position of the corner;
the lane change pre-crash zone pre-entry time TTZ2 is calculated: the lane-changing pre-collision area pre-entry time TTZ2 is the time required for a vehicle in an adjacent lane to enter a door-opening early warning area or a lane-changing pre-collision area when the vehicle runs according to the current speed, and the calculation formula is as follows:
TTZ2= D01/vel；
wherein D01And the distance from the dynamic target to the lane-changing pre-collision area, and the vel is the speed of the vehicle relative to the dynamic target.
The early warning strategy comprises the following steps:
if D01First threshold D11If yes, no early warning is carried out;
if D01< second threshold D21Triggering an early warning signal;
if the second threshold value D21＜ D01< first threshold D11Judging whether to give an early warning according to the pre-entry time TTZ2 of the lane-changing pre-collision area, and when the pre-entry time TTZ2 of the lane-changing pre-collision area is smaller than a set threshold T01And triggering an early warning signal, otherwise not early warning.
Further, the high-speed car-following distance over-close early warning mode comprises the following steps: under a high-speed working condition, vehicle detection and tracking are carried out on a front vehicle early warning area input by a front looking-around fish-eye camera, the vehicle following time HDT of the vehicle relative to the front vehicle is calculated, and a corresponding vehicle following prompt or early warning signal is sent out according to a set early warning threshold;
the front vehicle early warning area carries out vehicle detection and tracking: utilizing a marked vehicle training sample, training a vehicle detection classifier in an off-line manner, and searching the position of a vehicle in an on-line sliding window of a front vehicle early warning area input by a front looking-around fish-eye camera according to a preset aspect ratio;
the following time HDT:
wherein the content of the first and second substances,
the distance is a longitudinal relative vehicle distance, and V is an absolute vehicle speed of the vehicle;
the car following prompt or early warning signal is as follows:
if the following time HDT is smaller than a preset following time early warning threshold TG1, sending a following early warning signal;
if the following time HDT is larger than a preset following time early warning threshold TG2, no prompt is given;
and in other cases, displaying and prompting the following time HDT through the early warning reminding module.
Further, the expressway lane departure warning comprises: under the high-speed working condition, carrying out lane line detection on a deviation early warning area in the input of the side-looking fish-eye camera, calculating line pre-pressing time TLC of the vehicle and the corresponding lane line, and sending out a corresponding yaw early warning signal when no lane change intention is detected according to a set early warning threshold value:
the lane line detection: performing visual angle distortion correction and inverse perspective transformation on the input of a side-looking fish-eye camera, extracting edge features in a lane line detection area, and filtering binary edge features exceeding geometric features of a lane line through setting lane line width, distance and angle threshold values; fitting the characteristic points of the lane line by utilizing a plurality of sections of straight lines with fixed lengths; determining the type of the lane line through the setting of the length threshold of the lane line, and tracking the detected lane line;
the pre-pressing line time TLC calculation: transforming the projected coordinates of the lane line fitting result to a self-defined vehicle coordinate plane, fixing the original point to the midpoint of a front axle of the vehicle, enabling an X axle to be backward of the vehicle, enabling a Y axle to be lateral of the vehicle, extending the lane line to a point of intersection with the Y axle, and calculating the lateral offset of the nearest lane line of the vehicle relative to a front wheel on the side: d = Y0-W/2; estimating the pre-pressing line time TLC when the vehicle exceeds the nearest side lane line set distance threshold DL0, and adopting the following formula:
wherein the content of the first and second substances,
is the corner of the front wheel, W is the width of the vehicle body, and V is the vehicle speed;
if D is larger than a set threshold DL1, no early warning is given;
if D is smaller than a set threshold DL2, early warning is carried out;
and if D is smaller than a set threshold DL1 and larger than a set threshold DL2, judging whether to perform early warning according to TL, triggering an early warning signal when TLC is smaller than the set threshold TL, and otherwise not performing early warning.
The substantial effects of the invention are as follows: the invention provides a vehicle-mounted 360-degree all-around input-based advanced auxiliary driving system based on all-around input, and integrates lane departure, lane change and following distance early warning functions in a high-speed scene. Compared with a traditional look-around system, the intelligent analysis function based on visual input is added, and the high-speed auxiliary early warning function is designed aiming at the characteristic of the fish-eye camera input of the look-around system, and the auxiliary driving functions including car following prompt, lane auxiliary and the like can effectively avoid partial accidents caused by the human negligence of a driver.
Drawings
FIG. 1 is a general diagram of a system architecture of the present invention;
FIG. 2 is a block diagram of a mode decision of the present invention;
fig. 3 is a schematic view of a parking door opening early warning sensing area according to the present invention.
FIG. 4 is a schematic diagram of the high-speed following distance, the high-speed lane-changing collision and the high-speed lane departure region according to the present invention.
Detailed Description
The technical solution of the present invention is further specifically described below by way of specific examples in conjunction with the accompanying drawings.
A driving assistance early warning method based on vehicle-mounted 360-degree all-around input adopts a driving assistance early warning system, and the driving assistance early warning system comprises the following components as shown in figure 1: the all-round-looking fisheye camera is used for acquiring image information around the vehicle; the vehicle-mounted sensor unit is used for acquiring vehicle signals; the control module is connected with the all-round fisheye camera, connected with the vehicle-mounted sensor unit, and used for receiving vehicle signals and/or image information, analyzing and processing the vehicle signals and/or the image information and outputting early warning signals; and the early warning reminding module receives the early warning signal. The all-round-looking fisheye camera and the vehicle-mounted sensor unit are used as input of the system, the early warning reminding module is used as output of the system, and the control module is used for analyzing and processing received image information and vehicle signals and outputting reminding signals to the early warning reminding module. Specifically, a vehicle speed sensor, a gear signal sensor, a steering sensor and a look-around fish-eye camera are used as input of the system, auxiliary visual angle display and related early warning signals are output of the system, an early warning reminding module comprises an auxiliary display unit, a loudspeaker unit and a chart interface unit, and a control module comprises an image preprocessing and splicing display unit, an ROI image analysis unit and an early warning/control decision generation unit. The all-round-looking fisheye camera transmits the pictures to the image preprocessing and splicing display unit, and the processed images are output to the auxiliary display unit on one hand; and on the other hand, the signals are input to the early warning/control decision generation unit after passing through the ROI image analysis unit, the generated auditory early warning signals are output to the loudspeaker unit after being processed by the early warning/control decision generation unit in combination with the signals transmitted by the vehicle-mounted sensor unit, and the generated visual early warning signals are output to the chart interface unit.
The driving assistance early warning method comprises the following steps:
s1, the control module judges the enabling of the working mode according to the vehicle signal collected by the vehicle sensor unit;
the working modes comprise: a parking and door opening prompting early warning mode, a high-speed vehicle following distance too close early warning mode, a high-speed lane changing collision early warning mode and a high-speed lane departure early warning mode;
the control module judges the enabling of the working mode according to the vehicle signals collected by the vehicle-mounted sensor unit, and according to the signals collected by the vehicle speed sensor, the gear signal sensor and the steering sensor, as shown in figure 2,
(1) if the vehicle speed =0, stopping the vehicle, and enabling a parking and door opening prompt early warning mode;
(2) if the vehicle speed v > v1, enabling the high-speed early warning function, wherein v1 is a high-speed function starting threshold;
judging the gear of the vehicle:
if the vehicle is in the D gear, enabling the high-speed following distance in the early warning mode; at the same time, the user can select the desired position,
if the vehicle has the intention of steering, enabling a high-speed lane-changing collision early warning mode;
if the vehicle does not have an intention to turn, the highway lane departure warning mode is enabled.
S2, calculating the door opening early warning area pre-entry time TTZ 1; lane change pre-crash zone pre-entry time TTZ 2; following time HDT; pre-pressing line time TLC; according to the set early warning threshold value, the early warning reminding module executes a corresponding early warning strategy;
as shown in figures 3 and 4 of the drawings,
s21, the parking and door opening prompt early warning mode is as follows: when the vehicle stops, detecting and tracking a dynamic target input by a right circular viewing fisheye and in an open door early warning area, calculating the pre-entry time TTZ1 between the dynamic target and the open door early warning area of the vehicle, and sending a corresponding collision early warning signal when detecting that the vehicle has an intention to open the door according to a set early warning threshold;
the dynamic target detection comprises the following steps: and (4) inputting a door opening early warning area by a right look-around fisheye, generating sparse corner features, calculating motion information of the corner features, and selecting FAST or Shi-Tomasi corners. Calculating the motion information of the corner feature by using a Lucas-Kanade optical flow method, and updating the position of the corner feature; and selecting a characteristic point set moving towards the vehicle by utilizing the similarity of the motion information to generate a dynamic target region of interest.
The dynamic target tracking comprises the following steps: and calculating ACF characteristics in the generated dynamic target region of interest, and performing sliding window search for limiting the dimension and the aspect ratio on a characteristic map, wherein different aspect ratios correspond to different target categories including vehicles and bicycles, the range of the door opening early warning area and the size of a detection window are configurable parameters, and the adjustment is performed according to different fisheye camera installation parameters. And filtering the feature position of the target corner point, tracking the corner point feature in the candidate area passing the target verification, and updating the feature position of the corner point by using LK optical flow.
Calculating the door opening early warning area pre-entry time TTZ 1: the door opening early warning area pre-entry time TTZ1 is the time required for a vehicle in an adjacent lane to travel according to the current speed and enter a door opening early warning area or a lane change pre-collision area, and the calculation formula is as follows:
TTZ1= D0/vel；
d0 is the distance from the dynamic target to the door opening early warning area, and can be obtained by transforming the projection coordinates of the midpoint position at the bottom of the detected vehicle image area to the user-defined vehicle coordinate plane (using a camera to calibrate parameters), and vel is the speed of the detected vehicle relative to the current vehicle and can be obtained by a tracking algorithm.
The early warning strategy comprises the following steps:
if D0 is larger than the first threshold D1, no early warning is given;
if D1 is less than a second threshold value D2, triggering an early warning signal;
and if the second threshold value D2 is larger than the D0 and smaller than the first threshold value D1, judging whether to perform early warning according to TTZ1, triggering an early warning signal when the TTZ is smaller than a set threshold value T0, and otherwise not performing early warning.
S22, the high-speed lane change collision early warning mode: under a high-speed working condition, detecting and tracking a dynamic target in a lane changing pre-collision area in the input of a rear-view fisheye camera, calculating the time TTZ2 for the dynamic target to enter the lane changing pre-collision area, and sending a corresponding collision early warning signal when detecting that a lane changing intention exists according to a set early warning threshold value.
The dynamic target detection is as follows: inputting a lane-changing pre-collision area by a back-view fisheye camera to generate sparse angular point features, calculating motion information of the angular point features, and updating the positions of the angular point features; selecting a characteristic point set moving towards the vehicle to generate a dynamic target region of interest;
the dynamic target tracking comprises the following steps: in the generated dynamic target region of interest, ACF characteristics are calculated, sliding window searching with limited scale and aspect ratio is carried out on a characteristic map, a lane changing pre-collision area is changed, and the size of a detection window is adjusted according to fisheye camera installation parameters; filtering the feature position of the target corner, tracking the corner features in the candidate region which passes the target verification, and updating the feature position of the corner;
the lane change pre-crash zone pre-entry time TTZ2 is calculated: the lane-changing pre-collision area pre-entry time TTZ2 is the time required for a vehicle in an adjacent lane to enter a door-opening early warning area or a lane-changing pre-collision area when the vehicle runs according to the current speed, and the calculation formula is as follows:
TTZ2= D01/vel；
wherein D01And the distance from the dynamic target to the lane-changing pre-collision area, and the vel is the speed of the vehicle relative to the dynamic target.
The early warning strategy comprises the following steps:
if D01First threshold D11If yes, no early warning is carried out;
if D01< second threshold D21Triggering an early warning signal;
if the second threshold value D21＜ D01< first threshold D11Judging whether to give an early warning according to the pre-entry time TTZ2 of the lane-changing pre-collision area, and when the pre-entry time TTZ2 of the lane-changing pre-collision area is smaller than a set threshold T01And triggering an early warning signal, otherwise not early warning.
S23, the high-speed car-following distance over-close early warning mode: under a high-speed working condition, vehicle detection and tracking are carried out on a front vehicle early warning area input by a front looking-around fish-eye camera, the vehicle following time HDT of the vehicle relative to the front vehicle is calculated, and a corresponding vehicle following prompt or early warning signal is sent out according to a set early warning threshold;
the front vehicle early warning area carries out vehicle detection and tracking: utilizing a marked vehicle training sample, training a vehicle detection classifier in an off-line manner, and searching the position of a vehicle in an on-line sliding window of a front vehicle early warning area input by a front looking-around fish-eye camera according to a preset aspect ratio;
the following time HDT:
wherein the content of the first and second substances,
the distance is a longitudinal relative vehicle distance, and V is an absolute vehicle speed of the vehicle;
if the HDT is smaller than a preset following time early warning threshold TG1, a visual and buzzing early warning signal is sent out (the buzzing frequency is in direct proportion to (TG-HDT)), if the HDT is larger than a preset following time early warning threshold TG2, no prompt is given out, and otherwise, the HDT is prompted through an I/O display port.
S24, the high-speed lane departure early warning: under the high-speed working condition, carrying out lane line detection on a deviation early warning area in the input of the side-looking fish-eye camera, calculating line pre-pressing time TLC of the vehicle and the corresponding lane line, and sending out a corresponding yaw early warning signal when no lane change intention is detected according to a set early warning threshold value:
the lane line detection: performing visual angle distortion correction and inverse perspective transformation on the input of a side-looking fish-eye camera, extracting edge features in a lane line detection area, and filtering binary edge features exceeding geometric features of a lane line through setting lane line width, distance and angle threshold values; fitting the characteristic points of the lane line by utilizing a plurality of sections of straight lines with fixed lengths; determining the type of the lane line through the setting of the length threshold of the lane line, and tracking the detected lane line by utilizing Kalman filtering;
the pre-pressing line time TLC calculation: transforming the projected coordinates of the lane line fitting result to a self-defined vehicle coordinate plane, fixing the original point to the midpoint of a front axle of the vehicle, enabling an X axle to be backward of the vehicle, enabling a Y axle to be lateral of the vehicle, extending the lane line to a point of intersection with the Y axle, and calculating the lateral offset of the nearest lane line of the vehicle relative to a front wheel on the side: d = Y0-W/2; estimating the pre-pressing line time TLC when the vehicle exceeds the nearest side lane line set distance threshold DL0, and adopting the following formula:
wherein the content of the first and second substances,
is the corner of the front wheel, W is the width of the vehicle body, and V is the vehicle speed;
if D is larger than a set threshold DL1, no early warning is given;
if D is smaller than a set threshold DL2, early warning is carried out;
and if D is smaller than a set threshold DL1 and larger than a set threshold DL2, judging whether to perform early warning according to TL, triggering an early warning signal when TLC is smaller than the set threshold TL, and otherwise not performing early warning.
The above-described embodiment is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and other variations and modifications may be made without departing from the scope of the invention as set forth in the claims.
Claims (4)
1. The utility model provides a supplementary driving early warning method based on 360 degrees look around inputs of on-vehicle, its characterized in that adopts supplementary driving early warning system, supplementary driving early warning system includes:
the all-round-looking fisheye camera is used for acquiring image information around the vehicle;
the vehicle-mounted sensor unit is used for acquiring vehicle signals;
the control module is connected with the all-round fisheye camera, connected with the vehicle-mounted sensor unit, and used for receiving vehicle signals and/or image information, analyzing and processing the vehicle signals and/or the image information and outputting early warning signals; and the number of the first and second groups,
the early warning reminding module receives the early warning signal and executes an early warning strategy;
the driving assistance early warning method comprises the following steps: the control module is used for judging the enabling of the working mode according to the vehicle signal collected by the vehicle-mounted sensor unit; calculating the pre-entry time TTZ1 of the door opening early warning area, the pre-entry time TTZ2 of the lane change pre-collision area, the vehicle following time HDT and the pre-line pressing time TLC; according to the set early warning threshold value, the early warning reminding module executes a corresponding early warning strategy;
the vehicle-mounted sensor unit includes: the system comprises a vehicle speed sensor, a gear signal sensor and a steering sensor, wherein the vehicle speed sensor, the gear signal sensor and the steering sensor are all connected with a control module;
the working modes comprise: a parking and door opening prompting early warning mode, a high-speed vehicle following distance too close early warning mode, a high-speed lane changing collision early warning mode and a high-speed lane departure early warning mode;
the control module collects vehicle signals according to the vehicle-mounted sensor unit to carry out working mode enabling judgment, and the method comprises the following steps: according to the signals collected by the vehicle speed sensor, the gear signal sensor and the steering sensor,
if the vehicle speed is equal to 0, stopping the vehicle, and enabling a parking and door opening prompting early warning mode;
if the vehicle speed v > v1, enabling the high-speed early warning function, wherein v1 is a high-speed function starting threshold;
judging the gear of the vehicle:
if the vehicle is in the D gear, enabling the high-speed following distance in the early warning mode; at the same time, the user can select the desired position,
if the vehicle has the intention of steering, enabling a high-speed lane-changing collision early warning mode;
if the vehicle does not have the steering intention, enabling the high-speed lane departure early warning mode;
the parking door opening prompting early warning mode is as follows: when the vehicle stops, detecting and tracking a dynamic target input by a right circular viewing fisheye and in an open door early warning area, calculating the pre-entry time TTZ1 between the dynamic target and the open door early warning area of the vehicle, and sending a corresponding collision early warning signal when detecting that the vehicle has an intention to open the door according to a set early warning threshold;
the dynamic target detection comprises the following steps: inputting a right look-around fisheye into a door opening early warning area, generating sparse angular point features, calculating motion information of the angular point features, and updating angular point feature positions; selecting a characteristic point set moving towards the vehicle to generate a dynamic target region of interest;
the dynamic target tracking comprises the following steps: in the generated dynamic target region of interest, ACF characteristics are calculated, sliding window searching with limited scale and height-width ratio is carried out on a characteristic map, and the range of a door opening early warning area and the size of a detection window are adjusted according to fisheye camera installation parameters; filtering the feature position of the target corner, tracking the corner features in the candidate region which passes the target verification, and updating the feature position of the corner;
calculating the door opening early warning area pre-entry time TTZ 1: the door opening early warning area pre-entry time TTZ1 is the time required for a vehicle in an adjacent lane to travel according to the current speed and enter a door opening early warning area or a lane change pre-collision area, and the calculation formula is as follows:
TTZ1＝D0/vel；
d0 is the distance from the dynamic target to the door opening early warning area, and vel is the speed of the vehicle relative to the dynamic target;
if D0 is larger than the first threshold D1, no early warning is given;
if D1 is less than a second threshold value D2, triggering an early warning signal;
and if the second threshold value D2 is larger than the D0 and smaller than the first threshold value D1, judging whether to perform early warning according to TTZ1, triggering an early warning signal when the TTZ is smaller than a set threshold value T0, and otherwise not performing early warning.
2. The vehicle-mounted 360-degree look-around input-based auxiliary driving early warning method according to claim 1, wherein the high-speed lane change collision early warning mode comprises the following steps: under a high-speed working condition, detecting and tracking a dynamic target in a lane changing pre-collision area in the input of a rear-view fisheye camera, calculating the time TTZ2 for the dynamic target to enter the lane changing pre-collision area, and sending a corresponding collision early warning signal when detecting that a lane changing intention exists according to a set early warning threshold;
the dynamic target detection is as follows: inputting a lane-changing pre-collision area by a back-view fisheye camera to generate sparse angular point features, calculating motion information of the angular point features, and updating the positions of the angular point features; selecting a characteristic point set moving towards the vehicle to generate a dynamic target region of interest;
the dynamic target tracking comprises the following steps: in the generated dynamic target region of interest, ACF characteristics are calculated, sliding window searching with limited scale and aspect ratio is carried out on a characteristic map, a lane changing pre-collision area is changed, and the size of a detection window is adjusted according to fisheye camera installation parameters; filtering the feature position of the target corner, tracking the corner features in the candidate region which passes the target verification, and updating the feature position of the corner;
the lane change pre-crash zone pre-entry time TTZ2 is calculated: the lane-changing pre-collision area pre-entry time TTZ2 is the time required for a vehicle in an adjacent lane to enter a door-opening early warning area or a lane-changing pre-collision area when the vehicle runs according to the current speed, and the calculation formula is as follows:
TTZ2＝D01/vel；
wherein D01The distance from the dynamic target to the lane-changing pre-collision area, and vel is the speed of the vehicle relative to the dynamic target;
if D01First threshold D11If yes, no early warning is carried out;
if D01< second threshold D21Triggering an early warning signal;
if the second threshold value D21＜D01< first threshold D11Judging whether to give an early warning according to the pre-entry time TTZ2 of the lane-changing pre-collision area, and when the pre-entry time TTZ2 of the lane-changing pre-collision area is smaller than a set threshold T01And triggering an early warning signal, otherwise not early warning.
3. The vehicle-mounted 360-degree look-around input-based auxiliary driving early warning method according to claim 1, wherein the high-speed following distance too-close early warning mode comprises: under a high-speed working condition, vehicle detection and tracking are carried out on a front vehicle early warning area input by a front looking-around fish-eye camera, the vehicle following time HDT of the vehicle relative to the front vehicle is calculated, and a corresponding vehicle following prompt or early warning signal is sent out according to a set early warning threshold;
the front vehicle early warning area carries out vehicle detection and tracking: utilizing a marked vehicle training sample, training a vehicle detection classifier in an off-line manner, and searching the position of a vehicle in an on-line sliding window of a front vehicle early warning area input by a front looking-around fish-eye camera according to a preset aspect ratio;
the following time HDT:
wherein D isxThe distance is a longitudinal relative vehicle distance, and V is an absolute vehicle speed of the vehicle;
if the following time HDT is smaller than a preset following time early warning threshold TG1, sending a following early warning signal;
if the following time HDT is larger than a preset following time early warning threshold TG2, no prompt is given;
and in other cases, displaying and prompting the following time HDT through the early warning reminding module.
4. The vehicle-mounted 360-degree look-around input-based assistant driving early warning method according to claim 1, wherein the highway lane departure early warning comprises: under the high-speed working condition, carrying out lane line detection on a deviation early warning area in the input of the side-looking fish-eye camera, calculating line pre-pressing time TLC of the vehicle and the corresponding lane line, and sending out a corresponding yaw early warning signal when no lane change intention is detected according to a set early warning threshold value:
the lane line detection: performing visual angle distortion correction and inverse perspective transformation on the input of a side-looking fish-eye camera, extracting edge features in a lane line detection area, and filtering binary edge features exceeding geometric features of a lane line through setting lane line width, distance and angle threshold values; fitting the characteristic points of the lane line by utilizing a plurality of sections of straight lines with fixed lengths; determining the type of the lane line through the setting of the length threshold of the lane line, and tracking the detected lane line;
the pre-pressing line time TLC calculation: transforming the projected coordinates of the lane line fitting result to a self-defined vehicle coordinate plane, fixing the original point to the midpoint of a front axle of the vehicle, enabling an X axle to be backward of the vehicle, enabling a Y axle to be lateral of the vehicle, extending the lane line to a point of intersection with the Y axle, and calculating the lateral offset of the nearest lane line of the vehicle relative to a front wheel on the side: d is Y0-W/2; estimating the pre-pressing line time TLC when the vehicle exceeds the nearest side lane line set distance threshold DL0, and adopting the following formula:
wherein the content of the first and second substances,
is the corner of the front wheel, W is the width of the vehicle body, and V is the vehicle speed;
if D is larger than a set threshold DL1, no early warning is given;
if D is smaller than a set threshold DL2, early warning is carried out;
and if D is smaller than a set threshold DL1 and larger than a set threshold DL2, judging whether to perform early warning according to TL, triggering an early warning signal when TLC is smaller than the set threshold TL, and otherwise not performing early warning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810561025.7A CN108944668B (en) | 2018-06-01 | 2018-06-01 | Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810561025.7A CN108944668B (en) | 2018-06-01 | 2018-06-01 | Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108944668A CN108944668A (en) | 2018-12-07 |
| CN108944668B true CN108944668B (en) | 2020-11-17 |
Family
ID=64493032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810561025.7A Active CN108944668B (en) | 2018-06-01 | 2018-06-01 | Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108944668B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401114B (en) * | 2019-02-12 | 2023-09-19 | 深圳市艾为智能有限公司 | Method for detecting crossing object based on limited optical flow field crossing object detection device |
| CN110566058A (en) * | 2019-08-09 | 2019-12-13 | 武汉格罗夫氢能汽车有限公司 | Parking door opening safety protection method, protection device, storage equipment and hydrogen energy automobile |
| CN111696386A (en) * | 2020-05-13 | 2020-09-22 | 宁波吉利汽车研究开发有限公司 | Lane line-based auxiliary early warning system, method and medium |
| CN112849160B (en) * | 2021-01-04 | 2022-10-14 | 广州小鹏自动驾驶科技有限公司 | Vehicle control method and device based on automatic driving |
| CN113034928B (en) * | 2021-02-26 | 2022-05-31 | 山东交通学院 | Accompanying system of road maintenance safety warning robot |
| CN113031594B (en) * | 2021-02-26 | 2022-06-24 | 山东交通学院 | Maintenance operation-based adjoint type active safety warning robot, system and method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101450646A (en) * | 2008-12-29 | 2009-06-10 | 广东铁将军防盗设备有限公司 | Vehicle obstacle-avoidance warning system and corresponding circuit thereof |
| US9102261B2 (en) * | 2012-05-10 | 2015-08-11 | Zen Lee CHANG | Vehicular collision-activated information exchange method and apparatus using wireless communication radios |
| CN107031623B (en) * | 2017-03-16 | 2019-09-20 | 浙江零跑科技有限公司 | A kind of road method for early warning based on vehicle-mounted blind area camera |
| CN107650640A (en) * | 2017-09-20 | 2018-02-02 | 上海工程技术大学 | The method for early warning and early warning system that a kind of automobile parking based on image procossing opens the door |
-
2018
- 2018-06-01 CN CN201810561025.7A patent/CN108944668B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108944668A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108944668B (en) | Auxiliary driving early warning method based on vehicle-mounted 360-degree look-around input | |
| CN108638999B (en) | Anti-collision early warning system and method based on 360-degree look-around input | |
| WO2021259344A1 (en) | Vehicle detection method and device, vehicle, and storage medium | |
| US10573068B2 (en) | Method, device, terminal and system for visualization of vehicle's blind spot and a vehicle | |
| Gandhi et al. | Vehicle surround capture: Survey of techniques and a novel omni-video-based approach for dynamic panoramic surround maps | |
| US20110169957A1 (en) | Vehicle Image Processing Method | |
| US7050908B1 (en) | Lane marker projection method for a motor vehicle vision system | |
| EP3487172A1 (en) | Image generation device, image generation method, and program | |
| CN105678787A (en) | Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera | |
| US11482015B2 (en) | Method for recognizing parking space for vehicle and parking assistance system using the method | |
| US11288833B2 (en) | Distance estimation apparatus and operating method thereof | |
| Liu et al. | Development of a vision-based driver assistance system with lane departure warning and forward collision warning functions | |
| US20180162274A1 (en) | Vehicle side-rear warning device and method using the same | |
| US11833968B2 (en) | Imaging system and method | |
| WO2016157698A1 (en) | Vehicle detection system, vehicle detection apparatus, vehicle detection method, and vehicle detection program | |
| CN110780287A (en) | Distance measurement method and distance measurement system based on monocular camera | |
| Brauckmann et al. | Towards all around automatic visual obstacle sensing for cars | |
| KR20190067578A (en) | Collision warning device and method using heterogeneous cameras having overlapped capture area | |
| CN212220070U (en) | Vehicle real-time positioning system based on visual semantic segmentation technology | |
| Kim et al. | An intelligent and integrated driver assistance system for increased safety and convenience based on all-around sensing | |
| Yang | Estimation of vehicle's lateral position via the Lucas-Kanade optical flow method | |
| Song et al. | Lateral driving assistance using optical flow and scene analysis | |
| Miman et al. | Lane departure system design using with IR camera for night-time road conditions | |
| CN115088248A (en) | Image pickup apparatus, image pickup system, and image pickup method | |
| Saito et al. | Robust and accurate object velocity detection by stereo camera for autonomous driving |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 310051 1st and 6th floors, no.451 Internet of things street, Binjiang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Zero run Technology Co.,Ltd. Address before: 310051 1st and 6th floors, no.451 Internet of things street, Binjiang District, Hangzhou City, Zhejiang Province Patentee before: ZHEJIANG LEAPMOTOR TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |