CN107985200B - Right-turning safety early warning method for load-carrying truck - Google Patents

Right-turning safety early warning method for load-carrying truck Download PDF

Info

Publication number
CN107985200B
CN107985200B CN201711112829.0A CN201711112829A CN107985200B CN 107985200 B CN107985200 B CN 107985200B CN 201711112829 A CN201711112829 A CN 201711112829A CN 107985200 B CN107985200 B CN 107985200B
Authority
CN
China
Prior art keywords
truck
pedestrian
load
blind area
turning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711112829.0A
Other languages
Chinese (zh)
Other versions
CN107985200A (en
Inventor
张祖涛
朱勉宽
席超星
姚迪
潘宏烨
漆令飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest Jiaotong University
Original Assignee
Southwest Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest Jiaotong University filed Critical Southwest Jiaotong University
Priority to CN201711112829.0A priority Critical patent/CN107985200B/en
Publication of CN107985200A publication Critical patent/CN107985200A/en
Application granted granted Critical
Publication of CN107985200B publication Critical patent/CN107985200B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
    • B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T7/00—Brake-action initiating means
    • B60T7/12—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
    • B60R2300/202—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used displaying a blind spot scene on the vehicle part responsible for the blind spot
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
    • B60R2300/8026—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views in addition to a rear-view mirror system

Abstract

The invention discloses a safety early warning method for right turning of a load-carrying truck, and relates to the technical field of vehicle driving safety. The method mainly comprises the following steps: when the right turning signal is started, the fisheye camera and the ultrasonic sensor are started to acquire the right turning blind area information of the heavy-duty truck. And calibrating the fisheye camera through the disc calibration plate, and correcting the distorted image. Carrying out right turning blind area pedestrian detection on the load-carrying truck through an HOG + SVM algorithm, and then carrying out real-time tracking on the right turning blind area pedestrian of the load-carrying truck through a related filtering fusion algorithm based on color feature statistics and HOG feature representation. According to the transverse distance of the people and the vehicle measured by the ultrasonic sensor, the turning speed of the load-carrying truck is controlled, and misoperation of a driver is prevented.

Description

Right-turning safety early warning method for load-carrying truck
Technical Field
The invention relates to the technical field of vehicle driving safety, in particular to a right-turning safety early warning method for a load-carrying truck.
Background
In recent years, the holding amount of load trucks in China is increasing due to the needs of economic development and industrial transportation. The heavy-duty truck has many visual blind areas due to the characteristics of large size, long length and high height, and accordingly, vehicle accidents are caused continuously. Particularly, the accident of 'death moon' is that when the load-carrying lorry turns right, because of the existence of the inner wheel difference and the blind area of the right rearview mirror, the driver can not notice whether pedestrians exist around the body of the right side of the load-carrying lorry, thereby causing the accident.
Chinese patent publication No. CN104401254A discloses a system for sensing a dead area when a truck turns, which relates to an infrared sensing system. The system comprises a plurality of infrared sensors which are distributed on two sides of a truck and are respectively connected with steering, whistling and braking devices of the truck. When the truck turns, due to the fact that the dead area of about 2 meters exists in the difference of the inner wheels, a driver cannot observe the dead area through a rearview mirror, the sum of the sensing ranges of the infrared sensors covers the rectangular dead area, when the truck turns the sensing system to sense a human body, the infrared sensors can send signals to a whistle and a braking device, and accidents are avoided through the whistle and the brake. Although the pedestrian detection system can effectively detect pedestrians in a death area, the system is single in signal acquisition, and if the infrared sensor is accidentally broken down, the system cannot work normally.
Chinese patent publication No. CN104477094A discloses a trailer turning safety early warning system, and belongs to the field of traffic safety monitoring. The system comprises a vehicle body, a cab, a steering lamp control switch, a pedestrian mobile phone terminal, a driver mobile phone terminal, an information acquisition device and a warning device which are arranged on two sides of the vehicle body, and an alarm device and a central processing unit which are arranged in the cab. This system adopts two ways to obtain the signal and early warning driver and pedestrian, and one way signal is: the information acquisition device acquires road condition information and transmits the road condition information to the central processing unit, and the central processing unit performs conversion and logical operation and outputs control signals to the warning device and the alarm device. The other path of signals is as follows: the pedestrian mobile phone terminal and the driver mobile phone terminal are used for connecting the position information to the central processing unit in a communication mode, the central processing unit is used for carrying out operation processing, and mobile phone terminal alarm signals for triggering the driver and pedestrians in dangerous distances are output. Meanwhile, the system adopts a millimeter wave radar and a wide-angle camera as an information acquisition device, an LED soft light band as a warning device and a buzzer as an alarm device. Although the system overcomes the defect of single signal acquisition and can ensure the reliability of the system, the system has excessive equipment and poor signal processing real-time performance, and alarms are carried out through a mobile phone terminal. If the driver and the pedestrian do not carry the mobile terminal, the alarm will be invalid.
Disclosure of Invention
The invention aims to provide a right-turning safety early warning method for a load-carrying truck. The early warning problem that the dead zone of right turn of the load-carrying truck is obstructed can be effectively solved.
The purpose of the invention is realized by the following technical scheme: a safety early warning method for right turning of a load-carrying truck comprises the following steps:
step one, acquiring right turning blind area information of a load truck:
after a central control center of the load-carrying truck detects a right-turning signal, a fisheye camera at the back of a rear compartment of a truck cab is started to acquire a real-time right-turning blind area video, and meanwhile, an ultrasonic sensor at the right side of the truck compartment is started to acquire distance information;
step two, utilizing the disc calibration plate to calibrate the fisheye camera:
because the pictures shot by the fisheye lens are similar to the disc, the calibration of the fisheye camera is completed by using a disc calibration plate to provide accurate point-line characteristics, and internal and external parameters and distortion coefficients of the fisheye camera are obtained through calibration;
step three, correcting the fish eye distortion image:
correcting the distorted image by using a template method in a fisheye image function correction method, filling the corrected image by using a cubic convolution method, and preparing for detecting and tracking pedestrians in a right turning blind area;
step four, detecting and tracking the pedestrians in the right turning blind area of the load truck:
after the distorted images are corrected, detecting and tracking pedestrians in the signal processing unit, and selecting the corrected first frame image to detect pedestrians in a right turning blind area of the heavy-duty truck; performing pedestrian detection by adopting a characteristic representation of a histogram of oriented gradients HOG and a classifier algorithm of a support vector machine SVM; after detecting blind area pedestrians, extracting the position information of a tracking frame of a first frame, and tracking the blind area pedestrians by adopting a related filtering fusion algorithm based on color features and HOG feature representation;
step five, controlling the right turning speed of the load-carrying truck:
the real-time display of the right turning blind area pedestrian information of the load-carrying truck on the liquid crystal display screen can help the driver to know the blind area information, but in order to prevent the driver from stepping on the accelerator by mistake, a right turning vehicle speed control scheme of the load-carrying truck based on the transverse distance of the pedestrian and the vehicle is adopted, when the fisheye camera detects and tracks the blind area pedestrian, the transverse distance of the pedestrian and the truck is obtained by the ultrasonic sensor, and the central control center actively controls the accelerator controller of the truck according to the transverse distance between the current truck turning vehicle speed and the pedestrian to prevent the misoperation of the driver; the control method comprises the following specific steps:
when the throttle is suddenly increased at any transverse distance, the central control center performs emergency braking;
A. the distance between the truck and the pedestrian is 10m, the central control center controls the accelerator and limits the turning speed to 30 km/h;
B. 5m < the transverse distance from the truck to the pedestrian <10m, and the central control center controls the accelerator and limits the turning speed to 20 km/h;
C. 3m < the transverse distance from the truck to the pedestrian <5m, and the central control center controls the accelerator and limits the turning speed to 10 km/h;
D. 1m < the transverse distance from the truck to the pedestrian <3m, and the central control center controls the accelerator and limits the turning speed to 5 km/h;
E. the transverse distance between the truck and the pedestrian is less than 1m, the central control center controls the accelerator and limits the turning speed to 0 km/h;
and repeating the operations of the first step to the fifth step.
The purpose of the invention is realized by the following technical scheme:
in the second step, the fisheye camera is calibrated by using the disc calibration plate, and the method specifically comprises the following steps:
b1, shooting a plurality of black and white disc calibration plate images by using a fisheye camera;
b2, binarization is carried out, and the color image is converted into a gray image;
b3, inputting the number of distorted disk calibration images, namely the number of circle centers on the horizontal and vertical coordinates;
b4, taking the center of the left lower disk as the origin of the image coordinate, and extracting the position of the center of each disk as a feature point;
b5, sub-pixel precision is carried out, and the circle centers are well stored;
b6, calculating a rotation matrix and a translation vector in the external parameters according to the conversion principle of the camera coordinate system and the world coordinate system, and calculating the internal parameters according to the conversion principle of the camera coordinate system and the image physical coordinate system;
b7, calculating the radial distortion coefficient and the tangential distortion coefficient by using the nonlinear model formula of the camera.
In the fourth step, the pedestrian detection and tracking are carried out on the right turning blind area of the load-carrying truck, and the fourth step specifically comprises the following steps:
d1, importing n corrected truck blind area pictures, and selecting a t-th frame;
d2, extracting an interested region RIO, and performing PCA dimension reduction on the ROI region;
d3, extracting HOG (histogram of oriented gradients) features in the test sample, and identifying blind area pedestrians by using a trained SVM (support vector machine) classifier;
d4, judging whether a pedestrian is detected or not;
d5, if the pedestrian is not identified, selecting the next frame of image and carrying out D2;
d6, if the pedestrian is identified, extracting four corner position information of a tracking frame (grountruth);
d7, performing color feature training of the target in the frame according to the position information of the tracking frame (grountruth), and counting the color probability of the foreground target and the background area;
d8, extracting HOG characteristics of the detected pedestrians in the frame according to the position information of the tracking frame (grountruth), and training to obtain a related filter template;
d9, inputting a corrected image of the next frame;
d10, judging the probability that each pixel in the image belongs to the foreground by using a Bayes method, and then inhibiting the object with similar color at the edge to obtain a target tracking area of the current frame based on color characteristics;
d11, performing cosine window processing, multiplying the cosine window processing by a related filter template through fast Fourier transform, performing inverse Fourier transform to obtain a maximum response point diagram, and obtaining a target tracking area of the current frame obtained based on the related filtering;
d12, fusing the target tracking area based on color feature statistics and the maximum response point diagram obtained by the relevant filtering to obtain the final tracking frame (groudtruth) area of the target in the current frame, and then switching to D7.
The invention aims to realize the following by a right-turning safety early warning system of a load-carrying truck: the method needs to arrange the fisheye cameras at the rear half part of the truck head, more than one ultrasonic sensor is arranged at the right side of the truck carriage, the signal processing unit is arranged in the cab, the central control center is arranged in the cab, and the liquid crystal display screen is arranged in the cab.
The output end of the fisheye camera and the output end of the ultrasonic sensor are connected with the input end of the signal processing unit, the output end of the signal processing unit is connected with the input end of the central control center and the input end of the liquid crystal display screen, and the output end of the central control center is connected with the input end of the fisheye camera, the input end of the ultrasonic sensor and the input end of the throttle controller.
The invention has the beneficial effects that:
compared with the prior art, the method adopts the fisheye camera to acquire more image information of the right turning blind area of the truck. Through the calibration of the fisheye lens and the correction of a distorted image, pedestrians tracked to a right turning visual blind area of the truck can be timely and accurately detected by using an HOG + SVM pedestrian detection algorithm and a pedestrian tracking algorithm based on color and related filtering. The processed image is displayed on the liquid crystal display screen, and the function of warning a driver can be achieved. In addition, in order to prevent the driver from mistakenly operating due to the fact that the driver sees the pedestrians in the blind area, the ultrasonic sensor is used for obtaining the transverse distance between the blind area pedestrians and the truck, and the speed of the truck in turning is limited according to the distance and the operation condition of the driver.
In a word, the invention can warn the driver when the load-carrying truck turns, and can also control the turning speed to prevent the misoperation of the driver. The system has simple structure and strong popularization.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a flow chart of the present invention;
FIG. 3 is a flow chart of fisheye camera calibration according to the present invention;
FIG. 4 is a flow chart of a method for detecting and tracking pedestrians in a dead zone of right turn of a load-carrying truck according to the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments.
Example 1:
referring to fig. 1, fig. 1 is a schematic structural diagram of a right-turning safety warning system of a load-carrying truck in an embodiment 1 of the present invention. The embodiment provides a right-turning safety early warning system for a load-carrying truck, which comprises a fisheye camera 1, an ultrasonic sensor 2, a signal processing unit 3, a central control center 4, a liquid crystal display screen 5 and an accelerator controller 6;
the fisheye camera 1 is arranged on the rear half part of the truck head, the ultrasonic sensors 2 are arranged on the right sides of the truck carriage respectively, the signal processing unit 3 is arranged in the cab, the central control center 4 is arranged in the cab, and the liquid crystal display screen 5 is arranged in the cab;
the output of fisheye camera 1, the input of 2 output connection signal processing unit 3 of ultrasonic sensor, central control center 4's input and liquid crystal display 5's input are connected to signal processing unit 3's output, central control center 4's output is connected fisheye camera 1's input, ultrasonic sensor 2's input and throttle controller 6's input.
Example 2:
referring to fig. 2, fig. 2 is a flowchart of a right-turning safety warning method for a truck according to the present invention. This embodiment provides a method of using the present invention, which includes the following steps:
step one, acquiring right turning blind area information of a load truck:
after a central control center of the load-carrying truck detects a right-turning signal, a fisheye camera at the back of a rear compartment of a truck cab is started to acquire a real-time right-turning blind area video, and meanwhile, an ultrasonic sensor at the right side of the truck compartment is started to acquire distance information;
step two, utilizing the disc calibration plate to calibrate the fisheye camera:
because the pictures shot by the fisheye lens are similar to the disc, the calibration of the fisheye camera is completed by using a disc calibration plate to provide accurate point-line characteristics, and internal and external parameters and distortion coefficients of the fisheye camera are obtained through calibration;
step three, correcting the fish eye distortion image:
correcting the distorted image by using a template method in a fisheye image function correction method, filling the corrected image by using a cubic convolution method, and preparing for detecting and tracking pedestrians in a right turning blind area;
step four, detecting and tracking the pedestrians in the right turning blind area of the load truck:
after the distorted images are corrected, detecting and tracking pedestrians in the signal processing unit, and selecting the corrected first frame image to detect pedestrians in a right turning blind area of the heavy-duty truck; performing pedestrian detection by adopting a characteristic representation of a histogram of oriented gradients HOG and a classifier algorithm of a support vector machine SVM; after detecting blind area pedestrians, extracting the position information of a tracking frame of a first frame, and tracking the blind area pedestrians by adopting a related filtering fusion algorithm based on color features and HOG feature representation;
step five, controlling the right turning speed of the load-carrying truck:
the real-time display of the right turning blind area pedestrian information of the load-carrying truck on the liquid crystal display screen can help the driver to know the blind area information, but in order to prevent the driver from stepping on the accelerator by mistake, a right turning vehicle speed control scheme of the load-carrying truck based on the transverse distance of the pedestrian and the vehicle is adopted, when the fisheye camera detects and tracks the blind area pedestrian, the transverse distance of the pedestrian and the truck is obtained by the ultrasonic sensor, and the central control center actively controls the accelerator controller of the truck according to the transverse distance between the current truck turning vehicle speed and the pedestrian to prevent the misoperation of the driver; the control method comprises the following specific steps:
when the throttle is suddenly increased at any transverse distance, the central control center performs emergency braking;
A. the distance between the truck and the pedestrian is 10m, the central control center controls the accelerator and limits the turning speed to 30 km/h;
B. 5m < the transverse distance from the truck to the pedestrian <10m, and the central control center controls the accelerator and limits the turning speed to 20 km/h;
C. 3m < the transverse distance from the truck to the pedestrian <5m, and the central control center controls the accelerator and limits the turning speed to 10 km/h;
D. 1m < the transverse distance from the truck to the pedestrian <3m, and the central control center controls the accelerator and limits the turning speed to 5 km/h;
E. the transverse distance between the truck and the pedestrian is less than 1m, the central control center controls the accelerator and limits the turning speed to 0 km/h;
and repeating the operations of the first step to the fifth step.
Example 3:
referring to fig. 3, fig. 3 is a flowchart of fisheye camera calibration according to embodiment 3 of the invention. The method is realized in the second step of the embodiment 2, and comprises the following steps:
and B1, shooting a plurality of black and white disc calibration plate images by using the fisheye camera.
And B2, performing binarization, and converting the color image into a gray image.
B3, inputting the number of distorted disk calibration images, namely the number of circle centers on the horizontal and vertical coordinates.
B4, taking the center of the lower left disc as the origin of the image coordinate, extracting the position (x) of the center of each disc0,y0) As a feature point.
B5, sub-pixel precision is carried out, and the circle centers are well saved.
B6 midpoint (X) according to camera coordinate systemc，Yc，Zc) To the middle point (X) of the world coordinate systemw，Yw，Zw) The conversion principle is as follows:
Figure GDA0002524192020000051
wherein 0TTranspose matrix for 0. And calculating a rotation matrix R and a translation vector t in the extrinsic parameters. According to the midpoint (X) of the camera coordinate systemc，Yc，Zc) And the principle of point (x, y) conversion in the image physical coordinate system:
Figure GDA0002524192020000052
s is a scaling factor. The internal parameter f of the camera can be calculatedx,fy,cx,cy。
B7, nonlinear model formula using camera:
Figure GDA0002524192020000053
formula midpoint (x)d,yd) As the original position, point (x)p,yp) For the corrected new position, r is the distance of the point from the imager. The radial distortion coefficient k can be calculated1,k2,k3,k4And tangential distortion coefficient p1,p2。
Example 4:
referring to fig. 4, fig. 4 is a flowchart of a blind zone pedestrian detection and tracking method for right turn of a heavy-duty truck according to an embodiment 4 of the present invention. The method is realized in the four steps of the embodiment 2, and comprises the following steps:
d1, importing n corrected truck blind area pictures, and selecting a t-th frame;
d2, extracting an interested region RIO, and performing PCA dimension reduction on the ROI region;
d3, extracting HOG (histogram of oriented gradients) features in the test sample, and identifying blind area pedestrians by using a trained SVM (support vector machine) classifier;
d4, judging whether a pedestrian is detected or not;
d5, if the pedestrian is not identified, selecting the next frame of image and carrying out D2;
d6, if the pedestrian is identified, extracting four corner position information of a tracking frame (grountruth);
d7, performing color feature training of the target in the frame according to the position information of the tracking frame (grountruth), and counting the color probability of the foreground target and the background area;
d8, extracting HOG characteristics of the detected pedestrians in the frame according to the position information of the tracking frame (grountruth), and training to obtain a related filter template;
d9, inputting a corrected image of the next frame;
d10, judging the probability that each pixel in the image belongs to the foreground by using a Bayes method, and then inhibiting the object with similar color at the edge to obtain a target tracking area of the current frame based on color characteristics;
d11, performing cosine window processing, multiplying the cosine window processing by a related filter template through fast Fourier transform, performing inverse Fourier transform to obtain a maximum response point diagram, and obtaining a target tracking area of the current frame obtained based on the related filtering;
d12, fusing the target tracking area based on color feature statistics and the maximum response point diagram obtained by the relevant filtering to obtain the final tracking frame (groudtruth) area of the target in the current frame, and then switching to D7.

Claims (5)

1. A safety early warning method for right turning of a load-carrying truck comprises the following steps:
step one, acquiring right turning blind area information of a load truck:
after a central control center of the load-carrying truck detects a right-turning signal, a fisheye camera at the back of a rear compartment of a truck cab is started to acquire a real-time right-turning blind area video, and meanwhile, an ultrasonic sensor at the right side of the truck compartment is started to acquire distance information;
step two, utilizing the disc calibration plate to calibrate the fisheye camera:
because the pictures shot by the fisheye lens are similar to the disc, the calibration of the fisheye camera is completed by using a disc calibration plate to provide accurate point-line characteristics, and internal and external parameters and distortion coefficients of the fisheye camera are obtained through calibration;
step three, correcting the fish eye distortion image:
correcting the distorted image by using a template method in a fisheye image function correction method, filling the corrected image by using a cubic convolution method, and preparing for detecting and tracking pedestrians in a right turning blind area;
step four, detecting and tracking the pedestrians in the right turning blind area of the load truck:
after the distorted images are corrected, detecting and tracking pedestrians in the signal processing unit, and selecting the corrected first frame image to detect pedestrians in a right turning blind area of the heavy-duty truck; performing pedestrian detection by adopting a characteristic representation of a histogram of oriented gradients HOG and a classifier algorithm of a support vector machine SVM; after detecting blind area pedestrians, extracting the position information of a tracking frame of a first frame, and tracking the blind area pedestrians by adopting a related filtering fusion algorithm based on color features and HOG feature representation;
step five, controlling the right turning speed of the load-carrying truck:
the real-time display of the right turning blind area pedestrian information of the load-carrying truck on the liquid crystal display screen can help the driver to know the blind area information, but in order to prevent the driver from stepping on the accelerator by mistake, a right turning vehicle speed control scheme of the load-carrying truck based on the transverse distance of the pedestrian and the vehicle is adopted, when the fisheye camera detects and tracks the blind area pedestrian, the transverse distance of the pedestrian and the truck is obtained by the ultrasonic sensor, and the central control center actively controls the accelerator controller of the truck according to the transverse distance between the current truck turning vehicle speed and the pedestrian to prevent the misoperation of the driver; the control method comprises the following specific steps:
when the throttle is suddenly increased at any transverse distance, the central control center performs emergency braking;
A. the distance between the truck and the pedestrian is 10m, the central control center controls the accelerator and limits the turning speed to 30 km/h;
B. 5m < the transverse distance from the truck to the pedestrian <10m, and the central control center controls the accelerator and limits the turning speed to 20 km/h;
C. 3m < the transverse distance from the truck to the pedestrian <5m, and the central control center controls the accelerator and limits the turning speed to 10 km/h;
D. 1m < the transverse distance from the truck to the pedestrian <3m, and the central control center controls the accelerator and limits the turning speed to 5 km/h;
E. the transverse distance between the truck and the pedestrian is less than 1m, the central control center controls the accelerator and limits the turning speed to 0 km/h;
and repeating the operations of the first step to the fifth step.
2. The safety warning method for right turn of truck as claimed in claim 1, wherein in the second step, the fisheye camera is calibrated by using the disc calibration plate, and the method specifically comprises the following steps:
b1, shooting a plurality of black and white disc calibration plate images by using a fisheye camera;
b2, binarization is carried out, and the color image is converted into a gray image;
b3, inputting the number of distorted disk calibration images, namely the number of circle centers on the horizontal and vertical coordinates;
b4, taking the center of the lower left disc as the origin of the image coordinate, extracting the position (x) of the center of each disc0,y0) As a feature point;
b5, sub-pixel precision is carried out, and the circle centers are well stored;
b6, calculating a rotation matrix R and a translation vector t in the external parameters according to the conversion principle of the camera coordinate system and the world coordinate system, and calculating an internal parameter f according to the conversion principle of the camera coordinate system and the image physical coordinate systemx,fy,cx,cy；
B7, calculating the radial distortion coefficient k by using the nonlinear model formula of the camera1,k2,k3,k4And tangential distortion coefficient p1,p2。
3. The safety warning method for right turn of load-carrying truck as claimed in claim 1, wherein the step four is to detect and track the pedestrian in the dead zone of right turn of load-carrying truck, and the step includes:
d1, importing n corrected truck blind area pictures, and selecting a t-th frame;
d2, extracting an interested region RIO, and performing PCA dimension reduction on the ROI region;
d3, extracting HOG (histogram of oriented gradients) features in the test sample, and identifying blind area pedestrians by using a trained SVM (support vector machine) classifier;
d4, judging whether a pedestrian is detected or not;
d5, if the pedestrian is not identified, selecting the next frame of image and carrying out D2;
d6, if the pedestrian is identified, extracting the position information of four corners of the tracking frame;
d7, performing color feature training of the targets in the frame according to the position information of the tracking frame, and counting the color probability of the foreground target and the background area;
d8, extracting HOG characteristics of the detected pedestrians in the frame according to the position information of the tracking frame, and training to obtain a related filter template;
d9, inputting a corrected image of the next frame;
d10, judging the probability that each pixel in the image belongs to the foreground by using a Bayes method, and then inhibiting the object with similar color at the edge to obtain a target tracking area of the current frame based on color characteristics;
d11, performing cosine window processing, multiplying the cosine window processing by a related filter template through fast Fourier transform, performing inverse Fourier transform to obtain a maximum response point diagram, and obtaining a target tracking area of the current frame obtained based on the related filtering;
d12, fusing the target tracking area based on color feature statistics and the maximum response point diagram obtained by the relevant filtering to obtain the final tracking frame area of the target in the current frame, and then turning to D7.
4. The safety warning method for right turn of truck as claimed in claim 1, characterized in that: the method needs to connect the output end of the fisheye camera (1) and the output end of the ultrasonic sensor (2) with the input end of the signal processing unit (3), the output end of the signal processing unit (3) is connected with the input end of the central control center (4) and the input end of the liquid crystal display screen (5), and the output end of the central control center (4) is connected with the input end of the fisheye camera (1), the input end of the ultrasonic sensor (2) and the input end of the accelerator controller (6).
5. The safety warning method for right turn of truck as claimed in claim 4, wherein: the fish-eye camera (1) is arranged on the rear half part of the head of the truck, more than one ultrasonic sensor (2) is arranged and respectively located on the right side of the truck carriage, the signal processing unit (3) is arranged inside the cab, the central control center (4) is arranged inside the cab, and the liquid crystal display screen (5) is arranged inside the cab.
CN201711112829.0A 2017-11-13 2017-11-13 Right-turning safety early warning method for load-carrying truck Active CN107985200B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711112829.0A CN107985200B (en) 2017-11-13 2017-11-13 Right-turning safety early warning method for load-carrying truck

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711112829.0A CN107985200B (en) 2017-11-13 2017-11-13 Right-turning safety early warning method for load-carrying truck

Publications (2)

Publication Number Publication Date
CN107985200A CN107985200A (en) 2018-05-04
CN107985200B true CN107985200B (en) 2020-11-13

Family

ID=62031323

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711112829.0A Active CN107985200B (en) 2017-11-13 2017-11-13 Right-turning safety early warning method for load-carrying truck

Country Status (1)

Country Link
CN (1) CN107985200B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109109748A (en) * 2018-10-08 2019-01-01 南京云计趟信息技术有限公司 A kind of pedestrian's identification early warning system for blind area on the right side of heavy motor truck
CN109345874A (en) * 2018-12-07 2019-02-15 安徽江淮汽车集团股份有限公司 A kind of vehicle right-hand rotation method for early warning and system based on V2X
CN110228413B (en) * 2019-06-10 2020-07-14 吉林大学 Safety early warning system for avoiding pedestrians from being involved under large-scale vehicle during turning
CN110929606A (en) * 2019-11-11 2020-03-27 浙江鸿泉车联网有限公司 Vehicle blind area pedestrian monitoring method and device
TWI763150B (en) * 2020-11-26 2022-05-01 中華電信股份有限公司 System and method for driving safety and accident judgment and computer-readable medium
CN112466159B (en) * 2020-11-30 2022-05-06 浙江科技学院 Right-turning safety early warning system for large vehicle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6661449B1 (en) * 1996-06-06 2003-12-09 Fuji Jukogyo Kabushiki Kaisha Object recognizing apparatus for vehicle and the method thereof
CN102923000A (en) * 2012-11-01 2013-02-13 西南交通大学 Automobile active backing speed limiting control method based on binocular vision target detection
CN105678787A (en) * 2016-02-03 2016-06-15 西南交通大学 Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera
CN205601701U (en) * 2016-03-21 2016-09-28 南京信息工程大学 Freight train bend pedestrian detects warning system
CN106379317A (en) * 2016-09-09 2017-02-08 张晶 Automobile safety auxiliary blind area display system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI458653B (en) * 2011-10-25 2014-11-01 Altek Autotronics Corp Blind spot detection system and blind spot detection method thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6661449B1 (en) * 1996-06-06 2003-12-09 Fuji Jukogyo Kabushiki Kaisha Object recognizing apparatus for vehicle and the method thereof
CN102923000A (en) * 2012-11-01 2013-02-13 西南交通大学 Automobile active backing speed limiting control method based on binocular vision target detection
CN105678787A (en) * 2016-02-03 2016-06-15 西南交通大学 Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera
CN205601701U (en) * 2016-03-21 2016-09-28 南京信息工程大学 Freight train bend pedestrian detects warning system
CN106379317A (en) * 2016-09-09 2017-02-08 张晶 Automobile safety auxiliary blind area display system

Also Published As

Publication number Publication date
CN107985200A (en) 2018-05-04

Similar Documents

Publication Publication Date Title
CN107985200B (en) Right-turning safety early warning method for load-carrying truck
US11417116B2 (en) Vehicular trailer angle detection system
JP7210589B2 (en) Multiple operating modes for extended dynamic range
US8766816B2 (en) System for monitoring the area around a vehicle
US20220111839A1 (en) Methods and systems for providing depth maps with confidence estimates
CN203020198U (en) Auxiliary anti-collision reversing device for forklift
US11912199B2 (en) Trailer hitching assist system with trailer coupler detection
US20210358304A1 (en) Vehicle vision system with cross traffic detection
US11787339B2 (en) Trailer hitching assist system with trailer coupler detection
CN109389060B (en) Vision-based vehicle surrounding collision early warning method
CN112721802A (en) Vehicle-mounted equipment integrating panoramic all-round view and blind area pedestrian early warning and early warning method thereof
US11308641B1 (en) Oncoming car detection using lateral emirror cameras
JP3952305B2 (en) Moving object periphery monitoring device, moving object periphery monitoring method, control program, and readable recording medium
JP2011076214A (en) Obstacle detection device
CN108973918B (en) Device and method for monitoring vehicle blind area
JP5457224B2 (en) Road surface condition detection device
CN110929606A (en) Vehicle blind area pedestrian monitoring method and device
EP3703359A1 (en) Image processing device, image capturing device, driving support device, moving body, and image processing method
CN207115438U (en) Image processing apparatus for vehicle-mounted fisheye camera
Le Guilloux et al. PAROTO project: The benefit of infrared imagery for obstacle avoidance
CN107082049A (en) Vehicle glazing control device
CN112216067B (en) Image processing method based on vehicle-mounted wide-angle camera
WO2019003996A1 (en) Processor, image processing device, mobile entity, image processing method, and program
JP2007069777A (en) Obstacle detection system
CN113139986A (en) Integrated environment perception and multi-target tracking system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant