CN112977393A

CN112977393A - Automatic driving anti-collision avoiding device and method thereof

Info

Publication number: CN112977393A
Application number: CN202110436589.XA
Authority: CN
Inventors: 周宇
Original assignee: Individual
Current assignee: Individual
Priority date: 2021-04-22
Filing date: 2021-04-22
Publication date: 2021-06-18
Anticipated expiration: 2041-04-22
Also published as: WO2022222159A1; CN112977393B

Abstract

The application discloses low-cost automatic driving anti-collision avoidance device and method based on ' camera + millimeter wave radar + laser emitter ' as main sensor relates to automobile automatic driving technical field to solve this difficult problem that current ' camera + millimeter wave radar ' can't discern static barrier as main sensor. The application autopilot anticollision avoid device, include: the laser emitter is used for emitting a laser beam in a specific direction; the camera acquires an image containing laser beam generated light spots; the image processing module is used for acquiring data transmitted by the camera to perform calculation processing and analyzing whether an obstacle exists in front; the control unit is used for acquiring real-time feedback data of the signal processing unit and analyzing whether collision risks exist or not; and the anti-collision execution unit executes the control of the control unit and performs automatic driving anti-collision avoidance operation.

Description

Automatic driving anti-collision avoiding device and method thereof

Technical Field

The application relates to an automatic driving anti-collision avoidance device and a method thereof, in particular to an anti-collision device and a method for detecting a static obstacle by an automatic driving vehicle with a camera and a millimeter wave radar as main sensors.

Background

The mainstream sensors mounted on vehicles having an automatic driving function at present include cameras, millimeter wave radars, and laser radars. Laser radars are not used in bulk due to their high cost and low stability. The existing automatic driving scheme mainly adopts a camera and a millimeter wave radar as a main sensor. However, the use of "camera + millimeter wave radar" as the main sensor has been a difficulty in recognizing static obstacles. In recent years, several fatal accidents have occurred in an autonomous vehicle using a camera + millimeter wave radar as a main sensor, and the vehicle crashes in a vertical direction while an autonomous driving system of level L2 is turned on (L2 is one of the evaluation criteria for the level of autonomous driving at the present stage: the driver is responsible for monitoring the road surface and realizing partial autonomous driving). The reason for these accidents is that the "camera + millimeter wave radar" does not determine the stationary obstacle, and finally the system does not react.

Disclosure of Invention

The main object of the present disclosure is to provide a low-cost automatic driving anti-collision avoidance apparatus and method based on "camera + millimeter wave radar + laser transmitter" as a main sensor.

The application discloses low-cost autopilot anticollision avoiding device based on "camera + millimeter wave radar + laser emitter" as main sensor, this autopilot anticollision avoiding device possesses: the laser emitter is used for emitting a laser beam in a specific direction; the camera acquires an image containing laser beam generated light spots; the image processing module is used for acquiring data transmitted by the camera to perform calculation processing and analyzing whether an obstacle exists in front; the control unit is used for acquiring real-time feedback data of the signal processing unit and analyzing whether collision risks exist or not; and the anti-collision execution unit executes the control of the control unit and performs automatic driving anti-collision avoidance operation.

The camera is a common camera chip, including but not limited to a CCD image sensor and a COMS image sensor; the laser emitter emits a light beam that is recognizable by the camera, including but not limited to visible light.

As a further improvement of the application, the camera firstly judges whether an obstacle exists in front, if not, the image processing module judges the safe position where the laser beam can irradiate, the control unit operates the laser transmitter to emit the laser beam in the specific direction, and the laser beam irradiates the safe position to form a light spot.

As a further improvement of the present application, the image processing unit determining unsafe positions where the laser beam can be irradiated includes: 1) The position directly or indirectly irradiating the human face part, 2) the surface of flammable and explosive substances, and 3) the surface of an object which has a reflection function and cannot judge whether the laser is safe or not after being reflected, but the unsafe position is not limited to the above.

As a further improvement of the application, the installation distance between the laser emitter and the camera in the vertical direction of the ground is more than 10 cm.

As a further refinement of the present application, laser emitters include, but are not limited to: a point cloud laser emitter and a rotatable laser emitter.

As a further improvement of the present application, the laser generator may emit laser beams of multiple colors sequentially or simultaneously.

As a further improvement of the present application, the apparatus further comprises: and the motion state sensor is used for judging the inclination, acceleration and steering states of the vehicle and transmitting the states to the control unit to correct the deviation of the laser emission direction.

As a further improvement of the present application, the motion state sensor is integrated on the control unit main board.

The application also discloses a low-cost automatic driving anti-collision avoidance method based on the camera, the millimeter wave radar and the laser transmitter as a main sensor, which comprises the following steps: the signal processing unit receives data from a camera and a microwave radar which are installed on the automatic driving vehicle and judges whether an obstacle exists in the front; if not, the image processing module judges the safe position which can be irradiated by the laser beam, and the control unit receives the safe position signal and controls the laser emitter to emit the laser beam to irradiate the safe position; the camera receives light spot image data formed by the laser beam and transmits the light spot image data to the image processing module, and the image processing module judges whether an obstacle exists in front again according to the position of the laser light spot; in the process, as long as the obstacle in front is detected, the control unit judges whether potential safety hazards exist or not and controls the anti-collision execution unit to perform automatic driving anti-collision avoidance operation.

The application provides a low-cost autopilot anticollision avoidance device and method based on "camera + millimeter wave radar + laser emitter" as the main sensor, has compensatied this difficult problem that "camera + millimeter wave radar" can't discern static barrier as the main sensor, and simple structure is with low costs moreover, has very high using value.

Drawings

To more clearly illustrate the detailed description of the present application or the technical solutions in the prior art, the following description will be directed to specific embodiments

Brief description of the drawingsthe drawings needed for implementation or prior art description are briefly described, and it is apparent that in the following description

The drawings illustrate some embodiments of the invention and will be apparent to those skilled in the art without the benefit of this disclosure

It is also possible to derive other figures from these figures.

Fig. 1 is a schematic view of an automatic driving collision avoidance apparatus according to an embodiment of the present application.

Fig. 2 is a schematic diagram of determining an obstacle through laser ranging according to an embodiment of the present application.

Fig. 3 is a diagram of a condition that a vehicle according to an embodiment of the present application corrects a deviation of a laser beam direction due to a height difference.

Fig. 4 is a flowchart of collision avoidance processing provided in the embodiment of the present application.

Detailed Description

The technical solution of the present application will be described clearly and completely with reference to the accompanying drawings, and obviously, the described implementation

Examples are a part of the disclosure, and not all examples. Based on the embodiments in this application, the ordinary skill in the art

All other embodiments obtained by a person without inventive labour are within the scope of protection of this application.

In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience in describing the present application and simplifying the description, but do not indicate or imply that the system or element being referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

In the description of the present application, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; the connection can be mechanical connection or electrical connection; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art.

Hereinafter, an embodiment embodied as an automatic driving collision avoidance apparatus mounted on a vehicle will be described with reference to the drawings. The collision avoidance device according to the present embodiment emits a laser beam in a specific direction to detect whether an obstacle exists in front. First, a schematic configuration of a collision avoidance device for a vehicle according to the present embodiment will be described with reference to fig. 1. It should be understood that the apparatus and methods of the present application may be used with any type of vehicle, including conventional vehicles, Hybrid Electric Vehicles (HEVs), Extended Range Electric Vehicles (EREVs), electric-only vehicles (BEVs), motorcycles, passenger vehicles, Sport Utility Vehicles (SUVs), cross-country vehicles, trucks, vans, buses, Recreational Vehicles (RVs), and the like. These are just some of the possible applications, as the devices and methods described herein are not limited to the exemplary embodiments shown in fig. 1-4, and may be implemented in a number of different ways.

In fig. 1, a vehicle (50) includes: sensor unit (10) comprising: the system comprises a camera (101) for image acquisition, a microwave radar (102) for front obstacle detection, a motion state sensor (103) for vehicle inclination, acceleration and steering state identification, and a laser transmitter (104) for laser beam emission; signal processing unit (20) comprising: the device comprises an image processing module (201) for receiving camera image data for image processing, a radar processing module (202) for microwave radar data processing and control, a motion state processing module (203) for processing, converting and analyzing motion state sensor data, and a laser control module (204) for controlling laser beam emission; the control unit (30) is used for receiving and processing the data transmitted by the signal processing unit, controlling the signal processing unit to further detect signals and finally controlling the anti-collision execution unit to perform anti-collision avoiding action; an anti-collision execution unit (40) comprising: the vehicle speed controller (401) is used for executing the control unit signal and reducing or closing the oil path to control the vehicle speed, the brake device (402) is used for executing the control unit signal and switching on the brake circuit to brake the vehicle, the instrument (403) is used for executing the control unit signal and displaying the front obstacle alarm, and the brake lamp (404) is used for executing the control unit signal and reminding the rear vehicle.

The camera (101) is a common camera chip, including but not limited to a CCD image sensor and a COMS image sensor; the device is used for acquiring image signals, can identify light beams emitted by the laser emitter (104), can acquire image signals containing light spots formed by the light beams emitted by the laser emitter (104), and transmits the image signals to the image processing module (201) for calculation and analysis.

The microwave radar (102) adopts microwave with the frequency of 300MHz-3000GHz and the wavelength of 0.1 mm-10 m as a signal source. The microwave is a general name of decimetric wave, centimeter wave, millimeter wave, submillimeter wave and millimeter wave, so the microwave radar also includes millimeter wave radar. The microwave has good directivity, the speed is equal to the light speed, the microwave is reflected back when meeting the vehicle and then is received by the radar velometer, and therefore once the microwave reaches the target speed, the speed of the vehicle to be measured can be displayed on the nixie tube within a time of tens of ten-thousandth of seconds. When a person or object moves within the sensing range of the microwave, the sensor is activated. Therefore, microwave radars are widely applied to detection of front obstacles in automatic driving, but the microwave radars cannot identify static obstacles and is always a difficult problem in automatic driving.

The motion state sensor (103) includes: the accelerometer is characterized by comprising a 3-axis accelerometer, a 3-axis gyroscope, a 3-axis magnetometer, a barometric pressure sensor and an inclination angle sensor, wherein the sensors can be one or more of the combination. The relevant signals can be collected and transmitted to the motion state processing module (203) for calculation and analysis.

The laser transmitter (104) may be instructed by the control unit (30) to emit a laser beam in a particular direction, including but not limited to: a point cloud laser emitter and a rotatable laser emitter. And can emit laser beams with multiple light colors sequentially or simultaneously. And forming a light spot at the confirmed safety position, wherein the light spot image data is used for judging whether an obstacle exists in front.

The image processing module (201) is used for receiving the image data transmitted by the camera (101), analyzing whether an obstacle exists in front or not through an algorithm, if not, continuously analyzing the image data of the laser light spots, calculating and analyzing whether the obstacle exists in front or not and the distance of the obstacle according to whether the light spots deviate or not and the deviation amount, and transmitting the result to the control unit.

The radar processing module (202) receives microwave radar (102) data, analyzes the obstacle ahead, and transmits the analysis result to the control unit.

The motion state processing module (203) receives the motion state sensor data, calculates the vehicle inclination, acceleration and steering states through fusion of Kalman filtering, particle filtering and complementary filtering algorithms, and transmits the data to the control unit for deviation correction of the laser beam emission angle.

The laser control module (204) receives the control unit control signal and controls the laser transmitter to emit the laser beam in a specific direction.

The control unit (30) is equipped with a microcomputer, an interface for a wire harness, and the like. The microcomputer has a known configuration including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an I/O and CAN (Controller Area Network) communication device. The control unit (30) is mainly used for being connected with the signal processing unit, receiving the sensing signal analysis data, sending the data to control the laser transmitter, calculating and analyzing whether the front barrier is safe or not, and controlling the anti-collision execution unit (40) to perform anti-collision avoiding action.

The vehicle speed regulator (401) is used for reducing or closing an oil path to control the engine to restrain driving force so as to regulate and control the vehicle speed.

The braking device (402) may be part of any suitable vehicle braking system, including systems associated with disc brakes, drum brakes, electro-hydraulic brakes, electro-mechanical brakes, regenerative brakes, line control brakes, and the like.

An alarm indicator lamp is arranged on the instrument (403), when safety risks occur on the obstacle at the front side, the alarm indicator lamp gives out an alarm, and the driver is prompted to take over further measures while automatic collision avoidance is achieved.

When the brake lamp (404) judges that the front obstacle needs to be avoided, the brake device (402) is automatically started, and meanwhile, the brake lamp is turned on to prompt the rear vehicle to give an early warning for avoiding.

Turning now to fig. 2, a schematic diagram of laser ranging barrier determination in an autonomous driving collision avoidance device is shown. As shown in fig. 2 (i), when the vehicle (50) is configured with the camera (101) to feed back images for the first time and determine that no obstacle exists in front of the vehicle, the image processing module (201) continues to determine whether the point C is safe or not and whether a pedestrian or an inflammable and explosive article exists or not, and if not, the laser transmitter (104) transmits a laser beam to irradiate the point C. The camera (101) collects the laser spot forming image for the second time. Point C will appear at the point C imaging position, as shown in fig. 2 (iii) (a), and the absence of an obstacle in front is secondarily confirmed.

If the image processing module (201) processes the first feedback image and makes a misjudgment, and an unidentified obstacle (60) is actually arranged in the front of the vehicle, according to the light linear propagation principle, a laser beam (a connecting line of a point B and a point C at the center of a laser emitter) can form an E-point light spot on the obstacle (60), a vertical section X passing through the E point is formed in the vehicle driving direction, and the imaging of the E point on the X section is shown in fig. 2 (III) (B). According to the principle of ray linear propagation, the imaging point of the original point C on the X section is the intersection point D of the AC line (the connecting line of the point C and the point A of the camera center) and the X section, as shown in fig. 2 (III) (b). As shown in fig. 2 (iii) (b), the acquired image of the camera (101) is shifted by x distance, with point C and point D coinciding with each other and point E being located below point C. Therefore, if the point C is not at the preset position of the point C but has an offset of the distance x, the image processing module (201) can judge that an obstacle exists in front of the point C according to the facula image, and the first image judges that misjudgment exists.

If the obstacle (60) is located farther away from the vehicle (50), such as the obstacle (70) in fig. 2 (i), the laser beam (line connecting the center B and C of the laser transmitter) forms an F spot on the X' section of the obstacle (70). Similarly, in the formed image on the camera (101), the point F and the point G are the same, and the distance between the point F (G) and the point c (d) is x', as shown in fig. 2 (iii) (c). As can be seen from fig. 2 (ii): point f (g) is below point C (d) and point E is below point f (g), so the farther the imaging point is from the predetermined imaging point C, the closer the obstacle is to the vehicle (50).

On the other hand, the distance between the obstacle and the vehicle (50) can be calculated by the distance between the imaging point E and the preset imaging point C. As shown in fig. 2 (ii), first, the difference X between two points on the X section can be obtained by calculating the pixel position difference between the point E and the point C on the image. Then, the distance d1 between the obstacle and the vehicle (50), the distance d from the vehicle (50) to the point C, the distance from the projection point H of the point E to the ground to the point C is d2, and d1= d-d 2; y is the distance from the center B of the laser emitter to the ground, alpha is the included angle between the laser beam BC and the ground, and d = y ÷ tan alpha; the distance between the point E and the point D on the section X is X, the included angle between the connecting line of the point A and the point C at the center of the camera and the ground is beta, and D2= X ÷ (tan beta-tan alpha); and the distance d1= y/tan alpha-x/(tan beta-tan alpha) between the obstacle and the vehicle (50) is obtained by substituting the two formulas into d1= d-d 2.

As can be seen from the formula d1= y/tan α -x/(tan β -tan α), the calculation accuracy of d1 is determined by x and y. And y is the mounting height of the laser probe from the ground, and the larger the obtained laser mounting height y is, the higher the test precision is. And x is actually related to BA ', where a' is the intersection point of the AC connecting line and the vertical section of the vehicle advancing direction passing through the point B, and BA 'is related to the installation distance of the camera and the laser transmitter in the vertical direction of the ground, and if the BA' distance is too small, the measurement accuracy is reduced, all requirements are that: the installation distance between the laser transmitter and the camera in the vertical direction of the ground is more than 10 cm.

In the above description case, only one laser beam is introduced for the sake of clear explanation, and in the actual use case, the laser beam may be multiple beams, and the emission time of different beams may also be different, and the light color of the laser emitted by different beams may also be different, which is more complicated than the above case according to the actual situation.

Further, as shown in fig. 3, a more complicated situation is introduced, when the vehicle (50) encounters an uphill or downhill slope or uneven front and back, the laser transmitter will have an angle η deviation with respect to the horizontal position, which directly results in a change in the original laser emission angle γ. At the moment, the actual inclination angle of the vehicle is judged by means of the motion state sensor (103), and the actual inclination angle is converted and fed back to the control unit (30) to correct the real-time laser emission angle delta. Furthermore, besides the vehicle inclination, the motion state sensor (103) can also judge the speed, the acceleration and the steering state of the vehicle, and further correct the direction of the laser emission beam or the position point of the image processing.

Fig. 4 is a flow chart illustrating a method according to an embodiment of the present application. It should be understood that although the steps in the flowchart of fig. 4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, in different orders, and may be performed alternately or at least partially with respect to other steps or sub-steps or stages of other steps. The processing flow of the collision avoidance apparatus according to the present embodiment repeatedly executes the processing of fig. 4 for each predetermined control cycle.

First, the camera and the microwave radar collect data and transmit the data to the signal processing unit (S101). If the signal processing unit detects an obstacle (S102: YES), the data is transmitted to the control unit. The control unit judges whether a potential safety hazard exists or not and controls the collision avoidance execution unit to perform an automatic driving collision avoidance operation (S109). If the signal processing unit does not detect an obstacle (S102: No), the image processing module confirms the laser safety irradiation point (S103) and transmits the safety irradiation point coordinates to the control unit. The control unit corrects the irradiation direction according to the coordinate position in combination with the position state sensor data (S104), and transmits correct irradiation coordinates to the laser control module. The laser control module controls the laser emitter to emit laser searchlight according to the correct coordinates transmitted by the control unit (S105). The camera collects laser spot image data for the second time (S106) and transmits the laser spot image data to the image processing module. The image processing module extracts a laser spot detection point according to the image data (S107), and analyzes whether an obstacle exists in front or not according to the spot position deviation condition (S108). If the signal processing unit detects an obstacle (S108: YES), the data is transmitted to the control unit. The control unit judges whether a potential safety hazard exists or not and controls the collision avoidance execution unit to perform an automatic driving collision avoidance operation (S109). If the signal processing unit does not detect an obstacle (S108: NO), the flow ends and returns to the operation of starting to enter the next cycle.

In summary, the embodiment of the application provides a low-cost automatic driving anti-collision avoidance device and method based on a camera, a millimeter wave radar and a laser transmitter serving as a main sensor, which makes up the difficult problem that the camera and the millimeter wave radar serving as the main sensor cannot identify static obstacles, and has the advantages of simple structure, low cost and high application value.

The present disclosure has been described in terms of embodiments, but it should be understood that the present disclosure is not limited to the embodiments and configurations. The present disclosure also includes various modifications and variations within an equivalent range. In addition, various combinations and modes, and other combinations and modes including only one element, one or more elements, or one or less elements are also included in the scope and spirit of the present disclosure.

Claims

1. A collision avoidance apparatus for automatic driving, the apparatus comprising:

sensor unit (10) comprising: the laser emitter (104) is used for emitting laser beams in a specific direction, and the camera (101) acquires images containing laser beam generating spots;

signal processing unit (20) comprising: the image processing module (201) acquires data transmitted by the camera for calculation processing, and analyzes whether an obstacle exists in front;

the control unit (30) is used for acquiring real-time feedback data of the signal processing unit and analyzing whether collision risks exist or not;

and a collision avoidance execution unit (40) which executes the control of the control unit and performs an automatic driving collision avoidance operation.

2. The apparatus of claim 1, wherein: the camera firstly judges whether an obstacle exists in front or not, if not, the image processing module judges the safe position where the laser beam can irradiate, the control unit operates the laser transmitter to emit the laser beam in the specific direction, and the laser beam irradiates the safe position to form a light spot.

3. The apparatus of claim 2, wherein: the image processing unit determining unsafe positions where the laser beam can be irradiated comprises: 1) The position directly or indirectly irradiating the human face part, 2) the surface of flammable and explosive substances, and 3) the surface of an object which has a reflection function and cannot judge whether the laser is safe or not after being reflected, but the unsafe position is not limited to the above.

4. The apparatus of claim 1, wherein: the installation distance between the laser transmitter and the camera in the vertical direction of the ground is more than 10 cm.

5. The apparatus of claim 1, wherein: laser emitters include, but are not limited to: a point cloud laser emitter and a rotatable laser emitter.

6. The apparatus of claim 1, wherein: the laser generator can emit laser beams with multiple light colors sequentially or simultaneously.

7. The apparatus of claim 1, wherein: the sensor unit further includes: the motion state sensor (103) judges the vehicle inclination, acceleration and steering states and transmits them to the control unit to correct the deviation of the laser emitting direction.

8. The method of claim 7, wherein: the motion state sensor is integrated on the control unit main board.

9. A method for automated driving collision avoidance, the collision avoidance method comprising the steps of:

a) the signal processing unit receives data from a camera and a microwave radar which are installed on the automatic driving vehicle and judges whether an obstacle exists in the front;

b) if not, the image processing module judges the safe position where the laser beam can irradiate, and the control unit receives a safe position signal and controls the laser generator to emit the laser beam to irradiate the safe position;

c) the camera receives image data during laser irradiation and transmits the image data to the image processing module, and the image processing module judges whether an obstacle exists in front again according to the position of a laser spot;

d) in the process, as long as the obstacle in front is detected, the control unit judges whether potential safety hazards exist or not and controls the anti-collision execution unit to perform automatic driving anti-collision avoidance operation.