WO2021097844A1 - 基于多传感器数据融合的护栏估计方法和车载设备 - Google Patents

基于多传感器数据融合的护栏估计方法和车载设备 Download PDF

Info

Publication number
WO2021097844A1
WO2021097844A1 PCT/CN2019/120407 CN2019120407W WO2021097844A1 WO 2021097844 A1 WO2021097844 A1 WO 2021097844A1 CN 2019120407 W CN2019120407 W CN 2019120407W WO 2021097844 A1 WO2021097844 A1 WO 2021097844A1
Authority
WO
WIPO (PCT)
Prior art keywords
guardrail
vehicle
sample points
fitting
information
Prior art date
Application number
PCT/CN2019/120407
Other languages
English (en)
French (fr)
Inventor
王子涵
叶凌峡
Original Assignee
驭势（上海）汽车科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 驭势（上海）汽车科技有限公司 filed Critical 驭势（上海）汽车科技有限公司
Priority to US17/916,038 priority Critical patent/US20230168368A1/en
Priority to CN201980002607.4A priority patent/CN113227831B/zh
Priority to PCT/CN2019/120407 priority patent/WO2021097844A1/zh
Publication of WO2021097844A1 publication Critical patent/WO2021097844A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G01S13/862—Combination of radar systems with sonar systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G01S13/867—Combination of radar systems with cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9318—Controlling the steering
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/93185—Controlling the brakes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/932—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles using own vehicle data, e.g. ground speed, steering wheel direction
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9323—Alternative operation using light waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88—Radar or analogous systems specially adapted for specific applications
    • G01S13/93—Radar or analogous systems specially adapted for specific applications for anti-collision purposes
    • G01S13/931—Radar or analogous systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G01S2013/9324—Alternative operation using ultrasonic waves

Definitions

  • the embodiments of the present disclosure relate to the field of intelligent driving technology, and in particular to a guardrail estimation method based on multi-sensor data fusion, on-board equipment, and storage media.
  • the first scene to realize advanced assisted driving and semi-autonomous driving is the highway.
  • the highway working conditions are relatively simple, and the lane line quality is better, which is easier to implement.
  • the lateral control is mainly based on the lane line to keep the controlled vehicle in the center of the lane.
  • cameras and millimeter-wave radars are the main sensing sensors, and the perception of lane lines depends on the camera.
  • the single dependence on the camera to perceive the lane lines as the basis for lateral control will lead to system performance and stability. Sex becomes worse.
  • guardrail estimation solution to obtain guardrail information, which can then be used as a supplement to the lane line to improve the performance of lateral control from different data sources.
  • At least one embodiment of the present disclosure provides a guardrail estimation method based on multi-sensor data fusion, a vehicle-mounted device, and a storage medium.
  • an embodiment of the present disclosure proposes a guardrail estimation method based on multi-sensor data fusion, and the method includes:
  • the guardrail information is estimated.
  • the embodiments of the present disclosure also propose a vehicle-mounted device, including: a processor and a memory; the processor is configured to execute the steps of the method described in the first aspect by calling a program or instruction stored in the memory.
  • the embodiments of the present disclosure also propose a non-transitory computer-readable storage medium for storing a program or instruction, and the program or instruction causes a computer to execute the steps of the method described in the first aspect.
  • the guardrail information when the vehicle is laterally controlled, the guardrail information corrects the poor quality lane line; when the lane line cannot be detected, the guardrail information is used for the lateral control degradation processing, and the guardrail information is beneficial Assist to achieve fast and accurate highway lane-level positioning, and achieve higher-level assisted driving functions.
  • FIG. 1 is an overall architecture diagram of an intelligent driving vehicle provided by an embodiment of the present disclosure
  • Fig. 2 is a block diagram of an intelligent driving system provided by an embodiment of the present disclosure
  • Fig. 3 is a block diagram of a guardrail estimation module provided by an embodiment of the present disclosure
  • FIG. 4 is a block diagram of a vehicle-mounted device provided by an embodiment of the present disclosure.
  • FIG. 5 is a flowchart of a guardrail estimation method based on multi-sensor data fusion according to an embodiment of the present disclosure
  • Fig. 6 is a schematic diagram of a multi-sensor arrangement provided by an embodiment of the present disclosure.
  • FIG. 7 is a scene in which the driving trajectory and the road boundary shape are inconsistent according to an embodiment of the present disclosure.
  • embodiments of the present disclosure provide a guardrail based on multi-sensor data fusion
  • the estimation scheme can stably output whether there is a guardrail and estimate the shape of the guardrail when the highway has guardrails.
  • the guardrail information corrects the lane lines with poor quality; the lane lines cannot be detected.
  • the guardrail information is used for lateral control downgrade processing, and the guardrail information is helpful to assist in realizing fast and accurate expressway lane-level positioning, and realize higher-level driving assistance functions.
  • FIG. 1 is an overall architecture diagram of an intelligent driving vehicle provided by an embodiment of the disclosure.
  • the intelligent driving vehicle includes: a sensor group, an intelligent driving system 100, a vehicle underlying execution system, and other components that can be used to drive the vehicle and control the operation of the vehicle.
  • the sensor group is used to collect the data of the external environment of the vehicle and detect the position data of the vehicle.
  • the sensor group includes, but is not limited to, at least one of a camera, a lidar, a millimeter wave radar, an ultrasonic radar, a GPS (Global Positioning System, Global Positioning System), and an IMU (Inertial Measurement Unit), for example.
  • the sensor group is also used to collect dynamics data of the vehicle.
  • the sensor group includes, but is not limited to, at least one of a wheel speed sensor, a speed sensor, an acceleration sensor, a steering wheel angle sensor, and a front wheel angle sensor, for example.
  • the intelligent driving system 100 is used to obtain data of a sensor group, and all sensors in the sensor group transmit data at a higher frequency during the driving of the intelligent driving vehicle.
  • the intelligent driving system 100 is also used for environmental perception and vehicle positioning based on the data of the sensor group, path planning and decision-making based on environmental perception information and vehicle positioning information, and generating vehicle control instructions based on the planned path, thereby controlling the vehicle according to the plan Route driving.
  • the intelligent driving system 100 is also used for guardrail estimation based on multi-sensor data fusion to obtain guardrail information.
  • the intelligent driving system 100 obtains multi-sensor data and vehicle information; and then determines the vehicle's travel trajectory based on the vehicle information; thereby determines multiple guardrail sample points based on the multi-sensor data and travel trajectory; Guardrail sample point, estimate guardrail information.
  • the intelligent driving system 100 may be a software system, a hardware system, or a combination of software and hardware.
  • the intelligent driving system 100 is a software system that runs on an operating system
  • the on-board hardware system is a hardware system that supports the operation of the operating system.
  • the intelligent driving system 100 is also used for wireless communication with a cloud server to exchange various information.
  • the intelligent driving system 100 and the cloud server perform wireless communication via wireless communication networks (for example, including but not limited to wireless communication networks such as GPRS network, Zigbee network, Wifi network, 3G network, 4G network, 5G network, etc.).
  • wireless communication networks for example, including but not limited to wireless communication networks such as GPRS network, Zigbee network, Wifi network, 3G network, 4G network, 5G network, etc.
  • the cloud server is used to coordinate the management of intelligent driving vehicles. In some embodiments, the cloud server may be used to interact with one or more intelligent driving vehicles, to coordinate and manage the scheduling of multiple intelligent driving vehicles, and so on.
  • the cloud server is a cloud server established by a vehicle service provider to provide cloud storage and cloud computing functions.
  • the vehicle file is created in the cloud server.
  • various information uploaded by the intelligent driving system 100 is stored in the vehicle file.
  • the cloud server can synchronize the driving data generated by the vehicle in real time.
  • the cloud server may be a server or a server group.
  • Server groups can be centralized or distributed. Distributed server is conducive to task allocation and optimization among multiple distributed servers, and overcomes the shortcomings of traditional centralized server resource shortage and response bottleneck.
  • the cloud server may be local or remote.
  • the cloud server can be used to charge the vehicle for parking, tolls, and so on. In some embodiments, the cloud server is also used to analyze the driving behavior of the driver and evaluate the safety level of the driving behavior of the driver.
  • the cloud server can be used to obtain information about the road side unit (RSU: Road Side Unit) and the intelligent driving vehicle, and can send the information to the intelligent driving vehicle.
  • the cloud server may send the detection information corresponding to the intelligent driving vehicle in the road monitoring unit to the intelligent driving vehicle according to the information of the intelligent driving vehicle.
  • the road monitoring unit may be used to collect road monitoring information.
  • the road monitoring unit may be an environmental sensing sensor, such as a camera, a lidar, etc., or a road device, such as a V2X device, a roadside traffic light device, and the like.
  • the road monitoring unit may monitor the road conditions subordinate to the corresponding road monitoring unit, for example, the type, speed, priority level, etc. of passing vehicles. After the road monitoring unit collects the road monitoring information, the road monitoring information can be sent to the cloud server, or can be sent to the intelligent driving vehicle passing the road.
  • the bottom-level execution system of the vehicle is used to receive vehicle control instructions to control the driving of the vehicle.
  • the vehicle bottom-level execution system includes, but is not limited to: a steering system, a braking system, and a driving system.
  • the steering system, the braking system and the driving system are mature systems in the vehicle field and will not be repeated here.
  • the intelligent driving vehicle may further include a vehicle CAN bus that is not shown in FIG. 1, and the vehicle CAN bus is connected to the underlying execution system of the vehicle.
  • the information interaction between the intelligent driving system 100 and the underlying execution system of the vehicle is transmitted through the vehicle CAN bus.
  • the intelligent driving vehicle can be controlled by the driver and the intelligent driving system 100 to control the vehicle.
  • the driver drives the vehicle by operating a device that controls the traveling of the vehicle.
  • the devices that control the traveling of the vehicle include, but are not limited to, a brake pedal, a steering wheel, and an accelerator pedal, for example.
  • the device for controlling the driving of the vehicle can directly operate the execution system at the bottom of the vehicle to control the driving of the vehicle.
  • the intelligent driving vehicle may also be an unmanned vehicle, and the driving control of the vehicle is executed by the intelligent driving system 100.
  • FIG. 2 is a block diagram of an intelligent driving system 200 provided by an embodiment of the disclosure.
  • the intelligent driving system 200 may be implemented as the intelligent driving system 100 or a part of the intelligent driving system 100 in FIG. 1 for controlling the driving of the vehicle.
  • the intelligent driving system 200 may be divided into multiple modules, for example, may include: a perception module 201, a planning module 202, a control module 203, a guardrail estimation module 204, and other modules that can be used for intelligent driving.
  • the perception module 201 is used for environmental perception and positioning.
  • the sensing module 201 is used to obtain data such as sensor data, V2X (Vehicle to X, wireless communication for vehicles) data, and high-precision maps.
  • the sensing module 201 is configured to perform environmental perception and positioning based on at least one of acquired sensor data, V2X (Vehicle to X, wireless communication for vehicles) data, high-precision maps and other data.
  • the perception module 201 is used to generate perception positioning information to realize obstacle perception, recognition of the drivable area of the camera image, and positioning of the vehicle.
  • Environmental Perception can be understood as the ability to understand the scene of the environment, such as the location of obstacles, the detection of road signs/marks, the detection of pedestrians/vehicles, and the semantic classification of data.
  • environment perception can be realized by fusing data from multiple sensors such as cameras, lidars, millimeter wave radars, and so on.
  • Positioning can be: GPS positioning, GPS positioning accuracy is tens of meters to centimeters, high positioning accuracy; positioning can also use GPS and inertial navigation system (Inertial Navigation System) positioning method. Positioning can also use SLAM (Simultaneous Localization And Mapping, simultaneous positioning and map construction). The goal of SLAM is to construct a map while using the map for positioning. SLAM uses the observed environmental features to determine the current vehicle's location and current observation features s position.
  • V2X is the key technology of the intelligent transportation system, which enables the communication between vehicles, vehicles and base stations, base stations and base stations, so as to obtain a series of traffic information such as real-time road conditions, road information, pedestrian information, etc., to improve the safety of intelligent driving and reduce Congestion, improve traffic efficiency, provide on-board entertainment information, etc.
  • High-precision maps are geographic maps used in the field of intelligent driving. Compared with traditional maps, the differences are: 1) High-precision maps include a large amount of driving assistance information, for example, relying on the accurate three-dimensional representation of the road network: including intersections and intersections. The location of road signs, etc.; 2) The high-precision map also includes a lot of semantic information, such as reporting the meaning of different colors on the traffic lights, and for example indicating the speed limit of the road, and the position of the left turn lane; 3) The high-precision map can reach centimeters Class accuracy ensures the safe driving of intelligent driving vehicles.
  • the planning module 202 is configured to perform path planning and decision-making based on the perception positioning information generated by the perception module 201.
  • the planning module 202 is configured to perform path planning and decision-making based on the perception positioning information generated by the perception module 201 in combination with at least one of V2X data, high-precision maps and other data.
  • the planning module 202 is used to plan a route and make decisions: behaviors (including but not limited to following, overtaking, stopping, detouring, etc.), vehicle heading, vehicle speed, desired acceleration of the vehicle, desired steering wheel angle And so on, generate planning decision information.
  • the control module 203 is configured to perform path tracking and trajectory tracking based on the planning decision information generated by the planning module 202.
  • control module 203 is used to generate control instructions for the vehicle's bottom-level execution system, and issue control instructions so that the vehicle's bottom-level execution system controls the vehicle to drive along a desired path, for example, by controlling the steering wheel, brakes, and accelerator to control the vehicle. Horizontal and vertical control.
  • control module 203 is also used to calculate the front wheel angle based on the path tracking algorithm.
  • the desired path curve in the path tracking process has nothing to do with time parameters.
  • tracking control it can be assumed that the intelligent driving vehicle is moving at a constant speed at the current speed, and the driving path is approached to the desired path at a certain cost rule; and the trajectory
  • the expected path curve is related to time and space, and the intelligent driving vehicle is required to reach a preset reference path point within a specified time.
  • Path tracking is different from trajectory tracking. It is not subject to time constraints and only needs to track the desired path within a certain error range.
  • the guardrail estimation module 204 is configured to perform guardrail estimation based on multi-sensor data fusion to obtain guardrail information.
  • the guardrail estimation module 204 obtains multi-sensor data and vehicle information; and then determines the vehicle's driving trajectory based on the entire vehicle information; thereby determining multiple guardrail sample points based on the multi-sensor data and driving trajectory; Guardrail sample point, estimate guardrail information.
  • the function of the guardrail estimation module 204 can be integrated into the perception module 201, the planning module 202, or the control module 203, or it can be configured as a module independent of the intelligent driving system 200, and the guardrail estimation module 204 can be a software module. , Hardware modules or a combination of software and hardware modules.
  • the guardrail estimation module 204 is a software module running on an operating system
  • the vehicle-mounted hardware system is a hardware system that supports the running of the operating system.
  • FIG. 3 is a block diagram of a guardrail estimation module 300 provided by an embodiment of the disclosure.
  • the guardrail estimation module 300 may be implemented as the guardrail estimation module 204 or a part of the guardrail estimation module 204 in FIG. 2.
  • the guardrail estimation module 300 may include but is not limited to the following units: an acquisition unit 301, a first determination unit 302, a second determination unit 303, and an estimation unit 304.
  • the acquiring unit 301 is used to acquire multi-sensor data and vehicle information.
  • multiple sensors are arranged on the vehicle, at least for realizing forward detection and lateral detection of the vehicle.
  • the multi-sensor may include, but is not limited to: Millimeter-Wave Radar, visual sensor, and Ultrasonic Radar.
  • the visual sensor is, for example, a camera. Millimeter-wave radar and vision sensors realize forward detection, and ultrasonic radar realizes lateral detection of vehicles.
  • the arrangement of multiple sensors is shown in Figure 6.
  • the Millimeter-Wave Radar is arranged on the front bumper of the vehicle
  • the Camera is arranged directly above the front windshield of the vehicle
  • the Ultrasonic Radar is arranged on the front of the vehicle.
  • the arrangement of multi-sensors is not limited to the arrangement shown in Fig. 6, and other ways are also possible.
  • Ultrasonic Radar is also arranged on the left front side of the vehicle. This embodiment does not limit the specific arrangement of multi-sensors. It does not limit the specific installation location of the multi-sensor, nor does it limit the specific type of the multi-sensor.
  • the millimeter wave radar data includes at least track information or track information of a moving object or a stationary object, where the track information includes at least the distance, azimuth, range change rate, and radar cross section of the detected target;
  • the information includes at least range (Range), range rate (Range Rate), angle (Angle), radar cross section (RCS).
  • the visual sensor data is used at least to identify the drivable area, and then the boundary points of the drivable area can be determined, so as to obtain the position information and category information of the boundary points of the traveling area.
  • the position information includes a horizontal position and a vertical position.
  • the lateral direction can be understood as the transverse direction of the vehicle, and the longitudinal direction can be understood as the longitudinal direction of the vehicle.
  • the category information includes: guardrail, non-guardrail.
  • the ultrasonic radar data includes at least the distance of the closest target detected.
  • the guardrail detection within 3 meters can be realized.
  • the entire vehicle information is used at least to calculate the driving trajectory of the vehicle and guardrail tracking.
  • the entire vehicle information may include, but is not limited to: vehicle speed, yaw rate, and steering wheel angle.
  • the first determining unit 302 is configured to determine the driving trajectory of the vehicle based on the entire vehicle information. In some embodiments, the first determining unit 302 determines the turning radius of the vehicle based on the entire vehicle information; furthermore, determines the driving trajectory of the vehicle based on the turning radius. In some embodiments, the first determining unit 302 calculates the turning radius of the vehicle in two situations: calculation based on the steering wheel angle and a predetermined vehicle track at low speed; and calculation based on vehicle speed and yaw rate at high speed.
  • the first determining unit 302 determines that the vehicle speed is less than the preset vehicle speed, it determines the steering radius of the vehicle based on the steering wheel angle and the vehicle wheelbase; after determining that the vehicle speed is not less than the preset vehicle speed, it determines based on the vehicle speed and the yaw rate. The turning radius of the vehicle.
  • the steering radius of the vehicle is calculated by the following formula:
  • the steering radius of the vehicle is calculated by the following formula:
  • v x is the vehicle speed, which can also be understood as the longitudinal speed of the vehicle, in meters per second (m/s); ⁇ is the yaw rate, in rad/s; R is the turning radius of the vehicle, which can also be understood as The instantaneous turning radius of the vehicle; L is the wheelbase of the vehicle; ⁇ f is the front wheel angle, and the front wheel angle is calculated based on the steering wheel angle; v th is the preset vehicle speed, and those skilled in the art can set v th according to actual needs. The embodiment does not limit the specific value of v th.
  • the second determining unit 303 is configured to determine multiple guardrail sample points based on the multi-sensor data and the driving trajectory.
  • the second determining unit 303 filters the millimeter-wave radar data and the visual sensor data in the multi-sensor data by the driving path, which can quickly obtain the guardrail sample points with low computational consumption.
  • the second determining unit 303 determines, based on the millimeter wave radar data, a stationary millimeter wave radar target within a preset range of the driving track as the guardrail sample point. In some embodiments, taking 64 targets that can be detected by the millimeter wave radar as an example, the second determining unit 303 determines whether each target meets the following requirements: stationary and in the driving track based on the millimeter wave radar data corresponding to each millimeter wave radar target. If it is satisfied within the set range, the target is determined to be the guardrail sample point.
  • the preset range of the driving trajectory may be a belt-shaped area distributed along the driving trajectory, and those skilled in the art can set the preset range of the driving trajectory according to actual needs.
  • the second determining unit 303 determines, based on the visual sensor data, the boundary points of the drivable area that are classified as guardrails and are within the preset range of the driving track as guardrail sample points. In some embodiments, the second determining unit 303 determines whether all millimeter wave radar targets are guardrail sample points, and then determines all boundary points of the drivable area. Specifically, the second determining unit 303 determines whether each boundary point of the drivable area is satisfied: the category is a guardrail and is within the preset range of the driving track, and if it is satisfied, the boundary point of the drivable area is determined as the guardrail sample point.
  • the second determining unit 303 determines, based on the ultrasonic radar data, an ultrasonic target whose ultrasonic radar data is valid and whose lateral distance is within a preset lateral distance range as the guardrail sample point. In some embodiments, the second determining unit 303 first determines whether the receiving flag indicates that data has been received based on the receiving flag in the ultrasonic radar data, and if not, it determines that the ultrasonic radar data is invalid; secondly, the ultrasonic radar-based data is invalid. Detection distance limit, confirm that the ultrasonic radar data with the lateral distance within the preset lateral distance range is valid. Among them, those skilled in the art can set the lateral distance range based on actual requirements, and this embodiment does not limit the specific value of the lateral distance range.
  • the second determining unit 303 judges whether all the boundary points of the drivable area are guardrail sample points, and then judges all the ultrasonic radar targets. Specifically, the second determining unit 303 determines whether each ultrasonic radar target is satisfied based on the ultrasonic radar data corresponding to each ultrasonic radar target: the ultrasonic radar data is valid and the lateral distance is within the preset lateral distance range, if it is satisfied, The ultrasonic target is determined to be the guardrail sample point.
  • the method of screening guardrail sample points through the driving path cannot effectively screen out suitable and sufficient guardrail sample points.
  • the vehicle is changing lanes, and the trajectory of the vehicle is inconsistent with the shape of the road boundary.
  • the dashed-dotted line represents the fence boundary fitted in the previous cycle.
  • the millimeter wave radar target shown in Figure 7 is The boundary points of the drivable area are all near the boundary of the guardrail fitted for the previous cycle, which can be used as the guardrail sample point to ensure the effectiveness of guardrail tracking.
  • the second determining unit 303 filters the millimeter wave radar data and the visual sensor data again according to the guardrail information estimated in the previous period. It should be noted that again The screening is aimed at targets that have not been screened as effective sample points (including millimeter-wave radar targets and travelable area boundary points), and then more guardrail sample points are added.
  • the second determining unit 303 determines a plurality of guardrail sample points based on the multi-sensor data and driving trajectory, and then filters the multiple valid sample points in the multi-sensor data based on the guardrail information estimated in the previous period;
  • the unit 304 estimates the guardrail information based on the multiple guardrail sample points and the multiple effective sample points, that is, the multiple effective sample points are also regarded as the guardrail sample points, and participate in the guardrail information estimation together with the multiple guardrail sample points.
  • the second determining unit 303 screens multiple valid sample points in the multi-sensor data based on the guardrail information estimated in the previous cycle, including: determining the stationary guardrail estimated in the previous cycle based on the millimeter wave radar data The millimeter wave radar targets that are not guardrail sample points within the preset range of the boundary are valid sample points.
  • the second determining unit 303 judges whether each target meets the following criteria based on the millimeter wave radar data corresponding to each target: static and in the guardrail estimated in the previous period The sample point is within the preset range of the boundary and is not a guardrail; if it is satisfied, the target is determined to be a valid sample point.
  • the millimeter wave radar target shown in FIG. 7 is the effective sample point selected by the second determining unit 303 based on the guardrail information estimated in the previous period. It should be noted that those skilled in the art can set the preset range of the guardrail boundary according to actual needs, and this embodiment does not limit the specific value of the preset range of the guardrail boundary.
  • the second determining unit 303 screens multiple valid sample points in the multi-sensor data based on the guardrail information estimated in the previous cycle, including: determining the type of guardrail and the guardrail estimated in the previous cycle based on the visual sensor data The boundary points of the drivable area that are not the guardrail sample points within the preset range of the boundary are valid sample points.
  • the second determining unit 303 determines whether all millimeter-wave radar targets are guardrail sample points based on the guardrail information estimated in the previous cycle, and then based on the guardrail information estimated in the previous cycle, analyzes all drivable areas The boundary point is judged. Specifically, the second determining unit 303 judges whether each boundary point of the drivable area satisfies: the category is a guardrail, is within the preset range of the guardrail boundary estimated in the previous cycle and is not a guardrail sample point; if it is satisfied, the drivable area is determined The boundary points are valid sample points. For example, the boundary points of the drivable area shown in FIG. 7 are the effective sample points screened by the second determining unit 303 based on the guardrail information estimated in the previous cycle.
  • the estimation unit 304 is configured to estimate guardrail information based on multiple guardrail sample points.
  • the guardrail information may include, but is not limited to: guardrail function coefficients, guardrail fitting credibility, flag position, and longitudinal distance distribution. Among them, the flag bit is used to mark whether there is guardrail information.
  • the estimation unit 304 determines the number and longitudinal distance distribution of multiple guardrail sample points; and then determines the credibility of the guardrail fitting based on the number of guardrail sample points and the longitudinal distance distribution; Reliability, determine the coefficient of the guardrail function.
  • the estimation unit 304 determines the credibility of the guardrail fitting based on the number of sample points of the multiple guardrail and the longitudinal distance distribution, specifically: if the number of sample points of the multiple guardrail is greater than the preset threshold of the number of sample points and If the longitudinal distance distribution is greater than the preset value, it is considered that the credibility of the guardrail fitting is high, which can increase the credibility of the guardrail fitting of the previous period, and obtain the credibility of the guardrail fitting of the current period. The increase can be set according to actual needs. . It should be noted that the threshold value of the number of sample points and the preset value can be set based on actual conditions, and the specific value is not limited in this embodiment.
  • the estimation unit 304 determines the guardrail function coefficient based on the guardrail fitting credibility, specifically: judging whether the guardrail fitting credibility is greater than a preset credibility threshold; and then determining the guardrail function based on the judgment result coefficient.
  • the estimation unit 304 fits multiple guardrail sample points based on the judgment result that the credibility of the guardrail fitting is greater than the preset credibility threshold to obtain the fitting coefficient. In some embodiments, after the second determining unit 303 determines the multiple guardrail sample points, the estimation unit 304 further determines the weight of each guardrail sample point, and then based on the judgment result, the guardrail fitting credibility is greater than the preset credibility Degree threshold, based on the weight of each guardrail sample point, weighted fitting multiple guardrail sample points to obtain the fitting coefficient.
  • the estimation unit 304 further determines the weight of each guardrail sample point and the weight of each effective sample point after the second determining unit 303 screens multiple effective sample points; and then based on the judgment result, the guardrail fitting can be The reliability is greater than the preset credibility threshold. Based on the weight of each guardrail sample point and the weight of each effective sample point, multiple guardrail sample points and multiple effective sample points are weighted and fitted to obtain the fitting coefficient.
  • the estimation unit 304 fits multiple guardrail sample points in two cases to obtain fitting coefficients.
  • Case 1 After the estimation unit 304 determines that the number of the guardrail sample points is less than the preset number or the longitudinal distance distribution is less than the preset value, the multiple guardrail sample points are subjected to a first-order fitting to obtain a first-order fitting coefficient.
  • Case 2 After the estimation unit 304 determines that the number of multiple guardrail sample points is not less than the preset number and the longitudinal distance distribution is not less than the preset value, the multiple guardrail sample points are subjected to a second-order fitting to obtain a second-order fitting coefficient.
  • the guardrail sample points and the effective sample points are collectively referred to as guardrail measurement points, and the weighted least squares method is used to fit the guardrail measurement points.
  • the weighted least squares method needs to define the weight of each guardrail measurement point, and then constitute the weight matrix W.
  • the weight matrix W can be defined as the covariance matrix formed by the vector of measurement points.
  • the weight of the guardrail measurement point may be defined as the reciprocal of the variance of the lateral detection distance of the guardrail measurement point.
  • the linear expression of the first-order fitting is:
  • the guardrail measuring points are recorded as (x 1 ,y 1 ),(x 2 ,y 2 ), whil,(x n ,y n ), where n represents the number of guardrail measuring points.
  • the curve expression of the second-order fitting is:
  • the estimation unit 304 fits a plurality of guardrail sample points and obtains the fitting coefficients, and then further performs filtering processing on the fitting coefficients to obtain the guardrail function coefficients.
  • the filtering process is Kalman filtering. Filtering can smooth the fitting coefficients, prevent abnormal jumps, and increase the stability of guardrail tracking.
  • the estimating unit 304 determines the guardrail function coefficient based on the guardrail fitting credibility, specifically: the estimating unit 304 determines that the guardrail fitting credibility is not greater than the preset credibility threshold based on the judgment result, and is based on the vehicle Filtering processing is performed on the motion status information of the, and the guardrail function coefficients are obtained.
  • the filtering process is Kalman filtering. Since the credibility of the guardrail fitting is not greater than the credibility threshold, it means that no guardrail is detected. By filtering the vehicle motion state information, the guardrail function coefficients can be obtained. To ensure the accuracy of guardrail tracking for a certain period of time, until the guardrail fitting credibility is for the guardrail existence threshold, it is considered that the guardrail does not exist and the filtering process is stopped.
  • the state equation of Kalman filtering is as follows:
  • the subscript k represents the k-th calculation cycle
  • ⁇ T represents the time interval between two calculations
  • the guardrail function coefficient a corresponds to the slope
  • the slope corresponds to the tangent of the heading angle
  • the rate of change of the heading angle is horizontal when the guardrail is a straight line.
  • Swing angular velocity, denoted as w, parameter b corresponds to intercept
  • the rate of change of intercept is the longitudinal velocity along the direction of the vehicle’s vertical guardrail without considering the lateral velocity, and it is considered that in a short period of time, the vehicle speed and the horizontal The pendulum angular velocity remains unchanged.
  • each unit in the guardrail estimation module 300 is only a logical function division, and there may be other division methods in actual implementation, such as the acquisition unit 301, the first determination unit 302, the second determination unit 303, and the The estimation unit 304 may be implemented as one unit; the acquisition unit 301, the first determination unit 302, the second determination unit 303, or the estimation unit 304 may also be divided into multiple sub-units. It can be understood that each unit or subunit can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions.
  • Fig. 4 is a schematic structural diagram of a vehicle-mounted device provided by an embodiment of the present disclosure.
  • the on-board equipment can support the operation of the intelligent driving system.
  • the vehicle-mounted device includes: at least one processor 401, at least one memory 402, and at least one communication interface 403.
  • the various components in the vehicle-mounted device are coupled together through the bus system 404.
  • the communication interface 403 is used for information transmission with external devices. Understandably, the bus system 404 is used to implement connection and communication between these components.
  • the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear description, various buses are marked as the bus system 404 in FIG. 4.
  • the memory 402 in this embodiment may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory.
  • the memory 402 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: operating systems and applications.
  • the operating system includes various system programs, such as a framework layer, a core library layer, and a driver layer, which are used to implement various basic services and process hardware-based tasks.
  • Application programs including various application programs, such as Media Player, Browser, etc., are used to implement various application services.
  • the program for implementing the guardrail estimation method based on multi-sensor data fusion provided by the embodiments of the present disclosure may be included in the application program.
  • the processor 401 calls a program or instruction stored in the memory 402, specifically, it may be a program or instruction stored in an application program, and the processor 401 is configured to execute the multi-sensor-based data provided by the embodiment of the present disclosure.
  • the steps of each embodiment of the integrated guardrail estimation method are described in detail below.
  • the guardrail estimation method based on multi-sensor data fusion may be applied to the processor 401 or implemented by the processor 401.
  • the processor 401 may be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the foregoing method can be completed by an integrated logic circuit of hardware in the processor 401 or instructions in the form of software.
  • the aforementioned processor 401 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the guardrail estimation method based on multi-sensor data fusion can be directly embodied as execution and completion by a hardware decoding processor, or by a combination of hardware and software units in the decoding processor.
  • the software unit may be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402 and completes the steps of the method in combination with its hardware.
  • FIG. 5 is a flowchart of a guardrail estimation method based on multi-sensor data fusion according to an embodiment of the disclosure.
  • the execution body of the method is a vehicle-mounted device.
  • the execution body of the method is an intelligent driving system supported by the vehicle-mounted device.
  • the guardrail estimation method based on multi-sensor data fusion may include the following steps 501 to 504:
  • 501 Obtain multi-sensor data and vehicle information.
  • multiple sensors are arranged on the vehicle, at least for realizing forward detection and lateral detection of the vehicle.
  • the multi-sensor may include, but is not limited to: Millimeter-Wave Radar, visual sensor, and Ultrasonic Radar.
  • the visual sensor is, for example, a camera.
  • Millimeter-wave radar and vision sensors realize forward detection, and ultrasonic radar realizes lateral detection of vehicles.
  • the arrangement of multiple sensors is shown in Figure 6.
  • the Millimeter-Wave Radar is arranged on the front bumper of the vehicle
  • the Camera is arranged directly above the front windshield of the vehicle
  • the Ultrasonic Radar is arranged on the front of the vehicle.
  • the arrangement of multi-sensors is not limited to the arrangement shown in Fig. 6, and other ways are also possible.
  • Ultrasonic Radar is also arranged on the left front side of the vehicle. This embodiment does not limit the specific arrangement of multi-sensors. It does not limit the specific installation location of the multi-sensor, nor does it limit the specific type of the multi-sensor.
  • the millimeter-wave radar data includes at least track information or track information of a moving object or a stationary object, where the track information includes at least the distance, azimuth, detection range change rate, and radar cross section of the detected target;
  • the trace information includes at least range (Range), range rate (Range Rate), angle (Angle), radar cross section (RCS).
  • the visual sensor data is used at least to identify the drivable area, and then the boundary points of the drivable area can be determined, so as to obtain the position information and category information of the boundary points of the traveling area.
  • the position information includes a horizontal position and a vertical position.
  • the lateral direction can be understood as the transverse direction of the vehicle, and the longitudinal direction can be understood as the longitudinal direction of the vehicle.
  • the category information includes: guardrail, non-guardrail.
  • the ultrasonic radar data includes at least the distance of the closest target detected.
  • the guardrail detection within 3 meters can be realized.
  • the entire vehicle information is used at least to calculate the driving trajectory of the vehicle and guardrail tracking.
  • the entire vehicle information may include, but is not limited to: vehicle speed, yaw rate, and steering wheel angle.
  • the turning radius of the vehicle is determined based on the entire vehicle information; and then the driving trajectory of the vehicle is determined based on the turning radius.
  • the steering radius of the vehicle is calculated in two cases: calculation is based on the steering wheel angle and a predetermined vehicle track at low speed; and calculation is based on vehicle speed and yaw rate at high speed.
  • the steering radius of the vehicle is determined based on the steering wheel angle and the vehicle track; after determining that the vehicle speed is not less than the preset vehicle speed, the steering radius of the vehicle is determined based on the vehicle speed and the yaw rate.
  • the steering radius of the vehicle is calculated by the following formula:
  • v x is the vehicle speed, which can also be understood as the longitudinal speed of the vehicle, in meters per second (m/s); ⁇ is the yaw rate, in rad/s; R is the turning radius of the vehicle, which can also be understood as The instantaneous turning radius of the vehicle; L is the wheelbase of the vehicle; ⁇ f is the front wheel angle, and the front wheel angle is calculated based on the steering wheel angle; v th is the preset vehicle speed, and those skilled in the art can set v th according to actual needs. The embodiment does not limit the specific value of v th.
  • the millimeter-wave radar data and the visual sensor data in the multi-sensor data are filtered by the driving path, so that the guardrail sample points can be quickly obtained, and the calculation consumption is low.
  • the millimeter wave radar target that is stationary and within the preset range of the driving track is the guardrail sample point.
  • taking the millimeter-wave radar can detect 64 targets as an example, based on the millimeter-wave radar data corresponding to each millimeter-wave radar target, it is determined whether each target satisfies: static and within the preset range of the driving track, if If satisfied, the target is determined as the guardrail sample point.
  • the preset range of the driving trajectory may be a belt-shaped area distributed along the driving trajectory, and those skilled in the art can set the preset range of the driving trajectory according to actual needs.
  • the boundary points of the drivable area that are classified as guardrails and are within the preset range of the driving track are guardrail sample points.
  • judging whether all millimeter wave radar targets are guardrail sample points then judging all boundary points of the drivable area. Specifically, it is determined whether each boundary point of the drivable area is satisfied: the category is a guardrail and is within the preset range of the driving track, and if it is satisfied, it is determined that the boundary point of the drivable area is a guardrail sample point.
  • based on the ultrasonic radar data it is determined that an ultrasonic target whose ultrasonic radar data is valid and whose lateral distance is within a preset lateral distance range is a guardrail sample point.
  • the receiving flag in the ultrasonic radar data it is first determined whether the receiving flag indicates that data has been received, if not, the ultrasonic radar data is determined to be invalid; secondly, based on the detection range limit of the ultrasonic radar, it is determined Ultrasonic radar data whose lateral detection distance is within the preset lateral distance range is valid.
  • those skilled in the art can set the lateral distance range based on actual requirements, and this embodiment does not limit the specific value of the lateral distance range.
  • all ultrasonic radar targets are determined. Specifically, based on the ultrasonic radar data corresponding to each ultrasonic radar target, it is determined whether each ultrasonic radar target is satisfied: the ultrasonic radar data is valid and the lateral distance is within the preset lateral distance range. If it is satisfied, the ultrasonic target is determined It is the sample point of the guardrail.
  • the method of screening guardrail sample points through the driving path cannot effectively screen out suitable and sufficient guardrail sample points.
  • the vehicle is changing lanes, and the trajectory of the vehicle is inconsistent with the shape of the road boundary.
  • the dashed-dotted line represents the fence boundary fitted in the previous cycle.
  • the millimeter wave radar target shown in Figure 7 is The boundary points of the drivable area are all near the boundary of the guardrail fitted for the previous cycle, which can be used as the guardrail sample point to ensure the effectiveness of guardrail tracking.
  • the millimeter wave radar data and the visual sensor data are filtered again based on the guardrail information estimated in the previous cycle.
  • Targets that are screened as effective sample points including millimeter wave radar targets and travelable area boundary points, and then supplement more guardrail sample points.
  • the multiple valid sample points in the multi-sensor data are further filtered based on the guardrail information estimated in the previous period; and then based on the multiple guardrail sample points
  • the guardrail information is estimated, that is, multiple effective sample points are also regarded as guardrail sample points, and they participate in the estimation of guardrail information together with multiple guardrail sample points.
  • filtering multiple valid sample points in the multi-sensor data based on the guardrail information estimated in the previous period includes: determining that the stationary one is within the preset range of the guardrail boundary estimated in the previous period based on the millimeter wave radar data The millimeter wave radar target that is not the guardrail sample point is the effective sample point.
  • taking the millimeter-wave radar can detect 64 targets as an example, based on the millimeter-wave radar data corresponding to each target, it is determined whether each target satisfies: static and within the preset range of the guardrail boundary estimated in the previous period And it is not a guardrail sample point; if it is met, the target is determined to be a valid sample point.
  • the millimeter wave radar target shown in Fig. 7 is an effective sample point filtered based on the guardrail information estimated in the previous period. It should be noted that those skilled in the art can set the preset range of the guardrail boundary according to actual needs, and this embodiment does not limit the specific value of the preset range of the guardrail boundary.
  • filtering multiple valid sample points in the multi-sensor data based on the guardrail information estimated in the previous cycle includes: determining the category as guardrail based on the visual sensor data, which is within the preset range of the guardrail boundary estimated in the previous cycle
  • the boundary point of the drivable area of the non-guardrail sample point is the effective sample point.
  • the boundary points of the drivable area are judged based on the guardrail information estimated in the previous cycle. Specifically, determine whether each boundary point of the drivable area satisfies: the category is a guardrail, is within the preset range of the guardrail boundary estimated in the previous cycle and is not a guardrail sample point; if it is satisfied, the boundary point of the drivable area is determined to be a valid sample point.
  • the boundary points of the drivable area shown in FIG. 7 are effective sample points filtered based on the guardrail information estimated in the previous cycle.
  • the guardrail information may include, but is not limited to: guardrail function coefficients, guardrail fitting credibility, flag position, and longitudinal distance distribution. Among them, the flag bit is used to mark whether there is guardrail information.
  • the number and longitudinal distance distribution of the multiple guardrail sample points are determined; and then the guardrail fitting credibility is determined based on the number of guardrail sample points and the longitudinal distance distribution; thus based on the guardrail fitting credibility, Determine the coefficient of the guardrail function.
  • the credibility of the guardrail fitting is determined, specifically: if the number of guardrail sample points is greater than the preset number of sample points threshold and the longitudinal distance distribution If it is greater than the preset value, it is considered that the credibility of the guardrail fitting is high, and the credibility of the guardrail fitting of the previous cycle can be increased, and the credibility of the guardrail fitting of the current cycle can be obtained.
  • the amount of increase can be set according to actual needs. It should be noted that the threshold value of the number of sample points and the preset value can be set based on actual conditions, and the specific value is not limited in this embodiment.
  • determining the guardrail function coefficient based on the guardrail fitting credibility is specifically: judging whether the guardrail fitting credibility is greater than a preset credibility threshold; and then determining the guardrail function coefficient based on the judgment result.
  • a plurality of guardrail sample points are fitted to obtain the fitting coefficient.
  • the weight of each guardrail sample point is further determined, and then based on the judgment result, the credibility of the guardrail fitting is greater than the preset credibility threshold, based on each guardrail sample The weight of the point is weighted to fit multiple guardrail sample points to obtain the fitting coefficient.
  • the weight of each guardrail sample point and the weight of each effective sample point are further determined; and based on the judgment result, the credibility of the guardrail fitting is greater than the preset credibility Degree threshold, based on the weight of each guardrail sample point and the weight of each effective sample point, weighted fitting multiple guardrail sample points and multiple effective sample points to obtain the fitting coefficient.
  • multiple guardrail sample points are fitted in two cases to obtain fitting coefficients.
  • Case 1 After determining that the number of multiple guardrail sample points is less than the preset number or the longitudinal distance distribution is less than the preset value, the multiple guardrail sample points are subjected to a first-order fitting to obtain a first-order fitting coefficient.
  • Case 2 After determining that the number of multiple guardrail sample points is not less than the preset number and the longitudinal distance distribution is not less than the preset value, the multiple guardrail sample points are subjected to a second-order fitting to obtain a second-order fitting coefficient.
  • the guardrail sample points and the effective sample points are collectively referred to as guardrail measurement points, and the weighted least squares method is used to fit the guardrail measurement points.
  • the weighted least squares method needs to define the weight of each guardrail measurement point, and then constitute the weight matrix W.
  • the weight matrix W can be defined as the covariance matrix formed by the vector of measurement points.
  • the weight of the guardrail measurement point may be defined as the reciprocal of the variance of the lateral detection distance of the guardrail measurement point.
  • the linear expression of the first-order fitting is:
  • the guardrail measuring points are recorded as (x 1 ,y 1 ),(x 2 ,y 2 ), whil,(x n ,y n ), where n represents the number of guardrail measuring points.
  • the curve expression of the second-order fitting is:
  • the fitting coefficients are further filtered to obtain guardrail function coefficients.
  • the filtering process is Kalman filtering. Filtering can smooth the fitting coefficients, prevent abnormal jumps, and increase the stability of guardrail tracking.
  • determining the guardrail function coefficients based on the guardrail fitting credibility is specifically: based on the judgment result that the guardrail fitting credibility is not greater than a preset credibility threshold, and based on the vehicle's motion state information, Filter processing to obtain the guardrail function coefficients.
  • the filtering process is Kalman filtering. Since the credibility of the guardrail fitting is not greater than the credibility threshold, it means that no guardrail is detected. By filtering the vehicle motion state information, the guardrail function coefficients can be obtained. To ensure the accuracy of guardrail tracking for a certain period of time, until the guardrail fitting credibility is for the guardrail existence threshold, it is considered that the guardrail does not exist and the filtering process is stopped.
  • the state equation of Kalman filtering is as follows:
  • the subscript k represents the k-th calculation cycle
  • ⁇ T represents the time interval between two calculations
  • the guardrail function coefficient a corresponds to the slope
  • the slope corresponds to the tangent of the heading angle
  • the rate of change of the heading angle is horizontal when the guardrail is a straight line.
  • Swing angular velocity, denoted as w, parameter b corresponds to intercept
  • the rate of change of intercept is the longitudinal velocity along the direction of the vehicle’s vertical guardrail without considering the lateral velocity, and it is considered that in a short period of time, the vehicle speed and the horizontal The pendulum angular velocity remains unchanged.
  • the embodiments of the present disclosure also provide a non-transitory computer-readable storage medium that stores a program or instruction that enables a computer to execute a guardrail estimation method based on multi-sensor data fusion In order to avoid repetitive description, the steps of each embodiment will not be repeated here.
  • the guardrail information when the vehicle is laterally controlled, the guardrail information corrects the poor quality lane line; when the lane line cannot be detected, the guardrail information is used for the lateral control degradation processing, and the guardrail information helps to assist in the realization of a fast and accurate expressway Lane-level positioning to achieve higher levels of assisted driving functions, with industrial applicability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种基于多传感器数据融合的护栏估计方法、车载设备和存储介质，该方法包括：获取多传感器数据和整车信息（501）；基于整车信息，确定车辆的行驶轨迹（502）；基于多传感器数据和行驶轨迹，确定多个护栏样本点（503）；基于多个护栏样本点，估计护栏信息（504）。当横向控制车辆时，护栏信息对质量较差的车道线进行修正；车道线无法检测时，护栏信息用于横向控制降级处理，同时护栏信息有利于辅助实现快速准确的高速公路车道级定位，实现更高等级的辅助驾驶功能。

Description

基于多传感器数据融合的护栏估计方法和车载设备 技术领域
本公开实施例涉及智能驾驶技术领域，具体涉及一种基于多传感器数据融合的护栏估计方法、车载设备和存储介质。
背景技术
随着智能驾驶技术的发展，最先实现高级辅助驾驶和半自动驾驶的场景为高速公路，高速公路工况相对简单，车道线质量较好，较易实现。
在辅助驾驶控制领域大致分为横向控制和纵向控制，横向控制主要依据为车道线，将被控车辆保持在车道中心。当今量产的具备辅助驾驶功能的车型中，摄像头和毫米波雷达作为主要感知传感器，而车道线的感知正是依赖于摄像头，单一的依赖摄像头感知车道线作为横向控制依据会导致***性能和稳定性变差。
为此，亟需提供一种护栏估计方案，得到护栏信息，进而作为车道线的补充，从不同的数据源提高横向控制的性能。
上述对问题的发现过程的描述，仅用于辅助理解本公开的技术方案，并不代表承认上述内容是现有技术。
发明内容
为了解决现有技术存在的至少一个问题，本公开的至少一个实施例提供了一种基于多传感器数据融合的护栏估计方法、车载设备和存储介质。
第一方面，本公开实施例提出一种基于多传感器数据融合的护栏估计方法，所述方法包括：
获取多传感器数据和整车信息；
基于所述整车信息，确定车辆的行驶轨迹；
基于所述多传感器数据和所述行驶轨迹，确定多个护栏样本点；
基于所述多个护栏样本点，估计护栏信息。
第二方面，本公开实施例还提出一种车载设备，包括：处理器和存储器；所述处理器通过调用所述存储器存储的程序或指令，用于执行如第一方面所述方法的步骤。
第三方面，本公开实施例还提出一种非暂态计算机可读存储介质，用于存储程序或指令，所述程 序或指令使计算机执行如第一方面所述方法的步骤。
可见，本公开实施例的至少一个实施例中，横向控制车辆时，护栏信息对质量较差的车道线进行修正；车道线无法检测时，护栏信息用于横向控制降级处理，同时护栏信息有利于辅助实现快速准确的高速公路车道级定位，实现更高等级的辅助驾驶功能。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。
图1是本公开实施例提供的一种智能驾驶车辆的整体架构图；
图2是本公开实施例提供的一种智能驾驶***的框图；
图3是本公开实施例提供的一种护栏估计模块的框图；
图4是本公开实施例提供的一种车载设备的框图；
图5是本公开实施例提供的一种基于多传感器数据融合的护栏估计方法流程图；
图6是本公开实施例提供的一种多传感器布置示意图；
图7是本公开实施例提供的一种行驶轨迹和道路边界形状不一致的场景。
具体实施方式
为了能够更清楚地理解本公开的上述目的、特征和优点，下面结合附图和实施例对本公开作进一步的详细说明。可以理解的是，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。此处所描述的具体实施例仅仅用于解释本公开，而非对本公开的限定。基于所描述的本公开的实施例，本领域普通技术人员所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。
针对现有技术中车道线的感知依赖于摄像头，单一的依赖摄像头感知车道线作为横向控制依据会导致***性能和稳定性变差的问题，本公开实施例提供一种基于多传感器数据融合的护栏估计方案，可以在高速公路具有护栏的情况下，较稳定地输出护栏是否存在并对护栏的形状进行估计，实现横向控制车辆时，护栏信息对质量较差的车道线进行修正；车道线无法检测时，护栏信息用于横向控制降 级处理，同时护栏信息有利于辅助实现快速准确的高速公路车道级定位，实现更高等级的辅助驾驶功能。
在一些实施例中，本公开实施例提供的基于多传感器数据融合的护栏估计方案，可应用于智能驾驶车辆。图1为本公开实施例提供的一种智能驾驶车辆的整体架构图。
如图1所示，智能驾驶车辆包括：传感器组、智能驾驶***100、车辆底层执行***以及其他可用于驱动车辆和控制车辆运行的部件。
传感器组，用于采集车辆外界环境的数据和探测车辆的位置数据。传感器组例如包括但不限于摄像头、激光雷达、毫米波雷达、超声波雷达、GPS(Global Positioning System，全球定位***)和IMU(Inertial Measurement Unit，惯性测量单元)中的至少一个。
在一些实施例中，传感器组，还用于采集车辆的动力学数据，传感器组例如还包括但不限于车轮转速传感器、速度传感器、加速度传感器、方向盘转角传感器、前轮转角传感器中的至少一个。
智能驾驶***100，用于获取传感器组的数据，传感器组中所有传感器在智能驾驶车辆行驶过程中都以较高的频率传送数据。
智能驾驶***100，还用于基于传感器组的数据进行环境感知和车辆定位，并基于环境感知信息和车辆定位信息进行路径规划和决策，以及基于规划的路径生成车辆控制指令，从而控制车辆按照规划路径行驶。
在一些实施例中，智能驾驶***100，还用于基于多传感器数据融合进行护栏估计，得到护栏信息。在一些实施例中，智能驾驶***100获取多传感器数据和整车信息；进而基于整车信息，确定车辆的行驶轨迹；从而基于多传感器数据和行驶轨迹，确定多个护栏样本点；基于多个护栏样本点，估计护栏信息。
在一些实施例中，智能驾驶***100可以为软件***、硬件***或者软硬件结合的***。例如，智能驾驶***100是运行在操作***上的软件***，车载硬件***是支持操作***运行的硬件***。
在一些实施例中，智能驾驶***100，还用于与云端服务器无线通信，交互各种信息。在一些实施例中，智能驾驶***100与云端服务器通过无线通讯网络(例如包括但不限于GPRS网络、Zigbee网络、Wifi网络、3G网络、4G网络、5G网络等无线通讯网络)进行无线通信。
在一些实施例中，云端服务器用于统筹协调管理智能驾驶车辆。在一些实施例中，云端服务器可以用于与一个或多个智能驾驶车辆进行交互，统筹协调管理多个智能驾驶车辆的调度等。
在一些实施例中，云端服务器是由车辆服务商所建立的云端服务器，提供云存储和云计算的功能。 在一些实施例中，云端服务器中建立车辆端档案。在一些实施例中，车辆端档案中储存智能驾驶***100上传的各种信息。在一些实施例中，云端服务器可以实时同步车辆端产生的驾驶数据。
在一些实施例中，云端服务器可以是一个服务器，也可以是一个服务器群组。服务器群组可以是集中式的，也可以是分布式的。分布式服务器，有利于任务在多个分布式服务器进行分配与优化，克服传统集中式服务器资源紧张与响应瓶颈的缺陷。在一些实施例中，云端服务器可以是本地的或远程的。
在一些实施例中，云端服务器可用于对车辆端进行停车收费、过路收费等。在一些实施例中，云端服务器还用于分析驾驶员的驾驶行为，并且对驾驶员的驾驶行为进行安全等级评估。
在一些实施例中，云端服务器可用于获取道路监测单元(RSU：Road Side Unit)和智能驾驶车辆的信息，以及可以发送信息至智能驾驶车辆。在一些实施例中，云端服务器可以根据智能驾驶车辆的信息将道路监测单元中的与智能驾驶车辆相对应的检测信息发送给智能驾驶车辆。
在一些实施例中，道路监测单元可以用于收集道路监测信息。在一些实施例中，道路监测单元可以是环境感知传感器，例如，摄像头、激光雷达等，也可以是道路设备，例如V2X设备，路边红绿灯装置等。在一些实施例中，道路监测单元可以监控隶属于相应道路监测单元的道路情况，例如，通过车辆的类型、速度、优先级别等。道路监测单元在收集到道路监测信息后，可将所述道路监测信息发送给云端服务器，也可以发送给通过道路的智能驾驶车辆。
车辆底层执行***，用于接收车辆控制指令，实现对车辆行驶的控制。在一些实施例中，车辆底层执行***包括但不限于：转向***、制动***和驱动***。转向***、制动***和驱动***属于车辆领域成熟***，在此不再赘述。
在一些实施例中，智能驾驶车辆还可包括图1中未示出的车辆CAN总线，车辆CAN总线连接车辆底层执行***。智能驾驶***100与车辆底层执行***之间的信息交互通过车辆CAN总线进行传递。
在一些实施例中，智能驾驶车辆既可以通过驾驶员又可以通过智能驾驶***100控制车辆行驶。在人工驾驶模式下，驾驶员通过操作控制车辆行驶的装置驾驶车辆，控制车辆行驶的装置例如包括但不限于制动踏板、方向盘和油门踏板等。控制车辆行驶的装置可直接操作车辆底层执行***控制车辆行驶。
在一些实施例中，智能驾驶车辆也可以为无人车，车辆的驾驶控制由智能驾驶***100来执行。
图2为本公开实施例提供的一种智能驾驶***200的框图。在一些实施例中，智能驾驶***200 可以实现为图1中的智能驾驶***100或者智能驾驶***100的一部分，用于控制车辆行驶。
如图2所示，智能驾驶***200可划分为多个模块，例如可包括：感知模块201、规划模块202、控制模块203、护栏估计模块204以及其他一些可用于智能驾驶的模块。
感知模块201用于进行环境感知与定位。在一些实施例中，感知模块201用于获取传感器数据、V2X(Vehicle to X，车用无线通信)数据、高精度地图等数据。在一些实施例中，感知模块201用于基于获取的传感器数据、V2X(Vehicle to X，车用无线通信)数据、高精度地图等数据中的至少一种，进行环境感知与定位。
在一些实施例中，感知模块201用于生成感知定位信息，实现对障碍物感知、摄像头图像的可行驶区域识别以及车辆的定位等。
环境感知(Environmental Perception)可以理解为对于环境的场景理解能力，例如障碍物的位置，道路标志/标记的检测，行人/车辆的检测等数据的语义分类。在一些实施例中，环境感知可采用融合摄像头、激光雷达、毫米波雷达等多种传感器的数据进行环境感知。
定位(Localization)属于感知的一部分，是确定智能驾驶车辆相对于环境的位置的能力。定位可采用：GPS定位，GPS的定位精度在数十米到厘米级别，定位精度高；定位还可采用融合GPS和惯性导航***(Inertial Navigation System)的定位方法。定位还可采用SLAM(Simultaneous Localization And Mapping，同步定位与地图构建)，SLAM的目标即构建地图的同时使用该地图进行定位，SLAM通过利用已经观测到的环境特征确定当前车辆的位置以及当前观测特征的位置。
V2X是智能交通运输***的关键技术，使得车与车、车与基站、基站与基站之间能够通信，从而获得实时路况、道路信息、行人信息等一系列交通信息，提高智能驾驶安全性、减少拥堵、提高交通效率、提供车载娱乐信息等。
高精度地图是智能驾驶领域中使用的地理地图，与传统地图相比，不同之处在于：1)高精度地图包括大量的驾驶辅助信息，例如依托道路网的精确三维表征：包括交叉路口局和路标位置等；2)高精度地图还包括大量的语义信息，例如报告交通灯上不同颜色的含义，又例如指示道路的速度限制，以及左转车道开始的位置；3)高精度地图能达到厘米级的精度，确保智能驾驶车辆的安全行驶。
规划模块202用于基于感知模块201生成的感知定位信息，进行路径规划和决策。
在一些实施例中，规划模块202用于基于感知模块201生成的感知定位信息，并结合V2X数据、高精度地图等数据中的至少一种，进行路径规划和决策。
在一些实施例中，规划模块202用于规划路径，决策：行为(例如包括但不限于跟车、超车、停 车、绕行等)、车辆航向、车辆速度、车辆的期望加速度、期望的方向盘转角等，生成规划决策信息。
控制模块203用于基于规划模块202生成的规划决策信息，进行路径跟踪和轨迹跟踪。
在一些实施例中，控制模块203用于生成车辆底层执行***的控制指令，并下发控制指令，以使车辆底层执行***控制车辆按照期望路径行驶，例如通过控制方向盘、刹车以及油门对车辆进行横向和纵向控制。
在一些实施例中，控制模块203还用于基于路径跟踪算法计算前轮转角。
在一些实施例中，路径跟踪过程中的期望路径曲线与时间参数无关，跟踪控制时，可以假设智能驾驶车辆以当前速度匀速前进，以一定的代价规则使行驶路径趋近于期望路径；而轨迹跟踪时，期望路径曲线与时间和空间均相关，并要求智能驾驶车辆在规定的时间内到达某一预设好的参考路径点。
路径跟踪不同于轨迹跟踪，不受制于时间约束，只需要在一定误差范围内跟踪期望路径。
护栏估计模块204用于基于多传感器数据融合进行护栏估计，得到护栏信息。在一些实施例中，护栏估计模块204获取多传感器数据和整车信息；进而基于整车信息，确定车辆的行驶轨迹；从而基于多传感器数据和行驶轨迹，确定多个护栏样本点；基于多个护栏样本点，估计护栏信息。
在一些实施例中，护栏估计模块204的功能可集成到感知模块201、规划模块202或控制模块203中，也可配置为与智能驾驶***200相独立的模块，护栏估计模块204可以为软件模块、硬件模块或者软硬件结合的模块。例如，护栏估计模块204是运行在操作***上的软件模块，车载硬件***是支持操作***运行的硬件***。
图3为本公开实施例提供的一种护栏估计模块300的框图。在一些实施例中，护栏估计模块300可以实现为图2中的护栏估计模块204或者护栏估计模块204的一部分。
如图3所示，护栏估计模块300可包括但不限于以下单元：获取单元301、第一确定单元302、第二确定单元303和估计单元304。
获取单元301，用于获取多传感器数据和整车信息。在一些实施例中，多传感器布置在车辆上，至少用于实现车辆的前向探测和侧向探测。在一些实施例中，多传感器可包括但不限于：毫米波雷达(Millimeter-Wave Radar)、视觉传感器和超声波雷达(Ultrasonic Radar)。其中，视觉传感器例如为摄像头(Camera)。毫米波雷达和视觉传感器实现前向探测，超声波雷达实现车辆侧向探测。
在一些实施例中，多传感器的布置如图6所示，图6中，Millimeter-Wave Radar布置在车辆的前保险杠，Camera布置在车辆的前风挡玻璃的正上方，Ultrasonic Radar布置在车辆的右前侧。本领域技术人员可以理解多传感器的布置不限于图6所示的布置方式，还可以有其他方式，例如在车辆的左 前侧也布置Ultrasonic Radar，本实施例不限定多传感器的具体布置方式，也不限定多传感器的具体安装位置，也不限定多传感器的具体类型。
在一些实施例中，毫米波雷达数据至少包括运动物体或静止物体的点迹信息或航迹信息，其中，点迹信息至少包括探测目标的距离、方位、距离变化率及雷达散射截面；航迹信息至少包括距离(Range)、距离变化率(Range Rate)、角度(Angle)、雷达散射截面(RCS)。
在一些实施例中，视觉传感器数据至少用于识别可行驶区域，进而能够确定可行驶区域边界点，从而得到行驶区域边界点的位置信息和类别信息。其中，位置信息包括横向位置和纵向位置。横向可以理解为车辆的横向方向，纵向可以理解为车辆的纵向方向。类别信息包括：护栏、非护栏。
在一些实施例中，超声波雷达数据至少包括探测的最近目标的距离。在相对速度小于等于80公里每小时情况下，可以实现3米之内护栏的探测。
在一些实施例中，整车信息至少用于计算车辆的行驶轨迹和护栏追踪。在一些实施例中，整车信息可包括但不限于：车速、横摆角速度和方向盘转角。
第一确定单元302，用于基于整车信息，确定车辆的行驶轨迹。在一些实施例中，第一确定单元302基于整车信息，确定车辆的转向半径；进而基于转向半径，确定车辆的行驶轨迹。在一些实施例中，第一确定单元302分两种情况计算车辆的转向半径：低速时基于方向盘转角和预先确定的车辆轮距计算；高速时基于车速和横摆角速度计算。在一些实施例中，第一确定单元302确定车速小于预设车速后，基于方向盘转角和车辆轮距，确定车辆的转向半径；确定车速不小于预设车速后，基于车速和横摆角速度，确定车辆的转向半径。
在一些实施例中，第一确定单元302确定车速小于预设车速后，通过下式计算车辆的转向半径：
Figure PCTCN2019120407-appb-000001
v x≥v th
第一确定单元302确定车速不小于预设车速后，通过下式计算车辆的转向半径：
Figure PCTCN2019120407-appb-000002
v x＜v th
其中，v x为车速，也可以理解为车辆的纵向速度，单位为米每秒(m/s)；ω为横摆角速度，单位为rad/s；R为车辆的转向半径，也可以理解为车辆的瞬态转向半径；L为车辆轮距；δ f为前轮转角，且前轮转角基于方向盘转角计算得到；v th为预设车速，本领域技术人员可根据实际需要设置v th，本实施例不限定v th的具体取值。
第二确定单元303，用于基于多传感器数据和行驶轨迹，确定多个护栏样本点。本实施例中，第二确定单元303通过行驶路径筛选多传感器数据中的毫米波雷达数据和视觉传感器数据，可以快速得到护栏样本点，计算消耗少。
在一些实施例中，第二确定单元303基于毫米波雷达数据，确定静止的且处于行驶轨迹预设范围内的毫米波雷达目标为护栏样本点。在一些实施例中，以毫米波雷达可探测64个目标为例，第二确定单元303基于每个毫米波雷达目标对应的毫米波雷达数据，判断每个目标是否满足：静止且处于行驶轨迹预设范围内，若满足，则确定该目标为护栏样本点。在一些实施例中，行驶轨迹预设范围可以是沿行驶轨迹分布的带状区域，本领域技术人员可根据实际需要设置行驶轨迹预设范围。
在一些实施例中，第二确定单元303基于视觉传感器数据，确定类别为护栏且处于行驶轨迹预设范围内的可行驶区域边界点为护栏样本点。在一些实施例中，第二确定单元303对所有毫米波雷达目标均判断是否为护栏样本点后，再对所有可行驶区域边界点进行判断。具体地，第二确定单元303判断每个可行驶区域边界点是否满足：类别为护栏且处于行驶轨迹预设范围内，若满足，则确定该可行驶区域边界点为护栏样本点。
在一些实施例中，第二确定单元303基于超声波雷达数据，确定超声波雷达数据有效且侧向距离处于预设的侧向距离范围内的超声波目标为护栏样本点。在一些实施例中，第二确定单元303基于超声波雷达数据中的接收标志位，首先判断接收标志位是否表明一直有数据被接收，若否，则确定超声波雷达数据无效；其次，基于超声波雷达的探测距离限制，确定侧向距离处于预设的侧向距离范围内的超声波雷达数据有效。其中，本领域技术人员可基于实际需求设置侧向距离范围，本实施例不限定侧向距离范围的具体取值。
在一些实施例中，第二确定单元303对所有可行驶区域边界点均判断是否为护栏样本点后，再对所有超声波雷达目标进行判断。具体地，第二确定单元303基于每个超声波雷达目标对应的超声波雷达数据，判断每个超声波雷达目标是否满足：超声波雷达数据有效且侧向距离处于预设的侧向距离范围内，若满足，则确定该超声波目标为护栏样本点。
在一些实施例中，对于行驶轨迹和道路边界形状不一致的场景，如本车变道或者入弯出弯的场景，通过行驶路径筛选护栏样本点的方式无法有效筛选出合适且足够的护栏样本点。如图7所示，车辆正在进行变道，车辆的行驶轨迹和道路边界形状不一致，图7中，点划线表示上一周期拟合的护栏边界，图7中所示的毫米波雷达目标和可行驶区域边界点均处于为上一周期拟合的护栏边界附近，可以作为护栏样本点，保证护栏追踪的有效性。
为此，本实施例中，在通过行驶路径筛选护栏样本点的基础上，第二确定单元303通过上一周期估计的护栏信息再次筛选毫米波雷达数据和视觉传感器数据，需要说明的是，再次筛选针对的是未被筛选为有效样本点的目标(包括毫米波雷达目标和可行驶区域边界点)，进而补充更多的护栏样本点。
在一些实施例中，第二确定单元303基于多传感器数据和行驶轨迹，确定多个护栏样本点后，进一步基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点；进而估计单元304基于多个护栏样本点和多个有效样本点，估计护栏信息，也即多个有效样本点也被视为是护栏的样本点，与多个护栏样本点一起共同参与护栏信息的估计。
在一些实施例中，第二确定单元303基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点，包括：基于毫米波雷达数据，确定静止的、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的毫米波雷达目标为有效样本点。在一些实施例中，以毫米波雷达可探测64个目标为例，第二确定单元303基于每个目标对应的毫米波雷达数据，判断每个目标是否满足：静止、处于上一周期估计的护栏边界预设范围内且非护栏样本点；若满足，则确定该目标为有效样本点。例如图7中所示的毫米波雷达目标为第二确定单元303基于上一周期估计的护栏信息筛选的有效样本点。需要说明的是，本领域技术人员可根据实际需要设置护栏边界预设范围，本实施例不限定护栏边界预设范围的具体取值。
在一些实施例中，第二确定单元303基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点，包括：基于视觉传感器数据，确定类别为护栏、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的可行驶区域边界点为有效样本点。
在一些实施例中，第二确定单元303基于上一周期估计的护栏信息，对所有毫米波雷达目标均判断是否为护栏样本点后，再基于上一周期估计的护栏信息，对所有可行驶区域边界点进行判断。具体地，第二确定单元303判断每个可行驶区域边界点是否满足：类别为护栏、处于上一周期估计的护栏边界预设范围内且非护栏样本点；若满足，则确定该可行驶区域边界点为有效样本点。例如图7中所示的可行驶区域边界点为第二确定单元303基于上一周期估计的护栏信息筛选的有效样本点。
估计单元304，用于基于多个护栏样本点，估计护栏信息。在一些实施例中，护栏信息可包括但不限于：护栏函数系数、护栏拟合可信度、标志位和纵向距离分布。其中，标志位用于标志是否存在护栏信息。
在一些实施例中，估计单元304确定多个护栏样本点的数量和纵向距离分布；进而基于多个护栏样本点的数量和纵向距离分布，确定护栏拟合可信度；从而基于护栏拟合可信度，确定护栏函数系数。
在一些实施例中，估计单元304基于多个护栏样本点的数量和纵向距离分布，确定护栏拟合可信度，具体为：若多个护栏样本点的数量大于预设的样本点数量阈值且纵向距离分布大于预设值，认为护栏拟合可信度较高，可增加上一周期的护栏拟合可信度，得到本周期的护栏拟合可信度，增加量可根据实际需要进行设置。需要说明的是，样本点数量阈值和预设值可基于实际情况进行设置，本实施例不限定具体取值。
在一些实施例中，估计单元304基于护栏拟合可信度，确定护栏函数系数，具体为：判断护栏拟合可信度是否大于预设的可信度阈值；进而基于判断结果，确定护栏函数系数。
在一些实施例中，估计单元304基于判断结果为护栏拟合可信度大于预设的可信度阈值，拟合多个护栏样本点，得到拟合系数。在一些实施例中，估计单元304在第二确定单元303确定多个护栏样本点后，进一步确定每个护栏样本点的权重，进而基于判断结果为护栏拟合可信度大于预设的可信度阈值，基于每个护栏样本点的权重，加权拟合多个护栏样本点，得到拟合系数。
在一些实施例中，估计单元304在第二确定单元303筛选多个有效样本点后，进一步确定每个护栏样本点的权重和每个有效样本点的权重；进而基于判断结果为护栏拟合可信度大于预设的可信度阈值，基于每个护栏样本点的权重和每个有效样本点的权重，加权拟合多个护栏样本点和多个有效样本点，得到拟合系数。
在一些实施例中，估计单元304分两种情况拟合多个护栏样本点，得到拟合系数。情况一：估计单元304确定多个护栏样本点的数量小于预设数量或纵向距离分布小于预设值后，将多个护栏样本点进行一阶拟合，得到一阶拟合系数。情况二：估计单元304确定多个护栏样本点的数量不小于预设数量且纵向距离分布不小于预设值后，将多个护栏样本点进行二阶拟合，得到二阶拟合系数。
在一些实施例中，将护栏样本点和有效样本点统称为护栏测量点，采用加权最小二乘法拟合护栏测量点。加权最小二乘法需要定义每个护栏测量点的权重，进而构成权重矩阵W。权重矩阵W可定义为测量点向量构成的协方差矩阵。在一些实施例中，可以将护栏测量点的权重定义为护栏测量点侧向探测距离方差的倒数。
在一些实施例中，一阶拟合的直线表达式为：
y＝a+b·x
护栏测量点记为(x 1,y 1),(x 2,y 2),……,(x n,y n)，其中，n表示护栏测量点的数量。
设d i为第i个护栏测量点到拟合的直线之间的距离，则d i表达式如下：
d i＝[y i-(a+b·x i)]
设D为差方加权和，且第i组护栏测量点权重为W i，则有如下公式：
Figure PCTCN2019120407-appb-000003
根据D的偏导为0，则有如下公式：
Figure PCTCN2019120407-appb-000004
Figure PCTCN2019120407-appb-000005
对上述公式求解，即可得：
Figure PCTCN2019120407-appb-000006
求得a，将其带入公式即可得到b。
在一些实施例中，二阶拟合的曲线表达式为：
y＝a+b·x+c·x 2
在一些实施例中，估计单元304拟合多个护栏样本点，得到拟合系数后，进一步对拟合系数进行滤波处理，得到护栏函数系数。在一些实施例中，滤波处理为卡尔曼滤波。通过滤波处理可以平滑拟合系数，防止异常跳变，增加护栏追踪的稳定性。
在一些实施例中，估计单元304基于护栏拟合可信度，确定护栏函数系数，具体为：估计单元304基于判断结果为护栏拟合可信度不大于预设的可信度阈值，基于车辆的运动状态信息，进行滤波处理，得到护栏函数系数。在一些实施例中，滤波处理为卡尔曼滤波，由于护栏拟合可信度不大于可信度阈值，说明没有检测到护栏，通过对车辆的运动状态信息进行滤波处理，可得到护栏函数系数，保证一定时间护栏追踪的正确性，直至护栏拟合可信度对于护栏存在性阈值时，认为护栏不存在，停止滤波处理。
在一些实施例中，卡尔曼滤波的状态方程如下：
a k+1＝a k+w k·ΔT
b k+1＝b k+v k·a k·ΔT
w k+1＝w k
v k+1＝v k
其中，下标k代表第k个计算周期，ΔT代表两次计算的时间间隔，护栏函数系数a对应斜率，斜率对应航向角的正切，而航向角的变化率在护栏为直线的情况下即横摆角速度，记为w，参数b对应为截距，截距的变化率在不考虑侧向速度的情况下为纵向速度沿车辆垂直护栏方向的速度，并认为在很短时间内，车速和横摆角速度不变。
在一些实施例中，护栏估计模块300中各单元的划分仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如获取单元301、第一确定单元302、第二确定单元303和估计单元304可以实现为一个单元；获取单元301、第一确定单元302、第二确定单元303或估计单元304也可以划分为多个子单元。可以理解的是，各个单元或子单元能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能。
图4是本公开实施例提供的一种车载设备的结构示意图。车载设备可支持智能驾驶***的运行。
如图4所示，车载设备包括：至少一个处理器401、至少一个存储器402和至少一个通信接口403。车载设备中的各个组件通过总线***404耦合在一起。通信接口403，用于与外部设备之间的信息传输。可理解地，总线***404用于实现这些组件之间的连接通信。总线***404除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但为了清楚说明起见，在图4中将各种总线都标为总线***404。
可以理解，本实施例中的存储器402可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。
在一些实施方式中，存储器402存储了如下的元素，可执行单元或者数据结构，或者他们的子集，或者他们的扩展集：操作***和应用程序。
其中，操作***，包含各种***程序，例如框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务。应用程序，包含各种应用程序，例如媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。实现本公开实施例提供的基于多传感器数据融合的护栏估计方法的程序可以包含在应用程序中。
在本公开实施例中，处理器401通过调用存储器402存储的程序或指令，具体的，可以是应用程序中存储的程序或指令，处理器401用于执行本公开实施例提供的基于多传感器数据融合的护栏估计 方法各实施例的步骤。
本公开实施例提供的基于多传感器数据融合的护栏估计方法可以应用于处理器401中，或者由处理器401实现。处理器401可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器401中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器401可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
本公开实施例提供的基于多传感器数据融合的护栏估计方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件单元组合执行完成。软件单元可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器402，处理器401读取存储器402中的信息，结合其硬件完成方法的步骤。
图5为本公开实施例提供的一种基于多传感器数据融合的护栏估计方法流程图。该方法的执行主体为车载设备，在一些实施例中，该方法的执行主体为车载设备所支持的智能驾驶***。
如图5所示，基于多传感器数据融合的护栏估计方法可包括以下步骤501至504：
501、获取多传感器数据和整车信息。在一些实施例中，多传感器布置在车辆上，至少用于实现车辆的前向探测和侧向探测。
在一些实施例中，多传感器可包括但不限于：毫米波雷达(Millimeter-Wave Radar)、视觉传感器和超声波雷达(Ultrasonic Radar)。其中，视觉传感器例如为摄像头(Camera)。毫米波雷达和视觉传感器实现前向探测，超声波雷达实现车辆侧向探测。
在一些实施例中，多传感器的布置如图6所示，图6中，Millimeter-Wave Radar布置在车辆的前保险杠，Camera布置在车辆的前风挡玻璃的正上方，Ultrasonic Radar布置在车辆的右前侧。本领域技术人员可以理解多传感器的布置不限于图6所示的布置方式，还可以有其他方式，例如在车辆的左前侧也布置Ultrasonic Radar，本实施例不限定多传感器的具体布置方式，也不限定多传感器的具体安装位置，也不限定多传感器的具体类型。
在一些实施例中，毫米波雷达数据至少包括运动物体或静止物体的点迹信息或航迹信息，其中，点迹信息至少包括探测目标的距离、方位、探测距离变化率和雷达散射截面；航迹信息至少包括距离(Range)、距离变化率(Range Rate)、角度(Angle)、雷达散射截面(RCS)。
在一些实施例中，视觉传感器数据至少用于识别可行驶区域，进而能够确定可行驶区域边界点，从而得到行驶区域边界点的位置信息和类别信息。其中，位置信息包括横向位置和纵向位置。横向可以理解为车辆的横向方向，纵向可以理解为车辆的纵向方向。类别信息包括：护栏、非护栏。
在一些实施例中，超声波雷达数据至少包括探测的最近目标的距离。在相对速度小于等于80公里每小时情况下，可以实现3米之内护栏的探测。
在一些实施例中，整车信息至少用于计算车辆的行驶轨迹和护栏追踪。在一些实施例中，整车信息可包括但不限于：车速、横摆角速度和方向盘转角。
502、基于整车信息，确定车辆的行驶轨迹。在一些实施例中，基于整车信息，确定车辆的转向半径；进而基于转向半径，确定车辆的行驶轨迹。在一些实施例中，分两种情况计算车辆的转向半径：低速时基于方向盘转角和预先确定的车辆轮距计算；高速时基于车速和横摆角速度计算。在一些实施例中，确定车速小于预设车速后，基于方向盘转角和车辆轮距，确定车辆的转向半径；确定车速不小于预设车速后，基于车速和横摆角速度，确定车辆的转向半径。
在一些实施例中，确定车速小于预设车速后，通过下式计算车辆的转向半径：
Figure PCTCN2019120407-appb-000007
v x≥v th
确定车速不小于预设车速后，通过下式计算车辆的转向半径：
Figure PCTCN2019120407-appb-000008
v x＜v th
其中，v x为车速，也可以理解为车辆的纵向速度，单位为米每秒(m/s)；ω为横摆角速度，单位为rad/s；R为车辆的转向半径，也可以理解为车辆的瞬态转向半径；L为车辆轮距；δ f为前轮转角，且前轮转角基于方向盘转角计算得到；v th为预设车速，本领域技术人员可根据实际需要设置v th，本实施例不限定v th的具体取值。
503、基于多传感器数据和行驶轨迹，确定多个护栏样本点。本实施例中，通过行驶路径筛选多传感器数据中的毫米波雷达数据和视觉传感器数据，可以快速得到护栏样本点，计算消耗少。
在一些实施例中，基于毫米波雷达数据，确定静止的且处于行驶轨迹预设范围内的毫米波雷达目标为护栏样本点。在一些实施例中，以毫米波雷达可探测64个目标为例，基于每个毫米波雷达目标对应的毫米波雷达数据，判断每个目标是否满足：静止且处于行驶轨迹预设范围内，若满足，则确定该目标为护栏样本点。在一些实施例中，行驶轨迹预设范围可以是沿行驶轨迹分布的带状区域，本领 域技术人员可根据实际需要设置行驶轨迹预设范围。
在一些实施例中，基于视觉传感器数据，确定类别为护栏且处于行驶轨迹预设范围内的可行驶区域边界点为护栏样本点。在一些实施例中，对所有毫米波雷达目标均判断是否为护栏样本点后，再对所有可行驶区域边界点进行判断。具体地，判断每个可行驶区域边界点是否满足：类别为护栏且处于行驶轨迹预设范围内，若满足，则确定该可行驶区域边界点为护栏样本点。
在一些实施例中，基于超声波雷达数据，确定超声波雷达数据有效且侧向距离处于预设的侧向距离范围内的超声波目标为护栏样本点。在一些实施例中，基于超声波雷达数据中的接收标志位，首先判断接收标志位是否表明一直有数据被接收，若否，则确定超声波雷达数据无效；其次，基于超声波雷达的探测距离限制，确定侧向探测距离处于预设的侧向距离范围内的超声波雷达数据有效。其中，本领域技术人员可基于实际需求设置侧向距离范围，本实施例不限定侧向距离范围的具体取值。
在一些实施例中，对所有可行驶区域边界点均判断是否为护栏样本点后，再对所有超声波雷达目标进行判断。具体地，基于每个超声波雷达目标对应的超声波雷达数据，判断每个超声波雷达目标是否满足：超声波雷达数据有效且侧向距离处于预设的侧向距离范围内，若满足，则确定该超声波目标为护栏样本点。
在一些实施例中，对于行驶轨迹和道路边界形状不一致的场景，如本车变道或者入弯出弯的场景，通过行驶路径筛选护栏样本点的方式无法有效筛选出合适且足够的护栏样本点。如图7所示，车辆正在进行变道，车辆的行驶轨迹和道路边界形状不一致，图7中，点划线表示上一周期拟合的护栏边界，图7中所示的毫米波雷达目标和可行驶区域边界点均处于为上一周期拟合的护栏边界附近，可以作为护栏样本点，保证护栏追踪的有效性。
为此，本实施例中，在通过行驶路径筛选护栏样本点的基础上，通过上一周期估计的护栏信息再次筛选毫米波雷达数据和视觉传感器数据，需要说明的是，再次筛选针对的是未被筛选为有效样本点的目标(包括毫米波雷达目标和可行驶区域边界点)，进而补充更多的护栏样本点。
在一些实施例中，基于多传感器数据和行驶轨迹，确定多个护栏样本点后，进一步基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点；进而基于多个护栏样本点和多个有效样本点，估计护栏信息，也即多个有效样本点也被视为是护栏的样本点，与多个护栏样本点一起共同参与护栏信息的估计。
在一些实施例中，基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点，包括：基于毫米波雷达数据，确定静止的、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的毫 米波雷达目标为有效样本点。在一些实施例中，以毫米波雷达可探测64个目标为例，基于每个目标对应的毫米波雷达数据，判断每个目标是否满足：静止、处于上一周期估计的护栏边界预设范围内且非护栏样本点；若满足，则确定该目标为有效样本点。例如图7中所示的毫米波雷达目标为基于上一周期估计的护栏信息筛选的有效样本点。需要说明的是，本领域技术人员可根据实际需要设置护栏边界预设范围，本实施例不限定护栏边界预设范围的具体取值。
在一些实施例中，基于上一周期估计的护栏信息，筛选多传感器数据中多个有效样本点，包括：基于视觉传感器数据，确定类别为护栏、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的可行驶区域边界点为有效样本点。
在一些实施例中，基于上一周期估计的护栏信息，对所有毫米波雷达目标均判断是否为护栏样本点后，再基于上一周期估计的护栏信息，对所有可行驶区域边界点进行判断。具体地，判断每个可行驶区域边界点是否满足：类别为护栏、处于上一周期估计的护栏边界预设范围内且非护栏样本点；若满足，则确定该可行驶区域边界点为有效样本点。例如图7中所示的可行驶区域边界点为基于上一周期估计的护栏信息筛选的有效样本点。
504、基于多个护栏样本点，估计护栏信息。在一些实施例中，护栏信息可包括但不限于：护栏函数系数、护栏拟合可信度、标志位和纵向距离分布。其中，标志位用于标志是否存在护栏信息。
在一些实施例中，确定多个护栏样本点的数量和纵向距离分布；进而基于多个护栏样本点的数量和纵向距离分布，确定护栏拟合可信度；从而基于护栏拟合可信度，确定护栏函数系数。
在一些实施例中，基于多个护栏样本点的数量和纵向距离分布，确定护栏拟合可信度，具体为：若多个护栏样本点的数量大于预设的样本点数量阈值且纵向距离分布大于预设值，认为护栏拟合可信度较高，可增加上一周期的护栏拟合可信度，得到本周期的护栏拟合可信度，增加量可根据实际需要进行设置。需要说明的是，样本点数量阈值和预设值可基于实际情况进行设置，本实施例不限定具体取值。
在一些实施例中，基于护栏拟合可信度，确定护栏函数系数，具体为：判断护栏拟合可信度是否大于预设的可信度阈值；进而基于判断结果，确定护栏函数系数。
在一些实施例中，基于判断结果为护栏拟合可信度大于预设的可信度阈值，拟合多个护栏样本点，得到拟合系数。在一些实施例中，在确定多个护栏样本点后，进一步确定每个护栏样本点的权重，进而基于判断结果为护栏拟合可信度大于预设的可信度阈值，基于每个护栏样本点的权重，加权拟合多个护栏样本点，得到拟合系数。
在一些实施例中，在筛选多个有效样本点后，进一步确定每个护栏样本点的权重和每个有效样本点的权重；进而基于判断结果为护栏拟合可信度大于预设的可信度阈值，基于每个护栏样本点的权重和每个有效样本点的权重，加权拟合多个护栏样本点和多个有效样本点，得到拟合系数。
在一些实施例中，分两种情况拟合多个护栏样本点，得到拟合系数。情况一：确定多个护栏样本点的数量小于预设数量或纵向距离分布小于预设值后，将多个护栏样本点进行一阶拟合，得到一阶拟合系数。情况二：确定多个护栏样本点的数量不小于预设数量且纵向距离分布不小于预设值后，将多个护栏样本点进行二阶拟合，得到二阶拟合系数。
在一些实施例中，将护栏样本点和有效样本点统称为护栏测量点，采用加权最小二乘法拟合护栏测量点。加权最小二乘法需要定义每个护栏测量点的权重，进而构成权重矩阵W。权重矩阵W可定义为测量点向量构成的协方差矩阵。在一些实施例中，可以将护栏测量点的权重定义为护栏测量点侧向探测距离方差的倒数。
在一些实施例中，一阶拟合的直线表达式为：
y＝a+b·x
护栏测量点记为(x 1,y 1),(x 2,y 2),……,(x n,y n)，其中，n表示护栏测量点的数量。
设d i为第i个护栏测量点到拟合的直线之间的距离，则d i表达式如下：
d i＝[y i-(a+b·x i)]
设D为差方加权和，且第i组护栏测量点权重为W i，则有如下公式：
Figure PCTCN2019120407-appb-000009
根据D的偏导为0，则有如下公式：
Figure PCTCN2019120407-appb-000010
Figure PCTCN2019120407-appb-000011
对上述公式求解，即可得：
Figure PCTCN2019120407-appb-000012
求得a，将其带入公式即可得到b。
在一些实施例中，二阶拟合的曲线表达式为：
y＝a+b·x+c·x 2
在一些实施例中，拟合多个护栏样本点，得到拟合系数后，进一步对拟合系数进行滤波处理，得到护栏函数系数。在一些实施例中，滤波处理为卡尔曼滤波。通过滤波处理可以平滑拟合系数，防止异常跳变，增加护栏追踪的稳定性。
在一些实施例中，基于护栏拟合可信度，确定护栏函数系数，具体为：基于判断结果为护栏拟合可信度不大于预设的可信度阈值，基于车辆的运动状态信息，进行滤波处理，得到护栏函数系数。在一些实施例中，滤波处理为卡尔曼滤波，由于护栏拟合可信度不大于可信度阈值，说明没有检测到护栏，通过对车辆的运动状态信息进行滤波处理，可得到护栏函数系数，保证一定时间护栏追踪的正确性，直至护栏拟合可信度对于护栏存在性阈值时，认为护栏不存在，停止滤波处理。
在一些实施例中，卡尔曼滤波的状态方程如下：
a k+1＝a k+w k·ΔT
b k+1＝b k+v k·a k·ΔT
w k+1＝w k
v k+1＝v k
其中，下标k代表第k个计算周期，ΔT代表两次计算的时间间隔，护栏函数系数a对应斜率，斜率对应航向角的正切，而航向角的变化率在护栏为直线的情况下即横摆角速度，记为w，参数b对应为截距，截距的变化率在不考虑侧向速度的情况下为纵向速度沿车辆垂直护栏方向的速度，并认为在很短时间内，车速和横摆角速度不变。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员能够理解，本公开实施例并不受所描述的动作顺序的限制，因为依据本公开实施例，某些步骤可以采用其他顺序或者同时进行。另外，本领域技术人员能够理解，说明书中所描述的实施例均属于可选实施例。
本公开实施例还提出一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质存储程序或指令，所述程序或指令使计算机执行如基于多传感器数据融合的护栏估计方法各实施例的步骤，为避免重复描述，在此不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含， 从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本公开的范围之内并且形成不同的实施例。
本领域的技术人员能够理解，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
虽然结合附图描述了本公开的实施方式，但是本领域技术人员可以在不脱离本公开的精神和范围的情况下做出各种修改和变型，这样的修改和变型均落入由所附权利要求所限定的范围之内。
工业实用性
本公开实施例中，横向控制车辆时，护栏信息对质量较差的车道线进行修正；车道线无法检测时，护栏信息用于横向控制降级处理，同时护栏信息有利于辅助实现快速准确的高速公路车道级定位，实现更高等级的辅助驾驶功能，具有工业实用性。

Claims (16)

  1. 一种基于多传感器数据融合的护栏估计方法，其特征在于，所述方法包括：
    获取多传感器数据和整车信息；
    基于所述整车信息，确定车辆的行驶轨迹；
    基于所述多传感器数据和所述行驶轨迹，确定多个护栏样本点；
    基于所述多个护栏样本点，估计护栏信息。
  2. 根据权利要求1所述的方法，其特征在于，所述多传感器数据包括：毫米波雷达数据、视觉传感器数据和超声波雷达数据；
    所述整车信息包括：车速、横摆角速度和方向盘转角；
    所述护栏信息包括：护栏函数系数、护栏拟合可信度、标志位和纵向距离分布。
  3. 根据权利要求1所述的方法，其特征在于，基于所述整车信息，确定车辆的行驶轨迹，包括：
    基于所述整车信息，确定车辆的转向半径；
    基于所述转向半径，确定车辆的行驶轨迹。
  4. 根据权利要求3所述的方法，其特征在于，基于所述整车信息，确定车辆的转向半径，包括：
    确定车速小于预设车速后，基于方向盘转角和车辆轮距，确定车辆的转向半径；确定车速不小于预设车速后，基于车速和横摆角速度，确定车辆的转向半径。
  5. 根据权利要求2所述的方法，其特征在于，基于所述多传感器数据和所述行驶轨迹，确定多个护栏样本点，包括：
    基于毫米波雷达数据，确定静止的且处于所述行驶轨迹预设范围内的毫米波雷达目标为护栏样本点；
    基于视觉传感器数据，确定类别为护栏且处于所述行驶轨迹预设范围内的可行驶区域边界点为护栏样本点；
    基于超声波雷达数据，确定超声波雷达数据有效且侧向距离处于预设的侧向距离范围内的超声波目标为护栏样本点。
  6. 根据权利要求5所述的方法，其特征在于，基于所述多个护栏样本点，估计护栏信息，包括：
    确定所述多个护栏样本点的数量和纵向距离分布；
    基于所述数量和纵向距离分布，确定护栏拟合可信度；
    基于所述护栏拟合可信度，确定护栏函数系数。
  7. 根据权利要求6所述的方法，其特征在于，基于所述护栏拟合可信度，确定护栏函数系数，包括：
    判断所述护栏拟合可信度是否大于预设的可信度阈值；
    基于所述判断结果，确定护栏函数系数。
  8. 根据权利要求7所述的方法，其特征在于，基于所述判断结果，确定护栏函数系数，包括：
    基于所述判断结果为所述护栏拟合可信度大于预设的可信度阈值，拟合所述多个护栏样本点，得到拟合系数，并对所述拟合系数进行滤波处理，得到护栏函数系数。
  9. 根据权利要求7所述的方法，其特征在于，基于所述判断结果，确定护栏函数系数，包括：
    基于判断结果为护栏拟合可信度不大于预设的可信度阈值，基于车辆的运动状态信息，进行滤波处理，得到护栏函数系数。
  10. 根据权利要求8所述的方法，其特征在于，拟合所述多个护栏样本点，得到拟合系数，包括：
    确定所述数量小于预设数量或所述纵向距离分布小于预设值后，将所述多个护栏样本点进行一阶拟合，得到一阶拟合系数；
    确定所述数量不小于预设数量且所述纵向距离分布不小于预设值后，将所述多个护栏样本点进行二阶拟合，得到二阶拟合系数。
  11. 根据权利要求8所述的方法，其特征在于，所述确定多个护栏样本点后，所述方法还包括：
    确定每个护栏样本点的权重；
    基于所述每个护栏样本点的权重，加权拟合所述多个护栏样本点，得到拟合系数。
  12. 根据权利要求5所述的方法，其特征在于，基于所述多传感器数据和所述行驶轨迹，确定多个护栏样本点后，所述方法还包括：
    基于上一周期估计的护栏信息，筛选所述多传感器数据中多个有效样本点；
    相应地，基于所述多个护栏样本点和所述多个有效样本点，估计护栏信息。
  13. 根据权利要求12所述的方法，其特征在于，基于上一周期估计的护栏信息，筛选所述多传感器数据中多个有效样本点，包括：
    基于毫米波雷达数据，确定静止的、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的毫米波雷达目标为有效样本点；
    基于视觉传感器数据，确定类别为护栏、处于上一周期估计的护栏边界预设范围内的、非护栏样本点的可行驶区域边界点为有效样本点。
  14. 根据权利要求12所述的方法，其特征在于，所述方法还包括：
    确定每个护栏样本点的权重和每个有效样本点的权重；
    基于每个护栏样本点的权重和每个有效样本点的权重，加权拟合所述多个护栏样本点和所述多个有效样本点，得到拟合系数。
  15. 一种车载设备，其特征在于，包括：处理器和存储器；
    所述处理器通过调用所述存储器存储的程序或指令，用于执行如权利要求1至14任一项所述方法的步骤。
  16. 一种非暂态计算机可读存储介质，其特征在于，所述非暂态计算机可读存储介质存储程序或指令，所述程序或指令使计算机执行如权利要求1至14任一项所述方法的步骤。
PCT/CN2019/120407 2019-11-22 2019-11-22 基于多传感器数据融合的护栏估计方法和车载设备 WO2021097844A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/916,038 US20230168368A1 (en) 2019-11-22 2019-11-22 Guardrail estimation method based on multi-sensor data fusion, and vehicle-mounted device
CN201980002607.4A CN113227831B (zh) 2019-11-22 2019-11-22 基于多传感器数据融合的护栏估计方法和车载设备
PCT/CN2019/120407 WO2021097844A1 (zh) 2019-11-22 2019-11-22 基于多传感器数据融合的护栏估计方法和车载设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/120407 WO2021097844A1 (zh) 2019-11-22 2019-11-22 基于多传感器数据融合的护栏估计方法和车载设备

Publications (1)

Publication Number Publication Date
WO2021097844A1 true WO2021097844A1 (zh) 2021-05-27

Family

ID=75981162

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/120407 WO2021097844A1 (zh) 2019-11-22 2019-11-22 基于多传感器数据融合的护栏估计方法和车载设备

Country Status (3)

Country Link
US (1) US20230168368A1 (zh)
CN (1) CN113227831B (zh)
WO (1) WO2021097844A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114353817A (zh) * 2021-12-28 2022-04-15 重庆长安汽车股份有限公司 多源传感器车道线确定方法、***、车辆及计算机可读存储介质
CN115184891A (zh) * 2022-06-14 2022-10-14 北京理工大学重庆创新中心 一种毫米波雷达道路栅栏检测识别方法

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103786729A (zh) * 2012-10-26 2014-05-14 现代自动车株式会社 车道识别方法和***
US20150293216A1 (en) * 2014-04-15 2015-10-15 GM Global Technology Operations LLC Method and system for detecting, tracking and estimating stationary roadside objects
CN107250840A (zh) * 2015-02-04 2017-10-13 奥迪股份公司 用于求得机动车在路面上的横向位置信息的方法和机动车
CN107667047A (zh) * 2015-06-03 2018-02-06 法雷奥开关和传感器有限责任公司 用于将机动车辆侧向周围环境区域中细长静止物体分类的方法、驾驶员辅助***和机动车辆
CN108297795A (zh) * 2018-01-31 2018-07-20 成都华远科技有限公司 一种电动车的车道识别***
CN108344997A (zh) * 2018-01-16 2018-07-31 华域汽车***股份有限公司 一种基于点迹特征的道路护栏快速检测方法
CN108974007A (zh) * 2017-06-02 2018-12-11 德尔福技术有限公司 确定主动巡航控制的兴趣物体
CN109249928A (zh) * 2017-07-13 2019-01-22 现代自动车株式会社 车辆和用于控制车辆的方法
CN109254289A (zh) * 2018-11-01 2019-01-22 百度在线网络技术（北京）有限公司 道路护栏的检测方法和检测设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103786729A (zh) * 2012-10-26 2014-05-14 现代自动车株式会社 车道识别方法和***
US20150293216A1 (en) * 2014-04-15 2015-10-15 GM Global Technology Operations LLC Method and system for detecting, tracking and estimating stationary roadside objects
CN107250840A (zh) * 2015-02-04 2017-10-13 奥迪股份公司 用于求得机动车在路面上的横向位置信息的方法和机动车
CN107667047A (zh) * 2015-06-03 2018-02-06 法雷奥开关和传感器有限责任公司 用于将机动车辆侧向周围环境区域中细长静止物体分类的方法、驾驶员辅助***和机动车辆
CN108974007A (zh) * 2017-06-02 2018-12-11 德尔福技术有限公司 确定主动巡航控制的兴趣物体
CN109249928A (zh) * 2017-07-13 2019-01-22 现代自动车株式会社 车辆和用于控制车辆的方法
CN108344997A (zh) * 2018-01-16 2018-07-31 华域汽车***股份有限公司 一种基于点迹特征的道路护栏快速检测方法
CN108297795A (zh) * 2018-01-31 2018-07-20 成都华远科技有限公司 一种电动车的车道识别***
CN109254289A (zh) * 2018-11-01 2019-01-22 百度在线网络技术（北京）有限公司 道路护栏的检测方法和检测设备

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114353817A (zh) * 2021-12-28 2022-04-15 重庆长安汽车股份有限公司 多源传感器车道线确定方法、***、车辆及计算机可读存储介质
CN114353817B (zh) * 2021-12-28 2023-08-15 重庆长安汽车股份有限公司 多源传感器车道线确定方法、***、车辆及计算机可读存储介质
CN115184891A (zh) * 2022-06-14 2022-10-14 北京理工大学重庆创新中心 一种毫米波雷达道路栅栏检测识别方法
CN115184891B (zh) * 2022-06-14 2024-04-19 北京理工大学重庆创新中心 一种毫米波雷达道路栅栏检测识别方法

Also Published As

Publication number Publication date
CN113227831A (zh) 2021-08-06
CN113227831B (zh) 2023-01-20
US20230168368A1 (en) 2023-06-01

Similar Documents

Publication Publication Date Title
US11685360B2 (en) Planning for unknown objects by an autonomous vehicle
CN110603181B (zh) 一种智能驾驶车辆让行方法、装置及车载设备
US10281920B2 (en) Planning for unknown objects by an autonomous vehicle
WO2021102957A1 (zh) 一种车道保持方法、车载设备和存储介质
CN110861650B (zh) 车辆的路径规划方法、装置，车载设备和存储介质
US10234864B2 (en) Planning for unknown objects by an autonomous vehicle
US20210309231A1 (en) Driving Environment Information Generation Method, Driving Control Method, Driving Environment Information Generation Device
CN113196011A (zh) 运动图构建和车道级路线规划
CN112829753B (zh) 基于毫米波雷达的护栏估计方法、车载设备和存储介质
CN110807412B (zh) 一种车辆激光定位的方法、车载设备和存储介质
CN112567439B (zh) 一种交通流信息的确定方法、装置、电子设备和存储介质
US11747166B2 (en) Driving environment information generation method, driving control method, driving environment information generation device
US20210389133A1 (en) Systems and methods for deriving path-prior data using collected trajectories
CN110568847B (zh) 一种车辆的智能控制***、方法，车载设备和存储介质
CN113085852A (zh) 自动驾驶车辆的行为预警方法、装置及云端设备
CN114512007A (zh) 交叉路口通行协调方法和装置
WO2021097844A1 (zh) 基于多传感器数据融合的护栏估计方法和车载设备
CN110375786B (zh) 一种传感器外参的标定方法、车载设备及存储介质
CN110599790B (zh) 一种智能驾驶车辆进站停靠的方法、车载设备和存储介质
CN116935693A (zh) 碰撞预警方法、车载终端及存储介质
CN114655243A (zh) 基于地图的停止点控制

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19953521

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19953521

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19953521

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12/12/2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19953521

Country of ref document: EP

Kind code of ref document: A1