Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
  • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
  • G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
  • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
  • G01S17/88—Lidar systems specially adapted for specific applications
  • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
  • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles

Definitions

• ADAS Advanced Driver Assistance Systems
• ADASs are electronic systems that aid a vehicle driver to increase car safety and road safety generally.
• ADASs are developed to automate, adapt, and enhance vehicle systems for safety and better driving.
• ACC Adaptive Cruise Control
• ACC is a form of ADAS which automatically adjusts vehicle speed to maintain a safe distance from other vehicles.
• ACC is an example of an active safety system in that it must be able to actively sense the distance to the vehicle ahead and then control the vehicle to set the sensed distance to the safe distance.
• FIG. 1 is a block diagram and flow chart 100 for the execution of an ACC system on a moving vehicle.
• the grey boxes represent the operation of various subsystems of the vehicle while the white boxes represent functional steps in the flow chart executed by those subsystems.
• This format is also applied in the remaining block diagrams and flow charts in this disclosure.
• a sensing system 101 scans the road ahead of the vehicle. This is illustrated by scan area 151 emanating from vehicle 150. The sensing system 101 will pass the information obtained from scanning to an active safety system 110.
• the active safety system can then detect a vehicle on the road, determine the detected vehicle's speed, determine the local speed (i.e., the speed of the vehicle on which the active safety system is located), calculate a braking time based on the local speed, and calculate a safe following distance based on the acquired information. This sensing of the detected vehicle is illustrated by scan area 151 encountering vehicle 152. [0004] Upon calculating the following distance, the active safety system 110 can then either provide relevant information based thereon to a user via a user interface system 120, or issue commands to the vehicle control system 130.
• the user interface system 120 can include visible or auditory cues to the user if they have gone too far ahead of or too far behind the calculated following distance and prompt them to increase or decrease the cruising speed.
• control system 130 can control the throttle, brake, or gear in a negative feedback loop with the sensing system 101 and active safety system 110 to guide the vehicle to the safe following distance automatically.
• the action of the active safety system 110 is illustrated by distance 153.
• Variant active safety systems provide similar benefits in terms of increased safety or increases in the amount of information provided to a human operator to improve their ability to safely pilot their vehicle.
• Adaptive active safety systems are disclosed herein that utilize a sensing system to detect road conditions and actuate an active safety routine based on the detected road condition.
• the result is a road-aware active safety system that can provide feedback, directly to the vehicle or to the operator, that is optimized for current road conditions.
• the sensing system used to detect the road conditions can be the same sensing system which would be used by an equivalent active safety system without road condition awareness.
• the sensing system can also be a dedicated system used for road condition detection.
• the example of an ADAS is provided throughout this disclosure as an example of the types of systems that can be improved by the disclosed approaches. However, the approaches disclosed herein are equally applicable to any active safety system including those utilized by autonomous vehicles.
• LIDAR multi-spectral light imaging detection and ranging
• the current approaches can instead utilize any sensing system including any electromagnetic (e.g., radio frequency, visible light, optical), acoustic, gravimetric, barometric, or other sensing system.
• electromagnetic e.g., radio frequency, visible light, optical
• acoustic e.g., gravimetric, barometric, or other sensing system.
• Suboptimal outcomes can take many forms based on the aspect of the active safety system which is affected by the road conditions. For example, a system may either not take into account the longer brake times required by a given condition and lead to an unsafe scenario, or the system will overcorrect for a worst case conditions and lead to systems that cause traffic build up, a suboptimal usage of road space, or worse— a situation in which an operator feels the system is inefficient and slow, and deactivates the system thereby completely mitigating any improvements to vehicle and general road safety that would otherwise have been provided by the system.
• an electromagnetic sensing system located on a vehicle is utilized to detect a road condition in an anticipated region of travel for the vehicle and provide this information to an active safety system located on the vehicle.
• the electromagnetic sensor can be or include an optical, magnetic, radio frequency, or any other sensor used to detect the interaction of electrons or photons with the road in the anticipated region of travel.
• the electromagnetic sensor can be an imaging system using visible, ultraviolet, or infrared light sensing, whether active or passive, and any form of depth sensor or two-dimensional texture map sensor.
• the sensing systems can be augmented with computer vision processing capabilities to extract depth information from a two-dimensional image, segment two- or three-dimensional images, extract information regarding portions of the environment contained in the image, and semantically derive information regarding objects in the environment contained in the image.
• the sensing system can be passive and detect the interaction of the electrons or photons from an ambient or alternative source, or it can be active, and both produce and sense the electrons or photons that will be sensed by the sensor.
• the sensing system can be used both directly by the active safety system and to determine road surface condition information for usage by the active safety system, or it can be a dedicated system specifically for determining road surface conditions.
• the active safety system can actuate an active safety routine based on the detected condition of the road. Detection of the road condition can affect when an active safety routine is actuated or how the active safety routine is actuated, or both.
• the sensing system can detect that the road is wet or dry, that it includes snow, ice, oil, gravel, etc. and determine, in real time, a grip of the road and an optimum following distance based thereon.
• the resulting system is an example of a road aware ADAS that features real time calculation of the safe braking distance calculation for a given road condition in the anticipated region of travel of the vehicle, and accordingly provides continuous optimization of the following distance.
• a first object of the invention is a method is performed by a system embedded in a vehicle, comprising the steps of: transmitting, with a multispectral lidar system, a multispectral light beam directed at an anticipated region of travel of the vehicle within a road; analyzing a multispectral response, of a photodetector, to a return of the multispectral light beam; determining, based on the analyzing of the multispectral response, a hazardous surface condition in the anticipated region of travel of the vehicle; and actuating, based on the determination of the hazardous surface condition, an active safety routine; and wherein the analyzing of the multispectral response includes a time of flight analysis and a multispectral intensity analysis, the time of flight analysis determining a distance between the multispectral lidar system and at least one point inside the anticipated region of travel, and the multispectral intensity analysis determining signature information of material of said at least one point, said signature information comprising at least two intensities at different wavelength.
• the anticipated region of travel is a region of road excluding boundaries of the road by detecting boundaries features, boundaries features including lane markers, side rails, curbs, road signs, the boundaries features being detected by analyzing the multispectral response; and the hazardous surface condition is determined based on the analyzing of the multispectral response corresponding to points on the road located only inside the anticipated region of travel.
• the actuating is further based on the determination of the inclination of the anticipated region of travel relative to an absolute horizontal direction, to identify if the anticipated region of travel is a rising portion of road or if the anticipated region of travel is a descendant portion of road.
• the multispectral light beam transmitted by the multispectral system scans both a potential encountering object to be detected in front of the vehicle and the road for determining the hazardous surface condition of the anticipated region of travel on the road.
• the multispectral light beam transmitted by the multispectral system is temporally successively and repeatedly, firstly directed towards a first set of points located to a potential encountering object in front of the vehicle for detecting said potential encountering object and secondly directed at a second set of points located on the road for determining the hazardous surface condition of the road, the direction of the multispectral light beam being controlled by at least one scanning mirror of the multispectral lidar system.
• the points in the first set of points are obtained by a first mean direction of the multispectral light beam that is inclined vertically between +/- 2 degrees relative to a vehicle horizontal direction
• the points in the second set of points are obtained by a second mean direction of the multispectral light beam that is inclined vertically towards the road between 5 to 20 degrees relative to said vehicle horizontal direction.
• the points in the second set of points are scanned more frequently than the points of the first set of points for detecting a potential encountering object more frequently than sounding road to determine the hazardous surface condition in the anticipated region of travel of the vehicle.
• the points of the second set of points are scanned at least ten times more frequently than the points of the first set of points.
• the active safety routine is conducted by one of: (i) an adaptive cruise control (ACC) system; (ii) a collision avoidance system; (iii) an automatic emergency braking (AEB) system; (iv) a forward collision warning system; (v) a hill descent control system; (vi) an intelligent speed adaptation system; and (vii) an intelligent speed advice (ISA) system.
• ACC adaptive cruise control
• AEB automatic emergency braking
• ISA intelligent speed advice
• the method further comprises: the determining of a hazardous surface condition includes determining a degree of the hazardous surface condition. [0020] According to some aspects, the method further comprises: determining, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for the active safety routine.
• the method further comprises: determining, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for when the active safety routine is actuated.
• a second object of the invention is a system, embedded in a vehicle, for mitigating a hazardous surface condition of a road, the system comprising: a multispectral lidar system, configured to transmit a multispectral light beam directed at an anticipated region of travel of the vehicle within the road; an active safety system; one or more non-transitory computer readable media storing instructions to: (i) analyze a multispectral response, of a photodetector, to a return of the multispectral light beam;
• the analyzing of the response includes a time of flight analysis and a multispectral intensity analysis, the time of flight analysis determining a distance between the multispectral lidar system and at least one point inside the anticipated region of travel, and the multispectral intensity analysis determining signature information of material of said at least one point, said signature information comprising at least two intensities at different wavelength.
• the anticipated region of travel is a region of road excluding boundaries of the road by detecting boundaries features, boundaries features including lane markers, side rails, curbs, road signs, the boundaries features being detected by analyzing the multispectral response; and the hazardous surface condition is determined based on the analyzing of the multispectral response corresponding to points on the road located only inside the anticipated region of travel
• the multispectral light beam transmitted by the multispectral system scans both a potential encountering object to be detected in front of the vehicle and the road for determining the hazardous surface condition of the anticipated region of travel on the road.
• the active safety routine is conducted by one of: (i) an adaptive cruise control (ACC) system; (ii) a collision avoidance system; (iii) an automatic emergency braking (AEB) system; (iv) a forward collision warning system; (v) a hill descent control system; (vi) an intelligent speed adaptation system; and (vii) an intelligent speed advice (ISA) system.
• ACC adaptive cruise control
• AEB automatic emergency braking
• ISA intelligent speed advice
• Figure 2 includes a block diagram and flow chart 200 for a road aware ADAS.
• the ADAS is an ACC system and the same sensing system is used directly by the ADAS 210 and by a surface condition detection system 220 to determine road surface conditions.
• the information obtained from sensing system 201 is provided not only to ADAS 210 but also to a surface condition detection system 220.
• the sensing system 201 can obtain both signature information for the surface condition detection system 220 and distance information for ADAS 210.
• the sensing system could be a multispectral LIDAR system with beams that can be used to make both a time of flight analysis for determining distance (e.g., measuring the time of reflection from a potential obstruction or road boundary), and an intensity measurement for determining signature information (e.g., two intensities at different wavelengths providing signature information of a material).
• a single laser source can be used both for time of flight analyses and for obtaining signature information by splitting the beam into multiple wavelengths or sending out pulses with different wavelengths at different times.
• the determined surface condition can be used by the active safety system to affect, in real time, the manner in which the active safety system actuates an active safety routine.
• the information regarding the surface condition is used to calculate the braking time of the vehicle, which will then influence the calculation of the optimum following distance.
• FIG. 2 The result of these differing actuations of the active safety routine are illustrated in Figure 2 in which vehicle 221 in state 240 follows a vehicle 222 at a first distance when the surface condition is detected as being a flat dry road, and the same vehicle 221 follows vehicle 222, with all else equal, at a larger distance in state 250 after the surface condition detection system 220 on vehicle 221 has determined that the road is icy and braking will be more difficult.
• the output to user interface system 120 or vehicle control system 130 may have the same format in either case, but it will be optimized based on the detected road condition.
• the detection of the surface condition can be conceptualized as a second feedback loop which constrains the stabilization point of the first feedback loop discussed above.
• the surface condition detection system is focused upon an anticipated region of travel of the vehicle.
• the anticipated region of travel of the vehicle can be determined in various ways.
• the additional processing steps associated with this determination can be guided by the same sensing system used by the active safety system, the same sensing system used to generate information for determining the surface condition, or by an alternative dedicated system.
• the determination of the anticipated region of travel could depend on information gathered by sensing system 201.
• the determination can include analyzing the road ahead to segment the road from the surroundings.
• the determination can include analyzing road boundaries such as side rails or lane dividers.
• the determination can also include analyzing the road in light of information regarding the motion of the vehicle itself, such as those produced by vehicle control system 130 in Figure 2, to determine which portion of the road the vehicle will utilize. Focusing on the anticipated region of travel can help to further optimize the actuation of the active safety procedures and can, in specific cases, prevent false alarms provided by surface conditions located on the side of the road or in adjacent lanes that are unlikely to affect the nature of travel for a vehicle on the road.
• a method is provided. The method is performed by a system embedded in a vehicle. The method includes transmitting, with a multispectral LIDAR system, a multispectral light beam directed at an anticipated region of travel of the vehicle within a road.
• the method also includes analyzing a response, of a photodetector, to a return of the multispectral light beam.
• the method also includes determining, based on the analyzing of the response, a hazardous surface condition in the anticipated region of travel of the vehicle.
• the method also includes actuating, based on the determination of the hazardous surface condition, an active safety routine.
• a system is provided.
• the system is embedded in a vehicle and is for mitigating a hazardous surface condition of a road.
• the system includes a multispectral lidar system configured to transmit a multispectral light beam directed at an anticipated region of travel of the vehicle within the road.
• the system also includes an active safety system.
• the system also includes one or more non-transitory computer readable media storing instructions to: analyze a response, of a photodetector, to a return of the multispectral light beam; determine, based on the analyzing of the response, a hazardous surface condition in the anticipated region of travel of the vehicle; and (iii) actuate, using the active safety system and based on the determination of the hazardous surface condition, an active safety routine.
• another method is provided.
• the method is performed by a system embedded in a vehicle.
• the method includes transmitting an electromagnetic signal at an anticipated region of travel of the vehicle within a road.
• the method also includes analyzing, using a sensor, a return of the electromagnetic signal.
• the method also includes determining, based on the analyzing of the return, a hazardous surface condition in the anticipated region of travel of the vehicle.
• the method also includes modifying an actuation of an active safety routine based on the determination of the hazardous surface condition.
• Figure 1 illustrates a block diagram and flow chart showing the operation of an active safety system in accordance with the related art.
• Figure 2 illustrates a block diagram and flow chart showing the operation of an adaptive active safety system in accordance with specific embodiments of the invention disclosed herein.
• Figure 2A illustrates some features of the environment of use of the adaptive active safety system of figure 2.
• Figure 3 illustrates a flow chart for a set of methods for actuating an active safety routine that are in accordance with specific embodiments of the invention disclosed herein.
• Figure 4 illustrates a set of block diagrams for various sensing systems that can be utilized to generate information for a surface condition detection system in accordance with specific embodiments of the invention disclosed herein. DETAILED DESCRIPTION
• the sensing system utilized by the active safety system to determine information about the environment generally is the same sensing system used to detect road conditions.
• the road conditions can also be determined by a separate dedicated system.
• the road condition is determined for an anticipated region of travel of a vehicle. The anticipated region of travel can further be determined by the same sensing system, or a separate dedicated system.
• the flow chart is an example of implementation of an adaptive active safety system 200 according to specific embodiments of the invention.
• the example of vehicles in states 240 and 250 is an example of an Adaptive Cruise Control System (ACC) that adapts or tunes an optimum following distance on the bases of the road conditions.
• Figure 2A is an enlarged view of state 240 of figure 2.
• the adaptive active safety system 200 includes a sensing system 201 providing distance information 260 between the vehicle 221 and a potential encountering object in front of said vehicle 211, and a surface condition determination system 220 providing a road surface condition of an anticipated region of travel 270 of vehicle 211.
• the distance information 260 and the surface condition are both provided to the Advanced Driver Assistance System 210 (ADAS).
• ADAS Advanced Driver Assistance System
• the ADAS detects a potential encountering object that is a vehicle 222 in front of the vehicle 211.
• the ADAS system detects a potential encountering object (e.g. detects vehicle 222), determines the potential encountering object speed (e.g. determines vehicle speed of vehicle 222), determines the local speed (i.e. the vehicle speed of the vehicle 221 on which the ADAS is located). Then, the ADAS determines a braking time and/or a brake distance based on at least the local speed of vehicle 221 and the surface condition provided by the surface condition detection system 220. Then, the ADAS 210 calculates a safe following distance based on the acquired information, such as the encountering object speed, local speed, braking time and/or brake distance, and provides said following distance to the user interface system 120 and/or the vehicle control system 130.
• a potential encountering object e.g. detects vehicle 222
• determines the local speed i.e. the vehicle speed of the vehicle 221 on which the ADAS is located.
• the ADAS determines a braking time and/or a brake distance based on at
• the vehicle control system 130 can control the throttle of engine, the braking force of each and/or all brakes, gear of vehicle 211.
• the sensing system 201 is for example the same sensing system (for example a dual measure scanning system) that provides information to the ADAS system for detecting the potential encountering object (the vehicle 222) and to the surface condition determining system 220 for determining the road surface condition.
• the sensing system 201 is for example the same sensing system (for example a dual measure scanning system) that provides information to the ADAS system for detecting the potential encountering object (the vehicle 222) and to the surface condition determining system 220 for determining the road surface condition.
• the sensing system 201 is a multispectral lidar system transmitting a multispectral light beam directed to various directions for scanning both the potential encountering object to be detected in front of the vehicle (350) and the road for determining the surface condition of the anticipated region of travel on the road.
• the sensing system 201 of vehicle 221 is scanning a scanned region 270 of the road for example in front of the vehicle 221.
• the sensing system 201 can extract a portion of said scanned region 270 corresponding to the anticipated region of travel 271 of vehicle 221.
• the anticipated region of travel 271 is a region of the road where the vehicle 221 will travel in near future and for which evaluation of road surface condition is desired to anticipate the vehicle future displacement.
• the sensing system 201 is also scanning a potential encountering object (e.g. vehicle 222).
• the potential encountering object is for example ahead (in front of) the anticipated region of travel 271.
• the sensing system 201 is then can provide distance information 260 to the ADAS system 210.
• the distance information 260 is a distance that should be greater than a maximum distance of scanning of the anticipated region of travel 270 as the surface condition should be determined between the vehicle 211 and the potential encountering object (vehicle 222).
• the sensing system 201 is a multispectral lidar system transmitting a multispectral light beam.
• the multispectral light beam is temporally successively and repeatedly, firstly directed towards a first set of points PI located to the potential encountering object (vehicle 222) in front of the vehicle 221 for detecting said potential encountering object and secondly directed at a second set of points P2 located on the road for determining the surface condition of the road, the direction of the multispectral light beam being controlled for example by at least one scanning mirror of the multispectral lidar system.
• the points in the first set of points PI may be obtained by a first mean direction of the multispectral light beam
• the points in the second set of points P2 are obtained by a second mean direction of the multispectral light beam that is inclined vertically towards the road.
• the first mean direction is inclined vertically between +/- 2 degrees relative to said vehicle horizontal direction.
• the second mean direction is inclined between 5 to 20 degrees relative to a vehicle horizontal direction.
• the points in the first set of points PI are scanned more frequently than the points of the first set of points P2 for detecting the potential encountering object (vehicle 222) more frequently than sounding road to determine the road surface condition in the anticipated region of travel 271 of the vehicle 221.
• the points of the first set of points PI are scanned at least five or ten times more frequently than the points of the second set of points PI. Therefore, the potential encountering object can be detected more often than the road condition is analysed.
• the number of points in the second set of points P2 could be more numerous than the number of points in the first set of points PI to obtain a more accurate and more robust determination of surface condition.
• the determination of surface condition of said points on the road can take more time than the detection and/or determination of distance 260 of the potential encountering object (vehicle 222) as no intensity analysis and signature determination might be needed for that task.
• the processing of more frequently scanning the first set of points PI is more optimized. Additionally, such processing leads to a more safe control of vehicle 221, i.e. a more safe active safety system.
• the anticipated region of travel 271 is a region of road excluding boundaries of the road by detecting boundaries features.
• the boundaries features may include lane markers 280, divider lines 281, side rails, curbs, road signs.
• the boundaries features can be detected by analyzing the multispectral response of the multispectral lidar system.
• the road surface condition can be then determined based on the analyzing of the multispectral response corresponding to points on the road located only inside the anticipated region of travel 271.
• the actuating of an active safety routine is further based on the determination of the inclination of the anticipated region of travel 271 relative to an absolute horizontal direction, to identify if the anticipated region of travel is a rising portion of road or if the anticipated region of travel is a descendant portion of road, and influence the control provided by the active safety routine.
• Figure 3 includes a flow chart 300 for a set of methods for actuating an active safety routine that are in accordance with specific embodiments of the invention disclosed herein.
• the flow chart includes optional steps with dashed boundaries and sub-steps in tilted parallelograms attached to the steps for which they serve as component sub-steps.
• the flow chart serves as an outline for the remainder of this disclosure. The flow chart begins with steps executed by the sensing system, proceeds to steps associated with determining the road conditions, and concludes with steps taken to actuate an active safety routine based on the determined road conditions.
• Flow chart 300 begins with steps executed by a sensing system to obtain information regarding a road condition.
• the sensing systems utilized by the vehicle can take many different forms.
• sensing systems used in accordance with this disclosure can be active or passive.
• the sensing system could be a passive hyper spectral camera or an active LIDAR sensing system.
• active safety systems require some form of information regarding the environment of the vehicle, the same sensing system used to obtain that information can be used to obtain information regarding a road condition.
• an active LIDAR can determine the distance to another vehicle and also provide information regarding the road condition.
• the systems can also be different.
• an active LIDAR could determine the distance to another vehicle while a hyper spectral camera determined that the road was icy or wet.
• the same sensing system 351 is an active multispectral LIDAR sensing system used for both the active safety system and for detecting the road condition.
• the sensing systems disclosed herein can utilize sensor fusion to combine multiple sources of information.
• the fused information can be derived from active or passive sensors, data from the control system of the vehicle, or information pulled from a network connection on the vehicle. For example, readings from an active LIDAR sensing system which indicate a wet road surface, could be combined with information from a sensor used to detect rain for automatic windshield wipers and information from a network connection to a local weather surface to either reinforce a determination that a road surface is wet, or more accurately gauge how much water is on the road surface.
• the fused information could include optics for lane markers, text on signs, license plate numbers for tracking vehicles, weather information, and time of day information.
• the sensing and computations systems utilized for the methods disclosed herein can be located in various places depending upon the application.
• the sensing systems disclosed herein can be integrated with the vehicle on which the active safety routine will be actuated, can be located on a road infrastructure object for use in a V2I application, or can be located on an alternative vehicle for use in a V2V application.
• the sensing system 351 is a multispectral LIDAR system and is located on vehicle 350. Sensing system 351 is embedded along with a processing system 360 that can execute the determining, analyzing, and general computation steps associated with executing the methods illustrated by flow chart 300.
• Embedded system 360 can include a non-transitory computer readable medium 361 which stores instructions that can be executed by a processor to conduct the determining, analyzing, and generally computation steps disclosed herein.
• Embedded system 360 can also include a surface condition determination system 362, an active safety system 363, a user interface system 364, and a vehicle control system 365.
• the corresponding systems described above with reference to Figure 2 are specific examples of these systems.
• Active sensing systems used in accordance with this disclosure can include transmitting a sounding signal and analyzing the response of the sounding signal to the environment.
• the sounding signals disclosed herein can take on variant forms depending upon the desired analysis that will be conducted on the response of those sounding signals.
• the set of sounding signals for a given analysis can include a single signal, such as a laser pulse used for a time of flight ranging analysis, a set of multiple signals sent simultaneously using frequency or code multiplexing, or a set of multiple signals set at staggered times using time division multiplexing or phase shift multiplexing.
• the signals can be projected in patterns, such as structured light patterns, and the analysis can involve extracting depth information from a two-dimensional image of the return signals.
• flow chart 300 begins with step 301 in which a multispectral LIDAR system transmits a multispectral light beam and continues with a step 304 in which the response of a photodetector to the return of the multispectral light beam is analyzed.
• the sounding signals can be transmitted in various fashions depending upon the application.
• the sounding signals may be broadcast isotropically from the vehicle or directed.
• the signals can be directed by a fixed placement of the sensing system relative to the vehicle or actively directed via moving parts or the variation of the electronic properties of solid-state elements.
• the sounding signals disclosed herein can be directed at an anticipated region of travel of a vehicle within a road.
• the multispectral light beam transmitted in step 301 can be transmitted specifically at an anticipated region of travel of vehicle 350.
• the sounding signals may also be transmitted without ex ante knowledge of the anticipated region of travel.
• a multispectral light beam used as a sounding signal can be a single light beam with a temporally variant wavelength or multiple simultaneously projected light beams each with a variant wavelength.
• the light beams can be transmitted from vehicle 350 from one or more projection points that are either static with respect to vehicle 350 or movable.
• the vehicle may be designed to transmit the light beams in any direction from the vehicle based on the anticipated region of travel including movement in a reverse direction.
• the light beams may be actively directed at an anticipated region of travel if the anticipated region of travel is solved for or otherwise investigated prior to the execution of step 301.
• the peak power of the multispectral LIDAR system may be designed to generate actionable sounding signals at a distance of tens of meters. In specific approaches, the multispectral LIDAR system will emit a peak power above 20 Watts.
• the sensing system can include a laser source and a photodetector.
• the laser source can be a single broad-spectrum laser source used in combination with a beam splitter.
• the laser source can alternatively be a set of multiple laser sources.
• the detector of the sensing system can include a filter array or tunable filer.
• the multiple laser sources will be tuned to emit sounding signals with multiple wavelengths emitted at different times either at a specific point or in a patter.
• the single laser source will emit light through a filter that is tunable to change the wavelength of the emitted light temporally to generate different wavelength signals in frequency. More specific examples of these sensing systems are provided with reference to Figure 4 as follows.
• FIG 4 presents block diagrams of two sensing systems that can be used in accordance with specific embodiments of the invention.
• the sensing systems could be used in place of sensing system 351 in Figure 3.
• the illustrated sensing systems are configured to utilize multispectral light beams with different characteristics and using different approaches.
• the sensing systems are configured to both emit light and receive a response from that emission.
• the solid arrows in each diagram represent the light beam as it is emitted, and the dotted line represents the response.
• Figure 4 provides a multispectral LIDAR system 400 with a single laser source in the form of a broad-spectrum laser source 401.
• the multispectral light beam generated by multispectral LIDAR system 400 includes multiple wavelengths emitted at different times.
• System 400 includes a separating unit 402, a wavelength selection unit 404, and a scanning mirror 403.
• the scanning mirror can be a micro-electrical-mechanical (MEMS) system or any scanning mechanism capable of orienting itself relative to the light beam.
• the scanning mirror 403 could be a 1-dimensional MEMS mirror with a diameter of around 4 mm.
• Multispectral LIDAR system 400 can include a single photodetector 406, a beam splitter 405 located in separating unit 402, and a filter in wavelength selection unit 404.
• Photodetector 406 can be a broadband detector configured to detect light responsive to all the different wavelengths that can be emitted by multispectral LIDAR system 400.
• Element 404 can include a filter used to select the wavelength of light that will be emitted from the broad-spectrum laser source and the wavelength of light that will be admitted and routed to separating unit 402.
• the filter can be a filter array or tunable filter.
• Separating unit 402 can include a beam splitter 405 to deflect the received light and provide it to photodetector 406.
• beam splitter 405 could alternatively be configured to deflect light of different wavelengths to multiple photosensitive elements where each photosensitive element in the set of photosensitive elements is uniquely operatively sensitive to a spectrum of light in the multispectral light beam.
• the set of photosensitive elements and the set of potential wavelengths of light that can be emitted from LIDAR system 400 could have the same cardinality.
• multiple branches of elements 402, 403, and 404 could be attached to broad-spectrum laser source 401 with each branch optimized for transmitting light of a different wavelength.
• the multispectral light beam used by such a system could include multiple wavelengths emitted at the same time from a single laser source such as broad-spectrum laser source 401.
• Figure 4 also provides a multispectral LIDAR system 410 with a set of laser sources including laser source 411 and laser source 412.
• the system 410 can be configured to generate a multispectral light beam with a set of beams that are emitted simultaneously.
• the number of laser sources does not need to be set at two and is done so here strictly for illustrative purposes.
• the multispectral light beam generated by multispectral LIDAR system 410 includes multiple wavelengths emitted at the same time.
• System 410 includes multiple scanning mirrors 413 and 414 and multiple beam splitters 415 and 416.
• the scanning mirrors can be configured to transmit the multiple light beams to a single target.
• the scanning mirrors can have the same characteristics as scanning mirror 403.
• Multispectral LIDAR system 410 can include a photodetector with multiple photosensitive elements 417 and 418.
• the photosensitive element in the set of photosensitive elements can be uniquely operatively sensitive to a spectrum of light in the multispectral light beam.
• the photosensitive elements would be configured to be uniquely operatively sensitive to the spectrum of light defined by the light beam routed thereto by their associated beam splitter.
• the set of photosensitive elements and the set of beams in the multispectral light beam can have the same cardinality.
• step 304 of analyzing a response of a photodetector, to a return of the multispectral light beam transmitted in step 301.
• the photodetector could have the characteristics of photodetector 406 or the photodetector utilizing photosensitive elements 417 and 418.
• the analysis can be conducted by sensing system 351 itself, surface condition determination system 362, or by active safety system 363.
• the analysis can include detecting road boundaries for guiding further soundings or limiting further analysis on the response of the sounding signals.
• the analysis can include an analysis required by the active safety system such as a ranging analysis (e.g., a time of flight analysis) on a sounding signal.
• the analysis can include an analysis required by a road condition detection system such as a signature analysis.
• the signature analysis can include an intensity analysis to determine the absorption of a surface that reflected the sounding signal.
• the analysis can include a signature analysis such as a scaterometry analysis to determine the scattering properties of the surface that reflected the sounding signal.
• the analysis can include a signature analysis such as an interferometric analysis to determine the phase shifting properties of the surface that reflected the sounding signals.
• the sounding signals of the sensing system will be utilized in more than one analysis and by more than one system.
• Flow chart 300 includes a component step 305 in which the analyzing of the response step 304 includes both a time of flight analysis and an intensity analysis.
• the time of flight analysis can be a standard ranging analysis in which the round-trip time of the sounding signal is measured to extrapolate the distance from the target of the sounding signal to the vehicle.
• the time of flight analysis can be used by an active safety system, such as active safety system 363, to determine the distance to an obstruction, road boundary, or alternative vehicle.
• the intensity analysis can include measuring the absorption of a given target to more than one wavelength of light. The different wavelengths can be transmitted simultaneously, and the responses can be measured by dedicated photosensitive elements or they can be transmitted in sequence and be measured by dedicate elements or a detector and filter combination. The difference in the intensities can provide a description of the characteristics of the sounding target.
• the vehicle will determine its anticipated region of travel in order to focus the action of the sensing or analysis of road conditions. Determining the motion of travel can require the use of a dedicated sensing system or it can also utilize the sensing system used to determine road conditions (e.g., sensing system 351).
• the anticipated region of travel of the device can then be used to guide the sensing of the road condition such as by guiding the transmission of active sensing signals onto the road.
• the determination of the anticipated region of travel can be used to guide the analysis of the response the sensing system to limit the analysis to responses received from that anticipated region of travel.
• the direction of travel can be determined with reference to a detected set of road boundaries.
• determining the anticipated region of travel can be conducted entirely without reference to road boundaries such as by relying entirely on the known motion of the vehicle, the state of the vehicle's control system, and possibly predictive analytics regarding the future state of the vehicle.
• the vehicle will determine a set of at least one road boundaries on the road and utilize the obtained information to determine the anticipated region of travel of the vehicle.
• the set of at least one road boundaries can include a single landmark such as a road sign, or a collection of road boundaries such as painted dashed lines.
• the road boundaries may already be detected for purposes of the active safety system such that no additional software or hardware is required for step 302 to be conducted for the case of a road aware adaptive safety system.
• the set of road boundaries can be discovered using the same sensing signals used to determine the surface condition.
• the road boundaries in these approaches could be three-dimensional lane markers such as Botts' dots, curbs, or road signs.
• the set of road boundaries can be determined using a separate system such as an optical visible light, infrared, or ultraviolet sensing system.
• the sensing system could also be directed at a different angle than the main sensing system such as by being guided off to the sides of the vehicle for this purpose.
• the flow chart of Figure 3 is an example of approaches in which road boundaries are detected in a step 302, and the anticipated region of travel is determined in a step 303 based on the detection of those road boundaries. As illustrated, the anticipated region of travel is then used in a step 304 to guide the analysis of the received sounding signals.
• step 303, and potentially step 302 can be executed in combination with information from control system 365.
• control system 365 could include information that the vehicle is merging left or right in a controlled manner and ignore one set of road boundaries in the direction of the merge to assure that the road on the alternative side of the road boundaries and towards which the vehicle is transferring is properly analyzed.
• the system can determine, based on the analyzing of the response information, a surface condition of the road.
• the determination can be a determination regarding a hazardous surface condition.
• the determination can be made in the anticipated region of travel of the vehicle.
• the step 306 involves determining, based on the analyzing of the response in step 304, a hazardous surface condition in the direction of travel.
• the hazardous surface condition is determined in the direction of travel because knowledge of the direction of travel was used to focus the analysis in step 304.
• the determination of the surface condition can be made based upon a signature of the road surface derived from analyzing the sounding signals.
• the vehicle can store a database of signatures for this purpose (e.g., snow, water on asphalt, gravel, etc.) to be used to determine the surface condition.
• a multispectral signature produced in step 304 could be applied to a database of multispectral signatures associated with such conditions.
• the database can be accessed by applying the signature to a feature detector followed by a classifier to identify the signature applying to a specific class.
• the signature can also include additional data fused with the output of the analysis of the sensing system's data, and this fused data can likewise be applied to a feature detector and classifier, or otherwise used to search a database.
• the determination step could involve the determination of a gradation of the road condition, the binary presence of absence of a surface condition, or the determination of multiple conditions.
• the signature could be associated with entries "ice” or "no ice” in the database, alternatively the signature could indicate a degree of intensity of a condition such as on a scale of zero to one where zero represented a perfectly even surface and one represented a condition nearly certain to cause massive tire damage. Regardless of whether the response and analysis generated a signature, the execution of the determination step could involve determining such gradations.
• Step 306 is shown as having an optional component step 307 in which a gradation of a given condition is generated.
• the execution of optional component step 307 could capture the degree of intensity of a given hazardous condition.
• the degree of intensity can affect how the active safety system is actuated in response to the condition. For example, an automatic stabilization system could slow the car down when detecting ice of over 30% while the stabilization system would shut the car down is detecting ice over 60%.
• the determination can also involve an analysis of the surface topology of the road surface as generated during the analysis. For example, the sounding signals could have generated a signature indicative of the presence of potholes or other uneven surface in the road which would be determined by the surface condition determination system upon analyzing the signature.
• the determination can also be a determination of a mixed condition; for example, the determination could determine that a road surface is a wet road and that the road has only been wet for about 15 minutes. This determination could involve the vehicle keeping track of how long the car has been detecting a wet road for or could involve an application of sensor fusion in which a networked weather report centered on an anticipated region of travel indicated that it had been raining for 15 minutes.
• the recency of the wet condition could be indicative of a higher level of hazard as the water on the road may be mixed with oil that has not fully been rinsed from the road by the water.
• the same condition could be determined from a different analysis where the signature of the road surface was precise enough to distinguish a road that is wet from water with a road that is wet from a mixture of oil and water.
• the vehicle can actuate an active safety routine based on the road condition.
• the vehicle can actuate an active safety routine based on the determination of a hazardous surface condition.
• Figure 3 includes a step 309 of actuating, based on the determination of the hazardous surface condition in step 306, an active safety routine. Step 309 can involve triggering the active safety routine upon detecting the hazardous surface condition or modifying the actuation of the active safety routine upon detecting the hazardous surface condition.
• Figure 3 also includes the optional step 308 of determining based on the hazardous surface condition of the road determined in step 306, a friction value for the road.
• the friction value for the road could then be used as an input variable for the execution of an active safety routine to either alter when the routine was actuated or the manner in which the active safety routine was actuated.
• the actuating of the active safety routine in step 309 is based on the determination of the hazardous surface condition in step 306 in that the friction value 308 is used in a calculation of the active safety routine.
• the calculations of an active safety system can be affected by this degree.
• the degree can also be used as an input variable to the execution of the active safety routine in the same way that the friction variable is used in the example of Figure 3.
• the system can actuate an active safety routine based on that surface condition.
• the active safety routine could be a automatic emergency braking (AEB) ADAS which actuates at a point determined by the surface condition.
• the detection of the surface condition could include detecting the fact that the road is iced over and calculating an expected grip value for the road.
• the collision avoidance system could use this information to determine that an automatic braking action should be executed at 15 meters from an obstruction as opposed to 5 meters from that same obstruction with all else held equal.
• the active safety routine could be an ACC ADAS which alters the manner in which it is actuated based on the surface condition.
• the detection of the surface condition could include the same fact that the road is iced over and the same expected grip value could be determined.
• the ACC ADAS instead utilize that information to determine the safe following distance that should be applied during the actuation of the ADAS and follow a car at 20 meters instead of 5 meters.
• the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the actuating is conducted in response to the determining of a presence of the hazardous surface condition. For example, a collision avoidance ADAS could detect a surface hazard condition so acute that it would need to immediately execute to avoid the hazard entirely.
• Hill descent control systems can increase the internal relationship of the system between gear size and slope.
• Intelligent speed adaptation or ISA systems can warn or protect users from speeding in jurisdictions were the speed limit is set by road conditions.
• Lane departure warning systems or lane change assistance can assure a vehicle makes gradual corrections or more cautions lane changes when roads are wet.
• Pedestrian protection and intersection assistance ADAS can increase the guard bands of uncertainty for slippery or uneven roads.
• Vehicles can also transmit messages indicative of the road condition for usage by other vehicles or road infrastructure.
• the messages can be sent to infrastructure or directly to alternative vehicles.
• an optional step 310 involves transmitting a message, regarding the hazardous surface condition determined in step 306 to a second vehicle on the road.
• the message can be broadcast from the device for consumption by multiple vehicles or targeted specifically for an alternative vehicle.
• the message can be received by one or more alternative vehicles and allow them to actuate their own active safety routines in response to that determination.
• the messages can be sent along with identification information concerning the location of the hazardous condition.
• the receiving vehicles can then choose to operate on messages in which the identified hazardous condition is in an anticipated region of travel for that vehicle.
• the method of the invention includes the following provisions.
• Provision 1 A method, performed by a system embedded in a vehicle, comprising the steps of: transmitting, with a multispectral lidar system, a multispectral light beam directed at an anticipated region of travel of the vehicle within a road; analyzing a response, of a photodetector, to a return of the multispectral light beam; determining, based on the analyzing of the response, a hazardous surface condition in the anticipated region of travel of the vehicle; and actuating, based on the determination of the hazardous surface condition, an active safety routine.
• Provision 2 The method of provision 1, wherein the multispectral lidar system emits a peak power above 20 W.
• Provision 3 The method of provision 1, the analyzing of the response includes a time of flight analysis and an intensity analysis.
• Provision 4 The method of provision 1, the active safety routine is conducted by one of: (i) an adaptive cruise control (ACC) system; (ii) a collision avoidance system; (iii) an automatic emergency breaking (AEB) system; (iv) a forward collision warning system; (v) a hill descent control system; (vi) an intelligent speed adaptation system; and (vii) an intelligent speed advice (ISA) system.
• ACC adaptive cruise control
• AEB automatic emergency breaking
• ISA intelligent speed advice
• Provision 5 The method of provision 1, wherein: the multispectral lidar system includes a single laser source; and the multispectral light beam includes multiple wavelengths emitted at different times.
• Provision 6 The method of provision 1, wherein: the multispectral lidar system includes a single laser source; and the multispectral light beam includes multiple wavelengths emitted at the same time.
• Provision 7 The method of provision 1, wherein: the multispectral lidar system includes: (i) a single laser source; (ii) a beam splitter; and (iii) a filter; the photodetector includes a set of photosensitive elements; the multispectral light beam includes a set of beams; and the set of photosensitive elements and the set of beams have the same cardinality.
• Provision 8 The method of provision 1, wherein: the photodetector includes a set of photosensitive elements; and each photosensitive element in the set of photosensitive elements is uniquely operatively sensitive to a spectrum of in the multispectral light beam.
• Provision 9 The method of provision 1, wherein: the photodetector includes a photosensitive element; and the photosensitive element is operatively sensitive to all spectrums in the multispectral light beam.
• Provision 10 The method of provision 1, wherein: the determining includes determining a degree of the hazardous surface condition.
• Provision 11 The method of provision 1, further comprising: determining, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for the active safety routine.
• Provision 12 The method of provision 1, wherein: determining, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for when the active safety routine is actuated.
• Provision 13 The method of provision 1, wherein: the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the actuating is conducted in response to the determining of a presence of the hazardous surface condition.
• Provision 14 The method of provision 1, wherein: the actuating of the active safety routine is based on the determination of the hazardous surface condition in that a manner in which the active safety routine is actuated is modified according to a degree of the hazardous surface condition.
• the system of the invention includes the following provisions.
• Provision 15 A system, embedded in a vehicle, for mitigating a hazardous surface condition of a road, the system comprising: a multispectral lidar system, configured to transmit a multispectral light beam directed at an anticipated region of travel of the vehicle within the road; an active safety system; and one or more non-transitory computer readable media storing instructions to:
• Provision 16 The system of provision 15, wherein the multispectral lidar system emits a peak power above 20 W.
• Provision 17 The system of provision 15, the analyzing of the response includes a time of flight analysis and an intensity analysis.
• Provision 18 The system of provision 15, the active safety routine is conducted by one of: (i) an adaptive cruise control (ACC) system; (ii) a collision avoidance system; (iii) an automatic emergency breaking (AEB) system; (iv) a forward collision warning system; (v) a hill descent control system; (vi) an intelligent speed adaptation system; and (vii) an intelligent speed advice (ISA) system.
• ACC adaptive cruise control
• AEB automatic emergency breaking
• ISA intelligent speed advice
• Provision 19 The system of provision 15, wherein: the multispectral lidar system includes a single laser source; and the multispectral light beam includes multiple wavelengths emitted at different times.
• Provision 20 The system of provision 15, wherein: the multispectral lidar system includes a single laser source; and the multispectral light beam includes multiple wavelengths emitted at the same time.
• Provision 21 The system of provision 15, wherein: the multispectral lidar system includes: (i) a single laser source; (ii) a beam splitter; and (iii) a filter; the photodetector includes a set of photosensitive elements; the multispectral light beam includes a set of beams; and the set of photosensitive elements and the set of beams have the same cardinality.
• Provision 22 The system of provision 15, wherein: the photodetector includes a set of photosensitive elements; and each photosensitive element in the set of photosensitive elements is uniquely operatively sensitive to a spectrum of in the multispectral light beam.
• Provision 23 The system of provision 15, wherein: the photodetector includes a photosensitive element; and the photosensitive element is operatively sensitive to all spectrums in the multispectral light beam.
• Provision 24 The system of provision 15, wherein: the photodetector includes a photosensitive element; and the photosensitive element is operatively sensitive to all spectrums in the multispectral light beam.
• Provision 25 The system of provision 15, wherein: the determining includes determining a degree of the hazardous surface condition.
• Provision 26 The system of provision 15, wherein the one or more non-transitory computer readable media further store instructions to: determine, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for the active safety routine.
• Provision 27 The system of provision 20, wherein the computer readable media further store instructions to: determine, based on the hazardous surface condition of the road, a friction value for the road; and wherein the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the friction value is used in a calculation for when the active safety routine is actuated.
• Provision 28 The system of provision 15, wherein: the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the actuating is conducted in response to the determining of a presence of the hazardous surface condition.
• Provision 29 The system of provision 15, wherein: the actuating of the active safety routine is based on the determination of the hazardous surface condition in that the manner in which the active safety routine is actuating is modified according to a degree of the hazardous surface condition.
• Provision 30 A method, performed by a system embedded in a vehicle, comprising the steps of: transmitting an electromagnetic signal at an anticipated region of travel of the vehicle within a road; analyzing, using a sensor, a return of the electromagnetic signal; determining, based on the analyzing of the return, a hazardous surface condition in the anticipated region of travel of the vehicle; and modifying an actuation of an active safety routine based on the determination of the hazardous surface condition.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• General Physics & Mathematics (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Electromagnetism (AREA)
• Traffic Control Systems (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=73059934

Family Applications (1)

Country Status (2)

Family Cites Families (3)

• 2020
  • 2020-11-03 EP EP20800911.8A patent/EP4055414A1/de not_active Withdrawn
  • 2020-11-03 WO PCT/EP2020/080835 patent/WO2021089557A1/en unknown

Also Published As

Similar Documents

Legal Events