EP2012211A1 - A system for monitoring the surroundings of a vehicle - Google Patents
A system for monitoring the surroundings of a vehicle Download PDFInfo
- Publication number
- EP2012211A1 EP2012211A1 EP07111671A EP07111671A EP2012211A1 EP 2012211 A1 EP2012211 A1 EP 2012211A1 EP 07111671 A EP07111671 A EP 07111671A EP 07111671 A EP07111671 A EP 07111671A EP 2012211 A1 EP2012211 A1 EP 2012211A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- road
- vehicle
- monitoring system
- lane
- surroundings monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/167—Driving aids for lane monitoring, lane changing, e.g. blind spot detection
Definitions
- the invention relates to a surroundings monitoring system according to the preamble of claim 1.
- the invention particularly relates to a surroundings monitoring system which includes a radar arranged to detect a road boundary within a radar zone positioned ahead of the vehicle.
- Systems for monitoring the surroundings of a vehicle are well known in the art.
- Various types of systems for monitoring the surroundings of a vehicle are known. All such systems include sensors retrieving information about the surroundings of a vehicle, processing means arranged to interpret the retrieved information, and intervention means arranged to activate specific functions dependent on how the processing means have interpreted the retrieved information.
- a specific type of surroundings monitoring system for monitoring the surroundings of a vehicle comprises a camera being mounted on a vehicle for detecting an image of a road in front of said vehicle.
- Such surroundings monitoring systems are useful for detection of the position of a vehicle within a lane. Knowledge of the position of a vehicle relative to a lane is of importance for active safety systems such as lane departure warning systems, which may alert a driver in the event the vehicle is about to leave a lane under certain conditions.
- Surroundings monitoring systems which are arranged to detect lane markers ahead of the vehicle are well known in the art. Most such systems include a camera detecting an image of the road in front of the vehicle. An image recognition means provided in the system will interpret the image and specifically locate the position of lane markers. Image recognition means, which are well known in the art, identifies the presence and location of predefined objects in an image constituted by an array of pixels.
- the predefined objects may be constituted by simple objects such as lines, or circle segments or more complex objects such as vehicles.
- Lane markers are relatively easy to detect, since they are normally painted in a colour which forms a sharp contrast to the road surface, road shoulders and the environment external of the road which encloses the road.
- a suitable image recognition system for determining the position of lane markers on a road surface makes use of a Hough transform to identify the position of the lane markers.
- the Hough transform which is extensively described in the literature, is excellent for determination of the presence and position of simple objects such as lines in an image. In a more generalized form the transform can be used to determine the presence and location of circle segments and other geometrical shapes.
- a description of an image recognition system capable of determining complex patters is present in US3069654 by Paul Hough .
- Lane markers In order to accurately determine the position of the lane markers it is of importance that the image is of relatively high quality with sharp contrast. As the relative angle between the road surface and an optical axis of the camera decreases with the distance from the vehicle, the contrast between the lane marker and the road surface decreases as well. Lane markers can therefore only be detected within a relatively small vision zone positioned ahead of the vehicle. A normal vision zone extends about 20 - 50 meters from the vehicle.
- monitoring systems normally further comprises a lane departure warning unit, which is arranged to generate an intervention in the event said lane warning unit has determined that the vehicle is about to leave the lane.
- the lane warning system should only normally be allowed to intervene when a driver is involuntary leaving a lane.
- warning signs should only be generated when the driver has not used the direction indicator. Considering the normal behavior of drivers, this has shown not to be sufficient to separate between voluntary and involuntary departure of the lanes.
- a vehicle position determination unit being arranged to determine a lateral position of a vehicle with respect to a lane defined by the lane markers detected by said camera must be able to determine the position of the vehicle relative to the lane marker with an accuracy in the cm range.
- the lateral velocity must also be determined with a high accuracy.
- knowledge of the relative distance between the vehicle and lane marker is not sufficient.
- the curvature of the road must also be accounted for. The requirement of determining the curvature of the road adds to the uncertainty of the estimation, since both the position of lines identified in the image by the image recognition system and the inclination of the lines suffers from estimation errors.
- an image recognition system In order to cope with the problem of determining the lateral position of a vehicle relative to the lane markers and the lateral velocity of the vehicle relative to the lane markers for a vehicle traveling at a relatively high speed by use of cameras having limited resolution, an image recognition system according to the state of the art is used to determine the lateral position of the vehicle relative to the lane marker by use of information from an instant image.
- An example of a suitable system is sold under the name MobilEye. In figure 1 , such an image is shown.
- the lane markers 2 detected by the camera 2 within the vision zone 4 is used to determine the relative lateral position y(t 0 ) at the time to.
- the image recognition system 6 detects the presence of the lane markers and establishes the lateral distance y(t 0 ) of the vehicle relative to the lane markers 2 based on the position of the camera relative to the vehicle and the position of the lane markers relative to the camera.
- the determination of the relative position y(t 0 ) of the vehicle with respect to the road may be determined by the use of known systems including a camera which detects the image of the road and an image recognition system which detects lane markers within the vision zone of the vehicle. By use of for instance a Hough transform, an estimate of the position of the lane markers can be detected with a sufficient degree of accuracy.
- the estimate of the position of the lane markers will represent a curved or straight line 8, which is used to determine the lateral position of the vehicle by geometric calculation. Since the resolution of cameras is limited, the exact position and curvature of the road geometry ahead of the vehicle is not determined.
- the estimated positions of the lane markers are primarily used to determine the current lateral position of the vehicle.
- the current lateral position of the vehicle in relation to the lane markers may be determined according to the method disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624 .
- a warning signal is generated depending of the vehicles relative position to the lane marker.
- a surroundings monitoring system including a departure warning unit, which is well adapted to normal behavior of drivers such that the systems capacity to separate between involuntary and voluntary departures is improved.
- a surroundings monitoring system includes a radar being arranged to detect a road boundary within a radar zone, said radar zone being positioned a head of said vehicle and preferably extending further from said vehicle than said vision zone.
- the radar will detect objects positioned along the road boundary, such as guide rails, trees, stones etc.
- the detected objects will define a road boundary, which forms a transition from a generally flat surface to a border formed by the detected objects.
- the detected road boundary does not form a continuous line, but a large set of detected objects forming an uneven interrupted border, it may be necessary to from the detected road boundary form a continuous line that estimates the road boundary.
- the surroundings monitoring system therefore furthermore includes a shoulder edge determination unit arranged to determine a lateral position of a shoulder edge based on the detected road boundary.
- the shape of the shoulder edge is determined according to the same principles as is used for generally determining a road shape or the shape of the longitudinal extension of lane markers, but the road boundary detected by the radar is used as input data instead or in addition to lane markers detected by a camera.
- the process described in A. Polychronopoulos, et. al.; "Integrated object and road border tracking using a 77 GHz automotive radar",; IEE Proc. - Radar Sonar Navig., Vol 151, No 6, December 2004 may be used.
- the shape of the longitudinal extension may thus be performed by adapting an curve to lane markers present in the detection zone of the camera such that the curve follows the lane markers.
- an exact lateral position of the shoulder edge is determined.
- the lateral position of the shoulder edge is selected such that it forms a tangent to parts of the road boundary within the detection zone being closest to the road centre.
- the surroundings monitoring system furthermore includes a road departure warning unit which is arranged to generate an intervention based on relative position of the vehicle to the shoulder edge.
- a road departure warning unit which is arranged to generate an intervention based on relative position of the vehicle to the shoulder edge.
- the intervention may be made in the event said road departure warning unit has determined that the vehicle approaches the road boundary and is positioned at a certain gap from the shoulder edge.
- the surroundings monitoring system may furthermore include a warning line generation unit which is arranged to locate a warning line at a gap from the shoulder edge.
- the size of the gap may be dependent on the width of the road shoulder, which is the distance from the lane markers to the shoulder edge.
- the size of the gap may furthermore be dependent on the curvature radius of the road.
- the decision to intervene will be based on the relative lateral position of the vehicle with relation to the warning line, which depends on the position of the shoulder edge.
- the shoulder edge determination unit, and, if included, the warning line determination unit and/or a lane marker position determination unit all use the same estimator for forming the respective shapes of the shoulder edge, warning line, and if included a lane marker position estimate.
- This may be done in a common road geometry determination unit using the estimating process disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624 .
- the timing of the intervention may thus be made dependent on an estimated time for the vehicle to cross the shoulder edge or preferably the warning line or distance to the shoulder edge or preferably warning line or a combination thereof, that is the timing of the intervention will be dependent on both the time for the vehicle to cross the shoulder edge and distance to the shoulder edge or preferably the time for the vehicle to cross the warning line and distance to the warning line.
- the surroundings monitoring system will be better adapted to an accepted behavior of drivers to temporarily use road shoulders.
- the system will be especially advantageous for curved roads where drivers tend to cut inner curves, where road shoulders exists and the driver have a clear sight of the road ahead of the vehicle.
- the decision whether an intervention should be made or not by the road departure warning unit is based on the relative position of the vehicle to a warning line which is positioned at a certain gap G from a shoulder edge
- the surroundings monitoring system includes a road geometry determination unit, which is arranged to determine the road geometry ahead of the vehicle based on the road boundary detected by said radar.
- the road geometry determination may preferably be performed in the manner disclosed in A. Polychronopoulos, et. al.; "Integrated object and road border tracking using a 77 GHz automotive radar",; IEE Proc. - Radar Sonar Navig., Vol 151, No 6, December 2004 .
- a road border geometry is determined in a road border assignment process, which estimates the road border geometry from measurement data obtained by the radar.
- a probalistic and a geometric method are disclosed as alternatives to assign the road borders.
- a weighted combination of both methods is used to obtain an accurate estimate of the road geometry.
- said surroundings monitoring system further comprises a camera being mounted on a vehicle for detecting an image of a road within a vision zone and an image recognition system being arranged to detect lane markers within said vision zone.
- the detection of lane markers can be used to improve the estimate of the road geometry by disregarding measurement data that are too distant from the lane markers.
- said surroundings monitoring system further comprises a camera being mounted on a vehicle for detecting an image of a road within a vision zone and an image recognition system being arranged to detect lane markers within said vision zone.
- the road geometry may be completely determined from the position of the lane markers detected by the camera. The detection of lane markers can be used to improve the estimate of the by disregarding measurement data that are too distant from the lane markers.
- the road border geometry in the third embodiment is determined in a road border assignment process, which estimates the road border geometry from measurement data obtained by the radar.
- the detected position of the lane markers can furthermore be used to determine the presence and extension of a road shoulder.
- the surroundings monitoring system further comprises a road shoulder assessment unit, which is arranged to assess the lateral extension of a road shoulder as the difference in lateral position of the shoulder edge estimated from the road boundary detected by the radar and a lane marker adjacent to said shoulder, which lane marker being detected by the camera.
- the road departure warning unit may be furthermore arranged to intervene in dependence of the lateral extension of the road shoulder, i. e. on the difference in lateral position of the road boundary detected by said radar and a lane marker adjacent to said shoulder, which lane marker being detected by said camera.
- the road departure warning system may furthermore be arranged to intervene in dependence of the curvature of the road.
- the dependence may preferably be of such a nature that such that the distance from a centre line of the lane to the point of intervention is increased for curves being convex toward the centre line and the distance from a centre line of the lane to the point of interception is decreased for curves being concave toward the centre line., that is the point of intervention is moved to the right at curves turning to the left and vice versa.
- the presence of a radar determining the road boundary ahead of the vehicle is beneficial for adapting lane warning departure systems to the actual road geometry.
- One advantage is that the existence and size of road shoulder may be assessed from estimating the distance to a road boundary detected by the radar to a lane marker detected by the camera.
- the lane warning system can thereafter be adapted to intervene in dependence on the existence and size of the road shoulder.
- Another advantage is that a lane warning system can be adapted to the road geometry such that the vehicle will be allowed to cut certain types of curves, while being prevented to cut other type of curves.
- Figure 2 shows a schematic drawing of a surroundings monitoring system 10 determining the relative lateral position y(t) of a vehicle 1 on a road 12 with lane markers 14.
- Figure 3 shows a block diagram of a surroundings monitoring system 10 according to the invention.
- the surroundings monitoring system 10 includes a radar 22 being arranged to detect the road boundary 24 within a radar zone 26, said radar zone being positioned a head of said vehicle.
- the radar is thus necessarily directed in the forward direction of the vehicle.
- the road boundary 24 detected by the radar 22 is formed by a reflected radar beam. All objects that vertically extend in the height direction along the road will be detected. Examples of objects which generate a sharp detectable reflection are parked cars, guard rails, solid objects such as rocks, houses, trees, animals and human beings.
- the lateral distance to the road boundary may be directly used as input to a road departure warning unit.
- the input data from the radar may however be first treated by a road geometry determination unit 23.
- the road geometry determination unit may retrieve its input data from a camera 16..
- a suitable road boundary determination unit is sold under the trade name Mobil Eye.
- the road geometry determination unit may also be arranged as disclosed in A. Polychronopoulos, et.
- the road border geometry is determined in combined probalistic and geometric method disclosed in the document.
- a road boundary 24 will thus be determined by detection of the reflected radar signal.
- a shoulder edge 28 will be determined from the detected road boundary in a shoulder edge determination unit 27.
- the road shoulder determination unit is arranged to determine the shape and lateral position of a shoulder edge based on the detected road boundary by forming an essentially continuous shoulder edge.
- the shoulder edge determination unit 27 is preferably included in a road geometry determination unit 26.
- the shoulder edge representation may be a curved or straight line.
- a road boundary 24 and thus a shoulder edge 28 can be determined ahead of the vehicle at distances exceeding 150m.
- the use of a radar 22 to determine the road boundary ensures that the shoulder edge 28 can be determined at a distance extending further from said vehicle than an actual vision zone, in which lane markers are detected.
- a particular advantage in using a radar for detecting a road boundary is that vertically extending objects can be detected with a good accuracy at a large distance, while the lane markers are difficult to detect at a large distance since the relative angle between the cameras optical axis and the surface of the lane marker becomes very small for distant lane markers. It is therefore very difficult to detect lane markers, if at all possible, at a distance exceeding 20 - 50 m.
- the representation of the road boundary 24 is stored in a memory suitably as a vector R( ⁇ ((x,y)(t))) for angles ⁇ relative to the heading direction ⁇ of the vehicle positioned at the location (x,y) the time t.
- a representation of a road boundary R( ⁇ (x,y(t))) for a vehicle is illustrated.
- a vehicle position determination unit 29 included in the vehicle surroundings monitoring system is arranged to determine a lateral position y(t) of a vehicle on a road 12 with respect to a shoulder edge 28, or preferably a warning line 42 as will be explained below.
- the vehicle position determination unit may preferably include a camera 16 as an input device for deriving the actual lateral position of the vehicle.
- a camera 16 is mounted on a vehicle for detecting an image of a road within a vision zone 18.
- An image recognition system 20 is arranged to detect lane markers 14 in the image captured by the camera.
- the camera 16 and the image recognition system 20 may be any known arrangement which is able to capture an image of lane markers and determine the position of the vehicle relative to lane markers.
- the vision zone 18 preferably extends ahead of the vehicle as shown in figure 2 .
- the camera can be directed in the rearward direction. This is possible since the main task of the image recognition system is to determine the current lateral position of the vehicle, which may be determined either by image recognition detecting the lane markers in the front or rear direction of the vehicle.
- the radar zone preferably extends further from the vehicle than the vision zone.
- a road shoulder assessment unit 30 is included in the surroundings monitoring system, which road shoulder assessment unit 30 is arranged to assess the lateral extension D of a road shoulder 31 as the difference in lateral position of the shoulder edge estimated from road boundary 24 detected by said radar 22 and a lane marker 14 adjacent to said shoulder 31, which lane marker 14 being detected by said camera 16.
- the road shoulder is thus a relatively plane surface limited on one side of a lane marker 14 detected by a camera, and on the other side of a shoulder edge preferably forming a tangent to vertically extending objects, detected by the radar, and which are defining a road boundary 24.
- the surroundings monitoring system includes a warning line generation unit 33 which is arranged to locate a warning line 42 R , 42 L at a gap from the shoulder edge 28.
- the size of the gap G may be dependent on the width D of the road shoulder, which is the distance from the lane markers 14 to the shoulder edge 28.
- the size of the gap G may furthermore be dependent on the curvature radius R of the road.
- the decision to intervene will be based on the relative lateral position of the vehicle 1 with relation to the warning line 42 R , 42 L , which depends on the position of the shoulder edge. If a warning line generation unit is not included the decision to intervene will be based on the relative lateral position of the vehicle 1 with relation to the position of the shoulder edge directly.
- the shoulder edge determination unit 27, and, if included, the warning line determination unit 33 and/or a lane marker position determination unit all use the same estimator for forming the respective shapes of the shoulder edge, warning line, and if included an lane marker position estimate.
- This may be done in a common road geometry determination unit 23 using the estimating process disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624 .
- the camera will be used to generate indata to estimate y_off och Psi_rel.
- c0 is measured from input data from the camera and radar as is disclosed in the reference. Based on these equations the Kalman filter is created according to general practice.
- the respective lines will then be a set of parallel curved lines off set a certain distance from each other.
- the surroundings monitoring system furthermore comprises a road departure warning unit 32, which is arranged to generate an intervention in the event said road departure warning unit has determined that the vehicle is about to leave the road.
- the intervention may be provided in different forms, such as generation of a warning sign, activation of brakes, active steering of the vehicle, etc..
- the most common lane warning units will provide an intervention in the form of a warning alerting the driver.
- the warning may be visual or audial.
- a road departure warning unit 32 is arranged to determine that the vehicle is about to leave the road based on relative position of the vehicle to the warning line, which is positioned at a certain gap G from the shoulder edge 28.
- the size of the gap may be dependent on the lateral extension D of the road shoulder and/or of the curvature radius R of the road.
- the intervention will be made when the vehicle approaches the warning line and is positioned at a certain distance S from the warning line.
- the distance S may be determined in different manners.
- said road departure warning 32 unit is arranged to determine that the vehicle is about to leave the road, when an estimated time to departure T, that is the time T needed for the vehicle to cross the warning line, is below a threshold value, S: T ⁇ Threshold.
- road departure warning unit 32 is arranged to determine that the vehicle is about to leave the road, when an estimated distance to departure L, that is the distance required for the vehicle to travel in order to cross the warning line, is below a threshold value, S: L ⁇ Threshold.
- the road departure warning unit 32 is arranged to determine that the vehicle is about to leave the road, that is to pass the warning line, when a value of a function f of an estimated distance to departure L and an estimated time to departure T is below a threshold value, S: f(T,L) ⁇ Threshold.
- the warning line positioned at a gap G from the road shoulder will form a continuous warning line running along the extension of the road boundary 24.
- the gap size can be made dependent of the position of the lane markers.
- the road shoulder assessment unit 30 is arranged to assess the lateral extension D of a road shoulder 31 as the difference in lateral position of the shoulder edge 28 determined from the road boundary 24 detected by said radar 22 and a lane marker 14 adjacent to said shoulder 31.
- the lateral extension D is within a range, such that D min ⁇ D ⁇ D max , where D max is smaller than a normal lane width, it is possible to exclude misinterpretation of a neighboring lane as a road shoulder, which may be used by the vehicle. This criterion can also be used to prevent that a neighboring lane traveling in the opposite direction is interpreted as road shoulder.
- Different rules may be applied for detected flat ground on the left and right side of the vehicle.
- a road boundary to the right of the vehicle may be used when defining a road shoulder.
- This rule may be applicable in countries with right hand traffic. Consequently a rule that only a road boundary to the left of the vehicle may be used when defining a road shoulder may be used in countries with left hand traffic.
- the lateral distance to an actual lane marker can be positive, that is an intervention is generated before the vehicle passes the actual lane marker, or negative, that is an intervention is made after the vehicle has passed the actual lane marker.
- FIG 5 a road curved towards the left in the drawing is shown.
- the road is provided with lane markers 14 L , 14 M , 14 R forming a left, middle and right delimitation of two lanes 34, 36.
- a vehicle 1 is travelling in the right lane 36 in the upwards direction in the figure.
- Warning lines 40 L , 40 R forms the fixed reference in lateral position for a road departure warning unit.
- the road departure warning unit compares the vehicles relative position to the warning lines in order to determine whether an intervention should be made or not.
- the warning lines marks the position at which in intervention will be made by a road departure warning unit in the event the vehicle will pass the warning lines 40 L , 40 R .
- the timing of the intervention can be based on an estimated time to departure T when the vehicle is assumed to cross the warning line.
- a warning line exists for the left and right side of the lane in which the vehicle travels.
- the warning lines are positioned at gap G from the shoulder edge estimated from the road boundary 24 detected by the radar.
- the size of the gap is determined in a gap size determination unit 44, which calculates the position relative to a road boundary.
- Input to the gap size determination unit 44 is information from the road geometry determination unit 26, the shoulder assessment unit 30, a vehicle speed sensor 46 and a vehicle yaw rate sensor 48.
- the gap size may be dependent on the following input variables alone or taken in combination:
- the timing of the intervention may be based on:
- the warning line 40 L is positioned at a distance ⁇ in the lateral direction to the lane markers 14 R .
- the magnitude of the distance ⁇ may be positive, that is the warning line will be positioned in the lane, or negative, that is the warning line will be positioned outside of the lane.
- the warning line 40 R is positioned in the lane 36, which indicates that the distance ⁇ has a positive value
- the warning line 40 L is positioned outside of the lane 38, which indicates that the distance ⁇ has a negative value. If the distance ⁇ has a negative value an intervention will be made before the vehicle exits the lane, while if the distance ⁇ has a positive value an intervention will be made after the vehicle exits the lane.
- the lateral position of the warning line 40 L , 40 R with respect to the road boundary 24 is dependent on the curvature of the road such that a minimal lateral distance from a centre of curvature to said warning line is decreasing with a decreased radius of curvature of the road.
- This allows a vehicle to cut an inner curve with a certain distance, while preventing the vehicle to ride to close to the lane markers in an outer curve.
- it is possible to set different rules for the left and right lane markers in the event, for instance it should not be allowed to pass a lane marker positioned adjacent to an other lane, while allowing the vehicle to pass lane markers which are facing a road boundary, especially in the event a rod shoulder is positioned on the other side of the lane marker.
- the warning lines 42 L , 42 R marks the position at which in intervention will be made by a road departure warning unit in the event the vehicle will pass the warning lines 42 L , 42 R .
- the warning lines are positioned at gap G from the shoulder edge estimated from the road boundary 24 detected by the radar.
- the size of the gap is determined in a gap size determination unit 44, which calculates the position relative to a road boundary.
- Input to the gap size determination unit 44 is information from the road geometry determination unit 26, the shoulder assessment unit 30, a vehicle speed sensor 46 and a vehicle yaw rate sensor 48.
- the gap size may be dependent on the curvature radius R determined by the road geometry determination unit 26, the presence and lateral extension D of road shoulders determined by road shoulder assessment unit 30.
- warning lines 42 L , 42 R are shifted towards the right in stead of to the left as in figure 5a .
- the magnitude of the shift; that is the size of the gap is dependent on the curvature radius such that a smaller curvature radius R will generate a larger gap.
- Figure 5c is a schematic drawing showing the relations between the road boundary 24, shoulder edge 28, warning line 42 R , and lane marker 14.
- Figure 5c shows the geometry of the road including discontinuous set of radar reflections forming a road boundary 24.
- a continuous shoulder edge 28 is determined from the radar reflections forming the road boundary 24 in a shoulder edge determination unit 27, which may be included in the road geometry determination unit 26.
- a warning line 42 R determined in a warning line generation unit is located at a gap G from the shoulder edge 28.
- a lane marker 14 is positioned to the left in the figure.
- a road shoulder extending between the lane marker 14 and shoulder edge 28 has a lateral extension D.
- a decision to intervene is based on the relative lateral position S of a vehicle 1 to the warning line 42 R .
- the warning line 42 R is positioned at a distance ⁇ to the lane markers in the lateral direction.
- the position of the warning line 42 R may be on either side of the lane marker 14 depending on the geometry of the road and/or location of the shoulder edge relative to the lane marker.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
A surroundings monitoring system (10) for monitoring the surroundings (11) of a vehicle (1) comprising:
a radar (22) being arranged to detect a road boundary (24) within a radar zone (26), said radar zone (26) being positioned a head of said vehicle (1), a shoulder edge determination unit (27) arranged to determine a lateral position of a shoulder edge (28) based on the detected road boundary (24), a vehicle position determination unit (29) being arranged to determine a lateral position (y(t)) of the vehicle (1) with respect to said shoulder edge (28).
a radar (22) being arranged to detect a road boundary (24) within a radar zone (26), said radar zone (26) being positioned a head of said vehicle (1), a shoulder edge determination unit (27) arranged to determine a lateral position of a shoulder edge (28) based on the detected road boundary (24), a vehicle position determination unit (29) being arranged to determine a lateral position (y(t)) of the vehicle (1) with respect to said shoulder edge (28).
Description
- The invention relates to a surroundings monitoring system according to the preamble of
claim 1. The invention particularly relates to a surroundings monitoring system which includes a radar arranged to detect a road boundary within a radar zone positioned ahead of the vehicle. - Systems for monitoring the surroundings of a vehicle are well known in the art. Various types of systems for monitoring the surroundings of a vehicle are known. All such systems include sensors retrieving information about the surroundings of a vehicle, processing means arranged to interpret the retrieved information, and intervention means arranged to activate specific functions dependent on how the processing means have interpreted the retrieved information. A specific type of surroundings monitoring system for monitoring the surroundings of a vehicle comprises a camera being mounted on a vehicle for detecting an image of a road in front of said vehicle. Such surroundings monitoring systems are useful for detection of the position of a vehicle within a lane. Knowledge of the position of a vehicle relative to a lane is of importance for active safety systems such as lane departure warning systems, which may alert a driver in the event the vehicle is about to leave a lane under certain conditions.
- Surroundings monitoring systems which are arranged to detect lane markers ahead of the vehicle are well known in the art. Most such systems include a camera detecting an image of the road in front of the vehicle. An image recognition means provided in the system will interpret the image and specifically locate the position of lane markers. Image recognition means, which are well known in the art, identifies the presence and location of predefined objects in an image constituted by an array of pixels. The predefined objects may be constituted by simple objects such as lines, or circle segments or more complex objects such as vehicles. Lane markers are relatively easy to detect, since they are normally painted in a colour which forms a sharp contrast to the road surface, road shoulders and the environment external of the road which encloses the road.
- A suitable image recognition system for determining the position of lane markers on a road surface makes use of a Hough transform to identify the position of the lane markers. The Hough transform, which is extensively described in the literature, is excellent for determination of the presence and position of simple objects such as lines in an image. In a more generalized form the transform can be used to determine the presence and location of circle segments and other geometrical shapes. A description of an image recognition system capable of determining complex patters is present in
US3069654 by Paul Hough . - In order to accurately determine the position of the lane markers it is of importance that the image is of relatively high quality with sharp contrast. As the relative angle between the road surface and an optical axis of the camera decreases with the distance from the vehicle, the contrast between the lane marker and the road surface decreases as well. Lane markers can therefore only be detected within a relatively small vision zone positioned ahead of the vehicle. A normal vision zone extends about 20 - 50 meters from the vehicle.
- As stated above, monitoring systems normally further comprises a lane departure warning unit, which is arranged to generate an intervention in the event said lane warning unit has determined that the vehicle is about to leave the lane. In order for a lane warning system to be commercially acceptable, the lane warning system should only normally be allowed to intervene when a driver is involuntary leaving a lane. In order to separate between cases where the driver is voluntary leaving the lane from cases where the driver is involuntary leaving the lane, several suggestions have been proposed. In some systems it has been proposed that warning signs should only be generated when the driver has not used the direction indicator. Considering the normal behavior of drivers, this has shown not to be sufficient to separate between voluntary and involuntary departure of the lanes.
- In order to further improve the lane departure warning units, it has therefore been proposed to make use of information relating to the vehicles trajectory relative to the lane marker in order to separate between voluntary and involuntary departure from the lane. Frequently such systems base determination of whether to intervene or not on whether a measure of an estimated time to departure is below a threshold value or not. The time to departure will depend on the velocity of the vehicle, the vertical distance to the lane marker and the vehicles trajectory relative to the lane marker lane marker. In order to accurately determine if to intervene or not, the lateral position and lateral velocity of the vehicle relative to the lane marker must be determined with high accuracy.
- In order to with sufficient accuracy determine the position of the vehicle relative to the lane marker and the lateral velocity of the vehicle relative to the lane markers, a vehicle position determination unit being arranged to determine a lateral position of a vehicle with respect to a lane defined by the lane markers detected by said camera must be able to determine the position of the vehicle relative to the lane marker with an accuracy in the cm range. The lateral velocity must also be determined with a high accuracy. In order to establish the lateral velocity, knowledge of the relative distance between the vehicle and lane marker is not sufficient. The curvature of the road must also be accounted for. The requirement of determining the curvature of the road adds to the uncertainty of the estimation, since both the position of lines identified in the image by the image recognition system and the inclination of the lines suffers from estimation errors.
- In order to cope with the problem of determining the lateral position of a vehicle relative to the lane markers and the lateral velocity of the vehicle relative to the lane markers for a vehicle traveling at a relatively high speed by use of cameras having limited resolution, an image recognition system according to the state of the art is used to determine the lateral position of the vehicle relative to the lane marker by use of information from an instant image. An example of a suitable system is sold under the name MobilEye. In
figure 1 , such an image is shown. Thelane markers 2 detected by thecamera 2 within thevision zone 4 is used to determine the relative lateral position y(t0) at the time to. Theimage recognition system 6 detects the presence of the lane markers and establishes the lateral distance y(t0) of the vehicle relative to thelane markers 2 based on the position of the camera relative to the vehicle and the position of the lane markers relative to the camera. The determination of the relative position y(t0) of the vehicle with respect to the road may be determined by the use of known systems including a camera which detects the image of the road and an image recognition system which detects lane markers within the vision zone of the vehicle. By use of for instance a Hough transform, an estimate of the position of the lane markers can be detected with a sufficient degree of accuracy. The estimate of the position of the lane markers will represent a curved orstraight line 8, which is used to determine the lateral position of the vehicle by geometric calculation. Since the resolution of cameras is limited, the exact position and curvature of the road geometry ahead of the vehicle is not determined. The estimated positions of the lane markers are primarily used to determine the current lateral position of the vehicle. The current lateral position of the vehicle in relation to the lane markers may be determined according to the method disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624. - In current surroundings monitoring systems including a road departure warning system a warning signal is generated depending of the vehicles relative position to the lane marker.
- Even though progress have been made in the field of surroundings monitoring system including a departure warning unit, there still is a need to further improve such systems so that they better can separate between involuntary and voluntary departures and thereby to a higher degree alert the driver only when necessary.
- It is an object of the invention to further improve surrounding monitoring system. In particular it is an object of the invention to improve a surroundings monitoring system including a departure warning unit, which is well adapted to normal behavior of drivers such that the systems capacity to separate between involuntary and voluntary departures is improved.
- This object is achieved by a surroundings monitoring system according to the characterizing portion of
claim 1. - A surroundings monitoring system according to the invention includes a radar being arranged to detect a road boundary within a radar zone, said radar zone being positioned a head of said vehicle and preferably extending further from said vehicle than said vision zone. The radar will detect objects positioned along the road boundary, such as guide rails, trees, stones etc. The detected objects will define a road boundary, which forms a transition from a generally flat surface to a border formed by the detected objects.
- Since the detected road boundary does not form a continuous line, but a large set of detected objects forming an uneven interrupted border, it may be necessary to from the detected road boundary form a continuous line that estimates the road boundary.
- The surroundings monitoring system therefore furthermore includes a shoulder edge determination unit arranged to determine a lateral position of a shoulder edge based on the detected road boundary. The shape of the shoulder edge is determined according to the same principles as is used for generally determining a road shape or the shape of the longitudinal extension of lane markers, but the road boundary detected by the radar is used as input data instead or in addition to lane markers detected by a camera. Suitably the process described in A. Polychronopoulos, et. al.; "Integrated object and road border tracking using a 77 GHz automotive radar",; IEE Proc. - Radar Sonar Navig., Vol 151, may be used. The shape of the longitudinal extension may thus performed by adapting an curve to lane markers present in the detection zone of the camera such that the curve follows the lane markers. Once the shape of the shoulder edge is determined an exact lateral position of the shoulder edge is determined. Preferably the lateral position of the shoulder edge is selected such that it forms a tangent to parts of the road boundary within the detection zone being closest to the road centre.
- The surroundings monitoring system according to the invention furthermore includes a road departure warning unit which is arranged to generate an intervention based on relative position of the vehicle to the shoulder edge. In Pohl, J. and Ekmark, j. (2003). Development of a haptic intervention system for unintended lane departure. In Proceedings of the 2003 SAE World Congress, Detroit, MI, USA., a suitable system for handling interventions due to unintentional road departures is described.
- The intervention may be made in the event said road departure warning unit has determined that the vehicle approaches the road boundary and is positioned at a certain gap from the shoulder edge.
- The surroundings monitoring system may furthermore include a warning line generation unit which is arranged to locate a warning line at a gap from the shoulder edge. The size of the gap may be dependent on the width of the road shoulder, which is the distance from the lane markers to the shoulder edge. The size of the gap may furthermore be dependent on the curvature radius of the road. In the event a warning line generation unit is included, the decision to intervene will be based on the relative lateral position of the vehicle with relation to the warning line, which depends on the position of the shoulder edge.
- Preferably the shoulder edge determination unit, and, if included, the warning line determination unit and/or a lane marker position determination unit all use the same estimator for forming the respective shapes of the shoulder edge, warning line, and if included a lane marker position estimate. This may be done in a common road geometry determination unit using the estimating process disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624.
- The timing of the intervention may thus be made dependent on an estimated time for the vehicle to cross the shoulder edge or preferably the warning line or distance to the shoulder edge or preferably warning line or a combination thereof, that is the timing of the intervention will be dependent on both the time for the vehicle to cross the shoulder edge and distance to the shoulder edge or preferably the time for the vehicle to cross the warning line and distance to the warning line.
- By determining whether an intervention should be made or not by the road departure warning unit based on relative position of the vehicle to the shoulder edge being estimated from a detected road boundary instead of using a relative position to a lane marker, the surroundings monitoring system will be better adapted to an accepted behavior of drivers to temporarily use road shoulders. The system will be especially advantageous for curved roads where drivers tend to cut inner curves, where road shoulders exists and the driver have a clear sight of the road ahead of the vehicle. Preferably the decision whether an intervention should be made or not by the road departure warning unit is based on the relative position of the vehicle to a warning line which is positioned at a certain gap G from a shoulder edge
- In a first embodiment of the invention, the surroundings monitoring system includes a road geometry determination unit, which is arranged to determine the road geometry ahead of the vehicle based on the road boundary detected by said radar. The road geometry determination may preferably be performed in the manner disclosed in A. Polychronopoulos, et. al.; "Integrated object and road border tracking using a 77 GHz automotive radar",; IEE Proc. - Radar Sonar Navig., Vol 151, . A road border geometry is determined in a road border assignment process, which estimates the road border geometry from measurement data obtained by the radar. A probalistic and a geometric method are disclosed as alternatives to assign the road borders. Preferably a weighted combination of both methods is used to obtain an accurate estimate of the road geometry.
- In a second embodiment said surroundings monitoring system further comprises a camera being mounted on a vehicle for detecting an image of a road within a vision zone and an image recognition system being arranged to detect lane markers within said vision zone. The detection of lane markers can be used to improve the estimate of the road geometry by disregarding measurement data that are too distant from the lane markers.
- In a third embodiment said surroundings monitoring system further comprises a camera being mounted on a vehicle for detecting an image of a road within a vision zone and an image recognition system being arranged to detect lane markers within said vision zone. In this embodiment, the road geometry may be completely determined from the position of the lane markers detected by the camera. The detection of lane markers can be used to improve the estimate of the by disregarding measurement data that are too distant from the lane markers. The road border geometry in the third embodiment is determined in a road border assignment process, which estimates the road border geometry from measurement data obtained by the radar.
- The detected position of the lane markers can furthermore be used to determine the presence and extension of a road shoulder. In this event, the surroundings monitoring system further comprises a road shoulder assessment unit, which is arranged to assess the lateral extension of a road shoulder as the difference in lateral position of the shoulder edge estimated from the road boundary detected by the radar and a lane marker adjacent to said shoulder, which lane marker being detected by the camera.
- The road departure warning unit may be furthermore arranged to intervene in dependence of the lateral extension of the road shoulder, i. e. on the difference in lateral position of the road boundary detected by said radar and a lane marker adjacent to said shoulder, which lane marker being detected by said camera. According to this embodiment, it is possible to adjust the timing of intervention in dependence of the existence and width of road shoulders. If large road shoulders exist, the vehicle may be allowed to pass the lane marker with a certain distance before an intervention is made, on the other hand if no road shoulder exists, but the lane marker coincides with a guard rail, a warning can be issued well before the lane marker.
- The road departure warning system may furthermore be arranged to intervene in dependence of the curvature of the road. The dependence may preferably be of such a nature that such that the distance from a centre line of the lane to the point of intervention is increased for curves being convex toward the centre line and the distance from a centre line of the lane to the point of interception is decreased for curves being concave toward the centre line., that is the point of intervention is moved to the right at curves turning to the left and vice versa.
- In the type of image recognition system referred to in the paragraph starting on
page 4, line 5, where the relative lateral position is determined from the instant image, and where the relative lateral velocity is determined from estimating derivatives of the relative lateral position, the system is unaware of which type of curve is ahead of the vehicle. Such system may only determine what type of curve the vehicle is currently traveling in by use of a yaw rate sensor and the determined distance to the lane markers. In the event a vehicle is traveling in an S curve, where the centre of the curve moves from one side of the road to another side of the road, the current systems are unable to adapt the lane departure warning systems so as to allow certain type of curve cutting associated with S curves. With the new type of system, judgment of which type of curve is ahead of the vehicle is allowed for, which improves the capability of generating a lane departure warning system, which is more accurate in respect of alerting the driver only in situations when the driver unintentionally leaves the lane. - Generally the presence of a radar determining the road boundary ahead of the vehicle is beneficial for adapting lane warning departure systems to the actual road geometry. One advantage is that the existence and size of road shoulder may be assessed from estimating the distance to a road boundary detected by the radar to a lane marker detected by the camera. The lane warning system can thereafter be adapted to intervene in dependence on the existence and size of the road shoulder. Another advantage is that a lane warning system can be adapted to the road geometry such that the vehicle will be allowed to cut certain types of curves, while being prevented to cut other type of curves.
- An embodiment of the invention will be described below, with references to the appended drawings where:
- Fig. 1
- shows a state of the art system for determining the lateral position of the vehicle with respect to lane markers,
- Fig. 2
- shows a schematic drawing of a surroundings monitoring system determining the relative lateral position of a vehicle on a road with lane markers,
- Fig. 3
- shows a block diagram of a surroundings monitoring system according to the invention,
- Fig. 4
- shows a representation of a road boundary R(θ(x,y(t))) for a vehicle,
- Fig 5a
- shows a road scenario for a road curved towards the left in the drawing, and
- Fig 5b
- shows a road scenario for a road curved towards the right in the drawing, and
- Fig. 5c
- is a schematic drawing showing the relations between the road boundary, shoulder edge, warning line, and lane marker.
- An embodiment will be described below with references to
figures 2 and3 .Figure 2 shows a schematic drawing of asurroundings monitoring system 10 determining the relative lateral position y(t) of avehicle 1 on aroad 12 withlane markers 14.Figure 3 shows a block diagram of asurroundings monitoring system 10 according to the invention. - The
surroundings monitoring system 10 includes aradar 22 being arranged to detect theroad boundary 24 within aradar zone 26, said radar zone being positioned a head of said vehicle. - The radar is thus necessarily directed in the forward direction of the vehicle. The
road boundary 24 detected by theradar 22 is formed by a reflected radar beam. All objects that vertically extend in the height direction along the road will be detected. Examples of objects which generate a sharp detectable reflection are parked cars, guard rails, solid objects such as rocks, houses, trees, animals and human beings. The lateral distance to the road boundary may be directly used as input to a road departure warning unit. The input data from the radar may however be first treated by a roadgeometry determination unit 23. The road geometry determination unit may retrieve its input data from acamera 16.. A suitable road boundary determination unit is sold under the trade name Mobil Eye. The road geometry determination unit may also be arranged as disclosed in A. Polychronopoulos, et. al.; "Integrated object and road border tracking using a 77 GHz automotive radar",; IEE Proc. - Radar Sonar Navig., Vol 151, . Preferably the road border geometry is determined in combined probalistic and geometric method disclosed in the document.
Aroad boundary 24 will thus be determined by detection of the reflected radar signal. Ashoulder edge 28 will be determined from the detected road boundary in a shoulderedge determination unit 27. The road shoulder determination unit is arranged to determine the shape and lateral position of a shoulder edge based on the detected road boundary by forming an essentially continuous shoulder edge. The shoulderedge determination unit 27 is preferably included in a roadgeometry determination unit 26. The shoulder edge representation may be a curved or straight line. Since the radar with good accuracy may detect objects along the road which extend in the vertical direction, and which objects are positioned relatively far from the vehicle, aroad boundary 24 and thus ashoulder edge 28 can be determined ahead of the vehicle at distances exceeding 150m. Hence, the use of aradar 22 to determine the road boundary ensures that theshoulder edge 28 can be determined at a distance extending further from said vehicle than an actual vision zone, in which lane markers are detected. A particular advantage in using a radar for detecting a road boundary, is that vertically extending objects can be detected with a good accuracy at a large distance, while the lane markers are difficult to detect at a large distance since the relative angle between the cameras optical axis and the surface of the lane marker becomes very small for distant lane markers. It is therefore very difficult to detect lane markers, if at all possible, at a distance exceeding 20 - 50 m. - The representation of the
road boundary 24 is stored in a memory suitably as a vector R(θ((x,y)(t))) for angles θ relative to the heading direction θ of the vehicle positioned at the location (x,y) the time t. Infigure 4 , a representation of a road boundary R(θ(x,y(t))) for a vehicle is illustrated. - A vehicle
position determination unit 29 included in the vehicle surroundings monitoring system is arranged to determine a lateral position y(t) of a vehicle on aroad 12 with respect to ashoulder edge 28, or preferably a warning line 42 as will be explained below. The vehicle position determination unit may preferably include acamera 16 as an input device for deriving the actual lateral position of the vehicle. In this embodiment of thesurroundings monitoring system 10, acamera 16 is mounted on a vehicle for detecting an image of a road within avision zone 18. Animage recognition system 20 is arranged to detectlane markers 14 in the image captured by the camera. Thecamera 16 and theimage recognition system 20 may be any known arrangement which is able to capture an image of lane markers and determine the position of the vehicle relative to lane markers. Such arrangements are well known to a person skilled in the art. A suitable system is sold under the tradename MobilEye. Thevision zone 18 preferably extends ahead of the vehicle as shown infigure 2 . Alternatively, the camera can be directed in the rearward direction. This is possible since the main task of the image recognition system is to determine the current lateral position of the vehicle, which may be determined either by image recognition detecting the lane markers in the front or rear direction of the vehicle. The radar zone preferably extends further from the vehicle than the vision zone. - As the vehicle travels along the road, the camera will detect the
lane markers 14 positioned along the road. A roadshoulder assessment unit 30 is included in the surroundings monitoring system, which roadshoulder assessment unit 30 is arranged to assess the lateral extension D of aroad shoulder 31 as the difference in lateral position of the shoulder edge estimated fromroad boundary 24 detected by saidradar 22 and alane marker 14 adjacent to saidshoulder 31, whichlane marker 14 being detected by saidcamera 16. The road shoulder is thus a relatively plane surface limited on one side of alane marker 14 detected by a camera, and on the other side of a shoulder edge preferably forming a tangent to vertically extending objects, detected by the radar, and which are defining aroad boundary 24. - In a preferred embodiment of the invention the surroundings monitoring system includes a warning
line generation unit 33 which is arranged to locate a warning line 42R, 42L at a gap from theshoulder edge 28. The size of the gap G may be dependent on the width D of the road shoulder, which is the distance from thelane markers 14 to theshoulder edge 28. The size of the gap G may furthermore be dependent on the curvature radius R of the road. In the event a warning line generation unit is included, the decision to intervene will be based on the relative lateral position of thevehicle 1 with relation to the warning line 42R, 42L, which depends on the position of the shoulder edge. If a warning line generation unit is not included the decision to intervene will be based on the relative lateral position of thevehicle 1 with relation to the position of the shoulder edge directly. - Preferably the shoulder
edge determination unit 27, and, if included, the warningline determination unit 33 and/or a lane marker position determination unit all use the same estimator for forming the respective shapes of the shoulder edge, warning line, and if included an lane marker position estimate. This may be done in a common roadgeometry determination unit 23 using the estimating process disclosed in Andreas Eidehall and Fredrik Gustafsson, "Combined road prediction and target tracking in collision avoidance" in Proceedings of IEEE Intelligent Vehicles Symposium 2004, pages 619-624. - The equations governing the Kalman process which determines the road geometry and hence the geometry of the respective lines that is the shoulder edge, the warning line, a continuous line representing the lane markers is as follows:
Where,
y_off = host vehicle position relative to the center of the lane
Psi_rel = host vehicle heading angle relative to the lane
c0 = road curvature = 1/road radius
c1 = distance derivative of c0
v = host vehicle velocity
d= time derivative - The camera will be used to generate indata to estimate y_off och Psi_rel. c0 is measured from input data from the camera and radar as is disclosed in the reference.
Based on these equations the Kalman filter is created according to general practice. - The respective lines will then be a set of parallel curved lines off set a certain distance from each other.
- The surroundings monitoring system furthermore comprises a road
departure warning unit 32, which is arranged to generate an intervention in the event said road departure warning unit has determined that the vehicle is about to leave the road. The intervention may be provided in different forms, such as generation of a warning sign, activation of brakes, active steering of the vehicle, etc..The most common lane warning units will provide an intervention in the form of a warning alerting the driver. The warning may be visual or audial. - Generally a road
departure warning unit 32 is arranged to determine that the vehicle is about to leave the road based on relative position of the vehicle to the warning line, which is positioned at a certain gap G from theshoulder edge 28. The size of the gap may be dependent on the lateral extension D of the road shoulder and/or of the curvature radius R of the road. The intervention will be made when the vehicle approaches the warning line and is positioned at a certain distance S from the warning line. The distance S may be determined in different manners. In one embodiment said road departure warning 32 unit is arranged to determine that the vehicle is about to leave the road, when an estimated time to departure T, that is the time T needed for the vehicle to cross the warning line, is below a threshold value, S: T≤ Threshold. In another embodiment roaddeparture warning unit 32 is arranged to determine that the vehicle is about to leave the road, when an estimated distance to departure L, that is the distance required for the vehicle to travel in order to cross the warning line, is below a threshold value, S: L≤ Threshold. In a third embodiment the roaddeparture warning unit 32 is arranged to determine that the vehicle is about to leave the road, that is to pass the warning line, when a value of a function f of an estimated distance to departure L and an estimated time to departure T is below a threshold value, S: f(T,L) ≤ Threshold. - The warning line positioned at a gap G from the road shoulder will form a continuous warning line running along the extension of the
road boundary 24. In the event lane markers are detected by theimage recognition system 20, the gap size can be made dependent of the position of the lane markers. - In an embodiment of the invention, the road
shoulder assessment unit 30 is arranged to assess the lateral extension D of aroad shoulder 31 as the difference in lateral position of theshoulder edge 28 determined from theroad boundary 24 detected by saidradar 22 and alane marker 14 adjacent to saidshoulder 31. By assessing that the lateral extension D is within a range, such that Dmin < D < Dmax, where Dmax is smaller than a normal lane width, it is possible to exclude misinterpretation of a neighboring lane as a road shoulder, which may be used by the vehicle. This criterion can also be used to prevent that a neighboring lane traveling in the opposite direction is interpreted as road shoulder. Different rules may be applied for detected flat ground on the left and right side of the vehicle. For example only a road boundary to the right of the vehicle may be used when defining a road shoulder. This rule may be applicable in countries with right hand traffic. Consequently a rule that only a road boundary to the left of the vehicle may be used when defining a road shoulder may be used in countries with left hand traffic. - The lateral distance to an actual lane marker can be positive, that is an intervention is generated before the vehicle passes the actual lane marker, or negative, that is an intervention is made after the vehicle has passed the actual lane marker.
- In
figure 5a road curved towards the left in the drawing is shown. The road is provided withlane markers lanes vehicle 1 is travelling in theright lane 36 in the upwards direction in the figure. Warning lines 40L, 40R forms the fixed reference in lateral position for a road departure warning unit. The road departure warning unit compares the vehicles relative position to the warning lines in order to determine whether an intervention should be made or not. In one embodiment the warning lines marks the position at which in intervention will be made by a road departure warning unit in the event the vehicle will pass the warning lines 40L, 40R. In another embodiment the timing of the intervention can be based on an estimated time to departure T when the vehicle is assumed to cross the warning line. A warning line exists for the left and right side of the lane in which the vehicle travels. The warning lines are positioned at gap G from the shoulder edge estimated from theroad boundary 24 detected by the radar. - The size of the gap is determined in a gap
size determination unit 44, which calculates the position relative to a road boundary. Input to the gapsize determination unit 44 is information from the roadgeometry determination unit 26, theshoulder assessment unit 30, avehicle speed sensor 46 and a vehicleyaw rate sensor 48. - The gap size may be dependent on the following input variables alone or taken in combination:
- the curvature radius R determined by the road
geometry determination unit 26, - the presence and lateral extension D of road shoulders determined by road
shoulder assessment unit 30, - The gap size can be expressed as G= T(D,R).
- The timing of the intervention may be based on:
- a time to departure T, determined by the yaw rate sensor and a vehicle speed sensor and an estimated road geometry, and
- the distance to departure L.
- The warning line 40L is positioned at a distance Δ in the lateral direction to the
lane markers 14R. The magnitude of the distance Δ may be positive, that is the warning line will be positioned in the lane, or negative, that is the warning line will be positioned outside of the lane. Infigure 5a , the warning line 40R is positioned in thelane 36, which indicates that the distance Δ has a positive value, while the warning line 40L is positioned outside of the lane 38, which indicates that the distance Δ has a negative value. If the distance Δ has a negative value an intervention will be made before the vehicle exits the lane, while if the distance Δ has a positive value an intervention will be made after the vehicle exits the lane. - As can be seen in
figure 5a the lateral position of the warning line 40L, 40R with respect to theroad boundary 24 is dependent on the curvature of the road such that a minimal lateral distance from a centre of curvature to said warning line is decreasing with a decreased radius of curvature of the road. This allows a vehicle to cut an inner curve with a certain distance, while preventing the vehicle to ride to close to the lane markers in an outer curve. Naturally, it is possible to set different rules for the left and right lane markers in the event, for instance it should not be allowed to pass a lane marker positioned adjacent to an other lane, while allowing the vehicle to pass lane markers which are facing a road boundary, especially in the event a rod shoulder is positioned on the other side of the lane marker. - In
figure 5b road curved towards the right in the drawing is shown. The warning lines 42L, 42R marks the position at which in intervention will be made by a road departure warning unit in the event the vehicle will pass the warning lines 42L, 42R. - The warning lines are positioned at gap G from the shoulder edge estimated from the
road boundary 24 detected by the radar. - The size of the gap is determined in a gap
size determination unit 44, which calculates the position relative to a road boundary. Input to the gapsize determination unit 44 is information from the roadgeometry determination unit 26, theshoulder assessment unit 30, avehicle speed sensor 46 and a vehicleyaw rate sensor 48. - The gap size may be dependent on the curvature radius R determined by the road
geometry determination unit 26, the presence and lateral extension D of road shoulders determined by roadshoulder assessment unit 30. The gap size can be expressed as G= T(D, R). - It can here be noted that the warning lines 42L, 42R are shifted towards the right in stead of to the left as in
figure 5a . This indicated that the warning lines are normally shifted in the same direction as the curvature of the road; that is to the left in a lane curved toward the left and to the right in a lane curved to the right. The magnitude of the shift; that is the size of the gap is dependent on the curvature radius such that a smaller curvature radius R will generate a larger gap.
Figure 5c is a schematic drawing showing the relations between theroad boundary 24,shoulder edge 28, warning line 42R, andlane marker 14.
Figure 5c shows the geometry of the road including discontinuous set of radar reflections forming aroad boundary 24. Acontinuous shoulder edge 28 is determined from the radar reflections forming theroad boundary 24 in a shoulderedge determination unit 27, which may be included in the roadgeometry determination unit 26. A warning line 42R determined in a warning line generation unit is located at a gap G from theshoulder edge 28. Alane marker 14 is positioned to the left in the figure. A road shoulder extending between thelane marker 14 andshoulder edge 28 has a lateral extension D. - A decision to intervene is based on the relative lateral position S of a
vehicle 1 to the warning line 42R. The warning line 42R is positioned at a distance Δ to the lane markers in the lateral direction. The position of the warning line 42R may be on either side of thelane marker 14 depending on the geometry of the road and/or location of the shoulder edge relative to the lane marker. - The dependence of the curvature radius of the road for the gap G may be expressed as G=max(0,k/abs(R)), where k is set between 50 and 500, preferably between 80 and 150, that is k is preferably set to approximately 100.
Claims (10)
1. A surroundings monitoring system (10) for monitoring the surroundings (11) of a vehicle (1) comprising:
a radar (22) being arranged to detect a road boundary (24) within a radar zone (26), said radar zone (26) being positioned a head of said vehicle (1), a shoulder edge determination unit (27) arranged to determine a lateral position of a shoulder edge (28) based on the detected road boundary (24), a vehicle position determination unit (29) being arranged to determine a lateral position (y(t)) of the vehicle (1) with respect to said shoulder edge (28), characterized in that said surroundings monitoring system (10) further comprises a road departure warning unit (32), which is arranged to generate an intervention based on relative lateral position of the vehicle (1) to the shoulder edge (28).
2. A surroundings monitoring system according to claim 1, characterized in that the system furthermore includes a warning line generation unit (33) which is arranged to locate a warning line (42R, 42L) at a gap G from the shoulder edge (28) and that the road departure warning unit (32), is arranged to generate an intervention based on relative position (S) of the vehicle (1) to the warning line (42R, 42L).
3. A surroundings monitoring system according to claim 2, characterized in that the magnitude of the relative position (S) of the vehicle (1) to the warning line (42R, 42L) at which an interventions is made (S=S(T)) is dependent on an estimated time (T) for the vehicle to cross the warning line (42R, 42L)
4. A surroundings monitoring system according to claim 2 or 3, characterized in that the magnitude of the relative position (S) of the vehicle (1) to the warning line (1) at which an interventions is made (S=S(T); S=S(T,L) is dependent on distance L to the warning line (42R, 42L).
4. A surroundings monitoring system according to any of claims 1 - 3, characterized in that said surroundings monitoring system (10) further comprises a road geometry determination unit (23), which is arranged to determine the road geometry ahead of the vehicle (1) based on the road boundary (24) detected by said radar (22).
5. A surroundings monitoring system according to any of claims 1 - 4, characterized in that said surroundings monitoring system (10) further comprises a camera (16) being mounted on the vehicle (1) for detecting an image of a road within a vision zone (18) and an image recognition system (20) being arranged to detect lane markers (2) within said vision zone (18);
6. A surroundings monitoring system according claim 5, characterized in that said surroundings monitoring system (10) further comprises a road shoulder assessment unit (30), which is arranged to assess the lateral extension (D) of a road shoulder (31) as the difference in lateral position of the road boundary (24) detected by said radar (26) and a lane marker (14) adjacent to said road boundary (24), which lane marker (14) being detected by said camera (16).
7. A surroundings monitoring system according to claim 6, characterized in that the extension of said gap (G) is dependent on the extension of the road shoulder (D).
8. A surroundings monitoring system according to any of claims 1 - 7, characterized in that the extension of said gap (G) is dependent on the curvature (C) of the road.
9. A surroundings monitoring system according to claim 8, characterized in that said gap (G) will form a warning line (40R) running along the road boundary (24) and that a lateral position of the warning line with respect to the shoulder edge (28) is dependent on the curvature (R) of the road such that a minimal lateral distance from a centre of curvature to said warning line (40R) is decreasing with a decreased radius of curvature of the road.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP07111671A EP2012211A1 (en) | 2007-07-03 | 2007-07-03 | A system for monitoring the surroundings of a vehicle |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP07111671A EP2012211A1 (en) | 2007-07-03 | 2007-07-03 | A system for monitoring the surroundings of a vehicle |
Publications (1)
Publication Number | Publication Date |
---|---|
EP2012211A1 true EP2012211A1 (en) | 2009-01-07 |
Family
ID=38727347
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07111671A Withdrawn EP2012211A1 (en) | 2007-07-03 | 2007-07-03 | A system for monitoring the surroundings of a vehicle |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP2012211A1 (en) |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009103692A1 (en) * | 2008-02-20 | 2009-08-27 | Continental Teves Ag & Co. Ohg | Method and assistance system for detecting objects in the surrounding area of a vehicle |
WO2011020715A3 (en) * | 2009-08-21 | 2011-04-21 | Robert Bosch Gmbh | Method and control device for robustly detecting a lane change of a vehicle |
DE102010032063A1 (en) | 2010-06-09 | 2011-05-12 | Daimler Ag | Method for determining environment of vehicle, involves recognizing radar data from roadway surface elevated object by radar sensors and recognizing camera data from pavement marking by camera |
US8410919B2 (en) | 2009-05-22 | 2013-04-02 | Toyota Jidosha Kabushiki Kaisha | Driving support apparatus |
JP2014109945A (en) * | 2012-12-03 | 2014-06-12 | Fuji Heavy Ind Ltd | Vehicle driving support control device |
EP2842117A2 (en) * | 2012-04-26 | 2015-03-04 | Continental Teves AG & Co. oHG | Method for displaying vehicle surroundings |
JP2015523621A (en) * | 2012-05-08 | 2015-08-13 | オートリブ ディベロップメント エービー | Lane mark crossing warning system |
JP2017016360A (en) * | 2015-06-30 | 2017-01-19 | 富士通テン株式会社 | Information presentation device, on-vehicle device, information presentation system and information presentation method |
EP3141926A1 (en) * | 2015-09-10 | 2017-03-15 | Continental Automotive GmbH | Automated detection of hazardous drifting vehicles by vehicle sensors |
EP3217374A1 (en) | 2016-03-10 | 2017-09-13 | Volvo Car Corporation | Method and system for estimating a boundary of a road technical field |
WO2017163856A1 (en) * | 2016-03-22 | 2017-09-28 | 株式会社デンソー | Traffic lane deviation avoidance system |
US9829575B2 (en) | 2012-07-30 | 2017-11-28 | Conti Temic Microelectronic Gmbh | Method for representing a vehicle environment with position points |
WO2017202570A1 (en) * | 2016-05-27 | 2017-11-30 | Volkswagen Aktiengesellschaft | Method, device and computer readable storage medium with instructions for determining the lateral position of a vehicle relative to the lanes of a carriageway |
CN109189079A (en) * | 2018-11-05 | 2019-01-11 | 南京理工大学 | Mobile Robotics Navigation control method based on GPS positioning |
WO2019054183A1 (en) * | 2017-09-12 | 2019-03-21 | 愛知製鋼株式会社 | Position capturing system and position capturing method |
WO2019201650A1 (en) * | 2018-04-20 | 2019-10-24 | Robert Bosch Gmbh | Method and device for determining a highly precise position of a vehicle |
DE102009050941B4 (en) | 2009-10-28 | 2020-06-04 | Man Truck & Bus Se | Driver assistance system for a commercial vehicle |
CN111619587A (en) * | 2019-02-28 | 2020-09-04 | 本田技研工业株式会社 | Lane departure prevention assist system for vehicle |
WO2021001018A1 (en) * | 2019-07-01 | 2021-01-07 | Zenuity Ab | Sourced lateral offset for adas or ad features |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE19944542A1 (en) * | 1999-09-17 | 2001-04-12 | Daimler Chrysler Ag | Procedure for a vehicle, fitted with forwards facing radar, based determination of a vehicle trajectory in which the radar identifies road edge markers and uses these to determine the path to be followed |
EP1320082A2 (en) * | 2001-12-14 | 2003-06-18 | Audi Ag | Vehicle with an acoustical and/or optical distance warning system |
DE102005025387A1 (en) * | 2004-09-30 | 2006-05-04 | Daimlerchrysler Ag | Method and device for driver's warning or to actively intervene in the driving dynamics, if threatening to leave the lane |
DE102004057296A1 (en) * | 2004-11-26 | 2006-06-08 | Daimlerchrysler Ag | Lane departure warning with distinction between lane markings and the construction boundary of the lane |
DE102005003178A1 (en) * | 2005-01-19 | 2006-07-27 | Volkswagen Ag | Transverse guidance assistance device for vehicle, has arithmetic and logic unit to determine reference lane that is determined from free lane, which is detected by sensor, and safety margin, where free lane is reduced by margin |
DE102005039167A1 (en) * | 2005-08-17 | 2007-02-22 | Daimlerchrysler Ag | Lane departure warning driver assistance device for vehicle, has evaluating unit evaluating present surrounding situation by comparing location of boundary with markings or edges, and adjusting unit adjusting warning output to situation |
DE102005039895A1 (en) * | 2005-08-23 | 2007-03-01 | Siemens Ag | Lane Departure Warning and / or Lane Keeping System |
-
2007
- 2007-07-03 EP EP07111671A patent/EP2012211A1/en not_active Withdrawn
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE19944542A1 (en) * | 1999-09-17 | 2001-04-12 | Daimler Chrysler Ag | Procedure for a vehicle, fitted with forwards facing radar, based determination of a vehicle trajectory in which the radar identifies road edge markers and uses these to determine the path to be followed |
EP1320082A2 (en) * | 2001-12-14 | 2003-06-18 | Audi Ag | Vehicle with an acoustical and/or optical distance warning system |
DE102005025387A1 (en) * | 2004-09-30 | 2006-05-04 | Daimlerchrysler Ag | Method and device for driver's warning or to actively intervene in the driving dynamics, if threatening to leave the lane |
DE102004057296A1 (en) * | 2004-11-26 | 2006-06-08 | Daimlerchrysler Ag | Lane departure warning with distinction between lane markings and the construction boundary of the lane |
DE102005003178A1 (en) * | 2005-01-19 | 2006-07-27 | Volkswagen Ag | Transverse guidance assistance device for vehicle, has arithmetic and logic unit to determine reference lane that is determined from free lane, which is detected by sensor, and safety margin, where free lane is reduced by margin |
DE102005039167A1 (en) * | 2005-08-17 | 2007-02-22 | Daimlerchrysler Ag | Lane departure warning driver assistance device for vehicle, has evaluating unit evaluating present surrounding situation by comparing location of boundary with markings or edges, and adjusting unit adjusting warning output to situation |
DE102005039895A1 (en) * | 2005-08-23 | 2007-03-01 | Siemens Ag | Lane Departure Warning and / or Lane Keeping System |
Non-Patent Citations (2)
Title |
---|
EIDEHALL ET AL: "Joint road geometry estimation and vehicle tracking", 1 October 2007, CONTROL ENGINEERING PRACTICE, PERGAMON PRESS, OXFORD, GB, PAGE(S) 1484-1494, ISSN: 0967-0661, XP022282170 * |
POLYCHRONOPOULOS A ET AL: "Integrated object and road border tracking using 77 GHz automotive radars", IEE PROCEEDINGS: RADAR, SONAR & NAVIGATION, INSTITUTION OF ELECTRICAL ENGINEERS, GB, vol. 151, no. 6, 10 December 2004 (2004-12-10), pages 375 - 381, XP006023106, ISSN: 1350-2395 * |
Cited By (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8457359B2 (en) | 2008-02-20 | 2013-06-04 | Continental Teves Ag & Co. Ohg | Method and assistance system for detecting objects in the surrounding area of a vehicle |
WO2009103692A1 (en) * | 2008-02-20 | 2009-08-27 | Continental Teves Ag & Co. Ohg | Method and assistance system for detecting objects in the surrounding area of a vehicle |
US8410919B2 (en) | 2009-05-22 | 2013-04-02 | Toyota Jidosha Kabushiki Kaisha | Driving support apparatus |
DE102009046699B4 (en) * | 2009-05-22 | 2013-10-10 | Toyota Jidosha Kabushiki Kaisha | Driving support device |
US9592829B2 (en) | 2009-08-21 | 2017-03-14 | Robert Bosch Gmbh | Method and control unit for robustly detecting a lane change of a vehicle |
WO2011020715A3 (en) * | 2009-08-21 | 2011-04-21 | Robert Bosch Gmbh | Method and control device for robustly detecting a lane change of a vehicle |
CN102481952A (en) * | 2009-08-21 | 2012-05-30 | 罗伯特·博世有限公司 | Method and control device for robustly detecting a lane change of a vehicle |
DE102009050941B4 (en) | 2009-10-28 | 2020-06-04 | Man Truck & Bus Se | Driver assistance system for a commercial vehicle |
DE102010032063A1 (en) | 2010-06-09 | 2011-05-12 | Daimler Ag | Method for determining environment of vehicle, involves recognizing radar data from roadway surface elevated object by radar sensors and recognizing camera data from pavement marking by camera |
EP2842117B1 (en) * | 2012-04-26 | 2021-08-25 | Continental Teves AG & Co. OHG | Method for displaying vehicle surroundings |
US20150105937A1 (en) * | 2012-04-26 | 2015-04-16 | Continental Teves Ag & Co. Ohg | Method for Representing Vehicle Surroundings |
US9896092B2 (en) * | 2012-04-26 | 2018-02-20 | Continental Teves Ag & Co. Ohg | Method for representing vehicle surroundings |
EP2842117A2 (en) * | 2012-04-26 | 2015-03-04 | Continental Teves AG & Co. oHG | Method for displaying vehicle surroundings |
JP2015523621A (en) * | 2012-05-08 | 2015-08-13 | オートリブ ディベロップメント エービー | Lane mark crossing warning system |
US9829575B2 (en) | 2012-07-30 | 2017-11-28 | Conti Temic Microelectronic Gmbh | Method for representing a vehicle environment with position points |
JP2014109945A (en) * | 2012-12-03 | 2014-06-12 | Fuji Heavy Ind Ltd | Vehicle driving support control device |
JP2017016360A (en) * | 2015-06-30 | 2017-01-19 | 富士通テン株式会社 | Information presentation device, on-vehicle device, information presentation system and information presentation method |
EP3141926A1 (en) * | 2015-09-10 | 2017-03-15 | Continental Automotive GmbH | Automated detection of hazardous drifting vehicles by vehicle sensors |
WO2017042089A1 (en) * | 2015-09-10 | 2017-03-16 | Continental Automotive Gmbh | Automated detection of hazardous drifting vehicles by vehicle sensors |
US10864911B2 (en) | 2015-09-10 | 2020-12-15 | Continental Automotive Gmbh | Automated detection of hazardous drifting vehicles by vehicle sensors |
US10480948B2 (en) | 2016-03-10 | 2019-11-19 | Volvo Car Corporation | Method and system for estimating a boundary of a road |
CN107176167A (en) * | 2016-03-10 | 2017-09-19 | 沃尔沃汽车公司 | Method and system for estimating road boundary |
EP3217374A1 (en) | 2016-03-10 | 2017-09-13 | Volvo Car Corporation | Method and system for estimating a boundary of a road technical field |
CN108885838A (en) * | 2016-03-22 | 2018-11-23 | 株式会社电装 | Lane is detached from and avoids system |
US11254308B2 (en) * | 2016-03-22 | 2022-02-22 | Denso Corporation | Lane departure avoidance system |
JP2017174017A (en) * | 2016-03-22 | 2017-09-28 | 株式会社デンソー | Lane deviation avoidance system |
WO2017163856A1 (en) * | 2016-03-22 | 2017-09-28 | 株式会社デンソー | Traffic lane deviation avoidance system |
WO2017202570A1 (en) * | 2016-05-27 | 2017-11-30 | Volkswagen Aktiengesellschaft | Method, device and computer readable storage medium with instructions for determining the lateral position of a vehicle relative to the lanes of a carriageway |
US10943130B2 (en) | 2016-05-27 | 2021-03-09 | Volkswagen Aktiengesellschaft | Method, device and computer readable storage medium with instructions for determining the lateral position of a transportation vehicle relative to the lanes of a carriageway |
JP2019049500A (en) * | 2017-09-12 | 2019-03-28 | 愛知製鋼株式会社 | Position capturing system and position capturing method |
WO2019054183A1 (en) * | 2017-09-12 | 2019-03-21 | 愛知製鋼株式会社 | Position capturing system and position capturing method |
US11287263B2 (en) | 2017-09-12 | 2022-03-29 | Aichi Steel Corporation | Position capturing system and position capturing method |
WO2019201650A1 (en) * | 2018-04-20 | 2019-10-24 | Robert Bosch Gmbh | Method and device for determining a highly precise position of a vehicle |
CN109189079A (en) * | 2018-11-05 | 2019-01-11 | 南京理工大学 | Mobile Robotics Navigation control method based on GPS positioning |
CN111619587A (en) * | 2019-02-28 | 2020-09-04 | 本田技研工业株式会社 | Lane departure prevention assist system for vehicle |
CN111619587B (en) * | 2019-02-28 | 2023-05-26 | 本田技研工业株式会社 | Lane departure prevention assist system for vehicle |
WO2021001018A1 (en) * | 2019-07-01 | 2021-01-07 | Zenuity Ab | Sourced lateral offset for adas or ad features |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2012211A1 (en) | A system for monitoring the surroundings of a vehicle | |
CN110077399B (en) | Vehicle anti-collision method based on road marking and wheel detection fusion | |
KR101996419B1 (en) | Sensor integration based pedestrian detection and pedestrian collision prevention apparatus and method | |
US10864911B2 (en) | Automated detection of hazardous drifting vehicles by vehicle sensors | |
KR101996418B1 (en) | Sensor integration based pedestrian detection and pedestrian collision prevention apparatus and method | |
CN110065494B (en) | Vehicle anti-collision method based on wheel detection | |
CN109254289B (en) | Detection method and detection equipment for road guardrail | |
US7561032B2 (en) | Selectable lane-departure warning system and method | |
EP1947475B1 (en) | Object detection device | |
KR101075615B1 (en) | Apparatus and method for generating a auxiliary information of moving vehicles for driver | |
EP1304264B1 (en) | A 360 degree vision system for a vehicle | |
CN104554272B (en) | The path planning of avoidance steering operation when target vehicle and surrounding objects be present | |
US8630793B2 (en) | Vehicle controller | |
US9132837B2 (en) | Method and device for estimating the number of lanes and/or the lane width on a roadway | |
JP5410730B2 (en) | Automobile external recognition device | |
CN110865374A (en) | Positioning system | |
JP7279053B2 (en) | System and method for detecting collision risk between a motor vehicle and a secondary object in the driving lane next to the vehicle when changing lanes | |
CN103718061A (en) | Improved driver assistance systems using radar and video | |
CN105922991A (en) | Lane departure early warning method and system based on generation of virtual lane lines | |
JP2023010800A (en) | Display device | |
US20050278112A1 (en) | Process for predicting the course of a lane of a vehicle | |
JP2009086788A (en) | Vehicle surrounding monitoring device | |
JP2006155615A (en) | Lane deviation warning system by discrimination of driveway edge mark and structural boarder of driveway edge | |
JP2004531424A (en) | Sensing device for cars | |
EP3190021A1 (en) | System and method for collision warning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK RS |
|
AKX | Designation fees paid | ||
REG | Reference to a national code |
Ref country code: DE Ref legal event code: 8566 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20090708 |