CN110770744B

CN110770744B - Apparatus and method for determining a property of a surface in the surroundings of a vehicle

Info

Publication number: CN110770744B
Application number: CN201880043557.XA
Authority: CN
Inventors: D.M.M.福克; M-M.梅内克; F.瓦尔内克
Original assignee: Volkswagen AG
Current assignee: Volkswagen AG
Priority date: 2017-06-29
Filing date: 2018-06-28
Publication date: 2024-02-02
Anticipated expiration: 2038-06-28
Also published as: DE102017114571A1; US20200126247A1; CN110770744A; US11386568B2; EP3646237A1; WO2019002482A1

Abstract

The invention relates to a method for determining properties of a surface in the surroundings of a vehicle (1), wherein: three-dimensional surface coordinates of the surface are generated by means of sensor means (7, 8). The method according to the invention is characterized in that an approximation of the curved run of the surface in at least one direction is obtained on the basis of the surface coordinates and in that the classification of the surface coordinates for marking the properties of the surface is performed according to the curved run and/or the vertical spacing of the approximation of the curved run from the three-dimensional surface coordinates. The invention also relates to a device for carrying out the method.

Description

Apparatus and method for determining a property of a surface in the surroundings of a vehicle

Technical Field

The invention relates to a method for determining properties of a surface in the surroundings of a vehicle. In the case of this method, three-dimensional surface coordinates of the surface are generated by means of a sensor device. Furthermore, the invention relates to a device for determining a property of a surface in the surroundings of a vehicle. The device comprises sensor means by means of which three-dimensional surface coordinates of the surface can be generated.

Background

The method and the device according to the invention are particularly relevant for vehicles which travel automatically or partly automatically on unreinforced roadways. In this case, it is particularly important to identify, for example, depressions, slopes, furrows, obstacles, etc., in a defined and reliable manner. In particular in the automatic driving of trucks, the properties of the foundations to be passed should be known as precisely as possible.

Sensor devices are known with which the surface of the roadway or the environment surrounding the roadway can be measured in three dimensions. However, for applications such as automatic driving of a vehicle, the measurement accuracy of these sensor devices is insufficient. Measurement noise is often excessive or excessive measurement errors occur.

Disclosure of Invention

The invention is therefore based on the following task: an apparatus and a method of the type mentioned at the beginning are described with which the properties of a surface in the surroundings of a vehicle, in particular in the direction of travel of the vehicle, can be determined more precisely.

According to the invention, this object is achieved by a method having the features of claim 1 and by a device having the features of claim 14. Advantageous embodiments and developments emerge from the dependent claims.

The method according to the invention is characterized in that an approximation of the curved run of the surface in at least one direction is obtained on the basis of the three-dimensional surface coordinates and in that the classification of the surface coordinates is performed for marking the properties of the surface on the basis of the curved run and/or on the basis of the vertical spacing of the curved run approximation from the three-dimensional surface coordinates.

By means of the three-dimensional surface coordinates generated by the sensor device, a two-dimensional point grid for the surface in the surroundings of the vehicle is defined in the case of the method according to the invention. These surface coordinates include two horizontal coordinates. In addition, these surface coordinates also include vertical coordinates or vertical coordinates, which are described with respect to points defined by horizontal coordinates: at what height the surface is. Hereinafter, terms such as "upper" and "lower" refer to a vertical direction, i.e. the direction of gravitational force acting on the earth.

In the method according to the invention, the curvature of the surface is obtained from three-dimensional surface coordinates in at least one direction, and the curvature of the surface is obtained exactly by approximation. In contrast to interpolation, in this case, not only intermediate coordinates between the three-dimensional surface coordinates measured or generated by means of the sensor device are obtained, but also curves or surfaces which merely have to be close to the three-dimensional surface coordinates on which they are based are obtained in the case of using the three-dimensional surface coordinates as control points. In this way, the curved course of the real surface in the surroundings of the vehicle can be reproduced more precisely. In order to mark the properties of the surface, the curved course obtained in this way and/or the vertical spacing of the approximation of the curved course from the three-dimensional surface coordinates is then determined. Such vertical spacing may be derived because the approximated curve or plane does not have to traverse the three-dimensional surface coordinates on which the approximation is derived. The vertical spacing is thus the difference between the vertical coordinates of the three-dimensional surface coordinates and the approximated curve or plane for the horizontal coordinates of the same three-dimensional surface coordinates. The vertical spacing also includes symbols, i.e., the following information: whether the approximated surface or curve is above or below the three-dimensional surface coordinates.

It has been found that, in addition to the curved course of the curve or surface being approximated, the value of this vertical spacing can mark the properties of the surface as such: the surface may be classified as suitable in view of the possible passability by the vehicle.

Advantageously, this is achieved by the method according to the invention: a reliable determination of the surface course is also obtained in the case of strong measuring noises in the case of detecting surfaces in the surroundings of the vehicle. In this way, the nature of the surface in the surroundings of the vehicle can be reliably determined.

According to a further development of the method according to the invention, the approximation of the curved run is performed on the basis of the spline curve that is approximated. The spline curve to be approximated here approximates its own control point. In this case these control points are obtained from three-dimensional surface coordinates or a subset of said surface coordinates generated by means of the sensor device. The approximated spline curve describes a smooth curve course, which is determined by the control points, without the curve having to be extended through these control points as necessary.

The spline curve to be approximated may be a so-called base spline curve, which is also called B-spline curve. The basis function is used in the case of a basis spline curve. For example, to determine a point at a determined location on the curve, the individual control points are weighted according to the value of the basis function at that location. The weights are in particular chosen such that the influence of the control point decreases with increasing distance from the location.

As an extension of the base spline curve, NURBS (non-uniform rational B-spline) can be used, in which case the control points are weightedSplines or so-called P-splines, whereIn the case of splines or so-called P-splines, an additional cost function is applied to the base spline curve.

By means of the approximated spline curve, the curvature of the surface can advantageously be determined very precisely and reliably at least in a defined direction.

The classification can then be performed on the basis of the curved course of the approximated spline curve.

According to a further embodiment of the method according to the invention, the approximation of the curved course in the transverse direction is obtained on the basis of three-dimensional surface coordinates extending in the transverse direction with respect to the direction of travel of the vehicle. The classification is then performed according to the curved direction in the transverse direction. The nature of the surface in the surroundings of the vehicle in the transverse direction is particularly important in order to determine the lateral limit of the roadway. The unreinforced roadway may be limited, for example, by the course of a ditch, slope, or other non-trafficable surface. The lateral limit of the course can be determined in a defined and reliable manner by means of classification by means of an approximation of the course of the curve in the transverse direction.

Alternatively or additionally, an approximation of the curve run in the longitudinal direction can also be obtained on the basis of the surface coordinates extending in the longitudinal direction with respect to the driving direction. In this case, sorting may then alternatively or additionally be performed according to a curved direction in the longitudinal direction. By means of the curved course in the longitudinal direction, obstacles in the direction of travel of the vehicle can be identified, in particular, in a defined and reliable manner as a function of the classification. This is important in particular in the case of automatic travel of the vehicle, so that the vehicle does not collide with obstacles in the travel direction of the vehicle.

According to a further embodiment of the method according to the invention, an approximation of the curvature of the entire surface is obtained on the basis of the surface coordinates. A continuous curved surface is thus calculated from the three-dimensional surface coordinates, which curved surface represents a real surface in the surroundings of the vehicle. In this case, each row in the transverse direction or each column in the longitudinal direction, i.e. in the direction of travel, of the height map formed by the three-dimensional surface coordinates is therefore not described merely by the curve that is approximated. Instead, the entire height map is approximated by means of the approximated surface, in particular the surface formed by the spline curve. In this case the curvature profile in all directions can be determined and used for classifying the surface coordinates for marking the properties of the surface.

According to one embodiment of the method according to the invention, the passable region is determined from the curved course. The roadway is assigned to a continuous passable zone and the roadway surroundings are assigned to a zone adjacent to the continuous passable zone. In the case of automatic driving of the vehicle, the vehicle can be controlled as a function of the passable region in such a way that only the following driving maneuvers are carried out, which ensure that: the vehicle is always located in a passable area, i.e. a roadway.

A roadway is herein understood to be a surface which is suitable for being travelled by a vehicle. The roadway is in particular an unreinforced roadway or a so-called off-road roadway. The surroundings of a roadway are understood to be the regions adjacent to the roadway, in particular laterally to the roadway with respect to the direction of travel of the vehicle.

According to a further embodiment of the method according to the invention, the first class of classification is assigned to a group of three-dimensional surface coordinates which, on average, are arranged so far below the approximated curve that the first limit value is exceeded. Such surface coordinates relate in particular to surfaces such as those constituted by depressions. The limit value is thus defined such that the measured surface coordinates lie very far below the approximated curve. Furthermore, in this case, a set of surface coordinates is assigned to the first class only if it is within the passable area of the surface, i.e. on the roadway. Only in this case, for example, a depression is relevant for the driving of the vehicle.

According to a further embodiment of the method according to the invention, a region of the surface is assigned to the second class of classification, in which region an increasingly decreasing curvature is determined in a defined first section in the transverse direction. The second category is thus characterized by more strongly decreasing curves, for example, curves that decrease strongly in spline curves. By means of the second class, it is thus possible to determine regions which actually correspond to slopes or trenches. Such a slope or ditch is relevant, in particular laterally beside the roadway. The first section is thus defined in particular in the transverse direction with respect to the direction of travel. In this way, a slope or a ditch beside the roadway, i.e. in the surrounding of the roadway, can be detected. Exactly such a slope or groove is difficult to detect if only the three-dimensional surface coordinates generated by the sensor device are considered.

According to a further embodiment of the method according to the invention, a region of the surface is assigned to the third class, in which region in a defined second section of the surface direction an increasingly rising curvature and an increasingly falling curvature are determined. The second section is defined here, in particular, in the surface direction on the roadway. In this way, it is possible to detect floor waves or indentations in which there is an increasingly rising curvature course and an increasingly falling curvature course within the section. However, in contrast to indentations, the three-dimensional surface coordinates do not lie so far below the approximated curve, so that the regions comprising these surface coordinates should be assigned to the first class.

According to a further embodiment of the method according to the invention, a region of the surface is assigned to the fourth class of classifications, in which region a sudden rise in the vertical coordinates of a plurality of adjacent surface coordinates is determined. This abrupt rise in vertical coordinates occurs in the case of higher obstacles, such as parked vehicles or trees. The detection of the area of the surface of the fourth class may on the one hand be used to limit the roadway surroundings from the roadway. On the other hand, an obstacle in the traveling direction of the vehicle can be identified in this way.

According to a further embodiment of the method according to the invention, the roadway properties are determined from the values of the dispersion of the approximation of the curved course with the vertical spacing of the surface coordinates. For example, in this case, the vertical deviation of the approximation of the curved course can be determined for each surface coordinate. In this way, a value for the roughness of the roadway and/or a value for the average unevenness of the roadway can be obtained, for example. If the value for the roughness or for the average unevenness exceeds a defined limit value, it can be derived from this: the surface is no longer passable for the vehicle concerned.

According to a further embodiment of the method according to the invention, three-dimensional surface coordinates are generated in succession in time, and an approximation of the curved course is thus obtained for each set of three-dimensional surface coordinates recorded at a defined point in time. Based on the curved path and/or the approximation of the curved path to the vertical distance from the three-dimensional surface coordinates, an obstacle is then identified and the movement of the obstacle is determined from the time course of the three-dimensional surface coordinates assigned to the obstacle. In this way, a moving obstacle can be tracked by means of a plurality of measurement sequences in which the obstacle is identified.

According to a further embodiment of the method according to the invention, the position of the support surface (Auflagefl ä che) of the wheels of the vehicle on the surface is determined. An estimated trajectory of the position of the bearing surface of the wheels of the vehicle is then determined and the surface trafficability is checked for the estimated trajectory from a classification of the surface coordinates belonging to the trajectory. The position of the bearing surface of the wheel of the vehicle can be determined geometrically, since the position of the sensor means in relation to the bearing surface of the wheel of the vehicle is known. From the direction of movement and, if appropriate, the speed of the vehicle, the intended trajectory of the support surface of the wheel can then be determined. As set forth above, it can now be determined whether there are indentations or larger obstacles on the intended trajectory that limit the passability of the surface or cause the surface to no longer be passable. Such information can then be taken into account when the vehicle is automatically driving in order to avoid corresponding indentations or obstacles.

According to one embodiment of the method according to the invention, three-dimensional surface coordinates are generated by means of a stereo image. This has the following advantages: the three-dimensional surface coordinates can be determined by means of a very low cost sensor device. Furthermore, image data which can be used for other purposes is provided in this way. However, the three-dimensional surface coordinates obtained by means of stereoscopic images have the following drawbacks: measurement inaccuracy is so great that the obtained three-dimensional surface coordinates may not be sufficient as a basis for control for automatic travel. Furthermore, the slope of the roadway and other side restrictions may be identified only inadequately. In addition, in the case of strong measuring noise, depressions and grooves are no longer reliably detected. However, in the case of the method according to the invention, the generated three-dimensional surface coordinates are further processed. An approximation of the curved run of the surface is performed. Advantageously, the data of the stereo image can be preprocessed in such a way that dangerous areas, such as slopes, holes or lateral ditches, can be detected early in the expected lane of the vehicle and bypassed. In this way, the measurement problems of conventional stereo camera based measurement systems can be overcome. By means of the approximation of the curved course, in this case too, a reliable estimation of the surface course and of the possible slope can be performed in the case of strong measuring noise.

Furthermore, three-dimensional surface coordinates can be obtained by means of a laser scanner which samples the surface in the direction of travel in the transverse direction. Advantageously, by means of this way of obtaining three-dimensional surface coordinates, it can also function in bad weather conditions and at night. The sampling density is smaller compared to the use of a stereo camera in the direction of travel, i.e. perpendicular to the direction of rotation of the system in which the scanning is performed. This may result in: grooves and potholes are not identified. However, in the case of the method according to the invention, it is ensured by approximation of the curved course: such surface properties can be identified with certainty. The application of the laser scanner also has the following advantages: abrupt height changes in the surface can be identified very reliably.

The device according to the invention for determining the properties of a surface in the surroundings of a vehicle is characterized in that: a computing unit configured to obtain an approximation of the curved trend of the surface in at least one direction based on the three-dimensional surface coordinates; and a classification unit by means of which classification of the three-dimensional surface coordinates for marking the properties of the surface can be performed as a function of the curved run and/or the approximation of the curved run and the vertical spacing of the three-dimensional surface coordinates.

The device according to the invention is designed to carry out the method according to the invention described above. It thus has the same advantages as the method according to the invention.

The sensor device of the apparatus according to the invention comprises in particular a stereo camera. Furthermore, a laser measurement system that performs scanning or a measurement system based on laser intersection may also be used as the sensor device. The measuring system based on laser intersection has the following advantages: the ground plane can be detected very accurately. However, it has only a very limited range of action. Furthermore, the installation space requirements of such measuring systems are relatively high.

The method according to the invention and the device according to the invention are used in particular in vehicles, in particular trucks, which have a system for automatic driving. In this way, the vehicle can automatically travel on an unreinforced roadway without a driver. The system for automatic driving is supported by the method according to the invention or the device according to the invention. Information about the nature of the surface in the traveling direction of the vehicle is provided. It is determined here that: which area of the surface is passable and thus suitable as a roadway. Furthermore, the nature of the surroundings of the roadway is determined such that it can be taken into account when controlling the vehicle, which surface in the surroundings of the vehicle is suitable for the driving maneuver in the case of automatic driving.

The device according to the invention can be integrated for this purpose in a vehicle having a system for automatic driving. The sensor device detects the surface of the ground in the direction of travel of the vehicle, wherein the lateral area in front of the vehicle is also detected.

Drawings

Hereinafter, the present invention will be described in detail according to embodiments with reference to the accompanying drawings.

Fig. 1 shows a situation in which the method according to the invention can be applied;

fig. 2 shows a further situation in which the method according to the invention can be applied;

fig. 3 schematically shows the construction of an embodiment of the device according to the invention;

fig. 4 shows an example of a reconstructed height map as obtained in the case of an embodiment of the method according to the invention;

fig. 5 shows a further example of a reconstructed height map as obtained in the case of an embodiment of the method according to the invention;

fig. 6 illustrates an approximation of the curved course of a surface as performed in the case of an embodiment of the method according to the invention;

fig. 7 illustrates the approximation of the curvature course of a surface in the off-road situation as performed in the case of an embodiment of the method according to the invention;

fig. 8 illustrates the approximation of the curved course of a surface as is performed in the case of an embodiment of the method according to the invention when a road with a lateral ditch is travelled.

Detailed Description

In the following, first the situation in which the method according to the invention is performed will be explained with reference to fig. 1 and 2.

The vehicle 1 travels on an unreinforced surface. The vehicle 1 may be driven automatically, for example without driver support. The unreinforced surface comprises the roadway 2 and laterally beside it in the direction of travel comprises the roadway surroundings 3. The roadway surroundings 3 delimit the roadway 2. In the example shown in fig. 1, the roadway surroundings 3 laterally beside the roadway 2 comprise: and a trench 4. In the example case shown in fig. 2, the roadway 2 is bounded laterally by the earth wall 5 and by the tree 16 or bridge tower. While traveling on this surface, the wheels of the vehicle 1 form a bearing surface 11 on the roadway 2.

As set forth below, it is possible to identify by means of embodiments of the method according to the invention and of the device according to the invention: which area of the surface in the direction of travel of the vehicle 1 is passable and can therefore be allocated to the roadway. In addition, the properties of the surface of the roadway 2 and the properties of the roadway surroundings 3 can be determined.

The following embodiment of the device 6 according to the invention is elucidated with reference to fig. 3:

the device 6 comprises a stereo camera 7 and an image processing unit 8 connected thereto. The stereo camera 7 and the image processing unit 8 form a sensor device by means of which three-dimensional surface coordinates of the surface in the surroundings of the vehicle 1, in particular in the direction of travel of the vehicle 1, can be generated. With the aid of this sensor device, a three-dimensional height map of the surface in the direction of travel of the vehicle 1 can thus be obtained.

The image processing unit 8 is connected to a computing unit 9, to which the image processing unit transmits three-dimensional surface coordinates. As will be explained later with reference to this embodiment of the method according to the invention, the calculation unit 9 is designed to obtain an approximation of the curved course of the surface on the basis of three-dimensional surface coordinates.

The calculation unit 9 is connected to a classification unit 10, by means of which classification of the surface coordinates is performed according to the curved run and the approximation of the curved run and the vertical spacing of the three-dimensional surface coordinates for demarcating the roadway 2 from the roadway surroundings 3 and for marking the properties of the surface, as will also be explained later with reference to this embodiment of the method according to the invention.

In other embodiments, the sensor device may also use a scanning laser measurement system, a laser intersection based measurement system or other measurement system with which three-dimensional surface coordinates can be obtained.

The device 6 may finally be coupled to a control device 13 for the automatic travel of the vehicle 1. All data about the properties of the surfaces, in particular of the roadway 2 and the roadway surroundings 3, are transmitted to the control device 13.

In the following, embodiments of the method according to the invention which can be implemented by the device 6 described above are elucidated with reference to fig. 4 to 8:

the stereo camera 7 of the sensor device records, starting from the vehicle 1, temporally successive stereo images in the direction of travel. These images in different driving situations are shown in fig. 4, 5, 7 and 8. The image processing unit 8 processes the stereoscopic image and obtains three-dimensional surface coordinates in a manner known per se. Here, for a horizontal grid, the vertical coordinates for each point of the grid are determined. A distance is found between the points of the horizontal grid, which distance is determined by the resolution of the stereo camera and the resolution of the subsequent image processing. The reconstructed height map, which is formed from the three-dimensional surface coordinates and is obtained from the image of the stereo camera 7, is shown as a net 17 in the images shown in fig. 4, 5, 7 and 8.

The three-dimensional surface coordinates here include coordinates in the transverse direction with respect to the direction of travel. These three-dimensional surface coordinates in the transverse direction are also referred to as rows of a three-dimensional height map made up of three-dimensional surface coordinates. Further, these three-dimensional surface coordinates include coordinates in the longitudinal direction with respect to the traveling direction. The coordinates in the longitudinal direction with respect to the direction of travel are also referred to as columns of a three-dimensional height map.

In the case of the method according to the invention, an approximation of the curved course of the surface in at least one direction is obtained on the basis of the three-dimensional surface coordinates. In this embodiment, the approximation of the curved course is performed row by row, i.e. in a transverse direction with respect to the direction of travel. The three-dimensional surface coordinates of a row are used here as control points for the spline curve that is approximated. A base spline curve is used in this embodiment. However, as mentioned at the outset, other approximated spline curves can also be used. In this way, a continuous spline curve is derived for each row of the three-dimensional height map.

Such spline curves 14 for a plurality of rows of the three-dimensional height map are schematically shown in fig. 6. The spline 14 to be approximated is characterized in that, although it is determined by its control points, i.e. the three-dimensional surface coordinates of a row, it extends unnecessarily through these three-dimensional surface coordinates. In the case of each of the three-dimensional surface coordinates of each row, a vertical distance from the approximated spline 14 can thus be derived. The value of this vertical distance is shown in fig. 6 by the length of the arrow 15 in the case of the corresponding three-dimensional surface coordinates.

By means of the approximated spline curve 14, a continuous surface can thus be modeled row by row from the three-dimensional surface coordinates. Furthermore, the curved course can also be approximated in a plurality of directions, so that a continuous surface can also be modeled from three-dimensional surface coordinates. This is then also possible in the case of higher measurement noise in the generation of three-dimensional surface coordinates. This course of the spline 14 now allows the measured three-dimensional surface coordinates, i.e. the three-dimensional surface coordinates of the three-dimensional height map, to be divided according to different criteria. In one aspect, the vertical spacing of the three-dimensional surface coordinates from the spline 14 may be considered. On the other hand, the curved course of the spline 14 can be considered.

Hereinafter, classification of three-dimensional surface coordinates as performed by the classification unit 10 is set forth for marking properties of the surface.

According to the curved course of the spline 14, the region which can be passed through for the vehicle 1 is first determined. The continuous passable area is defined as a roadway 2. The non-passable surroundings of such a passable area are defined as roadway surroundings 3.

If a single three-dimensional surface coordinate in the vertical direction is significantly deviated from its neighboring plurality of three-dimensional surface coordinates, the three-dimensional surface coordinates are classified as abnormal values and are not considered hereinafter.

The obtained three-dimensional surface coordinates may furthermore be filtered over time in order to correct misclassification of the respective three-dimensional surface coordinates.

The average is arranged so far below the approximated curve, i.e. below the spline 14, that a set of three-dimensional surface coordinates exceeding a first limit value is assigned to the first class. The surface coordinates that form the depression through which the vehicle 1 should not travel belong to this class. The first limit value can be obtained here by a previously performed measurement of a depression having a defined depth. In this case, in particular, three-dimensional surface coordinates extending in the longitudinal direction with respect to the driving direction are considered.

Furthermore, a region of the surface is defined in which a progressively decreasing curvature course is defined in a defined first section in the transverse direction. The region is assigned to the second class of classifications. This second category should be especially detecting lateral slopes and furrows. The first section thus extends in the transverse direction. The curvature course is determined from a spline curve 14, wherein the curvature course is obtained from a row of a three-dimensional height map of the three-dimensional surface coordinates. If the spline has a more strongly decreasing curvature, then it is inferred that: in this region is a ramp or a groove.

Furthermore, a region of the surface is defined in which an increasingly higher curvature and an increasingly lower curvature are defined in a defined second section of the surface direction. The region is assigned to the third class of classifications. This third type marks ground waves, small depressions or vegetation. The second section is selected in this case such that it extends in any surface direction and is arranged on the roadway 2. The third class may alternatively also comprise only the region of the surface which is characterized by a strongly rising curvature in the spline curve 14.

Furthermore, a region of the surface is determined in which a sudden rise in the vertical coordinates of a plurality of adjacent surface coordinates is determined. The region is assigned to the fourth class of the classification. The fourth category includes larger obstacles such as parked vehicles, trees, and the like. Such larger obstacles can be determined not only on the roadway 2 but also in the roadway surroundings 3.

The roadway properties are finally determined from the values of the dispersion of the vertical spacing of the spline 14 from the corresponding surface coordinates. In this way, a value for the roughness of the roadway and a value for the average unevenness of the roadway can be obtained.

Three-dimensional surface coordinates are generated by the sensor device in time succession. During the running of the vehicle 1, stereoscopic images are sequentially recorded and three-dimensional surface coordinates are thereby obtained. The curved course is approximated by means of a spline curve 14 for each stereo image, i.e. for each set of three-dimensional vertical coordinates. Based on this curved course and the vertical distance of the spline 14 from the corresponding three-dimensional surface coordinates, obstacles can be identified as explained above. The movement of the obstacle may then be determined by a time course of the three-dimensional surface coordinates assigned to the obstacle. In this way, the obstacle can be tracked over time.

The classification unit 10 thus generates a classification of the three-dimensional surface coordinates of the three-dimensional height map from the vertical orientation of the spline 14 with respect to the approximation and from the course of the spline 14, in particular from the curved course of the spline 14.

In fig. 4, an example is shown in which an image recorded by the stereo camera 7 is shown in the left part. Overlapping the image, a reconstructed height map, i.e. a mesh 17 of three-dimensional surface coordinates, is inserted. In addition, a transverse height distribution is inserted in the figure. On the right side of the image is shown a classification of the three-dimensional surface coordinates according to a gray scale map. The illustration of fig. 4 relates here to a driving situation on a road, in which the roadway 2 is laterally limited by the ditch and the roadway. In these grey-scale images, different classifications are evident next to the right side of the image being reproduced. In particular, the passable area, i.e. the roadway 2, can be identified unambiguously.

Fig. 5 shows the corresponding contents for the driving situation on an unreinforced road with a lateral ditch. The lower right image in fig. 5 shows a classification of three-dimensional surface coordinates. It should be apparent that there are two furrows to the left and right of the roadway 2. There are three-dimensional surface coordinates of the area between which traffic has been classified as passable, i.e. the roadway 2.

In fig. 7, the image recorded by the stereo camera 7 is shown on the underside in the driving situation of an off-road section with a lateral drop on the right side next to the roadway 2. In this case too, a three-dimensional height map is inserted into the image according to the mesh 17. In the upper part of fig. 7, a spline curve 14 is shown for representing the roadway 2 and the roadway surroundings 3. In the right-hand part of the spline curves 14, a strong lateral drop can be deduced from the curvature of the spline curve 14 and from the vertical distance of the three-dimensional surface coordinates from the spline curve 14. The area is classified as non-passable.

Fig. 8 shows an image of a stereo camera 7 for a driving situation on a road with a lateral ditch on the right and left side next to the roadway 2. A three-dimensional height map from the net 17 is again shown overlapping the image. Above this image, an associated spline 14 is shown for representing the surroundings. A large deviation of the vertical coordinates of the three-dimensional surface coordinates from the corresponding vertical coordinates of the spline 14 is found in the upper right and upper left regions. In this way, the lateral sulcus is identified and the associated three-dimensional surface coordinates are classified accordingly.

In the case of the use of the method according to the invention and the device according to the invention in an autonomous vehicle 1, the support surface 11 of the wheels of the vehicle 1 on the surface is determined. The spatial relationship between the support surface 11 of the wheels of the vehicle 1 and the three-dimensional surface coordinates is determined by the spatial arrangement of the support surface 11 of the wheels relative to the stereo camera 7 and the angle of view of the stereo camera 7. Based on the data present in the control unit 13, the expected trajectory of the bearing surface 11 of the wheels of the vehicle 1 is determined. The availability of the surface is then checked by the classification unit 10 for the region, i.e. for the expected trajectory. Here, a classification of three-dimensional surface coordinates is considered, which lie on the expected trajectory of the bearing surface 11 of the wheels of the vehicle 1. In addition, adjacent regions can also be considered for this purpose. If a depression is found here, for example, on such a track, a corresponding signal is transmitted to the control unit 13. It is then possible to automatically carry out a driving maneuver in which the indentations are bypassed. Furthermore, the control unit 13 can ensure, from the data transmitted by the classification unit 10: the vehicle 1 is automatically moved over a passable area, i.e. a roadway 2. Furthermore, the roadway surroundings 3 can be taken into account in the case of driving maneuvers. For example, it is also possible to limit the trees of the roadway 2 laterally, so that the autonomous vehicle 1 does not collide laterally or above the obstacle even if the bearing surfaces 11 of the wheels of the vehicle 1 are completely on the roadway 2.

The method according to the invention and the device 6 according to the invention are used in particular in off-road applications of trucks. It can be applied, for example, in the case of trucks, which are used in mining areas. The truck may in this case be equipped with a system for automatic driving, which system makes use of the device 6 according to the invention and the method according to the invention in order to control the truck. Furthermore, the method according to the invention and the device 6 according to the invention can be used in off-road applications of passenger vehicles and in driving on unreinforced substrates. In this way, an assistance system can be provided which reliably detects obstacles and depressions and outputs information about the nature of the roadway to the driver. Finally, the method according to the invention and the device 6 according to the invention can be used in off-road applications of motorcycles and in the case of motorcycles running on unreinforced substrates. In this case too, an auxiliary system can be provided which reliably detects obstacles and depressions. In the case of applications in motorcycles, the auxiliary system can in particular indicate dangerous, particularly deep depressions and ground irregularities.

List of reference numerals

1. Vehicle with a vehicle body having a vehicle body support

2. Roadway

3. Surrounding environment of roadway

4. Ditch (groove)

5. Soil wall

6. Apparatus and method for controlling the operation of a device

7. Stereo camera

8. Image processing unit

9. Calculation unit

10. Classification unit

11. Bearing surface

13. Control device

14. Spline curve

15. Arrows

16. Tree and tree

17. Net

18. Transverse height distribution

Claims

1. Method for determining properties of a surface in the surroundings of a vehicle (1), wherein:

three-dimensional surface coordinates of the surface are generated by means of sensor means (7, 8),

an approximation of the curved trend of the surface in at least one direction is obtained based on the three-dimensional surface coordinates,

performing a classification of the three-dimensional surface coordinates in accordance with the curved run and/or an approximation of the curved run and a perpendicular spacing of the three-dimensional surface coordinates for marking the property of the surface,

it is characterized in that the method comprises the steps of,

the position (11) of the bearing surface of the wheels of the vehicle (1) on the surface is determined,

an estimated trajectory of the position (11) of the bearing surface of the wheel of the vehicle (1) is determined and the surface is checked for its trafficability according to a classification of the three-dimensional surface coordinates belonging to the trajectory.

2. The method according to claim 1,

it is characterized in that the method comprises the steps of,

the approximation of the curved trend is performed according to the approximated spline curve (14).

3. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

obtaining an approximation of the curved run in the transverse direction based on the three-dimensional surface coordinates extending in the transverse direction with respect to the direction of travel of the vehicle (1), and

sorting is performed according to the curved direction in the transverse direction.

4. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

an approximation of the curved trend of the entire surface is obtained based on the three-dimensional surface coordinates.

5. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

a passable area is determined as a function of the curve, and a roadway (2) is assigned to a continuous passable area and a roadway surroundings (3) is assigned to an area adjacent to the continuous passable area.

6. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

the first class of the classification is assigned to a group of three-dimensional surface coordinates which are averaged such that the first limit value is exceeded so far below the approximated curved run.

7. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

the second class of classification is assigned to regions of the surface in which increasingly descending curvature is determined in a defined first section in the transverse direction.

8. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

the surface is assigned to a third class of classes in which regions, in a defined second section of the surface direction, increasingly higher curvature and increasingly lower curvature are defined.

9. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

a fourth class of classifications is assigned to regions of the surface in which sudden increases in the vertical coordinates of a plurality of adjacent three-dimensional surface coordinates are determined.

10. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

the roadway properties are determined from the values of the dispersion of the vertical spacing of the approximation of the curved run from the surface coordinates.

11. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

three-dimensional surface coordinates are generated in succession in time, and approximations of the curved run are obtained respectively,

identifying obstacles from the curved run and/or the approximation of the curved run and the perpendicular spacing of the three-dimensional surface coordinates, and

the movement of the obstacle is determined from a time course of the three-dimensional surface coordinates assigned to the obstacle.

12. The method according to claim 1 or 2,

it is characterized in that the method comprises the steps of,

the three-dimensional surface coordinates are generated by means of a stereoscopic image.

13. Device (6) for determining properties of a surface in the surroundings of a vehicle (1), said device having:

sensor means (7, 8) by means of which three-dimensional surface coordinates of the surface can be generated,

-a computing unit (9) configured for obtaining an approximation of the curved trend of the surface in at least one direction based on the three-dimensional surface coordinates; and

a classification unit (10) by means of which classification of the three-dimensional surface coordinates for marking the properties of the surface can be performed as a function of the curved run and/or the approximation of the curved run and the vertical spacing of the three-dimensional surface coordinates;

it is characterized in that the method comprises the steps of,

the device is configured to determine a position (11) of a bearing surface of a wheel of the vehicle (1) on the surface,

an estimated trajectory of the position (11) of the bearing surface of the wheel of the vehicle (1) is determined, and the surface is checked for its trafficability according to a classification of the three-dimensional surface coordinates belonging to the trajectory.

14. The device (6) according to claim 13,

it is characterized in that the method comprises the steps of,

the sensor device (7, 8) comprises a stereo camera (7).