EP3103683A1 - Method for operating a camera-based vehicle system - Google Patents
Method for operating a camera-based vehicle system Download PDFInfo
- Publication number
- EP3103683A1 EP3103683A1 EP16001285.2A EP16001285A EP3103683A1 EP 3103683 A1 EP3103683 A1 EP 3103683A1 EP 16001285 A EP16001285 A EP 16001285A EP 3103683 A1 EP3103683 A1 EP 3103683A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- vehicle
- pattern
- camera
- image
- image area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000001454 recorded image Methods 0.000 claims 1
- 230000007704 transition Effects 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/40—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the details of the power supply or the coupling to vehicle components
- B60R2300/402—Image calibration
Definitions
- the invention relates to a method for operating a camera-based vehicle system, comprising a camera, which is arranged in the vehicle interior for receiving the vehicle apron or the vehicle rear field, and a control device processing the camera images.
- Modern motor vehicles have a number of vehicle systems that are intended to assist the driver or that are relevant to the operation of the motor vehicle. Some of these vehicle systems are camera-based, that is to say that their function is based on camera images of the vehicle surroundings being recorded, which are analyzed by a control device in order to output information or messages to the driver on the basis of the analyzed information Vehicle longitudinal or transverse guidance, etc.
- a camera z. B. installed in the region of the interior mirror on the windscreen, the camera is directed forward to record the vehicle apron.
- vehicles are installed in the area of the rear of the vehicle, for example in SUVs or vans, to accommodate the vehicle rear field.
- laterally arranged cameras are provided which absorb the side environment.
- the cameras can be mounted both inside and outside the vehicle. Often this is the part of the motor vehicle in the field of vision of the camera, for example, in a front camera usually the hood of the vehicle, in a rear camera, for example, the lower bumper or the like.
- the position of the hood is defined in advance in today's vehicle systems.
- the invention is based on the problem of providing a method which is improved in comparison and which makes it possible to improve the system performance.
- the method according to the invention provides that for determining an information-relevant image area given in at least one camera image by the control device, an image area shown in the camera image and showing part of the vehicle is determined in the camera image itself, which image area in FIG The processing of subsequently recorded camera images is disregarded.
- the invention provides to determine the image area to be "cut out" of the overall image, in which the vehicle part is shown, in one or more currently recorded camera image itself. This makes it possible to dimension the unobservable image area exactly as it actually appears in the respective camera image. For the actual representation shows the real constructive situation, so that exactly the image area can be suppressed, in which the vehicle part is shown, but not an additional, tolerances reasons considered image area.
- the information-relevant image area corresponds exactly to the image area, which actually contains information that can be detected or evaluated by the control device. There Consequently, this information-relevant image area is not prone to be prophylactically restricted for reasons of safety or tolerance, a larger image area is therefore available for evaluation, so that the system performance can be improved.
- the determination of the image area showing a part of the vehicle takes place on the basis of a pattern or object shown in the camera image or images that is visible in the information-relevant image area and hidden in the image area showing part of the vehicle which pattern or object is determined by the controller.
- the controller processes the camera image (s) using a corresponding algorithm that detects a pattern or object that is visible in the information-relevant area.
- This object extends as far as the dividing line or edge in which the information-relevant image area merges into the other image area showing the vehicle part, that is to say that the pattern or the object is then obscured after the transition from the vehicle part.
- the controller is now able to detect such a pattern or object and thus detect a point or area where the one image area merges into the other.
- the controller Based on the geometry of the vehicle part, so for example the hood, which is known in their design data control device side, now the exact course of the edge can be detected, so that the transition region can be defined exactly.
- the control device can determine the course of the edge or the image area transition via a suitable edge detection algorithm or the like.
- the pattern or the object can either be arranged on the vehicle itself, for example, in that it is applied to the bonnet or placed or is attached in the edge region.
- the controller now determines this pattern with the appropriate analysis algorithm or object and can determine from the information obtained, the dividing line of the image areas, respectively, the course of the vehicle edge part.
- the pattern or object is located in front of the vehicle.
- the vehicle is placed in a position to be defined in front of this pattern or object, after which the controller analyzes the camera image (s) again and first determines the pattern or object in the image, and then the point or area in which the pattern or object is at the image area transition "disappears". This means that the vehicle part begins in the image where it is no longer possible to recognize the pattern or object.
- the pattern or object and the vehicle may be fixed in position, that is, both are static. So they do not move relative to each other.
- the pattern or object moves relative to the vehicle or the vehicle relative to the pattern or object.
- One element thus moves, while the other element is fixed in position.
- a series of camera images are taken which show the pattern or object moving relative to the vehicle, regardless of which element is moving. It is a dynamic process in this case.
- the pattern or object may be moved substantially horizontally. This means that it is guided, for example via a linear guide on the ground, relative to the vehicle, preferably of course in the vehicle longitudinal axis, is moved. It can even be guided under the vehicle.
- the pattern or object can also be moved up and / or down, or moved horizontally across the vehicle. In this case, the pattern or object is moved, for example, vertically or laterally along a guide, so that it inevitably comes in the course of the movement, for example from above, from the vehicle part is covered.
- the respective movement is shown in the camera images, so that the place where the pattern or object "disappears" behind the vehicle part, can be detected exactly.
- a horizontal movement taking place transversely along the driver's side is also conceivable, and the part of the pattern or object visible in the successive images also changes.
- the pattern or object When the vehicle is moving relative to the pattern or object, the pattern or object may in turn be positioned on the ground as the vehicle passes over the pattern or object, thus ensuring that it necessarily disappears.
- the pattern or object may be, for example, a reference pattern, for example in the form of a checkerboard pattern printed on a surface, or a simple black and white surface with a defined edge.
- the reference pattern may also be designed as a light column or the like. In any case, it is a specially provided for this purpose and appropriately positioned pattern or object.
- the pattern or object may also be a natural pattern or object located in the vehicle apron or vehicle rear field, which is determined by the control device.
- a pattern is preferably determined such that has an edge structure visible in the image. It is conceivable, for example, a vertical column, a beam, a piece of furniture or the like, such as those in a workshop or the like, where for image purposes, the image area determination or calibration is made. This alternative is advantageous in that no special patterns or objects need to be kept.
- the detection of the hood or a unit obscuring another view can also take place via a temporal viewing of the images, that is to say a comparison of a plurality of images of the same built - in camera taken one after the other or at different times Area with z. B. the hood is less different than other areas.
- the invention further relates to a motor vehicle, comprising a camera with associated control device, which motor vehicle is designed to carry out the method according to the invention.
- Fig. 1 shows a motor vehicle 1 according to the invention, comprising a vehicle system 2, for example in the form of a driver assistance system or the like, with a control device 3, a camera 4 receiving the vehicle apron and exemplarily several devices 5 operated via the control device 3, which are a display element Control element or other components or assemblies in Can act vehicle that are controlled or operated via the vehicle system.
- a vehicle system 2 for example in the form of a driver assistance system or the like
- a control device 3 a camera 4 receiving the vehicle apron and exemplarily several devices 5 operated via the control device 3, which are a display element Control element or other components or assemblies in Can act vehicle that are controlled or operated via the vehicle system.
- the camera 4 takes as described on the vehicle apron. Part of the hood 6 is also located in the detection area of the camera.
- the image area which shows the hood 6 can not necessarily contain any information-relevant contents. It is therefore necessary to determine exactly where the boundary between the image area, which contains information-relevant information, and the image area, which shows the vehicle part, here the engine hood 6, runs.
- Fig. 2 shows by way of example a camera image 7, which is recorded via the camera 4.
- a reference pattern 8 is arranged in the region of the vehicle front, which extends to below the hood 6.
- the dashed area of the reference pattern 8 is hidden over the hood 6.
- the control device 3 is now able to determine the boundary line 11, where one image area 9 merges into the other image area 10. This is done by analyzing the camera image 7 for detecting the reference pattern 8, which is divided here like a checkerboard in black and white fields. The control device 3 now determines exactly the region 12 in which the regular structure, namely the reference pattern 8, coming from the upper image area 9 virtually disappears. On or off exactly at this image area 12, the reference pattern 8 is thus covered by the hood 6.
- the control device can now determine the remaining course of the hood edge, ie the line 11. This can be done either by using known geometry data of the hood is present on the part of the control device 3. Because the hood 6 has a defined, characteristic course. From this defined course of the entire course of the line 11, starting from the area 12, can now be determined. Alternatively, it is conceivable to determine the course of the line 11, which indeed defines a sharp, optical separation of the image areas 9 and 10, via a corresponding edge detection algorithm, again by image analysis. Finally, if the boundary line is virtually rectilinear due to the geometry of the hood 6, the determined image line of the area 12 could simply be used as the dividing line of the image areas 9 and 10
- control device 3 can now define the information-relevant image area 9, so that the information that will subsequently be used to control the equipment 5 can be obtained from this within the framework of a subsequent processing of the image content of the image area 9.
- An influence on the image analysis by the image area 10 is excluded since this image area remains taken into account during the processing. Consequently, almost the maximum relevant image area 9 of the analysis is used, so that the greatest possible information depth can be recorded.
- the analysis of the image area 6 is avoided, which is time-consuming and unnecessary because it contains no relevant information.
- Fig. 3 shows in the form of a schematic representation of an alternative to the fixed arrangement of the reference pattern 8 on the vehicle front, as in Fig. 1 shown.
- the reference pattern 8 as shown by the arrow P1, along a guide vertically moved up and down.
- the camera 4 can detect the reference pattern until it is virtually covered by the vehicle front, here in turn the hood 6, which is the case when the reference pattern is lowered below the line 13.
- the control device 3 now takes a multiplicity of images of the camera 4, which were taken during the lowering movement, for the purpose of analysis and in turn determines the corresponding transition region 12 in order to define the image regions 9 and 10 exactly.
- a reference pattern 8 is provided, which is here, however, horizontally movable, as shown by the arrow P2. It is located in front of the motor vehicle 1.
- the reference pattern 8 is pushed along the arrow P2 in the direction of the motor vehicle 1 and passes the line shown 13, so disappears with increasing approach movement.
- the control device 3 analyzes a plurality of camera images and detects in this way the exact movement path of the reference pattern 3 and thus also the transition region 12, based on what then the image areas 9 and 10 are defined.
- a reference pattern 8 is shown, which is static, so does not move relative to the vehicle, which is also fixed in position.
- the reference pattern 8 is designed here as a rod or rod. It has a first section 8a which, being static, is permanently visible in the image of the camera 4. The second, lower area 8b is covered by the hood 6, so it is not visible in the picture. The areas 8a and 8b are separated by the line 13.
- the control device 3 now includes in the camera image the reference pattern 8 and thus the area 8a and can thus define the image areas 9 and 10 exactly and distinguish them from one another.
- the embodiment according to Fig. 5 corresponds substantially to the embodiment according to Fig. 1 , only with the difference that the reference pattern 8 does not, as in Fig. 1 , Is attached to the vehicle itself, but positionally fixed in front of the vehicle is arranged.
- the determination made of the position of the line 11, that is, the definition of the image area 9 and 10 can be made once factory, if the system once set up or calibrated. However, it is also possible to carry out this calibration regularly during workshop visits.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
Verfahren zum Betrieb eines kamerabasierten Fahrzeugsystems, umfassend eine Kamera (4), die im Fahrzeuginneren zur Aufnahme des Fahrzeugvorfelds oder des Fahrzeugrückfelds angeordnet ist, sowie eine die Kamerabilder verarbeitende Steuerungseinrichtung (3), wobei zur Bestimmung eines in wenigstens einem Kamerabild (7) gegebenen informationsrelevanten Bildbereichs (9) seitens der Steuerungseinrichtung (3) ein in dem Kamerabild gezeigter, einen Teil des Fahrzeugs (1) zeigenden Bildbereich (10) im Kamerabild (7) selbst ermittelt wird, welcher Bildbereich (10) im Rahmen der Verarbeitung nachfolgend aufgenommener Kamerabilder (7) außer Betracht bleibt.Method for operating a camera-based vehicle system, comprising a camera (4), which is arranged in the vehicle interior for receiving the vehicle apron or the vehicle rear field, and a control device (3) processing the camera images, wherein for determining an information relevant in at least one camera image (7) Image area (9) on the part of the control device (3) a shown in the camera image, a part of the vehicle (1) facing image area (10) in the camera image (7) itself is determined which image area (10) in the context of processing subsequently captured camera images ( 7) is disregarded.
Description
Die Erfindung betrifft ein Verfahren zum Betrieb eines kamerabasierten Fahrzeugsystems, umfassend eine Kamera, die im Fahrzeuginneren zur Aufnahme des Fahrzeugvorfelds oder des Fahrzeugrückfelds angeordnet ist, sowie eine die Kamerabilder verarbeitende Steuerungseinrichtung.The invention relates to a method for operating a camera-based vehicle system, comprising a camera, which is arranged in the vehicle interior for receiving the vehicle apron or the vehicle rear field, and a control device processing the camera images.
Moderne Kraftfahrzeuge verfügen über eine Reihe von Fahrzeugsystemen, die dem Fahrer assistieren sollen oder die für den Betrieb des Kraftfahrzeugs relevant sind. Manche dieser Fahrzeugsysteme sind kamerabasiert, das heißt, dass ihre Funktion darauf beruht, dass über eine Kamera Bilder der Fahrzeugumgebung aufgenommen werden, die seitens einer Steuerungseinrichtung analysiert werden, um gestützt auf die analysierten Informationen entsprechende Informationen oder Mitteilungen an den Fahrer auszugeben, Eingriffe in die Fahrzeuglängs- oder Querführung vorzunehmen etc.Modern motor vehicles have a number of vehicle systems that are intended to assist the driver or that are relevant to the operation of the motor vehicle. Some of these vehicle systems are camera-based, that is to say that their function is based on camera images of the vehicle surroundings being recorded, which are analyzed by a control device in order to output information or messages to the driver on the basis of the analyzed information Vehicle longitudinal or transverse guidance, etc.
Häufig wird eine solche Kamera z. B. im Bereich des Innenspiegels an der Frontscheibe verbaut, wobei die Kamera nach vorne gerichtet ist, um das Fahrzeugvorfeld aufzunehmen. Mitunter sind aber auch Fahrzeuge im Bereich der Fahrzeugrückseite verbaut, beispielsweise bei SUVs oder Transportern, um das das Fahrzeugrückfeld aufzunehmen. Mitunter sind auch seitlich angeordnete Kameras vorgesehen, die das seitliche Umfeld aufnehmen. Die Kameras können sowohl im Inneren des Fahrzeugs als auch außen angebracht sein. Häufig befindet sich dabei im Sichtbereich der Kamera ein Teil des Kraftfahrzeugs, beispielsweise bei einer Frontkamera zumeist die Motorhaube des Fahrzeugs, bei einer Heckkamera beispielsweise die untere Stoßstange oder dergleichen. Um Wechselwirkungen und Einflüsse eines im Kamerabild sichtbaren Fahrzeugteils, also beispielsweise der Motorhaube, auf die Bildauswertung und damit die Informationsgewinnung zu vermeiden, wird in heutigen Fahrzeugsystemen die Lage der Motorhaube vorab definiert.Often, such a camera z. B. installed in the region of the interior mirror on the windscreen, the camera is directed forward to record the vehicle apron. Occasionally, however, vehicles are installed in the area of the rear of the vehicle, for example in SUVs or vans, to accommodate the vehicle rear field. Sometimes laterally arranged cameras are provided which absorb the side environment. The cameras can be mounted both inside and outside the vehicle. Often this is the part of the motor vehicle in the field of vision of the camera, for example, in a front camera usually the hood of the vehicle, in a rear camera, for example, the lower bumper or the like. In order to avoid interactions and influences of a vehicle visible in the camera image part, so for example the hood, on the image analysis and thus the information retrieval, the position of the hood is defined in advance in today's vehicle systems.
Sie wird aus Konstruktionsdaten des Kraftfahrzeugs und dem bekannten Positionsort der Kamera bestimmt und einmalig definiert. Diese Maße sind jedoch toleranzbehaftet, so dass ein Sicherheitspuffer im System vorgehalten werden muss, um sicherzugehen, dass diese Toleranzen nicht dazu führen, dass ein kleiner Teil der Motorhaube doch wieder im verarbeiteten Bildbereich erscheint. Es wird also quasi mehr aus dem aufgenommenen Kamerabild "ausgeschnitten", um die Motorhaube oder ein anderes Fahrzeugteil aus der Bildverarbeitung auszunehmen, als nötig, was die Systemperformance einschränkt.It is determined from construction data of the motor vehicle and the known location of the camera and defined once. However, these dimensions are tolerance tolerant, so a safety buffer must be kept in the system to ensure that these tolerances do not cause a small portion of the hood to reappear in the processed image area. It is almost "cut out" of the recorded camera image to exclude the hood or another vehicle part from the image processing, as necessary, which restricts the system performance.
Der Erfindung liegt das Problem zugrunde, ein Verfahren anzugeben, das demgegenüber verbessert ist und das es ermöglicht, die Systemperformance zu verbessern.The invention is based on the problem of providing a method which is improved in comparison and which makes it possible to improve the system performance.
Zur Lösung dieses Problems ist bei einem Verfahren der eingangs genannten Art erfindungsgemäß vorgesehen, dass zur Bestimmung eines in wenigstens einem Kamerabild gegebenen informationsrelevanten Bildbereichs seitens der Steuerungseinrichtung ein in dem Kamerabild gezeigter, einen Teil des Fahrzeugs zeigender Bildbereich im Kamerabild selbst ermittelt wird, welcher Bildbereich im Rahmen der Verarbeitung nachfolgend aufgenommener Kamerabilder außer Betracht bleibt.To solve this problem, the method according to the invention provides that for determining an information-relevant image area given in at least one camera image by the control device, an image area shown in the camera image and showing part of the vehicle is determined in the camera image itself, which image area in FIG The processing of subsequently recorded camera images is disregarded.
Die Erfindung sieht vor, den aus dem Gesamtbild "auszuschneidenden" Bildbereich, in dem der Fahrzeugteil gezeigt ist, in einem oder mehreren aktuell aufgenommenen Kamerabild selbst zu ermitteln. Dies ermöglich es, den nicht zu berücksichtigenden Bildbereich exakt so zu bemessen, wie er sich tatsächlich im jeweiligen Kamerabild darstellt. Denn die Ist-Darstellung zeigt die reale konstruktive Situation, so dass exakt der Bildbereich unterdrückt werden kann, in dem das Fahrzeugteil gezeigt ist, nicht aber ein zusätzlicher, aus Toleranzgründen berücksichtigter Bildbereich.The invention provides to determine the image area to be "cut out" of the overall image, in which the vehicle part is shown, in one or more currently recorded camera image itself. This makes it possible to dimension the unobservable image area exactly as it actually appears in the respective camera image. For the actual representation shows the real constructive situation, so that exactly the image area can be suppressed, in which the vehicle part is shown, but not an additional, tolerances reasons considered image area.
Hieraus resultiert, dass der informationsrelevante Bildbereich genau dem Bildbereich entspricht, der tatsächlich Informationen enthält, die seitens der Steuerungseinrichtung erfasst respektive ausgewertet werden können. Da dieser informationsrelevante Bildbereich nicht aus Sicherheitsgründen respektive Toleranzgründen prophylaktisch beschnitten ist, steht folglich ein größerer Bildbereich zur Auswertung zur Verfügung, so dass die Systemperformance verbessert werden kann.The result of this is that the information-relevant image area corresponds exactly to the image area, which actually contains information that can be detected or evaluated by the control device. There Consequently, this information-relevant image area is not prone to be prophylactically restricted for reasons of safety or tolerance, a larger image area is therefore available for evaluation, so that the system performance can be improved.
In Weiterbildung der Erfindung kann vorgesehen sein, dass die Ermittlung des einen Teil des Fahrzeugs zeigenden Bildbereichs anhand eines in dem oder den hierzu verarbeitenden Kamerabildern gezeigten Musters oder Objekt, das im informationsrelevanten Bildbereich sichtbar und in dem einen Teil des Fahrzeugs zeigenden Bildbereich verdeckt ist, erfolgt, welches Muster oder Objekt seitens der Steuerungseinrichtung ermittelt wird. Die Steuerungseinrichtung verarbeitet das oder die Kamerabilder unter Verwendung eines entsprechenden Algorithmus, der ein Muster oder ein Objekt erfasst, das im informationsrelevanten Bereich sichtbar ist. Dieses Objekt verläuft bis zur Trennlinie respektive Kante, in der der informationsrelevante Bildbereich in den anderen, das Fahrzeugteil zeigenden Bildbereich übergeht, das heißt, dass das Muster oder das Objekt dann nach dem Übergang von dem Fahrzeugteil verdeckt ist. Die Steuerungseinrichtung ist nun in der Lage, ein solches Muster oder ein solches Objekt zu erfassen und so einen Punkt oder Bereich zu detektieren, an dem der eine Bildbereich in den anderen übergeht. Anhand der Geometrie des Fahrzeugteils, also beispielsweise der Motorhaube, die in ihren Konstruktionsdaten steuerungseinrichtungsseitig bekannt ist, kann nun der genaue Verlauf der Kante erfasst werden, so dass der Übergangsbereich exakt definiert werden kann. Alternativ zur Ermittlung auf Basis der Geometriedaten des Fahrzeugteils ist es auch denkbar, dass die Steuerungseinrichtung über einen geeigneten Kantendetektionsalgorithmus oder dergleichen den Verlauf der Kante respektive des Bildbereichsübergangs ermitteln kann.In a further development of the invention, it can be provided that the determination of the image area showing a part of the vehicle takes place on the basis of a pattern or object shown in the camera image or images that is visible in the information-relevant image area and hidden in the image area showing part of the vehicle which pattern or object is determined by the controller. The controller processes the camera image (s) using a corresponding algorithm that detects a pattern or object that is visible in the information-relevant area. This object extends as far as the dividing line or edge in which the information-relevant image area merges into the other image area showing the vehicle part, that is to say that the pattern or the object is then obscured after the transition from the vehicle part. The controller is now able to detect such a pattern or object and thus detect a point or area where the one image area merges into the other. Based on the geometry of the vehicle part, so for example the hood, which is known in their design data control device side, now the exact course of the edge can be detected, so that the transition region can be defined exactly. Alternatively to the determination on the basis of the geometry data of the vehicle part, it is also conceivable that the control device can determine the course of the edge or the image area transition via a suitable edge detection algorithm or the like.
Das Muster oder das Objekt kann entweder am Fahrzeug selbst angeordnet sein, beispielsweise in dem es auf die Motorhaube aufgebracht oder gelegt wird respektive im Kantenbereich angebracht wird. Die Steuerungseinrichtung ermittelt nun mit dem geeigneten Analysealgorithmus dieses Muster oder Objekt und kann aus den gewonnenen Informationen die Trennlinie der Bildbereiche respektive den Verlauf der Fahrzeugteilkante bestimmen.The pattern or the object can either be arranged on the vehicle itself, for example, in that it is applied to the bonnet or placed or is attached in the edge region. The controller now determines this pattern with the appropriate analysis algorithm or object and can determine from the information obtained, the dividing line of the image areas, respectively, the course of the vehicle edge part.
Alternativ zur Anordnung des Musters oder des Objekts am Fahrzeug selbst ist es auch denkbar, dass sich das Muster oder Objekt vor dem Fahrzeug befindet. Das Fahrzeug wird in eine entsprechend zu definierende Position vor diesem Muster oder Objekt gebracht, wonach die Steuerungseinrichtung das oder die Kamerabilder wiederum analysiert und zunächst das Muster oder Objekt im Bild ermittelt, und anschließend den Punkt oder Bereich, in dem das Muster oder Objekt beim Bildbereichsübergang "verschwindet". Das heißt, dass das Fahrzeugteil im Bild dort beginnt, wo es nicht mehr gelingt, das Muster oder Objekt zu erkennen.Alternatively to the arrangement of the pattern or the object on the vehicle itself, it is also conceivable that the pattern or object is located in front of the vehicle. The vehicle is placed in a position to be defined in front of this pattern or object, after which the controller analyzes the camera image (s) again and first determines the pattern or object in the image, and then the point or area in which the pattern or object is at the image area transition "disappears". This means that the vehicle part begins in the image where it is no longer possible to recognize the pattern or object.
Während der Aufnahme des oder der Kamerabilder kann das Muster oder Objekt und das Fahrzeug positionsfest sein, das heißt, dass beide statisch sind. Sie bewegen sich also nicht relativ zueinander. Alternativ ist es denkbar, dass sich das Muster oder Objekt relativ zum Fahrzeug oder das Fahrzeug relativ zum Muster oder Objekt bewegt. Ein Element bewegt sich also, während das andere Element positionsfest ist. Hier werden folglich eine Reihe von Kamerabildern aufgenommen, die das sich - unabhängig davon, welches Element sich bewegt - relativ zum Fahrzeug bewegende Muster oder Objekt zeigen. Es handelt sich in diesem Fall also um ein dynamisches Verfahren.During capture of the camera image or images, the pattern or object and the vehicle may be fixed in position, that is, both are static. So they do not move relative to each other. Alternatively, it is conceivable that the pattern or object moves relative to the vehicle or the vehicle relative to the pattern or object. One element thus moves, while the other element is fixed in position. Hereby, a series of camera images are taken which show the pattern or object moving relative to the vehicle, regardless of which element is moving. It is a dynamic process in this case.
Bei einem sich relativ zum Fahrzeug bewegenden Muster oder Objekt kann das Muster oder Objekt im Wesentlichen horizontal bewegt werden. Das heißt, dass es, beispielsweise über eine Linearführung auf dem Boden geführt, relativ zum Fahrzeug, bevorzugt natürlich in der Fahrzeuglängsachse, bewegt wird. Es kann hierbei sogar unter das Fahrzeug geführt werden. Alternativ zu einer quasi horizontalen Bewegung kann das Muster oder Objekt auch auf- und/oder abbewegt, oder horizontal quer entlang des Fahrzeugs bewegt werden. Hierbei wird das Muster oder Objekt beispielsweise vertikal oder seitlich an einer Führung entlang bewegt, so dass es zwangsläufig im Laufe der Bewegung, beispielsweise von oben kommend, vom Fahrzeugteil verdeckt wird. Die jeweilige Bewegung ist in den Kamerabildern gezeigt, so dass der Ort, wo das Muster oder Objekt quasi hinter dem Fahrzeugteil "verschwindet", exakt erfasst werden kann. Auch eine horizontale, quer entlang der Fahrerseite erfolgende Bewegung ist denkbar, auch dabei ändert sich der in den nacheinander aufgenommen Bildern sichtbare Teil des Musters oder Objekts.In a pattern or object moving relative to the vehicle, the pattern or object may be moved substantially horizontally. This means that it is guided, for example via a linear guide on the ground, relative to the vehicle, preferably of course in the vehicle longitudinal axis, is moved. It can even be guided under the vehicle. As an alternative to a quasi-horizontal movement, the pattern or object can also be moved up and / or down, or moved horizontally across the vehicle. In this case, the pattern or object is moved, for example, vertically or laterally along a guide, so that it inevitably comes in the course of the movement, for example from above, from the vehicle part is covered. The respective movement is shown in the camera images, so that the place where the pattern or object "disappears" behind the vehicle part, can be detected exactly. A horizontal movement taking place transversely along the driver's side is also conceivable, and the part of the pattern or object visible in the successive images also changes.
Bewegt sich das Fahrzeug relativ zum Muster oder Objekt, so kann das Muster oder Objekt wiederum am Boden liegend positioniert sein, während das Fahrzeug das Muster oder Objekt überfährt, so dass sichergestellt ist, das es zwingend verschwindet.When the vehicle is moving relative to the pattern or object, the pattern or object may in turn be positioned on the ground as the vehicle passes over the pattern or object, thus ensuring that it necessarily disappears.
Das Muster oder Objekt kann beispielsweise ein Referenzmuster sein, beispielsweise in Form eines Schachbrettmusters, das auf eine Fläche gedruckt ist, oder eine einfache schwarz/weiß-Fläche mit einer definierten Kante. Alternativ kann das Referenzmuster auch als Lichtsäule ausgeführt sein oder ähnliches. In jedem Fall handelt es sich um ein extra für diesen Zweck vorgesehenes und entsprechend positioniertes Muster oder Objekt.The pattern or object may be, for example, a reference pattern, for example in the form of a checkerboard pattern printed on a surface, or a simple black and white surface with a defined edge. Alternatively, the reference pattern may also be designed as a light column or the like. In any case, it is a specially provided for this purpose and appropriately positioned pattern or object.
Alternativ dazu kann es sich bei dem Muster oder Objekt auch um ein natürliches, im Fahrzeugvorfeld oder Fahrzeugrückfeld befindliches Muster oder Objekt handeln, das seitens der Steuerungseinrichtung bestimmt wird. Als ein solches Muster wird bevorzugt ein solches bestimmt, dass eine im Bild sichtbare Kantenstruktur aufweist. Denkbar ist beispielsweise eine vertikale Säule, ein Balken, ein Möbel oder dergleichen, wie sie beispielsweise in einer Werkstatt oder ähnlichem, wo zu Einrichtungszwecken die Bildbereichsermittlung respektive Kalibrierung vorgenommen wird. Diese Alternative ist von Vorteil, als keine speziellen Muster oder Objekte vorgehalten werden müssen.Alternatively, the pattern or object may also be a natural pattern or object located in the vehicle apron or vehicle rear field, which is determined by the control device. As such a pattern is preferably determined such that has an edge structure visible in the image. It is conceivable, for example, a vertical column, a beam, a piece of furniture or the like, such as those in a workshop or the like, where for image purposes, the image area determination or calibration is made. This alternative is advantageous in that no special patterns or objects need to be kept.
Die Erkennung der Haube oder einer anderen Sicht verdeckenden Einheit kann auch über eine zeitliche Betrachtung der Bilder erfolgen, also einem Vergleich mehrerer zeitlich nacheinander bzw. zu verschiedenen Zeitpunkten aufgenommener Bilder der selben verbauten Kamera gegeneinander - der Bereich mit z. B. der Motorhaube unterscheidet sich weniger als andere Bereiche.The detection of the hood or a unit obscuring another view can also take place via a temporal viewing of the images, that is to say a comparison of a plurality of images of the same built - in camera taken one after the other or at different times Area with z. B. the hood is less different than other areas.
Neben dem Verfahren selbst betrifft die Erfindung ferner ein Kraftfahrzeug, umfassend eine Kamera mit zugeordneter Steuerungseinrichtung, welches Kraftfahrzeug zur Durchführung des erfindungsgemäßen Verfahrens ausgebildet ist.In addition to the method itself, the invention further relates to a motor vehicle, comprising a camera with associated control device, which motor vehicle is designed to carry out the method according to the invention.
Weitere Vorteile und Einzelheiten der Erfindung ergeben sich aus den im Folgenden beschriebenen Ausführungsbeispielen sowie anhand der Zeichnungen:
- Fig. 1
- eine Prinzipdarstellung eines erfindungsgemäßen Kraftfahrzeugs,
- Fig. 2
- eine Prinzipdarstellung eines Kamerabilds mit darin gezeigtem Referenzmuster,
- Fig. 3
- eine Prinzipdarstellung eines Verfahrensausführung mit vertikal beweglichem Referenzmuster,
- Fig. 4
- eine Prinzipdarstellung einer Verfahrensausführung mit horizontal beweglichem Referenzmuster und
- Fig. 5
- eine Prinzipdarstellung einer Verfahrensausführung mit statischem Referenzmuster.
- Fig. 1
- a schematic diagram of a motor vehicle according to the invention,
- Fig. 2
- a schematic diagram of a camera image with reference pattern shown therein,
- Fig. 3
- a schematic diagram of a method embodiment with a vertically movable reference pattern,
- Fig. 4
- a schematic representation of a process execution with horizontally movable reference pattern and
- Fig. 5
- a schematic representation of a process execution with static reference pattern.
Die Kamera 4 nimmt wie beschrieben das Fahrzeugvorfeld auf. Im Erfassungsbereich der Kamera befindet sich auch ein Teil der Motorhaube 6. Der Bildbereich, der die Motorhaube 6 zeigt, kann zwangsläufig keine informationsrelevanten Inhalte aufweisen. Es ist daher zu ermitteln, wo genau die Grenze zwischen dem Bildbereich, der informationsrelevante Informationen beinhaltet, und dem Bildbereich, der das Fahrzeugteil, hier die Motorhaube 6 zeigt, verläuft.The
Die Steuerungseinrichtung 3 ist nun in der Lage, die Grenzlinie 11, wo der eine Bildbereich 9 in den anderen Bildbereich 10 übergeht, zu ermitteln. Dies geschieht durch Analyse des Kamerabildes 7 zur Erfassung des Referenzmusters 8, das hier schachbrettartig in Schwarz-Weiß-Felder unterteilt ist. Die Steuerungseinrichtung 3 ermittelt nun genau den Bereich 12, in dem die regelmäßige Struktur, nämlich das Referenzmuster 8, vom oberen Bildbereich 9 kommend quasi verschwindet. An bzw. ab genau an diesem Bildbereich 12 wird das Referenzmuster 8 folglich durch die Motorhaube 6 verdeckt.The
Ist die Bildzeile oder der Bildpunkt bekannt, wo genau dieser Übergangsbereich 12 ist, kann die Steuerungseinrichtung nun den restlichen Verlauf der Motorhaubenkante, also der Linie 11 ermitteln. Dies kann entweder dadurch geschehen, dass auch bekannte Geometriedaten der Motorhaube zurückgegriffen wird, die seitens der Steuerungseinrichtung 3 vorliegen. Denn die Motorhaube 6 hat einen definierten, charakteristischen Verlauf. Aus diesem definierten Verlauf kann nun der gesamte Verlauf der Linie 11, beginnend ab dem Bereich 12, ermittelt werden. Alternativ ist es denkbar, den Verlauf der Linie 11, die ja eine scharfe, optische Trennung der Bildbereiche 9 und 10 definiert, über einen entsprechenden Kantendetektionsalgorithmus wiederum durch Bildanalyse zu ermitteln. Schließlich könnte, wenn die Grenzlinie aufgrund der Geometrie der Motorhaube 6 quasi geradlinig ist, einfach die ermittelte Bildzeile des Bereichs 12 als Trennungslinie der Bildbereiche 9 und 10 verwendet werdenIf the image line or the pixel is known exactly where this
Unabhängig davon kann die Steuerungseinrichtung 3 nun den informationsrelevanten Bildbereich 9 definieren, so dass im Rahmen einer nachfolgenden Verarbeitung des Bildinhalts des Bildbereichs 9 nur aus diesem die Informationen gewonnen werden können, die nachfolgend zur Ansteuerung der Gerätschaften 5 verwendet werden. Eine Beeinflussung der Bildanalyse durch den Bildbereich 10 ist ausgeschlossen, da dieser Bildbereich im Rahmen der Verarbeitung berücksichtigt bleibt. Es wird folglich nahezu der maximale relevante Bildbereich 9 der Analyse zugrunde gelegt, so dass größt mögliche Informationstiefe erfasst werden kann. Gleichzeitig wird die Analyse des Bildbereichs 6 vermieden, was zeitaufwendig und unnötig ist, da darin keine relevanten Informationen enthalten sind.Regardless of this, the
Bei der Ausgestaltung nach
Bei der Ausführungsform gemäß
Die Steuerungseinrichtung 3 umfasst nun im Kamerabild das Referenzmuster 8 und damit den Bereich 8a und kann so exakt die Bildbereiche 9 und 10 definieren und voneinander unterscheiden.The
Die Ausführungsform gemäß
Anstelle der Bewegung des Referenzmusters 8 ist es natürlich auch möglich, dass das Kraftfahrzeug 1 relativ zum positionsfesten Referenzmuster zu bewegen.Of course, instead of the movement of the
Die vorgenommene Bestimmung der Lage der Linie 11, also die Definition des Bildbereichs 9 und 10 kann einmalig werkseitig erfolgen, wenn das System einmalig eingerichtet respektive kalibriert wird. Es ist aber auch möglich, diese Kalibrierung regelmäßig bei Werkstattbesuchen vorzunehmen.The determination made of the position of the
Claims (11)
dadurch gekennzeichnet,
dass zur Bestimmung eines in wenigstens einem Kamerabild (7) gegebenen informationsrelevanten Bildbereichs (9) seitens der Steuerungseinrichtung (3) ein in dem Kamerabild gezeigter, einen Teil des Fahrzeugs (1) zeigenden Bildbereich (10) im Kamerabild (7) selbst ermittelt wird, welcher Bildbereich (10) im Rahmen der Verarbeitung nachfolgend aufgenommener Kamerabilder (7) außer Betracht bleibt.Method for operating a camera-based vehicle system, comprising a camera (4), which is arranged in or on the vehicle for receiving the vehicle surroundings, and a control device (3) processing the camera images,
characterized,
in that for determining an information-relevant image area (9) given in at least one camera image (7), an image area (10) shown in the camera image and showing a part of the vehicle (1) is determined in the camera image (7) itself, which image area (10) is disregarded as part of the processing of subsequently recorded camera images (7).
dadurch gekennzeichnet,
dass die Ermittlung des einen Teil des Fahrzeugs (1) zeigenden Bildbereichs (10) anhand eines in dem oder den zu verarbeitenden Kamerabildern (7) gezeigten Musters oder Objekts (8) das im informationsrelevanten Bildbereich (9) sichtbar und im des Teil des Fahrzeugs (1) zeigenden Bildbereich (10) verdeckt ist, erfolgt, welches Muster oder Objekt (8) seitens der Steuerungseinrichtung (3) ermittelt wird.Method according to claim 1,
characterized,
in that the image area (10) showing a part of the vehicle (1) is visible in the information-relevant image area (9) and in the part of the vehicle (in the form of a pattern or object (8) shown in the camera image (s) to be processed. 1), the pattern or object (8) is detected by the control device (3).
dadurch gekennzeichnet,
dass das Muster oder Objekt (8) am Fahrzeug (1) angeordnet ist, oder sich vor dem Fahrzeug (1) befindet.Method according to claim 2,
characterized,
that the pattern or object (8) is arranged on the vehicle (1), or in front of the vehicle (1).
dadurch gekennzeichnet,
dass während der Aufnahme der Kamerabilder (7) das Muster oder Objekt (8) und das Fahrzeug (1) positionsfest sind, oder dass sich das Muster oder Objekt (8) relativ zum Fahrzeug (1) oder das Fahrzeug (1) relativ zum Muster oder Objekt (8) bewegt.Method according to claim 3,
characterized,
that during the taking of the camera images (7) the pattern or object (8) and the vehicle (1) are fixed in position, or that the Moving pattern or object (8) relative to the vehicle (1) or the vehicle (1) relative to the pattern or object (8).
dadurch gekennzeichnet,
dass bei sich relativ zum Fahrzeug (1) bewegendem Muster oder Objekt (8) das Muster oder Objekt (8) im Wesentlichen horizontal bewegt und gegebenenfalls unter das Fahrzeug (1) geführt wird, oder dass das Muster oder Objekt (8) auf- und/oder abbewegt wird.Method according to claim 4,
characterized,
that when relative to the vehicle (1) moving pattern or object (8), the pattern or object (8) to move substantially horizontally and optionally with the vehicle (1) is guided, or that the pattern or object (8) up and / or is moved.
dadurch gekennzeichnet,
dass bei sich relativ zum Muster oder Objekt (8) zu bewegendem Fahrzeug (1) das Fahrzeug (1) das Muster oder Objekt (8) überfährt.Method according to claim 4,
characterized,
that in relative to the sample or object (8) to moving the vehicle (1) the vehicle (1) passes over the pattern or object (8).
dadurch gekennzeichnet,
dass das Muster oder Objekt ein Referenzmuster (8) ist.Method according to one of the preceding claims,
characterized,
that the pattern or object is a reference pattern (8).
dadurch gekennzeichnet,
dass das Muster oder Objekt ein natürliches, im Fahrzeugvorfeld oder Fahrzeugrückfeld befindliches Muster oder Objekt ist, das seitens der Steuerungseinrichtung (3) bestimmt wird.Method according to one of claims 1 - 6,
characterized,
in that the pattern or object is a natural pattern or object located in the vehicle apron or vehicle return field, which is determined by the control device (3).
dadurch gekennzeichnet,
dass als Muster oder Objekt ein solches bestimmt wird, das eine im Bild sichtbare Kantenstruktur aufweist.Method according to claim 8,
characterized,
that a pattern or object is determined which has an edge structure visible in the image.
dadurch gekennzeichnet,
das der im Kamerabild gezeigte Teil des Kraftfahrzeugs durch einen Vergleich zeitlich nacheinander aufgenommener Bilder ermittelt wird.Method according to claim 1,
characterized
the part of the motor vehicle shown in the camera image is determined by a comparison of temporally successively recorded images.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102015007391.5A DE102015007391A1 (en) | 2015-06-10 | 2015-06-10 | Method for operating a camera-based vehicle system |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3103683A1 true EP3103683A1 (en) | 2016-12-14 |
EP3103683B1 EP3103683B1 (en) | 2018-08-15 |
Family
ID=56116179
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16001285.2A Active EP3103683B1 (en) | 2015-06-10 | 2016-06-08 | Method for operating a camera-based vehicle system |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP3103683B1 (en) |
DE (1) | DE102015007391A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102017220492B4 (en) | 2017-11-16 | 2019-08-14 | Audi Ag | Method and device for verifying sensor data in an environment of a vehicle |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0691599A2 (en) * | 1994-07-05 | 1996-01-10 | Hitachi, Ltd. | Environment recognition device provided with image pickup device |
DE102005003254A1 (en) * | 2005-01-24 | 2006-08-10 | Sick Ag | Optoelectronic sensor for protecting e.g. press brake, has locally resolving optoelectronic receiver, and evaluation and control unit coupled with another receiver, where operating parameter is identifiable by unit based on received signals |
DE102008028303A1 (en) * | 2007-06-15 | 2009-01-08 | Denso Corp., Kariya-shi | Display system and program |
DE102009059797A1 (en) * | 2009-12-21 | 2011-06-22 | Krauss-Maffei Wegmann GmbH & Co. KG, 80997 | Optical marker and object detection device |
DE102012103495A1 (en) * | 2012-03-29 | 2013-10-02 | Sick Ag | Optoelectronic device for measuring structure or object sizes and method for calibration |
DE102012214283A1 (en) * | 2012-08-10 | 2014-02-13 | Siemens Aktiengesellschaft | Method for supporting target recording area setting of imaging device e.g. X-ray device, involves adjusting recording area of imaging device based on detecting position of markers representing target recoding area of object |
WO2015041005A1 (en) * | 2013-09-19 | 2015-03-26 | 富士通テン株式会社 | Image generation device, image display system, image generation method, and image display method |
-
2015
- 2015-06-10 DE DE102015007391.5A patent/DE102015007391A1/en not_active Withdrawn
-
2016
- 2016-06-08 EP EP16001285.2A patent/EP3103683B1/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0691599A2 (en) * | 1994-07-05 | 1996-01-10 | Hitachi, Ltd. | Environment recognition device provided with image pickup device |
DE102005003254A1 (en) * | 2005-01-24 | 2006-08-10 | Sick Ag | Optoelectronic sensor for protecting e.g. press brake, has locally resolving optoelectronic receiver, and evaluation and control unit coupled with another receiver, where operating parameter is identifiable by unit based on received signals |
DE102008028303A1 (en) * | 2007-06-15 | 2009-01-08 | Denso Corp., Kariya-shi | Display system and program |
DE102009059797A1 (en) * | 2009-12-21 | 2011-06-22 | Krauss-Maffei Wegmann GmbH & Co. KG, 80997 | Optical marker and object detection device |
DE102012103495A1 (en) * | 2012-03-29 | 2013-10-02 | Sick Ag | Optoelectronic device for measuring structure or object sizes and method for calibration |
DE102012214283A1 (en) * | 2012-08-10 | 2014-02-13 | Siemens Aktiengesellschaft | Method for supporting target recording area setting of imaging device e.g. X-ray device, involves adjusting recording area of imaging device based on detecting position of markers representing target recoding area of object |
WO2015041005A1 (en) * | 2013-09-19 | 2015-03-26 | 富士通テン株式会社 | Image generation device, image display system, image generation method, and image display method |
Also Published As
Publication number | Publication date |
---|---|
DE102015007391A1 (en) | 2016-12-15 |
EP3103683B1 (en) | 2018-08-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1018839B2 (en) | Method and apparatus for inspecting the rear observation space of a vehicle | |
EP3177474B1 (en) | Method for generating a surroundings map, and driver assistance system | |
EP2753533B1 (en) | Determination of the position of structural elements of a vehicle | |
EP3011225B1 (en) | Device and method for securing a machine that operates in an automated manner | |
EP2920778B1 (en) | Method for carrying out an at least semi-autonomous parking process of a motor vehicle into a garage, parking assistance system and motor vehicle | |
DE102009015142B4 (en) | Vehicle surroundings recognition device and control system for tracking a preceding vehicle | |
EP1928687A1 (en) | Method and driver assistance system for sensor-based driving off control of a motor vehicle | |
DE102013114369B4 (en) | CAMERA MODULE FOR A VEHICLE AND SUBSEQUENT MONITORING SYSTEM | |
EP3335419B1 (en) | Vehicle camera device for capturing the surroundings of a motor vehicle and driver assistance device for detecting objects with such a vehicle camera device | |
DE102009046726A1 (en) | Method for detecting and selecting e.g. longitudinal parking spaces, for aiding driver of car for transporting persons or materials, involves displaying surfaces as possible parking spaces, and selecting suitable parking spaces | |
DE102006060893A1 (en) | Device and method for determining a free space in front of a vehicle | |
DE102018217746A1 (en) | Method for operating a driver assistance system of a motor vehicle and motor vehicle | |
DE102013021616A1 (en) | Motor vehicle and method for checking a calibration of a camera | |
EP2856755B1 (en) | Device and method for recording images of a vehicle underbody.. | |
WO2009077445A1 (en) | Method and apparatus for optically detecting an area surrounding a vehicle | |
DE102016010373A1 (en) | Method and device for detecting the opening state of a garage door | |
EP3103683B1 (en) | Method for operating a camera-based vehicle system | |
DE102011076795A1 (en) | Method for determining a pitching movement of a camera installed in a vehicle and method for controlling a light emission of at least one headlight of a vehicle | |
EP2275318B1 (en) | Method for determining whether a motor vehicle in front is changing lane | |
DE102015217049B4 (en) | Device and method for testing an optical sensor for a vehicle | |
DE102016218079B3 (en) | Camera for an assistance system of a vehicle, assistance system and vehicle | |
EP3421674A1 (en) | Method and device for imaging areas | |
DE102014017128A1 (en) | motor vehicle | |
EP3833576B1 (en) | Surveillance camera system | |
DE102021003474A1 (en) | Device for detecting a vehicle environment and vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20170614 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06T 7/00 20060101ALI20180411BHEP Ipc: B60R 1/00 20060101AFI20180411BHEP Ipc: G06K 9/78 20060101ALI20180411BHEP Ipc: G06K 19/06 20060101ALI20180411BHEP Ipc: G06K 9/80 20060101ALI20180411BHEP |
|
INTG | Intention to grant announced |
Effective date: 20180507 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH Ref country code: AT Ref legal event code: REF Ref document number: 1029361 Country of ref document: AT Kind code of ref document: T Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502016001625 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181116 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181115 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181215 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181115 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502016001625 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20190516 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20190630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190608 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190608 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190630 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190630 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181215 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20160608 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180815 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1029361 Country of ref document: AT Kind code of ref document: T Effective date: 20210608 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20210608 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230530 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230623 Year of fee payment: 8 Ref country code: DE Payment date: 20230630 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20240625 Year of fee payment: 9 |