EP3967030A1 - Verfahren zur erzeugung eines bildes einer fahrzeugumgebung und vorrichtung zur erzeugung eines bildes einer fahrzeugumgebung - Google Patents
Verfahren zur erzeugung eines bildes einer fahrzeugumgebung und vorrichtung zur erzeugung eines bildes einer fahrzeugumgebungInfo
- Publication number
- EP3967030A1 EP3967030A1 EP20727143.8A EP20727143A EP3967030A1 EP 3967030 A1 EP3967030 A1 EP 3967030A1 EP 20727143 A EP20727143 A EP 20727143A EP 3967030 A1 EP3967030 A1 EP 3967030A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- vehicle
- image
- hdr
- cameras
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000005540 biological transmission Effects 0.000 claims 1
- 238000013507 mapping Methods 0.000 description 4
- 238000002156 mixing Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20208—High dynamic range [HDR] image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the invention relates to a method and a device for generating an image of a vehicle environment.
- Driver assistance systems that support the driver in performing driving maneuvers.
- Some of these driver assistance systems contain camera surround view systems that allow the driver of the vehicle to see the surroundings of the vehicle.
- Such camera surround view systems include several vehicle cameras which deliver real images of the vehicle environment, which are combined in particular by a data processing unit of the camera surround view system to form an image of the surroundings of the vehicle. The image of the vehicle surroundings is then advantageously displayed to the driver on a display unit.
- Adjacent cameras of surround view systems usually have overlapping fields of view. Since more information is available in these areas than is required, the images in these areas are usually mixed by alpha blending. This can lead to strong transitions in the mixing areas if the brightness in the adjacent images varies too much.
- one known approach is to equalize the brightness in the images before mixing. This is done in particular through image harmonization. Image harmonization has some disadvantages, however. For example, it is computationally intensive; the sampling must be close to perfect; the brightness cannot be easily adjusted in some color formats to adjust; and the sample pattern must be random enough not to match a pattern in the images. Overall, the results are heavily dependent on the current situation.
- the object of the invention is to provide a method and a device with which the existing problems based on the differences in brightness between neighboring cameras can be eliminated.
- Embodiments are the subject of the subclaims.
- a device is the subject of the independent claim.
- the present invention relates to a method for generating an image of a vehicle environment, comprising the steps:
- the method according to the invention ensures, in particular, that the very complex and relatively error-prone image harmonization can be dispensed with, since in particular the differences in brightness in the individual images are eliminated.
- the multiple vehicle cameras each capture at least a partial area of the vehicle environment.
- the vehicle cameras or the viewing areas of the vehicle cameras point in particular in different directions, with adjacent vehicle cameras normally having viewing areas that overlap at least in certain areas.
- HDR images are understood to mean in particular images with a high dynamic range. These are also known to the person skilled in the art as high dynamic range images. Such images make it possible for large differences in brightness to be reproduced in great detail in the camera's field of view.
- a single, entire HDR image is made from the multiple, individual HDR images.
- the joining of the individual images to form the overall image works in a manner known to the person skilled in the art, preferably by stitching. This also applies in particular to the areas of overlap.
- LDR images are understood to mean in particular images with a low dynamic range. These are also known to the person skilled in the art as low dynamic range images.
- the LDR image is obtained in particular by compressing the dynamic range of the HDR image.
- the individual HDR images are transmitted to a control unit.
- the control unit preferably creates the entire HDR image from the multiple, individual HDR images. In particular finds that Merging of the individual HDR images into a single, complete HDR image takes place in the control unit.
- the calculation and / or generation of the individual HDR images takes place in the individual vehicle cameras.
- the calculation of the individual HDR images takes place outside the vehicle cameras.
- the raw data of the images or the image data can be sent to a device that can be located, for example, in the vehicle, but also outside the vehicle, where the calculation then takes place.
- the LDR image is calculated from the HDR image using tone mapping.
- the calculation is not limited to any specific tone mapping process.
- the use of global operators is basically conceivable.
- a function is used here which assigns a dynamically compressed value to each HDR value and which is then applied to each pixel.
- the pixels of the HDR image or the output image are processed independently of one another.
- the HDR image is linearly compressed; a factor is used for all pixels.
- the use of local or frequency-based operators is also possible.
- image regions with high contrast are compressed more strongly and image regions with low contrast less strongly.
- the neighborhood, in particular neighboring pixels is preferably included.
- each vehicle camera records several exposures per image.
- the individual HDR images are preferably reconstructed from an exposure series, advantageously from an exposure series of images with a low dynamic range, in particular low dynamic ranks.
- exposure series is understood to mean, in particular, a graduated series of exposures.
- the same motif is preferably imaged, in particular several times, with different exposure settings.
- Each image region is preferably correctly exposed in at least one of the individual images.
- the change in exposure can take place over the duration of the exposure time.
- the change in exposure can also be controlled via the f-number or by changing the sensitivity setting.
- Exposure series are advantageously recorded to calculate the individual HDR images.
- the entire LDR image is displayed on a display unit.
- the display unit can be a display, a display and / or a head-up display.
- the display unit can be part of the vehicle. It is also possible, however, for the display unit to be part of a mobile terminal.
- cameras of a surround view system supply the image data of the vehicle environment. These are in particular four cameras, which are ideally arranged on different sides of the vehicle.
- the cameras are preferably fisheye cameras.
- the invention relates to a device for generating an image of a vehicle environment, comprising:
- Vehicle cameras which are attached in particular to a vehicle body of a vehicle and are designed in such a way that they generate HDR images or can calculate HDR images, and
- control unit that is designed to control the multiple Combine individual HDR images into a complete HDR image and calculate an LDR image from this.
- the device according to the invention is preferably suitable for carrying out a method according to the invention.
- the device comprises a display unit which outputs the LDR image calculated by the control device.
- the vehicle cameras are advantageously designed to be identical.
- identical construction is to be understood in particular to the effect that the vehicle cameras have the same sensor and / or the same optics. It is advantageous if all vehicle cameras use the same values for exposure time (s), sensitivity (s), aperture, AD converter and / or ISP.
- the vehicle cameras are preferably designed as fisheye cameras.
- the vehicle cameras are arranged on different sides of the vehicle. It is advantageous if exactly four cameras are used, in particular a vehicle camera being arranged on a front side, a vehicle camera on a rear side, a vehicle camera on the right side and a vehicle camera on a left side of the vehicle body.
- the vehicle cameras are in particular cameras of a surround view system.
- FIG. 1 a schematic representation of a flow chart of a method according to the invention for generating an image of a vehicle environment in one embodiment
- Fig. 2 a schematic representation of an inventive
- FIG. 1 shows a schematic representation of a flow chart of a method according to the invention for generating an image of a vehicle environment in one embodiment.
- a first step S1 several vehicle cameras 12, 14,
- the cameras 12, 14, 16, 18 are arranged in particular on a vehicle body of a vehicle, the vehicle cameras 12, 14, 16, 18 or the viewing areas of the vehicle cameras 12, 14, 16, 18 preferably pointing in different directions. Adjacent vehicle cameras 12, 14, 16, 18 in particular partially have a
- a second step S2 individual HDR images are calculated and / or generated from image data or images from vehicle cameras 12, 14, 16, 18.
- the individual HDR images are preferably calculated in each of the individual vehicle cameras 12, 14, 16, 18.
- the individual HDR images are advantageously obtained in that they are reconstructed from an exposure series, preferably from an exposure series of images with a low dynamic range.
- each vehicle camera 12, 14, 16, 18 takes in particular several per image
- a third step S3 the multiple, individual HDR images are combined to form an overall HDR image.
- the individual HDR images can be combined in a control device 20.
- an image with a low dynamic range is generated from the entire HDR image. calculated.
- the LDR image is preferably calculated from the HDR image using tone mapping.
- the entire LDR image can then be displayed on a display unit 22.
- FIG. 2 shows a schematic representation of a device 10 according to the invention for generating an image of a vehicle environment in one embodiment.
- the device comprises several vehicle cameras 12, 14, 16, 18, which are designed such that they can generate HDR images or that HDR images can be calculated therefrom.
- the vehicle cameras 12, 14, 16, 18 are preferably attached to a vehicle body of a vehicle.
- the vehicle cameras 12, 14, 16, 18 are arranged on different sides of the vehicle.
- the vehicle cameras 12, 14, 16, 18 are ideally designed to be identical. In particular, they have the same sensor and / or the same optics.
- the vehicle cameras 12, 14, 16, 18 can be designed as fisheye cameras. It is advantageous if the vehicle cameras 12, 14, 16, 18 are cameras of a surround view system.
- the device 10 according to the invention further comprises a control device 20.
- the control device 20 is designed to combine the multiple, individual HDR images to form an overall HDR image and then to calculate an LDR image therefrom.
- the device 10 according to the invention can also include a display unit 22 which outputs or displays the LDR image calculated by the control device 20.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Closed-Circuit Television Systems (AREA)
- Studio Devices (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
- Human Computer Interaction (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102019206581.3A DE102019206581A1 (de) | 2019-05-08 | 2019-05-08 | Verfahren zur Erzeugung eines Bildes einer Fahrzeugumgebung und Vorrichtung zur Erzeugung eines Bildes einer Fahrzeugumgebung |
PCT/DE2020/200030 WO2020224730A1 (de) | 2019-05-08 | 2020-05-05 | Verfahren zur erzeugung eines bildes einer fahrzeugumgebung und vorrichtung zur erzeugung eines bildes einer fahrzeugumgebung |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3967030A1 true EP3967030A1 (de) | 2022-03-16 |
Family
ID=70779404
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP20727143.8A Pending EP3967030A1 (de) | 2019-05-08 | 2020-05-05 | Verfahren zur erzeugung eines bildes einer fahrzeugumgebung und vorrichtung zur erzeugung eines bildes einer fahrzeugumgebung |
Country Status (6)
Country | Link |
---|---|
US (1) | US20220215668A1 (ja) |
EP (1) | EP3967030A1 (ja) |
JP (1) | JP2022529657A (ja) |
CN (1) | CN113812144A (ja) |
DE (1) | DE102019206581A1 (ja) |
WO (1) | WO2020224730A1 (ja) |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4325642B2 (ja) * | 2006-05-31 | 2009-09-02 | ソニー株式会社 | 移動体カメラシステム |
JP4869795B2 (ja) * | 2006-06-06 | 2012-02-08 | 富士フイルム株式会社 | 撮像制御装置、撮像システム、および撮像制御方法 |
JP5182042B2 (ja) * | 2008-11-28 | 2013-04-10 | 富士通株式会社 | 画像処理装置、画像処理方法及びコンピュータプログラム |
JP5409577B2 (ja) * | 2010-10-05 | 2014-02-05 | 株式会社ソニー・コンピュータエンタテインメント | パノラマ画像生成装置およびパノラマ画像生成方法 |
CN105469375B (zh) * | 2014-08-28 | 2021-09-07 | 北京三星通信技术研究有限公司 | 处理高动态范围全景图的方法和装置 |
DE102014118314A1 (de) * | 2014-12-10 | 2016-06-16 | Connaught Electronics Ltd. | Verfahren zum Anpassen einer Helligkeit eines Hochkontrastbilds, Kamerasystem und Kraftfahrzeug |
US9418408B1 (en) * | 2015-07-22 | 2016-08-16 | Rockwell Collins, Inc. | Dynamic range optimization |
US20170090194A1 (en) * | 2015-09-24 | 2017-03-30 | Halo Augmented Reality Ltd. | System And Method For Subtractive Augmented Reality And Display Contrast Enhancement |
WO2017205492A1 (en) * | 2016-05-25 | 2017-11-30 | Gopro, Inc. | Three-dimensional noise reduction |
WO2018031441A1 (en) * | 2016-08-09 | 2018-02-15 | Contrast, Inc. | Real-time hdr video for vehicle control |
CN109923580B (zh) * | 2016-11-04 | 2023-12-15 | 特利丹菲力尔有限责任公司 | 用于热视频的动态范围压缩 |
JP7122729B2 (ja) * | 2017-05-19 | 2022-08-22 | 株式会社ユピテル | ドライブレコーダー、ドライブレコーダー用表示装置及びプログラム |
JP2019023787A (ja) * | 2017-07-24 | 2019-02-14 | 株式会社デンソー | 画像認識システム |
CN111226256A (zh) * | 2017-11-09 | 2020-06-02 | 深圳市大疆创新科技有限公司 | 用于图像动态范围调整的***和方法 |
EP3672267A1 (en) * | 2018-12-20 | 2020-06-24 | InterDigital VC Holdings, Inc. | Methods for processing audio and/or video contents and corresponding signal, devices, electronic assembly, system, computer readable program products and computer readable storage media |
US11150664B2 (en) * | 2019-02-01 | 2021-10-19 | Tesla, Inc. | Predicting three-dimensional features for autonomous driving |
US10956755B2 (en) * | 2019-02-19 | 2021-03-23 | Tesla, Inc. | Estimating object properties using visual image data |
-
2019
- 2019-05-08 DE DE102019206581.3A patent/DE102019206581A1/de active Pending
-
2020
- 2020-05-05 JP JP2021561823A patent/JP2022529657A/ja active Pending
- 2020-05-05 US US17/595,073 patent/US20220215668A1/en active Pending
- 2020-05-05 WO PCT/DE2020/200030 patent/WO2020224730A1/de unknown
- 2020-05-05 EP EP20727143.8A patent/EP3967030A1/de active Pending
- 2020-05-05 CN CN202080033410.XA patent/CN113812144A/zh active Pending
Also Published As
Publication number | Publication date |
---|---|
JP2022529657A (ja) | 2022-06-23 |
CN113812144A (zh) | 2021-12-17 |
DE102019206581A1 (de) | 2020-11-12 |
WO2020224730A1 (de) | 2020-11-12 |
US20220215668A1 (en) | 2022-07-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2623374B1 (de) | Sichtsystem für Nutzfahrzeuge zur Darstellung von gesetzlich vorgeschriebenen Sichtfeldern eines Hauptspiegels und eines Weitwinkelspiegels | |
DE102006036933B4 (de) | Verfahren zur Erzeugung eines Gesamtbilds aus überlappenden Einzelbildern | |
WO2016162245A1 (de) | Verfahren zur darstellung einer fahrzeugumgebung eines fahrzeuges | |
DE112012004352B4 (de) | Stereobildgebungsvorrichtung | |
EP3607740B1 (de) | Kameravorrichtung und verfahren zur erfassung eines umgebungsbereichs eines fahrzeugs | |
DE102018108751A1 (de) | Method, system and device of obtaining 3D-information of objects | |
DE102019208338B4 (de) | Rundumsicht-überwachungssystem und verfahren zum betrieb davon | |
DE102011077398A1 (de) | Fahrzeugkamerasystem zur Bereitstellung eines lückenlosen Bildes der Fahrzeugumgebung und entsprechendes Verfahren | |
DE102011014368A1 (de) | Verfahren und Vorrichtung für ein bildgebendes Fahrerassistenzsystem | |
DE102014118314A1 (de) | Verfahren zum Anpassen einer Helligkeit eines Hochkontrastbilds, Kamerasystem und Kraftfahrzeug | |
DE102019105630A1 (de) | Anzeigesteuervorrichtung, Fahrzeugumgebungsanzeigesystem und Computerprogramm | |
DE102015105529A1 (de) | Verfahren zum Transformieren eines Bildes einer virtuellen Kamera, Computerprogrammprodukt, Anzeigesystem und Kraftfahrzeug | |
EP3073446B1 (de) | Verfahren zum darstellen einer fahrzeugumgebung eines fahrzeuges | |
DE102015208343B4 (de) | Verfahren zum Erzeugen eines Gesamtbildes einer Fahrzeugumgebung eines Fahrzeuges und entsprechende Vorrichtung | |
EP3967030A1 (de) | Verfahren zur erzeugung eines bildes einer fahrzeugumgebung und vorrichtung zur erzeugung eines bildes einer fahrzeugumgebung | |
DE102012025580A1 (de) | Verfahren zum Betreiben eines Kamerasystems mit einer Master-Kamera, Kamerasystemund Kraftfahrzeug | |
DE102017000307A1 (de) | Verfahren zum Kalibrieren einer Kamera für ein Kraftfahrzeug unter Berücksichtigung eines Kalibrierfehlers, Kamera sowie Kraftfahrzeug | |
EP3420533B1 (de) | Verfahren zum kalibrieren eines optischen messaufbaus | |
DE102010034127A1 (de) | Verfahren zum Anzeigen von Bildern auf einer Anzeigeeinrichtung in einem Kraftfahrzeug, Fahrerassistenzsystem und Kraftfahrzeug | |
DE102013011844A1 (de) | Verfahren zum Anpassen einer Gammakurve eines Kamerasystems eines Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug | |
DE102015204213A1 (de) | Verfahren zum Zusammensetzen von zwei Bildern einer Fahrzeugumgebung eines Fahrzeuges und entsprechende Vorrichtung | |
DE102014219428A1 (de) | Selbstkalibrierung eines Stereokamerasystems im Auto | |
DE102013020952A1 (de) | Verfahren zum Einstellen eines für die Helligkeit und/oder für den Weißabgleich einer Bilddarstellung relevanten Parameters in einem Kamerasystem eines Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug | |
DE102014209863A1 (de) | Verfahren und Vorrichtung zum Betreiben einer Stereokamera für ein Fahrzeug sowie Stereokamera für ein Fahrzeug | |
EP2816802A1 (de) | Verfahren und Vorrichtung zum Anpassen eines Bildparameters |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20211208 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: CONTINENTAL AUTONOMOUS MOBILITY GERMANY GMBH |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20220810 |