WO2014095782A1 - Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system - Google Patents

Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system Download PDF

Info

Publication number
WO2014095782A1
WO2014095782A1 PCT/EP2013/076800 EP2013076800W WO2014095782A1 WO 2014095782 A1 WO2014095782 A1 WO 2014095782A1 EP 2013076800 W EP2013076800 W EP 2013076800W WO 2014095782 A1 WO2014095782 A1 WO 2014095782A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
white balance
pixels
color
motor vehicle
Prior art date
Application number
PCT/EP2013/076800
Other languages
French (fr)
Inventor
Brian Michael Thomas DEEGAN
Patrick Eoghan Denny
Vladimir Zlokolica
Original Assignee
Connaught Electronics Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Connaught Electronics Ltd. filed Critical Connaught Electronics Ltd.
Publication of WO2014095782A1 publication Critical patent/WO2014095782A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • H04N23/84Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/88Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control

Definitions

  • the invention relates to a method for white balance of an image presentation displayed on a display device of a motor vehicle.
  • the image presentation is generated from a partial region of an image, which is provided by means of a camera of the motor vehicle.
  • For performing the white balance respective color values of pixels of the image are acquired, and a white balance parameter for the white balance is determined depending on the color values for at least one color channel of the image presentation.
  • the invention relates to a camera system formed for performing such a method, as well as to a motor vehicle with such a camera system.
  • the invention is not restricted to the provision of such a plan view as the image
  • the so-called “grey world assumption” conveys that the average values of all of the color channels of the image are to result in a grey value on average.
  • grey world assumption For example, if the image is considered in the RGB color space (red, green, blue), thus, the red, the green and the blue color components can be affected such that the color average values of all of the three color channels are the same. The color of the presented image then rather corresponds to reality.
  • the white balance In performing the white balance, thus, first the color average or median values of all of the color channels are calculated, and the color channels of the image are then each multiplied by a gain factor (white balance parameter) in order to match the color average values of the color channels to each other.
  • a gain factor white balance parameter
  • the white balance can be performed such that the average values of all of the color channels result in a grey value on average. This approach is referred to as grey world assumption.
  • multiple cameras can be attached to a motor vehicle, the images of which are collectively processed to a common image presentation, which is then displayed on the display.
  • a common image presentation which is then displayed on the display.
  • the above mentioned "bird eye view” can be generated as the image presentation.
  • the display Especially in such situations, in which only a partial region of the respective image of the cameras is displayed on the display, the
  • performance of the white balance presents a particular challenge. If all of the pixels of the image are taken into account in determining the color average values and thus in determining the gain factors of the color channels for the white balance of the displayed image presentation, thus, it can occur that a monochromatic object is depicted in a non- displayed image region of the image, which then causes increase of one of the color average values and thus a color cast in the displayed image presentation. Such a color cast then results in an unnatural presentation on the display.
  • this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims.
  • Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.
  • a method according to the invention serves for white balance of an image presentation displayed on a display device of a motor vehicle.
  • An image of an environmental region of the motor vehicle is provided by means of at least one camera of the motor vehicle.
  • the image presentation for display on the display device is generated exclusively from a partial region of the image.
  • a white balance parameter e.g. the associated color channel gain factor
  • For performing the white balance a subset of pixels of the image is selected according to a preset selection criterion, wherein the color values exclusively of the selected subset of pixels are taken into account in determining the white balance parameter for the at least one color channel.
  • a color channel gain factor is understood by the white balance parameter, which is determined for the at least one color channel of the image presentation based on color values of the selected subset of pixels.
  • a gain factor can also be separately determined for all of the color channels of the image presentation.
  • the respective color channels of the image presentation can be corrected such that for example the respective color average or median values of the color channels are the same (grey world assumption).
  • a color average or median value of this color channel is determined, namely from the color values of the selected subset of pixels.
  • This color average/median value can then be compared to the color average/median values of the other color channels, and depending on this comparison, the white balance parameter can be determined, with which the color channel is corrected. Therein, the determination of the color
  • the average/median values can be performed in any color space such that the invention is not restricted to a certain color space.
  • the color average/median values can be determined in the YUV color space or else in the RGB color space.
  • the at least one camera is preferably a video camera providing a sequence of images per second.
  • the partial region is respectively "cut out” from these images and used for generating the image presentation for the display device.
  • the image presentation is therefore generated in real time in the form of a video from the respectively current images.
  • the camera can for example be a CCD camera or a CMOS camera.
  • the subset of pixels is selected according to a preset selection criterion.
  • this selection criterion preferably, the position of the pixels in the image is taken into account.
  • those pixels are selected according to the selection criterion, which are within the partial region of the image, thus exclusively those pixels, which are displayed on the display device or contribute to the generation of the image presentation for the display device.
  • the white balance parameter thus, exclusively the color values of pixels are taken into account, which are within the partial region of the image, which is used for generating the image presentation for the display device.
  • color casts in the image presentation are avoided, which else could be caused by high color values of pixels, which are outside of the used partial region of the image.
  • an image presentation can be provided, which is free of such color casts and therefore has a homogeneous and uniform color distribution.
  • the respective color values of the pixels can also be taken into account in the selection criterion. This has the advantage that for example exclusively those pixels can be selected in determining the white balance parameter, which are very close to the grey value.
  • those pixels are preferably selected, the color values of which are within a preset range of values, in particular around the zero value.
  • determining the white balance parameter thus, exclusively pixels are taken into account, the color value amounts of which are below a preset limit value.
  • pixels remain unconsidered, which have relatively high color values and therefore can negatively affect the white balance algorithm. Namely, these pixels could excessively affect the determination of the respective color average or median value of the color channels and thus result in color casts in the image
  • At least a first and a second camera of the motor vehicle each provide an image of an environmental region of the motor vehicle.
  • the image presentation for display on the display device is then generated from a partial region of the image of the first camera as well as from a partial region of the image of the second camera.
  • the subset of pixels is selected according to the preset selection criterion both from pixels of the image of the first camera and from pixels of the image of the second camera.
  • pixels of the two images are taken into account, namely exclusively those pixels, which are selected from the image of the first camera as well as from the image of the second camera according to the mentioned selection criterion.
  • an image presentation can be provided, which has a uniform and realistic color distribution without color leaps and without color casts.
  • the white balance parameter it can occur that the number of the selected pixels within the subset is relatively low. This is in particular the case if exclusively those pixels are selected from the partial region of the image according to the selection criterion, the color values of which are around the zero value within the preset range of values. If a large monochromatic object is depicted in the partial region of the image, such that most of the pixels have relatively high color values, thus, the selected subset only includes a low number of pixels satisfying the above mentioned criterion. In this case, a reliable white balance based on this subset of few pixels is not possible.
  • the previously adjusted value of the white balance parameter is maintained or the white balance parameter is adjusted to a preset default value if the number of the pixels within the selected subset is below the preset limit value. Therein, it can be examined if the previously adjusted white balance parameter can be maintained or else adjustment of the white balance parameter to the preset default value is required.
  • the white balance parameter is adjusted to the preset default value if at least one of the mentioned conditions is satisfied, in particular if the two following conditions are satisfied at the same time:
  • a color average or median value of the at least one color channel determined from color values of all of the pixels of the partial region is outside of a preset range of values around the zero value, thus, its value is greater than a preset limit value, and/or
  • the white balance parameter (thus the gain factor of the at least one color channel) is adjusted to the preset default value, and the algorithm can be restarted because the number of the selected pixels will again exceed the limit value.
  • the algorithm can be restarted because the number of the selected pixels will again exceed the limit value.
  • a criterion can for example provide that a ratio between the gain factors of different color channels is determined. If this ratio then exceeds a preset limit value, thus, this indicates color faults in the image presentation. It is also possible that this detection of the color casts can also be performed by evaluating a gain offset of the color channels.
  • the adjustment of the white balance parameter to the preset default value too can be performed in different manners. This change of the white balance parameter can e.g. be performed stepwise such that the white balance parameter is stepwise adjusted to the preset default value via intermediate values. Direct adjustment to the default value is also possible without using intermediate values.
  • the invention also relates to a camera system for a motor vehicle including at least one camera for providing an image of an environmental region of the motor vehicle, including an image processing device for generating an image presentation for display on a display device from a partial region of the image, and including a correcting device for performing a white balance of the image presentation, wherein the correcting device is formed for performing a method according to the invention.
  • the invention relates to a motor vehicle with a camera system according to the invention.
  • FIG. 1 in schematic illustration a motor vehicle with a camera system according to an embodiment of the invention
  • FIG. 2 in schematic illustration a block diagram of the camera system according to an embodiment of the invention
  • Fig. 3 and 4 schematic illustrations of a color space, wherein a method according to an embodiment of the invention is explained in more detail.
  • a motor vehicle 1 illustrated in Fig. 1 is for example a passenger car.
  • the motor vehicle 1 includes a camera system 2, which has a plurality of cameras 3, 4, 5, 6 in the
  • a first camera 3 is for example disposed on the front bumper of the motor vehicle 1 .
  • a second camera 4 is e.g. disposed in the rear region, for instance on the rear bumper or on a tailgate.
  • the two lateral cameras 5, 6 can e.g. be integrated in the respective exterior mirror.
  • the cameras 3, 4, 5, 6 are electrically coupled to an image processing device 7, which in turn is coupled to a display device 8.
  • the display device 8 can be an LCD display.
  • the camera 3 captures an environmental region 9 in front of the motor vehicle 1 .
  • the camera 4 captures an environmental region 10 behind the motor vehicle 1.
  • the camera 5 captures a lateral environmental region 1 1 to the left beside the motor vehicle 1 , while the camera 6 captures an environmental region 12 to the right of the motor vehicle 1.
  • the cameras 3, 4, 5, 6 provide images of the respective environmental regions 9, 10, 1 1 , 12 and communicate these images to the image processing device 7.
  • the cameras 3, 4, 5, 6 are video cameras, which are each able to capture a sequence of images per time unit.
  • the cameras 3, 4, 5, 6 can also be so-called fish eye cameras, which have a relatively wide capturing angle or opening angle, for instance in a range of values from 150° to 200°.
  • the image processing device 7 From the images of the cameras 3, 4, 5, 6, the image processing device 7 generates an image presentation, which is then displayed on the display device 8. The white balance of this image presentation is also performed.
  • the white balance can be performed internally in the image processing device 7 in an embodiment.
  • the image processing device 7 can also be integrated in one of the cameras 3, 4, 5, 6.
  • a block diagram of the camera system 2 is illustrated in extremely abstract illustration.
  • the cameras 3, 4, 5, 6 communicate images B3, B4, B5, B6 to the image processing device 7.
  • the image processing device 7 generates the image presentation 13 from respective partial regions B3', B4', B5', B6' of the images B3, B4, B5, B6, which is presented on the display device 8.
  • This image presentation 13 can for example be a plan view presentation, which shows the motor vehicle 1 and its
  • This image presentation 13 is generated from the respective partial regions B3', B4', B5', B6', which show the respective
  • the cameras 3, 4, 5, 6 can communicate exclusively the partial regions B3', B4', B5', B6' to the image processing device 7, such that "cutting out” the partial regions B3', B4', B5', B6' is performed internally in the cameras 3, 4, 5, 6.
  • the image presentation 13 is then subjected to a white balance.
  • the white balance is performed by means of a correcting device 14, which can also be integrated in the image processing device 7.
  • the image presentation 13 is then communicated to the display device 8.
  • a situation is shown, in which the white balance is performed on the finished image presentation 13.
  • the white balance can already be performed before providing the image presentation 13 by subjecting the partial regions B3', B4', B5', B6' to the white balance.
  • the performance of the white balance of the image presentation 13 is explained in more detail.
  • the YUV color space is used.
  • the invention is not restricted to the processing of the pixels in this color space and can also be implemented in other color spaces.
  • the invention can be used in any color spaces isometric with the considered YUV space (e.g., HSV space).
  • pixels of the partial regions B3', B4', B5', B6' used for generating the image presentation 13 are taken into account.
  • the pixels of the remaining image regions are not taken into account.
  • the color values of each pixel can be described by a pair of values (U, V). From the pixels of the partial regions B3 ⁇ B4', B5 ⁇ B6 ⁇ now, a subset of pixels is selected, which satisfy a predetermined criterion. Namely, pixels are selected, the color values U, V of which are within a preset range of values around the zero value, thus, if the two following conditions are satisfied:
  • the limit values a and ⁇ can be identical values or else values different from each other.
  • the color values of the selected pixels are then used for calculating a color average or median value (below referred to as color average value) of the respective color channel.
  • the color average value of the U color channel is denoted by U s ; the color average value of the V color channel is denoted by V s .
  • the color average values U s und V s represent color temperatures of the two color channels.
  • the above mentioned criterion for selecting the subset of pixels can be illustrated with reference to Fig. 3.
  • the range of values from -a to a on the one hand and the range of values from - ⁇ to ⁇ on the other hand is defined around the origin of the UV color space.
  • Exclusively pixels are selected, the color values U, V of which come within the rectangle illustrated in Fig. 3.
  • the respective gain factor Gu and G v is determined for each color channel U and V.
  • the color channels can be corrected by multiplying the color values (U, V) of the pixels by the respective gain factor Gu, G v .
  • correction of the color average values U s and V s of the color channels is effected such that the color average values U s and V s are for example matched to each other.
  • the gain factors Gu, G v represent white balance parameters and are determined depending on the respective color average value U s and V s .
  • the number of the pixels is determined, the color values U, V of which satisfy the above mentioned criterion (come within the rectangle according to Fig. 3). If the number of the selected pixels is greater than a preset limit value, new gain factors Gu, G v are determined for the current image presentation depending on the current color average values U s and V s , respectively. If the number of the selected pixels is below the limit value, the following algorithm is performed:
  • color casts can for example arise in that the brightness of the scene is suddenly changed if for example a dominant light source with a high color temperature is switched off. Therein, with switched on light source, a relatively severe correction of the color temperature of the image presentation 13 is performed, and this severe correction causes color casts in the image presentation 13 after switching off the light source. In this case, it can occur that only very few pixels come within the rectangle according to Fig. 3. If the number of the selected pixels is smaller than the limit value, thus, it is examined whether or not such color casts are present.
  • color average values U T , V T are calculated from respective color values of all of the pixels of the partial regions B3', B4', B5', B6'. On the one hand, these color average values U T , V T are examined; on the other hand, the currently adjusted gain factors of the color channels are also examined.
  • the gain factors are adjusted to a preset standard value (so-called default value), namely either directly or stepwise via intermediate values.
  • the adjustment of the gain factors to the preset value is effected. Otherwise, the current gain factors can be maintained.
  • the calculation of new gain factors can be again restarted based on the current images because the number of the selected pixels will again exceed the limit value.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)
  • Processing Of Color Television Signals (AREA)

Abstract

The invention relates to a method for white balance of an image presentation (13), which is displayed on a display device (8) of a motor vehicle, wherein an image (B3, B4, B5, B6) of an environmental region of the motor vehicle is provided by means of at least one camera (3, 4, 5, 6) of the motor vehicle and the image presentation (13) for display on the display device (8) is generated from a partial region (Β3', Β4', Β5', Β6') of the image (B3, B4, B5, B6), and wherein for performing the white balance, respective color values of pixels of the image (B3, B4, B5, B6) are acquired and a white balance parameter for the white balance is determined depending on the color values for at least one color channel of the image presentation (13), wherein a subset of pixels of the image (B3, B4, B5, B6) is selected according to a preset selection criterion for performing the white balance, and that the color values exclusively of the selected subset of pixels are taken into account in determining the white balance parameter for the at least one color channel.

Description

Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system
The invention relates to a method for white balance of an image presentation displayed on a display device of a motor vehicle. The image presentation is generated from a partial region of an image, which is provided by means of a camera of the motor vehicle. For performing the white balance, respective color values of pixels of the image are acquired, and a white balance parameter for the white balance is determined depending on the color values for at least one color channel of the image presentation. In addition, the invention relates to a camera system formed for performing such a method, as well as to a motor vehicle with such a camera system.
Methods for performing a white balance of an image are already prior art. Presently, the interest is directed to a camera of a motor vehicle, which captures an environmental region of the motor vehicle. It is already known to attach a plurality of cameras to a motor vehicle, which capture the environment around the motor vehicle. Then, an image presentation can be displayed on a display of the motor vehicle, which is based on the images of the cameras. For example, the so-called "bird eye view" can here be provided from the images of all of the cameras, thus, a plan view presentation showing the motor vehicle and its environment from a bird's eye view. Such a plan view presentation is generated from respective partial regions of the images of all of the cameras, wherein that partial region of the image is respectively used, which images the respective
environmental region up to a predetermined distance from the motor vehicle. However, the invention is not restricted to the provision of such a plan view as the image
presentation. Other types of image presentations can also be generated from each one partial region of the images. At this point, three-dimensional views are exemplarily mentioned, which can involve a projection to a concave surface, such as for example to the surface of a hemisphere, a paraboloid, a hyperboloid or a similar concave surface.
In the known methods for automatic white balance of an image, usually, the so-called "grey world assumption" is used, which conveys that the average values of all of the color channels of the image are to result in a grey value on average. For example, if the image is considered in the RGB color space (red, green, blue), thus, the red, the green and the blue color components can be affected such that the color average values of all of the three color channels are the same. The color of the presented image then rather corresponds to reality.
In performing the white balance, thus, first the color average or median values of all of the color channels are calculated, and the color channels of the image are then each multiplied by a gain factor (white balance parameter) in order to match the color average values of the color channels to each other. By corresponding selection of the gain factors of the color channels, thus, the white balance can be performed such that the average values of all of the color channels result in a grey value on average. This approach is referred to as grey world assumption.
As already explained, multiple cameras can be attached to a motor vehicle, the images of which are collectively processed to a common image presentation, which is then displayed on the display. For example, the above mentioned "bird eye view" can be generated as the image presentation. Especially in such situations, in which only a partial region of the respective image of the cameras is displayed on the display, the
performance of the white balance presents a particular challenge. If all of the pixels of the image are taken into account in determining the color average values and thus in determining the gain factors of the color channels for the white balance of the displayed image presentation, thus, it can occur that a monochromatic object is depicted in a non- displayed image region of the image, which then causes increase of one of the color average values and thus a color cast in the displayed image presentation. Such a color cast then results in an unnatural presentation on the display.
It is an object of the invention to demonstrate a solution, how in a method of the initially mentioned kind the white balance of the image presentation on the display can be improved compared to the prior art such that in particular a realistic presentation of the environment can be presented on the display device in the motor vehicle to the driver.
According to the invention, this object is solved by a method, by a camera system as well as by a motor vehicle having the features according to the respective independent claims. Advantageous implementations of the invention are the subject matter of the dependent claims, of the description and of the figures.
A method according to the invention serves for white balance of an image presentation displayed on a display device of a motor vehicle. An image of an environmental region of the motor vehicle is provided by means of at least one camera of the motor vehicle. The image presentation for display on the display device is generated exclusively from a partial region of the image. For performing the white balance, respective color values of pixels of the image are acquired, and for at least one color channel of the image presentation, a white balance parameter (e.g. the associated color channel gain factor) for the white balance is determined depending on the color values. For performing the white balance, a subset of pixels of the image is selected according to a preset selection criterion, wherein the color values exclusively of the selected subset of pixels are taken into account in determining the white balance parameter for the at least one color channel.
For performing the white balance of the image presentation, thus, not all of the pixels of the captured image are taken into account, but a subset of pixels is selected according to a preset selection criterion. In determining the white balance parameter (e.g. the mentioned gain factor for a color channel), then, exclusively the selected pixels are taken into account, while the remaining pixels remain unconsidered. Such an approach proves particularly advantageous in particular if exclusively a partial region is selected from the provided image and displayed on the display device in the interior of the vehicle, optionally together with other partial regions of images provided by means of further cameras, as it is for example the case in the "bird eye view". By the method according to the invention, monochromatic objects depicted in the non-displayed image region can be prevented from causing a too great increase of a color average or median value and thus a color casts in the displayed image presentation. The selection of the pixels for the determination of the white balance parameter can be specifically performed such that exclusively color values of relevant pixels are taken into account, which allow a reliable white balance. Overall, the method according to the invention allows improved white balance of the displayed image presentation compared to the prior art.
Preferably, a color channel gain factor is understood by the white balance parameter, which is determined for the at least one color channel of the image presentation based on color values of the selected subset of pixels. Such a gain factor can also be separately determined for all of the color channels of the image presentation. With the respective gain factors (white balance parameters), the respective color channels of the image presentation can be corrected such that for example the respective color average or median values of the color channels are the same (grey world assumption).
For determining the white balance parameter for the at least one color channel, preferably, a color average or median value of this color channel is determined, namely from the color values of the selected subset of pixels. This color average/median value can then be compared to the color average/median values of the other color channels, and depending on this comparison, the white balance parameter can be determined, with which the color channel is corrected. Therein, the determination of the color
average/median values can be performed in any color space such that the invention is not restricted to a certain color space. For example, the color average/median values can be determined in the YUV color space or else in the RGB color space.
The at least one camera is preferably a video camera providing a sequence of images per second. The partial region is respectively "cut out" from these images and used for generating the image presentation for the display device. The image presentation is therefore generated in real time in the form of a video from the respectively current images. The camera can for example be a CCD camera or a CMOS camera.
The subset of pixels is selected according to a preset selection criterion. With this selection criterion, preferably, the position of the pixels in the image is taken into account. In an embodiment, it is provided that those pixels are selected according to the selection criterion, which are within the partial region of the image, thus exclusively those pixels, which are displayed on the display device or contribute to the generation of the image presentation for the display device. In determining the white balance parameter, thus, exclusively the color values of pixels are taken into account, which are within the partial region of the image, which is used for generating the image presentation for the display device. Thus, color casts in the image presentation are avoided, which else could be caused by high color values of pixels, which are outside of the used partial region of the image. Thus, an image presentation can be provided, which is free of such color casts and therefore has a homogeneous and uniform color distribution.
Additionally or alternatively, the respective color values of the pixels can also be taken into account in the selection criterion. This has the advantage that for example exclusively those pixels can be selected in determining the white balance parameter, which are very close to the grey value.
Therein, according to the selection criterion, those pixels are preferably selected, the color values of which are within a preset range of values, in particular around the zero value. In determining the white balance parameter, thus, exclusively pixels are taken into account, the color value amounts of which are below a preset limit value. For determining the white balance parameter, thus, pixels remain unconsidered, which have relatively high color values and therefore can negatively affect the white balance algorithm. Namely, these pixels could excessively affect the determination of the respective color average or median value of the color channels and thus result in color casts in the image
presentation. These pixels with too high color values are now filtered out and therefore remain unconsidered in determining the white balance parameter.
In an embodiment, it can be provided that at least a first and a second camera of the motor vehicle each provide an image of an environmental region of the motor vehicle. The image presentation for display on the display device is then generated from a partial region of the image of the first camera as well as from a partial region of the image of the second camera. For determining the white balance parameter, the subset of pixels is selected according to the preset selection criterion both from pixels of the image of the first camera and from pixels of the image of the second camera. In determining the white balance parameter for the displayed image presentation, thus, pixels of the two images are taken into account, namely exclusively those pixels, which are selected from the image of the first camera as well as from the image of the second camera according to the mentioned selection criterion. Thus, an image presentation can be provided, which has a uniform and realistic color distribution without color leaps and without color casts.
If exclusively pixels from the partial region of the image are used for determining the white balance parameter, thus, it can occur that the number of the selected pixels within the subset is relatively low. This is in particular the case if exclusively those pixels are selected from the partial region of the image according to the selection criterion, the color values of which are around the zero value within the preset range of values. If a large monochromatic object is depicted in the partial region of the image, such that most of the pixels have relatively high color values, thus, the selected subset only includes a low number of pixels satisfying the above mentioned criterion. In this case, a reliable white balance based on this subset of few pixels is not possible. For this reason, in an embodiment, it is proposed to determine the number of the selected pixels within the subset and to perform the determination of a new value for the white balance parameter based on the subset of pixels and thus adaptation of the previous white balance parameter only on condition that the number of the selected pixels is above a preset limit value. Thus, erroneous calculation of the white balance parameter is prevented.
Now, a relatively difficult situation can occur, in which a relatively great correction of the color channels (relative great correction of the color temperature) is performed within the scope of the white balance and the imaged scene suddenly changes because for example a light source with an extreme color temperature is suddenly switched off. Then, color casts arise in the image presentation, and it is possible that none of the pixels of the partial region satisfies the selection criterion. In this case, the number of the pixels within the subset is very low and is below the preset limit value. This could result in the color casts no longer being able to be corrected because the adaptation of the white balance parameter is omitted. In order to prevent such a situation, in an embodiment, it can be provided that the previously adjusted value of the white balance parameter is maintained or the white balance parameter is adjusted to a preset default value if the number of the pixels within the selected subset is below the preset limit value. Therein, it can be examined if the previously adjusted white balance parameter can be maintained or else adjustment of the white balance parameter to the preset default value is required.
If the number of the selected pixels is below the limit value, thus, the white balance parameter is adjusted to the preset default value if at least one of the mentioned conditions is satisfied, in particular if the two following conditions are satisfied at the same time:
if a color average or median value of the at least one color channel determined from color values of all of the pixels of the partial region is outside of a preset range of values around the zero value, thus, its value is greater than a preset limit value, and/or
if the previously adjusted value of the white balance parameter (thus the correction of the color temperature) is greater than a preset limit value.
If the two conditions are satisfied, thus, this indicates that color casts are present in the image presentation, which are to be compensated for. In this case, the white balance parameter (thus the gain factor of the at least one color channel) is adjusted to the preset default value, and the algorithm can be restarted because the number of the selected pixels will again exceed the limit value. Thus, color casts in the image presentation can be continuously prevented without any interruption.
In order to be able to detect color casts in the image presentation, additionally or alternatively to the above mentioned two criteria, other criteria can also be defined. A criterion can for example provide that a ratio between the gain factors of different color channels is determined. If this ratio then exceeds a preset limit value, thus, this indicates color faults in the image presentation. It is also possible that this detection of the color casts can also be performed by evaluating a gain offset of the color channels. The adjustment of the white balance parameter to the preset default value too can be performed in different manners. This change of the white balance parameter can e.g. be performed stepwise such that the white balance parameter is stepwise adjusted to the preset default value via intermediate values. Direct adjustment to the default value is also possible without using intermediate values.
The invention also relates to a camera system for a motor vehicle including at least one camera for providing an image of an environmental region of the motor vehicle, including an image processing device for generating an image presentation for display on a display device from a partial region of the image, and including a correcting device for performing a white balance of the image presentation, wherein the correcting device is formed for performing a method according to the invention.
Furthermore, the invention relates to a motor vehicle with a camera system according to the invention.
The preferred embodiments presented with respect to the method according to the invention and the advantages thereof correspondingly apply to the camera system according to the invention as well as to the motor vehicle according to the invention.
Further features of the invention are apparent from the claims, the figures and the description of figures. All of the features and feature combinations mentioned above in the description as well as the features and feature combinations mentioned below in the description of figures and/or shown in the figures alone are usable not only in the respectively specified combination, but also in other combinations or else alone.
Now, the invention is explained in more detail based on a preferred embodiment as well as with reference to the attached drawings.
There show:
Fig. 1 in schematic illustration a motor vehicle with a camera system according to an embodiment of the invention;
Fig. 2 in schematic illustration a block diagram of the camera system according to an embodiment of the invention; and Fig. 3 and 4 schematic illustrations of a color space, wherein a method according to an embodiment of the invention is explained in more detail.
A motor vehicle 1 illustrated in Fig. 1 is for example a passenger car. The motor vehicle 1 includes a camera system 2, which has a plurality of cameras 3, 4, 5, 6 in the
embodiment, which are disposed distributed on the motor vehicle 1 . Therein, four cameras 3, 4, 5, 6 are provided in the embodiment, wherein the invention is not restricted to such a number and arrangement of the cameras 3, 4, 5, 6. Basically, any number of cameras can be used, which can be mounted at different locations of the motor vehicle 1. Alternatively to such a multi-camera system 2, a stereo camera can also be used, which includes two image sensors.
A first camera 3 is for example disposed on the front bumper of the motor vehicle 1 . A second camera 4 is e.g. disposed in the rear region, for instance on the rear bumper or on a tailgate. The two lateral cameras 5, 6 can e.g. be integrated in the respective exterior mirror. The cameras 3, 4, 5, 6 are electrically coupled to an image processing device 7, which in turn is coupled to a display device 8. The display device 8 can be an LCD display.
The camera 3 captures an environmental region 9 in front of the motor vehicle 1 . The camera 4 captures an environmental region 10 behind the motor vehicle 1. The camera 5 captures a lateral environmental region 1 1 to the left beside the motor vehicle 1 , while the camera 6 captures an environmental region 12 to the right of the motor vehicle 1. The cameras 3, 4, 5, 6 provide images of the respective environmental regions 9, 10, 1 1 , 12 and communicate these images to the image processing device 7.
The cameras 3, 4, 5, 6 are video cameras, which are each able to capture a sequence of images per time unit. The cameras 3, 4, 5, 6 can also be so-called fish eye cameras, which have a relatively wide capturing angle or opening angle, for instance in a range of values from 150° to 200°.
From the images of the cameras 3, 4, 5, 6, the image processing device 7 generates an image presentation, which is then displayed on the display device 8. The white balance of this image presentation is also performed. The white balance can be performed internally in the image processing device 7 in an embodiment. Optionally, the image processing device 7 can also be integrated in one of the cameras 3, 4, 5, 6.
In Fig. 2, a block diagram of the camera system 2 is illustrated in extremely abstract illustration. The cameras 3, 4, 5, 6 communicate images B3, B4, B5, B6 to the image processing device 7. Then, the image processing device 7 generates the image presentation 13 from respective partial regions B3', B4', B5', B6' of the images B3, B4, B5, B6, which is presented on the display device 8. This image presentation 13 can for example be a plan view presentation, which shows the motor vehicle 1 and its
environment 9, 10, 1 1 , 12 from a bird's eye view. This image presentation 13 is generated from the respective partial regions B3', B4', B5', B6', which show the respective
environmental region 9, 10, 1 1 , 12 up to a predetermined distance from the motor vehicle 1 and which are processed together to the image presentation 13.
Alternatively, the cameras 3, 4, 5, 6 can communicate exclusively the partial regions B3', B4', B5', B6' to the image processing device 7, such that "cutting out" the partial regions B3', B4', B5', B6' is performed internally in the cameras 3, 4, 5, 6.
The image presentation 13 is then subjected to a white balance. The white balance is performed by means of a correcting device 14, which can also be integrated in the image processing device 7. After performing the white balance, the image presentation 13 is then communicated to the display device 8. In Fig. 2, a situation is shown, in which the white balance is performed on the finished image presentation 13. However, the white balance can already be performed before providing the image presentation 13 by subjecting the partial regions B3', B4', B5', B6' to the white balance.
Below, the performance of the white balance of the image presentation 13 is explained in more detail. In the described white balance algorithm, the YUV color space is used.
However, the invention is not restricted to the processing of the pixels in this color space and can also be implemented in other color spaces. In particular, the invention can be used in any color spaces isometric with the considered YUV space (e.g., HSV space).
For performing the white balance, exclusively pixels of the partial regions B3', B4', B5', B6' used for generating the image presentation 13 are taken into account. The pixels of the remaining image regions are not taken into account. The color values of each pixel can be described by a pair of values (U, V). From the pixels of the partial regions B3\ B4', B5\ B6\ now, a subset of pixels is selected, which satisfy a predetermined criterion. Namely, pixels are selected, the color values U, V of which are within a preset range of values around the zero value, thus, if the two following conditions are satisfied:
-a < U < a
and
-β < ν < β.
Therein, the limit values a and β can be identical values or else values different from each other. The color values of the selected pixels are then used for calculating a color average or median value (below referred to as color average value) of the respective color channel. The color average value of the U color channel is denoted by Us; the color average value of the V color channel is denoted by Vs. The color average values Us und Vs represent color temperatures of the two color channels.
The above mentioned criterion for selecting the subset of pixels can be illustrated with reference to Fig. 3. The range of values from -a to a on the one hand and the range of values from -β to β on the other hand is defined around the origin of the UV color space. Exclusively pixels are selected, the color values U, V of which come within the rectangle illustrated in Fig. 3.
For performing the white balance, the respective gain factor Gu and Gv is determined for each color channel U and V. With this gain factor Gu, Gv, then, the color channels can be corrected by multiplying the color values (U, V) of the pixels by the respective gain factor Gu, Gv. Thus, correction of the color average values Us and Vs of the color channels is effected such that the color average values Us and Vs are for example matched to each other. Therein, the gain factors Gu, Gv represent white balance parameters and are determined depending on the respective color average value Us and Vs.
Before the adaptation of the gain factors Gu and Gv is performed depending on the currently selected subset of pixels, first, the number of the pixels is determined, the color values U, V of which satisfy the above mentioned criterion (come within the rectangle according to Fig. 3). If the number of the selected pixels is greater than a preset limit value, new gain factors Gu, Gv are determined for the current image presentation depending on the current color average values Us and Vs, respectively. If the number of the selected pixels is below the limit value, the following algorithm is performed:
It is examined whether or not color casts are present in the current image presentation. Such color casts can for example arise in that the brightness of the scene is suddenly changed if for example a dominant light source with a high color temperature is switched off. Therein, with switched on light source, a relatively severe correction of the color temperature of the image presentation 13 is performed, and this severe correction causes color casts in the image presentation 13 after switching off the light source. In this case, it can occur that only very few pixels come within the rectangle according to Fig. 3. If the number of the selected pixels is smaller than the limit value, thus, it is examined whether or not such color casts are present. For this examination, color average values UT, VT are calculated from respective color values of all of the pixels of the partial regions B3', B4', B5', B6'. On the one hand, these color average values UT, VT are examined; on the other hand, the currently adjusted gain factors of the color channels are also examined.
If the color average values UT, VT are outside of a preset range of values 15 according to Fig. 4 and if the current gain factors are additionally also greater than a limit value, thus, it is assumed that color faults are present in the image presentation 13. In this case, the gain factors are adjusted to a preset standard value (so-called default value), namely either directly or stepwise via intermediate values.
Thus, it is examined if:
-GU < UT < GU
and
-GV < VT < GV.
If one of these criteria is not satisfied and if the gain factors are greater than the limit value, the adjustment of the gain factors to the preset value is effected. Otherwise, the current gain factors can be maintained.
After adjusting the gain factors to the default value, the calculation of new gain factors can be again restarted based on the current images because the number of the selected pixels will again exceed the limit value.

Claims

Claims
1 . Method for white balance of an image presentation (13), which is displayed on a display device (8) of a motor vehicle (1 ), wherein an image (B3, B4, B5, B6) of an environmental region (9, 10, 1 1 , 12) of the motor vehicle (1 ) is provided by means of at least one camera (3, 4, 5, 6) of the motor vehicle (1 ) and the image presentation (13) for display on the display device (8) is generated from a partial region (Β3', Β4', Β5', Β6') of the image (B3, B4, B5, B6), and wherein for performing the white balance respective color values (U, V) of pixels of the image (B3, B4, B5, B6) are acquired and a white balance parameter (Gu, Gv) for the white balance is determined depending on the color values (U, V) for at least one color channel (U, V) of the image presentation (13),
characterized in that
a subset of pixels of the image (B3, B4, B5, B6) is selected according to a preset selection criterion for performing the white balance, and that the color values (U, V) exclusively of the selected subset of pixels are taken into account in determining the white balance parameter (Gu, Gv) for the at least one color channel (U, V).
2. Method according to claim 1 ,
characterized in that
the position of the pixels in the image (B3, B4, B5, B6) is taken into account in the selection criterion.
3. Method according to claim 2,
characterized in that
according to the selection criterion, those pixels are selected, which are within the partial region (B3\ B4\ B5\ B6') of the image (B3, B4, B5, B6).
4. Method according to anyone of the preceding claims,
characterized in that
the respective color values (U, V) of the pixels are taken into account in the selection criterion.
5. Method according to claim 4,
characterized in that
according to the selection criterion, those pixels are selected, the color values (U, V) of which are within a preset range of values (-a to a; -β to β).
6. Method according to anyone of the preceding claims,
characterized in that
at least a first and a second camera (3, 4, 5, 6) of the motor vehicle (1 ) each provide an image (B3, B4, B5, B6) of an environmental region (9, 10, 1 1 , 12) of the motor vehicle (1 ) and the image presentation (13) for display on the display device (8) is generated from a partial region (B3\ B4\ B5\ B6') of the image (B3, B4, B5, B6) of the first camera (3, 4, 5, 6) and a partial region (Β3', Β4', Β5', Β6') of the image (B3, B4, B5, B6) of the second camera (3, 4, 5, 6), wherein for determining the white balance parameter (Gu, Gv) the subset of pixels is selected according to the preset selection criterion both from pixels of the image (B3, B4, B5, B6) of the first camera (3, 4, 5, 6) and from pixels of the image (B3, B4, B5, B6) of the second camera (3, 4, 5, 6).
7. Method according to anyone of the preceding claims,
characterized in that
the number of the selected pixels within the subset is determined and the
determination of the white balance parameter (Gu, Gv) based on the subset is effected only on condition that the number of the selected pixels is above a preset limit value.
8. Method according to claim 7,
characterized in that
if the number is below the limit value, the previously adjusted value of the white balance parameter (Gu, Gv) is maintained or the white balance parameter (Gu, Gv) is adjusted to a preset value.
9. Method according to claim 8,
characterized in that if the number is below the limit value, the white balance parameter (Gu, Gv) is adjusted to a preset value if:
a color median or average value (UT, VT) of the at least one color channel determined from color values (U, V) of all of the pixels of the partial region (Β3', Β4', Β5', Β6') is outside of a preset range of values (-GU to GU; -GV to GV), and/or
the previously adjusted value of the white balance parameter is greater than a preset limit value.
10. Camera system (2) for a motor vehicle (1 ), including at least one camera (3, 4, 5, 6) for providing an image (B3, B4, B5, B6) of an environmental region (9, 10, 1 1 , 12) of the motor vehicle (1 ), including an image processing device (7) for generating an image presentation (13) for display on a display device (8) from a partial region (Β3', Β4', Β5', Β6') of the image (B3, B4, B5, B6), and including a correcting device (14) for performing a white balance of the image presentation (13), wherein the correcting device (14) is adapted to acquire respective color values (U, V) of pixels of the image (B3, B4, B5, B6) and to determine a white balance parameter (Gu, Gv) for the white balance depending on the color values (U, V) for at least one color channel (U, V) of the image presentation (13) for performing the white balance, characterized in that
the correcting device (14) is adapted to select a subset of pixels of the image (B3, B4, B5, B6) according to a preset selection criterion for performing the white balance and to consider the color values (U, V) exclusively of the selected subset of pixels in determining the white balance parameter (Gu, Gv) for the at least one color channel (U, V).
1 1 . Motor vehicle (1 ) with a camera system (2) according to claim 10.
PCT/EP2013/076800 2012-12-17 2013-12-17 Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system WO2014095782A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102012024661.7 2012-12-17
DE102012024661.7A DE102012024661A1 (en) 2012-12-17 2012-12-17 Method for white balance of an image representation taking into account color values excluding a subset of pixels, camera system and motor vehicle with a camera system

Publications (1)

Publication Number Publication Date
WO2014095782A1 true WO2014095782A1 (en) 2014-06-26

Family

ID=49955292

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2013/076800 WO2014095782A1 (en) 2012-12-17 2013-12-17 Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system

Country Status (2)

Country Link
DE (1) DE102012024661A1 (en)
WO (1) WO2014095782A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107911682B (en) * 2017-11-28 2020-02-18 Oppo广东移动通信有限公司 Image white balance processing method, device, storage medium and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006022630A1 (en) * 2004-07-26 2006-03-02 Silicon Optix, Inc. Panoramic vision system and method
EP1973074A1 (en) * 2006-01-10 2008-09-24 Matsushita Electric Industrial Co., Ltd. Color correction device, color correction method, dynamic camera color correction device, and video search device using the same
US20100020221A1 (en) * 2008-07-24 2010-01-28 David John Tupman Camera Interface in a Portable Handheld Electronic Device
US20120242852A1 (en) * 2011-03-21 2012-09-27 Apple Inc. Gesture-Based Configuration of Image Processing Techniques

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010003668B4 (en) * 2010-04-07 2020-03-26 Robert Bosch Gmbh Color mask for an image sensor of a vehicle camera
JP5386538B2 (en) * 2011-05-12 2014-01-15 富士重工業株式会社 Environment recognition device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006022630A1 (en) * 2004-07-26 2006-03-02 Silicon Optix, Inc. Panoramic vision system and method
EP1973074A1 (en) * 2006-01-10 2008-09-24 Matsushita Electric Industrial Co., Ltd. Color correction device, color correction method, dynamic camera color correction device, and video search device using the same
US20100020221A1 (en) * 2008-07-24 2010-01-28 David John Tupman Camera Interface in a Portable Handheld Electronic Device
US20120242852A1 (en) * 2011-03-21 2012-09-27 Apple Inc. Gesture-Based Configuration of Image Processing Techniques

Also Published As

Publication number Publication date
DE102012024661A1 (en) 2014-06-18

Similar Documents

Publication Publication Date Title
EP3729803B1 (en) Multi camera image processing
EP3394833B1 (en) Dynamic image blending for multiple-camera vehicle systems
US11477372B2 (en) Image processing method and device supporting multiple modes and improved brightness uniformity, image conversion or stitching unit, and computer readable recording medium realizing the image processing method
EP2471691B1 (en) Obstacle detection device, obstacle detection system provided therewith, and obstacle detection method
US7969480B2 (en) Method of controlling auto white balance
US10187590B2 (en) Multi-camera vehicle vision system with image gap fill
KR101367637B1 (en) Monitoring apparatus
EP2160017A3 (en) Optical condition design method for a compound-eye imaging device
EP3343912B1 (en) Imaging device
US9214034B2 (en) System, device and method for displaying a harmonized combined image
CN111630837B (en) Image processing apparatus, output information control method, and program
US10455159B2 (en) Imaging setting changing apparatus, imaging system, and imaging setting changing method
US20170347008A1 (en) Method for adapting a brightness of a high-contrast image and camera system
US9105105B2 (en) Imaging device, imaging system, and imaging method utilizing white balance correction
KR20180001869A (en) Image Improving Apparatus for AVM System and Improving Method thereof
WO2014095782A1 (en) Method for white balance of an image presentation considering color values exclusively of a subset of pixels, camera system and motor vehicle with a camera system
JP7148384B2 (en) Semiconductor device, image processing method and program
JP4857556B2 (en) In-vehicle camera video presentation device
JP2021097348A (en) Imaging apparatus, control method, and program
EP2843937B1 (en) Method for adapting a gamma curve of a camera system of a motor vehicle, camera system and motor vehicle
US11017508B2 (en) Image matching method and apparatus
US20210287335A1 (en) Information processing device and program
KR20180028354A (en) Method for displaying image in multiple view modes
WO2013156417A1 (en) Method for white balance of an image considering the colouring of the motor vehicle
Yamamoto et al. Color correction for multi-camera system by using correspondences

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13820751

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13820751

Country of ref document: EP

Kind code of ref document: A1