EP3725069A1 - Verfahren und vorrichtung zum aufnehmen eines digitalbildes - Google Patents
Verfahren und vorrichtung zum aufnehmen eines digitalbildesInfo
- Publication number
- EP3725069A1 EP3725069A1 EP18819308.0A EP18819308A EP3725069A1 EP 3725069 A1 EP3725069 A1 EP 3725069A1 EP 18819308 A EP18819308 A EP 18819308A EP 3725069 A1 EP3725069 A1 EP 3725069A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- pixel
- image sensor
- digital
- value
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000009467 reduction Effects 0.000 claims description 15
- 230000004044 response Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 2
- 101100273797 Caenorhabditis elegans pct-1 gene Proteins 0.000 claims 1
- 230000000875 corresponding effect Effects 0.000 description 13
- 238000013213 extrapolation Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000001955 cumulated effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 238000004574 scanning tunneling microscopy Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/50—Control of the SSIS exposure
- H04N25/57—Control of the dynamic range
- H04N25/58—Control of the dynamic range involving two or more exposures
- H04N25/587—Control of the dynamic range involving two or more exposures acquired sequentially, e.g. using the combination of odd and even image fields
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/47—Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data
Definitions
- the invention relates to a method and a device for recording a digital image by means of a recording device, wherein an image sensor is exposed and a digital image is then determined in response to the exposure of the image sensor.
- Digital photography has largely replaced analogue photography.
- an image sensor consisting of one or more sensor pixels is exposed, and based on the exposure of the individual sensor pixels in response to the incident light, an electric charge is accumulated proportional to the amount of incident light and of the incident light Sensor recorded Lich tes is.
- the accumulated electrical charge is read at the end of the exposure time, the amount of electrical charge being assigned a quantified value, which then later represents the corresponding pixel value at the corresponding image position.
- the value that corresponds to the accumulated charge at the respective image position is usually specified digitally and generally has a predetermined value range, which can assume values between 0 and 255 for an 8-bit depth. It also sensors with 12 to 14-bit depth and rarely even with special cameras up to 16-bit depth are known.
- the dynamic range is defined as the quotient between the value of the brightest area of the scene and the darkest area of the scene. The larger the dynamic range is, the more difficult a reasonable exposure of the image becomes, if it is neither overexposed nor underpainted. If a scene has a high dynamic range, a short exposure time displays the bright areas of the scene to be photographed well, while leaving the dark areas underexposed, depending on the dynamic range. The other way around is when the exposure time is set to be very long, enough to capture the dark areas of the scene, while overexposing the bright areas.
- HDR High Dynamic Range
- an imaging system which includes an image sensor having an array of dual gain pixels.
- Each pixel can be driven by an improved triple-read method and an improved quadruple-read method, so that all signals can be read in a high-gain configuration in order to prevent an electrical offset in the signal strengths.
- dual-gain pixels with two read-out paths represent a modification of the hardware, so that already existing recording devices can not be retrofitted accordingly.
- the object is achieved by the method according to claim 1 and theretevorrich- device according to claim 12 according to the invention.
- Image sensor of the recording device over a predetermined exposure period is illuminated.
- the image sensor pixels contained in the digital image sensor are also exposed, each of these image sensor pixels representing one pixel or a group of pixels.
- electrical signals are read out from the individual image sensor pixels by means of an electronic readout circuit, the electrical signals of each individual image sensor pixel representing in each case an accumulated charge in response to the incident light of the respective image sensor pixel.
- a digital pixel value is now determined for each image sensor pixel by means of an electronic arithmetic unit, the respective pixel value corresponding to the accumulated charge represented by the read-out electrical signal of the respective image sensor pixel.
- a digital image is generated based on the determined digital pixel values by means of the electronic arithmetic unit and, if necessary, stored in a digital data memory.
- the pixel values may be a representation of brightness and / or color values.
- this generic method for recording a digital image is modified and supplemented in such a way that the electrical signals from at least part of the image sensor pixels are read out several times during the exposure period at discrete readout times within the exposure period using the electronic readout circuit
- the respective image sensor pixel is only read once by means of the readout circuit, as is conventionally the case at the end of the exposure period according to the prior art.
- a digital pixel intermediate value based on the read-out signal of the respective image sensor pixel is then determined for each of the image sensor pixels read several times at the respective discrete readout times, wherein this pixel intermediate value of the accumulated charge represented by the read-out electrical signal of the respective image sensor pixel is the respective discrete one Readout time corresponds.
- image sensor pixels After readout by means of the electronic read-out circuit, image sensor pixels are not reset or resetted, so that at the end of the exposure period the entire accumulated charge is contained in the respective image sensor pixels. If, for example, all image sensor pixels of the digital image sensor are read out at each readout time within the exposure period using the electronic readout circuit and the pixel intermediate values are determined accordingly, there is an intermediate image of the recorded scene at each discrete readout time within the exposure period, that at the respective discrete readout time is defined by the pixel intermediate values.
- Image sensor pixels are read out. It is conceivable and also advantageous if each image sensor pixel has its own discrete readout times, which may be at least partially different from each other. Thus, it may be that a first group of image sensor pixels has a first discrete readout time, which differs from the first discrete readout time of a second group of image sensor pixels. In other words, each image sensor pixel or groups of image sensor pixels may have their own individual discrete readout times.
- Each relevant image sensor pixel is thus read out a plurality of times within the exposure period, once at each image sensor pixel, for the discrete readout time, so that at each discrete readout time point a respective pixel intermediate value can be determined so that a plurality of pixel intermediate values result for the relevant image sensor pixel over the entire exposure period.
- a digital pixel value with respect to the end of the exposure period is estimated as a function of the determined digital pixel intermediate values at the respective readout times when the accumulated charge of the respective multiple read sensor pixel has reached a charge saturation at the end of the exposure period. If the accumulated charge of the respective image sensor pixel read a plurality of times has reached its charge saturation over the exposure period, then a digital pixel value based thereon would have reached a predefined value maximum.
- the later digital image would be overexposed in these areas.
- the value maximum of the charge to be accumulated of the respective image sensor pixel is increased, specifically beyond the physical charge saturation, which also increases a possible value maximum with respect to the pixel values to be determined.
- a pixel value based on the accumulated charge of an image sensor pixel can be determined as if the image sensor pixel had no charge saturation with respect to the accumulated charge during the exposure time.
- the digital image is generated, whereby the quotient between individual image values and possibly even all digital pixel values is estimated the brightest pixel value and the darkest pixel value compared to conventional digital images is increased and thus overall the dynamic range is increased.
- the image sensor only contains a single image sensor pixel, which may be the case, for example, in scanning tunneling microscopy. Therefore, all statements are to be understood as meaning that only a single image sensor pixel is present whose natural charge saturation has been computationally increased.
- Image sensors are used to record a corresponding HDR digital image according to the teachings of the present invention can.
- an extrapolation with respect to the end of the exposure period based on the determined digital pixel intermediate values is understood in particular, although other methods including noise suppression methods may also be used.
- the extrapolation of a pixel value for an image sensor pixel determines the pixel value that would result at the end of the exposure time if the respective image sensor pixel had no physical charge saturation.
- the individual intermediate pixel values, which were determined at the respectively discrete readout time points based on the charge accumulated at the respective readout time, are used in order to be able to determine a corresponding extrapolated digital pixel value outside the actual value range.
- the pixels are read out individually or simultaneously or that the pixels are read line by line in order to determine the individual values.
- the longest possible exposure period is selected, whereby many of the image sensor pixels are overexposed in the conventional sense and reach their charge saturation, but due to the present invention exactly these image sensor pixels are corrected with correspondingly extrapolated pixel values. It is therefore particularly advantageous if the image sensor pixels are exposed without interruption over the entire exposure period in order to achieve the necessary intermediate samples for extrapolating the pixel values.
- the exposure period is chosen such that more than 25%, preferably more than 50%, of the exposed one
- Image sensor pixels have reached a predetermined charge threshold.
- This predetermined charge threshold may be, for example, the charge saturation, i. the exposure period is selected so that at 25% or 50% of the image sensor pixels the accumulated charge reaches the maximum or the charge saturation and thus a pixel value determined at the end of the exposure period has reached its predetermined maximum value.
- the exposure period is selected for a sufficiently long time to ensure that a correspondingly large dynamic range is created.
- the digital image is then generated on the basis of the extrapolated pixel values as well as the pixel values determined at the end of the exposure period when the relevant image sensor pixel has not reached the charge maximum, so that the digital image results both from extrapolated pixel values and at the end of the exposure period determined pixel values can exist. Consequently, the charge maximum corresponds to an original value maximum of the pixel values, wherein an extrapolated digital pixel value is greater than or equal to the original value maximum and thus increases the original value maximum, theoretically arbitrary.
- a pixel noise reduction is carried out, wherein for the pixel noise reduction of at least one pixel intermediate value at a readout time other pixel intermediate values of the same readout time are used.
- pixel noise reduction is advantageously performed at each of the readout times for each pixel intermediate value determined at this readout timing so as to reduce errors in determining the pixel intermediate values.
- the pixel noise reduction is advantageously carried out for all pixel intermediate values of at least one readout time, so as to generate a noise-reduced intermediate image.
- pixel noise reduction of at least one intermediate pixel value at a readout time other pixel intermediate values are used whose image sensor pixels are different from the image sensor pixel of the intermediate value and, for example, lie in a predefined area around the image sensor pixel of the intermediate value the pixel noise reduction is performed.
- all other intermediate pixel values are used to reduce the noise of a pixel intermediate value.
- the other pixel intermediate values may belong to the same readout time or be interpolated values from temporally adjacent readout times.
- Image sensor pixel performed a time-related pixel noise reduction, wherein for the time-related pixel noise reduction of a pixel intermediate value of the at least one image sensor pixel or of the estimated digital pixel value is used for the determined pixel intermediate values of the at least one image sensor pixel.
- a recording device for recording a digital image is also proposed according to the invention, wherein the recording device has a digital image sensor which has at least one image sensor pixel.
- the recording device furthermore has a readout circuit for reading out an electrical signal of the at least one image sensor pixel, wherein the electrical signal in each case represents an accumulated charge in response to incident light of the respective image sensor pixel.
- the recording device further has a computing unit for determining at least one digital pixel value based on the at least one read-out electrical signal and for generating a digital image based on the at least one determined digital pixel value.
- the receiving device in particular the electronic calculating unit, is designed to carry out the above-mentioned method.
- the recording device has an image sensor having a plurality of image sensor pixels, wherein one or more electrical signals can be read out for each image sensor pixel. For each image sensor pixel, a digital pixel value can now be determined based on the electrical signals.
- Figure 1 Schematic representation of a recording device for
- FIG. 1 shows a recording device 10 with which a scene 11 to be recorded can be recorded digitally and a corresponding digital image can be generated.
- the receiving device 10 usually has a housing 12 which has on one of the sides of the housing 12 an exposure optics 13 or another device with which a digital image sensor 14 can be exposed.
- the exposure optics 13 are designed such that they open as required and close again after a predetermined exposure time or an exposure period ,e, during which the light of the scene 11 falls on the digital image sensor 14 during the opening of the exposure optics 13. After closing the exposure optics 13, the interior of the recording device 10 is completely darkened again, so that the digital image sensor is not accidentally exposed.
- the digital image sensor 14 has a plurality of image sensor pixels 15, which for example in the manner of a photodiode in response to the incident light during the exposure time ⁇ e accumulate an electrical charge in a charge storage.
- Each of these image sensor pixels 15 in this case has such a charge storage, wherein, at the end of the exposure time, a pixel value, which later corresponds to the corresponding pixel in the generated digital image based on the accumulated charge, is correlated with the accumulated charge in the respective image sensor pixel 15.
- the accumulated charges of the individual image sensor pixels 15 can be read out and correspondingly quantified, so that an electronic computing unit 17 can then be based thereon
- an electronic computing unit 17 For each of the read-out image sensor pixels 15 can determine a corresponding pixel value. In the simplest case, such a pixel value corresponds to the quantified accumulated charge of the read-out image sensor pixel.
- the image sensor pixels 15 are not, as known from the prior art, only once at the end of the exposure time ⁇ e by the electronic readout circuit
- each of the respective image sensor pixels 15 is repeatedly read out over the exposure period ⁇ e, so that a plurality of pixel values of the respective image sensor pixel can be determined for each of these image sensor pixels 15 within the exposure period ⁇ e, which coincides with the charge accumulated at the read-out time correlate to respective image sensor pixels 15.
- These pixel values are called pixel intermediate values because they represent a pixel value within the exposure period.
- a pixel value for the end of the exposure period is then extrapolated based on the respective pixel intermediate values if the respective image sensor pixel 15 has reached its charge maximum or charge saturation before reaching the end of the exposure period.
- the digital image is then generated and stored in the digital memory 18.
- the lower diagram shows the operating principle of the present invention.
- the exposure period is subdivided into five time periods in each case, with the respective image sensor pixels using the electronic readout circuit at the discrete points in time ti-U and ⁇ e
- the discrete times ti, t2, t3 and U there is in each case still a valid value with respect to the accumulator
- the charge of the individual readout times is determined, whereby the sensor pixel would run into its charge saturation Vmax between U and the end of the exposure period ⁇ e.
- the value at time ⁇ e is therefore no longer measurable.
- pixel values vi-v 4 which are respectively read out at the discrete readout times t.sub.i-U, which show the value development during the exposure of the respective sensor pixel, are therefore present.
- the value VE at the time of the end of the exposure period are extrapolated ⁇ e now, this value VE above the possible charge maximum v ma x and thus would be physically not measurable.
- the intermediate values vi-v 4 previously determined in the respective intermediate steps, it is nevertheless possible to extrapolate such a value VE at the end of the exposure period ⁇ e.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102017129770.7A DE102017129770B4 (de) | 2017-12-13 | 2017-12-13 | Verfahren und Vorrichtung zum Aufnehmen eines Digitalbildes |
PCT/EP2018/084582 WO2019115623A1 (de) | 2017-12-13 | 2018-12-12 | Verfahren und vorrichtung zum aufnehmen eines digitalbildes |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3725069A1 true EP3725069A1 (de) | 2020-10-21 |
Family
ID=64665354
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18819308.0A Withdrawn EP3725069A1 (de) | 2017-12-13 | 2018-12-12 | Verfahren und vorrichtung zum aufnehmen eines digitalbildes |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP3725069A1 (de) |
DE (1) | DE102017129770B4 (de) |
WO (1) | WO2019115623A1 (de) |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7061524B2 (en) * | 2001-11-13 | 2006-06-13 | The Board Of Trustees Of The Leland Stanford Junior University | Motion/saturation detection system and method for synthesizing high dynamic range motion blur free images from multiple captures |
US7009636B2 (en) * | 2001-11-13 | 2006-03-07 | The Board Of Trustees Of The Leland Stanford Junior University | Photocurrent estimation from multiple captures for simultaneous SNR and dynamic range improvement in CMOS image sensors |
DE102006046720A1 (de) * | 2006-10-02 | 2008-04-03 | Arnold & Richter Cine Technik Gmbh & Co. Betriebs Kg | Digitale Laufbildkamera |
JP5644400B2 (ja) * | 2010-11-15 | 2014-12-24 | セイコーエプソン株式会社 | 撮影装置、撮影方法および撮影プログラム |
US9948875B2 (en) * | 2015-10-01 | 2018-04-17 | Semiconductor Components Industries, Llc | High dynamic range imaging pixels with improved readout |
-
2017
- 2017-12-13 DE DE102017129770.7A patent/DE102017129770B4/de not_active Expired - Fee Related
-
2018
- 2018-12-12 EP EP18819308.0A patent/EP3725069A1/de not_active Withdrawn
- 2018-12-12 WO PCT/EP2018/084582 patent/WO2019115623A1/de unknown
Also Published As
Publication number | Publication date |
---|---|
DE102017129770B4 (de) | 2020-08-13 |
DE102017129770A1 (de) | 2019-06-13 |
WO2019115623A1 (de) | 2019-06-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE102007026337B4 (de) | Digitale Kamera | |
DE60014814T2 (de) | Elektronische Stehbildkamera und zugehöriges Signalverarbeitungsverfahren zur Entfernung von Rauschanteilen in Bilddaten | |
DE3687877T2 (de) | Kamera und verfahren zur bilderzeugung. | |
DE69314370T2 (de) | Festkörperbildaufnahmevorrichtung mit automatischer Korrektur defekter Bildelemente | |
DE602005005553T2 (de) | Abbildungsvorrichtung und Abbildungsverfahren | |
DE4106825A1 (de) | Belichtungskontrollvorrichtung fuer einzelbild-videokameras | |
DE3734957A1 (de) | Logarithmische farbabbildungsvorrichtung | |
EP1909492B1 (de) | Digitale Laufbildkamera | |
DE102009053281A1 (de) | Bildsensor | |
EP1357742B1 (de) | Digitale Laufbildkamera | |
EP2567539B1 (de) | Bildsensor und verfahren zum aufnehmen eines bildes | |
DE112015005599T5 (de) | Bildverarbeitungsvorrichtung, Bildverarbeitungsverfahren und Programm | |
DE102014013446A1 (de) | Bildaufnahmevorrichtung und -system und Steuerverfahren für die Bildaufnahmevorrichtung | |
WO1999008440A1 (de) | Verfahren zur kompensation des dunkelstroms eines mehrere pixel aufweisenden elektronischen sensors | |
DE112015005499T5 (de) | Bildverarbeitungsvorrichtung, Bildverarbeitungsverfahren und Programm | |
DE10213564A1 (de) | Verfahren zur Unterdrückung von Geistbildartefakten bei Röntgenbildern und Röntgenvorrichtung zur Durchführung dieses Verfahrens | |
DE102010051986A1 (de) | Bildsensor | |
DE3134098A1 (de) | Messeinrichtung | |
DE2060581A1 (de) | Elektrische Schaltungsanordnung zur Vergroesserung kleiner Signalwertaenderungen gegenueber Stoerpegelwerten | |
EP1761071A1 (de) | Verfahren zur Aufnahme eines Bildes und Digitalkamera | |
DE112017005810T5 (de) | Bildaufnahmevorrichtung und Steuerverfahren dafür | |
DE102017129770B4 (de) | Verfahren und Vorrichtung zum Aufnehmen eines Digitalbildes | |
WO2012130208A1 (de) | Videoanordnung zur bereitstellung von hdr videoströmen | |
DE102013209164B4 (de) | Digitale Filmkamera | |
DE102008052679A1 (de) | Digitalkamerasensor, Digitalkamera sowie Verfahren zum Aufnehmen eines Kamerabildes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20200610 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20220701 |