EP3301640B1 - Method and thermal camera for the contactless measurement of a temperature or for observing quickly moving ir scenes with a thermal ir imager - Google Patents
Method and thermal camera for the contactless measurement of a temperature or for observing quickly moving ir scenes with a thermal ir imager Download PDFInfo
- Publication number
- EP3301640B1 EP3301640B1 EP17193958.0A EP17193958A EP3301640B1 EP 3301640 B1 EP3301640 B1 EP 3301640B1 EP 17193958 A EP17193958 A EP 17193958A EP 3301640 B1 EP3301640 B1 EP 3301640B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- filter
- sensor
- thermal
- transfer function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 21
- 238000005259 measurement Methods 0.000 title claims description 14
- 238000012546 transfer Methods 0.000 claims description 46
- 239000011159 matrix material Substances 0.000 claims description 42
- 230000009021 linear effect Effects 0.000 claims description 22
- 230000000694 effects Effects 0.000 claims description 17
- 230000004044 response Effects 0.000 claims description 15
- 238000009529 body temperature measurement Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000003706 image smoothing Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000003707 image sharpening Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 41
- 238000012937 correction Methods 0.000 description 16
- 238000001931 thermography Methods 0.000 description 13
- 238000007781 pre-processing Methods 0.000 description 11
- 239000013598 vector Substances 0.000 description 11
- 230000002123 temporal effect Effects 0.000 description 10
- 125000004122 cyclic group Chemical group 0.000 description 7
- 238000003491 array Methods 0.000 description 6
- 230000003111 delayed effect Effects 0.000 description 6
- 238000001914 filtration Methods 0.000 description 6
- 238000005316 response function Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 4
- 229910021417 amorphous silicon Inorganic materials 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000001364 causal effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012892 rational function Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20182—Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
Definitions
- the invention relates to a method and a thermal imaging camera for non-contact temperature measurement or observation of fast-moving IR scenes with a thermal IR image sensor, in particular for uncooled thermal IR cameras.
- Microbolometers are thermal IR image sensors and have relatively long response times due to their mode of action. They are therefore hardly suitable for applications with higher dynamic requirements. Fast radiometric temperature measurements are not possible or severely error-prone. The inertia of the microbolometer also manifests itself in motion blur in camera pans or in objects that move quickly through the image field.
- microbolometer arrays The dynamic behavior of microbolometer arrays is discussed in numerous publications. In general, the thermal time constant of the pixels ⁇ th - in addition to the frame rate of the sensor - is considered as an important parameter and as a significant limiting factor for the "speed" of an IR camera. However, there are no solutions for dynamic compensation in the literature.
- the thermal inertia of microbolometers is simply called fact. Only for thermal single sensors, for example in thermometers, a frequency response correction is known. At present, the focus is solely on the use of new, faster sensors. However, the technical possibilities are limited (eg reduction of the "thermal mass”) and can not usually be influenced by the users.
- thermal inertia of thermal imaging is switched to cooled photonic sensors.
- time-critical measurement tasks are not regarded as the domain of the thermal sensors, since the dynamic measurement error after the period of time ⁇ th is still about 37% (as described in US Pat Fig. 5 for an a-Si microbolometer). Precise measurements are only possible after a period of several ⁇ th .
- thermoelectric sensors (“Thermopiles"), pyroelectric / ferroelectric sensors and sensors based on bimorph microcantilever arrays.
- the invention has for its object to find a new way to correct the dynamic inertia of thermal IR sensor arrays, by means of the dynamic measurement error of a radiometric or thermal thermal imaging camera can be significantly reduced. In an extended task also a reduction of motion blur should be achieved.
- the readout of the sensor array of the IR image sensor with a frame rate that is at least twice the characteristic frequency that the thermal time constant ⁇ th is assigned to the sensor elements of the sensor array of the IR image sensor is.
- the readout of the sensor array is preferably carried out with a frame rate which is at least three times the characteristic frequency.
- the adaptation of the parameters of the image filter for influencing the eigenvalues of the at least one convolution matrix is expediently carried out by numerically solving the linear equation system by means of approximate solutions, preferably based on the least squares of errors.
- amplitude eigenvalues can also be adjusted for the configuration of the image filter by extending the linear equation system with other linear equations.
- the object of the invention in a thermal imaging camera for non-contact temperature measurement or observation of fast-moving IR scenes with a thermal IR image sensor achieved in that at least one spatial-temporal image filter is arranged downstream of the output of the IR image sensor, that is formed image filter as at least an image processing unit for realizing a matrix restructuringen transfer function, wherein in calculations of each resultant pixel value v xy (t) is a respective current sensor pixel value u xy (t) is used and a number of time of past sensor pixel values u xy (tk ⁇ at), a number adjacent sensor pixel values u (x ⁇ m ⁇ ⁇ x) (y ⁇ n ⁇ ⁇ y) (t) and a number of temporally past adjacent sensor pixel values u (x ⁇ m ⁇ ⁇ x) (y ⁇ n ⁇ ⁇ y) (tk ⁇ ⁇ t).
- the image filter is at least temporarily linked with means for adjusting parameters of the image filter by influencing the position of eigenvalues of at least one convolution matrix by at least one parameter calculation unit (5) in order to adapt at least zeros or poles of the transfer function to a specific measurement task or observation task, wherein the means for setting the parameters, a computer unit for solving a linear equation system for influencing the eigenvalues of the at least one convolution matrix by means of numerical approximation methods.
- the transfer function of the spatio-temporal image filter in a complex s-plane in the image area of a Laplace transform has one or more zeros to compensate for the poles of the transfer function of the IR image sensor or expediently has one or more poles for limiting the noise bandwidth.
- one or more filter coefficients are either of matrix value and convolution matrix in spatial space or scalar valued and so dimensioned that the effects of the thermal inertia of the sensor pixels in the associated result pixel values of the output result image data are reduced by means of the transfer function of the image filter.
- one or more filter coefficients are matrix-valued and designed as a convolution matrix in the spatial space such that the effects of crosstalk of the sensor pixels in the associated result pixel values of the output result image data are reduced by means of the transfer function of the image filter.
- one or more filter coefficients are matrix-valued and designed as a convolution matrix in the spatial space such that the output image data of result pixel values have spatial image smoothing or image sharpening by means of the transfer function of the image filter.
- one or more filter coefficients are scalar-valued and dimensioned so that the noise bandwidth of the sensor pixels in the associated result pixel values of the output result image data is reduced by means of the transfer function of the image filter.
- one or more filter coefficients of the image filter are continuously variably adjustable depending on the image content, so that by means of The transfer function of the image filter can always be an optimized filter effect with minimal artifacts in the output result image data can be set.
- a desired transfer function of the image filter is implemented particularly advantageously by an IIR system.
- the invention is based on the fundamental idea that the measurement efficiency of a thermal imaging camera based on a microbolometer array suffers from rapid thermal changes in the scene due to the thermal inertia of its individual sensor elements, which operate as thermal resistance bridges. This can only be slightly improved by changing the thermal properties of the sensor array (eg by reducing the thermal mass of each individual element).
- the reason for this is the thermal inertia of the bolometer elements, which can be described by a thermal time constant ⁇ th as the characteristic variable of the heat transfer behavior.
- the transient behavior corresponds to that in Fig.
- the thermal time constant ⁇ th corresponds to the time duration after which the [1-exp (-1)] fold (ie approximately 63%) of the jump height is reached.
- the thermal time constant ⁇ th (which is typically in the range between 7 and 15 ms in the case of the well-known manufacturers of bolometer arrays) can barely be significantly reduced due to the sensor design by hardware optimizations.
- the invention solves this problem by digital image processing based on a spatial-temporal filtering of read sensor data.
- measured values of a considered sensor pixel as well as a number of temporally preceding values of the considered sensor pixel, a number of spatially adjacent pixel values and a number of temporally preceding values of the spatially adjacent pixels flow into the calculation of each generated result pixel value.
- the spatial-temporal image filter has a transfer function in which, in a complex s-plane as the image area of a Laplace transform, at least one zero for compensating pole positions of the Signal transmission function of the sensor and at least one pole to find a limitation of a noise bandwidth.
- the transfer function of the spatio-temporal image filter can be represented mathematically as a rational function. It is matrix-valued if one or more filter coefficients are matrix-valued.
- the matrix-valued filter coefficients can act as convolution matrix in the spatial domain and be dimensioned such that the transfer function compensates for the thermal inertia and the crosstalk of the sensor pixels.
- one or more filter coefficients have a matrix value, act as a convolution matrix in spatial space and are dimensioned such that the transfer function of the image filter results in spatial image smoothing or image sharpening.
- a further embodiment provides that one or more filter coefficients are scalar-valued and dimensioned so that the thermal inertia of the sensor pixels is compensated for by means of the transfer function of the filter and / or the noise bandwidth is reduced.
- one or more filter coefficients can be variably adjusted continuously depending on the image content in order to achieve an optimum filter effect with minimal artifacts.
- numerical approximations are used by means of recursive systems, in particular filters with infinite impulse response, so-called IIR filters (Infinite Impulse Response Filter) are used.
- Fig. 1 shown schematically for a thermal imaging camera.
- This comprises a thermal IR image sensor 1, which contains an IR sensor array 11, preferably in the form of a microbolometer array, and a read-out circuit 12, usually already equipped with signal preprocessing, as well as a spatio-temporal image filter 3.
- the raw signal values supplied by the sensor array 11 as sensor images must generally be processed in order to achieve a signal quality that is sufficient for successful filter application.
- Such a preparation will include as preprocessing steps at least one nonuniformity correction (NUC). This can be carried out, for example, as a single-point correction, wherein depending on temperature influences drifting raw signal values are adjusted by means of a reference image (offset correction).
- NUC nonuniformity correction
- the preprocessing steps may also include a defect pixel replacement scheme.
- the aforementioned corrections of a signal preprocessing can already be contained in an on-chip circuit of the IR image sensor 1 or else in a downstream separate image preprocessing unit 13 (only in FIG Fig. 2 drawn).
- a downstream separate image preprocessing unit 13 only in FIG Fig. 2 drawn.
- these corrections are at least predominantly already implemented in the readout circuit 12, which then often contains a bias control to compensate for the uneven offsets and / or sensitivities of the individual sensor elements of the IR sensor array 11.
- the result of the signal pre-processing of the read-out circuit 12 is an image data stream 2 of processed sensor images, which is provided as filter input data 31 in the form u xy (t) .
- the spatio-temporal image filter 3 then outputs corrected result image data 4 in the form v xy (t) .
- the image filter 3 it is fundamentally irrelevant whether the read-out image data stream 2 is transferred from sensor raw images directly into the data stream of the processed sensor images directly in an online process through the preprocessing steps described above or is used directly as filter input data 31.
- the filter input data 31 can also be provided, for example, in such a way that sensor images processed by preprocessing steps are buffered in a buffer memory (not shown here) and then read again successively to provide filter input data 31.
- a buffer memory not shown here
- image stream 2 edited sensor images as an ordered sequence of image files permanently on a mass storage (not shown) store.
- the image filter 3 can then work completely separated in an offline mode, providing the Filter input data 31 is then only in the orderly reading the stored image files.
- the spatio-temporal image filter 3 represents a three-dimensional filter in which the two-dimensionally read image data stream 2 from the IR sensor array 11 accesses the two spatial dimensions 32 of the filter input data 31 and, as a third dimension 33, the time profile resulting from the image data stream 2 Filter input data 31 is considered.
- the image filter 3 thus effects a combination of spatial and temporal components of the image data stream 2, as will be explained in more detail below.
- Fig. 2 shows a schematic of the thermal imager, in which the method according to Fig. 1 is modified, wherein the signal preprocessing (at least predominantly) in a separate the IR image sensor 1 downstream image preprocessing 13 runs.
- the filter input data 31 thus has at least one unevenness correction (NUC), an offset correction by reference image comparison in live mode and a complete unevenness correction with defect pixel replacement scheme.
- the thus preprocessed filter input data 31 thus have a significantly lower noise and noise signal quota than the raw data from the image data stream 2, whereby the image filter 3 can align itself with the pure compensation of effects of the dynamic measurement errors by rapid changes in the observed IR scene.
- the combination of the incoming image signals as filter input data 31 takes place, as already above Fig.
- a parameter calculation unit 5 is provided, which is used at least once for configuring the image filter 3 before the start of a defined observation task by adjusting zeros or poles of the transfer function to a specific measurement task by influencing the position of eigenvalues of one or more convolution matrices, or else during the current operation Operation of the thermal imager (in Fig. 2 shown stylized with thermal IR image sensor 1) is in use.
- the spatial-temporal image filter 3 is advantageously parametrized according to the invention as a temporal image filter 3, so that, depending on the configuration, a smoothing (integrating) or sharpening (differentiating) characteristic arises.
- T 2 can initially be arbitrary time-independent matrices.
- the amplitude-locking matrix A in Eq. (2) could also be removed from the description by considering A v instead of v , which should not be done here.
- said matrices need not be invertible or diagonalizable. They also need not be assumed to be real, although this will usually be the case in the application.
- Real parts of eigenvalues of T 2 and of reciprocal eigenvalues of ⁇ 1 have the meaning of time constants. That by Eq. (1) described system is to be understood so that u reacts to the input vector f . From Eq. (1) So the solution is needed.
- f itself should, according to Eq. (2) react to v. In this sense, Eq. (2) already written down in the right direction. The cascading of both systems results in a relatively general image filter 3 whose transmission behavior is now to be investigated.
- Eigenvalues with a vanishing real part can be allowed in the sense of a subsequent border crossing. It should be noted that solutions of the homogeneous equation belonging to eigenvalues with a positive real part can initially also grow in time (in principle arbitrarily far), before they finally have to subside asymptotically after (in principle arbitrarily long time). This phenomenon can only occur with non-diagonalizable matrices, ie with matrices whose eigenvectors are not complete. The arbitrary initial time t 0 in Eq. (3) under these circumstances may be moved back to the distant past ( t 0 ⁇ - ⁇ ).
- ⁇ _ T 2 ⁇ A + s 1 + ⁇ 1 - 1 ⁇ 1 - ⁇ 1 ⁇ T 2 ⁇ A ,
- the matrices acting on states e.g. In Eq. (8) or (22) have ( N 1 N 2 ) x ( N 1 N 2 ) elements.
- the scalar case in which A, ⁇ 1 and T 2 are all multiples of the unit matrix 1.
- the pixels are then completely decoupled, ie the temporal course of the signal values of each pixel is filtered purely temporally on its own (in the same way for all pixels).
- the states and matrices then need to be related to only a single pixel, ie they effectively become scalars.
- the (scalar) transfer function has exactly one zero and exactly one pole, the position of which results directly from the (scalar) filter parameters. Eq. (8)].
- a more general case where the solution of eigenvalue problems is possible is that of matrices that act on the vector components as convolution. Because of the two indices on the basis vectors, the components are M n 1 n 2 n ' 1 n ' 2 a matrix M is now characterized by four indices (or two index pairs). For a matrix that acts as a convolution, the components only depend on n 1 - n ' 1 and n 2 - n' 2 , so they can be written again with only two indices (2D convolution matrix). It is favorable, the pictures u n 1 n 2 etc.
- n 2 1 N 1 N 2 e i 2 ⁇ k 1 n 1 / N 1 + k 2 n 2 / N 2 form an orthonormal basis as the original e n 1 , n 2 and are common eigenvectors of all cyclic convolution matrices.
- the base change from the e mn to the b k 1 k 2 corresponds to the discrete, finite Fourier transform of the associated vector components.
- the eigenvalue equation F ⁇ b k 1 k 2 ⁇ k 1 k 2 b k 1 k 2 for a (cyclic) folding matrix F is written out in components ⁇ n 1 ' n 2 ' F n 1 - n 1 ' .
- (28) can be regarded as a "scalar" transfer function, which (in the ⁇ or s range) indicates the transmission of spatial Fourier components to the respective same spatial frequency pair.
- This transfer function describes the response of the pixel value u n 1 n 2 to the whole set of all pixel values v n ' 1 n ' 2 without having to refer directly to the spatial Fourier components.
- Matrices and also A matrices, which act as (cyclic) convolution result for the discrete-time case from Eq.
- the curly brace is again the "scalar" transfer function which (in the z- domain) indicates the transfer of similar spatial Fourier components.
- the "scalar” case is included again.
- Eq. (29) results ⁇ n 1 - n 1 ' .
- n 1 - n 2 ' z ⁇ k 1 k 2 a k 1 k 2 N 1 N 2 ⁇ t ⁇ 1 k 1 k 2 z + 1 - z ⁇ 2 k 1 k 2 1 - z ⁇ 1 k 1 k 2 e i 2 ⁇ k 1 n 1 - n 1 ' / N 1 + k 2 n 2 - n 2 ' / N 2 ,
- Eq. (27) applies to the eigenvalues of convolution matrices, regardless of size.
- ⁇ k 1 k 2 2 A cos 2 ⁇ k 1 / N 1 + k 2 / N 2 + cos 2 ⁇ k 1 / N 1 - k 2 / N 2 + + 2 B cos 2 ⁇ k 1 / N 1 + cos 2 ⁇ k 2 / N 2 + C
- Equation (33) applies to all eigenvalues of the convolution matrices of the type described in Equation (32) and thus is uniformly applicable to zeros, poles and amplitude eigenvalues. It can be seen that, by adapting the few parameters A, B and C, the position of the N 1 N 2 eigenvalues of a convolution matrix can of course only be manipulated in a limited manner. Of course, the restrictions are reduced if more than three independent parameters and possibly larger folding matrices are taken into account.
- ⁇ _ ⁇ k 1 . k 2 a k 1 . k 2 s t 2 1 k 1 . k 2 + 1 s + ⁇ 1 1 k 1 . k 2 s t 2 2 k 1 . k 2 + 1 s + ⁇ 1 2 k 1 . k 2 ... b k 1 k 2 b k 1 k 2 T can be generalized. The further explication can be omitted here.
- Fig. 3 shows - schematically sketched - a signal plan in a canonical normal form.
- Current intermediate signal data w ( t ) (belonging at a time t ) are linearly combined from current (at the same time t belonging) filter input data u ( t ) and the previous discrete time, delayed intermediate signal data (symbol z -1 for the time delay).
- Actual and delayed intermediate signal data are linearly combined to the corrected result signal v ( t ).
- the coefficients used to form the linear combinations are scalar-valued when the image filter 3 operates on a pixel-by-pixel basis (purely temporal filtering). With true spatio-temporal filtering, they act as convolution matrix on the filter input data 31 or intermediate signal data, so that pixel values are also spatially mixed.
- the image filter 3 is according to FIG Fig. 4 realized.
- Fig. 4 schematically a signal flowchart is sketched in a preferred embodiment.
- Current intermediate signal data w ( t ) is linearly combined from delayed intermediate signal data and the difference of current filter input data u ( t ) and delayed intermediate signal data (symbol z -1 for the time delay).
- Current filter input data and delayed intermediate signal data are linearly combined to the corrected result signal v (t) .
- the coefficients used to form the linear combinations (here denoted by ⁇ 1 , ⁇ 0 and ⁇ 1 ) are again scalar-valued when the image filter 3 operates purely temporally, and act as a convolution matrix with true spatio-temporal filtering.
- the method according to the invention and the thermal imaging camera according to the invention can be used in systems which differ in terms of their functional definition (frequency response correction, noise suppression, image smoothing and image sharpening) and their realization (scalar / matrix-valued coefficients).
- the illustrated technical solution reduces the dynamic temperature measurement error and motion blur of an infrared thermal imager. This opens up areas of application previously reserved exclusively for cooled photonic IR sensors for low-cost microbolometer arrays (for example, measuring the tire temperature of passing trucks to detect dangerous situations). It is advantageous that various functions can be implemented with a fixed system design of the spatial-temporal image filter 3 merely by changing parameters, such as frequency response correction, noise suppression, image smoothing or image sharpening.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Description
Die Erfindung betrifft ein Verfahren sowie eine Wärmebildkamera zur berührungslosen Temperaturmessung oder zur Beobachtung von schnell bewegten IR-Szenen mit einem thermischen IR-Bildsensor, insbesondere für ungekühlte thermische IR-Kameras.The invention relates to a method and a thermal imaging camera for non-contact temperature measurement or observation of fast-moving IR scenes with a thermal IR image sensor, in particular for uncooled thermal IR cameras.
In vielen messenden Wärmebildkameras und IR-Beobachtungsgeräten werden heute häufig kostengünstige ungekühlte Mikrobolometerarrays eingesetzt. Mikrobolometer sind thermische IR-Bildsensoren und haben auf Grund ihres Wirkprinzips relativ lange Ansprechzeiten. Sie eignen sich daher kaum für Anwendungen mit höheren dynamischen Anforderungen. Schnelle radiometrische Temperaturmessungen sind nicht möglich oder stark fehlerbehaftet. Die Trägheit der Mikrobolometer äußert sich auch in Bewegungsunschärfe bei Kameraschwenks oder bei Objekten, die schnell durch das Bildfeld wandern.Cost-effective uncooled microbolometer arrays are frequently used today in many measuring thermal imaging cameras and IR observation devices. Microbolometers are thermal IR image sensors and have relatively long response times due to their mode of action. They are therefore hardly suitable for applications with higher dynamic requirements. Fast radiometric temperature measurements are not possible or severely error-prone. The inertia of the microbolometer also manifests itself in motion blur in camera pans or in objects that move quickly through the image field.
Das dynamische Verhalten von Mikrobolometerarrays ist in zahlreichen Publikationen thematisiert. Generell wird die thermische Zeitkonstante der Pixel τth - neben der Bildrate des Sensors - als eine wichtige Kenngröße und als ein wesentlicher limitierender Faktor für die "Schnelligkeit" einer IR-Kamera angesehen. Es finden sich in der Literatur jedoch keine Lösungsansätze zur dynamischen Kompensation. Die thermische Trägheit von Mikrobolometern wird lediglich als Fakt genannt. Lediglich für thermische Einzelsensoren, beispielsweise in Thermometern, ist eine Frequenzgangkorrektur bekannt.
Derzeit setzt man ausschließlich auf den Einsatz neuer, schnellerer Sensoren. Die technischen Möglichkeiten sind jedoch begrenzt (z. B. Reduzierung der "thermischen Masse") und von den Anwendern meist nicht beeinflussbar. Oftmals bleibt als Ausweg zur Vermeidung der thermischen Trägheit der Wärmebildaufnahmen nur der Umstieg auf gekühlte photonische Sensoren.
Allgemein gelten zeitkritische Messaufgaben nicht als Domäne der thermischen Sensoren, da der dynamische Messfehler nach der Zeitdauer τth noch etwa 37 % beträgt (wie es in
At present, the focus is solely on the use of new, faster sensors. However, the technical possibilities are limited (eg reduction of the "thermal mass") and can not usually be influenced by the users. Often, the only way out to avoid the thermal inertia of thermal imaging is to switch to cooled photonic sensors.
In general, time-critical measurement tasks are not regarded as the domain of the thermal sensors, since the dynamic measurement error after the period of time τ th is still about 37% (as described in US Pat
Der Erfindung liegt die Aufgabe zugrunde, eine neue Möglichkeit zur Korrektur der dynamischen Trägheit von thermischen IR-Sensorarrays zu finden, mittels der dynamische Messfehler einer radiometrischen bzw. thermischen Wärmebildkamera deutlich reduziert werden können. In einer erweiterten Aufgabenstellung soll auch eine Verringerung der Bewegungsunschärfe erreicht werden.The invention has for its object to find a new way to correct the dynamic inertia of thermal IR sensor arrays, by means of the dynamic measurement error of a radiometric or thermal thermal imaging camera can be significantly reduced. In an extended task also a reduction of motion blur should be achieved.
Erfindungsgemäß wird die Aufgabe bei einem Verfahren zur berührungslosen Temperaturmessung oder zur Beobachtung von schnell bewegten IR-Szenen mit einem thermischen IR-Bildsensor durch die folgenden Schritte gelöst:
- Aufnehmen einer IR-Szene mittels des thermischen IR-Bildsensors,
- Auslesen eines Sensorarrays des IR-Bildsensors mit einer Bildrate, die höher ist als eine der thermischen Zeitkonstante τth des IR-Bildsensors zugeordnete charakteristische Frequenz und Bereitstellen von Sensorsignalen in Form von vorverarbeiteten Filtereingangsdaten,
- Anwenden eines räumlich-temporalen Bildfilters, bei dem eine matrixwertige Übertragungsfunktion einstellbare Parameter enthält und in Form von Matrixelementen das Responseverhalten jedes einzelnen korrigierten Ergebnispixelwertes von ausgegebenen Ergebnisbilddaten auf einen gesamten Satz zeitlich vorangegangener Pixelwerte der Filtereingangsdaten charakterisiert, wobei eine Einstellung einer gewünschten Filtercharakteristik durch Anpassen von Parametern des Bildfilters zur Beeinflussung der Lage von Eigenwerten wenigstens einer Faltungsmatrix erfolgt, um mindestens Nullstellen oder Polstellen in der Übertragungsfunktion der Konfiguration des Bildfilters an eine konkrete Messaufgabe oder Beobachtungsaufgabe anzupassen, indem ein lineares Gleichungssystem zur Beeinflussung der Eigenwerte der wenigstens einen Faltungsmatrix mittels numerischer Verfahren mindestens näherungsweise gelöst wird.
- Recording an IR scene by means of the thermal IR image sensor,
- Reading a sensor array of the IR image sensor at a frame rate which is higher than a characteristic frequency associated with the thermal time constant τ th of the IR image sensor and providing sensor signals in the form of preprocessed filter input data,
- Applying a spatial-temporal image filter, wherein a matrix-valued transfer function contains adjustable parameters and characterizes in the form of matrix elements the response behavior of each corrected result pixel value of output result image data to an entire set of temporally preceding pixel values of the filter input data, wherein adjustment of a desired filter characteristic by adjusting parameters the image filter is used to influence the position of eigenvalues of at least one convolution matrix in order to adapt at least zeros or poles in the transfer function of the configuration of the image filter to a specific measurement task or observation task by at least approximately a linear equation system for influencing the eigenvalues of the at least one convolution matrix by means of numerical methods is solved.
Vorteilhaft geschieht das Auslesen des Sensorarrays des IR-Bildsensors mit einer Bildrate, die mindestens das Doppelte der charakteristischen Frequenz, die der thermischen Zeitkonstanten τth der Sensorelemente des Sensorarrays des IR-Bildsensors zugeordnet ist, beträgt. Dabei erfolgt das Auslesen des Sensorarrays bevorzugt mit einer Bildrate, die mindestens das Dreifache der charakteristischen Frequenz beträgt.Advantageously, the readout of the sensor array of the IR image sensor with a frame rate that is at least twice the characteristic frequency that the thermal time constant τ th is assigned to the sensor elements of the sensor array of the IR image sensor is. The readout of the sensor array is preferably carried out with a frame rate which is at least three times the characteristic frequency.
Das Anpassen der Parameter des Bildfilters zur Beeinflussung der Eigenwerte der mindestens einen Faltungsmatrix erfolgt zweckmäßig durch numerisches Lösen des linearen Gleichungssystems mittels Näherungslösungen, bevorzugt auf Basis der kleinsten Fehlerquadrate.
Zusätzlich zu den Polstellen und Nullstellen können für die Konfiguration des Bildfilters auch Amplituden-Eigenwerte durch Erweiterung des linearen Gleichungssystems mit weiteren linearen Gleichungen angepasst werden.
Bei der numerischen Lösung des linearen Gleichungssystems erweist es sich als vorteilhaft, zusätzlich eine Gewichtung von bestimmten Ortsfrequenzen vorzunehmen.The adaptation of the parameters of the image filter for influencing the eigenvalues of the at least one convolution matrix is expediently carried out by numerically solving the linear equation system by means of approximate solutions, preferably based on the least squares of errors.
In addition to the poles and zeros, amplitude eigenvalues can also be adjusted for the configuration of the image filter by extending the linear equation system with other linear equations.
In the case of the numerical solution of the linear system of equations, it proves to be advantageous additionally to carry out a weighting of specific spatial frequencies.
Des Weiteren wird die Aufgabe der Erfindung bei einer Wärmebildkamera zur berührungslosen Temperaturmessung oder zur Beobachtung von schnell bewegten IR-Szenen mit einem thermischen IR-Bildsensor, dadurch gelöst, dass mindestens ein räumlich-temporales Bildfilter dem Ausgang des IR-Bildsensors nachgeordnet ist, dass das Bildfilter als mindestens eine Bildverarbeitungseinheit zur Realisierung einer matrixwertigen Übertragungsfunktion ausgebildet ist, wobei in Berechnungen jedes Ergebnispixelwertes vxy(t) ein jeweils aktueller Sensorpixelwert uxy(t) eingesetzt ist sowie eine Anzahl zeitlich zurückliegender Sensorpixelwerte uxy(t-k·Δt), eine Anzahl benachbarter Sensorpixelwerte u(x±m·Δx)(y±n·Δy)(t) und eine Anzahl zeitlich zurückliegender benachbarter Sensorpixelwerte u(x±m·Δx)(y±n·Δy)(t-k·Δt) berücksichtigt sind.Furthermore, the object of the invention in a thermal imaging camera for non-contact temperature measurement or observation of fast-moving IR scenes with a thermal IR image sensor, achieved in that at least one spatial-temporal image filter is arranged downstream of the output of the IR image sensor, that is formed image filter as at least an image processing unit for realizing a matrixwertigen transfer function, wherein in calculations of each resultant pixel value v xy (t) is a respective current sensor pixel value u xy (t) is used and a number of time of past sensor pixel values u xy (tk · at), a number adjacent sensor pixel values u (x ± m · Δx) (y ± n · Δy) (t) and a number of temporally past adjacent sensor pixel values u (x ± m · Δx) (y ± n · Δy) (tk · Δt).
Vorteilhaft ist das Bildfilter durch wenigstens eine Parameterberechnungseinheit (5) mindestens temporär mit Mitteln zur Einstellung von Parametern des Bildfilters durch Beeinflussung der Lage von Eigenwerten wenigstens einer Faltungsmatrix verknüpft, um mindestens Nullstellen oder Polstellen der Übertragungsfunktion an eine konkrete Messaufgabe oder Beobachtungsaufgabe anzupassen, wobei die Mittel zur Einstellung der Parameter eine Rechnereinheit zur Lösung eines linearen Gleichungssystems für die Beeinflussung der Eigenwerte der wenigstens einen Faltungsmatrix mittels numerischer Näherungsverfahren beinhalten.Advantageously, the image filter is at least temporarily linked with means for adjusting parameters of the image filter by influencing the position of eigenvalues of at least one convolution matrix by at least one parameter calculation unit (5) in order to adapt at least zeros or poles of the transfer function to a specific measurement task or observation task, wherein the means for setting the parameters, a computer unit for solving a linear equation system for influencing the eigenvalues of the at least one convolution matrix by means of numerical approximation methods.
Vorzugsweise hat die Übertragungsfunktion des räumlich-temporalen Bildfilters in einer komplexen s-Ebene im Bildbereich einer Laplace-Transformation eine oder mehrere Nullstellen zur Kompensation der Polstellen der Übertragungsfunktion des IR-Bildsensors oder weist zweckmäßig eine oder mehrere Polstellen zur Begrenzung der Rauschbandbreite auf.Preferably, the transfer function of the spatio-temporal image filter in a complex s-plane in the image area of a Laplace transform has one or more zeros to compensate for the poles of the transfer function of the IR image sensor or expediently has one or more poles for limiting the noise bandwidth.
In einer ersten bevorzugten Variante sind ein oder mehrere Filterkoeffizienten entweder matrixwertig und als Faltungsmatrix im Ortsraum ausgebildet oder skalarwertig und so bemessen, dass mittels der Übertragungsfunktion des Bildfilters die Auswirkungen der thermische Trägheit der Sensorpixel in den zugehörigen Ergebnispixelwerten der ausgegebenen Ergebnisbilddaten reduziert werden.In a first preferred variant, one or more filter coefficients are either of matrix value and convolution matrix in spatial space or scalar valued and so dimensioned that the effects of the thermal inertia of the sensor pixels in the associated result pixel values of the output result image data are reduced by means of the transfer function of the image filter.
In einer zweiten vorteilhaften Variante sind ein oder mehrere Filterkoeffizienten matrixwertig und als Faltungsmatrix im Ortsraum ausgebildet so bemessen, dass mittels der Übertragungsfunktion des Bildfilters die Auswirkungen des Übersprechens der Sensorpixel in den zugehörigen Ergebnispixelwerten der ausgegebenen Ergebnisbilddaten reduziert werden.In a second advantageous variant, one or more filter coefficients are matrix-valued and designed as a convolution matrix in the spatial space such that the effects of crosstalk of the sensor pixels in the associated result pixel values of the output result image data are reduced by means of the transfer function of the image filter.
Bei einer zweckmäßigen dritten Variante sind ein oder mehrere Filterkoeffizienten matrixwertig und als Faltungsmatrix im Ortsraum ausgebildet so bemessen, dass mittels der Übertragungsfunktion des Bildfilters die ausgegebenen Ergebnisbilddaten von Ergebnispixelwerten eine räumliche Bildglättung oder Bildschärfung aufweisen.In an expedient third variant, one or more filter coefficients are matrix-valued and designed as a convolution matrix in the spatial space such that the output image data of result pixel values have spatial image smoothing or image sharpening by means of the transfer function of the image filter.
In einer vierten zweckmäßige Variante sind ein oder mehrere Filterkoeffizienten skalarwertig und so bemessen, dass mittels der Übertragungsfunktion des Bildfilters die Rauschbandbreite der Sensorpixel in den zugehörigen Ergebnispixelwerten der ausgegebenen Ergebnisbilddaten reduziert ist.In a fourth expedient variant, one or more filter coefficients are scalar-valued and dimensioned so that the noise bandwidth of the sensor pixels in the associated result pixel values of the output result image data is reduced by means of the transfer function of the image filter.
In einer weiteren bevorzugten Variante sind ein oder mehrere Filterkoeffizienten des Bildfilters in Abhängigkeit vom Bildinhalt fortlaufend variabel anpassbar, sodass mittels der Übertragungsfunktion des Bildfilters stets eine optimierte Filterwirkung bei minimalen Artefakten in den ausgegebenen Ergebnisbilddaten eingestellt werden kann. Eine gewünschte Übertragungsfunktion des Bildfilters wird besonders vorteilhaft durch ein IIR-System approximativ realisiert.In a further preferred variant, one or more filter coefficients of the image filter are continuously variably adjustable depending on the image content, so that by means of The transfer function of the image filter can always be an optimized filter effect with minimal artifacts in the output result image data can be set. A desired transfer function of the image filter is implemented particularly advantageously by an IIR system.
Die Erfindung basiert auf der Grundüberlegung, dass die Messeffizienz einer Wärmebildkamera auf Basis eines Mikrobolometerarrays bei schnellen Änderungen der Szene an der thermischen Trägheit seiner einzelnen Sensorelemente, die als thermische Widerstandsbrücken arbeiten, leidet. Daran lässt sich auch durch Änderung thermischer Eigenschaften des Sensorarrays (z. B. durch Verringerung der thermischen Masse jedes Einzelelements) nur geringfügig etwas verbessern. Grund dafür ist die thermische Trägheit der Bolometerelemente, die durch eine thermische Zeitkonstante τth als charakteristische Größe des Wärmeübergangsverhaltens beschrieben werden kann. Im Rahmen dieses Modells entspricht das Übergangsverhalten, wie es in
Die thermische Zeitkonstante τth (die bei den namhaften Herstellern von Bolometerarrays typischerweise im Bereich zwischen 7 und 15 ms liegt) ist aufgrund der Sensorkonstruktion durch Hardware-Optimierungen kaum noch signifikant zu verringern. Die Erfindung löst dieses Problem durch digitale Bildverarbeitung auf Basis einer räumlich-temporalen Filterung von ausgelesenen Sensordaten. Dabei fließen in die Berechnung jedes erzeugten Ergebnispixelwertes jeweils Messwerte eines betrachteten Sensorpixels sowie eine Anzahl zeitlich vorausgegangener Werte des betrachteten Sensorpixels, eine Anzahl räumlich benachbarter Pixelwerte und eine Anzahl zeitlich vorausgegangener Werte der räumlich benachbarten Pixel ein.
Dies wird dadurch erreicht, dass das räumlich-temporale Bildfilter eine Übertragungsfunktion aufweist, bei der in einer komplexen s-Ebene als Bildbereich einer Laplace-Transformation mindestens eine Nullstelle zur Kompensation von Polstellen der Signalübertragungsfunktion des Sensors und mindestens eine Polstelle zur Begrenzung einer Rauschbandbreite aufzufinden sind.
Die Übertragungsfunktion des räumlich-temporalen Bildfilters lässt sich mathematisch als rationale Funktion darstellen. Sie ist matrixwertig, wenn ein oder mehrere Filterkoeffizienten matrixwertig sind. Die matrixwertigen Filterkoeffizienten können im Ortsraum als Faltungsmatrix wirken und so bemessen sein, dass die Übertragungsfunktion die thermische Trägheit und das Übersprechen der Sensorpixel kompensiert.
In einer anderen Adaption sind ein oder mehrere Filterkoeffizienten matrixwertig, wirken als Faltungsmatrix im Ortsraum und sind so bemessen, dass die Übertragungsfunktion des Bildfilters zu einer räumlichen Bildglättung oder Bildschärfung führt. Eine weitere Ausführung sieht vor, dass ein oder mehrere Filterkoeffizienten skalarwertig und so bemessen sind, dass mittels der Übertragungsfunktion des Filters die thermische Trägheit der Sensorpixel kompensiert und/oder die Rauschbandbreite reduziert wird. In allen Fällen lassen sich ein oder mehrere Filterkoeffizienten variabel in Abhängigkeit vom Bildinhalt fortlaufend anpassen, um eine optimale Filterwirkung mit minimalen Artefakten zu erreichen.
Zur Realisierung der Filterfunktion werden numerische Approximationen mittels rekursiver Systeme angewendet, wobei insbesondere Filter mit unendlicher Impulsantwort, sog. IIR-Filter (Infinite Impulse Response Filter) verwendet werden.The invention is based on the fundamental idea that the measurement efficiency of a thermal imaging camera based on a microbolometer array suffers from rapid thermal changes in the scene due to the thermal inertia of its individual sensor elements, which operate as thermal resistance bridges. This can only be slightly improved by changing the thermal properties of the sensor array (eg by reducing the thermal mass of each individual element). The reason for this is the thermal inertia of the bolometer elements, which can be described by a thermal time constant τ th as the characteristic variable of the heat transfer behavior. In the context of this model, the transient behavior corresponds to that in
The thermal time constant τ th (which is typically in the range between 7 and 15 ms in the case of the well-known manufacturers of bolometer arrays) can barely be significantly reduced due to the sensor design by hardware optimizations. The invention solves this problem by digital image processing based on a spatial-temporal filtering of read sensor data. In each case, measured values of a considered sensor pixel as well as a number of temporally preceding values of the considered sensor pixel, a number of spatially adjacent pixel values and a number of temporally preceding values of the spatially adjacent pixels flow into the calculation of each generated result pixel value.
This is achieved in that the spatial-temporal image filter has a transfer function in which, in a complex s-plane as the image area of a Laplace transform, at least one zero for compensating pole positions of the Signal transmission function of the sensor and at least one pole to find a limitation of a noise bandwidth.
The transfer function of the spatio-temporal image filter can be represented mathematically as a rational function. It is matrix-valued if one or more filter coefficients are matrix-valued. The matrix-valued filter coefficients can act as convolution matrix in the spatial domain and be dimensioned such that the transfer function compensates for the thermal inertia and the crosstalk of the sensor pixels.
In another adaptation, one or more filter coefficients have a matrix value, act as a convolution matrix in spatial space and are dimensioned such that the transfer function of the image filter results in spatial image smoothing or image sharpening. A further embodiment provides that one or more filter coefficients are scalar-valued and dimensioned so that the thermal inertia of the sensor pixels is compensated for by means of the transfer function of the filter and / or the noise bandwidth is reduced. In all cases, one or more filter coefficients can be variably adjusted continuously depending on the image content in order to achieve an optimum filter effect with minimal artifacts.
To implement the filter function, numerical approximations are used by means of recursive systems, in particular filters with infinite impulse response, so-called IIR filters (Infinite Impulse Response Filter) are used.
Mit der Erfindung ist es somit möglich,
- den dynamischen Messfehler einer radiometrischen Wärmebildkamera mit Mitteln der digitalen Bildverarbeitung zu reduzieren, indem die thermische Zeitkonstante τth der Sensorelemente und das Übersprechen der Pixel mit numerischen Mitteln kompensiert bzw. in ihrer Auswirkung reduziert werden,
- durch Anwenden der gleichen Methode die Bewegungsunschärfe zu verringern,
- ein räumlich-temporales Bildfilter zur digitalen Signalverarbeitung in einer Wärmebildkamera zu beschreiben, das bei festem Systemdesign allein durch Wahl der Koeffizienten unterschiedliche Funktionen, wie Frequenzgangkorrektur (temporal differenzierend), Reduzierung der Rauschbandbreite (temporal integrierend) und Bildglättung (räumlich integrierend), realisiert.
- to reduce the dynamic measurement error of a radiometric thermal imaging camera with digital image processing means by compensating or reducing the effect of the thermal time constant τ th of the sensor elements and the crosstalk of the pixels by numerical means,
- by applying the same method to reduce motion blur,
- to describe a spatial-temporal image filter for digital signal processing in a thermal imaging camera, the fixed system design alone by selecting the coefficients different functions, such as frequency response correction (temporal differentiating), reducing the noise bandwidth (temporal integrating) and image smoothing (spatially integrating) realized.
Die Erfindung soll nachfolgend anhand von Ausführungsbeispielen und Abbildungen näher erläutert werden. Dabei zeigen:
- Fig. 1:
- eine schematische Darstellung des erfindungsgemäßen Verfahrens mit der Struktur eines räumlich-temporalen Bildfilters;
- Fig. 2:
- eine schematische Darstellung der Signalverarbeitung des Bilddatenstroms einer erfindungsgemäßen Wärmebildkamera mit einem räumlich-temporalen Bildfilter;
- Fig. 3:
- eine erste Ausführung der Erfindung in Form eines IIR-Systems als rekursive Umsetzung des räumlich-temporalen Bildfilters;
- Fig. 4:
- eine bevorzugte Ausführung der Erfindung für eine Wärmebildkamera mit räumlich-temporalem Bildfilter als rekursives IIR-System;
- Fig. 5:
- eine Darstellung des Übergangsverhaltens eines a-Si-Mikrobolometers mit Kennzeichnung der thermischen Zeitkonstante τth.
- Fig. 1:
- a schematic representation of the inventive method with the structure of a spatial-temporal image filter;
- Fig. 2:
- a schematic representation of the signal processing of the image data stream of a thermal imaging camera according to the invention with a spatial-temporal image filter;
- 3:
- a first embodiment of the invention in the form of an IIR system as a recursive implementation of the spatial-temporal image filter;
- 4:
- a preferred embodiment of the invention for a thermal imaging camera with spatiotemporal image filter as a recursive IIR system;
- Fig. 5:
- a representation of the transition behavior of an a-Si microbolometer with identification of the thermal time constant τ th .
Das erfindungsgemäße Verfahren ist in
Die vom Sensorarray 11 als Sensorbilder gelieferten Rohsignalwerte müssen in der Regel aufbereitet werden, um eine für eine erfolgreiche Filteranwendung ausreichende Signalqualität zu erreichen. Eine solche Aufbereitung wird als Vorverarbeitungsschritte wenigstens eine Ungleichmäßigkeitskorrektur (NUC, non-uniformity correction) beinhalten. Diese kann zum Beispiel als Einpunktkorrektur ausgeführt sein, wobei in Abhängigkeit von Temperatureinflüssen driftende Rohsignalwerte mit Hilfe eines Referenzbildes abgeglichen werden (Offset-Korrektur). Gängiger ist es jedoch, eine Zweipunktkorrektur auszuführen, wobei zusätzlich zur Offset-Korrektur mit einem Verstärkungsfaktorfeld (engl. gain field) eventuelle Unterschiede der Empfindlichkeiten der Sensorelemente und zugleich auch der natürliche Randlichtabfall und Transmissionsinhomogenitäten einer dem IR-Bildsensor 1 vorgelagerten Optik (hier nicht gezeichnet) ausgeglichen werden können. Solche Verstärkungsfaktorfelder können werksseitig einkalibriert sein. Die für die Offset-Korrektur benötigten Referenzbilder werden im Live-Betrieb üblicherweise in gewissen Abständen mittels eines Shutters ermittelt bzw. aktualisiert oder sie können von shutterlosen Verfahren geliefert werden.
Zusätzlich zur Ungleichmäßigkeitskorrektur können die Vorverarbeitungsschritte beispielsweise auch noch ein Defektpixelersetzungsschema beinhalten.
Die vorgenannten Korrekturen einer Signalvorverarbeitung können bereits in einer chipinternen Schaltung des IR-Bildsensors 1 enthalten sein oder auch in einer nachgeordneten separaten Bildvorverarbeitungseinheit 13 (nur in
The raw signal values supplied by the
For example, in addition to the unevenness correction, the preprocessing steps may also include a defect pixel replacement scheme.
The aforementioned corrections of a signal preprocessing can already be contained in an on-chip circuit of the
Das Ergebnis der Signalvorverarbeitung der Ausleseschaltung 12 ist ein Bilddatenstrom 2 von aufbereiteten Sensorbildern, der als Filtereingangsdaten 31 in der Form uxy(t) bereitgestellt wird. Das räumlich-temporale Bildfilter 3 gibt dann korrigierte Ergebnisbilddaten 4 in der Form vxy(t) aus. Für die Funktion des Bildfilters 3 ist es dabei grundsätzlich unerheblich, ob der ausgelesene Bilddatenstrom 2 von Sensor-Rohbildern direkt in einem Online-Vorgang durch die oben beschriebenen Vorverarbeitungsschritte in den Datenstrom der aufbereiteten Sensorbilder überführt oder unmittelbar als Filtereingangsdaten 31 verwendet wird. Stattdessen können die Filtereingangsdaten 31 beispielsweise auch so bereitgestellt werden, dass durch Vorverarbeitungsschritte aufbereitete Sensorbilder in einem Pufferspeicher zwischengespeichert (hier nicht gezeichnet) und zur Bereitstellung von Filtereingangsdaten 31 dann sukzessive wieder gelesen werden. Natürlich ist es auch möglich, den in
Das räumlich-temporale Bildfilter 3 stellt ein dreidimensionales Filter dar, bei dem aus dem zweidimensional ausgelesenen Bilddatenstrom 2 aus dem IR-Sensorarray 11 auf die zwei räumlichen Dimensionen 32 der Filtereingangsdaten 31 zugegriffen und als dritte Dimension 33 der zeitliche Verlauf der aus dem Bilddatenstrom 2 resultierenden Filtereingangsdaten 31 betrachtet wird. Das Bildfilter 3 bewirkt somit eine Verknüpfung räumlicher und zeitlicher Komponenten des Bilddatenstroms 2, wie nachfolgend noch detailliert erläutert wird.The result of the signal pre-processing of the read-
The spatio-
Die so vorverarbeiteten Filtereingangsdaten 31 weisen so eine deutlich geringere Stör- und Rauschsignalquote auf als die Rohdaten aus dem Bilddatenstrom 2, wodurch sich das Bildfilter 3 auf die reine Kompensation von Auswirkungen der dynamischen Messfehler durch schnelle Änderungen in der beobachteten IR-Szene ausrichten kann. Die Verknüpfung der einlaufenden Bildsignale als Filtereingangsdaten 31 erfolgt dabei, wie oben schon zu
Für die nachfolgend noch näher erläuterte Bildfilterung mittels im Bildfilter 3 enthaltener Koeffizientenmatizen ist in der Ausführung von
The thus preprocessed
For the below described in more detail image filtering by means contained in the
Das räumlich-temporale Bildfilter 3 wird erfindungsgemäß möglichst vorteilhaft als temporales Bildfilter 3 parametrisiert, so dass je nach Konfiguration eine glättende (integrierende) oder schärfende (differenzierende) Charakteristik entsteht. Um einfach räumlich-temporale Verallgemeinerungen gleich mit abzudecken, wählt man zweckmäßig von vornherein eine Vektor- und Matrixschreibweise.
Begonnen wird mit der Beschreibung in stetiger Zeit und mit Betrachtung eines einfachen linearen Systems der Art:
It starts with the description in continuous time and considering a simple linear system of the kind:
Die allgemeine Lösung des Anfangswertproblems für Gl. (1) kann mit Hilfe des Propagators V 1(τ) = exp(-τ·Γ1) als
Dazu müssen die Eigenwerte von Γ1 alle einen positiven Realteil haben. Eigenwerte mit verschwindendem Realteil können im Sinne eines nachträglichen Grenzübergangs mit erlaubt werden. Man sollte beachten, dass zu Eigenwerten mit positivem Realteil gehörige Lösungen der homogenen Gleichung zeitlich zunächst auch (im Prinzip beliebig weit) anwachsen können, bevor sie nach (im Prinzip beliebig langer Zeit) letztlich doch asymptotisch abklingen müssen. Dieses Phänomen kann nur bei nichtdiagonalisierbaren Matrizen auftreten, d. h. bei Matrizen, deren Eigenvektoren nicht vollständig sind.
Der beliebige Anfangszeitpunkt t 0 in Gl. (3) darf unter diesen Umständen in die ferne Vergangenheit (t 0 → -∞) zurückverlegt werden. Das System "vergisst" dann seine Anfangsbedingung u 0 nach hinreichend langer Zeit und es verbleibt nur der "getriebene" Anteil der Lösung,
For this, the eigenvalues of Γ 1 must all have a positive real part. Eigenvalues with a vanishing real part can be allowed in the sense of a subsequent border crossing. It should be noted that solutions of the homogeneous equation belonging to eigenvalues with a positive real part can initially also grow in time (in principle arbitrarily far), before they finally have to subside asymptotically after (in principle arbitrarily long time). This phenomenon can only occur with non-diagonalizable matrices, ie with matrices whose eigenvectors are not complete.
The arbitrary initial time t 0 in Eq. (3) under these circumstances may be moved back to the distant past ( t 0 → -∞). The system then "forgets" its initial condition u 0 after a sufficiently long time and leaves only the "driven" portion of the solution,
Diese (retardierte) Form der Lösung wird meist als Ausdruck des Kausalitätsprinzips angesehen. Wie man sieht, ist es in diesem Fall keine unabhängige Forderung; unter den genannten Bedingungen verhält sich das (Teil-)System von selbst "kausal". Die Responsefunktion (retardierte Greensche Funktion; Impulsantwort; Suszeptibilitätskern im Zeitbereich) des (Teil-)Systems ist durch χ 1(τ)=θ(τ)V 1(τ) gegeben [θ(τ), Heaviside-Sprungfunktion bzw. Einheitssprung], und ihre Fouriertransformierte liefert die Übertragungsfunktion (Transferfunktion; Systemfunktion; verallgemeinerte Suszeptibilität) in der komplexen Frequenzebene,
Es kann auch die Laplace-Variable s benutzt werden, ω=is. Mit der genannten Annahme über Γ1 ist dieses Integral zumindest für Re s > 0 konvergent und liefert
In der linken komplexen s-Halbebene gilt dieser Ausdruck weiter. Auf der rechten Seite von Gl. (6) erscheint die Resolvente von -Γ1. Sie hat Polstellen genau bei den Eigenwerten von -Γ1. Ihre explizite analytische Bestimmung erfordert in der Regel die analytische Lösung des Eigenwertproblems für Γ1.
Für das Teilsystem Gl. (2) lässt sich die Responsefunktion aus der Gleichung selbst unmittelbar ablesen, χ 2(τ)=[δ'(τ)T 2+δ(τ)1]·A. Sie ist nicht retardiert, sondern instantan, jedoch ebenfalls kausal, χ 2(τ<0)=0. Die zugehörige Übertragungsfunktion lautet
For the subsystem Eq. (2) the response function can be read directly from the equation itself, χ 2 ( τ ) = [ δ ' ( τ ) T 2 + δ ( τ ) 1] · A. It is not retarded, but instantaneously, but also causally, χ 2 ( τ <0) = 0. The associated transfer function is
Das kaskadierte Gesamtsystem, d. h. die lineare Response von u auf v, wird somit durch die Übertragungsfunktion χ (ω)= χ 1(ω)· χ 2(ω), d. h.
Dies lässt sich auch durch direktes Umformen von Gl. (8) nachweisen. Es sei hier noch angemerkt, dass die Wirkung des "differenzierenden" Teilsystems offensichtlich durch die Wahl T2 = 0 (und A = 1) gänzlich "abgeschaltet" werden kann. Ein völliges Abschalten des "integrierenden" Teilsystems ist durch passende Wahl von Γ1 zwar nicht unmittelbar möglich, jedoch braucht hierzu in einer Realisierung des Bildfilters 3 lediglich auch das "Zwischensignal" f(t) abgegriffen werden.
Für eine zeitdiskrete Beschreibung kann man an mit Rückwärts-Differenzen angenäherte Formen von Gl. (1) und Gl. (2) anknüpfen,
Bezeichnet man die Zeitpunkte auf dem Raster mit tn =t 0+nΔ (t 0 beliebig) und führt die Bezeichnung u (n)=u(tn ) etc. ein, geht Gl. (11) in
For a discrete-time description, one can approximate to backward-difference approximated forms of Eq. (1) and Eq. (2) tie in,
If the times on the grid are denoted by t n = t 0 + n Δ ( t 0 arbitrary) and the designation u ( n ) = u ( t n ) etc. is entered, Eq. (11) in
Wie im kontinuierlichen Fall ist das (Teil-)System kausal, wenn unsere Annahme über Γ1 erfüllt ist. Analog zum Integral in Gl. (4) ist die Reihe in Gl. (16) dann (für vernünftige Eingangssignale f) konvergent.
Die diskrete Responsefunktion des "integrierenden" Teilsystems ist folglich
The discrete response function of the "integrating" subsystem is therefore
Die diskrete Responsefunktion Y(N) des Gesamtsystems lässt sich aus Gl. (17) und Gl. (18) zu
Zur Anwendung auf die Bildfilterung wird der Zustandsvektor u (und analog f, v etc.) als Vektor in einem (N 1 x N 2)-dimensionalen Raum angenommen und seine (zweidimensional indizierten) Komponenten u n
For application to image filtering, the state vector u (and analog f, v, etc.) is assumed to be a vector in a ( N 1 x N 2 ) -dimensional space and its (two-dimensionally indexed) components u n
Es gibt jedoch Fälle, in denen dies doch möglich ist, zum einen natürlich den skalaren Fall, bei dem A, Γ1 und T 2 allesamt Vielfache der Einheitsmatrix 1 sind. Die Pixel sind dann völlig entkoppelt, d. h. der zeitliche Verlauf der Signalwerte jedes Pixels wird für sich genommen rein temporal gefiltert (auf dieselbe Weise für alle Pixel). Die Zustände und Matrizen brauchen dann auch auf nur ein einzelnes Pixel bezogen werden, d. h. sie werden effektiv zu Skalaren. In diesem Fall weist die (skalare) Übertragungsfunktion genau eine Nullstelle und genau eine Polstelle auf, deren Lage unmittelbar aus den (skalaren) Filterparametern hervorgeht [siehe z. B. Gl.(8)].However, there are cases in which this is possible, on the one hand, of course, the scalar case, in which A, Γ 1 and T 2 are all multiples of the
Ein allgemeinerer Fall, bei dem die Lösung der Eigenwertprobleme möglich ist, ist der von Matrizen, die auf die Vektorkomponenten als Faltung wirken. Wegen der zwei Indizes an den Basisvektoren sind die Komponenten M n
Die Vektoren
Die Eigenwertgleichung F·b k
The vectors
The eigenvalue equation F · b k
Die geschweifte Klammer ist wieder die "skalare" Übertragungsfunktion, die (im z-Bereich) die Übertragung gleichartiger räumlicher Fourierkomponenten angibt. In den Gln. (28) und (30) ist auch der "skalare" Fall wieder mit eingeschlossen. Als Analogon von Gl. (29) ergibt sich
Die Gleichung (33) gilt für alle Eigenwerte der Faltungsmatrizen der in Gleichung (32) beschriebenen Art und ist somit einheitlich für Nullstellen, Polstellen und Amplituden-Eigenwerte anwendbar. Man sieht, dass durch Anpassung der wenigen Parameter A, B und C die Lage der N 1 N 2 Eigenwerte einer Faltungsmatrix natürlich nur auf eingeschränkte Weise manipuliert werden kann. Die Einschränkungen werden freilich geringer, wenn mehr als drei unabhängige Parameter und ggf. größere Faltungsmatrizen in Betracht gezogen werden.Equation (33) applies to all eigenvalues of the convolution matrices of the type described in Equation (32) and thus is uniformly applicable to zeros, poles and amplitude eigenvalues. It can be seen that, by adapting the few parameters A, B and C, the position of the N 1 N 2 eigenvalues of a convolution matrix can of course only be manipulated in a limited manner. Of course, the restrictions are reduced if more than three independent parameters and possibly larger folding matrices are taken into account.
Die für die Filterkonfiguration entscheidenden Nullstellen s=-1/(t 2) k
Diese linearen Gleichungssysteme sind - außer bei Faltungsmatrizen in Vollbildgröße - natürlich überbestimmt, d. h. sie besitzen keine exakte Lösung. Es können jedoch numerisch dafür optimale Näherungslösungen bestimmt werden, die das jeweilige Gleichungssystem im Sinne der kleinsten Fehlerquadrate bestmöglich befriedigen. Dabei kann auch noch eine bestimmte Gewichtung der einzelnen Ortsfrequenzen berücksichtigt werden, wodurch im Sinne einer Kompromissfindung eine angestrebte Filterwirkung in bestimmten Ortsfrequenzbereichen besser, jedoch zulasten der Wirkung in anderen Ortsfrequenzbereichen, erreicht wird. Je nach Größe der Restfehler kann man dann hoffen, dass so ermittelte Filterparameter eine Filtercharakteristik herbeiführen, die das angestrebte Verhalten mehr oder weniger gut annähert.The zeros s = -1 / ( t 2 ) k decisive for the filter configuration
These linear systems of equations are, of course, overdetermined, except in the case of full-size convolution matrices, ie they have no exact solution. However, optimal approximate solutions can be determined numerically for this, which satisfy the respective system of equations in the sense of the least squares in the best possible way. In this case, even a certain weighting of the individual spatial frequencies can be taken into account, whereby a desired filter effect in certain spatial frequency ranges better, but at the expense of the effect in other spatial frequency ranges, is achieved in the sense of a compromise finding. Depending on the size of the residual errors, it can then be hoped that filter parameters determined in this way will produce a filter characteristic that more or less approximates the desired behavior.
In obigen Überlegungen stand die Kaskadierung zweier Teilsysteme erster Ordnung am Anfang, wobei ein "differenzierendes" Teilsystem Gl. (2) und ein "integrierendes" Teilssystem Gl. (1) über den Zwischensignalvektor f hintereinandergeschaltet waren. Wie die für die Übertragungsfunktionen erhaltenen Ausdrücke zeigen, ergab dies (im "skalaren" Fall) die Möglichkeit, durch eine Nullstelle eine Polstelle und/oder durch eine Polstelle eine Nullstelle mittels des kaskadierten Gesamtfilters zu kompensieren. Es ist klar, dass mit Hilfe von zusätzlichen Zwischensignalvektoren ohne Weiteres ganz analog auch mehrere Teilsysteme erster Ordnung von einem oder beiden Teilsystemtypen kaskadiert werden können. Im skalaren Fall führt dies zu der Möglichkeit, mehrere Polstellen und/oder Nullstellen mit dem entstehenden Gesamtfilter zu kompensieren.In the above considerations, the cascading of two first order subsystems was at the beginning, with a "differentiating" subsystem Eq. (2) and an "integrating" subsystem Eq. (1) were connected in series via the intermediate signal vector f . As the expressions obtained for the transfer functions show, this (in the "scalar" case) has the potential to compensate for a pole by a zero and / or a pole by a pole by means of the cascaded overall filter. It is clear that, with the aid of additional intermediate signal vectors, several first-order subsystems of one or both subsystem types can also be cascaded in a completely analogous manner. In the scalar case, this leads to the possibility of compensating for multiple poles and / or zeros with the resulting total filter.
Dieselbe Verallgemeinerung ist aber auch im Fall von Koeffizienten, die Matrizen sind, völlig analog möglich. Von Γ1 und T 2 (und ggf. auch A) gibt es dann mehrere Ausgaben Γ1 (α) bzw. T 2 (β). Wenn die Matrizen als Faltungsmatrizen fungieren und man auf zusätzliche Duplikate von A verzichtet, ist ohne erneute Rechnung klar, dass beispielsweise Gl. (28) zu
In
In einer zweiten bevorzugten Ausführung der erfindungsgemäßen Wärmebildkamera ist das Bildfilter 3 gemäß
In
In
Das erfindungsgemäße Verfahren und die erfindungsgemäße Wärmebildkamera sind bei Systemen, die sich hinsichtlich ihrer funktionellen Bestimmung (Frequenzgangkorrektur, Rauschunterdrückung, Bildglättung und Bildschärfung) und ihrer Realisierung (skalare / matrixwertige Koeffizienten) unterscheiden, verwendbar.
Die dargestellte technische Lösung verringert den dynamischen Temperaturmessfehler und die Bewegungsunschärfe einer IR-Wärmebildkamera. Damit öffnen sich Anwendungsgebiete, die bisher ausschließlich den gekühlten photonischen IR-Sensoren vorbehalten waren, für preiswerte Mikrobolometerarrays (beispielsweise Messung der Reifentemperatur vorbeifahrender LKW zur Erkennung von Gefahrensituationen). Von Vorteil ist, dass sich mit einem festen Systemdesign des räumlich-temporalen Bildfilters 3 allein durch Ändern von Parametern verschiedenartige Funktionen realisieren lassen, wie Frequenzgangkorrektur, Rauschunterdrückung, Bildglättung oder Bildschärfung.The method according to the invention and the thermal imaging camera according to the invention can be used in systems which differ in terms of their functional definition (frequency response correction, noise suppression, image smoothing and image sharpening) and their realization (scalar / matrix-valued coefficients).
The illustrated technical solution reduces the dynamic temperature measurement error and motion blur of an infrared thermal imager. This opens up areas of application previously reserved exclusively for cooled photonic IR sensors for low-cost microbolometer arrays (for example, measuring the tire temperature of passing trucks to detect dangerous situations). It is advantageous that various functions can be implemented with a fixed system design of the spatial-
- 11
- (thermischer) IR-Bildsensor(thermal) IR image sensor
- 1111
- (IR-)Sensorarray(IR) sensor array
- 1212
- Ausleseschaltungreadout circuit
- 1313
- Bildvorverarbeitungseinheitimage pre-processing
- 22
- BilddatenstromImage data stream
- 33
- Bildfilterimage filters
- 3131
- FiltereingangsdatenFilter input data
- 3232
- räumliche Dimensionspatial dimension
- 3333
- zeitliche Dimensiontemporal dimension
- 44
- (korrigierte) Ergebnisbilddaten(corrected) result image data
- 55
- Parameterberechnungseinheit (für Bildfiltereinstellung bzw. -anpassung)Parameter calculation unit (for image filter adjustment)
Claims (15)
- Method for contactless temperature measurement or for observation of quickly moving IR scenes with a thermal IR image sensor (1) comprising the steps of:- capturing an IR scene by means of the thermal IR image sensor (1),- reading out a sensor array (11) of the IR image sensor (1) at a frame rate which is higher than a characteristic frequency associated with the thermal time constant τth of the IR image sensor and providing sensor signals in the form of pre-processed filter input data (31),- applying a spatio-temporal image filter (3), in which a matrix-valued transfer function contains adjustable parameters and in the form of matrix elements characterizes the response behaviour of each individual corrected result pixel value of output result image data (4) to an entire set of temporally preceding pixel values of the filter input data (31), wherein a desired filter characteristic is adjusted by:• adjusting parameters of the image filter (3) for influencing the position of eigenvalues of at least one convolution matrix in order to adapt at least zeros or poles in the transfer function of the configuration of the image filter (3) to a specific measurement task or observation task, wherein a linear equation system for influencing the eigenvalues of the at least one convolution matrix is solved at least approximately by means of numerical methods.
- Method according to claim 1, wherein the sensor array (11) of the IR image sensor (1) is read out at a frame rate which is at least twice the characteristic frequency associated with the thermal time constant τth of the sensor elements of the sensor array (11) of the IR image sensor (1).
- Method according to claim 1, wherein the sensor array (11) of the IR image sensor (1) is read out at a frame rate which is at least three times the characteristic frequency associated with the thermal time constant τth of the sensor elements of the sensor array (11) of the IR image sensor (1).
- Method according to claim 1, wherein the parameters of the image filter (3) are adjusted for influencing the eigenvalues of the at least one convolution matrix by numerically solving the linear equation system by means of approximate solutions based on the least error squares.
- Method according to claim 1, wherein, in addition to the poles and zeros, also amplitude eigenvalues of the configuration of the image filter (3) are adjusted by extending the linear equation system by other linear equations.
- Method according to claim 1, wherein certain spatial frequencies are additionally weighted in the numerical solution of the linear equation system.
- Thermal camera for contactless temperature measurement or for observation of quickly moving IR scenes with a thermal IR image sensor (1), characterized in that- at least one spatio-temporal image filter (3) is arranged downstream of the output of the IR image sensor (1), wherein a sensor array (11) of the IR image sensor (1) is read out at a frame rate which is higher than a characteristic frequency associated with the thermal time constant τth of the IR image sensor and wherein sensor signals in the form of pre-processed filter input data (31) are provided,- the image filter (3) is constructed as at least one image processing unit for implementing a matrix-valued transfer function, wherein- the matrix-valued transfer function contains adjustable parameters and in the form of matrix elements characterizes the response behaviour of each individual corrected result pixel value of output result image data (4) to an entire set of temporally preceding pixel values of the filter input data (31), wherein a desired filter characteristic is adjusted by adjusting parameters of the image filter (3) for influencing the position of eigenvalues of at least one convolution matrix in order to adapt at least zeros or poles in the transfer function of the configuration of the image filter (3) to a specific measurement task or observation task, wherein a linear equation system for influencing the eigenvalues of the at least one convolution matrix is solved at least approximately by means of numerical methods.
- Thermal camera according to claim 7, characterized in that in at least one parameter calculation unit (5) the image filter (3) at least temporarily has means for adjusting parameters of the image filter (3) by influencing the position of eigenvalues of at least one convolution matrix in order to adapt at least zeros or poles of the transfer function to a specific measurement task or observation task, wherein the means for adjusting the parameters include a computer unit for solving a linear equation system for influencing the eigenvalues of the at least one convolution matrix by means of numerical approximation methods.
- Thermal camera according to claim 7, characterized in that in a complex s-plane in the image area of a Laplace transformation the transfer function of the spatio-temporal image filter (3) has one or more zeros to compensate for the poles of the transfer function of the IR image sensor (1) or has one or more poles to limit the noise bandwidth.
- Thermal camera according to claim 7, characterized in that one or more filter coefficients are either matrix-valued and configured as a convolution matrix in the spatial domain or scalar-valued and dimensioned such that by means of the transfer function of the image filter (3) the effects of the thermal inertia of the sensor pixels in the associated result pixel values of the output result image data (4) are reduced.
- Thermal camera according to claim 7, characterized in that one or more filter coefficients are matrix-valued and configured as at least one convolution matrix in the spatial domain are dimensioned such that by means of the transfer function of the image filter (3) the effects of the crosstalk of the sensor pixels of the IR sensor array (11) in the associated result pixel values of the output result image data (4) are reduced.
- Thermal camera according to claim 7, characterized in that one or more filter coefficients are matrix-valued and configured as at least one convolution matrix in the spatial domain are dimensioned such that by means of the transfer function of the image filter (3) the output result image data (4) of result pixel values have a spatial image smoothing or image sharpening.
- Thermal camera according to claim 7, characterized in that one or more filter coefficients are scalar-valued and dimensioned such that by means of the transfer function of the image filter (3) the noise bandwidth of the sensor pixels in the associated result pixel values of the output result image data (4) is reduced.
- Thermal camera according to claim 7, characterized in that one or more filter coefficients of the image filter (3) are continuously variably adjustable depending on the image content, so that by means of the transfer function of the image filter (3) there is always an optimised filter effect set with minimal artefacts in the output result image data (4).
- Thermal camera according to claim 7, characterized in that a desired transfer function of the image filter (3) is implemented approximately by an IIR system.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102016118638.4A DE102016118638A1 (en) | 2016-09-30 | 2016-09-30 | Method and thermal imaging camera for non-contact temperature measurement or observation of fast-moving IR scenes with a thermal IR image sensor |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3301640A1 EP3301640A1 (en) | 2018-04-04 |
EP3301640B1 true EP3301640B1 (en) | 2018-11-28 |
Family
ID=59997216
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17193958.0A Active EP3301640B1 (en) | 2016-09-30 | 2017-09-29 | Method and thermal camera for the contactless measurement of a temperature or for observing quickly moving ir scenes with a thermal ir imager |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP3301640B1 (en) |
DE (1) | DE102016118638A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109919071B (en) * | 2019-02-28 | 2021-05-04 | 沈阳天眼智云信息科技有限公司 | Flame identification method based on infrared multi-feature combined technology |
US20210378519A1 (en) * | 2019-05-23 | 2021-12-09 | Yildiz Teknik Universitesi | A body temperature monitoring system |
FR3103939B1 (en) * | 2019-11-28 | 2022-07-22 | Office National Detudes Rech Aerospatiales | METHOD FOR CAPTURING IMAGES USING SENSITIVE ELEMENTS WITH MEMORY EFFECT |
EP4084467B1 (en) | 2021-04-30 | 2023-02-01 | Axis AB | A method and a thermal camera having a microbolometer detector for capturing a sequence of image frames |
CN116652681A (en) * | 2023-05-24 | 2023-08-29 | 浙江沃沃阀门有限公司 | Automatic multi-station machining equipment and method for forged steel valve cover |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0864224B1 (en) * | 1995-12-01 | 2002-09-25 | QinetiQ Limited | Imaging system |
US6211515B1 (en) * | 1998-10-19 | 2001-04-03 | Raytheon Company | Adaptive non-uniformity compensation using feedforward shunting and wavelet filter |
US6330371B1 (en) * | 1998-10-19 | 2001-12-11 | Raytheon Company | Adaptive non-uniformity compensation using feedforward shunting and min-mean filter |
DE10110294A1 (en) * | 2001-02-23 | 2001-10-18 | Univ Dresden Tech | Digital signal sampling rate conversion method by decimating, down-sampling and up-sampling |
US6910060B2 (en) * | 2001-05-21 | 2005-06-21 | Computational Sensor Corp. | Spatio-temporal filter and method |
US9756264B2 (en) * | 2009-03-02 | 2017-09-05 | Flir Systems, Inc. | Anomalous pixel detection |
CN102798849B (en) * | 2012-08-14 | 2014-03-26 | 中国科学院光电技术研究所 | Method for eliminating light spot centroid shift caused by crosstalk |
-
2016
- 2016-09-30 DE DE102016118638.4A patent/DE102016118638A1/en not_active Withdrawn
-
2017
- 2017-09-29 EP EP17193958.0A patent/EP3301640B1/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
EP3301640A1 (en) | 2018-04-04 |
DE102016118638A1 (en) | 2018-04-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3301640B1 (en) | Method and thermal camera for the contactless measurement of a temperature or for observing quickly moving ir scenes with a thermal ir imager | |
DE69319677T2 (en) | Device for compensating jitter | |
DE112016001040T5 (en) | Method and system for real-time noise removal and image enhancement of high dynamic range images | |
DE69113889T2 (en) | Passive distance measurement and fast autofocusing. | |
DE102010023166B4 (en) | Thermal camera | |
DE102010023168B4 (en) | Method and device for correcting non-uniform sensitivity of detector elements in thermal imaging cameras | |
DE69132203T2 (en) | System and method for real-time mixing of video images from multiple sources | |
DE69010411T2 (en) | SCENE-RELATED COMPENSATION OF INHOMOGENOUSLY DISTRIBUTED DETECTOR SENSITIVITY OF AN IMAGE DETECTOR MATRIX. | |
DE69422784T2 (en) | Scene-dependent correction of the non-uniformity of scanning infrared sensors | |
DE68925456T2 (en) | Process for image contrast enhancement | |
EP2791896B1 (en) | Method for generating super-resolution images having improved image resolution and measuring device | |
DE4000343A1 (en) | AUTOMATIC FOCUSING DEVICE | |
DE102012106584A1 (en) | Method and apparatus for image reconstruction | |
DE3905591A1 (en) | DEVICE FOR OBTAINING CONTRAST HIGH IMAGES | |
DE102010023170A1 (en) | Thermal imaging camera and method for recording and / or modification and reproduction of thermal images of a scene and / or an object | |
DE60222127T2 (en) | Apparatus and method for reading composite microbolometer matrices | |
DE112017006779T5 (en) | CAMERA SYSTEM INCLUDING OBJECTIVE WITH MAGNIFICATION GRADIENT | |
WO2016087277A1 (en) | Digital image capture system and method for correcting errors in such a system | |
DE112006001641T5 (en) | Signal processing device | |
DE202013008745U1 (en) | Thermal camera | |
DE102017112484A1 (en) | Method and device for image correction | |
DE102006036845A1 (en) | Method and device for scanning a document by means of a line camera | |
EP3391329B1 (en) | Determination of brightness values of virtual pixels | |
DE102015109878B3 (en) | Correction of a patterned image defect | |
DE69231417T2 (en) | Method and filter for removing noise in an image sequence |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180425 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06T 5/20 20060101AFI20180530BHEP |
|
INTG | Intention to grant announced |
Effective date: 20180622 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1071108 Country of ref document: AT Kind code of ref document: T Effective date: 20181215 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502017000422 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D Ref country code: NL Ref legal event code: FP |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190328 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190228 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
REG | Reference to a national code |
Ref country code: NO Ref legal event code: T2 Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190301 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190328 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502017000422 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20190829 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R082 Ref document number: 502017000422 Country of ref document: DE Representative=s name: WALDAUF, ALEXANDER, DIPL.-ING. DR.-ING., DE Ref country code: DE Ref legal event code: R081 Ref document number: 502017000422 Country of ref document: DE Owner name: JENOPTIK OPTICAL SYSTEMS GMBH, DE Free format text: FORMER OWNER: JENOPTIK ADVANCED SYSTEMS GMBH, 22880 WEDEL, DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PFUS Owner name: JENOPTIK OPTICAL SYSTEMS GMBH, DE Free format text: FORMER OWNER: JENOPTIK ADVANCED SYSTEMS GMBH, DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: PD Owner name: JENOPTIK OPTICAL SYSTEMS GMBH; DE Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), MERGE; FORMER OWNER NAME: JENOPTIK ADVANCED SYSTEMS GMBH Effective date: 20200304 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20200305 AND 20200311 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190929 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190929 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20190930 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190930 |
|
REG | Reference to a national code |
Ref country code: NO Ref legal event code: CHAD Owner name: JENOPTIK OPTICAL SYSTEMS GMBH, DE Ref country code: NO Ref legal event code: CREP Representative=s name: BRYN AARFLOT AS, STORTINGSGATA 8, 0161 OSLO, NORGE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20170929 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181128 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: NO Payment date: 20230919 Year of fee payment: 7 Ref country code: NL Payment date: 20230920 Year of fee payment: 7 Ref country code: GB Payment date: 20230921 Year of fee payment: 7 Ref country code: AT Payment date: 20230915 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: SE Payment date: 20230921 Year of fee payment: 7 Ref country code: FR Payment date: 20230918 Year of fee payment: 7 Ref country code: DE Payment date: 20230919 Year of fee payment: 7 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20231001 Year of fee payment: 7 |