CN102800059B - Image visibility enhancing method with assistance of near-infrared image - Google Patents

Abstract

The invention provides an image visibility enhancing method with assistance of a near-infrared image. The method comprises the following steps of: firstly, acquiring a pair of visible image and near-infrared image of a scene; converting the visible image into a luminance-chrominance space, and performing multiscale edge-preserving decomposition on luminance information of the visible image and the near-infrared image; enhancing a large-scale information layer; subsequently, enhancing a contrast ratio of a detailed layer of the visible image according to detailed layer information of the near-infrared image; and finally, re-combining enhanced large-scale information, detailed information and chrominance information to obtain a visibility enhancement result. The image visibility enhancing method utilizes the advantages of the near-infrared image that atmosphere media have little influence on the near-infrared image and thus the near-infrared image can acquire more detailed scene information than the visible image, and combines rich high-frequency detailed information and strong contrast in the near-infrared image to perform quick and effective visibility enhancement on the visible image.

Description

A kind of visibility of image Enhancement Method auxiliary based on near-infrared image

Technical field

The present invention relates to image vision and strengthen technical field, particularly a kind of visibility of image Enhancement Method auxiliary based on near-infrared image.

Background technology

At present, the system of computer vision field and algorithm suppose that the collection environment of input picture is vacuum usually, namely do not consider the impact of air medium, suppose that light intensity that scene gives off under illumination effect is taken system record completely.But in a practical situation, usually and be false, imaging system usually can by the impact of fog, rain, snow etc. to a certain extent for above-mentioned hypothesis.When there is the weather such as fog, rain, snow, active in imaging is complicated hydrone condensate, and medium to the projection decay of imaging fibre and scattering, such that the open air collection image color under above-mentioned severe weather conditions degrades, contrast reduces.

For the problems referred to above, the existing method to the visibility of image enhancing under severe weather conditions mainly contains two kinds: a kind of is the image utilizing several special shootings, it is mainly divided into two classes again, one class obtains the collection image of Same Scene under different weather condition, the method is only applicable to fixed scene, fixed camera, and need longer image acquisition time there is the change of weather, therefore practical application is weak, Equations of The Second Kind is before collecting device, add that polarization optical filter is to obtain the two width images of Same Scene under different polarization condition, the hypotheses that the method realizes is that the part due to medium scatters in image has polarization property, airlight is partial poolarized light and degree of polarization does not change with distance, but this hypothesis is not necessarily set up in actual imaging, thus cause image enhancement effects unsatisfactory.The method that the second visibility of image strengthens carries out observability based on single image recover and strengthen, this method has stronger practicality, but, the quantity of information comprised due to single image is little, therefore need based on stronger hypothesis or priori, such as, suppose that the actual acquisition image under fair weather condition has very high contrast, or suppose that scene has abundant colouring information etc., but, in some practical applications, as airport monitoring etc., gather image only there is less texture information and color and airlight color close, do not meet above-mentioned hypothesis, make the enhancing effect based on single image undesirable.

In addition, a kind of physics imaging model for severe weather conditions is also had:

I=J·t+A·(1-t)，t=e ^-β·z

Wherein, I represent current scene actual acquisition to the image affected by severe weather conditions; J represents the image that current scene is corresponding under fair weather condition, namely not by the imaging results of the medium influences such as mist; Z is the depth information of current scene; β is the sign factor of weather condition, and β is larger, and the mist of corresponding weather condition is denseer; T represents the situation that the radius of scene reality is decayed by medium influence in atmosphere, it is generally acknowledged that it exponentially changes with the degree of depth, i.e. t=e ^{-β z}, 0<t < 1, the degree of depth is less, and t, more close to 1, shows that current acquired image is subject to the impact of medium less, and what the light of the direct radiation of scene surface was more is directly delivered in imaging system; A represents the color of airlight, scattering is there is in the light that scene directly gives off owing to being subject to the effect of air medium, this some light departs from original travel path and disperses to all directions, after the Multiple Scattering of droplet, it can abstractly be finally the effect of airlight in imaging model, point Range Imaging plane in scene is far away, and the effect of this part is stronger.Original picture rich in detail to be recovered according to this physics imaging model, just need estimating depth and airlight.But in the processing procedure of reality, because the height of this problem is less qualitative, make these two parameters be difficult to be estimated accurately.

In recent years, near-infrared image obtains in computer vision and calculating shooting field and payes attention to widely and develop, and achieves certain achievement in research.The wavelength of visible ray is 400nm-700nm, and near infrared wavelength is 700nm-1100nm.Due to the difference of wavelength, the two scattering properties is also obviously different, particularly when light transmission medium be mist and other pollutants time, very little near infrared propagation effect, the near-infrared image obtained is more clear, and can retain more details to object at a distance.Meanwhile, the color of cloud and sky is closely similar in visible images, and the contrast of the two obviously increases in near-infrared image.

In sum, existing visibility of image enhancing result of study is not very good.And under severe weather conditions, near-infrared image by the impact of atmospheric medium, can not collect more scene detailed information than visible images, thus be conducive to auxiliary visible images and carry out observability fast and effectively and strengthen.Therefore, how to utilize near-infrared image to assist carrying out visibility of image enhancing is the technical matters needing solution at present badly.

Summary of the invention

The present invention is intended at least solve the technical matters existed in prior art, especially innovatively proposes a kind of visibility of image Enhancement Method auxiliary based on near-infrared image.

In order to realize above-mentioned purpose of the present invention, the invention provides a kind of visibility of image Enhancement Method auxiliary based on near-infrared image, it comprises the steps:

S1: the visible images and the near-infrared image pair that obtain scene;

S2: visible images is transformed into luminance-chrominance space, carries out multi-scale edge to visible images monochrome information and near-infrared image and keeps decomposing;

S3: the large scale Information Level of visible images monochrome information is strengthened;

S4: according to the contrast of near-infrared image levels of detail information enhancement visible ray levels of detail;

S5: large scale information, detailed information and chrominance information after being strengthened by visible images reconfigure and obtain observability enhancing result.

It is little that the visibility of image Enhancement Method auxiliary based on near-infrared image of the present invention utilizes near-infrared image to affect by atmospheric medium, the advantage of more scene detailed information is collected than visible images, in conjunction with detail of the high frequency abundant in near-infrared image and strong contrast, observability is fast and effectively carried out to visible images and strengthens.No matter whether the method exist the impact of air dielectric in image, when realizing image enhaucament without when personal error, and can not need to estimate scene depth and airlight, thus make the image gathered can serve various application better.

Additional aspect of the present invention and advantage will part provide in the following description, and part will become obvious from the following description, or be recognized by practice of the present invention.

Accompanying drawing explanation

Above-mentioned and/or additional aspect of the present invention and advantage will become obvious and easy understand from accompanying drawing below combining to the description of embodiment, wherein:

Fig. 1 is the process flow diagram of a kind of preferred implementation that the present invention is based on the visibility of image Enhancement Method that near-infrared image is assisted;

Fig. 2 is the Hybrid camera acquisition system adopted in a kind of preferred implementation of the present invention;

Fig. 3 is visible images under greasy weather condition of the scene that obtains in a kind of preferred implementation of the present invention and near-infrared image pair;

Fig. 4 is the result figure after using the method for existing enhancing visibility of image to strengthen the visible images in Fig. 3;

Fig. 5 is the result figure after using visibility of image Enhancement Method shown in Fig. 1 of the present invention to strengthen the visible images in Fig. 3.

Embodiment

Be described below in detail embodiments of the invention, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has element that is identical or similar functions from start to finish.Being exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not limitation of the present invention being interpreted as.

The present invention proposes a kind of visibility of image Enhancement Method auxiliary based on near-infrared image, the method comprises the steps:

S1: the visible images and the near-infrared image pair that obtain scene;

S2: visible images is transformed into luminance-chrominance space, carries out multi-scale edge to visible images monochrome information and near-infrared image and keeps decomposing;

S3: the large scale Information Level of visible images monochrome information is strengthened;

S4: according to the contrast of near-infrared image levels of detail information enhancement visible ray levels of detail;

S5: large scale information, detailed information and chrominance information after being strengthened by visible images reconfigure and obtain observability enhancing result.

Fig. 1 is the process flow diagram of a kind of preferred implementation that the present invention is based on the visibility of image Enhancement Method that near-infrared image is assisted, Fig. 2 is the Hybrid camera acquisition system adopted in a kind of preferred implementation of the present invention, and Hybrid camera acquisition system comprises Visible Light Camera, near infrared camera and spectroscope.Shown in composition graphs 1 and Fig. 2, visibility of image Enhancement Method of the present invention specifically comprises the steps:

The first step: the visible images and the near-infrared image pair that obtain scene.Concrete grammar utilizes spectroscope that the visible ray in light and near infrared light are divided into two-way, obtained respectively by calibrated Visible Light Camera and near infrared camera, obtains scene visible images and the near-infrared image pair of alignment.

Second step: visible images is transformed into luminance-chrominance space, carries out multi-scale edge to visible images monochrome information V and near-infrared image N and keeps decomposing.Specifically comprise the steps:

S21: visible images is transformed into luminance-chrominance space, the monochrome information V process only to image; ;

S22: apply multi-scale edge respectively to visible images monochrome information and near-infrared image and keep decomposing, obtain large scale Information Level and corresponding levels of detail, the large scale Information Level of visible images monochrome information V and near-infrared image N is respectively { V ^b _k} _k=1...n, { N ^b _k} _k=1...n, the levels of detail of visible images monochrome information V and near-infrared image N is respectively { V ^d _k} _k=1...n, { N ^d _k} _k=1...n, computing formula is:

${I^b}_k=\underset{{I^b}_k}{\arg \min }{\mathrm{\&Sigma;}}_i({\left({{I_i^b}_k-I}_i\right)}^2+{\mathrm{\&lambda;}}_{k-1}(a_{x,i}{\left(\frac{{{\&PartialD;I}^b}_k}{\&PartialD;x}\right)}_i^2+a_{y,i}{\left(\frac{{{\&PartialD;I}^b}_k}{\&PartialD;y}\right)}_i^2)),$

${I^d}_k=\frac{{I^b}_{k-1}-{I^b}_k}{{I^b}_k+\mathrm{\&epsiv;}},$

Wherein, at calculating { V ^b _k} _k=1...n, { N ^b _k} _k=1...ntime, I ^b _k{ V respectively ^b _k} _k=1...n, { N ^b _k} _k=1...n.At calculating { V ^d _k} _k=1...n, { N ^d _k} _k=1...ntime, I ^d _kbe respectively { V ^d _k} _k=1...n, { N ^d _k} _k=1...n.

N is progression,

ε be avoid denominator be zero little numerical constant,

λ is coefficient of balance,

A _x,i, a _y,ifor smoothing factor,

$a_{x,i}={({\left|\frac{\&PartialD;\log \left(I\right)}{\&PartialD;x}\right|}_i^{\mathrm{\&alpha;}}+\mathrm{\&epsiv;})}^{-1},$

$a_{y,i}={({\left|\frac{\&PartialD;\log \left(I\right)}{\&PartialD;y}\right|}_i^{\mathrm{\&alpha;}}+\mathrm{\&epsiv;})}^{-1},$

α controls smoothing factor to the parameter of image gradient sensitivity.

In the present embodiment, get progression n=6, ε=0.0001, α controls the sensitivity to image gradient information, and span is 1.2-2.0.For every one-level K, the value of parameter lambda is λ _k=λ ₀× 2 ^k, wherein, λ ₀=0.1.

3rd step: to large scale Information Level V ^b _nstrengthen.In the present embodiment, the gradient constraint of near-infrared image and wavelet coefficient is utilized to retrain large scale Information Level V ^b _nstrengthen, specifically comprise the steps:

S31: to the large scale Information Level V of visible images monochrome information V and near-infrared image N ^b _nand N ^b _ncarry out wavelet decomposition, obtain V ₀, VH, VV, VD tetra-components and correspondence N ₀, NH, NV, ND tetra-components.

S32: the low frequency component V that wavelet decomposition is obtained ₀, N ₀ask for gradient V _g, N _g,

$V_G=\sqrt{V_{\mathrm{Gx}}^2+V_{\mathrm{Gy}}^2}=\sqrt{{\left(\frac{{\&PartialD;V}_0}{\&PartialD;x}\right)}^2+{\left(\frac{{\&PartialD;V}_0}{\&PartialD;y}\right)}^2}$

$N_G=\sqrt{N_{\mathrm{Gx}}^2+N_{\mathrm{Gy}}^2}=\sqrt{{\left(\frac{{\&PartialD;N}_0}{\&PartialD;x}\right)}^2+{\left(\frac{{\&PartialD;N}_0}{\&PartialD;y}\right)}^2}$

According to near-infrared image low frequency component gradient N _gdistribution situation, to the gradient V of visible images brightness _gcarry out Histogram Matching, obtain the gradient information V upgraded _g'.

S33: the gradient information utilizing corrected visible ray brightness,

${V_{\mathrm{Gx}}}^{\&prime;}=\frac{{V_G}^{\&prime;}}{V_G}\&times;V_{\mathrm{Gx}}$

${V_{\mathrm{Gy}}}^{\&prime;}=\frac{{V_G}^{\&prime;}}{V_G}\&times;V_{\mathrm{Gy}}$

Rebuild by Poisson and obtain corresponding image V ₀'.

S34: the high fdrequency component VH obtained wavelet decomposition, VV, VD, is replaced by the wavelet coefficient high fdrequency component NH of near-infrared image, NV, ND.

S35: by heavy for two parts Combination nova { V ₀', NH, NV, ND}, the large scale Information Level V of the visible images monochrome information after being enhanced by wavelet inverse transformation ^b _n'.

4th step: according to the contrast of near-infrared image levels of detail information enhancement visible ray levels of detail.In one embodiment of the invention, concrete grammar is that formula is by the updated value of the higher value of visible ray levels of detail and near infrared levels of detail corresponding point as visible ray levels of detail pixel:

V ^d _k'=max(V ^d _k,N ^d _k)。

Wherein, k=1 ..., n.

5th step: by the large scale information V after enhancing ^b _n', detailed information { V ^d _k' _k=1...nand chrominance information C reconfigure obtain observability strengthen result,

$V^{\&prime;}={{V^b}_n}^{\&prime;}{\mathrm{\&Pi;}}_{k=1}^n({{V^d}_k}^{\&prime;}+1).$

Use the image enchancing method of prior art and image enchancing method of the present invention to process same collection image below respectively, by comparing the image after enhancing, advantage of the present invention will become more obvious.If Fig. 3 is the visible images of scene under greasy weather condition and near-infrared image pair that use Hybrid camera acquisition system to obtain in a kind of preferred implementation of the present invention, wherein, Fig. 3 (a) is visible images, and Fig. 3 (b) is near-infrared image.Fig. 4 is the result figure after using the visible images of method to Fig. 3 (a) of existing enhancing visibility of image to strengthen, and as seen from the figure, cross-color is more serious.Fig. 5 is the result figure after using the visible images of visibility of image Enhancement Method of the present invention to Fig. 3 (a) to strengthen.Contrasted from Fig. 4 and Fig. 5, the enhancing image effect using the method for enhancing visibility of image of the present invention to obtain is better.

It is little that the present invention utilizes near-infrared image to affect by atmospheric medium, collects more scene detailed information than visible images, thus be conducive to auxiliary visible images carry out fast and effectively observability strengthen.And, no matter whether the method exist the impact of air dielectric in image, when realizing image enhaucament without when personal error, and can not need to estimate scene depth and airlight, thus make the image gathered can serve various application better.

In the description of this instructions, specific features, structure, material or feature that the description of reference term " embodiment ", " some embodiments ", " example ", " concrete example " or " some examples " etc. means to describe in conjunction with this embodiment or example are contained at least one embodiment of the present invention or example.In this manual, identical embodiment or example are not necessarily referred to the schematic representation of above-mentioned term.And the specific features of description, structure, material or feature can combine in an appropriate manner in any one or more embodiment or example.

Although illustrate and describe embodiments of the invention, those having ordinary skill in the art will appreciate that: can carry out multiple change, amendment, replacement and modification to these embodiments when not departing from principle of the present invention and aim, scope of the present invention is by claim and equivalents thereof.

Claims (8)

1., based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, comprise the steps:

S1: the visible images and the near-infrared image pair that obtain scene;

S2: visible images is transformed into luminance-chrominance space, apply multi-scale edge respectively to visible images monochrome information and near-infrared image to keep decomposing, obtain large scale Information Level and corresponding levels of detail, wherein, apply multi-scale edge respectively to described visible images monochrome information V and near-infrared image N to keep decomposing, obtain large scale Information Level { V ^b _k} _k=1...n, { N ^b _k} _k=1...nwith the levels of detail { V of correspondence ^d _k} _k=1...n, { N ^d _k} _k=1...nformula be:

${I^b}_k=\underset{{I^b}_k}{\arg \min }{\mathrm{\&Sigma;}}_i({(I_{\mathrm{i\; k}}^b-I_i)}^2+{\mathrm{\&lambda;}}_{k-1}(a_{x,i}{\left(\frac{{{\&PartialD;I}^b}_k}{\&PartialD;x}\right)}_i^2+a_{y,i}{\left(\frac{{{\&PartialD;I}^b}_k}{\&PartialD;y}\right)}_i^2))$

${I^d}_k=\frac{{I^b}_{k-1}-{I^b}_k}{{I^b}_k+\mathrm{\&epsiv;}}$

Wherein, n is progression,

ε be avoid denominator be zero little numerical constant,

λ is coefficient of balance,

A _x,i, a _y,ifor smoothing factor,

$a_{x,i}={({\left|\frac{\&PartialD;\log \left(I\right)}{\&PartialD;x}\right|}_i^{\mathrm{\&alpha;}}+\mathrm{\&epsiv;})}^{-1},$

$a_{y,i}={({\left|\frac{\&PartialD;\log \left(I\right)}{\&PartialD;y}\right|}_i^{\mathrm{\&alpha;}}+\mathrm{\&epsiv;})}^{-1},$

α controls smoothing factor to the parameter of image gradient sensitivity;

S3: the large scale Information Level of visible images monochrome information is strengthened;

S4: according to the contrast of near-infrared image levels of detail information enhancement visible ray levels of detail;

S5: large scale information, detailed information and chrominance information after being strengthened by visible images reconfigure and obtain observability enhancing result.

2. as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, the visible images of scene is obtained and the right method of near-infrared image is: utilize spectroscope that visible ray and near infrared are divided into two-way in described step S1, respectively by calibrated Visible Light Camera and near infrared camera, obtain scene visible images and the near-infrared image pair of alignment.

3., as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, get progression n=6 ε=0.0001.

4., as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, the span of α is 1.2-2.0.

5., as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, the value of parameter lambda is λ _k=λ ₀× 2 ^k, wherein, λ ₀=0.1.

6. as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, in described step S3, utilize the gradient constraint of near-infrared image and wavelet coefficient constraint to strengthen large scale information.

7., as claimed in claim 6 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, comprise the steps:

S31: wavelet decomposition is carried out to the large scale Information Level of visible images monochrome information and near-infrared image;

S32: ask for gradient to the low frequency component that wavelet decomposition obtains, according to the distribution situation of near-infrared image low frequency component gradient, carries out Histogram Matching to the gradient of visible ray brightness;

S33: the gradient information utilizing corrected visible ray brightness, is rebuild by Poisson and obtains corresponding image;

S34: the high fdrequency component obtained wavelet decomposition, is replaced by the wavelet coefficient high fdrequency component of near-infrared image;

S35: by heavy for two parts Combination nova, the large scale Information Level after being enhanced by wavelet inverse transformation.

8. as claimed in claim 1 based on the visibility of image Enhancement Method that near-infrared image is auxiliary, it is characterized in that, in described step S4, increase the method for picture contrast in levels of detail by near-infrared image details is: by the higher value of visible ray levels of detail and near infrared levels of detail corresponding point as the updated value of visible ray levels of detail pixel.