CN103034984B - A kind of image defogging method capable based on the calculus of variations - Google Patents

Abstract

An image defogging method capable based on the calculus of variations, relates to image processing. Provide and ensureing under the prerequisite of subjective vision effect, processing has mist image rapidly, obtains good sharpening result, meets a kind of image defogging method capable based on the calculus of variations of embedded device real time implementation processing demands. 1) under variation framework, construct the object function of large impervious image; 2) according to the object function constructing, utilize the operation of opening in morphology to calculate large impervious image to image; 3) according to atmospheric scattering physical model, and the large impervious image calculating, to there being mist image to carry out mist elimination restoration disposal. Be based upon on the basis of the calculus of variations, computing is simple, and memory consumption is very little, can meet the demand of embedded device real time implementation processing.

Description

A kind of image defogging method capable based on the calculus of variations

Technical field

The present invention relates to image processing, especially relate to a kind of image defogging method capable based on the calculus of variations.

Background technology

Fog is a kind of common weather phenomenon of occurring in nature. In area, He Jin river, coastal waters, be subject to the impact of water evaporation, thisThe generation of common natural weather is just more frequent. Mist can play and well intersperse effect natural landscape, but also exists simultaneouslyDisturbing to a great extent people's normal life, as affecting outdoor monitoring effect, increase the incidence etc. of traffic accident. MistGas is very large on the impact of the intelligent equipment performance based on image and video, and under the weather condition that has fog, video camera obtainsThe scene image of getting is second-rate, is embodied in: contrast is lower, and details is smudgy and color is in various degree inclined to one sideMove.

The mist elimination algorithm research of Misty Image is for the robustness and the reliability tool that promote the intelligent equipment based on image and videoThere are the wide application prospect of great meaning and deep social value. Although current main flow mist elimination algorithm makes in effectPeople is satisfied, but the mist elimination processing time is longer, is difficult to be applied in the middle of reality. Under this background, we have proposed based on changeThe rapid image defogging method capable of point-score, can ensure under the prerequisite of subjective vision effect, processing has mist image rapidly.

Chinese patent 201010139441.1 discloses a kind of automated graphics defogging method capable based on dark primary, and the method utilization is secretly formerLook priori is asked for transmitting image, and multiple dimensioned Retinex asks for luminance component image, and its processing speed is slow, under transfer functionLimit threshold value can not dynamic self-adapting, after treatment day dummy section there is halation.

Summary of the invention

The object of the present invention is to provide and ensureing under the prerequisite of subjective vision effect, processing has mist image rapidly, obtainsGood sharpening result, meets a kind of image defogging method capable based on the calculus of variations of embedded device real time implementation processing demands.

The present invention includes following steps:

1) under variation framework, construct the object function of large impervious image;

2) according to the object function constructing, utilize the operation of opening in morphology to calculate large impervious to imageImage;

3) according to atmospheric scattering physical model, and the large impervious image calculating, to there being mist image to carry out mist elimination recoveryProcess.

In step 1), the concrete steps of described object function of constructing large impervious image under variation framework are as follows:

The first step: the equation to atmospheric scattering physical model is normalized is removed atmosphere illumination under each color spaceValue A, atmosphere illumination value A can think known terms, and its value can directly obtain from input picture, and this normalization process existsR, G, is described as under tri-color spaces of B:

$\frac{I_i(x,y)}{A}=\frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y)),i\∈\{R,G,B\}$

Second step: at above-mentioned R, G, the value of selected pixels value minimum in B passage, and give tacit consent to atmospheric propagation coefficient t (x, y) at threeIn color space, get identical value:

$\min \frac{I_i(x,y)}{A}=\min \frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y)),i\∈\{R,G,B\}$

Making above formula left side Section 1 is the observed image s under smallest passage, and above formula the right Section 1 is the scene under smallest passageImage j, above formula the right Section 2 is large impervious image v, rewriteeing above formula has:

s＝j+v

Target becomes: at known observed image s, in the situation of unknown large impervious image v, obtain scene image j, this is askedTopic is an ill-conditioning problem, therefore needs certain constraint in addition that it can be separated;

The 3rd step: the observed image of smallest passage is made to certain a priori assumption and constraint according to its greasy weather imaging model,The optimization function being constructed as follows:

$\mathrm{Minimize}:F\left[v\right]=\underset{\mathrm{\Ω}}{\∫}{|\▿v|}^2+\mathrm{\α}{(s-v)}^2+\mathrm{\β}{|\▿(s-v)|}^2\mathrm{dxdy}$

Subjectto:s≥v

Wherein, Ω represents the space that observed image s produces;For penalty term, for ensureing the sky of large impervious image vBetween flatness; α (s-v)²For data fidelity item, for the estimation v that ensures large impervious image close to observed image s;For penalty term, for ensureing the spatial smoothness of restored image.

In step 2) in, the object function that described basis constructs, utilizes the operation of opening in morphology to count imageThe concrete steps that calculation obtains large impervious image are as follows:

The first step: the observed image s under smallest passage is carried out to the computing of a gray level image morphological erosion with structural element b,Obtain transfer image acquisition

Wherein, structural element b being chosen to diameter is 11 circular configuration element;

Second step: use structural element b to transfer image acquisitionCarry out the computing of a gray level image morphological dilations, obtain large imperviousImage: $v\≈\tilde{s}\⊕b;$

Above-mentioned two steps can be merged into a step in morphology, and Morphological Grayscale image is opened operation, logical to minimumObserved image s under road opens operation, obtains large impervious image v:v ≈ s ο b.

In step 3), described according to atmospheric scattering physical model, and the large impervious image calculating, to there being mist figureThe concrete grammar that picture carries out mist elimination restoration disposal can be:

According to the large impervious image v calculating and large impervious image expression formula: 1-t (x, y) calculates atmospheric propagationCoefficient t:t (x, y)=1-v (x, y), wherein, (x, y) is the coordinate figure of each pixel in image; Finally restore according to mist eliminationThe expression formula of image:Try to achieve the restored image after mist elimination.

In step 3), described atmospheric scattering physical model is:

I(x,y)＝J(x,y)t(x,y)+A(1-t(x,y))；

Wherein, I (x, y) is observed image, by " decay " image J (x, y) t (x, y) and " large impervious " image A (1-t (x, y))Composition; Wherein, (x, y) is the coordinate figure of each pixel in image, and I is for there being mist image, and t is atmospheric propagation coefficient, AFor atmosphere illumination value, J is the image after restoring;

According to atmospheric scattering physical model, the expression formula of mist elimination restored image is:Simultaneously to the formula right sideThe denominator of limit Section 2 carries out threshold value constraint max (t (x, y), t₀), object is to prevent that t is tending towards 0, thus whole fractional value is tending towards nothingPoor large, cause restored image distortion, t₀The value is here generally 0.1.

The present invention is based upon on the basis of the calculus of variations, and main feature of the present invention is that computing is simple, and memory consumption is very little, energyEnough meet the demand of embedded device real time implementation processing.

Outstanding advantages of the present invention is as follows:

1. the rapid image defogging method capable based on the calculus of variations has been proposed first. Under variation framework, construct large impervious imageObject function.

2. according to the object function of structure, the observed image under smallest passage is opened to operation, obtain large impervious figureThe best approximation of picture, finally obtains the restored image after mist elimination by large impervious image calculation.

3. because the inventive method is simple and quick, the rapid solving method of the object function to calculus of variations structure is many, memory consumptionMeasure very littlely, greasy weather of suitable embedded platform strengthens processes, and application prospect is very extensive.

Brief description of the drawings

Fig. 1 is overall flow figure of the present invention.

Fig. 2 is gray scale corrosion output and the recovery effect figure that has mist image, smallest passage image, smallest passage image. ?In Fig. 2, the gray scale that is followed successively by from left to right mist image, smallest passage image, smallest passage image is corroded output and multipleFormer effect.

Fig. 3 is the gray scale corrosion output that has mist image, smallest passage image, smallest passage image, the ash of smallest passage imageDegree expand output, final recovery effect figure. In Fig. 3, be respectively from left to right have mist image, smallest passage image,The gray scale corrosion output of smallest passage image, the gray scale expansion output of smallest passage image, final recovery effect.

Fig. 4 is for having mist image, the bright method effect of He Kai, Tarel method effect, the inventive method design sketch. In Fig. 4,Mist image, the bright method effect of He Kai, Tarel method effect, the inventive method effect are followed successively by from left to right; A is 600× 400, b is that 600 × 450, c is that 531 × 800, d is 768 × 1024.

Detailed description of the invention

For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing, embodiment of the present invention is doneFurther instruction:

Referring to Fig. 1, the implementation method of mist elimination mainly contains three steps:

Step 1: obtain the observed image under smallest passage, construct the object function of large impervious image under variation framework.

Variational method is by constructing a series of functions that can reflect variable intrinsic property, and with order that these functions were formedThe optimum of scalar functions is output as target, finally obtains a kind of Mathematical method of variable estimated value. For the light of greasy weather imagingLearn model, constructed the object function of large impervious image under variation framework, concrete steps are as follows:

The first step: the equation to atmospheric scattering physical model is normalized is removed atmosphere illumination under each color spaceValue A. Atmosphere illumination value A can think known terms, and its value can directly obtain from input picture, and this normalization process existsR, G, is described as under tri-color spaces of B:

$\frac{I_i(x,y)}{A}=\frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y)),i\∈\{R,G,B\}$

Second step: at above-mentioned R, G, in B passage, respective pixel value is got minimum of a value, and gives tacit consent to atmospheric propagation coefficient t (x, y) three faceIn the colour space, get identical value:

$\min \frac{I_i(x,y)}{A}=\min \frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y)),i\∈\{R,G,B\}$

Making above formula left side Section 1 is the observed image s under smallest passage, and above formula the right Section 1 is the scene graph under smallest passagePicture j, above formula the right Section 2 is large impervious image v, rewriteeing above formula has:

s=j+v

Target becomes: at known observed image s, in the situation of unknown large impervious image v, obtain scene image j, this is askedTopic is an ill-conditioning problem, therefore needs certain constraint in addition that it can be separated.

The 3rd step: the observed image of smallest passage is made to following a priori assumption and constraint according to its greasy weather imaging model:

(1) large impervious image v meeting spatial flatness in entire image;

(2) large impervious image v is less than observed image s, i.e. s >=v in entire image;

(3) observe and obtain by experiment: in greasy weather outdoor scene image, large impervious image v is close to observed image, with twoPerson's numerical value proximity (s-v)²Construct this data fidelity item;

(4) the estimated value j of scene image, with very large probability, close to (s-v), and natural image generally can be with very high generalRate meeting spatial flatness, meeting spatial flatness is answered in the now estimation of its restored image (s-v), can use bound termExpress.

According to above-mentioned a priori assumption and constraint, the optimization function being constructed as follows:

$\mathrm{Minimize}:F\left[v\right]=\underset{\mathrm{\Ω}}{\∫}{|\▿v|}^2+\mathrm{\α}{(s-v)}^2+\mathrm{\β}{|\▿(s-v)|}^2\mathrm{dxdy}$

Subjectto:s≥v

Wherein, Ω represents the space that observed image s produces;For penalty term, for ensureing the sky of large impervious image vBetween flatness; α (s-v)²For data fidelity item, for the estimation v that ensures large impervious image close to observed image s;For penalty term, for ensureing the spatial smoothness of restored image.

By a series of prior-constrained condition, the optical model of greasy weather imaging is done after appropriate processing, under variation frameworkConstruct the object function for large impervious image v, this process meaning is in the present invention effectively morbid state originallySolve problems can solution.

Step 2: according to the object function constructing, observed image is carried out to the operation of morphology opening operation, obtain large impervious figurePicture.

Through qualitative analysis, after balance compromise between data fidelity item and bound term, draw the following conclusions, i.e. large impervious figureEstimation as v should present following character: have the characteristic of Piecewise Smooth, and have good at scene abrupt change of information edgeRetentivity; At burst intra-zone, large impervious image tends to the minimum of a value in this region, i.e. minimum of a value in burst neighborhoodCan be diffused in whole neighborhood. Based on this conclusion, can solve with multiple fast algorithm the majorized function of invention,In the present invention, adopted a kind of large impervious Image estimation method based on morphologic filtering, therefore the present invention is not limited to and makesWith morphologic filtering, as long as finally solving the method for the object function being constructed by the calculus of variations, all at protection domain of the present inventionIn.

Morphology in image processing field is a kind of extraction from image using mathematical morphology as instrument for expressing and describingThe method of the useful picture content of region shape, such as border, skeleton and convex hull etc.

Expansion is a kind of basic operation of mathematical morphological operation, and the morphological dilations process of gray level image is with the convolution mistake of imageJourney is similarly, and difference is to have replaced convolution and computing by maximum operation, has replaced convolution product with add operation.If the value of structure element be on the occasion of, gray scale dilation operation can make output image brighten, simultaneously whether dark details according to littleBe eliminated or weaken in structural element.

With the operation definition that structural element B (x, y) carries out gray scale expansion to input picture A (x, y) be:

$(A\⊕B)(s,t)=\max \{A(s-x,t-y)+B(x,y)\}$

Wherein: { (s-x), (t-y) ∈ D_A;(x,y)∈D_B, D in above formula_AAnd D_BRepresent respectively determining of A (x, y) and B (x, y)Justice territory.

Corrosion is the another kind of basic operation of mathematical morphological operation, similar with expansive working, and the corrosion operation of gray level image is to useMinimum operation has replaced image convolution and computing. If structure element be on the occasion of, gray scale erosion operation can make output imageDarker, meanwhile, whether bright details basis is less than structural element is eliminated or weakens.

With the operation definition that structural element B (x, y) carries out gray scale corrosion to input picture A (x, y) be:

(A!B)(s,t)＝min{A(s+x,t+y)-B(x,y)}

Wherein: { (s+x), (t+y) ∈ D_A;(x,y)∈D_B, D in above formula_AAnd D_BRepresent respectively determining of A (x, y) and B (x, y)Justice territory.

The basic operation of the dilation and corrosion based on above-mentioned gray level image, can obtain a kind of very important gray level image formLearn basic operation: gray level image is opened operation.

It is to use same structural element first to carry out erosion operation to target image that gray level image is opened operation, then carries out dilation operationProcess. Entered to open after operation, image can be removed isolated point, burr and little connected region, removal wisp,The border of level and smooth larger object, large area does not change image area simultaneously. The mathematic(al) representation that gray level image is opened operation is:

Adopt in the present invention a kind of filtering method based on morphology theory to approach the optimum of large impervious Image estimationSeparate, the approximate algorithm computing of this kind based on morphologic filtering method is simple, and memory consumption is very little, can meet embedded establishingThe demand of standby real time implementation processing.

Concrete steps are as follows:

The first step: the observed image s under smallest passage is carried out to gray level image morphological erosion process one time with structural element b,Obtain estimated image

Wherein, in the present invention, structural element b being chosen to diameter is 11 circular configuration element. After processing, obtaining above formulaEstimated imageCan be used as is the estimation of large impervious image v. In recovery result from Fig. 2, can find, on the sceneSudden change place of depth of field degree, such as close shot has fog-zone intersection or scene information level sudden change place at a distance without fog-zone and distant view, occurs" white edge " phenomenon.

Second step: in order to solve above-mentioned white edge phenomenon, reuse morphologic method, optimize it to large impervious image vEstimation. With structural element b to estimated imageCarry out gray level image morphological dilations process one time, obtain large impervious image: $v\≈\tilde{s}\⊕b;$

After expansive working, for the marginal portion of scene information sudden change place, expansion curve of output can be good at matching, fromAnd effectively suppressing the generation of most white edge phenomenons, Output rusults is at this moment just rationally estimating large impervious image vMeter.

Above-mentioned two steps can be merged into a step in morphology, and Morphological Grayscale is opened operation. Under smallest passageObserved image s open operation, obtain large impervious image v:v ≈ s ο b.

Step 3: obtain the restored image after mist elimination according to atmospheric scattering physical model.

According to the large impervious image v calculating and large impervious image expression formula: 1-t (x, y)

Can calculate atmospheric propagation coefficient t:t (x, y)=1-ω × v (x, y), wherein, (x, y) is each pixel in imageCoordinate figure, be multiplied by parameter ω and be in order to ensure that sky part can distortion, increase picture depth information, value of the present invention is095。

Finally according to the expression formula of mist elimination restored image:Can try to achieve the restored image after mist elimination.In recovery result from Fig. 3, can find, the phenomenon of white edge is inhibited, and recovery effect is very good.

Outstanding advantages of the present invention is as follows: proposed first the rapid image defogging method capable based on the calculus of variations 1.. At variation frameworkLower object function of having constructed large impervious image; 2. according to the object function of structure, the observed image under smallest passage is enteredRow is opened operation, obtains the best approximation of large impervious image, finally obtains after mist elimination by large impervious image calculationRestored image. 3. because the inventive method is simple and quick, the rapid solving method of the object function to calculus of variations structure is many, inDeposit consumption very little, the greasy weather of suitable embedded platform strengthens processes, and application prospect is very extensive.

The present invention with at present in the world the mist elimination algorithm of two main flows: He Kaiming at CVPR09 ' and Tarel at ICCV09 'Running time and the recovery effect of institute's put forward the methods compare. The present invention chooses four width mist image, is being configured toPentium (R) Dual-CoreE53002.60GHz internal memory 4GB, tests contrast on the PC of Matlab7.5.0.Mist elimination compares referring to table 1 running time.

The table 1 mist elimination Operational Timelines

Experimental data by table 1 and Fig. 4 can find out, the present invention in mist elimination effect and other two kinds of algorithm effects substantiallyQuite, but will be far below other two kinds of algorithms on running time. If by hardware-accelerated method employing of the present invention, when operationBetween will further reduce, thereby ensureing, under the prerequisite of subjective effect, to realize the real-time of image and video mist elimination.

Claims (2)

1. the image defogging method capable based on the calculus of variations, is characterized in that comprising the following steps:

1) under variation framework, construct the object function of large impervious image; Describedly under variation framework, construct large impervious imageThe concrete steps of object function are as follows:

The first step: the equation to atmospheric scattering physical model is normalized is removed atmosphere illumination under each color spaceValue A, atmosphere illumination value A can think known terms, and its value can directly obtain from observed image, and this normalized process existsR, G, is described as under tri-color spaces of B:

$\frac{I_i(x,y)}{A}=\frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y\left)\right),i\∈\{R,G,B\}$

Second step: at above-mentioned R, G, the value of selected pixels value minimum in the passage of B, and give tacit consent to atmospheric propagation coefficient t (x, y) threeIn individual color space, get identical value:

$min\frac{I_i(x,y)}{A}=min\frac{J_i(x,y)t(x,y)}{A}+(1-t(x,y\left)\right),i\∈\{R,G,B\}$

Making above formula left side Section 1 is the observed image s under smallest passage, and above formula the right Section 1 is the scene image under smallest passageJ, above formula the right Section 2 is the estimation v of large impervious image, rewriteeing above formula has:

s＝j+v

Target becomes: at known observed image s, in the situation of the estimation v of unknown large impervious image, obtain scene image j,Because this is an ill-conditioning problem, therefore need certain constraint in addition that it can be separated;

The 3rd step: the observed image under smallest passage is made to certain a priori assumption and constraint according to its greasy weather imaging model,The object function being constructed as follows:

Minimize：

Subjectto:s≥v

Wherein, Ω represents the space that observed image s produces;For penalty term, for ensureing the estimation v of large impervious imageSpatial smoothness; α (s-v)²For data fidelity item, for the estimation v that ensures large impervious image close to observed image s;For penalty term, for ensureing the spatial smoothness of restored image;

2) according to the object function constructing, utilize the operation of opening in morphology to calculate large impervious figure to imageThe estimation of picture; The object function that described basis constructs, utilizes the operation of opening in morphology to calculate greatly imageThe concrete steps of the estimation of impervious image are as follows:

The first step: the observed image s under smallest passage is carried out to the computing of a gray level image morphological erosion with structural element b,Obtain transfer image acquisition

Wherein, structural element b being chosen to diameter is 11 circular configuration element;

Second step: use structural element b to transfer image acquisitionCarry out the computing of a gray level image morphological dilations, obtain large impervious figureThe estimation of picture:

Above-mentioned two steps can be merged into a step in morphology, and Morphological Grayscale image is opened operation, to smallest passageUnder observed image s open operation, obtain the estimation v:v ≈ s ο b of large impervious image;

3) according to atmospheric scattering physical model, and the estimation of the large impervious image calculating, to there being mist image to carry out mist eliminationRestoration disposal; Described according to atmospheric scattering physical model, and the estimation of the large impervious image calculating, to there being mist imageThe concrete grammar that carries out mist elimination restoration disposal is:

Estimation expression formula according to the estimation v of the large impervious image calculating and large impervious image: 1-t (x, y), calculatesGo out atmospheric propagation coefficient t (x, y): t (x, y)=1-v, wherein, (x, y) is the coordinate figure of each pixel in image; Last basisThe expression formula of the restored image after mist elimination:Try to achieve the restored image after mist elimination, wherein I (x, y) is for there being mist figurePicture.

2. a kind of image defogging method capable based on the calculus of variations as claimed in claim 1, is characterized in that in step 3) in, instituteStating atmospheric scattering physical model is:

I(x,y)＝Jt(x,y)+A(1-t(x,y))；

Wherein, I (x, y) is for there being mist image, by " decay " image Jt (x, y) and " large impervious " image A (1-t (x, y)) groupBecome; Wherein, (x, y) is the coordinate figure of each pixel in image, and t (x, y) is atmospheric propagation coefficient, and A is atmosphere illumination value,J is the restored image after mist elimination;

According to atmospheric scattering physical model, the expression formula of the restored image after mist elimination is:Simultaneously to the formula right sideThe denominator of limit Section 2 carries out threshold value constraint max (t (x, y), t₀), object is to prevent that t (x, y) is tending towards 0, thus whole fractional value is tending towardsInfinity, causes restored image distortion, t₀Value be 0.1.