CN110097522A - A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks - Google Patents
A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks Download PDFInfo
- Publication number
- CN110097522A CN110097522A CN201910397724.7A CN201910397724A CN110097522A CN 110097522 A CN110097522 A CN 110097522A CN 201910397724 A CN201910397724 A CN 201910397724A CN 110097522 A CN110097522 A CN 110097522A
- Authority
- CN
- China
- Prior art keywords
- neural networks
- convolutional neural
- multiple dimensioned
- layer
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 31
- 239000003595 mist Substances 0.000 claims description 21
- 238000001914 filtration Methods 0.000 claims description 16
- 238000013507 mapping Methods 0.000 claims description 15
- 230000002146 bilateral effect Effects 0.000 claims description 14
- 239000011248 coating agent Substances 0.000 claims description 14
- 238000000576 coating method Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 5
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 2
- 238000000889 atomisation Methods 0.000 claims 1
- 239000000155 melt Substances 0.000 claims 1
- 230000006870 function Effects 0.000 abstract description 30
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000010276 construction Methods 0.000 abstract 1
- 230000001537 neural effect Effects 0.000 description 6
- 238000005267 amalgamation Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- -1 steam Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20028—Bilateral filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks, belong to computer vision field.The present invention constructs training sample set the following steps are included: according to atmospherical scattering model；Based on deep learning thought, multiple dimensioned convolutional neural networks are built；According to the multiple dimensioned convolutional neural networks built, objective function is constructed；Objective function based on construction, the multiple dimensioned convolutional neural networks of training.The present invention is not necessarily to obtain the priori knowledge of outdoor image, and can effectively save the information such as edge, texture, color, contrast and the saturation degree of image.
Description
Technical field
The invention belongs to computer vision fields, more particularly, to a kind of single width family based on multiple dimensioned convolutional neural networks
Outer image defogging method.
Background technique
Mist is that a kind of traditional meteor is formed by by particles such as steam, dust, cigarettes.Mist can make locating for vision system
The image of reason is fuzzy, contrast reduces, saturation degree deviation, but will hinder the visual tasks such as classification, identification, detection and tracking into
Row, even results in multi view mission failure.Therefore, how to remove mist from outdoor image becomes computer vision field
Problem, and the extensive concern by scholars.
There are mainly two types of information types handled by existing defogging method: outdoor video and single width outdoor image.Based on family
The defogging method of outer video is relatively fewer, and main cause is that the defogging method based on video is needed first by Video segmentation into several views
Then frequency frame successively carries out defogging to the video frame after segmentation again, handle substantially or to single width outdoor image.Cause
This, existing defogging method is typically based on the realization of single width outdoor image.Meanwhile the existing defogging based on single width outdoor image
There is the problems such as need to obtaining priori knowledge in advance, edge and texture are lost, and color, contrast and saturation degree are distorted in method.
K.M.He and J.Sun is in article " Single image haze removal using dark channel
prior.Proceedings of the IEEE Conference on Computer Vision and Pattern
Based on helping secretly proposed in Recognition Workshops:IEEE Computer Society, 2009:1956-1963 "
The defogging method of road priori is one of mist removal most classic method in field, but the priori that this method need to obtain image in advance is known
Know.Chen Shuzhen and Ren Zhanguang is in article " based on the single image defogging algorithm automation for improving dark channel prior and Steerable filter
Journal .2016,42 (3): based on improved dark channel prior defogging method, there are dark threshold values proposed in 455-465 "
With mixing dark brightness maxima cannot adaptively choose, and after defogging image color distortion the problems such as.C.Z.He
With C.D.Zhang in article " A haze density aware adaptive perceptual single image haze
removal algorithm.Proceedings of the IEEE International Conference on
Mentioned method exists in Information and Automation, IEEE Computer Society, 2016:1933-1938 "
Image border and texture information recovery extent are to be improved after defogging.B Cai and X Xu are in article " DehazeNet:An End-
to-End System for Single Image Haze Removal.IEEE Transactions on Image
Processing, 2016,25 (11): that there are contrasts is low for method proposed in 5187-5198 ", saturation degree is low and edge letter
The problems such as breath distortion.Z.G.Ling and G.F.Fan is in article " Perception oriented transmission
estimation for high quality image dehazing.Neurocomputing,2017,224(2):82-95”
Proposed in method the problems such as being distorted there are contrast and saturation degree.Z.G.Li and H.Jing is in article " Single Image
De-Hazing Using Globally Guided Image Filtering.IEEE Transactions on Image
Processing, 2018,27 (1): outdoor image color distortion and contrast after method defogging proposed in 442-450 "
It is lower.
Summary of the invention
Existing single width Method of defogging image of outdoor scenes there are aiming at the problem that, the purpose of the present invention is to propose to a kind of based on more
The single width Method of defogging image of outdoor scenes of scale convolutional neural networks, by building the multiple dimensioned convolutional neural networks for defogging,
Obtain mapping relations between foggy image and fog free images, and the multiple dimensioned convolutional neural networks that training is built, successive optimization
Parameter in network, to achieve the purpose that any single width outdoor image defogging.
In order to solve the above technical problems, the technical solution adopted by the present invention are as follows: one kind is based on multiple dimensioned convolutional neural networks
Single width Method of defogging image of outdoor scenes, which comprises the steps of:
(1) training sample obtains: obtaining fog free images sample, carries out mist to fog free images sample using atmospherical scattering model
Change processing and obtain foggy image sample, using fog free images sample and corresponding foggy image sample as training sample；
(2) multiple dimensioned convolutional neural networks model: the convolutional layer that building is no less than three；The output end of each convolutional layer connects
The pond a Max Pooling layer is connect, the output end of the pond each Max Pooling layer connects one based on amendment linear unit
The Nonlinear Mapping layer of ReLu；The output end of all Nonlinear Mapping layers is connect with Fusion Features layer；The Fusion Features layer
Output end connect a bilateral filtering layer that transmissivity is handled, using bilateral filtering layer output transmissivity to convolution
The foggy image sample of layer input carries out defogging processing；
(3) multiple dimensioned convolutional neural networks model training: using the foggy image sample in step (1) as multiple dimensioned volume
The input of product neural network model, the differentiation exported using the fog free images sample in step (1) as multiple dimensioned neural network
Standard is trained multiple dimensioned convolutional neural networks model for the purpose of the minimization of object function, carries out parametric solution；Its
Middle objective function are as follows:
Wherein, ci、wiAnd hiIt is the corresponding average RGB numerical value of i-th of sample, average contrast and average staturation respectively；
The parameter of multiple dimensioned convolutional neural networks is Φ, and i-th of foggy image sample is Ii, the corresponding nothing of i-th of foggy image sample
Mist image pattern is Ji, the number of training sample is N；
(4) the multiple dimensioned convolutional neural networks model after being solved using step (3) goes foggy image to be processed
Mist processing.
Further technical solution is that there are three the convolutional layers, and convolution kernel is 7 × 7,5 × 5 and 3 × 3 respectively；Or institute
It states there are four convolutional layers, convolution kernel is 11 × 11,7 × 7,5 × 5 and 3 × 3 respectively.
Further technical solution is that the objective function is constructed according to mean square error MSE and L-2 norm.
Further technical solution is that the objective function is minimized according to stochastic gradient descent method.
Further technical solution is that in step (1) sample acquisition, fog free images sample architecture is corresponding with mist image pattern
Calculation method it is as follows:
IT(x)=JT(x)tT(x)+αT(1-tT(x))
Wherein, JTIt (x) is the i.e. collected image of fog free images sample, tTIt (x) is transmissivity, αTFor global air light value,
ITIt (x) is foggy image sample.
Further technical solution is that the convolutional layer form is as follows:
Wherein, I (x) is the foggy image sample to defogging, and q is convolution kernel size,It is convolutional layer filter,
It is convolutional layer biasing, * is convolution algorithm,It is the output of multiple dimensioned convolutional neural networks convolutional layer.
Further technical solution is that the pond Max Pooling layer form is as follows:
Wherein,It is the output of multiple dimensioned convolutional neural networks pond layer,It is multiple dimensioned convolutional Neural
The output of network convolutional layer.
Further technical solution is that it is more that the Nonlinear Mapping layer carries out Nonlinear Mapping acquisition to the feature after dimensionality reduction
Scale feature figure, constructed active coating form are as follows:
Wherein,It is active coating filter,It is active coating biasing,It is multiple dimensioned convolutional neural networks
The output of active coating,It is the output of multiple dimensioned convolutional neural networks pond layer.
Further technical solution is that the Fusion Features layer merges Analysis On Multi-scale Features figure, to get transmissivityAnalysis On Multi-scale Features figure amalgamation mode is as follows:
Wherein, λ1,λ2,…,λnIt is characteristic pattern weight coefficient, h respectively(q)、c(q)And s(q)It is each scale feature figure respectively
Average RGB numerical value, mean contrast value and average intensity value；It is the defeated of n scale convolutional neural networks pond layer
Out.
Further technical solution is that the bilateral filtering layer is using bilateral filtering to transmissivityIt is handled, from
And fining transmissivity t (x) is obtained, calculation method is as follows:
D (ξ, y)=| | ξ-y | |2
Wherein, y is transmissivityIn pixel, ξ is the pixel adjacent with y, and c (ξ, y) is space weighting function,
σcIt is the variance between two pixels, d (ξ, y) is between two pixels apart from measure function,WithIt is with pixel respectively
The transmissivity that 8 neighborhood territory pixel blocks centered on point ξ and y are constituted,It is similar weight calculation function, σsIt is two
Variance between transmissivity,It is two transmissivities apart from measure function.
The present invention is compared with the advantage of the prior art:
(1) it is based on deep learning thought, builds multiple dimensioned convolutional neural networks, and sufficiently excavate the spy of each scale feature figure
Point so that the information such as outdoor image color, contrast, saturation degree after defogging with initially have mist outdoor image close.
(2) transmissivity merged by characteristic pattern is handled using bilateral filtering, so that the open air after defogging
Image border and texture information save complete.
(3) it is based on MSE and L-2 norm, constructs objective function, to realize to the effective quasi- of multiple dimensioned convolutional neural networks
It closes, realizes the more effective removal to mist in outdoor image.
Detailed description of the invention
Fig. 1 is the flow chart of defogging method proposed by the invention；
Fig. 2 is part training sample schematic diagram；
Fig. 3 is 3 scale convolutional neural networks structural schematic diagrams；
Fig. 4 is 4 scale convolutional neural networks structural schematic diagrams.
Specific embodiment
Clear to be more clear technical solution of the present invention, the present invention will be further described below with reference to the accompanying drawings.
A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks is elaborated in the embodiment of the present invention,
It is characterized by comprising the following steps:
(1) training sample obtains: obtaining fog free images sample, carries out mist to fog free images sample using atmospherical scattering model
Change processing and obtain foggy image sample, using fog free images sample and corresponding foggy image sample as training sample；
(2) multiple dimensioned convolutional neural networks model: the convolutional layer that building is no less than three；The output end of each convolutional layer connects
The pond a Max Pooling layer is connect, the output end of the pond each Max Pooling layer connects one based on amendment linear unit
The Nonlinear Mapping layer of ReLu；The output end of all Nonlinear Mapping layers is connect with Fusion Features layer；The Fusion Features layer
Output end connect a bilateral filtering layer that transmissivity is handled, using bilateral filtering layer output transmissivity to convolution
The foggy image sample of layer input carries out defogging processing；
(3) multiple dimensioned convolutional neural networks model training: using the foggy image sample in step (1) as multiple dimensioned volume
The input of product neural network model, the differentiation exported using the fog free images sample in step (1) as multiple dimensioned neural network
Standard is trained multiple dimensioned convolutional neural networks model for the purpose of the minimization of object function, carries out parametric solution；Its
Middle objective function are as follows:
Wherein, ci、wiAnd hiIt is the corresponding average RGB numerical value of i-th of sample, average contrast and average staturation respectively；
The parameter of multiple dimensioned convolutional neural networks is Φ, and i-th of foggy image sample is Ii, the corresponding nothing of i-th of foggy image sample
Mist image pattern is Ji, the number of training sample is N；
(4) the multiple dimensioned convolutional neural networks model after being solved using step (3) goes foggy image to be processed
Mist processing.
There are three convolutional layers described in the embodiment of the present invention, and convolution kernel is 7 × 7,5 × 5 and 3 × 3 respectively；Or the convolution
There are four layers, and convolution kernel is 11 × 11,7 × 7,5 × 5 and 3 × 3 respectively.
Objective function described in the embodiment of the present invention is constructed according to mean square error MSE and L-2 norm.
Objective function described in the embodiment of the present invention is minimized according to stochastic gradient descent method.
In the embodiment of the present invention in step (1) sample acquisition, fog free images sample architecture is corresponding with the meter of mist image pattern
Calculation method is as follows:
IT(x)=JT(x)tT(x)+αT(1-tT(x))
Wherein, JTIt (x) is the i.e. collected image of fog free images sample, tTIt (x) is transmissivity, αTFor global air light value,
ITIt (x) is foggy image sample.
Convolutional layer form described in the embodiment of the present invention is as follows:
Wherein, I (x) is the foggy image sample to defogging, and q is convolution kernel size,It is convolutional layer filter,
It is convolutional layer biasing, * is convolution algorithm,It is the output of multiple dimensioned convolutional neural networks convolutional layer.
The pond Max Pooling described in embodiment of the present invention layer form is as follows:
Wherein,It is the output of multiple dimensioned convolutional neural networks pond layer,It is multiple dimensioned convolutional Neural
The output of network convolutional layer.
It is multiple dimensioned that Nonlinear Mapping layer described in the embodiment of the present invention carries out Nonlinear Mapping acquisition to the feature after dimensionality reduction
Characteristic pattern, constructed active coating form are as follows:
Wherein,It is active coating filter,It is active coating biasing,It is multiple dimensioned convolutional neural networks
The output of active coating,It is the output of multiple dimensioned convolutional neural networks pond layer.
Fusion Features layer described in the embodiment of the present invention merge Analysis On Multi-scale Features figure, to get transmissivity
Analysis On Multi-scale Features figure amalgamation mode is as follows:
Wherein, λ1,λ2,…,λnIt is characteristic pattern weight coefficient, h respectively(q)、c(q)And s(q)It is each scale feature figure respectively
Average RGB numerical value, mean contrast value and average intensity value；It is the output of n scale convolutional neural networks pond layer.
Bilateral filtering layer described in the embodiment of the present invention is using bilateral filtering to transmissivityIt is handled, to obtain
It refines transmissivity t (x), calculation method is as follows:
D (ξ, y)=| | ξ-y | |2
Wherein, y is transmissivityIn pixel, ξ is the pixel adjacent with y, and c (ξ, y) is space weighting function,
σcIt is the variance between two pixels, d (ξ, y) is between two pixels apart from measure function,WithIt is with pixel respectively
The transmissivity that 8 neighborhood territory pixel blocks centered on point ξ and y are constituted,It is similar weight calculation function, σsIt is two
Variance between transmissivity,It is two transmissivities apart from measure function.
As shown in Figure 1, a kind of single width outdoor image based on multiple dimensioned convolutional neural networks in the embodiment of the present invention
Defogging method the following steps are included:
Step 1: training sample being obtained according to atmospherical scattering model, constructs training sample data collection.
1.1) the fogless outdoor image under 3000 width different scenes is collected from internet.
1.2) 3000 collected width outdoor images are directed to, and set JTIt (x) is the i.e. collected figure of fogless outdoor image
Picture, tTIt (x) is transmissivity, αTFor global air light value, ITIt (x) is to have mist outdoor image.Since training sample to be guaranteed wraps as far as possible
Containing a variety of situations, therefore choose different tT(x), global air light value αTFixed value is chosen, definition has mist outdoor image IT
(x) it is
IT(x)=JT(x)tT(x)+αT(1-tT(x)) (1)
1.3 3000 width outdoor images of traversal, and obtain 3000 width according to above formula and have mist outdoor image as training sample, from
And construct training sample data collection.
Training sample data concentrated part training sample and its Ground Truth are as shown in Figure 2.Wherein, first be classified as from
The fogless outdoor image that internet is got, second is classified as and has mist outdoor image through what formula (1) calculated.
Step 2: building multiple dimensioned convolutional neural networks.
2.1) convolutional layer for the multiple dimensioned convolutional neural networks built is rolled up by 7 × 7,5 × 5,3 × 3 three kinds of different scales
Product core is constituted, if the training sample to defogging is I (x), q represents convolution kernel size and q ∈ { 7,5,3 },Represent convolutional layer
Filter,Convolutional layer biasing is represented, * represents convolution algorithm, then the output of multiple dimensioned convolutional neural networks convolutional layerIt is represented by
2.2) the pond layer for the multiple dimensioned neural network built is constructed by Max Pooling, and purport is to by convolution
Layer and calculated feature carries out dimensionality reduction, to get the feature with translation invariance, rotational invariance, then multiple dimensioned volume
The output of product neural network pond layerIt is represented by
2.3) the multiple dimensioned convolutional neural networks built construct active coating according to linear unit R eLu is corrected, after dimensionality reduction
Feature carry out Nonlinear Mapping, to obtain Analysis On Multi-scale Features figure,Active coating filter is represented,Represent activation
Layer biasing, the then output of multiple dimensioned convolutional neural networks active coatingIt can be expressed as
2.4) it after obtaining Analysis On Multi-scale Features figure, sufficiently excavates in color each scale feature graph coloring, saturation degree and contrast
Feature merges each scale feature figure, so that the corresponding transmissivity of input picture I (x) be calculatedFusion function
It is defined as
Wherein, λ, μ and γ are characteristic pattern weight coefficient, h respectively(q)、c(q)And s(q)It is being averaged for each scale feature figure respectively
RGB numerical value, mean contrast value and average intensity value.
2.5) since the calculated transmissivity of existing defogging method institute is more coarse, so that the outdoor image after defogging is general
Store-through saves the problems such as imperfect on boundary and texture.Therefore, transmission of the bilateral filtering to merging by characteristic pattern is selected
Rate is handled, in the hope of obtaining fining transmissivity t (x), to transmissivityBilateral filtering process is carried out to be represented by
Wherein, y is transmissivityIn pixel, ξ is the pixel adjacent with y, and c (ξ, y) is space weighting function,
σcIt is the variance between two pixels, d (ξ, y) is between two pixels apart from measure function,WithIt is with pixel respectively
The transmissivity that 8 neighborhood territory pixel blocks centered on point ξ and y are constituted,It is similar weight calculation function, σsIt is two
Variance between transmissivity,It is two transmissivities apart from measure function.
2.6) after obtaining fining transmissivity t (x), selecting overall situation air light value α is each pixel in input picture I (x)
The corresponding maximum brightness value of point.At this point, atmospherical scattering model is deformed, so that the outdoor image J (x) after defogging is calculated,
Calculation method is
Multiple dimensioned convolutional neural networks structure is as shown in Figure 3.
Step 3: according to mean square error MSE and L-2 norm, constructing objective function.
Single width outdoor image defogging problem is typical supervised learning problem, and supervised learning needs to establish convolutional Neural
Mapping relations G between network inputs (having mist outdoor image) and output (fogless outdoor image).If multiple dimensioned convolutional Neural net
The parameter of network is
I-th of training sample is Ii, the corresponding Ground Truth of i-th of training sample is Ji, the number of training sample is N, multiple dimensioned
The parameter Φ of convolutional neural networks can be obtained by minimizing objective function, be constructed by mean square error MSE and L-2 norm
Objective function, form are
Wherein, ci、wiAnd hiIt is the corresponding average RGB numerical value of i-th of sample, average contrast and average staturation respectively.
Step 4: the multiple dimensioned convolutional neural networks of training.
Firstly, randomly selecting 20000 64 × 64 foggy image blocks from the training sample concentration constructed, respectively there is mist figure
As block possesses corresponding Ground Truth；Then, constructed objective function is minimized using stochastic gradient descent method；Most
It afterwards, is objective function given threshold represents multiple dimensioned convolutional Neural when minimizing objective function result less than given threshold
The parameter Φ of network it has been determined that complete the training to convolutional neural networks, and then can realize and carry out to any outdoor image at this time
Defogging processing；As shown in figure 3, the output of its multiple dimensioned convolutional neural networks is treated picture.
The improvement that the embodiment of the present invention does upper one embodiment in improvement, builds multiple dimensioned convolutional Neural net in step 2
The convolutional layer of network is made of 11 × 11,7 × 7,5 × 5,3 × 3 four kinds of different scale convolution kernels, if the training sample to defogging is I
(x), q represents convolution kernel size and q ∈ { 11,7,5,3 }, and its Analysis On Multi-scale Features figure amalgamation mode is as follows:
Wherein, λ, μ, γ and β are characteristic pattern weight coefficient, h respectively(q)、c(q)And s(q)It is the flat of each scale feature figure respectively
Equal RGB numerical value, mean contrast value and average intensity value.As shown in figure 4, the output of its four scale convolutional neural networks is
Treated picture.
Embodiment described above is only that preferred embodiments of the present invention will be described, not to the scope of the present invention
It is defined, without departing from the spirit of the design of the present invention, those of ordinary skill in the art are to technical solution of the present invention
The various changes and improvements made should all be fallen into the protection scope that claims of the present invention determines.
Claims (10)
1. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks, which is characterized in that including walking as follows
It is rapid:
(1) training sample obtains: obtaining fog free images sample, is carried out at atomization using atmospherical scattering model to fog free images sample
Reason obtains foggy image sample, using fog free images sample and corresponding foggy image sample as training sample；
(2) multiple dimensioned convolutional neural networks model: the convolutional layer that building is no less than three；The output end connection one of each convolutional layer
The pond a Max Pooling layer, the output end of the pond each Max Pooling layer connect one based on the linear unit R eLu of amendment
Nonlinear Mapping layer；The output end of all Nonlinear Mapping layers is connect with Fusion Features layer；The Fusion Features layer it is defeated
Outlet connects a bilateral filtering layer handled transmissivity, and the transmissivity exported using bilateral filtering layer is defeated to convolutional layer
The foggy image sample entered carries out defogging processing；
(3) multiple dimensioned convolutional neural networks model training: using the foggy image sample in step (1) as multiple dimensioned convolution mind
Input through network model, the discrimination standard exported using the fog free images sample in step (1) as multiple dimensioned neural network,
For the purpose of the minimization of object function, multiple dimensioned convolutional neural networks model is trained, carries out parametric solution；Wherein target
Function are as follows:
Wherein, ci、wiAnd hiIt is the corresponding average RGB numerical value of i-th of sample, average contrast and average staturation respectively；More rulers
The parameter for spending convolutional neural networks is Φ, and i-th of foggy image sample is Ii, the corresponding fogless figure of i-th of foggy image sample
Decent is Ji, the number of training sample is N；
(4) the multiple dimensioned convolutional neural networks model after being solved using step (3) carries out at defogging foggy image to be processed
Reason.
2. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, there are three the convolutional layers, and convolution kernel is 7 × 7,5 × 5 and 3 × 3 respectively；Or there are four the convolutional layers, convolution
Core is 11 × 11,7 × 7,5 × 5 and 3 × 3 respectively.
3. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the objective function is constructed according to mean square error MSE and L-2 norm.
4. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the objective function is minimized according to stochastic gradient descent method.
5. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, in step (1) sample acquisition, the calculation method that fog free images sample architecture is corresponding with mist image pattern is as follows:
IT(x)=JT(x)tT(x)+αT(1-tT(x))
Wherein, JTIt (x) is the i.e. collected image of fog free images sample, tTIt (x) is transmissivity, αTFor global air light value, IT(x)
For foggy image sample.
6. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1 or 2,
It is characterized in that, the convolutional layer form is as follows:
Wherein, I (x) is the foggy image sample to defogging, and q is convolution kernel size,It is convolutional layer filter,It is
Convolutional layer biasing, * is convolution algorithm,It is the output of multiple dimensioned convolutional neural networks convolutional layer.
7. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the pond Max Pooling layer form is as follows:
Wherein,It is the output of multiple dimensioned convolutional neural networks pond layer,It is multiple dimensioned convolutional neural networks
The output of convolutional layer.
8. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the Nonlinear Mapping layer carries out Nonlinear Mapping to the feature after dimensionality reduction and obtains Analysis On Multi-scale Features figure, constructed
Active coating form it is as follows:
Wherein,It is active coating filter,It is active coating biasing,It is multiple dimensioned convolutional neural networks activation
The output of layer,It is the output of multiple dimensioned convolutional neural networks pond layer.
9. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the Fusion Features layer merges Analysis On Multi-scale Features figure, to get transmissivityAnalysis On Multi-scale Features figure melts
Conjunction mode is as follows:
Wherein, λ1,λ2,…,λnIt is characteristic pattern weight coefficient, h respectively(q)、c(q)And s(q)It is being averaged for each scale feature figure respectively
RGB numerical value, mean contrast value and average intensity value；It is the output of n scale convolutional neural networks pond layer.
10. a kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks according to claim 1,
It is characterized in that, the bilateral filtering layer is using bilateral filtering to transmissivityIt is handled, to obtain fining transmissivity t
(x), calculation method is as follows:
D (ξ, y)=| | ξ-y | |2
Wherein, y is transmissivityIn pixel, ξ is the pixel adjacent with y, and c (ξ, y) is space weighting function, σcIt is
Variance between two pixels, d (ξ, y) are between two pixels apart from measure function,WithBe respectively with pixel ξ with
The transmissivity that 8 neighborhood territory pixel blocks centered on y are constituted,It is similar weight calculation function, σsIt is two transmissivities
Between variance,It is two transmissivities apart from measure function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910397724.7A CN110097522B (en) | 2019-05-14 | 2019-05-14 | Single outdoor image defogging method based on multi-scale convolution neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910397724.7A CN110097522B (en) | 2019-05-14 | 2019-05-14 | Single outdoor image defogging method based on multi-scale convolution neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110097522A true CN110097522A (en) | 2019-08-06 |
| CN110097522B CN110097522B (en) | 2021-03-19 |
Family
ID=67447928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910397724.7A Active CN110097522B (en) | 2019-05-14 | 2019-05-14 | Single outdoor image defogging method based on multi-scale convolution neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110097522B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570371A (en) * | 2019-08-28 | 2019-12-13 | 天津大学 | image defogging method based on multi-scale residual error learning |
| CN110738622A (en) * | 2019-10-17 | 2020-01-31 | 温州大学 | Lightweight neural network single image defogging method based on multi-scale convolution |
| CN111369472A (en) * | 2020-03-12 | 2020-07-03 | 北京字节跳动网络技术有限公司 | Image defogging method and device, electronic equipment and medium |
| CN112164010A (en) * | 2020-09-30 | 2021-01-01 | 南京信息工程大学 | Multi-scale fusion convolution neural network image defogging method |
| CN114049274A (en) * | 2021-11-13 | 2022-02-15 | 哈尔滨理工大学 | Defogging method for single image |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
| CN107749052A (en) * | 2017-10-24 | 2018-03-02 | 中国科学院长春光学精密机械与物理研究所 | Image defogging method and system based on deep learning neutral net |
| CN108269244A (en) * | 2018-01-24 | 2018-07-10 | 东北大学 | It is a kind of based on deep learning and prior-constrained image defogging system |
| CN108564549A (en) * | 2018-04-20 | 2018-09-21 | 福建帝视信息科技有限公司 | A kind of image defogging method based on multiple dimensioned dense connection network |
| CN109087254A (en) * | 2018-04-26 | 2018-12-25 | 长安大学 | Unmanned plane image haze sky and white area adaptive processing method |
| WO2018235746A1 (en) * | 2017-06-21 | 2018-12-27 | キヤノン株式会社 | Image processing device, imaging device, image processing method, program, and storage medium |
| CN109360156A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single image rain removing method based on the image block for generating confrontation network |
| CN109360155A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single-frame images rain removing method based on multi-scale feature fusion |
| CN109410144A (en) * | 2018-10-31 | 2019-03-01 | 聚时科技(上海)有限公司 | A kind of end-to-end image defogging processing method based on deep learning |
| CN109584188A (en) * | 2019-01-15 | 2019-04-05 | 东北大学 | A kind of image defogging method based on convolutional neural networks |
| CN109712083A (en) * | 2018-12-06 | 2019-05-03 | 南京邮电大学 | A kind of single image to the fog method based on convolutional neural networks |
-
2019
- 2019-05-14 CN CN201910397724.7A patent/CN110097522B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106960425A (en) * | 2017-04-05 | 2017-07-18 | 上海矽奥微电子有限公司 | Single frames defogging method based on multiple dimensioned filtering of deconvoluting |
| WO2018235746A1 (en) * | 2017-06-21 | 2018-12-27 | キヤノン株式会社 | Image processing device, imaging device, image processing method, program, and storage medium |
| CN107749052A (en) * | 2017-10-24 | 2018-03-02 | 中国科学院长春光学精密机械与物理研究所 | Image defogging method and system based on deep learning neutral net |
| CN108269244A (en) * | 2018-01-24 | 2018-07-10 | 东北大学 | It is a kind of based on deep learning and prior-constrained image defogging system |
| CN108564549A (en) * | 2018-04-20 | 2018-09-21 | 福建帝视信息科技有限公司 | A kind of image defogging method based on multiple dimensioned dense connection network |
| CN109087254A (en) * | 2018-04-26 | 2018-12-25 | 长安大学 | Unmanned plane image haze sky and white area adaptive processing method |
| CN109360156A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single image rain removing method based on the image block for generating confrontation network |
| CN109360155A (en) * | 2018-08-17 | 2019-02-19 | 上海交通大学 | Single-frame images rain removing method based on multi-scale feature fusion |
| CN109410144A (en) * | 2018-10-31 | 2019-03-01 | 聚时科技(上海)有限公司 | A kind of end-to-end image defogging processing method based on deep learning |
| CN109712083A (en) * | 2018-12-06 | 2019-05-03 | 南京邮电大学 | A kind of single image to the fog method based on convolutional neural networks |
| CN109584188A (en) * | 2019-01-15 | 2019-04-05 | 东北大学 | A kind of image defogging method based on convolutional neural networks |
Non-Patent Citations (4)
| Title |
|---|
| WENQI REN等: "Single Image Dehazing via Multi-scale Convolutional Neural Networks", 《ECCV2016》 * |
| YUAN-KAI WANG等: "Single Image Defogging by Multiscale Depth Fusion", 《IEEE TRANSACTIONS ON IMAGE PROCESSING 》 * |
| 郭继昌等: "多尺度卷积神经网络的单幅图像去雨方法", 《哈尔滨工业大学学报》 * |
| 陈清江等: "基于卷积神经网络的图像去雾算法", 《液晶与显示》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570371A (en) * | 2019-08-28 | 2019-12-13 | 天津大学 | image defogging method based on multi-scale residual error learning |
| CN110570371B (en) * | 2019-08-28 | 2023-08-29 | 天津大学 | Image defogging method based on multi-scale residual error learning |
| CN110738622A (en) * | 2019-10-17 | 2020-01-31 | 温州大学 | Lightweight neural network single image defogging method based on multi-scale convolution |
| CN111369472A (en) * | 2020-03-12 | 2020-07-03 | 北京字节跳动网络技术有限公司 | Image defogging method and device, electronic equipment and medium |
| CN111369472B (en) * | 2020-03-12 | 2021-04-23 | 北京字节跳动网络技术有限公司 | Image defogging method and device, electronic equipment and medium |
| CN112164010A (en) * | 2020-09-30 | 2021-01-01 | 南京信息工程大学 | Multi-scale fusion convolution neural network image defogging method |
| CN114049274A (en) * | 2021-11-13 | 2022-02-15 | 哈尔滨理工大学 | Defogging method for single image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110097522B (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110097522A (en) | A kind of single width Method of defogging image of outdoor scenes based on multiple dimensioned convolutional neural networks | |
| CN107680054B (en) | Multi-source image fusion method in haze environment | |
| CN108615226B (en) | Image defogging method based on generation type countermeasure network | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| Kuanar et al. | Night time haze and glow removal using deep dilated convolutional network | |
| CN111079556A (en) | Multi-temporal unmanned aerial vehicle video image change area detection and classification method | |
| CN107301624B (en) | Convolutional neural network defogging method based on region division and dense fog pretreatment | |
| Li et al. | Deep dehazing network with latent ensembling architecture and adversarial learning | |
| Ju et al. | BDPK: Bayesian dehazing using prior knowledge | |
| CN107590427B (en) | Method for detecting abnormal events of surveillance video based on space-time interest point noise reduction | |
| CN110111346B (en) | Remote sensing image semantic segmentation method based on parallax information | |
| Guo et al. | Haze and thin cloud removal using elliptical boundary prior for remote sensing image | |
| CN111582074A (en) | Monitoring video leaf occlusion detection method based on scene depth information perception | |
| CN114241372A (en) | Target identification method applied to sector-scan splicing | |
| Riaz et al. | Multiscale image dehazing and restoration: An application for visual surveillance | |
| Culibrk | Neural network approach to Bayesian background modeling for video object segmentation | |
| Soumya et al. | Self-organized night video enhancement for surveillance systems | |
| CN115937019A (en) | Non-uniform defogging method combining LSD (local Scale decomposition) quadratic segmentation and deep learning | |
| Koley et al. | Single image visibility restoration using dark channel prior and fuzzy logic | |
| Fan et al. | Image defogging approach based on incident light frequency | |
| Sudhakara et al. | Multi-scale fusion for underwater image enhancement using multi-layer perceptron | |
| Dai et al. | Detecting moving object from dynamic background video sequences via simulating heat conduction | |
| Zhou et al. | Low‐light image enhancement for infrared and visible image fusion | |
| Várkonyi-Kóczy | New advances in digital image processing | |
| Zhang et al. | Single image haze removal for aqueous vapour regions based on optimal correction of dark channel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |