CN110288550A - The single image defogging method of confrontation network is generated based on priori knowledge guiding conditions - Google Patents
The single image defogging method of confrontation network is generated based on priori knowledge guiding conditions Download PDFInfo
- Publication number
- CN110288550A CN110288550A CN201910580136.7A CN201910580136A CN110288550A CN 110288550 A CN110288550 A CN 110288550A CN 201910580136 A CN201910580136 A CN 201910580136A CN 110288550 A CN110288550 A CN 110288550A
- Authority
- CN
- China
- Prior art keywords
- image
- defogging
- training
- network
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 239000003595 mist Substances 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 52
- 230000008030 elimination Effects 0.000 claims abstract description 47
- 238000003379 elimination reaction Methods 0.000 claims abstract description 47
- 230000006735 deficit Effects 0.000 claims abstract description 8
- 238000000889 atomisation Methods 0.000 claims abstract description 7
- 230000003044 adaptive effect Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 5
- 230000004913 activation Effects 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 abstract description 11
- 230000000694 effects Effects 0.000 abstract description 10
- 230000007812 deficiency Effects 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 14
- 230000009466 transformation Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 230000001737 promoting effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 208000001491 myopia Diseases 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 238000009738 saturating Methods 0.000 description 1
- 238000010189 synthetic method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of single image defogging methods that confrontation network is generated based on priori knowledge guiding conditions, comprising steps of one, foundation atomization training set of images；Two, the preliminary defogging of individual random foggy image；Three, the defogging training of preliminary mist elimination image；Four, the truth value of the defogging training image with reference to true value image and preliminary mist elimination image is calculated；Five, image impairment objective function is calculated；Six, weight parameter set is updated；Seven, individual new random foggy image, circulation step two to step 6 are transferred, until truth value reaches setting value；Eight, individual practical foggy image defogging.Present invention utilizes priori knowledges to instruct coding network to carry out fogless result generation, the part useful information that priori knowledge obtains is utilized, the feature modeling ability that deep neural network is utilized again simultaneously compensates for the deficiency of priori knowledge, it does not need the display in deep neural network and establishes atmospherical scattering model, but the condition for being regarded as image generates, defog effect is good.
Description
Technical field
The invention belongs to technical field of image processing, and in particular to one kind generates confrontation net based on priori knowledge guiding conditions
The single image defogging method of network.
Background technique
It is existing that quality degradation can occur due to the effect of atmospheric scattering in the image acquired under the bad weather of mist, haze etc
As making the inclined canescence of color of image, contrast reduces, and object features are difficult to recognize, not only visual effect is made to be deteriorated, and image is ornamental
Property reduce, there is deviation in the understanding for also resulting in picture material.Image defogging just refers to specific ways and means, makes air
Middle suspended particulates even are eliminated the adverse effect reduction of image.Single image defogging refers to the item in only one foggy image
Under part, defogging is carried out to it and handles to obtain clearly image.
Single image defogging method is broadly divided into three categories at present: the first kind is the method based on image enhancement, the second class
It is the method based on physical model, third class is the method based on deep learning.
The essence of method based on image enhancement is enhanced the image being degraded, and the quality of image is improved.Such as
Common histogram equilibrium, logarithmic transformation, power law transformation, sharpening, wavelet transformation etc..Enhance the contrast of image by these methods
Or the feature of prominent image.Different from common contrast enhancement process, the method for another common image enhancement is to be based on
Color constancy and the sheaf theoretic Retinex method of retina skin.Picture breakdown is essential image and illumination image by this method
Product, thus eliminate because by the illumination factor that haze blocks on image imaging influenced.Retinex method is than traditional pair
It is compared than degree method for improving, obtained mist elimination image has better local contrast, and cross-color is smaller.But due to
Retinex method itself is also an ill-conditioning problem, can only carry out approximate evaluation, thus also affects image to a certain extent
Defog effect.
Using atmospherical scattering model, (I=JT+ (1-T) A, wherein I indicates foggy image, J table to method based on physical model
Showing fog free images) estimation scene medium perspective rate T and global atmosphere light are according to A respectively, to obtain clearly fog free images.However
Under the conditions of only individual foggy image, estimation T and A is also an ill-conditioning problem, can only carry out near-sighted estimation.It is dissipated using atmosphere
Penetrate the method that foggy image is restored to fog free images by model, can generally be divided into three classes: the 1st class is based on depth information
Method；2nd class is the defogging algorithm based on atmosphere light polarization；3rd class is the method based on priori knowledge.Preceding two classes method
Hand fit is usually required, can just obtain preferable as a result, and the 3rd class method is method relatively common at present, such as base
In the method for dark statistics priori, the method based on Color Statistical priori.These methods are obtained by then passing through statistical information
Knowledge, do not adapt to all scenes, such as saturating based on the region estimation brighter to sky of the method for dark channel prior knowledge
Viewing system just will appear deviation, and the image after leading to defogging is whole partially dark.
Based on the method for deep learning using the technologies such as artificial synthesized foggy image data set and convolutional neural networks come
It realizes defogging, is specifically divided into two classes: (1) being to indicate atmospherical scattering model using deep neural network, learn automatically and estimate
Corresponding T and A.Different from being shone based on the methods of priori knowledge estimation perspective with atmosphere light, such methods are mainly from data
Learnt, to overcome the deviation of part priori knowledge, but such methods usually require known scene depth and could synthesize
T is obtained, so as to the study that exercises supervision；(2) on the basis of not making any hypothesis or estimation to T and A, directly by defogging process
It is considered as the transformation either image synthesis of image.The side such as contrast enhancing, white balance is usually utilized based on image synthetic method
Method pre-processes foggy image, then again by neural network learning weighting function, after carrying out fusion pretreatment
Image, to realize defogging.But this method is easy have stronger dependence to pretreatment image, and single-frame images is handled
Time is longer.Method based on image transformation directly utilizes non-linear between neural network learning foggy image and fog free images
Transforming function transformation function, to obtain fog free images.But this method is in default of the control of real scene, thus to the dependence of data
It is very strong.
Summary of the invention
In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is that providing a kind of based on priori
Knowledge elicitation condition generates the single image defogging method of confrontation network, and priori knowledge is utilized to instruct coding network to carry out nothing
Mist result generates, and the part useful information that priori knowledge obtains is utilized, while the feature that deep neural network is utilized again is built
Mould ability compensates for the deficiency of priori knowledge, does not need the display in deep neural network and establishes atmospherical scattering model, but will
Its condition for being considered as image generates, and defog effect is good, convenient for promoting the use of.
In order to solve the above technical problems, the technical solution adopted by the present invention is that: based on the generation pair of priori knowledge guiding conditions
The single image defogging method of anti-network, which is characterized in that method includes the following steps:
Step 1: establishing atomization training set of images: using the image data set of known depth, being closed according to atmospherical scattering model
At groups of foggy image training set；
Step 2: the preliminary defogging of individual random foggy image: being mentioned at random in the atomization training set of images in step 1
A foggy image is taken, according to formulaPreliminary mist elimination image is obtained by priori knowledgeWherein, Ih
For individual random foggy image, A is that global atmosphere light is shone, and T is medium perspective rate；
Step 3: the defogging training of preliminary mist elimination image: according to formulaIndividual is random
Foggy image IhWith preliminary mist elimination imageIt is overlapped, is sent to condition and generates in the generator G of confrontation network, obtain preliminary
The defogging training image of mist elimination imageWherein, concat () is superpositing function, fg(·)θIt indicates using θ as weight parameter collection
The condition of conjunction generates the coding network of confrontation network generator G, fg(·)θIt is made of multiple convolution blocks；
Step 4: calculating the truth value of the defogging training image with reference to true value image and preliminary mist elimination image: according to formulaCalculate the defogging training image of the truth value D (J) and preliminary mist elimination image with reference to true value image J
Truth valueWherein, with reference to the truth value D (J) of true value image J and the defogging training image of preliminary mist elimination image
Truth valueValue range be (0,1], fd(·)θ'It indicates with the volume for the arbiter D that θ ' is weight parameter set
Code network, fd(·)θ'It is made of multiple convolution blocks；
Step 5: according to formula L=λ1L1+λ2Ledge+λ3LGAN, calculate image impairment objective function L, wherein L1For reference
The defogging training image of true value image J and preliminary mist elimination imageConfrontation loss andN is image slices
Plain total number, n are image pixel variable,For the defogging training image of preliminary mist elimination imageNth pixel point pixel
Value, Jn be with reference to true value image J withThe pixel value of the pixel of corresponding position, LedgeFor with reference to true value image J and tentatively
The defogging training image of mist elimination imageEdge-smoothing loss and Ledge=| grad (JG)-grad (J) |, grad () is figure
As gradient function, LGANFor the defogging training image with reference to true value image J and preliminary mist elimination imageAbsolute Error Loss andΕ () is expectation function, λ1For L1Weight coefficient, λ2For LedgePower
Weight coefficient, λ3For LGANWeight coefficient；
Step 6: updating weight parameter set: image impairment objective function L being sent into Adam optimizer, in step 3
Condition generate confrontation network generator G coding network fg(·)θWeight parameter set θ and step 4 in arbiter D
Coding network fd(·)θ'Weight parameter set θ ' be updated；
Step 7: individual new random foggy image is transferred, circulation step two to step 6, until
At this point, obtaining the coding network f that condition generates confrontation network generator Gg(·)θWeight parameter set θ training result be θ,
Condition generates the coding network f of confrontation network generator Gg(·)θTraining result be fg(·)θ, wherein Δ1For first true and false
Threshold value, Δ2For the second true and false threshold value；
Step 8: individual practical foggy image defogging: generating the coding of confrontation network generator G using trained condition
Network fg(·)θTo individual practical foggy image defogging, individual practical mist elimination image J is obtainedG, i.e.,Wherein, IhFor individual practical foggy image,For tentatively going for individual practical foggy image
Mist image and
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
Global atmosphere light passes through dark defogging algorithm, median filtering defogging algorithm, multiple dimensioned according to A and medium perspective rate T in step 2
Retinex algorithm for image enhancement, adaptive histogram equalization algorithm or adaptive contrast and the image algorithm of color range enhancing
It obtains.
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
The convolution block includes the successively convolution of operation, activation primitive, batch normalization operation.
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
Step 3 conditional generates the coding network f of confrontation network generator Gg(·)θWhen first used, it is raw to generate confrontation network for condition
Grow up to be a useful person the coding network f of Gg(·)θWeight parameter set θ carry out random initializtion；The coding network of arbiter D in step 4
fd(·)θ'When first used, the coding network f of arbiter Dd(·)θ'Weight parameter set θ ' carry out random initializtion.
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
λ in step 51Value range be 100~200；λ2Value range be 10~20；λ3Value range be 1~2.
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
First true and false threshold value Δ in step 71With the second true and false threshold value Δ2Value range be 0~0.1.
The above-mentioned single image defogging method that confrontation network is generated based on priori knowledge guiding conditions, it is characterised in that:
The image data set of the known depth includes NYU image data set.
Compared with the prior art, the present invention has the following advantages:
1, it after the present invention carries out preliminary defogging to foggy image using priori knowledge, then is overlapped with foggy image, work
It is sent into and is generated in confrontation network for condition, after training by great amount of samples, fog free images can be generated using generator,
In, the preliminary defogging result of foggy image can be schemed by accomplished in many ways, such as common dark channel prior method, Retinex
Preliminary mist elimination image is superimposed by image intensifying method etc. with foggy image on this basis, is then re-fed into condition and is generated confrontation network
It is trained, and realizes the defogging of single image by trained generator, any tradition transcendental method can carry out tentatively
Defogging, it is adaptable, convenient for promoting the use of.
2, the present invention can effectively combine the advantage of traditional priori knowledge method and the method based on deep learning, have and make an uproar
The advantage that acoustic jamming is small, mist elimination image fidelity is high, processing speed is fast, reliable and stable, using effect is good.
3, the method for the present invention step is simple, is tentatively gone based on the method for priori knowledge to foggy image by traditional
Then mist recycles this result that condition is instructed to generate confrontation network and generates fog free images, single image defogging thinking is novel, even if
It is that still can obtain preferable defogging as a result, convenient for promoting the use of under condition of small sample.
4, priori knowledge guiding conditions of the present invention generate the single image defogging frame of confrontation network, can be more with effective integration
The traditional defogging method of kind, models weight system between multiple images without showing, utilizes the training of disclosed standard data set
Experimental result, method applicability is strong, and image procossing precision is high, and defog effect is good.
In conclusion present invention utilizes priori knowledges to instruct coding network to carry out fogless result generation, elder generation is utilized
The part useful information that knowledge obtains is tested, while the feature modeling ability that deep neural network is utilized again compensates for priori knowledge
Deficiency, do not need in deep neural network display and establish atmospherical scattering model, but the condition for being regarded as image generates,
Defog effect is good, convenient for promoting the use of.
Below by drawings and examples, technical scheme of the present invention will be described in further detail.
Detailed description of the invention
Fig. 1 is the method flow block diagram of the method for the present invention.
Fig. 2 is individual practical foggy image of one scene of the present embodiment.
Fig. 3 is individual practical mist elimination image of Fig. 2.
Fig. 4 is individual practical foggy image of another scene of the present embodiment.
Fig. 5 is individual practical mist elimination image of Fig. 4.
Fig. 6 is individual practical foggy image of the present embodiment third scene.
Fig. 7 is individual practical mist elimination image of Fig. 6.
Specific embodiment
As shown in Figure 1, the single image defogging method of the invention that confrontation network is generated based on priori knowledge guiding conditions,
The following steps are included:
Step 1: establishing atomization training set of images: using the image data set of known depth, being closed according to atmospherical scattering model
At groups of foggy image training set, the effective image data amount for expanding foggy image training set；
In the present embodiment, the image data set of the known depth includes NYU image data set.
Step 2: the preliminary defogging of individual random foggy image: being mentioned at random in the atomization training set of images in step 1
A foggy image is taken, according to formulaPreliminary mist elimination image is obtained by priori knowledgeWherein, Ih
For individual random foggy image, A is that global atmosphere light is shone, and T is medium perspective rate；
In the present embodiment, global atmosphere light passes through dark defogging algorithm, intermediate value according to A and medium perspective rate T in step 2
Filter defogging algorithm, multiple dimensioned Retinex algorithm for image enhancement, adaptive histogram equalization algorithm or adaptive contrast and
The image algorithm of color range enhancing obtains.
It should be noted that after carrying out preliminary defogging to foggy image using priori knowledge, then folded with foggy image
Add, is sent into and is generated in confrontation network as condition, after training by great amount of samples, fogless figure can be generated using generator
Picture, wherein the preliminary defogging result of foggy image can by accomplished in many ways, such as common dark channel prior method,
Preliminary mist elimination image is superimposed by Retinex image enhancement method etc. with foggy image on this basis, is then re-fed into condition generation
Confrontation network is trained, and the defogging of single image is realized by trained generator, any tradition transcendental method
Preliminary defogging is carried out, it is adaptable.
Step 3: the defogging training of preliminary mist elimination image: according to formulaIndividual is random
Foggy image IhWith preliminary mist elimination imageIt is overlapped, is sent to condition and generates in the generator G of confrontation network, obtain preliminary
The defogging training image of mist elimination imageWherein, concat () is superpositing function, fg(·)θIt indicates using θ as weight parameter collection
The condition of conjunction generates the coding network of confrontation network generator G, fg(·)θIt is made of multiple convolution blocks；
In the present embodiment, step 3 conditional generates the coding network f of confrontation network generator Gg(·)θWhen first used,
Condition generates the coding network f of confrontation network generator Gg(·)θWeight parameter set θ carry out random initializtion；In step 4
The coding network f of arbiter Dd(·)θ'When first used, the coding network f of arbiter Dd(·)θ'Weight parameter set θ ' into
Row random initializtion.
In the present embodiment, the convolution block includes the successively convolution of operation, activation primitive, batch normalization operation.
It should be noted that condition generates the coding network f of confrontation network generator Gg(·)θIt is made of multiple convolution blocks,
Convolution block includes the successively convolution of operation, activation primitive, batch normalization operation, avoids using full connection and f caused by pondizationg
(·)θIt is unstable.
It should be noted that carrying out preliminary defogging to foggy image based on the method for priori knowledge by traditional, then
It recycles this result that condition is instructed to generate confrontation network and generates fog free images, single image defogging thinking is novel, even small
Under sample conditions, preferable defogging result can be still obtained.
Step 4: calculating the truth value of the defogging training image with reference to true value image and preliminary mist elimination image: according to formulaCalculate the defogging training image of the truth value D (J) and preliminary mist elimination image with reference to true value image J
Truth valueWherein, with reference to the truth value D (J) of true value image J and the defogging training image of preliminary mist elimination image
Truth valueValue range be (0,1], fd(·)θ'It indicates with the volume for the arbiter D that θ ' is weight parameter set
Code network, fd(·)θ'It is made of multiple convolution blocks；
It should be noted that with reference to the truth value D (J) closer 1 of true value image J, the defogging training of preliminary mist elimination image
ImageTruth valueCloser to 0, declaration condition generates the coding network f of confrontation network generator Gg(·)θTraining
As a result better.
Step 5: according to formula L=λ1L1+λ2Ledge+λ3LGAN, calculate image impairment objective function L, wherein L1For reference
The defogging training image of true value image J and preliminary mist elimination imageConfrontation loss andN is image
Total number of pixels, n are image pixel variable,For the defogging training image of preliminary mist elimination imageNth pixel point picture
Element value, Jn be with reference to true value image J withThe pixel value of the pixel of corresponding position, LedgeFor with reference to true value image J and just
Walk the defogging training image of mist elimination imageEdge-smoothing loss and Ledge=| grad (JG)-grad (J) |, grad () is
Image gradient function, LGANFor the defogging training image with reference to true value image J and preliminary mist elimination imageAbsolute Error Loss andΕ () is expectation function, λ1For L1Weight coefficient, λ2For LedgePower
Weight coefficient, λ3For LGANWeight coefficient；
In the present embodiment, λ in step 51Value range be 100~200；λ2Value range be 10~20；λ3Take
Being worth range is 1~2.
Step 6: updating weight parameter set: image impairment objective function L being sent into Adam optimizer, in step 3
Condition generate confrontation network generator G coding network fg(·)θWeight parameter set θ and step 4 in arbiter D
Coding network fd(·)θ'Weight parameter set θ ' be updated；
It should be noted that image impairment objective function L is sent into Adam optimizer, Adam optimizer optimizes automatically
Operation, anti-pass gradient realize that the condition in step 3 generates the coding network f of confrontation network generator Gg(·)θWeight parameter
The coding network f of arbiter D in set θ and step 4d(·)θ'Weight parameter set θ ' update, can effectively combine
The advantage of traditional priori knowledge method and the method based on deep learning, mist elimination image fidelity height, place small with noise jamming
Manage fireballing advantage.
Step 7: individual new random foggy image is transferred, circulation step two to step 6, until
At this point, obtaining the coding network f that condition generates confrontation network generator Gg(·)θWeight parameter set θ training result be θ,
Condition generates the coding network f of confrontation network generator Gg(·)θTraining result be fg(·)θ, wherein Δ1For first true and false
Threshold value, Δ2For the second true and false threshold value；
In the present embodiment, the first true and false threshold value Δ in step 71With the second true and false threshold value Δ2Value range be 0~
0.1。
It should be noted that priori knowledge guiding conditions generate the single image defogging frame of confrontation network, it can be effective
A variety of traditional defogging methods are merged, weight system between multiple images is modeled without showing, utilizes disclosed normal data
Collect training experimental result, method applicability is strong, and image procossing precision is high, and defog effect is good.
Step 8: individual practical foggy image defogging: generating the coding of confrontation network generator G using trained condition
Network fg(·)θTo individual practical foggy image defogging, individual practical mist elimination image J is obtainedG, i.e.,Wherein, IhFor individual practical foggy image,For tentatively going for individual practical foggy image
Mist image and
The present invention is in use, as shown in Figures 2 to 7, the coding of confrontation network generator G is generated using trained condition
Network fg(·)θRespectively to individual practical foggy image defogging, i.e. Fig. 2, Fig. 4 and Fig. 6 in three scenes, three are respectively obtained
Priori knowledge is utilized to instruct coding network to carry out nothing in scene corresponding individual practical mist elimination image, i.e. Fig. 3, Fig. 5 and Fig. 7
Mist result generates, and the part useful information that priori knowledge obtains is utilized, while the feature that deep neural network is utilized again is built
Mould ability compensates for the deficiency of priori knowledge, does not need the display in deep neural network and establishes atmospherical scattering model, but will
Its condition for being considered as image generates, and defog effect is good.
The above is only presently preferred embodiments of the present invention, is not intended to limit the invention in any way, it is all according to the present invention
Technical spirit any simple modification to the above embodiments, change and equivalent structural changes, still fall within skill of the present invention
In the protection scope of art scheme.
Claims (7)
1. generating the single image defogging method of confrontation network based on priori knowledge guiding conditions, which is characterized in that this method packet
Include following steps:
Step 1: establishing atomization training set of images: using the image data set of known depth, according to atmospherical scattering model synthesis at
The foggy image training set of group；
Step 2: the preliminary defogging of individual random foggy image: extracting one at random in the atomization training set of images in step 1
Foggy image is opened, according to formulaPreliminary mist elimination image is obtained by priori knowledgeWherein, IhFor list
Zhang Suiji foggy image, A are that global atmosphere light is shone, and T is medium perspective rate；
Step 3: the defogging training of preliminary mist elimination image: according to formulaIndividual there is into mist at random
Image IhWith preliminary mist elimination imageIt is overlapped, is sent to condition and generates in the generator G of confrontation network, obtain preliminary defogging
The defogging training image of imageWherein, concat () is superpositing function, fg(·)θIt indicates using θ as weight parameter set
Condition generates the coding network of confrontation network generator G, fg(·)θIt is made of multiple convolution blocks；
Step 4: calculating the truth value of the defogging training image with reference to true value image and preliminary mist elimination image: according to formulaCalculate the defogging training image of the truth value D (J) and preliminary mist elimination image with reference to true value image J
Truth valueWherein, with reference to the truth value D (J) of true value image J and the defogging training image of preliminary mist elimination image
Truth valueValue range be (0,1], fd(·)θ'It indicates with the coding for the arbiter D that θ ' is weight parameter set
Network, fd(·)θ'It is made of multiple convolution blocks；
Step 5: according to formula L=λ1L1+λ2Ledge+λ3LGAN, calculate image impairment objective function L, wherein L1For with reference to true value
The defogging training image of image J and preliminary mist elimination imageConfrontation loss andN is that image pixel is total
Number, n are image pixel variable,For the defogging training image of preliminary mist elimination imageNth pixel point pixel value,
JnFor with reference to true value image J withThe pixel value of the pixel of corresponding position, LedgeTo refer to true value image J and preliminary defogging
The defogging training image of imageEdge-smoothing loss and Ledge=| grad (JG)-grad (J) |, grad () is image ladder
Spend function, LGANFor the defogging training image with reference to true value image J and preliminary mist elimination imageAbsolute Error Loss andΕ () is expectation function, λ1For L1Weight coefficient, λ2For LedgePower
Weight coefficient, λ3For LGANWeight coefficient；
Step 6: updating weight parameter set: image impairment objective function L being sent into Adam optimizer, to the item in step 3
Part generates the coding network f of confrontation network generator Gg(·)θWeight parameter set θ and step 4 in arbiter D coding
Network fd(·)θ'Weight parameter set θ ' be updated；
Step 7: individual new random foggy image is transferred, circulation step two to step 6, untilAt this point,
Obtain the coding network f that condition generates confrontation network generator Gg(·)θWeight parameter set θ training result be θ, condition
Generate the coding network f of confrontation network generator Gg(·)θTraining result be fg(·)θ, wherein Δ1For the first true and false threshold value,
Δ2For the second true and false threshold value；
Step 8: individual practical foggy image defogging: generating the coding network of confrontation network generator G using trained condition
fg(·)θTo individual practical foggy image defogging, individual practical mist elimination image J is obtainedG, i.e.,Its
In, IhFor individual practical foggy image,For individual practical foggy image preliminary mist elimination image and
2. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: global atmosphere light according to A and medium perspective rate T passes through dark defogging algorithm, median filtering defogging in step 2
Algorithm, multiple dimensioned Retinex algorithm for image enhancement, adaptive histogram equalization algorithm or adaptive contrast and color range enhancing
Image algorithm obtain.
3. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: the convolution block includes the successively convolution of operation, activation primitive, batch normalization operation.
4. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: step 3 conditional generates the coding network f of confrontation network generator Gg(·)θWhen first used, condition generates
Fight the coding network f of network generator Gg(·)θWeight parameter set θ carry out random initializtion；Arbiter D in step 4
Coding network fd(·)θ'When first used, the coding network f of arbiter Dd(·)θ'Weight parameter set θ ' carry out it is random
Initialization.
5. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: λ in step 51Value range be 100~200；λ2Value range be 10~20；λ3Value range be 1
~2.
6. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: the first true and false threshold value Δ in step 71With the second true and false threshold value Δ2Value range be 0~0.1.
7. the single image defogging method described in accordance with the claim 1 that confrontation network is generated based on priori knowledge guiding conditions,
It is characterized by: the image data set of the known depth includes NYU image data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580136.7A CN110288550B (en) | 2019-06-28 | 2019-06-28 | Single-image defogging method for generating countermeasure network based on priori knowledge guiding condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580136.7A CN110288550B (en) | 2019-06-28 | 2019-06-28 | Single-image defogging method for generating countermeasure network based on priori knowledge guiding condition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110288550A true CN110288550A (en) | 2019-09-27 |
| CN110288550B CN110288550B (en) | 2020-04-24 |
Family
ID=68019837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910580136.7A Active CN110288550B (en) | 2019-06-28 | 2019-06-28 | Single-image defogging method for generating countermeasure network based on priori knowledge guiding condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110288550B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807749A (en) * | 2019-11-06 | 2020-02-18 | 广西师范大学 | Single image raindrop removing method based on dense multi-scale generation countermeasure network |
| CN111179202A (en) * | 2019-12-31 | 2020-05-19 | 内蒙古工业大学 | Single image defogging enhancement method and system based on generation countermeasure network |
| CN111709888A (en) * | 2020-06-03 | 2020-09-25 | 中科九度(北京)空间信息技术有限责任公司 | Aerial image defogging method based on improved generation countermeasure network |
| CN112750093A (en) * | 2021-01-25 | 2021-05-04 | 中国人民解放***箭军工程大学 | Video image defogging method based on time sequence label transmission |
| CN112767275A (en) * | 2021-01-25 | 2021-05-07 | 中国人民解放***箭军工程大学 | Single image defogging method based on artificial sparse annotation information guidance |
| CN113658051A (en) * | 2021-06-25 | 2021-11-16 | 南京邮电大学 | Image defogging method and system based on cyclic generation countermeasure network |
| WO2023019507A1 (en) * | 2021-08-19 | 2023-02-23 | 浙江吉利控股集团有限公司 | Road image generation method and apparatus based on deep learning, and device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108665432A (en) * | 2018-05-18 | 2018-10-16 | 百年金海科技有限公司 | A kind of single image to the fog method based on generation confrontation network |
| CN108765325A (en) * | 2018-05-17 | 2018-11-06 | 中国人民解放军陆军工程大学 | Small unmanned aerial vehicle blurred image restoration method |
| CN109064422A (en) * | 2018-07-17 | 2018-12-21 | 中国海洋大学 | A kind of underwater image restoration method based on fusion confrontation network |
| US20190005603A1 (en) * | 2017-06-30 | 2019-01-03 | Intel Corporation | Approximating image processing functions using convolutional neural networks |
| CN109493303A (en) * | 2018-05-30 | 2019-03-19 | 湘潭大学 | A kind of image defogging method based on generation confrontation network |
| US10325201B1 (en) * | 2019-01-31 | 2019-06-18 | StradVision, Inc. | Method and device for generating deceivable composite image by using GAN including generating neural network and discriminating neural network to allow surveillance system to recognize surroundings and detect rare event more accurately |
-
2019
- 2019-06-28 CN CN201910580136.7A patent/CN110288550B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190005603A1 (en) * | 2017-06-30 | 2019-01-03 | Intel Corporation | Approximating image processing functions using convolutional neural networks |
| CN108765325A (en) * | 2018-05-17 | 2018-11-06 | 中国人民解放军陆军工程大学 | Small unmanned aerial vehicle blurred image restoration method |
| CN108665432A (en) * | 2018-05-18 | 2018-10-16 | 百年金海科技有限公司 | A kind of single image to the fog method based on generation confrontation network |
| CN109493303A (en) * | 2018-05-30 | 2019-03-19 | 湘潭大学 | A kind of image defogging method based on generation confrontation network |
| CN109064422A (en) * | 2018-07-17 | 2018-12-21 | 中国海洋大学 | A kind of underwater image restoration method based on fusion confrontation network |
| US10325201B1 (en) * | 2019-01-31 | 2019-06-18 | StradVision, Inc. | Method and device for generating deceivable composite image by using GAN including generating neural network and discriminating neural network to allow surveillance system to recognize surroundings and detect rare event more accurately |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807749A (en) * | 2019-11-06 | 2020-02-18 | 广西师范大学 | Single image raindrop removing method based on dense multi-scale generation countermeasure network |
| CN110807749B (en) * | 2019-11-06 | 2022-11-25 | 联友智连科技有限公司 | Single image raindrop removing method based on dense multi-scale generation countermeasure network |
| CN111179202B (en) * | 2019-12-31 | 2022-09-02 | 内蒙古工业大学 | Single image defogging enhancement method and system based on generation countermeasure network |
| CN111179202A (en) * | 2019-12-31 | 2020-05-19 | 内蒙古工业大学 | Single image defogging enhancement method and system based on generation countermeasure network |
| CN111709888A (en) * | 2020-06-03 | 2020-09-25 | 中科九度(北京)空间信息技术有限责任公司 | Aerial image defogging method based on improved generation countermeasure network |
| CN111709888B (en) * | 2020-06-03 | 2023-12-08 | 中科九度(北京)空间信息技术有限责任公司 | Aerial image defogging method based on improved generation countermeasure network |
| CN112750093A (en) * | 2021-01-25 | 2021-05-04 | 中国人民解放***箭军工程大学 | Video image defogging method based on time sequence label transmission |
| CN112767275B (en) * | 2021-01-25 | 2021-10-22 | 中国人民解放***箭军工程大学 | Single image defogging method based on artificial sparse annotation information guidance |
| CN112750093B (en) * | 2021-01-25 | 2021-10-22 | 中国人民解放***箭军工程大学 | Video image defogging method based on time sequence label transmission |
| CN112767275A (en) * | 2021-01-25 | 2021-05-07 | 中国人民解放***箭军工程大学 | Single image defogging method based on artificial sparse annotation information guidance |
| CN113658051A (en) * | 2021-06-25 | 2021-11-16 | 南京邮电大学 | Image defogging method and system based on cyclic generation countermeasure network |
| CN113658051B (en) * | 2021-06-25 | 2023-10-13 | 南京邮电大学 | Image defogging method and system based on cyclic generation countermeasure network |
| WO2023019507A1 (en) * | 2021-08-19 | 2023-02-23 | 浙江吉利控股集团有限公司 | Road image generation method and apparatus based on deep learning, and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110288550B (en) | 2020-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110288550A (en) | The single image defogging method of confrontation network is generated based on priori knowledge guiding conditions | |
| CN108876735B (en) | Real image blind denoising method based on depth residual error network | |
| CN107103591B (en) | Single image defogging method based on image haze concentration estimation | |
| CN109300090A (en) | A kind of single image to the fog method generating network based on sub-pix and condition confrontation | |
| CN110363727A (en) | Image defogging method based on multiple dimensioned dark channel prior cascade deep neural network | |
| CN109255758A (en) | Image enchancing method based on full 1*1 convolutional neural networks | |
| CN105447825B (en) | Image defogging method and its system | |
| CN111598791A (en) | Image defogging method based on improved dynamic atmospheric scattering coefficient function | |
| CN113284061B (en) | Underwater image enhancement method based on gradient network | |
| CN111242911A (en) | Method and system for determining image definition based on deep learning algorithm | |
| CN109949239B (en) | Self-adaptive sharpening method suitable for multi-concentration multi-scene haze image | |
| CN110349113B (en) | Adaptive image defogging method based on dark primary color priori improvement | |
| CN112419163B (en) | Single image weak supervision defogging method based on priori knowledge and deep learning | |
| CN110189262B (en) | Image defogging method based on neural network and histogram matching | |
| CN115937019A (en) | Non-uniform defogging method combining LSD (local Scale decomposition) quadratic segmentation and deep learning | |
| CN112070691A (en) | Image defogging method based on U-Net | |
| CN108765337B (en) | Single color image defogging processing method based on dark channel prior and non-local MTV model | |
| CN109685735B (en) | Single picture defogging method based on fog layer smoothing prior | |
| CN111553856A (en) | Image defogging method based on depth estimation assistance | |
| CN117422653A (en) | Low-light image enhancement method based on weight sharing and iterative data optimization | |
| CN107767348B (en) | Single tunnel image rapid enhancement method based on imaging model constraint | |
| CN114627005A (en) | Rain density classification guided dual-stage single image rain removing method | |
| CN114549343A (en) | Defogging method based on dual-branch residual error feature fusion | |
| CN110796607B (en) | Deep learning low-illumination image enhancement method based on retina cerebral cortex theory | |
| CN110992293A (en) | Self-adaptive video defogging method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |