CN109035142A - A kind of satellite image ultra-resolution method fighting network integration Aerial Images priori - Google Patents
A kind of satellite image ultra-resolution method fighting network integration Aerial Images priori Download PDFInfo
- Publication number
- CN109035142A CN109035142A CN201810777731.5A CN201810777731A CN109035142A CN 109035142 A CN109035142 A CN 109035142A CN 201810777731 A CN201810777731 A CN 201810777731A CN 109035142 A CN109035142 A CN 109035142A
- Authority
- CN
- China
- Prior art keywords
- image
- loss
- resolution
- model
- satellite
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 230000010354 integration Effects 0.000 title claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 72
- 238000012805 post-processing Methods 0.000 claims abstract description 12
- 238000003707 image sharpening Methods 0.000 claims abstract description 9
- 238000001914 filtration Methods 0.000 claims abstract description 7
- 230000000007 visual effect Effects 0.000 claims abstract description 6
- 230000006870 function Effects 0.000 claims description 35
- 230000008447 perception Effects 0.000 claims description 27
- 230000008569 process Effects 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 239000000203 mixture Substances 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 230000003321 amplification Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000004321 preservation Methods 0.000 claims description 3
- 238000002474 experimental method Methods 0.000 abstract description 9
- 230000000694 effects Effects 0.000 description 12
- 238000012360 testing method Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000015556 catabolic process Effects 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 206010021403 Illusion Diseases 0.000 description 1
- 235000009754 Vitis X bourquina Nutrition 0.000 description 1
- 235000012333 Vitis X labruscana Nutrition 0.000 description 1
- 240000006365 Vitis vinifera Species 0.000 description 1
- 235000014787 Vitis vinifera Nutrition 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013100 final test Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a kind of satellite image ultra-resolution methods for fighting network integration Aerial Images priori, data training image super-resolution model of clearly taking photo by plane is recycled to training denoising model by using 16 grades of corresponding images constituted without noise image of 16 grades of noisy acoustic images first.Since there is no the situation of satellite image and Aerial Images pair, when carrying out post processing of image to the super resolution image of generation, using priori dictionary outside clearly Aerial Images building GMM model, and thus, the internal unsharp satellite image of guidance is rebuild.It is further promotion picture quality after reconstruction, carries out image sharpening using the mode of gaussian filtering.The full resolution pricture of protosatellite image is finally obtained, and realizes that the visual quality of images in protosatellite image basis is promoted.By experiment link it can also be seen that the validity of this programme.Satellite image super-resolution and picture quality to solve to have ready conditions under limited case in reality provide effective thinking.
Description
Technical field
The invention belongs to Image Super-resolution technical fields, and in particular to one kind is lost based on multiple dimensioned perception and fought with generation
The satellite image ultra-resolution method of network integration Aerial Images priori.
Background technique
Image resolution ratio is the important indicator of picture quality, and the higher image of resolution ratio clearer can be shown more
Details, but influenced in obtaining image process by hardware and external environment, the image resolution ratio of acquisition is lower, to generate
The problem of how obtaining full resolution pricture from the image of low resolution.Currently, the increase of the quantity with satellite, satellite can be covered
Earth range is more than 90%, this makes the range that can be monitored by satellite be significantly larger than the model that other means acquisition image is covered
It encloses, but satellite image is influenced by many-sided reason, resolution ratio is lower.Such as relative to Aerial Images, satellite image is opposite
It is fuzzy to lack detailed information, but the covering surface of Aerial Images is far away from satellite image, so it is higher how to obtain resolution ratio
Satellite image has great significance and is worth.
In Image Super-resolution field, the combination of deep neural network and traditional images super-resolution problem, so that image oversubscription
The technology of distinguishing has new breakthrough.With the development of computer hardware equipment, the cost that extensive computation accelerates is significantly reduced, training
The cost of deep neural network reduces, and researcher is greatly facilitated, but also this technical application is extensively in every field.
Network is fought from the generation of network SRCNN of the deep learning initially proposed in conjunction with super-resolution problem till now
The super resolution algorithm SRGAN that (Generative Adversarial Nets, GAN) is realized, by using low resolution and high-resolution
Image is trained network parameter, to obtain from low-resolution image to full resolution pricture transformation model, at only low point
Full resolution pricture is generated in the case where distinguishing image.
Image Super-resolution problem is described as follows:
Image Super-resolution problem refers to that the image from a low resolution obtains the mistake of corresponding high-resolution image
Journey, the limitation of technological break-through original system imaging h ardware condition in this way, obtains clearer image.In Image Super-resolution
In technology, super-resolution problem in the case of generally can be divided into two kinds: ultra-resolution method based on single image and it is based on several figures
The ultra-resolution method of picture.Single image super-resolution improves image resolution by the amplification to low resolution image, by algorithm for reconstructing
The method of rate.Super resolution algorithm based on multiple image is rebuild using the method for the similar image sequences fusion of multiframe
High-resolution image out.
In the ultra-resolution method based on single image, algorithm is by establishing between low-resolution image and high-definition picture
Relationship.To rebuild high-resolution image by low-resolution image.Traditional algorithm is simulated low by various modes
The origin cause of formation of image in different resolution constructs various degradation models to be fitted the process of low-resolution image generation to construct low resolution
Relationship between image and high-definition picture, to predict to generate high-definition picture.Such simulation process is available following public
Formula description:
IL=HIH+n
Wherein ILFor low-resolution image, IHFor ILCorresponding high-definition picture, H are to generate moving back for low-resolution image
Change model, n is the noise jamming factor generated during low-resolution image.H can be indicated again as degradation model are as follows:
H=DSub×B×G
Wherein, DSubDownsapling method is represented, B is fuzzy factor, and G is the geometric deformation factor.
The method for solving the above degradation model building mainly has, the method based on interpolation, the method based on image reconstruction, with
And the method based on study.In interpolation method, by decomposing to image, interpolation and the method for returning to interpolated value realize figure
The super-resolution of picture, have the speed of service it is fast, can parallel computation, the requirement of realtime graphic super-resolution can be met.But interpolation method can not
The high-frequency information lost in from low-resolution image to high-definition picture is predicted, the high-definition picture of generation lacks texture
Details and clearly edge.In the super resolution algorithm based on image reconstruction, and it is divided into spatial domain method and frequency domain method, by airspace
Or establish the corresponding relationship of low-resolution image and high-definition picture in frequency domain, engineer's corresponding relationship model is realized
Process from low-resolution image to high-definition picture.Such as more classical projections onto convex sets, maximum a-posteriori estimation etc..
The defect of such method is that engineer's model can not be adapted to diversified image detail and restore, and building model is only
Good effect can be obtained in a small number of data, and the clear journey of image detail can not be further improved in the increased situation of data
Degree.
In the method based on study, similarly with the method based on image reconstruction, they are all by establishing low resolution
Relationship between rate image and high-definition picture, but the method based on study is using external trainer sample acquisition about low resolution
The priori knowledge of relationship between image and full resolution pricture, to realize the transformation from low resolution image to full resolution pricture.Such as
Method based on manifold learning, the method based on rarefaction representation and the method based on deep neural network.In rarefaction representation etc.
The reason of being difficult to ensure in learning method by building dictionary size and Deta sparseness limitation, can not obtain stable image
Super-resolution effect.In the ultra-resolution method based on deep neural network, it has been suggested that based on residual error network and based on generate
It fights in the methods of network, requires the learning training by quantity of parameters to low resolution image and full resolution pricture pair, in this way
Method equally exist and need mass data to training, when training, is easy to generate data over-fitting, and when test can not obtain very
The problems such as robustness got well.Meanwhile when predicting the high-frequency information of full resolution pricture, it still will appear missing, so that texture-rich
Region seems smooth.
There are also the limitation of some current conditions in satellite image super-resolution problem, very high-resolution can not be obtained at present
Satellite image, this is difficult to the data for obtaining high-resolution satellite image Yu low resolution image pair when allowing to carry out Image Super-resolution,
Much low-resolution image and the ultra-resolution method of full resolution pricture pair is needed to cannot be directly used to such satellite image super-resolution
Task.In the acquisition of satellite image, influence of noise is serious, so that grain noise is obvious in the image obtained, directly carries out single
Width Image Super-resolution can be such that the noise in image amplifies, and influence clarity.As auxiliary data, although Aerial Images covering surface is remote
Not as good as satellite image, but in Aerial Images and satellite image, similar place is very more, and takes photo by plane relative to satellite image
Image has extraordinary clarity.The image data and satellite image data that current acquisition is taken photo by plane do not have in pairs yet
Property, i.e., non-same place and the shooting of same period.How under existing limited conditions, satellite image is denoised,
Super-resolution and how using clearly Aerial Images data to satellite data carry out clarity enhancing become one need
It solves the problems, such as.
Summary of the invention
In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is that providing a kind of based on more rulers
The satellite image ultra-resolution method of degree perception loss and generation confrontation network integration Aerial Images priori, can make up commonly only makes
It is raw with the super resolution algorithm of satellite image in the process to the problem of the priori deficiency of clear image (lacking clearly satellite image)
At the satellite image being more clear.Meanwhile in the case where only using satellite data, since multiple dimensioned perception loss is added,
Produce the super resolution image being more clear than method for distinguishing.
The invention adopts the following technical scheme:
It is a kind of fight network integration Aerial Images priori satellite image ultra-resolution method, using 16 grades of noisy acoustic images with
Its corresponding 16 grades constitute image to training denoising model without noise image, then utilize data training image super-resolution of taking photo by plane
Model；Using priori dictionary outside Aerial Images building GMM model, and internal unsharp satellite image is guided to be rebuild,
The post-processing to the super resolution image of generation is completed, then image sharpening is carried out using the mode of gaussian filtering, finally obtains original
The full resolution pricture of satellite image realizes that the visual quality of images in protosatellite image basis is promoted.
Specifically, the following steps are included:
S1, definition generate the generator in confrontation network, and network is lost in decision device and multiple dimensioned perception；
S2, using in existing satellite data since 18 grades extract image down sampling to 16 grades, if obtain 16 grades defend
Target I of the star chart picture as denoisingD_H, the satellite data extracted from 16 grades is as noisy image ID_LImage pair is constituted, if generating
Not noisy satellite image be ID_GH；
S3, the image pair to be constituted in step S2, carry out initialization training to the generator in denoising model, are initializing
Pixel in training, using mean square error as loss function, between the corresponding target image of the image that calculating generator generates
Mean square error obtain MSE generator loss function lossMSE, calculate gradient and return adjustment model parameter；
S4, after the initialization of 100 epoch training, carry out complete model training, calculate loss and corresponding ladder
The parameter model in adjustment generator and decision device is spent and returns, network VGG19 not adjusting parameter is lost in perception；
S5, reach convergence according to 200 epoch of arrangement above training, preservation model, trained generator is at denoising
Reason uses, and the image after the denoising of acquisition is ID_GHAs the input of Image Super-resolution, satellite image super-resolution model is defined；
S6, step S3~S5 is repeated, completes super-resolution network training process and denoising model, then generates super resolution image
ISR_GH, external priori dictionary is constructed using gauss hybrid models；
17 grades of images of clearly taking photo by plane, are divided into the fritter of 15*15, then according to Europe by priori dictionary outside S7, building GMM
Formula distance is tentatively grouped；
S8, the inside graph block packet reconstruction satellite mapping according to reconstruction carry out image sharpening operation to the satellite mapping of reconstruction,
Obtain final result figure.
Further, in step S1, the generator in confrontation network is generated is defined as: use a residual error network as life
It grows up to be a useful person, includes 16 residual error modules in residual error network, include three convolutional layers in each residual error module；
Decision device structure is defined as: use one 10 layers of convolutional neural networks as decision device, convolutional neural networks
Convolutional layer uses empty convolution；
Multiple dimensioned perception loss is defined as: using on IMAGENET1000 class taxonomy database pre-training cross
VGG19 network perceptually loses network, by usingconv2_2,conv3_4,conv4_4, Analysis On Multi-scale Features figure in multilayer, building
Multiple dimensioned perception loss.
Further, in step S3, MSE generator loss function lossMSEIt is as follows:
lossMSE=MSE (ID_GH,ID_H)。
Further, in step S4, when model training, by the MSE generator loss function in generator loss function
lossMSE, perceive loss function lossvggWith confrontation loss function lossGANGenerator when whole training is constituted after weighting summation
Loss function is as follows:
lossG=lossMSE+lossvgg+lossGAN。
Further, perception loss lossvggIt is as follows:
lossvgg=10-6×(lossmse_conv2_2+lossmse_conv3_4+lossmse_conv4_4)
lossmse_conv2_2=MSE (fi_conv2_2,ft_conv2_2)
lossmse_conv3_4=MSE (fi_conv3_4,ft_conv3_4)
lossmse_conv4_4=MSE (fi_conv4_4,ft_conv4_4)
Wherein, fi_conv2_2, fi_conv3_4, fi_conv4_4It is corresponding into sensor model that image is generated for inputconv2_2,conv3_4,conv4_4
Layer characteristic pattern, ft_conv2_3, ft_conv3_3, ft_conv4_3It is right obtained in target image input sensor model to correspond to for generation image
It answersconv2_2,conv3_4,conv4_4Layer characteristic pattern；
Fight loss function lossGANIt is as follows:
lossGAN=10-4×cross_entropy(ID_GH,True)
cross_entropy(ID_GH, True) and=log (D (ID_GH))
Wherein, D () is decision device.
Further, in step S4, decision device loss function loss when entirety is trainedDIs defined as:
lossD=loss1+loss2
loss1=sigmoid_cross_entropy (ID_GH,False)
loss2=sigmoid_cross_entropy (ID_H,True)。
Further, in step S5, super-resolution model includes generator, sensor model and decision device, sensor model and is sentenced
Certainly device is identical as structure used in denoising model, and the generator defined in Image Super-resolution model is as follows:
By building residual error module, then multiple residual error modules fold composition network structure main body, and it is real to pass through sub-pix convolutional layer
Now the amplification of image is used.
Further, the data that the generator training of super-resolution model uses are data of taking photo by plane, and are inputted as ISR_LLow point
16 grades debated are taken photo by plane 17 grades of figure I that take photo by plane of figure high-resolution corresponding with itsSR_HThe image pair of composition, generator output are ISR_GH, fixed
The loss function of adopted generator is as follows:
lossMSE_SR=MSE (ISR_GH,ISR_H)。
Further, step S7 is specific as follows:
S701, GMM model is constructed according to the image block of grouping, SVD decomposition is carried out to covariance matrix in obtained model,
Dictionary is constructed, the reconstruction of satellite image below is guided as external priori；
S702, the I to be exported in the super-resolution model of frontSR_GHIt is inputted as internal image, 15*15 piecemeal is pressed after input,
It is clustered using GMM model guidance piecemeal when constructing external priori dictionary；
S703, the dictionary guidance internal image block constituted using external priori again pull up internal dictionary；
S704, the inside graph block for carrying out sparse coding to internal dictionary, and combining former internal image block packet reconstruction new
Grouping.
Compared with prior art, the present invention at least has the advantages that
A kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori of the present invention, in realistic situation
Wish to improve satellite image resolution ratio and effect of visualization, but the corresponding clearly Aerial Images of satellite image are not present
Pair situation, devise a set of satellite image super-resolution process, include image denoising, Image Super-resolution and post processing of image three
Part steps up the method flow of final satellite image super-resolution result within the scope of availability data
Further, the denoising model in satellite image super-resolution process and super-resolution model have used generation to fight
Network is constituted, and multiple dimensioned perception loss is added in proposition on this basis, is further promoted generation confrontation network and is being realized
Image denoising and performance when Image Super-resolution, the effect for perceiving loss be, generator is generated from property field image and
It corresponds to the constraint between target, so that generation image and real goal image are visually closer.Multiple dimensioned perception loss
Even more the perception loss of multiple scales is combined, added stronger constraint, has obtained further mentioning so generating effect
It rises.
Further, as the disparate modules generated in confrontation network, generator, decision device play the role of different.This
In realize satellite image denoising with satellite image super-resolution when respectively define play a significant role.Generator is mainly for point pair
Point pixel building loss, also focuses more on the high-frequency information for extracting image in network principal (by residual error structure).And differentiate
Device is more concern high-level semantic level, guarantees the consistency of the image generated and true target image, needs bigger
Receptive field (is realized) by empty convolution.Multiple dimensioned perception loss is then to generating between image and real goal image in property field
Constraint, here by using the network implementations that pre-training is crossed on IMAGENET.
Further, generator generates not Noise and with true clear image in the similar image of pixel scale, so
Loss function uses the MSE function based on difference between pixel.
Further, decision device constrains the similitude for generating image and true clear image from high-level semantics level.It uses
Intersecting entropy function is the loss function based on judgement probability, it is desirable to generate image and true target image semantically by
It is judged to same category of maximum probability.It generates image and real goal image is similar as far as possible.
Further, in super-resolution model, the effect of decision device and sensor model is identical as in noise model, so
Identical structure is used.Generator part, network principal is similar (but there is still a need for more high-frequency informations are generated, equally to be used
Residual error structure), but since super-resolution model needs to generate image more larger-sized than input low-resolution image, used here as Asia
Design that pixel convolutional layer cooperates with grape convolutional layer is realized.
Further, under reality, can not obtain clearer satellite image (close to the clarity of Aerial Images) with
Low resolution satellite image pair, this problem limit the realization effect of satellite image super-resolution, and the present invention is proposed in image oversubscription
The post processing of image method used after distinguishing further improves image viewing effect, by using the data GMM that clearly takes photo by plane
It constructs the internal dictionary of external priori dictionary guidance satellite image building and reconstructs clearer satellite image.
In conclusion the present invention is by being implemented in combination with denoising in the constraint of Pixel-level, semantic class and Analysis On Multi-scale Features domain
Model and Image Super-resolution model, while being directed to no pairs of satellite image training data, introduce Aerial Images carry out image into
The satellite mapping that the training of row super-resolution model and the GMM model dictionary in post processing of image construct that reconstruction is guided to be more clear
Picture.
Below by drawings and examples, technical scheme of the present invention will be described in further detail.
Detailed description of the invention
Fig. 1 is overall flow figure；
Fig. 2 is the structure chart of generator in denoising model；
Fig. 3 is the structure chart of arbiter in denoising model；
Fig. 4 is the structure chart for denoising VGG19 in device；
Fig. 5 is the generator structure chart in Image Super-resolution model；
Fig. 6 is the flow chart for constructing GMM model using Aerial Images and satellite image being guided to rebuild；
Fig. 7 is effect picture of the present invention；
Fig. 8 is that result of the present invention compares figure.
Specific embodiment
Defending for network integration Aerial Images priori is fought with generation based on multiple dimensioned perception loss the present invention provides a kind of
Star image super-resolution method, 16 grades of corresponding by using 16 grades of noisy acoustic images first figures constituted without noise image
As recycling data training image super-resolution model of clearly taking photo by plane to training denoising model.Since there is no satellite image with
The situation of Aerial Images pair is constructed when carrying out post processing of image to the super resolution image of generation using clearly Aerial Images
Priori dictionary outside GMM model, and thus the internal unsharp satellite image of guidance is rebuild.It is further to be promoted after reconstruction
Picture quality carries out image sharpening using the mode of gaussian filtering.The full resolution pricture of protosatellite image is finally obtained, and real
Visual quality of images in present protosatellite image basis is promoted.By experiment link it can also be seen that the validity of this programme.For solution
The satellite image super-resolution and picture quality certainly having ready conditions under limited case in reality, which are promoted, provides effective thinking.
It is lost based on multiple dimensioned perception referring to Fig. 1, the present invention is a kind of and generates confrontation network integration Aerial Images priori
Satellite image ultra-resolution method, the specific steps are as follows:
S1, the generation confrontation network for realizing denoising function include three parts, generator, decision device and a use
The good VGG19 network of IMAGENET database pre-training；
S101, definition generate the generator in confrontation network, used here as a residual error network as generator, wherein wrapping
It include three convolutional layers in each residual error module containing 16 residual error modules.The when denoising function for needing exist for realizing is not required to here
Image is amplified, specific structure is shown in Fig. 2.
S102, decision device structure is defined, decision device uses one 10 layers of convolutional neural networks here, and wherein convolutional layer makes
Increase the size of receptive field under conditions of without using pond layer by the way that the range size of empty convolution is arranged with empty convolution,
The accuracy of decision device is improved, specific structure is shown in Fig. 3, includes 10 convolutional layers, every layer of convolution kernel in the structure of decision device
Number is respectively that 64,128,256,512,1024,512,256,128,128,128 modes incremented by successively successively decreased again arrange, and preceding 7
The convolution kernel size of layer is 4*4, and step-length 2 successively carries out sliding convolution, and incremental convolution kernel number means as more as possible
Characteristic type.The last layer convolution kernel is used having a size of 1*1, and effect is to reduce parameter amount.Due to front convolution kernel number
Increase, port number is consequently increased, and needs to be added such one layer here and is adjusted.
S103, define multiple dimensioned perception loss, using on IMAGENET1000 class taxonomy database pre-training cross
VGG19 network perceptually loses network, unlike other perception losses, by usingconv2_2,conv3_4,conv4_4Multilayer
In Analysis On Multi-scale Features figure, construct multiple dimensioned perception loss, promote generator generation picture quality, specific structure is shown in Fig. 4, wherein
It include two convolutional layers and a pond layer comprising two kinds of convolution modules, in the first, comprising four in second of convolution module
Convolutional layer and a pond layer.Here all convolutional layers are all made of the convolution kernel of 3*3, and step-length 1, convolution kernel number is using similar
The mode being successively incremented by decision device is successively are as follows: and 64,64,128,128,256,256,256,256,512,512,512,512,
512,512,512,512. whereinconv2_2,conv3_4,conv4_4The output of respectively second convolution module, third convolution module
Output and the 4th convolution module output.
S2, the image down sampling extracted since 18 grades in existing satellite data to 16 grades of (general common satellite mappings is utilized
As being 16 grades, 18 grades of data procurement costs are higher), 16 grades of data being achieved in that are relatively clear, but due to 18 grades of satellites
Data acquisition cost is very high, and such clear data are considerably less.
If target I of the 16 grades of satellite images obtained as denoisingD_H, the common direct satellite data extracted from 16 grades
As noisy image ID_L, image pair is constituted in this way, if the not noisy satellite image generated is ID_GH；
S3, the image pair to be constituted in step S2, carry out initialization training to the generator in denoising model, are initializing
In training, using mean square error (MSE) as loss function, between the corresponding target image of the image that calculating generator generates
The mean square error of pixel calculates gradient and returns adjustment model parameter lossMSEIt is as follows:
lossMSE=MSE (ID_GH,ID_H)
S4, by about 100 epoch, (epoch refers to that image data all in image library all training are gone over
Can be regarded as an epoch) initialization training after, carry out the training of complete model；
At this moment three networks are intended to participate in training, but VGG19 not adjusting parameter, it is only necessary to which output perception loss is transmitted to generation
Device and decision device adjusting parameter；When whole training, the loss function of generator is different when individually initialization is trained relatively.
When whole training, the loss function of generator includes three parts: the loss of MSE generator, perception loss and confrontation
Loss, these three partial weightings constitute generator loss function when whole training after being added:
lossG=lossMSE+lossvgg+lossGAN
Wherein, lossMSEAs loss function when initialization training, lossvggIt is lost for perception:
lossvgg=10-6×(lossmse_conv2_2+lossmse_conv3_4+lossmse_conv4_4)
lossmse_conv2_2=MSE (fi_conv2_2,ft_conv2_2)
lossmse_conv3_4=MSE (fi_conv3_4,ft_conv3_4)
lossmse_conv4_4=MSE (fi_conv4_4,ft_conv4_4)
Wherein, fi_conv2_2, fi_conv3_4, fi_conv4_4It is corresponding into sensor model that image is generated for inputconv2_2
,conv3_4,conv4_4Layer characteristic pattern, ft_conv2_3, ft_conv3_3, ft_conv4_3Target image input sensor model is corresponded to generate image
Obtained in it is correspondingconv2_2,conv3_4,conv4_4Layer characteristic pattern；
lossGANTo fight loss function:
lossGAN=10-4×cross_entropy(ID_GH,True)
cross_entropy(ID_GH, True) and=log (D (ID_GH))
Wherein, D () is decision device.
Decision device loss function when whole training is defined as:
lossD=loss1+loss2
loss1=sigmoid_cross_entropy (ID_GH,False)
loss2=sigmoid_cross_entropy (ID_H,True)
Wherein, lossDFor decision device loss, calculates loss and corresponding gradient and return the parameter mould in adjustment decision device
Type.
S5, reach convergence, preservation model, wherein after trained generator is used for according to 200 epoch of arrangement above training
Face denoising uses, and the image after the denoising of acquisition is ID_GH, as the input of Image Super-resolution below, next definition is defended
Star Image Super-resolution model；
Super-resolution model also mainly includes three parts, i.e. generator, sensor model and decision device.Wherein sensor model and
Decision device is identical structure using used in structure and front denoising model.
Define the generator in Image Super-resolution model: generator portion body structure also uses residual error network, i.e., logical
Cross building residual error module then multiple residual error modules be folded and constitute network structure main body, behind realize the amplification of image used
Sub-pix convolutional layer (subpixel), specific structure are shown in Fig. 5, in the structure of super-resolution generator and previously defined denoising model
Structure it is similar, using multiple residual error modules be superimposed mode, wherein convolutional layer is all made of the convolution kernel of 3*3, convolution kernel number
It is 64, what subsequent sub-pix convolutional layer and its convolutional layer being correspondingly connected with were all made of is 256 convolution kernels, and convolutional layer uses 3*
The scale=1 of 3 convolution kernel, first sub-pix convolutional layer in the super-resolution model for realizing x2, second sub-pix convolution
The scale=2 of layer.
The generator of super-resolution model, training use data of taking photo by plane, if input is ISR_L16 grades of low explanation are taken photo by plane
Scheme high-resolution 17 grade take photo by plane figure I corresponding with itsSR_HThe image pair of composition, generator output is ISR_GH。
The loss function of generator is defined as:
lossMSE_SR=MSE (ISR_GH,ISR_H)
S6, step S3~S5 is repeated, completes super-resolution network training process and denoising model, then generates super resolution image
ISR_GH, for further combined with clearly priori uses gauss hybrid models (GMM) structure into satellite image here in Aerial Images
External priori dictionary is built, the super-resolution to guide the method that internal image is rebuild and image sharpening combines further to promote generation is defended
The quality of star chart picture；
Priori dictionary guidance internal image rebuilds clearer satellite image and (is originally used for image to go outside S7, building GMM
It makes an uproar).Here with the situation that can not constitute image pair between Aerial Images and satellite image, the life that can not directly propose before use
It is trained at confrontation network model, clear Aerial Images can be introduced indirectly using the mode of priori dictionary outside building GMM
In the satellite image that is generated to super-resolution of abundant details in；Priori dictionary outside GMM is constructed, will clearly take photo by plane 17 grades of images
It is divided into the fritter of 15*15, preliminary grouping (according to Euclidean distance) is carried out after block, as shown in Figure 6；
S701, GMM model is constructed according to the image block of grouping, SVD decomposition is carried out to covariance matrix in obtained model,
Dictionary is constructed, the reconstruction of satellite image below is guided as external priori；
S702, the I to be exported in the super-resolution model of frontSR_GHIt is inputted as internal image, piecemeal (15*15) after input,
It is clustered using GMM model guidance piecemeal when constructing external priori dictionary；
S703, the dictionary guidance internal image block constituted simultaneously using external priori again pull up internal dictionary；
S704, the inside graph block for carrying out sparse coding to internal dictionary, and combining former internal image block packet reconstruction new
Grouping.
S8, it is grouped according to the inside graph block of reconstruction, rebuilds satellite mapping, the behaviour of image sharpening is carried out to the satellite mapping of reconstruction
Make, so that the edge in image, which is more clear, obtains final result figure.
The present invention is real in the case where there is certain condition limited case by losing and generating confrontation network integration multiple dimensioned perception
The super-resolution of existing satellite image.Wherein, it using satellite image training one network for the purpose of denoising, is instructed using Aerial Images
Practice the network of a realization Image Super-resolution, and it is first that the feature that gauss hybrid models extract in clearly Aerial Images is used in combination
It tests, the image after Super-resolution Reconstruction is further rebuild.A gaussian filtering is again passed by carry out the edge in image
Edge contrast finally generates relatively sharp satellite image.
The present invention solves the problems, such as that the image under restrictive condition puts resolution and image quality improvement.By using more rulers
The perceptual distortion of degree is lost, and the multiple dimensioned constraint to characteristics of image domain is generated is realized, to generate effect better image.
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.The present invention being described and shown in usually here in attached drawing is real
The component for applying example can be arranged and be designed by a variety of different configurations.Therefore, below to the present invention provided in the accompanying drawings
The detailed description of embodiment be not intended to limit the range of claimed invention, but be merely representative of of the invention selected
Embodiment.Based on the embodiments of the present invention, those of ordinary skill in the art are obtained without creative efforts
The every other embodiment obtained, shall fall within the protection scope of the present invention.
A, experiment condition
1. experiment uses database
Present invention experiment is the satellite image data provided in satellite image super-resolution project and Aerial Images using data
Data.Non-public data set only does part displaying here.Satellite image data includes:
Data type 1: the satellite image (including the apparent noise of graininess) extracted since 16 grades, readability is not
It is high；
Data type 2: the satellite image (graininess noise is unobvious) extracted since 18 grades, clarity is slightly high by one
A bit.When the satellite image extracted since 18 grades is down-sampled to 16 grades, the opposite extraction satellite image since 16 grades can be obtained
Become apparent from some satellite images.But the satellite image higher cost due to being extracted since 18 grades, generally it is difficult largely to obtain
, generally common is the satellite image extracted since 16 grades.So the realization of this project, by obtaining sub-fraction
Training image super-resolution model on the basis of the satellite image extracted from 18 grades, then realizes root by the technology of Image Super-resolution
The similar extraction satellite image even more than since 18 grades is obtained (by making according to the satellite image for the low definition extracted from 16 grades
Assisted with clearly Aerial Images), there is great research significance and value.This categorical data obtain here it is less,
But have with the overlay area in data type 1 overlapping, it is possible to constitute a small amount of image to progress model training.
Data type 3: data of clearly taking photo by plane, due to shooting height and style of shooting, relative satellite image
It is more clear.With the Aerial Images of satellite image same level, Aerial Images are clearly more, and include texture abundant
Information.But Aerial Images covering surface is limited, limited source, can not obtain and same position in data type 1 and data type 2
The Aerial Images of similar time section, there is no the images pair that satellite image and Aerial Images are constituted, and cannot be used directly to train, such as
Shown in table 1.
1 data set of table and its distribution situation
| Data type/classification | 15 grades | 16 grades | 17 grades | 18 grades | It is total |
| Data type 1 | 12989 | 51956 | Nothing | Nothing | 64945 |
| Data type 2 | 1583 | 6332 | 25328 | 101302 | 134555 |
| Data type 3 | 1689 | 7104 | 27988 | 111952 | 148733 |
2. requirement of experiment
Experiment is divided into three parts: denoising model training, Image Super-resolution model training and post processing of image are tested.
Denoising model training: it is adopted under after utilizing the satellite image (Noise) extracted from 16 grades and being extracted since 18 grades
Sample to 16 grades satellite image (not Noise, but clarity is not high) constitute image pair.As training data, training this programme
The generation of middle proposition fights network.After training is completed, using Maker model, the satellite image for inputting a width Noise can be obtained
To the satellite image of not Noise.For the robustness for guaranteeing model, test uses the satellite for being with training different cities region
Image is similarly the noisy acoustic image extracted from 16 grades.
Image Super-resolution model training: 16 grades of boats that model training is obtained using 17 grades of Aerial Images and 17 grades of down-samplings
Image is clapped, constitutes image to progress model training.After the completion of the generation confrontation network proposed in training this programme, generator is utilized
Model, corresponding 17 grades of full resolution prictures can be generated in 16 grades of satellite images of one width of input not Noise.And compare generation
The visual effect of full resolution pricture
Post processing of image experiment: to the satellite image by denoising and Image Super-resolution processing, further to promote image
Quality does post processing of image.Firstly, priori dictionary is obtained outside GMM as guiding using the training of clearly 17 grades of Aerial Images,
17 grades of satellite images that input super-resolution obtains, the internal dictionary of building, which is laid equal stress on, under the guidance of external priori builds image, in conjunction with
The satellite image of clear priori in Aerial Images.And behaviour is sharpened to image using the method for gaussian filtering on this basis
Make, obtains finally post-processing the image completed.The readability and visual effect of comparison result image and original image.
3. experiment parameter is arranged
In training denoising model and Image Super-resolution model using identical setting.It is the initial of generator first
Change training, initial learning rate is set as 0.0001, and cycle of training is that (it is one that training data is all gone over to 100 epoch
epoch).When network is integrally trained, initial learning rate is still set as 0.0001, is set as 200 epcoh cycle of training, learns
Rate decays once when reaching half cycle of training, decays to 0.00001.
In post processing of image, priori dictionary includes following parameter outside building GMM model: setting piecemeal step-length is 3, point
Block size is 15*15, and it includes 32 in GMM model that when cluster, which chooses most similar 10 image blocks of Euclidean distance as one group,
Gauss model is fitted 32 classifications.Gaussian filtering is used when image sharpening, and filter radius 1.5, sharpening intensities 2. are set
B, experimental result evaluation criterion
Since actual test input is, from 16 grades of satellite images extracted (including noise), there is no corresponding clearly 17 grades
Satellite image.It can not directly be measured using measurement standards such as general PSNR and SSIM.Here by enumerating some tests
The validity of figure comparative descriptions this programme as a result.
C, comparative test scheme
Please refer to Fig. 7 and Fig. 8, test result listed above shows scheme that this is proposed in practical situations effective
Property.The restrictive condition having in this programme background, cause general pattern super resolution algorithm can not direct training managing, need to borrow
Help a series of image processing algorithm that can achieve the desired results.Final test generates image effect in original packet Noise
On the basis of 16 grades of satellite images, noise is not removed only, also achieves super-resolution to 17 grades (i.e. dimensionally length and width * 2).And
And by take photo by plane clear (non-same place), the promotion and improvement for generating 17 grades of satellite image readabilities are realized.
The above content is merely illustrative of the invention's technical idea, and this does not limit the scope of protection of the present invention, all to press
According to technical idea proposed by the present invention, any changes made on the basis of the technical scheme each falls within claims of the present invention
Protection scope within.
Claims (10)
1. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori, which is characterized in that contained using 16 grades
Noise image corresponding 16 grades constitute image to training denoising model without noise image, then utilize data training of taking photo by plane
Image Super-resolution model；Using priori dictionary outside Aerial Images building GMM model, and guide internal unsharp satellite image
It is rebuild, completes the post-processing to the super resolution image of generation, then carry out image sharpening using the mode of gaussian filtering, most
The full resolution pricture of protosatellite image is obtained eventually, realizes that the visual quality of images in protosatellite image basis is promoted.
2. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 1,
It is characterized in that, comprising the following steps:
S1, definition generate the generator in confrontation network, and network is lost in decision device and multiple dimensioned perception；
S2, using in existing satellite data since 18 grades extract image down sampling to 16 grades, if obtain 16 grades of satellite mappings
As the target I as denoisingD_H, the satellite data extracted from 16 grades is as noisy image ID_LImage pair is constituted, if generating not
Noisy satellite image is ID_GH；
S3, the image pair to be constituted in step S2, carry out initialization training to the generator in denoising model, train in initialization
In, using mean square error as loss function, calculate the equal of the pixel between the corresponding target image of image that generator generates
Square error obtains MSE generator loss function lossMSE, calculate gradient and return adjustment model parameter；
S4, after the initialization of 100 epoch training, carry out complete model training, calculate loss and corresponding gradient simultaneously
Network VGG19 not adjusting parameter is lost in parameter model in passback adjustment generator and decision device, perception；
S5, reach convergence, preservation model according to 200 epoch of arrangement above training, trained generator makes for denoising
With the image after the denoising of acquisition is ID_GHAs the input of Image Super-resolution, satellite image super-resolution model is defined；
S6, step S3~S5 is repeated, completes super-resolution network training process and denoising model, then generates super resolution image
ISR_GH, external priori dictionary is constructed using gauss hybrid models；
S7, building GMM outside priori dictionary, 17 grades of images of clearly taking photo by plane are divided into the fritter of 15*15, then according to it is European away from
From being tentatively grouped；
S8, the inside graph block packet reconstruction satellite mapping according to reconstruction carry out image sharpening operation to the satellite mapping of reconstruction, obtain
Final result figure.
3. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
It is characterized in that, in step S1, generates the generator in confrontation network is defined as: use a residual error network as generator, it is residual
Include 16 residual error modules in poor network, includes three convolutional layers in each residual error module；
Decision device structure is defined as: use one 10 layers of convolutional neural networks as decision device, the convolution of convolutional neural networks
Layer uses empty convolution；
Multiple dimensioned perception loss is defined as: use the VGG19 net that pre-training is crossed on IMAGENET1000 class taxonomy database
Network perceptually loses network, by using conv2_2, conv3_4, conv4_4, Analysis On Multi-scale Features figure in multilayer, and building
Multiple dimensioned perception loss.
4. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
It is characterized in that, in step S3, MSE generator loss function lossMSEIt is as follows:
lossMSE=MSE (ID_GH,ID_H)。
5. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
It is characterized in that, in step S4, when model training, by the MSE generator loss function loss in generator loss functionMSE, perception
Loss function lossvggWith confrontation loss function lossGANGenerator loss function after weighting summation when the training of composition entirety is such as
Under:
lossG=lossMSE+lossvgg+lossGAN。
6. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 5,
It is characterized in that, perception loss lossvggIt is as follows:
lossvgg=10-6×(lossmse_conv2_2+lossmse_conv3_4+lossmse_conv4_4)
lossmse_conv2_2=MSE (fi_conv2_2,ft_conv2_2)
lossmse_conv3_4=MSE (fi_conv3_4,ft_conv3_4)
lossmse_conv4_4=MSE (fi_conv4_4,ft_conv4_4)
Wherein, fi_conv2_2, fi_conv3_4, fi_conv4_4Image is generated for input, and conv2_2, conv3_ are corresponded into sensor model
4, conv4_4 layers of characteristic pattern, ft_conv2_3, ft_conv3_3, ft_conv4_3It is corresponded in target image input sensor model to generate image
Obtained correspondence conv2_2, conv3_4, conv4_4 layer characteristic pattern；
Fight loss function lossGANIt is as follows:
lossGAN=10-4×cross_entropy(ID_GH,True)
cross_entropy(ID_GH, True) and=log (D (ID_GH))
Wherein, D () is decision device.
7. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
It is characterized in that, in step S4, decision device loss function loss when entirety is trainedDIs defined as:
lossD=loss1+loss2
loss1=sigmoid_cross_entropy (ID_GH,False)
loss2=sigmoid_cross_entropy (ID_H,True)。
8. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
Be characterized in that, in step S5, super-resolution model includes generator, sensor model and decision device, sensor model and decision device with go
Structure used in model of making an uproar is identical, and the generator defined in Image Super-resolution model is as follows:
By building residual error module, then multiple residual error modules fold composition network structure main body, pass through the realization pair of sub-pix convolutional layer
The amplification of image uses.
9. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 8,
It is characterized in that, the data that the generator training of super-resolution model uses are data of taking photo by plane, and are inputted as ISR_L16 grades of boats of low explanation
17 grades of figure I that take photo by plane of bat figure high-resolution corresponding with itsSR_HThe image pair of composition, generator output are ISR_GH, define generator
Loss function is as follows:
lossMSE_SR=MSE (ISR_GH,ISR_H)。
10. a kind of satellite image ultra-resolution method for fighting network integration Aerial Images priori according to claim 2,
It is characterized in that, step S7 is specific as follows:
S701, GMM model is constructed according to the image block of grouping, SVD decomposition, building is carried out to covariance matrix in obtained model
Dictionary guides the reconstruction of satellite image below as external priori；
S702, the I to be exported in the super-resolution model of frontSR_GHIt is inputted as internal image, 15*15 piecemeal is pressed after input, utilized
GMM model guidance piecemeal when constructing external priori dictionary is clustered；
S703, the dictionary guidance internal image block constituted using external priori again pull up internal dictionary；
S704, the inside graph block grouping for carrying out sparse coding to internal dictionary, and combining former internal image block packet reconstruction new.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810777731.5A CN109035142B (en) | 2018-07-16 | 2018-07-16 | Satellite image super-resolution method combining countermeasure network with aerial image prior |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810777731.5A CN109035142B (en) | 2018-07-16 | 2018-07-16 | Satellite image super-resolution method combining countermeasure network with aerial image prior |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109035142A true CN109035142A (en) | 2018-12-18 |
| CN109035142B CN109035142B (en) | 2020-06-19 |
Family
ID=64642502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810777731.5A Active CN109035142B (en) | 2018-07-16 | 2018-07-16 | Satellite image super-resolution method combining countermeasure network with aerial image prior |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109035142B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886875A (en) * | 2019-01-31 | 2019-06-14 | 深圳市商汤科技有限公司 | Image super-resolution rebuilding method and device, storage medium |
| CN110009568A (en) * | 2019-04-10 | 2019-07-12 | 大连民族大学 | The generator construction method of language of the Manchus image super-resolution rebuilding |
| CN110070505A (en) * | 2019-04-12 | 2019-07-30 | 北京迈格威科技有限公司 | Enhance the method and apparatus of image classification plant noise robustness |
| CN110119780A (en) * | 2019-05-10 | 2019-08-13 | 西北工业大学 | Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network |
| CN110120024A (en) * | 2019-05-20 | 2019-08-13 | 百度在线网络技术（北京）有限公司 | Method, apparatus, equipment and the storage medium of image procossing |
| CN110807762A (en) * | 2019-09-19 | 2020-02-18 | 温州大学 | Intelligent retinal blood vessel image segmentation method based on GAN |
| CN111209854A (en) * | 2020-01-06 | 2020-05-29 | 苏州科达科技股份有限公司 | Method and device for recognizing unbelted driver and passenger and storage medium |
| CN111351502A (en) * | 2018-12-21 | 2020-06-30 | 赫尔环球有限公司 | Method, apparatus and computer program product for generating an overhead view of an environment from a perspective view |
| CN112270654A (en) * | 2020-11-02 | 2021-01-26 | 浙江理工大学 | Image denoising method based on multi-channel GAN |
| WO2021052261A1 (en) * | 2019-09-17 | 2021-03-25 | 中国科学院空天信息创新研究院 | Image super-resolution reconstruction method and apparatus for sharpening of label data |
| CN113361508A (en) * | 2021-08-11 | 2021-09-07 | 四川省人工智能研究院(宜宾) | Cross-view-angle geographic positioning method based on unmanned aerial vehicle-satellite |
| CN113535996A (en) * | 2021-05-27 | 2021-10-22 | 中国人民解放***箭军工程大学 | Road image data set preparation method and device based on aerial image |
| US11257185B2 (en) * | 2019-09-17 | 2022-02-22 | Maxar International Sweden Ab | Resolution enhancement of aerial images or satellite images |
| CN112686801B (en) * | 2021-01-05 | 2023-06-20 | 金陵科技学院 | Water quality monitoring method based on aerial image and serial echo state network |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590296A (en) * | 2015-12-07 | 2016-05-18 | 天津大学 | Dual-dictionary learning-based single-frame image super-resolution reconstruction method |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN107977932A (en) * | 2017-12-28 | 2018-05-01 | 北京工业大学 | It is a kind of based on can differentiate attribute constraint generation confrontation network face image super-resolution reconstruction method |
| CN108171656A (en) * | 2018-01-12 | 2018-06-15 | 西安电子科技大学 | Adaptive Global Dictionary remote sensing images ultra-resolution method based on rarefaction representation |
-
2018
- 2018-07-16 CN CN201810777731.5A patent/CN109035142B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105590296A (en) * | 2015-12-07 | 2016-05-18 | 天津大学 | Dual-dictionary learning-based single-frame image super-resolution reconstruction method |
| US20180075581A1 (en) * | 2016-09-15 | 2018-03-15 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN107154023A (en) * | 2017-05-17 | 2017-09-12 | 电子科技大学 | Face super-resolution reconstruction method based on generation confrontation network and sub-pix convolution |
| CN107977932A (en) * | 2017-12-28 | 2018-05-01 | 北京工业大学 | It is a kind of based on can differentiate attribute constraint generation confrontation network face image super-resolution reconstruction method |
| CN108171656A (en) * | 2018-01-12 | 2018-06-15 | 西安电子科技大学 | Adaptive Global Dictionary remote sensing images ultra-resolution method based on rarefaction representation |
Non-Patent Citations (3)
| Title |
|---|
| LIUJUAN CAO 等: "Vehicle detection from highway satellite images via transfer learning", 《INFORMATION SCIENCES》 * |
| ZHOU FUQIANG 等: "High-frequency details enhancing DenseNet for super-resolution", 《NEUROCOMPUTING》 * |
| 张宗祥: "基于地物类别的高光谱图像超分辨率复原算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11720992B2 (en) | 2018-12-21 | 2023-08-08 | Here Global B.V. | Method, apparatus, and computer program product for generating an overhead view of an environment from a perspective image |
| CN111351502A (en) * | 2018-12-21 | 2020-06-30 | 赫尔环球有限公司 | Method, apparatus and computer program product for generating an overhead view of an environment from a perspective view |
| CN109886875B (en) * | 2019-01-31 | 2023-03-31 | 深圳市商汤科技有限公司 | Image super-resolution reconstruction method and device and storage medium |
| CN109886875A (en) * | 2019-01-31 | 2019-06-14 | 深圳市商汤科技有限公司 | Image super-resolution rebuilding method and device, storage medium |
| CN110009568A (en) * | 2019-04-10 | 2019-07-12 | 大连民族大学 | The generator construction method of language of the Manchus image super-resolution rebuilding |
| CN110070505A (en) * | 2019-04-12 | 2019-07-30 | 北京迈格威科技有限公司 | Enhance the method and apparatus of image classification plant noise robustness |
| CN110119780A (en) * | 2019-05-10 | 2019-08-13 | 西北工业大学 | Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network |
| CN110119780B (en) * | 2019-05-10 | 2020-11-27 | 西北工业大学 | Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network |
| CN110120024B (en) * | 2019-05-20 | 2021-08-17 | 百度在线网络技术（北京）有限公司 | Image processing method, device, equipment and storage medium |
| CN110120024A (en) * | 2019-05-20 | 2019-08-13 | 百度在线网络技术（北京）有限公司 | Method, apparatus, equipment and the storage medium of image procossing |
| US11645735B2 (en) | 2019-05-20 | 2023-05-09 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for processing image, device and computer readable storage medium |
| US11257185B2 (en) * | 2019-09-17 | 2022-02-22 | Maxar International Sweden Ab | Resolution enhancement of aerial images or satellite images |
| WO2021052261A1 (en) * | 2019-09-17 | 2021-03-25 | 中国科学院空天信息创新研究院 | Image super-resolution reconstruction method and apparatus for sharpening of label data |
| CN110807762B (en) * | 2019-09-19 | 2021-07-06 | 温州大学 | Intelligent retinal blood vessel image segmentation method based on GAN |
| CN110807762A (en) * | 2019-09-19 | 2020-02-18 | 温州大学 | Intelligent retinal blood vessel image segmentation method based on GAN |
| CN111209854A (en) * | 2020-01-06 | 2020-05-29 | 苏州科达科技股份有限公司 | Method and device for recognizing unbelted driver and passenger and storage medium |
| CN112270654A (en) * | 2020-11-02 | 2021-01-26 | 浙江理工大学 | Image denoising method based on multi-channel GAN |
| CN112686801B (en) * | 2021-01-05 | 2023-06-20 | 金陵科技学院 | Water quality monitoring method based on aerial image and serial echo state network |
| CN113535996A (en) * | 2021-05-27 | 2021-10-22 | 中国人民解放***箭军工程大学 | Road image data set preparation method and device based on aerial image |
| CN113535996B (en) * | 2021-05-27 | 2023-08-04 | 中国人民解放***箭军工程大学 | Road image dataset preparation method and device based on aerial image |
| CN113361508A (en) * | 2021-08-11 | 2021-09-07 | 四川省人工智能研究院(宜宾) | Cross-view-angle geographic positioning method based on unmanned aerial vehicle-satellite |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109035142B (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035142A (en) | A kind of satellite image ultra-resolution method fighting network integration Aerial Images priori | |
| CN111784602B (en) | Method for generating countermeasure network for image restoration | |
| CN108460726A (en) | A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network | |
| CN112184554B (en) | Remote sensing image fusion method based on residual mixed expansion convolution | |
| CN110120020A (en) | A kind of SAR image denoising method based on multiple dimensioned empty residual error attention network | |
| CN110119780A (en) | Based on the hyperspectral image super-resolution reconstruction method for generating confrontation network | |
| CN108830913B (en) | Semantic level line draft coloring method based on user color guidance | |
| CN108416755A (en) | A kind of image de-noising method and system based on deep learning | |
| CN109087375B (en) | Deep learning-based image cavity filling method | |
| CN112819737B (en) | Remote sensing image fusion method of multi-scale attention depth convolution network based on 3D convolution | |
| CN112465827A (en) | Contour perception multi-organ segmentation network construction method based on class-by-class convolution operation | |
| CN113012172A (en) | AS-UNet-based medical image segmentation method and system | |
| CN109214989A (en) | Single image super resolution ratio reconstruction method based on Orientation Features prediction priori | |
| CN109410149A (en) | A kind of CNN denoising method extracted based on Concurrent Feature | |
| CN112836602B (en) | Behavior recognition method, device, equipment and medium based on space-time feature fusion | |
| CN108416397A (en) | A kind of Image emotional semantic classification method based on ResNet-GCN networks | |
| CN110415253A (en) | A kind of point Interactive medical image dividing method based on deep neural network | |
| CN111814891A (en) | Medical image synthesis method, device and storage medium | |
| Xu et al. | Infrared and visible image fusion using a deep unsupervised framework with perceptual loss | |
| CN116739899A (en) | Image super-resolution reconstruction method based on SAUGAN network | |
| He et al. | VLMAE: Vision-language masked autoencoder | |
| Zhang et al. | Petsgan: Rethinking priors for single image generation | |
| CN115018711A (en) | Image super-resolution reconstruction method for warehouse scheduling | |
| CN109559278A (en) | Super resolution image reconstruction method and system based on multiple features study | |
| CN114972619A (en) | Single-image face three-dimensional reconstruction method based on self-alignment double regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |