CN113487476A - Online-updating image blind super-resolution reconstruction method and device - Google Patents
Online-updating image blind super-resolution reconstruction method and device Download PDFInfo
- Publication number
- CN113487476A CN113487476A CN202110558894.6A CN202110558894A CN113487476A CN 113487476 A CN113487476 A CN 113487476A CN 202110558894 A CN202110558894 A CN 202110558894A CN 113487476 A CN113487476 A CN 113487476A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- super
- reconstructed
- resolution reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 230000015556 catabolic process Effects 0.000 claims abstract description 140
- 238000006731 degradation reaction Methods 0.000 claims abstract description 140
- 230000000007 visual effect Effects 0.000 claims abstract description 16
- 230000006870 function Effects 0.000 claims description 43
- 238000005070 sampling Methods 0.000 claims description 20
- 125000004122 cyclic group Chemical group 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 8
- 238000012360 testing method Methods 0.000 abstract description 20
- 230000000694 effects Effects 0.000 abstract description 16
- 238000010586 diagram Methods 0.000 description 9
- 238000005457 optimization Methods 0.000 description 9
- 238000009826 distribution Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 5
- 238000000605 extraction Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000000052 comparative effect Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003892 spreading Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4076—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution using the original low-resolution images to iteratively correct the high-resolution images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides an online updated image blind super-resolution reconstruction method and device, wherein the method comprises the following steps: initializing a degradation estimation module and a super-resolution reconstruction module in a learning network; inputting the images to be reconstructed into a super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images; determining a super-resolution reconstructed image of an image to be reconstructed based on the visual effects of the plurality of candidate super-resolution reconstructed images; in each learning period, the degradation estimation module and the super-resolution reconstruction module are optimized alternately to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode. The method does not depend on a low-resolution-high-resolution sample pair, can optimize the model parameters according to the degradation modes of different low-resolution test pictures, and obtains the model specific to the degradation mode of the test pictures, thereby performing targeted super-resolution reconstruction on the image to be reconstructed, and improving the super-resolution reconstruction effect and robustness.
Description
Technical Field
The invention relates to the technical field of digital image processing, in particular to an online updated image blind super-resolution reconstruction method and device.
Background
Images typically contain a large amount of visual information with intuitive and efficient description capabilities. With the rapid development of information technology, image applications are gradually spreading in a plurality of fields such as medicine, remote sensing, astronomy, security monitoring and the like. The image resolution is an important parameter reflecting the richness of detailed information contained in the image, and the capability of an imaging system for actually reflecting object details is reflected. High resolution images contain greater pixel density, richer texture details, and higher confidence than low resolution images. In actual life, however, factors such as hardware conditions of imaging devices, weather environments, network transmission media, bandwidth and the like cause imaging blur, and image quality is reduced. Aiming at the problem, the method for optimizing the image quality through the super-resolution reconstruction algorithm is low in cost and easy to implement. Specifically, the image super-resolution reconstruction technique reconstructs a corresponding high-resolution image from a known low-resolution image through a specific algorithm using a digital image processing technique.
With the wide application of the deep learning in the field of image processing, the super-resolution reconstruction method based on the deep learning has obvious effect improvement compared with the traditional interpolation-based method and the statistical-based method. Currently, most deep learning based approaches use a large number of low-resolution-high-resolution image pairs to supervised train the network model so that the trained model can perform super-resolution reconstruction from the input data. However, these methods are not image-specific and have two significant drawbacks: (1) the methods depend on external data seriously, the model weight is completely determined by external synthetic data, and the characteristics of the test image are not considered; (2) the model weights of these methods are fixed when the test picture is hyper-differentially reconstructed, which means that the same model with the same network weights is used for the hyper-differential reconstruction regardless of the degradation mode of the test picture. However, in the real situation, the degradation mode of the low resolution test picture to be reconstructed is unknown and various, and the above method is difficult to be applied to various degradation situations, and the effect is poor.
Disclosure of Invention
The invention provides an online updated image blind super-resolution reconstruction method and device, which are used for solving the defect of poor super-resolution reconstruction effect in the prior art.
The invention provides an online updated image blind super-resolution reconstruction method, which comprises the following steps:
initializing a degradation estimation module and a super-resolution reconstruction module in a learning network;
inputting the images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images;
determining a super-resolution reconstructed image of the image to be reconstructed based on the visual effects of the candidate super-resolution reconstructed images;
and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
According to the online updated image blind super-resolution reconstruction method provided by the invention, in any learning period, the degradation estimation module is optimized based on the following mode:
based on the super-resolution reconstruction module, the image to be reconstructed is subjected to up-sampling to obtain a first super-resolution image;
based on the degradation estimation module, down-sampling the first super-resolution image to obtain a first low-resolution image in a resolution spatial domain of the image to be reconstructed;
and fixing the hyper-resolution reconstruction module as a known condition parameter, and updating the parameter of the degradation estimation module based on the image to be reconstructed and the first low-resolution image.
According to the online updated image blind super-resolution reconstruction method provided by the invention, in any learning period, the super-resolution reconstruction module is optimized based on the following mode:
based on the degradation estimation module, down-sampling the sample high-resolution image to obtain a second low-resolution image;
based on the super-resolution reconstruction module, the second low-resolution image is subjected to up-sampling to obtain a second super-resolution image in a resolution spatial domain of the sample high-resolution image;
and fixing the degradation estimation module as a known condition parameter, and updating the parameter of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image.
According to the online updated image blind super-resolution reconstruction method provided by the invention, the updating of the parameters of the degradation estimation module based on the image to be reconstructed and the first low-resolution image specifically comprises the following steps:
updating the parameters of the degradation estimation module based on the discrimination results of the image to be reconstructed, the first low-resolution image and the second low-resolution image and the discrimination result of the image to be reconstructed;
the judgment result of the image to be reconstructed and the judgment result of the second low-resolution image are obtained by respectively judging the resolutions of the image to be reconstructed and the second low-resolution image based on a low-resolution discriminator of the learning network.
According to the online updated image blind super-resolution reconstruction method provided by the invention, the loss function when the degradation estimation module is optimized
Comprises the following steps:
wherein L isIBAs a function of internal cyclic losses, LGAN，LFor the generation of the penalty function for the low resolution discriminator, λ is the weight, GdTo the degradation estimation module, DdThe low resolution discriminator;
y is the image to be reconstructed, Gr(y) is the first super-resolution image, Gd(Gr(y)) is the first low resolution image,
refers to the mathematical expectation of the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dd(y) is the result of the discrimination of the image to be reconstructed, xeFor high resolution images of the sample, Gd(xe) For said second low resolution image, Dd(Gd(xe) Is the discrimination result of the second low-resolution image.
According to the online updated image blind super-resolution reconstruction method provided by the invention, the updating of the parameters of the super-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image specifically comprises the following steps:
updating parameters of the hyper-resolution reconstruction module based on the second super-resolution image, the sample high-resolution image, the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image;
the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image are obtained by respectively judging the resolution of the sample high-resolution image and the resolution of the first super-resolution image based on a high-resolution discriminator of the learning network.
According to the invention, the online telephone is providedNovel image blind super-resolution reconstruction method, loss function when optimizing super-resolution reconstruction module
Comprises the following steps:
wherein L isEBAs a function of external cyclic losses, LGAN，HFor the generation of the high-resolution discriminator a penalty function is determined, λ being the weight, GrTo said hyper-resolution reconstruction module, DrIs the high resolution discriminator;
xefor high resolution images of the sample, Gd(xe) For said second low resolution image, Gr(Gd(xe) Is the second super-resolution image is obtained,
refers to the mathematical expectation of the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dr(xe) Is the discrimination result of the sample high-resolution image, y is the image to be reconstructed, Gr(y) is the first super-resolution image, Dr(Gr(y)) is a discrimination result of the first super-resolution image.
The invention also provides an online updated image blind super-resolution reconstruction device, which comprises:
the initialization unit is used for initializing a degradation estimation module and a hyper-resolution reconstruction module in the learning network;
the super-resolution reconstruction unit is used for inputting images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images;
the reconstructed image confirming unit is used for confirming the super-resolution reconstructed image of the image to be reconstructed based on the visual effect of the candidate super-resolution reconstructed images;
and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the online updated image blind super-resolution reconstruction method as described in any of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the online updated image blind super-resolution reconstruction method as set forth in any of the above.
According to the on-line updating image blind super-resolution reconstruction method and device, the degradation mode of the image to be reconstructed is learned by alternately optimizing the degradation estimation module and the super-resolution reconstruction module in each learning period, how to perform super-resolution reconstruction based on the degradation mode is learned, the low-resolution-high-resolution sample pair is not depended on, model parameter optimization can be performed according to the degradation modes of different low-resolution test pictures, a model specific to the degradation mode of the test picture is obtained, the image to be reconstructed is subjected to targeted super-resolution reconstruction, and the super-resolution reconstruction effect and robustness are improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of an online-updated image blind super-resolution reconstruction method according to the present invention;
FIG. 2 is a schematic structural diagram of a degradation estimation module according to the present invention;
FIG. 3 is a schematic structural diagram of a hyper-resolution reconstruction module provided in the present invention;
FIG. 4 is a second flowchart of the online-updated image blind super-resolution reconstruction method according to the present invention;
FIG. 5 is a third schematic flow chart of the online updated image blind super-resolution reconstruction method provided by the present invention;
FIG. 6 is a comparative graph of the hyper-resolution reconstruction effect with the hyper-resolution magnification of 2 according to the present invention;
FIG. 7 is a comparative graph of the hyper-resolution reconstruction effect with the hyper-resolution magnification of 4 according to the present invention;
FIG. 8 is a schematic structural diagram of an online-updated image blind super-resolution reconstruction apparatus provided in the present invention;
fig. 9 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow diagram of an online-updated image blind super-resolution reconstruction method according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
in each learning period, the degradation estimation module and the super-resolution reconstruction module are optimized alternately to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
In particular, the learning network may be a learnable network based on a deep neural network. The learning network comprises a degradation estimation module GdAnd a super-resolution reconstruction module Gr. In the learning process of the learning network, the degradation estimation module can learn the degradation mode of the specific image, and the super-resolution reconstruction module can learn how to perform targeted super-resolution reconstruction on the specific image according to the learned degradation mode.
Here, the degradation estimation module GdCan be constructed based on convolutional layers and downsampled layers. Fig. 2 is a schematic structural diagram of a degradation estimation module according to an embodiment of the present invention, and as shown in fig. 2, the degradation estimation module GdIt can be composed of 3 convolutional layers with different sizes and 1 downsampling layer. As an example, consider that the larger the downsampling multiple, the more severe the image degradation. Therefore, in practical implementation, for x 2 times of degradation estimation, the size of the 3-layer convolutional layer can be set to 3 × 3, 7 × 7, 9 × 9 in sequence; for the x 4 times degradation estimation, the sizes of the 3 convolutional layers may be 9 × 9, 15 × 15, and 17 × 17 in sequence, and the specific size of the convolutional layer may be set according to an actual application scenario, which is not specifically limited in this embodiment of the present invention.
Super-resolution reconstruction module GrThe structure of the model can be any existing super-resolution reconstruction model trained end-to-end, and the embodiment of the invention also does not specifically limit the modelAnd (4) determining. As an example, the super-divide reconstruction module may employ RRDBNet. Fig. 3 is a schematic structural diagram of a hyper-resolution reconstruction module according to an embodiment of the present invention, and as shown in fig. 3, the RRDBNet includes a shallow feature extraction layer, a cascaded hierarchical feature extraction block, and an upsampling module. Wherein, the layer feature extraction Block is composed of 23 Residual error Dense connecting blocks (RRDB). In addition, the super-resolution reconstruction module can also adopt network structures such as EDSR, RDN, RCAN and the like.
The above two modules may be initialized before optimizing the degradation estimation module and the super-resolution reconstruction module. For example, the degradation estimation block G may be estimated using an isotropic two-dimensional gaussian function with a mean of 0 and a variance of 1dInitializing each of the convolution layers. The isotropic two-dimensional gaussian function is expressed as follows:
where, (x, y) is a coordinate point, and σ is a variance.
In addition, DF2K data set synthesized by a bicubic downsampling method can be used for the super-resolution reconstruction module GrPerforming end-to-end pre-training, and performing over-point reconstruction on the module G by using the trained weightrInitialization is performed.
After initialization, the degradation estimation module G can be periodically traineddAnd a super-resolution reconstruction module Gr. And in each learning period, alternately optimizing the degradation estimation module and the over-resolution reconstruction module, namely fixing the parameters of one module and only training and updating the parameters of the other module. Through the optimization and degradation estimation module, the degradation mode of the image to be reconstructed, which needs to be subjected to super-resolution reconstruction, can be learned, and then through the optimization and super-resolution reconstruction module, how to perform the super-resolution reconstruction based on the specific degradation mode learned before is learned. The alternate optimization mode can enable the degradation estimation module and the hyper-resolution reconstruction module to be better optimized in a cooperative mode, and the training effect is improved.
The parameters of the two modules can be fixed every other learning period, and the image to be reconstructed is input to the super-resolution reconstruction module for super-resolution reconstruction, so that a plurality of candidate super-resolution reconstruction images are obtained. When the optimization termination condition is reached, an image with the best visual effect can be selected from the multiple candidate hyper-resolution reconstructed images to serve as a hyper-resolution reconstructed image of the image to be reconstructed.
According to the method provided by the embodiment of the invention, the degradation mode of the image to be reconstructed is learned by alternately optimizing the degradation estimation module and the super-resolution reconstruction module in each learning period, how to perform super-resolution reconstruction based on the degradation mode is learned, the method does not depend on a low-resolution-high-resolution sample pair, model parameter optimization can be performed according to the degradation modes of different low-resolution test pictures, a model specific to the degradation mode of the test picture is obtained, and therefore, the image to be reconstructed is subjected to targeted super-resolution reconstruction, and the super-resolution reconstruction effect and robustness are improved.
Based on the above embodiment, in any learning period, the degradation estimation module is optimized based on the following ways:
based on a super-resolution reconstruction module, an image to be reconstructed is sampled to obtain a first super-resolution image;
based on a degradation estimation module, down-sampling the first super-resolution image to obtain a first low-resolution image in a resolution spatial domain of the image to be reconstructed;
and fixing the hyper-resolution reconstruction module as a known condition parameter, and updating the parameter of the degradation estimation module based on the image to be reconstructed and the first low-resolution image.
Specifically, the optimization process of the degradation estimation module is an internal learning branch, the low-resolution image to be reconstructed can be used as input to learn the degradation mode of the image to be reconstructed, and the degradation estimation module G is optimizeddThe parameter (c) of (c).
In particular, the image y to be reconstructed may be passed through a hyper-resolution reconstruction module GrUp-sampling to obtain a first super-resolution image Gr(y)；
The first super-resolution image Gr(y) passing through a degradation estimation module GdDown-sampling to obtain a first low-resolution image Gd(Gr(y))Returning to the resolution spatial domain of the image to be reconstructed;
subsequently, based on the image to be reconstructed and the first low resolution image, the degradation estimation module G is optimizeddThe parameter (c) of (c). Will exceed and rebuild the module GrAs the known condition parameter is fixed, only G is updateddTo learn the degradation mode of the image y to be reconstructed. The objective function of the optimization degradation estimation module may be as follows:
wherein L isIBIs an internal cyclic loss function that can employ the L1 loss function to minimize the difference between the first low resolution image and the image to be reconstructed; n is the number of image blocks used to optimize the model parameters.
Based on any embodiment, in any learning period, the hyper-resolution reconstruction module is optimized based on the following mode:
based on a degradation estimation module, down-sampling the sample high-resolution image to obtain a second low-resolution image;
based on a super-resolution reconstruction module, the second low-resolution image is up-sampled to obtain a second super-resolution image in a resolution spatial domain of the sample high-resolution image;
and fixing the degradation estimation module as a known condition parameter, and updating the parameter of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image.
Specifically, the optimization process of the hyper-resolution reconstruction module is an external learning branch, and a sample high-resolution image in an external data set can be used as input through the degradation estimation module GdConstructing corresponding low-resolution-high-resolution sample pairs by the learned degradation mode so as to optimize the hyper-resolution reconstruction module GrThe parameter (c) of (c).
In particular, the sample high resolution images xe in the external data set may be randomly selected to pass through the degradation estimation module GdDown-sampling to obtain a second low-resolution image Gd(xe)；
The second low resolution image Gd(xe) passing through a super-resolution reconstruction block GrUp-sampling to obtain a second super-resolution image Gr(Gd(xe)), back into the resolution spatial domain of the sample high resolution image;
subsequently, the super-resolution reconstruction module G is optimized based on the second super-resolution image and the sample high-resolution imagerThe parameter (c) of (c). Will degrade the estimation module GdAs the known condition parameter is fixed, only G is updatedrTo learn how to perform super-resolution reconstruction based on the learned degradation. The objective function of the optimized hyper-divided reconstruction module may be as follows:
wherein L isEBIs an external cyclic loss function that can employ the L1 loss function to minimize the difference between the second super-resolved image and the sample high-resolution image.
Based on any of the above embodiments, updating the parameters of the degradation estimation module based on the image to be reconstructed and the first low-resolution image specifically includes:
updating parameters of the degradation estimation module based on the discrimination results of the image to be reconstructed, the first low-resolution image and the second low-resolution image and the discrimination result of the image to be reconstructed;
the judgment result of the image to be reconstructed and the judgment result of the second low-resolution image are obtained by respectively judging the resolutions of the image to be reconstructed and the second low-resolution image based on a low-resolution discriminator of a learning network.
Specifically, in order to better recover the texture details in the image and improve the visual effect of the reconstructed image, a low resolution discriminator may be introduced for counterlearning, so that the second low resolution image G is useddThe data distribution characteristic of (xe) is closer to the image to be reconstructed, that is, the down-sampling result of the degradation estimation module is closer to the data distribution of the real low-resolution space. Thus, a low resolution discriminator based on a learning network can be usedAnd respectively judging the resolution of the image to be reconstructed and the second low-resolution image to obtain a judgment result of the image to be reconstructed and a judgment result of the second low-resolution image, so as to perform counterstudy. And updating the parameters of the degradation estimation module by combining the image to be reconstructed and the first low-resolution image. The objective function of the counterlearning may be as follows:
wherein L isGAN，LIs the generation penalty function of the low resolution arbiter.
Based on any one of the above embodiments, the loss function when optimizing the degradation estimation module
Comprises the following steps:
wherein L isIBAs a function of internal cyclic losses, LGAN，LFor the generation of the penalty function for the low resolution discriminators, λ is the weight, GdFor a degradation estimation module, DdA low resolution discriminator;
y is the image to be reconstructed, Gr(y) is the first super-resolution image, Gd(Gr(y)) is the first low resolution image,
refers to the mathematical expectation that the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dd(y) is the result of discrimination of the image to be reconstructed, xeFor high resolution images of the sample, Gd(xe) For the second low-resolution image, Dd(Gd(xe) ) is the discrimination result of the second low-resolution image.
Based on any of the above embodiments, updating the parameters of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image specifically includes:
updating parameters of a hyper-resolution reconstruction module based on the second super-resolution image, the sample high-resolution image, the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image;
the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image are obtained by respectively judging the resolution of the sample high-resolution image and the first super-resolution image based on a high-resolution discriminator of a learning network.
Specifically, in order to better recover the texture details in the image and improve the visual effect of the reconstructed image, a high-resolution discriminator may be introduced for counterstudy, so that the first super-resolution image G is obtainedrThe data distribution characteristic of (y) is closer to the sample high-resolution image, namely the up-sampling result of the super-resolution reconstruction module is closer to the data distribution of the real high-resolution space. Therefore, counterstudy can be performed by performing resolution discrimination on the sample high-resolution image and the first super-resolution image respectively based on the high-resolution discriminator of the learning network to obtain a discrimination result of the sample high-resolution image and a discrimination result of the first super-resolution image. And updating parameters of the super-resolution reconstruction module by combining the second super-resolution image and the sample high-resolution image. The objective function of the counterlearning may be as follows:
wherein L isGAN，HIs the generation penalty function of the high resolution arbiter.
Based on any one of the embodiments, the loss function during the module hyper-resolution reconstruction is optimized
Comprises the following steps:
wherein L isEBAs a function of external cyclic losses, LGAN，HFor the generation of the penalty function for the high resolution discriminator, λ is the weight, GrFor the super-resolution reconstruction module, DrA high resolution discriminator;
xefor high resolution images of the sample, Gd(xe) For the second low-resolution image, Gr(Gd(xe) Is a second super-resolution image
Refers to the mathematical expectation that the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample.
Dr(xe) Is the discrimination result of the sample high-resolution image, y is the image to be reconstructed, Gr(y) is the first super-resolution image, Dr(Gr(y)) is the discrimination result of the first super-resolution image.
Based on any of the above embodiments, fig. 4 is a second schematic flow chart of the online-updated image blind super-resolution reconstruction method according to the embodiment of the present invention, as shown in fig. 4, the method includes:
constructing a learnable network based on Deep Neural Networks (DNN); the network comprises a degradation estimation module, an over-resolution reconstruction module, a low-resolution discriminator and a high-resolution discriminator;
initializing the weight of a degradation estimation module and a super-resolution reconstruction module;
learning the degradation mode of the low-resolution image to be reconstructed in the internal learning branch, and optimizing the parameters of the degradation estimation module;
generating corresponding sample pairs by using a learned degradation mode in an external learning branch, and optimizing the model parameters of the hyper-resolution reconstruction module;
enabling the intermediate results of the internal and external learning branches to be closer to real distribution by using a discriminator through a counterstudy mode;
inputting the test pictures into a hyper-resolution reconstruction module at fixed step intervals to generate a hyper-resolution image;
and if the maximum updating step number is reached, outputting the super-resolution image with the best visual effect.
Based on any of the above embodiments, fig. 5 is a third schematic flow chart of the online-updated image blind super-resolution reconstruction method provided in the embodiment of the present invention, as shown in fig. 5, the method includes:
the method comprises the steps of automatically setting the patch size and the batch size of an input internal learning branch and an input external learning branch, wherein the input size of the input external learning branch is s times of that of the input internal learning branch, and s is an amplification factor of the over-fraction reconstruction. In the embodiment of the present invention, the super-resolution reconstruction multiple s may be 4, the size of the low-resolution image patch input by the internal learning branch may be 32 × 32, the size of the high-resolution image patch input by the external learning branch may be 128 × 128, and the size of the patch may be 10.
Randomly selecting 10 patches in the test image, and randomly selecting 1 patch in each of 10 external high-resolution images, and respectively inputting the patches into the internal learning branch and the external learning branch.
In the internal learning branch, the input y passes through the hyper-resolution reconstruction module GrObtaining a super-resolution intermediate result G after shallow feature extraction, different-level feature extraction and upsampling processingr(y) size 128 × 128. Then intermediate result Gr(y) input degradation estimation block GdObtaining low resolution output G after degradation estimation and down sampling processingd(Gr(y)), the size was 32 × 32.
In the external learning branch, the degradation estimation block GdIs determined by the degradation mode learned during the internal learning process. The input xe is processed by the quality-reducing fuzzy and down-sampling of the quality-reducing estimation module to obtain a low-resolution intermediate result Gd(xe), size 32 × 32. Then intermediate result Gd(xe) input hyper-resolution reconstruction block GrObtaining high resolution output G through super-resolution reconstructionr(Gd(xe)), the size is 128 × 128.
Generating super-resolution intermediate result G of internal learning branchr(y), and the true external high resolution image xe is input to the high resolution discriminator DrUsing a mode of counterlearning to make GrThe data distribution characteristic of (y) is closer to the distribution characteristic of a real high-resolution image.
Low resolution intermediate result G generated by external learning branchd(xe), and the true low resolution test image y is input to a low resolution discriminator DdUsing a mode of counterlearning to make GdThe data distribution characteristics of (xe) are closer to those of a real low-resolution image.
Calculating an internal cyclic loss function LIBExternal cyclic loss function LEBLow resolution generation of the penalty function LGAN，LAnd high resolution generation of the penalty function LGAN，H. Using Adam optimizer, the gradient is calculated and propagated backwards, updating the network weights. Wherein the low resolution generates the penalty function LGAN，LThe specific formula of the optimized degradation estimation module Gd is as follows:
similarly, high resolution generation resists loss function LGAN，HOptimized super-resolution reconstruction module GrThe specific formula of (A) is as follows:
using separately optimized strategies, i.e. using different pairs of penalty functions GdAnd GrRespectively optimizing, wherein the formulas are respectively as follows:
wherein, the value of the network balance parameter λ may be 0.0001.
Setting the maximum online updating step number T as 200 and the testing interval step number T as 10, alternately optimizing the degradation estimation module and the super-resolution reconstruction module, when the current step number i% T is 0, fixing the network weight, and inputting the testing image y into the super-resolution reconstruction module GrGenerating a hyper-resolution image xi. When the online updating reaches the maximum step number, a plurality of obtained hyper-resolution reconstructed images { x1，x2，…，xnAnd selecting the image with the best visual effect as a final result.
Compared with the existing method, the method provided by the embodiment of the invention has the advantages that the test data set is DIV2KRK, the data set comprises 100 test images, and the degradation modes of the images are different. The criteria for experimental effect are the average Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM).
Wherein the formula for PSNR is as follows:
wherein n represents the number of pixel bits, generally 8 is taken, MSE is the mean square error of the current image a and the reference image b, and the specific formula is as follows:
wherein, muaAnd mubRepresenting the mean, σ, of images a and baAnd σbRepresenting the variance, σ, of images a and babRepresenting the covariance of images a and b. c. C1＝(k1L)2，c2＝(k2L)2Is a constant used to maintain stability, and L is the dynamic range of pixel values, generally taken as 255, k1＝0.01，k2＝0.03。
As shown in table 1, the method provided by the embodiment of the present invention performs better on the standard test data set DIV2KRK than the existing non-blind hyper-separation and blind hyper-separation techniques. In addition, as can be seen from table 1, the performance of the method provided by the embodiment of the present invention is significantly improved by optimizing the rdbnet through online updating compared with the rdbnet trained offline, which also fully illustrates the effectiveness of online updating for solving the blind over-score problem.
TABLE 1
Fig. 6 and 7 exemplarily show the visualization effect of the method provided by the embodiment of the present invention in table 1 and other comparison methods. Wherein, FIG. 6 is a comparative graph of the hyper-resolution reconstruction effect with the hyper-resolution magnification of 2; fig. 7 is a graph comparing the effect of the super-resolution reconstruction at a magnification of 4. Compared with other methods, the method provided by the embodiment of the invention has the advantages that the reconstructed image is closer to the target high-resolution image and clearer details are recovered, the perception effect is very good, the effectiveness of the method provided by the embodiment of the invention is proved, and the reconstruction effect is better.
Based on any of the above embodiments, fig. 8 is a schematic structural diagram of an online-updating image blind super-resolution reconstruction apparatus according to an embodiment of the present invention, as shown in fig. 8, the apparatus includes: an initialization unit 810, a hyper-resolution reconstruction unit 820, and a reconstructed image confirmation unit 830.
The initialization unit 810 is configured to initialize a degradation estimation module and a super-resolution reconstruction module in the learning network;
the super-resolution reconstruction unit 820 is used for inputting the images to be reconstructed into a super-resolution reconstruction module for super-resolution reconstruction every other learning cycle to obtain a plurality of candidate super-resolution reconstruction images;
the reconstructed image confirming unit 830 is configured to determine a super-resolution reconstructed image of an image to be reconstructed based on visual effects of a plurality of candidate super-resolution reconstructed images;
in each learning period, the degradation estimation module and the super-resolution reconstruction module are optimized alternately to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
According to the device provided by the embodiment of the invention, the degradation mode of the image to be reconstructed is learned by alternately optimizing the degradation estimation module and the super-resolution reconstruction module in each learning period, how to perform super-resolution reconstruction based on the degradation mode is learned, the low-resolution-high-resolution sample pair is not relied on, model parameters can be optimized according to the degradation modes of different low-resolution test pictures, a model specific to the degradation mode of the test picture is obtained, the image to be reconstructed is subjected to targeted super-resolution reconstruction, and the super-resolution reconstruction effect and robustness are improved.
Based on any embodiment, in any learning period, the degradation estimation module is optimized based on the following modes:
based on a super-resolution reconstruction module, an image to be reconstructed is sampled to obtain a first super-resolution image;
based on a degradation estimation module, down-sampling the first super-resolution image to obtain a first low-resolution image in a resolution spatial domain of the image to be reconstructed;
and fixing the hyper-resolution reconstruction module as a known condition parameter, and updating the parameter of the degradation estimation module based on the image to be reconstructed and the first low-resolution image.
Based on any embodiment, in any learning period, the hyper-resolution reconstruction module is optimized based on the following mode:
based on a degradation estimation module, down-sampling the sample high-resolution image to obtain a second low-resolution image;
based on a super-resolution reconstruction module, the second low-resolution image is up-sampled to obtain a second super-resolution image in a resolution spatial domain of the sample high-resolution image;
and fixing the degradation estimation module as a known condition parameter, and updating the parameter of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image.
Based on any of the above embodiments, updating the parameters of the degradation estimation module based on the image to be reconstructed and the first low-resolution image specifically includes:
updating parameters of the degradation estimation module based on the discrimination results of the image to be reconstructed, the first low-resolution image and the second low-resolution image and the discrimination result of the image to be reconstructed;
the judgment result of the image to be reconstructed and the judgment result of the second low-resolution image are obtained by respectively judging the resolutions of the image to be reconstructed and the second low-resolution image based on a low-resolution discriminator of a learning network.
Based on any one of the above embodiments, the loss function when optimizing the degradation estimation module
Comprises the following steps:
wherein L isIBAs a function of internal cyclic losses, LGAN，LFor the generation of the penalty function for the low resolution discriminators, λ is the weight, GdFor a degradation estimation module, DdA low resolution discriminator;
y is the image to be reconstructed, Gr(y) is the first super-resolution image, Gd(Gr(y)) is the first low resolution image,
refers to the mathematical expectation that the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dd(y) is the result of discrimination of the image to be reconstructed, xeFor high resolution images of the sample, Gd(xe) For the second low-resolution image, Dd(Gd(xe) ) is the discrimination result of the second low-resolution image.
Based on any of the above embodiments, updating the parameters of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image specifically includes:
updating parameters of a hyper-resolution reconstruction module based on the second super-resolution image, the sample high-resolution image, the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image;
the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image are obtained by respectively judging the resolution of the sample high-resolution image and the first super-resolution image based on a high-resolution discriminator of a learning network.
Based on any one of the embodiments, the loss function during the module hyper-resolution reconstruction is optimized
Comprises the following steps:
wherein L isEBAs a function of external cyclic losses, LGAN，HFor the generation of the penalty function for the high resolution discriminator, λ is the weight, GrFor the super-resolution reconstruction module, DrA high resolution discriminator;
xefor high resolution images of the sample, Gd(xe) For the second low-resolution image, Gr(Gd(xe) Is a second super-resolution image
Refers to the mathematical expectation that the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dr(xe) Is the discrimination result of the sample high-resolution image, y is the image to be reconstructed, Gr(y) is the first super-resolution image, Dr(Gr(y)) is the discrimination result of the first super-resolution image.
Fig. 9 illustrates a physical structure diagram of an electronic device, and as shown in fig. 9, the electronic device may include: a processor (processor)910, a communication Interface (Communications Interface)920, a memory (memory)930, and a communication bus 940, wherein the processor 910, the communication Interface 920, and the memory 930 communicate with each other via the communication bus 940. Processor 910 may invoke logic instructions in memory 930 to perform an online updated image blind super-resolution reconstruction method comprising: initializing a degradation estimation module and a super-resolution reconstruction module in a learning network; inputting the images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images; determining a super-resolution reconstructed image of the image to be reconstructed based on the visual effects of the candidate super-resolution reconstructed images; and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
Furthermore, the logic instructions in the memory 930 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the online updated image blind super-resolution reconstruction method provided by the above methods, the method comprising: initializing a degradation estimation module and a super-resolution reconstruction module in a learning network; inputting the images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images; determining a super-resolution reconstructed image of the image to be reconstructed based on the visual effects of the candidate super-resolution reconstructed images; and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements the method for blind super-resolution reconstruction of an online updated image provided above, the method comprising: initializing a degradation estimation module and a super-resolution reconstruction module in a learning network; inputting the images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images; determining a super-resolution reconstructed image of the image to be reconstructed based on the visual effects of the candidate super-resolution reconstructed images; and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An online updated image blind super-resolution reconstruction method is characterized by comprising the following steps:
initializing a degradation estimation module and a super-resolution reconstruction module in a learning network;
inputting the images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images;
determining a super-resolution reconstructed image of the image to be reconstructed based on the visual effects of the candidate super-resolution reconstructed images;
and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
2. The on-line updating image blind super-resolution reconstruction method according to claim 1, wherein in any learning period, the degradation estimation module is optimized based on the following ways:
based on the super-resolution reconstruction module, the image to be reconstructed is subjected to up-sampling to obtain a first super-resolution image;
based on the degradation estimation module, down-sampling the first super-resolution image to obtain a first low-resolution image in a resolution spatial domain of the image to be reconstructed;
and fixing the hyper-resolution reconstruction module as a known condition parameter, and updating the parameter of the degradation estimation module based on the image to be reconstructed and the first low-resolution image.
3. The on-line updating image blind super-resolution reconstruction method according to claim 2, wherein in any learning period, the super-resolution reconstruction module is optimized based on the following ways:
based on the degradation estimation module, down-sampling the sample high-resolution image to obtain a second low-resolution image;
based on the super-resolution reconstruction module, the second low-resolution image is subjected to up-sampling to obtain a second super-resolution image in a resolution spatial domain of the sample high-resolution image;
and fixing the degradation estimation module as a known condition parameter, and updating the parameter of the hyper-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image.
4. The on-line updating image blind super-resolution reconstruction method according to claim 3, wherein the updating the parameters of the degradation estimation module based on the image to be reconstructed and the first low-resolution image specifically comprises:
updating the parameters of the degradation estimation module based on the discrimination results of the image to be reconstructed, the first low-resolution image and the second low-resolution image and the discrimination result of the image to be reconstructed;
the judgment result of the image to be reconstructed and the judgment result of the second low-resolution image are obtained by respectively judging the resolutions of the image to be reconstructed and the second low-resolution image based on a low-resolution discriminator of the learning network.
5. The on-line updating image blind super-resolution reconstruction method according to claim 4, wherein a loss function when optimizing the degradation estimation module
Comprises the following steps:
wherein L isIBAs a function of internal cyclic losses, LGAN，LFor the generation of the penalty function for the low resolution discriminator, λ is the weight, GdTo the degradation estimation module, DdThe low resolution discriminator;
y is the image to be reconstructed, Gr(y) is the first super-resolution image, Gd(Gr(y)) is the first low resolution image,
refers to the mathematical expectation of the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dd(y) is the result of the discrimination of the image to be reconstructed, xeFor high resolution images of the sample, Gd(xe) For said second low resolution image, Dd(Gd(xe) Is the discrimination result of the second low-resolution image.
6. The on-line updated image blind super-resolution reconstruction method according to claim 3, wherein the updating the parameters of the super-resolution reconstruction module based on the second super-resolution image and the sample high-resolution image specifically comprises:
updating parameters of the hyper-resolution reconstruction module based on the second super-resolution image, the sample high-resolution image, the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image;
the judgment result of the sample high-resolution image and the judgment result of the first super-resolution image are obtained by respectively judging the resolution of the sample high-resolution image and the resolution of the first super-resolution image based on a high-resolution discriminator of the learning network.
7. The on-line updating image blind super-resolution reconstruction method according to claim 6, wherein a loss function when optimizing the super-resolution reconstruction module
Comprises the following steps:
wherein,
as a function of external cyclic losses, LGAN，HFor the generation of the high-resolution discriminator a penalty function is determined, λ being the weight, GrTo said hyper-resolution reconstruction module, DrIs the high resolution discriminator;
xefor high resolution images of the sample, Gd(xe) For said second low resolution image, Gr(Gd(xe) Is the second super-resolution image is obtained,
refers to the mathematical expectation of the image to be reconstructed,
refers to the mathematical expectation of a high resolution image of the sample;
Dr(xe) Is the discrimination result of the sample high-resolution image, y is the image to be reconstructed, Gr(y) is the first super-resolution image, Dr(Gr(y)) is a discrimination result of the first super-resolution image.
8. An online-updated image blind super-resolution reconstruction device, comprising:
the initialization unit is used for initializing a degradation estimation module and a hyper-resolution reconstruction module in the learning network;
the super-resolution reconstruction unit is used for inputting images to be reconstructed into the super-resolution reconstruction module for super-resolution reconstruction every other learning period to obtain a plurality of candidate super-resolution reconstruction images;
the reconstructed image confirming unit is used for confirming the super-resolution reconstructed image of the image to be reconstructed based on the visual effect of the candidate super-resolution reconstructed images;
and in each learning period, alternately optimizing the degradation estimation module and the super-resolution reconstruction module to learn the degradation mode of the image to be reconstructed and learn super-resolution reconstruction based on the degradation mode.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the online updated image blind super-resolution reconstruction method according to any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the online updated blind super-resolution image reconstruction method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110558894.6A CN113487476A (en) | 2021-05-21 | 2021-05-21 | Online-updating image blind super-resolution reconstruction method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110558894.6A CN113487476A (en) | 2021-05-21 | 2021-05-21 | Online-updating image blind super-resolution reconstruction method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113487476A true CN113487476A (en) | 2021-10-08 |
Family
ID=77932994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110558894.6A Pending CN113487476A (en) | 2021-05-21 | 2021-05-21 | Online-updating image blind super-resolution reconstruction method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113487476A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101604442A (en) * | 2009-07-24 | 2009-12-16 | 武汉大学 | Three-linear array Level 1 image method for rebuilding blind super-resolution |
| CN105913392A (en) * | 2016-04-08 | 2016-08-31 | 西安电子科技大学昆山创新研究院 | Degraded image overall quality improving method in complex environment |
| CN106204438A (en) * | 2016-06-28 | 2016-12-07 | 北京航空航天大学 | The super-resolution rebuilding of lack sampling degraded image optimizes restoration methods |
| CN110363716A (en) * | 2019-06-25 | 2019-10-22 | 北京工业大学 | One kind is generated based on condition and fights network combined degraded image high quality method for reconstructing |
| CN111062872A (en) * | 2019-12-17 | 2020-04-24 | 暨南大学 | Image super-resolution reconstruction method and system based on edge detection |
| CN111508508A (en) * | 2020-04-15 | 2020-08-07 | 腾讯音乐娱乐科技（深圳）有限公司 | Super-resolution audio generation method and equipment |
| CN111815516A (en) * | 2020-07-08 | 2020-10-23 | 北京航空航天大学 | Super-resolution reconstruction method for weak supervision infrared remote sensing image |
| CN112163998A (en) * | 2020-09-24 | 2021-01-01 | 肇庆市博士芯电子科技有限公司 | Single-image super-resolution analysis method matched with natural degradation conditions |
-
2021
- 2021-05-21 CN CN202110558894.6A patent/CN113487476A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101604442A (en) * | 2009-07-24 | 2009-12-16 | 武汉大学 | Three-linear array Level 1 image method for rebuilding blind super-resolution |
| CN105913392A (en) * | 2016-04-08 | 2016-08-31 | 西安电子科技大学昆山创新研究院 | Degraded image overall quality improving method in complex environment |
| CN106204438A (en) * | 2016-06-28 | 2016-12-07 | 北京航空航天大学 | The super-resolution rebuilding of lack sampling degraded image optimizes restoration methods |
| CN110363716A (en) * | 2019-06-25 | 2019-10-22 | 北京工业大学 | One kind is generated based on condition and fights network combined degraded image high quality method for reconstructing |
| CN111062872A (en) * | 2019-12-17 | 2020-04-24 | 暨南大学 | Image super-resolution reconstruction method and system based on edge detection |
| CN111508508A (en) * | 2020-04-15 | 2020-08-07 | 腾讯音乐娱乐科技（深圳）有限公司 | Super-resolution audio generation method and equipment |
| CN111815516A (en) * | 2020-07-08 | 2020-10-23 | 北京航空航天大学 | Super-resolution reconstruction method for weak supervision infrared remote sensing image |
| CN112163998A (en) * | 2020-09-24 | 2021-01-01 | 肇庆市博士芯电子科技有限公司 | Single-image super-resolution analysis method matched with natural degradation conditions |
Non-Patent Citations (7)
| Title |
|---|
| SHANG LI 等: "From general to specific: Online updating for blind super-resolution", 《PATTERN RECOGNITION》, vol. 127, 31 July 2022 (2022-07-31), pages 1 - 31 * |
| ZHENGXIONG LUO 等: "Unfolding the alternating optimization for blind super resolution", 《NIPS \'20: PROCEEDINGS OF THE 34TH INTERNATIONAL CONFERENCE ON NEURAL INFORMATION PROCESSING SYSTEMS》, 6 December 2020 (2020-12-06), pages 5632 - 5643 * |
| 乔鹏: "基于可学习图像扩散的自然图像复原技术研究", 《中国博士学位论文全文数据库 信息科技辑》, no. 2021, 15 January 2021 (2021-01-15), pages 138 - 142 * |
| 吴攀超: "模糊图像中待测目标边缘轮廓的三维测量", 《中国博士学位论文全文数据库 信息科技辑》, no. 2017, 15 December 2017 (2017-12-15), pages 138 - 95 * |
| 王心哲: "基于卷积神经网络的图像盲复原方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 2019, 15 May 2019 (2019-05-15), pages 138 - 1267 * |
| 连帅龙: "基于生成对抗网络的图像超分辨算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 2020, 15 February 2020 (2020-02-15), pages 138 - 1922 * |
| 陈赛健 等: "基于生成对抗网络的文本图像联合超分辨率与去模糊方法", 《计算机应用》, vol. 40, no. 03, 19 September 2019 (2019-09-19), pages 859 - 864 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | A unified learning framework for single image super-resolution | |
| CN111709895A (en) | Image blind deblurring method and system based on attention mechanism | |
| CN110675347B (en) | Image blind restoration method based on group sparse representation | |
| CN111598778B (en) | Super-resolution reconstruction method for insulator image | |
| Sun et al. | Lightweight image super-resolution via weighted multi-scale residual network | |
| CN106127688B (en) | A kind of super-resolution image reconstruction method and its system | |
| CN107133923B (en) | Fuzzy image non-blind deblurring method based on adaptive gradient sparse model | |
| CN106169174B (en) | Image amplification method | |
| CN105488759B (en) | A kind of image super-resolution rebuilding method based on local regression model | |
| Min et al. | Blind deblurring via a novel recursive deep CNN improved by wavelet transform | |
| CN116681584A (en) | Multistage diffusion image super-resolution algorithm | |
| CN113538246A (en) | Remote sensing image super-resolution reconstruction method based on unsupervised multi-stage fusion network | |
| CN115984117A (en) | Variational self-coding image super-resolution method and system based on channel attention | |
| CN115880158A (en) | Blind image super-resolution reconstruction method and system based on variational self-coding | |
| CN115147274A (en) | Method for acquiring super-resolution image, acquisition system device and storage medium | |
| CN115345791A (en) | Infrared image deblurring algorithm based on attention mechanism residual error network model | |
| Deng et al. | Efficient test-time adaptation for super-resolution with second-order degradation and reconstruction | |
| Khattab et al. | A Hybrid Regularization-Based Multi-Frame Super-Resolution Using Bayesian Framework. | |
| Chen et al. | Multi-view self-supervised disentanglement for general image denoising | |
| Guo et al. | Image blind deblurring using an adaptive patch prior | |
| CN114862699B (en) | Face repairing method, device and storage medium based on generation countermeasure network | |
| CN108492264B (en) | Single-frame image fast super-resolution method based on sigmoid transformation | |
| CN116630152A (en) | Image resolution reconstruction method and device, storage medium and electronic equipment | |
| Khattab et al. | Regularized Multiframe Super‐Resolution Image Reconstruction Using Linear and Nonlinear Filters | |
| CN113487476A (en) | Online-updating image blind super-resolution reconstruction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |