CN111383200B - CFA image demosaicing method based on generated antagonistic neural network - Google Patents
CFA image demosaicing method based on generated antagonistic neural network Download PDFInfo
- Publication number
- CN111383200B CN111383200B CN202010239207.XA CN202010239207A CN111383200B CN 111383200 B CN111383200 B CN 111383200B CN 202010239207 A CN202010239207 A CN 202010239207A CN 111383200 B CN111383200 B CN 111383200B
- Authority
- CN
- China
- Prior art keywords
- layer
- convolution
- image
- output
- batch normalization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 12
- 230000003042 antagnostic effect Effects 0.000 title claims abstract description 11
- 238000012549 training Methods 0.000 claims abstract description 53
- 238000012360 testing method Methods 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000002474 experimental method Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 138
- 238000010606 normalization Methods 0.000 claims description 116
- 230000004913 activation Effects 0.000 claims description 87
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 43
- 238000013256 Gubra-Amylin NASH model Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 230000008485 antagonism Effects 0.000 claims description 3
- 230000003416 augmentation Effects 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000004088 simulation Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 4
- 238000003475 lamination Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a CFA image demosaicing method based on a generated antagonistic neural network, which comprises the following steps: step 1, constructing a training data set, and preprocessing data to obtain a training data set with sufficient quantity; step 2, constructing a generated type countermeasure network GAN; step 3, performing a series of pixel operations on the training data set obtained in the step 1, and using the training data set as input of the GAN network built in the step 2; step 4, setting the loss function and super parameters of the GAN network built in the step 2, and selecting a network optimization algorithm to optimize the loss function; step 5, training the constructed generation type countermeasure network GAN; and 6, performing a test experiment on the trained network model in the step 5, and measuring demosaicing images by using the signal-to-noise ratio of the color peak and the structural similarity index to illustrate the network performance. The method aims at reducing the artifact phenomenon of the demosaicing image and better recovering the texture information of high-frequency parts (corners or edges) in the image.
Description
Technical Field
The invention belongs to the technical field of image processing methods, and particularly relates to a CFA image demosaicing method based on a generated type antagonistic neural network.
Background
Images are one of the important means for humans to acquire information, express information, and communicate information. Digital cameras are currently becoming mainstream imaging devices, and are widely applied to the fields of intelligent transportation, medical imaging, remote sensing technology and the like. The most widely used in daily life is a digital color image, in which each pixel has three color values: red, green, and blue. In order to obtain accurate image color information, the digital camera needs three color sensors to receive three color component information of red, green and blue of each pixel, respectively, and then synthesizes the three components into one color image. The placement problem of three color sensors can affect subsequent color synthesis. In addition, three-sensor cameras are often expensive and relatively bulky, with most digital cameras placing a Color Filter Array (CFA) in front of the sensor element. For CFA, one of the most popular modes of Bayer pattern, and many demosaicing algorithms are designed for this mode. In order to recover a high quality color image from a CFA image, it is important to use a demosaicing technique.
CFA image demosaicing is essentially a pathological inverse problem, and many algorithms currently exist for image demosaicing, among which the simplest and widely used are image interpolation algorithms, the most common linear interpolation methods, which are simple and quick to implement, but they have some inherent drawbacks, such as artifacts and blurring of details at the edge. The nearest interpolation method is a relatively fast interpolation method, and the interpolation processing is completed by copying the pixel value with the nearest spatial position. However, since this method does not take into account the correlation between color and space, the interpolation results in a lot of false color and aliasing. Bilinear interpolation, which estimates the non-captured pixels from neighboring pixels, however, the reconstructed image by this method has many artifacts and color distortions. Bicubic is an enlarged gray image interpolation algorithm, which can achieve the purpose of demosaicing by independently interpolating each color component, but the output image of Bicubic tends to have artifacts. The conventional demosaicing method can obtain high precision, and a good effect can be obtained by using a simple interpolation algorithm for processing a smooth area with approximately the same color and gradually changed brightness in a large area, but the method still has great challenges in demosaicing the edges and the corner faces of the image. Overall, these methods have two problems. Firstly, interpolation is carried out by using a filter which is made by hand, when high-frequency signals change (such as edges and angles), space deviation can occur to each color channel, and when adjacent values are interpolated, interpolation results can show color artifacts or zipper effects; second, some conventional demosaicing methods ignore the correlation between different color channels and do not take into account all of the image information in the CFA. In order to obtain a smoother image, the correlation between channels must be considered.
Disclosure of Invention
The invention aims to provide a CFA image demosaicing method based on a generative type anti-neural network, which aims to reduce artifacts of demosaicing images, and better recover texture information of high-frequency parts (corners or edges) in the images so as to obtain images which are closer to facts.
The technical scheme adopted by the invention is that the CFA image demosaicing method based on the generated antagonistic neural network is implemented according to the following steps:
step 2, constructing a generated type countermeasure network GAN, wherein the GAN comprises two parts: a generator and a discriminator using a U-Net model for the generator and a dense residual network for the discriminator;
step 3, performing a series of pixel operations on the training data set obtained in the step 1, and using the training data set as input of the GAN network built in the step 2;
step 4, setting the loss function and super parameters of the GAN network built in the step 2, and selecting a network optimization algorithm to optimize the loss function;
And 6, performing a test experiment on the trained network model in the step 5, and measuring demosaicing images by using the signal-to-noise ratio of the color peak and the structural similarity index to illustrate the network performance.
The present invention is also characterized in that,
the step 1 specifically comprises the following steps:
step 1.1, randomly finding out a plurality of color images from an existing database to serve as color image data sets, performing downsampling operation on each color image by using a filter, wherein the obtained downsampled images are color filter array images, all CFA images form a CFA image data set, and the color image data sets and the CFA image data sets form a training data set;
step 1.2, preprocessing the training data set obtained in the step 1.1, scaling each image in the training data set by 0.7,0.8,0.9,1 times, and selecting a sliding window with a proper scale to perform translation operation according to the size of the image in the training data set, namely completing the segmentation of the small blocks of the image to obtain a plurality of small block images so as to improve the network training performance; an augmentation operation of 90 deg. rotation, 180 deg. rotation, 270 deg. rotation and flip-up and down is then performed on each tile image, resulting in a sufficient number of training data sets.
The step 2 specifically comprises the following steps:
step 2.1, constructing a generator part in GAN, and setting parameters of each layer of the network by adopting a U-Net model;
and 2.2, constructing a discriminator part in the GAN, and setting parameters of each layer of the network by adopting a dense residual network model.
In step 2.1, the generator structure adopts a U-Net model, and the structure is as follows in sequence: input layer-1 st convolution layer-1 st PReLU activation function layer-2 nd convolution layer-1 st batch normalization operation layer-2 nd PReLU activation function layer-3 rd convolution layer-2 nd batch normalization operation layer-3 rd PReLU activation function layer-4 th convolution layer-3 rd batch normalization operation layer-4 th PReLU activation function layer-5 th convolution layer-4 th batch normalization operation layer-5 th PReLU activation function layer-6 th convolution layer-5 th batch normalization operation layer-6 th PReLU activation function layer-7 th convolution layer-6 th batch normalization operation layer-7 th PReLU activation function layer-8 th convolution layer-7 th batch normalization operation layer-8 th PReLU activation function layer the method comprises the steps of deconvolution layer 1, batch normalization operation layer 8, PReLU activation function layer 9, deconvolution layer 2, batch normalization operation layer 9, PReLU activation function layer 10, deconvolution layer 3, batch normalization operation layer 10, PReLU activation function layer 11, deconvolution layer 4, batch normalization operation layer 11, PReLU activation function layer 12, deconvolution layer 5, batch normalization operation layer 12, PReLU activation function layer 13, deconvolution layer 6, batch normalization operation layer 13, PReLU activation function layer 14, deconvolution layer 7, batch normalization operation layer 14, deconvolution layer 8, tanh activation function layer 1 and output layer;
Wherein the input layer represents a four-way compressed CFA image and the output layer represents an output image; in the structure, the 1 st convolution layer output is connected with the 14 th batch normalization operation layer output, the 1 st batch normalization operation layer output is connected with the 13 th batch normalization operation layer output, the 2 nd batch normalization operation layer output is connected with the 12 th batch normalization operation layer output, the 3 rd batch normalization operation layer output is connected with the 11 th batch normalization operation layer output, the 4 th batch normalization operation layer output is connected with the 10 th batch normalization operation layer output, the 5 th batch normalization operation layer output is connected with the 9 th batch normalization operation layer output, and the 6 th batch normalization operation layer output is connected with the 8 th batch normalization operation layer output to form symmetrical connection.
In step 2.1, parameters of each layer of the generator structure in the constructed GAN model are set as follows:
setting an input channel of an input layer to 4; setting the convolution kernel scale of the 1 st convolution layer as 3*3, the convolution step length as 1*1 and the number of feature maps as 16; setting the convolution kernel scale of the 2 nd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 32; setting the convolution kernel scale of the 3 rd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 64; setting the convolution kernel scale of the 4 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 128; setting the convolution kernel scale of the 5 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; setting the convolution kernel scale of the 6 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; the convolution kernel scale of the 7 th convolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 8 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 512; the convolution kernel scale of the 1 st deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; the convolution kernel scale of the 2 nd deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 3 rd deconvolution layer as 4*4, the convolution step length as 2 x 2, and the number of feature maps as 256; the convolution kernel scale of the 4 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 256; the convolution kernel scale of the 5 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 128; the convolution kernel scale of the 6 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 64; the convolution kernel scale of the 7 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 32; the convolution kernel scale of the 8 th deconvolution layer is set to 3*3, the convolution step size is set to 1*1, and the number of feature maps is set to 3;
For all PReLU activation function layers in the generator, its functions are defined as:
PReLU(x p )＝max(0,x p )+κ×min(0,x p ) (1)
wherein, kappa represents a positive constant, kappa is E (0, 1), and kappa is set to be 0.1; x is x p An input vector representing a PReLU activation function layer;
for the Tanh activation function layer in the generator, its function is defined as:
wherein x is t Representing the input vector of the Tanh activation function layer.
In step 2.2, the discriminator structure adopts a dense residual error network model, and the structure is as follows in sequence: input layer-1 st dense residual block-2 nd dense residual block-3 rd dense residual block-4 th dense residual block-5 th dense residual block-convolution layer-Sigmoid activation function layer-output layer;
the input layer generates an output image of the generator and a corresponding real image; the output layer is the judgment result of the discriminator, which is 0 or 1; the outputs of the 1 st dense residual block, the 2 dense residual blocks, the 3 dense residual blocks and the 4 dense residual blocks are respectively connected with the output of the 5 th dense residual block to form long jump connection among the dense residual blocks;
the structure of all dense residual blocks in the discriminator is: the 1 st PReLU activation function layer-the 1 st convolution layer-the 1 st batch normalization operation layer-the 2 nd PReLU activation function layer-the 2 nd convolution layer-the 2 nd batch normalization operation layer-the 3 rd PReLU activation function layer-the 3 rd convolution layer-the 3 rd batch normalization operation layer;
The input of the 1 st PReLU activation function layer is connected with the output of the 1 st batch normalization operation layer; the output of the 1 st batch normalization operation layer is connected with the output of the 2 nd batch normalization operation layer; the output of the 2 nd batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer; the 1 st PReLU activation function layer input is connected with the 2 nd batch normalization operation layer output; the output of the 1 st batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer, and finally, dense connection and long jump connection of a dense residual block are formed;
the parameters of each layer of the discriminator structure in the constructed GAN model are set as follows:
for all PReLU activation function layers in the discriminator, the function definition is as shown in equation (1), and the parameter settings are consistent with those in equation (1).
For the Sigmoid activation function layer in the discriminator, its function is defined as:
wherein x is s An input vector representing a Sigmoid activation function layer. The convolution kernel scale of all the convolution layers of the dense residual block in the discriminator is set to 3*3, the convolution step size is set to 1*1, and the number of feature maps is set to 64.
The step 3 specifically comprises the following steps:
taking the CFA image dataset in the training dataset of the step 1 as an input CFA image, and performing a series of pixel operations on the input CFA image: taking out three color components of red, green and blue in the CFA image to form a three-channel CFA image; then further separating the green component into two channels, thereby forming a four-channel CFA image; finally, carrying out pixel compression on the four-channel image, and extracting only pixel values which are not 0; taking the finally obtained four-channel compressed CFA image as the input of a generator, wherein the output of the generator is an interpolated output image; the output image is input to the discriminator at the same time as the real image, and the output of the discriminator is fed back to the generator.
The step 4 specifically comprises the following steps:
step 4.1, setting a loss function of the GAN network and setting y i Compressing the CFA image for four channels obtained in step 3, x i For the corresponding real image, where i=1, 2, …, N represents the number of images, for the generator part the antagonism loss function L a Expressed as:
wherein G (y) i ) Representing four-way compressed CFA image y i With the output image after the generator, D (G (y i ) Represents the output image G (y) i ) By the output result of the discriminator, additionally using VGG feature perception function L p It is defined as:
wherein F [ G (y) i )]Representing extraction of the output image G (y) with respect to the generator according to conv2 layer in a pretrained VGG network i ) Is a feature mapping matrix of F (x) i ) Representing extraction of the true image x from conv2 layer in a pretrained VGG network i In addition to using the reconstruction loss function L r It is defined as:
in the method, in the process of the invention,
represents the total variance, λ represents the regularization weight, L r The function can delete detail artifacts, preserve details in the picture, and finally L a 、L p And L r Three losses combine to jointly constrain the proposed generator, defining the loss function of the generator as:
L G ＝αL a +βL p +γL r (7)
wherein, alpha, beta and gamma represent positive weights; setting parameters α=0.5, β=1, γ=1, λ=10- 5 After the parameters of the generator are correspondingly modified according to the above formula, the discriminator is updated, and the loss function of the discriminator is as follows:
wherein D (x i ) Representing a real image x i With the output result of the discriminator, D (G (y i ) Represents the output image G (y) i ) Output results from the discriminator;
the sum of the loss function of the generator and the loss function of the discriminator, i.e. L G +L D ；
Step 4.2, setting super parameters of the GAN network, and specifically enveloping input batch, learning rate and iteration times of the network;
setting the input batch to 64, the iteration number to 200, the initial learning rate to 0.01, and setting the learning rate to be reduced by 1/10 after every 40 iterations;
and 4.3, setting an optimization algorithm of the GAN network, wherein the optimization algorithm uses an adaptive moment estimation algorithm to optimize the loss function in the step 4.1.
The step 6 specifically comprises the following steps:
step 6.1, selecting a test color image, which is not in the network training set, and performing downsampling operation on the test color image by using a filter, wherein the obtained downsampled image is a test CFA image;
step 6.2, inputting the test CFA image obtained in the step 6.1 into the trained network model obtained in the step 4, and further obtaining an output image of the network, namely a test demosaicing image;
Step 6.3, in order to illustrate network performance, measuring the test demosaicing image obtained in step 6.2 by using the color peak signal-to-noise ratio and the structural similarity index, wherein the standard of the CPSNR index is as follows: the larger PSNR is above 20dB, which means that the image has better demosaicing effect; the SSIM index has a value ranging from [0,1], and a larger value indicates that the image structure is closer to the original image.
The beneficial effects of the invention are as follows: according to the method, the built network frame is trained, the trained network is used for directly completing image demosaicing, and all parameters of the network do not need to be manually adjusted.
Drawings
FIG. 1 is a flow chart of a method for demosaicing CFA images based on a generated antagonistic neural network according to the invention;
FIG. 2 is a model constructed in a CFA image demosaicing method based on a generated antagonistic neural network in accordance with the present invention;
FIG. 3 is a schematic diagram of the structure of a generator in the GAN model constructed in the method of the present invention;
FIG. 4 is a schematic diagram of the structure of a discriminator in a GAN model constructed in the method of the invention;
FIG. 5 is a simulation diagram in a simulation experiment of the present invention;
in fig. 5, fig. 5 (a) is a test color image used in the simulation experiment of the present invention, fig. 5 (b) is a corresponding enlarged area image at the square in fig. 5 (a), fig. 5 (c) is a simulation result image using the nearest neighbor interpolation method, fig. 5 (d) is a corresponding enlarged area image at the square in fig. 5 (c), fig. 5 (e) is a simulation result image using the bilinear interpolation method, fig. 5 (f) is a corresponding enlarged area image at the square in fig. 5 (e), fig. 5 (g) is a simulation result of the method of the present invention, and fig. 5 (h) is a corresponding enlarged area image at the square in fig. 5 (g).
In the figure, 1. Input CFA image, 2. Three-channel CFA image (red, green, blue color components in top-down order in the figure), 3. Four-channel CFA image (red, green, blue color components in top-down order in the figure), 4. Four-channel compressed CFA image (red, green, blue color components in top-down order in the figure), 5. Output image (red, green, blue color components in top-down order in the figure), 6. Real image (red, green, blue color components in top-down order in the figure), 7. Generator, 8. Discriminator, 9. Convolution layer, 10. Deconvolution layer, 11.PReLU activation function layer, 12. Batch Normalization (BN) operation layer, 13.Tanh activation function layer, 14. Dense residual block, 15.Sigmoid activation function layer.
Detailed Description
The invention will be described in detail below with reference to the drawings and the detailed description.
The invention discloses a CFA image demosaicing method based on a generated antagonistic neural network, which is implemented as shown in fig. 1, and specifically comprises the following steps:
the step 1 specifically comprises the following steps:
step 1.1, randomly finding out 400 color images from an existing database to serve as color image data sets, performing downsampling operation on each color image by using a filter, wherein the obtained downsampled images are Color Filter Array (CFA) images, all the CFA images form a CFA image data set, and the color image data set and the CFA image data set form a training data set;
step 1.2, preprocessing the training data set obtained in the step 1.1, scaling each image in the training data set by 0.7,0.8,0.9,1 times, and selecting a sliding window with a proper scale to perform translation operation according to the size of the image in the training data set, namely completing the segmentation of the small blocks of the image to obtain a plurality of small block images so as to improve the network training performance; an augmentation operation of 90 deg. rotation, 180 deg. rotation, 270 deg. rotation and flip-up and down is then performed on each tile image, resulting in a sufficient number of training data sets.
Step 2, constructing a generated type countermeasure network GAN, wherein the GAN comprises two parts: a generator and a discriminator using a U-Net model for the generator and a dense residual network for the discriminator;
the step 2 specifically comprises the following steps:
step 2.1, constructing a generator part in GAN, and setting parameters of each layer of the network by adopting a U-Net model;
as shown in fig. 3, in step 2.1, the generator structure adopts a U-Net model, and the structure thereof is as follows: input layer-1 st convolution layer 9-1 st PReLU activation function layer 11-2 nd convolution layer 9-1 st batch normalization operation layer 12-2 nd PReLU activation function layer 11-3 rd convolution layer 9-2 nd batch normalization operation layer 12-3 rd PReLU activation function layer 11-4 th convolution layer 9-3 rd batch normalization operation layer 12-4 th PReLU activation function layer 11-5 th convolution layer 9-4 th batch normalization operation layer 12-5 th PReLU activation function layer 11-6 th convolution layer 9-5 th batch normalization operation layer 12-6 th PReLU activation function layer 11-7 th convolution layer 9-6 th batch normalization operation layer 12-7 th PReLU activation function layer 11-8 th convolution layer 9-7 th batch normalization operation layer 12-8 th PReLU activation function layer 11-8 th batch normalization operation layer the method comprises the steps of (1) deconvolution lamination 10-8 th batch normalization operation layer 12-9 th PReLU activation function layer 11-2 nd deconvolution layer 10-9 th batch normalization operation layer 12-10 th PReLU activation function layer 11-3 rd deconvolution layer 10-10 th batch normalization operation layer 12-11 th PReLU activation function layer 11-4 th deconvolution lamination 10-11 th batch normalization operation layer 12-12 th PReLU activation function layer 11-5 th deconvolution lamination 10-12 th batch normalization operation layer 12-13 th PReLU activation function layer 11-6 th deconvolution layer 10-13 th batch normalization operation layer 12-14 th PReLU activation function layer 11-7 th deconvolution layer 10-14 th batch normalization operation layer 12-8 th deconvolution layer 10-1 st Tanh activation function layer 13-output layer;
Wherein the input layer represents a four-way compressed CFA image and the output layer represents an output image; in the structure, the 1 st convolution layer output is connected with the 14 th batch normalization operation layer output, the 1 st batch normalization operation layer output is connected with the 13 th batch normalization operation layer output, the 2 nd batch normalization operation layer output is connected with the 12 th batch normalization operation layer output, the 3 rd batch normalization operation layer output is connected with the 11 th batch normalization operation layer output, the 4 th batch normalization operation layer output is connected with the 10 th batch normalization operation layer output, the 5 th batch normalization operation layer output is connected with the 9 th batch normalization operation layer output, and the 6 th batch normalization operation layer output is connected with the 8 th batch normalization operation layer output to form symmetrical connection.
In step 2.1, parameters of each layer of the generator structure in the constructed GAN model are set as follows:
setting an input channel of an input layer to 4; setting the convolution kernel scale of the 1 st convolution layer as 3*3, the convolution step length as 1*1 and the number of feature maps as 16; setting the convolution kernel scale of the 2 nd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 32; setting the convolution kernel scale of the 3 rd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 64; setting the convolution kernel scale of the 4 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 128; setting the convolution kernel scale of the 5 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; setting the convolution kernel scale of the 6 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; the convolution kernel scale of the 7 th convolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 8 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 512; the convolution kernel scale of the 1 st deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; the convolution kernel scale of the 2 nd deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 3 rd deconvolution layer as 4*4, the convolution step length as 2 x 2, and the number of feature maps as 256; the convolution kernel scale of the 4 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 256; the convolution kernel scale of the 5 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 128; the convolution kernel scale of the 6 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 64; the convolution kernel scale of the 7 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 32; the convolution kernel scale of the 8 th deconvolution layer is set to 3*3, the convolution step size is set to 1*1, and the number of feature maps is set to 3;
For all PReLU activation function layers in the generator, its functions are defined as:
PReLU(x p )＝max(0,x p )+κ×min(0,x p ) (1)
wherein, kappa represents a positive constant, kappa is E (0, 1), and kappa is set to be 0.1; x is x p An input vector representing a PReLU activation function layer;
for the Tanh activation function layer in the generator, its function is defined as:
wherein x is t An input vector representing a Tanh activation function layer;
step 2.2, constructing a discriminator part in the GAN, and setting each layer of parameters of the network by adopting a dense residual error network model;
as shown in fig. 4, in step 2.2, the discriminator structure adopts a dense residual network model, and the structure is as follows: input layer-1 st dense residual block 14-2 nd dense residual block 14-3 rd dense residual block 14-4 th dense residual block 14-5 th dense residual block 14-convolution layer-Sigmoid activation function layer 15-output layer;
the input layer generates an output image of the generator and a corresponding real image; the output layer is the judgment result of the discriminator, which is 0 or 1; the outputs of the 1 st dense residual block, the 2 dense residual blocks, the 3 dense residual blocks and the 4 dense residual blocks are respectively connected with the output of the 5 th dense residual block to form long jump connection among the dense residual blocks;
the structure of all dense residual blocks in the discriminator is: the 1 st PReLU activation function layer-the 1 st convolution layer-the 1 st batch normalization operation layer-the 2 nd PReLU activation function layer-the 2 nd convolution layer-the 2 nd batch normalization operation layer-the 3 rd PReLU activation function layer-the 3 rd convolution layer-the 3 rd batch normalization operation layer;
The input of the 1 st PReLU activation function layer is connected with the output of the 1 st batch normalization operation layer; the output of the 1 st batch normalization operation layer is connected with the output of the 2 nd batch normalization operation layer; the output of the 2 nd batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer; the 1 st PReLU activation function layer input is connected with the 2 nd batch normalization operation layer output; the output of the 1 st batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer, and finally, dense connection and long jump connection of a dense residual block are formed;
the parameters of each layer of the discriminator structure in the constructed GAN model are set as follows:
for all PReLU activation function layers in the discriminator, the function definition is as shown in equation (1), and the parameter settings are consistent with those in equation (1).
For the Sigmoid activation function layer in the discriminator, its function is defined as:
wherein x is s An input vector representing a Sigmoid activation function layer. The convolution kernel scale of all the convolution layers of the dense residual block in the discriminator is set to 3*3,the convolution step size is set to 1*1 and the number of feature maps is set to 64.
Step 3, performing a series of pixel operations on the training data set obtained in the step 1, and using the training data set as input of the GAN network built in the step 2;
The step 3 specifically comprises the following steps:
as shown in fig. 2, the CFA image dataset in the training dataset of step 1 is taken as an input CFA image 1, and a series of pixel operations are performed on the input CFA image: taking out three color components of red, green and blue in the CFA image to form a three-channel CFA image 2 (the three-channel CFA image 2 comprises the color components of red, green and blue from top to bottom in sequence); then the green color component is further separated into two channels, thereby forming a four-channel CFA image 3 (red, green, blue color components in the four-channel CFA image 3 from top to bottom in order); finally, carrying out pixel compression on the four-channel image, and extracting only pixel values which are not 0; taking the finally obtained four-channel compressed CFA image 4 (red, green and blue color components are sequentially arranged from top to bottom in the four-channel compressed CFA image 4) as an input of a generator 7, wherein the output of the generator 7 is an interpolated output image 5 (red, green and blue color components are sequentially arranged from top to bottom in the output image 5); the output image 5 is input to the discriminator 8 simultaneously with the real image 6, and the output of the discriminator 8 is fed back to the generator 7.
Step 4, setting the loss function and super parameters of the GAN network built in the step 2, and selecting a network optimization algorithm to optimize the loss function;
The step 4 specifically comprises the following steps:
step 4.1, setting a loss function of the GAN network and setting y i Compressing the CFA image for four channels obtained in step 3, x i For the corresponding real image, where i=1, 2, …, N represents the number of images, for the generator part the antagonism loss function L a Expressed as:
wherein G (y) i ) Representing four-channel compressionCFA image y i With the output image after the generator, D (G (y i ) Represents the output image G (y) i ) By the output result of the discriminator, additionally using VGG feature perception function L p It is defined as:
wherein F [ G (y) i )]Representing extraction of the output image G (y) with respect to the generator according to conv2 layer in a pretrained VGG network i ) Is a feature mapping matrix of F (x) i ) Representing extraction of the true image x from conv2 layer in a pretrained VGG network i The above equation can help detail recovery, additionally use the reconstruction loss function L r It is defined as:
in the method, in the process of the invention,
represents the total variance, λ represents the regularization weight, L r The function can delete detail artifacts, preserve details in the picture, and finally L a 、L p And L r Three losses combine to jointly constrain the proposed generator, defining the loss function of the generator as:
L G ＝αL a +βL p +γL r (7)
wherein, alpha, beta and gamma represent positive weights; setting parameters α=0.5, β=1, γ=1, λ=10 -5 After the parameters of the generator are correspondingly modified according to the above formula, the discriminator is updated, and the loss function of the discriminator is as follows:
wherein D (x i ) Representing realityImage x i With the output result of the discriminator, D (G (y i ) Represents the output image G (y) i ) Output results from the discriminator;
the sum of the loss function of the generator and the loss function of the discriminator, i.e. L G +L D ；
Step 4.2, setting super parameters of the GAN network, and specifically enveloping input batch, learning rate and iteration times of the network;
setting the input batch to 64, the iteration number to 200, the initial learning rate to 0.01, and setting the learning rate to be reduced by 1/10 after every 40 iterations;
and 4.3, setting an optimization algorithm of the GAN network, wherein the optimization algorithm uses an adaptive moment estimation algorithm to optimize the loss function in the step 4.1.
The step 6 specifically comprises the following steps:
step 6.1, selecting a test color image, which is not in the network training set, and performing downsampling operation on the test color image by using a filter, wherein the obtained downsampled image is a test CFA image;
step 6.2, inputting the test CFA image obtained in the step 6.1 into the trained network model obtained in the step 4, and further obtaining an output image of the network, namely a test demosaicing image;
step 6.3, to illustrate network performance, the test demosaicing image obtained in step 6.2 is measured with a color peak signal-to-noise ratio (CPSNR) and a Structural Similarity Index (SSIM), wherein the CPSNR index is based on: the larger PSNR is above 20dB, which means that the image has better demosaicing effect; the SSIM index has a value ranging from [0,1], and a larger value indicates that the image structure is closer to the original image.
The effects of the present invention will be further described with reference to simulation experiments.
1. Simulation conditions:
the simulation experiment of the invention is trained in a Tensorflow environment, and the installation environment is provided with Nvidia
Computer with MX250GPU and InterCore i5-8265U CPU.
2. Simulation content and result analysis:
fig. 5 is a simulation diagram of the present invention, in which fig. 5 (a) is a test color image used in the simulation experiment of the present invention, fig. 5 (b) is a corresponding enlarged area image at the square frame in fig. 5 (a), fig. 5 (c) is a simulation result image using the nearest neighbor interpolation method, fig. 5 (d) is a corresponding enlarged area image at the square frame in fig. 5 (c), fig. 5 (e) is a simulation result image using the bilinear interpolation method, fig. 5 (f) is a corresponding enlarged area image at the square frame in fig. 5 (e), fig. 5 (g) is a simulation result of the method of the present invention, fig. 5 (h) is a corresponding enlarged area image at the square frame in fig. 5 (g), and it can be seen that the present invention can better identify high frequency characteristics (edges and angles) of an input signal, effectively restore texture and edge information of an image, can be eliminated to some extent, and the present invention can be applied to the actual practice.
Claims (3)
1. The CFA image demosaicing method based on the generated antagonistic neural network is characterized by comprising the following steps of:
step 1, constructing a training data set, and preprocessing data to obtain a training data set with sufficient quantity;
step 2, constructing a generated type countermeasure network GAN, wherein the GAN comprises two parts: a generator and a discriminator using a U-Net model for the generator and a dense residual network for the discriminator;
step 3, performing a series of pixel operations on the training data set obtained in the step 1, and using the training data set as input of the GAN network built in the step 2;
step 4, setting the loss function and super parameters of the GAN network built in the step 2, and selecting a network optimization algorithm to optimize the loss function;
step 5, training the constructed generation type countermeasure network GAN, and training a training data set according to the network super-parameters, the loss function and the selected network optimization algorithm set in the step 4 to obtain a trained network model corresponding to the training data set;
step 6, performing a test experiment on the trained network model in the step 5, and measuring demosaicing images by using the signal-to-noise ratio of the color peak and the structural similarity index to illustrate the network performance;
The step 1 specifically comprises the following steps:
step 1.1, randomly finding out a plurality of color images from an existing database to serve as color image data sets, performing downsampling operation on each color image by using a filter, wherein the obtained downsampled images are color filter array images, all CFA images form a CFA image data set, and the color image data sets and the CFA image data sets form a training data set;
step 1.2, preprocessing the training data set obtained in the step 1.1, scaling each image in the training data set by 0.7,0.8,0.9,1 times, and selecting a sliding window with a proper scale to perform translation operation according to the size of the image in the training data set, namely completing the segmentation of the small blocks of the image to obtain a plurality of small block images so as to improve the network training performance; performing the augmentation operation of 90 DEG rotation, 180 DEG rotation, 270 DEG rotation and up-down turning on each small image, thereby obtaining a training data set with sufficient quantity;
the step 2 specifically comprises the following steps:
step 2.1, constructing a generator part in GAN, and setting parameters of each layer of the network by adopting a U-Net model;
step 2.2, constructing a discriminator part in the GAN, and setting each layer of parameters of the network by adopting a dense residual error network model;
In the step 2.1, the generator structure adopts a U-Net model, and the structure is as follows in sequence: input layer-1 st convolution layer-1 st PReLU activation function layer-2 nd convolution layer-1 st batch normalization operation layer-2 nd PReLU activation function layer-3 rd convolution layer-2 nd batch normalization operation layer-3 rd PReLU activation function layer-4 th convolution layer-3 rd batch normalization operation layer-4 th PReLU activation function layer-5 th convolution layer-4 th batch normalization operation layer-5 th PReLU activation function layer-6 th convolution layer-5 th batch normalization operation layer-6 th PReLU activation function layer-7 th convolution layer-6 th batch normalization operation layer-7 th PReLU activation function layer-8 th convolution layer-7 th batch normalization operation layer-8 th PReLU activation function layer the method comprises the steps of deconvolution layer 1, batch normalization operation layer 8, PReLU activation function layer 9, deconvolution layer 2, batch normalization operation layer 9, PReLU activation function layer 10, deconvolution layer 3, batch normalization operation layer 10, PReLU activation function layer 11, deconvolution layer 4, batch normalization operation layer 11, PReLU activation function layer 12, deconvolution layer 5, batch normalization operation layer 12, PReLU activation function layer 13, deconvolution layer 6, batch normalization operation layer 13, PReLU activation function layer 14, deconvolution layer 7, batch normalization operation layer 14, deconvolution layer 8, tanh activation function layer 1 and output layer;
Wherein the input layer represents a four-way compressed CFA image and the output layer represents an output image; in the structure, the 1 st convolution layer output is connected with the 14 th batch normalization operation layer output, the 1 st batch normalization operation layer output is connected with the 13 th batch normalization operation layer output, the 2 nd batch normalization operation layer output is connected with the 12 th batch normalization operation layer output, the 3 rd batch normalization operation layer output is connected with the 11 th batch normalization operation layer output, the 4 th batch normalization operation layer output is connected with the 10 th batch normalization operation layer output, the 5 th batch normalization operation layer output is connected with the 9 th batch normalization operation layer output, and the 6 th batch normalization operation layer output is connected with the 8 th batch normalization operation layer output to form symmetrical connection;
in step 2.1, parameters of each layer of the generator structure in the constructed GAN model are set as follows:
setting an input channel of an input layer to 4; setting the convolution kernel scale of the 1 st convolution layer as 3*3, the convolution step length as 1*1 and the number of feature maps as 16; setting the convolution kernel scale of the 2 nd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 32; setting the convolution kernel scale of the 3 rd convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 64; setting the convolution kernel scale of the 4 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 128; setting the convolution kernel scale of the 5 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; setting the convolution kernel scale of the 6 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 256; the convolution kernel scale of the 7 th convolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 8 th convolution layer as 4*4, the convolution step length as 2 x 2 and the number of feature maps as 512; the convolution kernel scale of the 1 st deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; the convolution kernel scale of the 2 nd deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 512; setting the convolution kernel scale of the 3 rd deconvolution layer as 4*4, the convolution step length as 2 x 2, and the number of feature maps as 256; the convolution kernel scale of the 4 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 256; the convolution kernel scale of the 5 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 128; the convolution kernel scale of the 6 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 64; the convolution kernel scale of the 7 th deconvolution layer is set to 4*4, the convolution step length is set to 2 x 2, and the number of feature maps is set to 32; the convolution kernel scale of the 8 th deconvolution layer is set to 3*3, the convolution step size is set to 1*1, and the number of feature maps is set to 3;
For all PReLU activation function layers in the generator, its functions are defined as:
PReLU(x p )＝max(0,x p )+κ×min(0,x p ) (1)
wherein, kappa represents a positive constant, kappa is E (0, 1), and kappa is set to be 0.1; x is x p An input vector representing a PReLU activation function layer;
for the Tanh activation function layer in the generator, its function is defined as:
wherein x is t An input vector representing a Tanh activation function layer;
in step 2.2, the discriminator structure adopts a dense residual error network model, and the structure is as follows in sequence: input layer-1 st dense residual block-2 nd dense residual block-3 rd dense residual block-4 th dense residual block-5 th dense residual block-convolution layer-Sigmoid activation function layer-output layer;
the input layer generates an output image of the generator and a corresponding real image; the output layer is the judgment result of the discriminator, which is 0 or 1; the outputs of the 1 st dense residual block, the 2 dense residual blocks, the 3 dense residual blocks and the 4 dense residual blocks are respectively connected with the output of the 5 th dense residual block to form long jump connection among the dense residual blocks;
the structure of all dense residual blocks in the discriminator is: the 1 st PReLU activation function layer-the 1 st convolution layer-the 1 st batch normalization operation layer-the 2 nd PReLU activation function layer-the 2 nd convolution layer-the 2 nd batch normalization operation layer-the 3 rd PReLU activation function layer-the 3 rd convolution layer-the 3 rd batch normalization operation layer;
The input of the 1 st PReLU activation function layer is connected with the output of the 1 st batch normalization operation layer; the output of the 1 st batch normalization operation layer is connected with the output of the 2 nd batch normalization operation layer; the output of the 2 nd batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer; the 1 st PReLU activation function layer input is connected with the 2 nd batch normalization operation layer output; the output of the 1 st batch normalization operation layer is connected with the output of the 3 rd batch normalization operation layer, and finally, dense connection and long jump connection of a dense residual block are formed;
the parameters of each layer of the discriminator structure in the constructed GAN model are set as follows:
for all PReLU activation function layers in the discriminator, the function definition is as shown in formula (1), and the parameter setting is consistent with that in formula (1);
for the Sigmoid activation function layer in the discriminator, its function is defined as:
wherein x is s An input vector representing a Sigmoid activation function layer; the convolution kernel scale of all the convolution layers of the dense residual block in the discriminator is set to 3*3, the convolution step size is set to 1*1, and the number of feature maps is set to 64;
the step 3 specifically comprises the following steps:
taking the CFA image dataset in the training dataset of the step 1 as an input CFA image, and performing a series of pixel operations on the input CFA image: taking out three color components of red, green and blue in the CFA image to form a three-channel CFA image; then further separating the green component into two channels, thereby forming a four-channel CFA image; finally, carrying out pixel compression on the four-channel image, and extracting only pixel values which are not 0; taking the finally obtained four-channel compressed CFA image as the input of a generator, wherein the output of the generator is an interpolated output image; the output image is input to the discriminator at the same time as the real image, and the output of the discriminator is fed back to the generator.
2. The method for demosaicing CFA images based on a generated antagonistic neural network according to claim 1, wherein said step 4 specifically comprises the steps of:
step 4.1, setting a loss function of the GAN network and setting y i Compressing the CFA image for four channels obtained in step 3, x i For the corresponding real image, where i=1, 2, …, N represents the number of images, for the generator part the antagonism loss function L a Expressed as:
wherein G (y) i ) Representing four-way compressed CFA image y i With the output image after the generator, D (G (y i ) Represents the output image G (y) i ) By the output result of the discriminator, additionally using VGG feature perception function L p It is defined as:
wherein F [ G (y) i )]Representing extraction of the output image G (y) with respect to the generator according to conv2 layer in a pretrained VGG network i ) Is a feature mapping matrix of F (x) i ) Representing extraction of the true image x from conv2 layer in a pretrained VGG network i In addition to using the reconstruction loss function L r It is defined as:
in the method, in the process of the invention,
represents the total variance, λ represents the regularization weight, L r The function can delete detail artifacts, preserve details in the picture, and finally L a 、L p And L r Three losses combine to jointly constrain the proposed generator, defining the loss function of the generator as:
L G ＝αL a +βL p +yL r (7)
Wherein, alpha, beta and gamma represent positive weights; setting parameters α=0.5, β=1, γ=1, λ=10- 5 After the parameters of the generator are correspondingly modified according to the above formula, the discriminator is updated, and the loss function of the discriminator is as follows:
wherein D (x i ) Representing a real image x i With the output result of the discriminator, D (G (y i ) Represents the output image G (y) i ) Output results from the discriminator;
the sum of the loss function of the generator and the loss function of the discriminator, i.e. L G +L D ；
Step 4.2, setting super parameters of the GAN network, and specifically enveloping input batch, learning rate and iteration times of the network;
setting the input batch to 64, the iteration number to 200, the initial learning rate to 0.01, and setting the learning rate to be reduced by 1/10 after every 40 iterations;
and 4.3, setting an optimization algorithm of the GAN network, wherein the optimization algorithm uses an adaptive moment estimation algorithm to optimize the loss function in the step 4.1.
3. The method for demosaicing CFA images based on a generated antagonistic neural network according to claim 1, wherein said step 6 specifically comprises the steps of:
step 6.1, selecting a test color image, which is not in the network training set, and performing downsampling operation on the test color image by using a filter, wherein the obtained downsampled image is a test CFA image;
Step 6.2, inputting the test CFA image obtained in the step 6.1 into the trained network model obtained in the step 4, and further obtaining an output image of the network, namely a test demosaicing image;
step 6.3, in order to illustrate network performance, measuring the test demosaicing image obtained in step 6.2 by using the color peak signal-to-noise ratio and the structural similarity index, wherein the standard of the CPSNR index is as follows: the larger PSNR is above 20dB, which means that the image has better demosaicing effect; the SSIM index has a value ranging from [0,1], and a larger value indicates that the image structure is closer to the original image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010239207.XA CN111383200B (en) | 2020-03-30 | 2020-03-30 | CFA image demosaicing method based on generated antagonistic neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010239207.XA CN111383200B (en) | 2020-03-30 | 2020-03-30 | CFA image demosaicing method based on generated antagonistic neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111383200A CN111383200A (en) | 2020-07-07 |
| CN111383200B true CN111383200B (en) | 2023-05-23 |
Family
ID=71220039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010239207.XA Active CN111383200B (en) | 2020-03-30 | 2020-03-30 | CFA image demosaicing method based on generated antagonistic neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111383200B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132104B (en) * | 2020-10-09 | 2021-08-03 | 哈尔滨工业大学 | ISAR ship target image domain enhancement identification method based on loop generation countermeasure network |
| CN112308119B (en) * | 2020-10-15 | 2021-11-05 | 中国医学科学院北京协和医院 | Immunofluorescence classification method and device for glomerulonephritis |
| CN112508929A (en) * | 2020-12-16 | 2021-03-16 | 奥比中光科技集团股份有限公司 | Method and device for training generation of confrontation network |
| CN113099121B (en) * | 2021-04-15 | 2022-05-06 | 西安交通大学 | ISP implementation method based on weak supervised learning |
| CN116128735B (en) * | 2023-04-17 | 2023-06-20 | 中国工程物理研究院电子工程研究所 | Multispectral image demosaicing structure and method based on densely connected residual error network |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11024009B2 (en) * | 2016-09-15 | 2021-06-01 | Twitter, Inc. | Super resolution using a generative adversarial network |
| CN110473154B (en) * | 2019-07-31 | 2021-11-16 | 西安理工大学 | Image denoising method based on generation countermeasure network |
| CN110570353B (en) * | 2019-08-27 | 2023-05-12 | 天津大学 | Super-resolution reconstruction method for generating single image of countermeasure network by dense connection |
-
2020
- 2020-03-30 CN CN202010239207.XA patent/CN111383200B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111383200A (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111383200B (en) | CFA image demosaicing method based on generated antagonistic neural network | |
| CN111311490B (en) | Video super-resolution reconstruction method based on multi-frame fusion optical flow | |
| CN111739078B (en) | Monocular unsupervised depth estimation method based on context attention mechanism | |
| CN111080724B (en) | Fusion method of infrared light and visible light | |
| CN109671023B (en) | Face image super-resolution secondary reconstruction method | |
| JP4214409B2 (en) | High resolution color image generation method, high resolution color image generation apparatus, and high resolution color image generation program | |
| US7373019B2 (en) | System and method for providing multi-sensor super-resolution | |
| US20090252411A1 (en) | Interpolation system and method | |
| US8345971B2 (en) | Method and system for spatial-temporal denoising and demosaicking for noisy color filter array videos | |
| Zhou et al. | Deep residual network for joint demosaicing and super-resolution | |
| US20100067820A1 (en) | Image processing apparatus and storage medium storing image processing program | |
| CN108288256B (en) | Multispectral mosaic image restoration method | |
| WO2008150342A1 (en) | Noise reduced color image using panchromatic image | |
| CN110211038A (en) | Super resolution ratio reconstruction method based on dirac residual error deep neural network | |
| EP1110414A1 (en) | Method and apparatus for synthesizing high-resolution imagery using one high-resolution camera and a lower resolution camera | |
| CN114170286B (en) | Monocular depth estimation method based on unsupervised deep learning | |
| CN111626927A (en) | Binocular image super-resolution method, system and device adopting parallax constraint | |
| CN104036468A (en) | Super-resolution reconstruction method for single-frame images on basis of pre-amplification non-negative neighbor embedding | |
| CN113610912B (en) | System and method for estimating monocular depth of low-resolution image in three-dimensional scene reconstruction | |
| CN103313068A (en) | White balance corrected image processing method and device based on gray edge constraint gray world | |
| Paul et al. | Maximum accurate medical image demosaicing using WRGB based Newton Gregory interpolation method | |
| CN111696167A (en) | Single image super-resolution reconstruction method guided by self-example learning | |
| US11669939B1 (en) | Burst deblurring with kernel estimation networks | |
| CN104574320B (en) | A kind of image super-resolution restored method based on sparse coding coefficients match | |
| CN109658358B (en) | Rapid Bayer color reconstruction method based on multi-guide filtering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |