CN113344825A

CN113344825A - Image rain removing method and system

Info

Publication number: CN113344825A
Application number: CN202110747294.4A
Authority: CN
Inventors: 盖杉; 刘鸿辉
Original assignee: Nanchang Hangkong University
Current assignee: Nanchang Hangkong University
Priority date: 2021-07-02
Filing date: 2021-07-02
Publication date: 2021-09-03
Anticipated expiration: 2041-07-02
Also published as: CN113344825B

Abstract

The invention discloses an image rain removing method and system. The method comprises the following steps: acquiring an image to be rained; inputting the image to be subjected to rain removal into a rain removal model to obtain an image subjected to rain removal; the rain removing model is obtained by training a deep learning network by adopting training data; the deep learning network comprises a first convolution layer, a long-short term memory module, a first residual block, a second residual block and a second convolution layer which are connected in sequence; the first residual block and the second residual block respectively comprise a third convolution layer, a fourth convolution layer and a linear unit function with leakage correction, which are connected in sequence. The invention can improve the rain removing effect of the image.

Description

Image rain removing method and system

Technical Field

The invention relates to the field of image processing, in particular to an image rain removing method and system.

Background

Rainy weather, a common dynamic weather, often interferes with imaging systems. Due to the dynamics and randomness of raindrops, accumulated raindrops can degrade image quality and seriously affect the performance of an outdoor vision system, such as automatic driving, pedestrian detection, target recognition and the like, and therefore, image rain removal is particularly important.

The current single image rain removing method mainly comprises two types: the first is a priori based method such as a bilateral filtering method that separates rain marks by decomposing the rain image into low and high frequency parts; aiming at the similar and repeated pattern of rain marks in an imaging scene, a rain removing method of a rain removing low-rank representation model is established; and separating a rain layer and a non-rain layer from the rain image, and then approaching image blocks of two layers by using discrimination sparse coding on a dictionary with strong mutual exclusivity so as to remove rain marks. The other method is a deep learning-based method, such as a fully supervised rain removal method based on a convolution neural network, which learns the mapping or residual error of a rain map to an rainless map by using full convolution; unsupervised and semi-supervised based rain removal algorithms.

At present, the image rain removing method based on deep learning adopts a network with a good rain removing effect as follows: a double RECURSIVE NETWORK FOR fast image rain removal (DUAL recording NETWORK FOR FAST IMAGE DERAINING, DRN NETWORK), but this NETWORK suffers from the following disadvantages: (1) although the feature map information can be extracted as much as possible by a double recursion mode, the shallow network structure still has the problems of losing the feature map information and extracting less feature information, and the rain removing effect is still to be improved; (2) the convolutional neural network algorithm adopted by the network has the advantages that an activation function used by a hidden layer is a modified Linear Unit (ReLU) function, and the ReLU function can cause neuron death in the neural network due to the self-reason, so that the use efficiency of nodes of the neural network is low, available information can be reduced, and the rain removing effect is influenced.

Disclosure of Invention

Based on this, the embodiment of the invention provides an image rain removing method and system, so as to improve the rain removing effect of an image.

In order to achieve the purpose, the invention provides the following scheme:

an image rain removal method comprising:

acquiring an image to be rained;

inputting the image to be subjected to rain removal into a rain removal model to obtain an image subjected to rain removal;

the rain removing model is obtained by training a deep learning network by adopting training data;

the deep learning network comprises a first convolution layer, a long-short term memory module, a first residual block, a second residual block and a second convolution layer which are connected in sequence; the first residual block and the second residual block respectively comprise a third convolution layer, a fourth convolution layer and a linear unit function with leakage correction, which are sequentially connected.

Optionally, the rain removing model determining method includes:

selecting the training data from a Rain100H dataset; the training data comprises rain-containing training images and corresponding rain-removing training images;

constructing the deep learning network;

inputting the training data into the deep learning network, performing repeated iterative training by adopting a random gradient descent method to obtain a trained deep learning network, and determining the rain removing model by the trained deep learning network;

the deep learning network performs T times of inter-stage recursive computation in each iterative training process, and performs T times of intra-stage recursive computation in each inter-stage recursive computation process;

and for the Nth inter-stage recursion, inputting the inter-stage rain-removing image which contains the rain training image and is output by the Nth-1 st inter-stage recursion into the first convolution layer, performing t times of recursion calculation between the first residual block and the second residual block after passing through the long-short term memory module, and outputting the inter-stage rain-removing image by the Nth inter-stage recursion through the second convolution layer.

Optionally, the inputting the training data into the deep learning network, and performing multiple iterative training by using a random gradient descent method to obtain a trained deep learning network specifically includes:

inputting the training data into the deep learning network, training according to set parameters, and judging whether an iteration termination condition is reached according to a target loss function; the target loss function is determined by the rain removing image and the rain removing training image which are output under the current iteration times;

if so, taking the deep learning network under the current iteration times as a well-trained deep learning network;

if not, the next iteration is carried out.

Optionally, the determining the rain removal model by the trained deep learning network specifically includes:

selecting the test data from the Rain100H dataset; the test data comprises a rain-containing test image and a corresponding rain-removing test image;

inputting the test data into the trained deep learning network to obtain a predicted image;

calculating the peak signal-to-noise ratio of the predicted image and the rain removal test image;

and if the peak signal-to-noise ratio is larger than a set value, determining the trained deep learning network as the rain removing model.

Optionally, the first convolution layer includes 16 filters with convolution kernel size of 3 × 3; the second convolutional layer comprises 3 filters with convolution kernel size of 3 x 3; the input of the first residual block, the output of the first residual block, the input of the second residual block, and the output of the second residual block each include 16 channels.

The present invention also provides an image rain removing system, comprising:

the image acquisition module is used for acquiring an image to be subjected to rain removal;

the rain removing module is used for inputting the image to be subjected to rain removing into a rain removing model to obtain an image subjected to rain removing;

Optionally, the image rain removing system further includes: a rain removal model determination module for determining the rain removal model; the rain removing model determining module specifically comprises:

a training data acquisition unit for selecting the training data from the Rain100H data set; the training data comprises rain-containing training images and corresponding rain-removing training images;

the network construction unit is used for constructing the deep learning network;

the training unit is used for inputting the training data into the deep learning network, performing repeated iterative training by adopting a random gradient descent method to obtain a trained deep learning network, and determining the rain removing model by the trained deep learning network;

Optionally, the training unit specifically includes:

the training subunit is used for inputting the training data into the deep learning network, training according to the set parameters, and judging whether an iteration termination condition is reached according to a target loss function; the target loss function is determined by the rain removing image and the rain removing training image which are output under the current iteration times;

the network determination subunit is used for taking the deep learning network under the current iteration times as a trained deep learning network if the iteration termination condition is reached; and if the iteration termination condition is not reached, performing the next iteration.

Optionally, the training unit further includes:

a test data acquisition subunit, configured to select the test data from the Rain100H dataset; the test data comprises a rain-containing test image and a corresponding rain-removing test image;

the predicted image determining subunit is used for inputting the test data into the trained deep learning network to obtain a predicted image;

the signal-to-noise ratio calculating subunit is used for calculating the peak signal-to-noise ratio of the predicted image and the rain removal test image;

and the model determining subunit is used for determining the trained deep learning network as the rain removing model if the peak signal-to-noise ratio is greater than a set value.

Compared with the prior art, the invention has the beneficial effects that:

the embodiment of the invention provides an image rain removing method and system, wherein a rain removing model is obtained by training a deep learning network by adopting training data; the deep learning network comprises a first convolution layer, a long short term memory module (LSTM), a first residual block, a second residual block and a second convolution layer which are connected in sequence; the first residual block and the second residual block each include a third convolutional layer, a fourth convolutional layer, and a linear unit function with leakage correction (leakage _ ReLU function) connected in sequence. According to the deep learning network constructed by the invention, the LSTM network is added in the stage of data feature extraction, so that deep features between different stages can be excavated, and compared with a method for extracting features only by using a convolutional layer, the rain removing effect is improved; the Leaky _ Relu function is used as an activation function to activate the hidden layer, so that the use efficiency of nodes and the like of the neural network is higher, and the reduction of available information is avoided; meanwhile, the Leaky _ Relu function makes full use of the image information input into the network, so that the network can learn the characteristics of the image more efficiently and fully, and the rain removing effect is improved. Therefore, the invention combines the LSTM network with the residual block (Resblock) comprising the Leaky _ Relu function, can extract deeper features and simultaneously prevent the problems of gradient disappearance and gradient explosion in the training model, thereby improving the rain removing effect of the image.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.

FIG. 1 is a flow chart of an image rain removal method according to an embodiment of the present invention;

FIG. 2 is a block diagram of a deep learning network according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a test result of testing and visualizing the rain removal performance of the trained deep learning network according to the embodiment of the present invention; wherein, part (a) of fig. 3 is a rain-containing test image, part (b) of fig. 3 is a rain-removing test image, and part (c) of fig. 3 is a prediction image;

FIG. 4 is a comparison of a first rain removal effect provided by an embodiment of the present invention; wherein, part (a) of fig. 4 is a first rain image, part (b) of fig. 4 is an effect diagram for removing rain on the first rain image by using the DRN network, and part (c) of fig. 4 is an effect diagram for removing rain on the first rain image by using the method of the present embodiment;

FIG. 5 is a comparison of a second rain removal effect provided by an embodiment of the present invention; wherein, part (a) of fig. 5 is a second rain image, part (b) of fig. 5 is an effect diagram for removing rain on the second rain image by using the DRN network, and part (c) of fig. 5 is an effect diagram for removing rain on the second rain image by using the method of the present embodiment;

fig. 6 is a structural diagram of an image rain removing system according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

The embodiments provided herein relate to deep learning, convolutional neural networks, cyclic neural networks, long and short term memories, residual blocks, convolutional layers, convolutional kernels, activation functions, loss functions, gradient descent optimization learning algorithms, and learning rates, and are explained first.

Deep learning: deep learning is derived from the research of artificial neural networks, wherein deep refers to the number of hidden layers in the neural networks, and deep learning refers to extracting and combining low-level features through the neural networks to form more abstract high-level representation attribute categories or features so as to find distributed feature representations of data.

A convolutional neural network: the convolutional neural network is proposed by a mechanism of biological Receptive Field (received Field). The receptive field mainly refers to some properties of neurons in the auditory system, the proprioceptive system and the visual system, and the artificial neurons can respond to peripheral units and can be used for large-scale image processing. The convolutional neural network includes convolutional layers and pooling layers.

Recurrent Neural Network (RNN): the recurrent neural network is a recurrent neural network which takes sequence data as input, recurses in the evolution direction of the sequence and all nodes (recurrent units) are connected in a chain manner, and the memory and parameter sharing of the recurrent neural network have certain advantages in learning the nonlinear characteristics of the sequence.

Long short-term memory (LSTM): the method is a special RNN, and mainly aims to solve the problems of gradient extinction and gradient explosion in the long sequence training process. In short, LSTM can perform better in longer sequences than normal RNNs.

Residual block: the principle of the residual block is to directly skip the data output of several previous layers to the input part of the following data layer. Simply speaking, the data with the "clear" front and the data with the "lossy compression" back are used together as the input of the data of the network, so that the network can learn richer contents.

And (3) rolling layers: the method is mainly used for extracting local features of the image through a convolution method.

And (3) convolution kernel: is an operator, namely a weight matrix used in convolution, the weight matrix has the same size as the used image area, and the row and the column of the weight matrix are both odd numbers.

Activation function: activation functions have a very important role for an artificial neural network model to learn and understand very complex and nonlinear functions. It refers to how the "features of activated neurons" are functionally preserved and mapped out.

Loss function: the criteria for evaluating the training model and generally preferring the function to be optimized easily.

Gradient descent optimization learning algorithm: is a method of minimizing the objective function by updating the parameters in the direction opposite to the gradient direction of the objective function, which will go all the way down along the slope direction of the ramp generated by the objective function until it reaches the bottom of the valley.

Learning rate: the speed of the process of reaching the optimal value of the trainable parameters in the gradient descent optimization process is high or low. The value size of the method determines the gradient descending amplitude in the optimization process, and directly determines the performance of the learning algorithm.

The method for removing rain from an image by using a DRN network is described below.

1) Training and testing data: the training and testing data sets used by the method are all from Rian100H in a rain removing data set, wherein 1800 rain images containing dense rain marks and 1800 corresponding rain-free clear pictures are used as training sets to train network model parameters; and taking 100 rain images and 100 rain-free clear images corresponding to the rain images as a test set for performance test of the final training model.

2) Parameter setting and network training: the DRN of the method consists of 2 convolutional layers and 2 residual blocks, namely 1 convolutional layer is used for receiving input, 2 residual blocks are used for recursive computation to extract features, 1 convolutional layer is used for outputting a clean image, an activation function of a hidden layer adopts a ReLU function to activate neurons of each layer, and the parameters are shared in multiple stages. The specific network structure and the structure of the parameter setting details are as follows:

the network structure is as follows:

a first layer: conv + ReLU, convolution kernel size 3 × 3, step size 1, padding 1. A total of 16 convolution kernels are used to generate 16 feature maps and then the ReLU function is used to perform the activation of neurons at each layer that receives the input.

A second layer: the method is characterized by comprising two identically structured residual blocks, wherein each residual block has the structure of Conv + ReLU → Conv + ReLU, the input and the output of each residual block are 16 channels, and the layer is used for extracting features.

And a third layer: conv, this layer contains only one convolution layer without activation. It takes the 16 channel characteristics in the residual block as input, and outputs 3 channel final rain-removed image.

Setting parameters:

the network only uses 2 convolution layers and 2 residual blocks to carry out T-layer recursion structure, thus forming a lightweight network, and the parameters in the recursion process are shared. The training process first performs weight initialization, and then takes the negative Structural Similarity (SSIM) between the output image and the corresponding rain-free image (GT map) as a loss function. SSIM is an index for measuring the similarity of two images.

Evaluation and comparative analysis of the model: after the training data set is trained and optimized by the model, the trained model is evaluated by test data, and performance evaluation is performed by peak signal-to-noise ratio (PNSR) and Structural Similarity (SSIM). The experiment was performed in a Pytorch environment and on a PC equipped with two invar GTX 1080Ti GPUs.

The DRN network for fast image rain removal recursively expands a 3-layer residual network through a loop between an outer loop and an inner residual block. Image restoration quality and computational cost can be well balanced due to the relatively lightweight network structure and parameters. The method and the experimental data show that the rain removing effect of the rain removing method is greatly improved compared with the effect of the traditional rain removing algorithm, the bottleneck problem of the traditional rain removing method is solved, and the experimental data taking the average peak signal-to-noise ratio (PSNR) as the measurement standard are compared as follows: unit (dB).

The method used	PSNR
		GMM	15.05
DDN	21.92
		LPNET	23.73
DRN	26.99

However, the method has the following disadvantages:

1. even if feature map information is extracted as much as possible by a double recursion mode, the shallow network structure still has the problems of losing the feature map information and extracting less feature information, and the final effect has a certain promotion space.

2. The convolutional neural network algorithm in the DRN network, the activation function used by the hidden layer of the convolutional neural network algorithm is a currently popular ReLU function, and the mathematical expression of the convolutional neural network algorithm is as follows: f (x) max (0, x).

When the input signal x is less than 0, the outputs are all 0, and when the input signal x is greater than 0, the outputs are equal to the inputs. Meanwhile, the ReLU function plays a role in sparse activation, the gradient dispersion problem in the neural network training process is mainly solved, and compared with sigmoid/tanh36, the ReLU function can obtain an activation value only by one threshold value without carrying out complex operation, so that the calculation speed is increased, and the convergence is accelerated. Experimental data for DRN network based methods illustrate these characteristics of the ReLU function, but the ReLU function appears fragile when the neural network is trained, and is easily "die" as: a very large gradient flows through a ReLU neuron, which never activates any data after the parameters have been updated. If this happens, the gradient of this neuron will always be 0, and if the learning rate is set to be larger in actual operation, 40% of neurons in the neural network will be "die", so that the negative axis information is totally lost and relatively usable information becomes less.

In summary, the main purpose of this embodiment is to solve the above two disadvantages, that is, adding a layer of LSTM network between the first layer and the second layer of network structure to solve the problem of feature information loss, and using the leak ReLU function instead of the ReLU function, which is shown by experiments to achieve a significant improvement in the performance of image rain removal.

The function expression of the Leaky _ Relu function:

the image rain removing method of the present embodiment will be described in detail below.

Fig. 1 is a flowchart of an image rain removing method according to an embodiment of the present invention.

Referring to fig. 1, the image rain removing method of the present embodiment includes:

step 101: and acquiring an image to be rained.

Step 102: inputting the image to be subjected to rain removal into a rain removal model to obtain an image subjected to rain removal; the rain removing model is obtained by training a deep learning network by adopting training data; the deep learning network comprises a first convolution layer, a long-short term memory module, a first residual block, a second residual block and a second convolution layer which are connected in sequence; the first residual block and the second residual block respectively comprise a third convolution layer, a fourth convolution layer and a linear unit function with leakage correction, which are sequentially connected.

The deep learning network is shown in fig. 2, wherein the first rolling layer is a first rolling layer, the second rolling layer is a long-short term memory module, the third is a first residual block, the fourth is a second residual block, and the fifth is a second rolling layer, in practical application, a splicing operation layer C is connected before the first rolling layer and after the fifth second rolling layer, and the splicing operation layer C is used for splicing different input data in rows, for example, the splicing operation layer C before the first rolling layer is used for splicing training data and output of a previous stage in rows and then inputting the training data and the output of the previous stage as input data into the first rolling layer. In the deep learning network training process, during each iteration, T times of inter-stage recursive computation are carried out, and in the process of the inter-stage recursive computation, T times of intra-stage recursive computation are carried out between a first residual block (III) and a second residual block (IV), wherein the output characteristics of the first convolution layer (I) and the output characteristics of the two residual block (IV) after the intra-stage recursive computation (internal circulation) are activated by an LReLU (Leaky _ ReLU) function and then input to the next layer, so that the first convolution layer (I) is also activated by the Leaky-ReLU function between a long-short term memory module and a second residual block (IV) and between the second convolution layer (V).

The rain removing model determining method comprises the following steps:

1) selecting the training data from a Rain100H dataset; the training data includes rain-containing training images and corresponding rain-removing training images. Specifically, the method comprises the following steps:

the training data used in this example was a Rain100H dataset. This data set contains 1800 rain maps, wherein the background of 546 rain maps is almost consistent with that of part of the test set, and in order not to affect the model performance evaluation, the 546 rain maps are removed. The remaining 1254 pictures were cropped to yield 18810 rainy pictures with a patch _ size of 100 × 100 as the training set.

2) And constructing the deep learning network.

The deep learning network is composed of two convolutional layers (a first convolutional layer and a second convolutional layer), a long-short term memory module and two Resblock (a first residual block and a second residual block) circulating in two stages, and the whole network is circulated between the stages.

Wherein, convolutional layer (Conv): the first convolutional layer at the beginning of the network consists of 16 filters with convolution kernel size 3 x 3 for receiving data, and the second convolutional layer consists of 3 filters with convolution kernel size 3 x 3 for outputting results.

Long short term memory module (LSTM): the transmission element state of the LSTM is composed of two activation functions (sigmoid and tanh) respectively constituting a forgetting gate, an input gate and an output gate. The sigmoid outputs the input parameters to be numerical values between 0 and 1, and is mainly used for judging the influence degree of certain characteristics on the model, and the influence degree is maximum when the influence degree is 1. The module is used for excavating deep features between different stages, and feature dependence between stages can promote rain.

Residual block (Resblock): each ResBlock has two convolutional layers followed by a leakage-ReLU function. In order to keep the dimensions consistent in the recursive computation, the input and output of ResBlock have 16 channels, i.e. the input of the first residual block, the output of the first residual block, the input of the second residual block and the output of the second residual block each comprise 16 channels. These two resblocks can recursively compute t times to extract the deep features, rather than pass forward once.

3) And inputting the training data into the deep learning network, and performing repeated iterative training by adopting a random gradient descent method to obtain the trained deep learning network. The method specifically comprises the following steps:

inputting the training data into the deep learning network, training according to set parameters, and judging whether an iteration termination condition is reached according to a target loss function; the target loss function is determined from the rain-removed image output at the current iteration number and the rain-removed training image. Wherein, the neural network training parameter setting: the optimization algorithm of the neural network is selected as an adaptive matrix estimation (Adam) algorithm in a random gradient descent method, the initial learning rate is set to be 0.001, and the model is better converged by dynamically changing the learning rate when the initial learning rate is reduced to half of the original learning rate after 30, 50 and 80 epochs of iteration. Each batch of training samples (batch-size) was set to 16. The total training ethics (epochs) was 100.

If so, taking the deep learning network under the current iteration times as a well-trained deep learning network; if not, the next iteration is carried out.

The deep learning network performs T times of inter-stage recursive computation in each iterative training process, and performs T times of intra-stage recursive computation in each inter-stage recursive computation process.

And for the Nth inter-stage recursion, inputting the inter-stage rain-removing image which contains the rain training image and is output by the Nth-1 st inter-stage recursion into the first convolution layer, performing t times of recursion calculation between the first residual block and the second residual block after passing through the long-short term memory module, and outputting the inter-stage rain-removing image by the Nth inter-stage recursion through the second convolution layer. And adjusting model parameters of the deep learning network by calculating the loss of the interstage rain removing image and the rain removing training image output by the N-th interstage recursion, and performing the (N + 1) -th interstage recursion. In practical application, the cycle number T of 2 residual block (Resblock) recursively calculated within a phase is set to 7, and the cycle number T recursively calculated between phases of the overall neural network is set to 7.

4) And determining the rain removing model by the trained deep learning network. The method specifically comprises the following steps:

selecting the test data from the Rain100H dataset; the test data includes a rain-containing test image and a corresponding rain-removed test image. Test data the Rain removal results were evaluated using 100 test pictures from Rain100H and 100 corresponding no-Rain pictures as labels.

And inputting the test data into the trained deep learning network to obtain a predicted image.

And calculating the peak signal-to-noise ratio of the prediction image and the rain removal test image.

The rain removing performance of the trained deep learning network of the embodiment is tested through the test data set, and the test result is visualized, as shown in fig. 3.

In order to verify the effectiveness of the image rain removing method of the present embodiment, the conventional rain removing method (GMM, DDN, LPNET), the rain removing method based on the DRN network, and the rain removing method of the present embodiment are compared.

In order to make the method of this embodiment more comparable and persuasive with the related art in the prior art, and at the same time, in order to exclude the interference of other external factors on the rain removing effect of this embodiment, the technical experiments related to this embodiment are all completed under the Pycharm (2020) environment, on the intel (r) core (tm) i5-5820K cpu3.30ghz PC and one Nvidia Titan X GPU.

The final Rain removal performance of this example was measured by the average peak signal-to-noise ratio and compared to the Rain removal performance of the conventional method and DRN on the Rain100H test data set, with the results shown in table 1 (unit: dB).

TABLE 1

The comparative data in table 1 shows that the method of the present embodiment is indeed superior in rain removal performance to the current several more advanced conventional rain removal methods and the rain removal method of the DRN network. The rain removal effect compared to the DRN network is shown in fig. 4 and 5.

The key points of the image rain removing method of the embodiment are as follows:

(1) compared with the method of extracting the features only by convolution, the method has certain improvement on performance by adding the LSTM network in the stage of extracting the data features, and particularly, the neural network can better extract the features of circulation through a plurality of stages.

(2) A network structure with multi-stage circulation is adopted, and parameters are shared among stages.

(3) The Leaky _ Relu function is used as the activation function to activate the hidden layer, so that the problem of neuron death in the neural network caused by the Relu activation function due to the self-reason is well solved, and the use efficiency of nodes and the like of the neural network is higher. Meanwhile, the Leaky _ Relu function makes full use of the image information in the input network, so that the neural network can learn the characteristics of the image more efficiently and fully.

The advantages of the image rain removing method of the embodiment are as follows:

(1) the combination of LSTM and Resblock can extract deeper features while preventing the problems of gradient disappearance and gradient explosion in the training model.

(2) The network structure and parameter sharing of the multi-stage recursion can greatly reduce network parameters and lighten the network, thereby reducing the time and computing resources of model training.

(3) The average peak signal-to-noise ratio is adopted to measure the rain removing performance, and experimental data show that the rain removing effect of the embodiment exceeds that of the DRN method.

The invention also provides an image rain removing system, and fig. 6 is a structural diagram of the image rain removing system provided by the embodiment of the invention.

Referring to fig. 6, the system comprises:

an image obtaining module 201, configured to obtain an image to be rained.

A rain removing module 202, configured to input the image to be subjected to rain removal into a rain removing model, so as to obtain a rain-removed image; the rain removing model is obtained by training a deep learning network by adopting training data; the deep learning network comprises a first convolution layer, a long-short term memory module, a first residual block, a second residual block and a second convolution layer which are connected in sequence; the first residual block and the second residual block respectively comprise a third convolution layer, a fourth convolution layer and a linear unit function with leakage correction, which are sequentially connected.

As an optional implementation, the image rain removing system further includes: a rain removal model determination module for determining the rain removal model; the rain removing model determining module specifically comprises:

a training data acquisition unit for selecting the training data from the Rain100H data set; the training data includes rain-containing training images and corresponding rain-removing training images.

And the network construction unit is used for constructing the deep learning network.

And the training unit is used for inputting the training data into the deep learning network, performing repeated iterative training by adopting a random gradient descent method to obtain a trained deep learning network, and determining the rain removing model by the trained deep learning network.

As an optional implementation manner, the training unit specifically includes:

the training subunit is used for inputting the training data into the deep learning network, training according to the set parameters, and judging whether an iteration termination condition is reached according to a target loss function; the target loss function is determined from the rain-removed image output at the current iteration number and the rain-removed training image.

As an optional implementation, the training unit further includes:

a test data acquisition subunit, configured to select the test data from the Rain100H dataset; the test data includes a rain-containing test image and a corresponding rain-removed test image.

And the predicted image determining subunit is used for inputting the test data into the trained deep learning network to obtain a predicted image.

And the signal-to-noise ratio calculating subunit is used for calculating the peak signal-to-noise ratio of the predicted image and the rain removal test image.

As an alternative embodiment, the first convolution layer comprises 16 filters with convolution kernel size 3 x 3; the second convolutional layer comprises 3 filters with convolution kernel size of 3 x 3; the input of the first residual block, the output of the first residual block, the input of the second residual block, and the output of the second residual block each include 16 channels.

The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.

The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims

1. An image rain removing method, comprising:

acquiring an image to be rained;

2. An image rain removing method according to claim 1, characterized in that the rain removing model is determined by:

constructing the deep learning network;

3. The image rain removing method according to claim 2, wherein the training data is input into the deep learning network, and a random gradient descent method is adopted to perform a plurality of times of iterative training, so as to obtain a trained deep learning network, and specifically comprises:

if not, the next iteration is carried out.

4. The image rain removing method according to claim 2, wherein the determining the rain removing model by the trained deep learning network specifically comprises:

5. The method of claim 1, wherein the first convolution layer includes 16 filters with a convolution kernel size of 3 x 3; the second convolutional layer comprises 3 filters with convolution kernel size of 3 x 3; the input of the first residual block, the output of the first residual block, the input of the second residual block, and the output of the second residual block each include 16 channels.

6. An image rain removal system, comprising:

7. An image de-raining system according to claim 6, further comprising: a rain removal model determination module for determining the rain removal model; the rain removing model determining module specifically comprises:

8. The image rain removing system according to claim 7, wherein the training unit specifically comprises:

9. An image rain removal system according to claim 7, wherein the training unit further comprises:

10. The image rain removal system according to claim 6, wherein in the rain removal module, the first convolution layer comprises 16 filters with convolution kernel size of 3 x 3; the second convolutional layer comprises 3 filters with convolution kernel size of 3 x 3; the input of the first residual block, the output of the first residual block, the input of the second residual block, and the output of the second residual block each include 16 channels.