CN112907479B - Residual single image rain removing method based on attention mechanism - Google Patents
Residual single image rain removing method based on attention mechanism Download PDFInfo
- Publication number
- CN112907479B CN112907479B CN202110244785.7A CN202110244785A CN112907479B CN 112907479 B CN112907479 B CN 112907479B CN 202110244785 A CN202110244785 A CN 202110244785A CN 112907479 B CN112907479 B CN 112907479B
- Authority
- CN
- China
- Prior art keywords
- image
- rain
- neural network
- attention
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000007246 mechanism Effects 0.000 title claims abstract description 11
- 238000003062 neural network model Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000000452 restraining effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241001085205 Prenanthella exigua Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a residual single image rain removing method based on an attention mechanism, which mainly solves the problems of limitation and unsatisfactory processing effect of the existing single image rain removing technology. The scheme comprises the following steps: 1) Preprocessing an input image to obtain a preprocessed image; 2) Constructing an attention residual error neural network model comprising a residual error network module and a coder-decoder network module; 3) Inputting the preprocessed image into an attention residual error neural network model for training, constraining the attention residual error neural network model by using a loss function, and then performing back propagation for parameter updating to obtain a trained rain-removing neural network model; 4) And inputting the rain image to be processed into a rain-removing neural network model for image processing to obtain a clear image without rain. The method can effectively remove rain marks in a single rain-containing image to obtain a clear image; meanwhile, the background information in the original image is fully reserved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a residual single image rain removing method based on an attention mechanism, which can be used for the sharpening processing of single images.
Background
Vision, one of the most important human perception systems, is the main source from which we obtain knowledge. As pictures and videos are increasingly abundant in the internet age, images become an important part of our lives. Therefore, computer vision is a very important discipline in the background of the modern times, and is an important component of all fields in our lives, such as manufacturing, military industry and various intelligent system fields.
In a severe weather environment, pictures or videos shot by people are often disturbed and blurred by rain, snow and fog. The acquired picture or video information is damaged to different degrees, and even the main body of the picture is seriously interfered. The variable weather mainly includes rain, snow, smoke and dust. For smog and dust, rain is a more disordered picture noise, which changes constantly and also causes different effects on the picture along with the magnitude of the rain. For example, light rain can cause bright white rain lines in some areas of the picture, while heavy rain can cause occlusion or severe blurring of the entire picture. Therefore, the rainwater removing work is a work with high difficulty in the fields of image denoising and image restoration.
Rain is a dynamic weather-because its constituent particles are relatively large, it can be easily captured by the camera. Due to the introduction of unintended visual artifacts in outdoor vision systems, computer vision tasks such as subsequent image matching can be adversely affected. Raindrops typically produce dense streaks on the image that are unpredictable, such as orientation, rainfall intensity, non-uniform rainfall density, and the like. Thus, the complex pixel variations and additional gradients caused by these dense fringes can obscure the advantageous information conveyed in the image, preventing efficient detection of reliable features.
However, rainwater, the most common natural phenomenon in our lives, occurs very frequently, and particularly in the southern plum rain season, continuous rainfall occurs. Therefore, the method has practical significance for the research of image rain removal.
In recent years, the image rain-removing task has a very high attention, and in general, the image rain-removing task can be roughly divided into two types: 1) Video-based methods and 2) single-image based methods.
Chen Jun in its patent of "a video rain removing method based on multi-scale mixed index model" (application number: cn201910298090.X, publication number: CN110070506 a) discloses a video rain removing method based on multi-scale mixed index model. A rain removing video is obtained by optimizing a rain video statistical model under a single scale, pyramid decomposition reconstruction is further adopted on the rain removing video of each scale, and a final result is obtained by adding and averaging; for the video shot by a static camera, the method can effectively remove the rain stripes, however, for the video shot by a mobile camera, the method has great limitation and is not applicable.
Ding Xing discloses a method for removing rain from a single image based on a convolutional neural network in the patent of ' method for removing rain from a single image based on a convolutional neural network ' (application number: CN201610592134.6, publication number: CN 106204499A) ', which establishes an image library by artificially raining a clean and clear image through a screen blend model; then, taking out image block pairs with rain or without rain from the image library, and performing network training by taking the image block pairs as training samples; and finally, taking blocks of the single rain-containing image in an overlapping mode, inputting the blocks into a trained rain-removing filtering system, obtaining corresponding rain-free image blocks, and carrying out weighted average to obtain a rain-free image. The method has the disadvantage that the designed convolutional neural network is too simple in structure and cannot achieve a good removal effect.
Disclosure of Invention
The invention aims to provide a residual single image rain removing method based on an attention mechanism aiming at the defects of the prior art, which is used for improving the rain removing performance of a single image; firstly, preprocessing an input image, and then inputting the preprocessed image into a constructed attention residual error neural network model for training; constraining the attention residual error neural network model through a loss function, and then performing parameter updating through back propagation to obtain a trained rain-removing neural network model; and inputting the rain image into the trained rain removing neural network model to obtain a clear rain-free image. The method can effectively remove rain marks in a single rain-containing image to obtain a clear image; meanwhile, the background information in the original image is fully reserved.
The invention realizes the purpose and comprises the following steps:
(1) Preprocessing an input image to obtain a preprocessed rainy image;
(2) Building an attention residual error neural network model:
(2.1) adopting convolution layers, convolution residual blocks formed by short jump connection and long jump connection to form a residual network module, and extracting image characteristics of the preprocessed rain image;
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block;
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter a decoder network after being processed by a channel attention module SE _ Block and a convolutional layer, and the output characteristic information of decoder networks on other layers and the corresponding layers of the encoder network are overlapped and fused in a hopping connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
and (2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and rain print information output by the coding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model.
(3) Training an attention residual error neural network model by adopting the preprocessed rainy image, restraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain a rain-removing neural network model;
(4) And taking the rain image to be processed as the input of a rain removing neural network model, carrying out rain removing processing on the image by using the rain removing neural network model, and outputting a clean image after rain removing.
Compared with the prior art, the invention has the following advantages:
firstly, the network model adopts a residual error network structure, so that the network is deepened, the degradation problem of a deep neural network is avoided, richer image characteristic information can be extracted, and the network training process is simple;
secondly, as the attention mechanism is introduced, the interference of background information is reduced, so that the model focuses more on the detailed information of the rain streak in the image, and meanwhile, the implementation is simpler and the rain removing effect is good.
Drawings
FIG. 1 is a flow chart of an implementation of the method of the present invention;
FIG. 2 is a schematic diagram of a residual error network module according to the present invention;
FIG. 3 is a schematic diagram of a convolution residual block of the present invention;
FIG. 4 is a schematic diagram of a deep learning model of the method of the present invention;
fig. 5 is a schematic diagram of a channel attention module SE _ Block according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described with reference to the accompanying drawings.
Example 1:
referring to the attached figure 1, the invention provides a residual single image rain removing method based on an attention mechanism, which comprises the following steps:
step 1: preprocessing an input image to obtain a preprocessed rainy image;
the pretreatment specifically comprises the following steps: the pixel values of the input image are normalized [0,1], and the pixel size is clipped to 256 × 3.
Step 2: building an attention residual error neural network model;
(2.1) adopting convolution layers, convolution residual blocks formed by short jump connection and long jump connection to form a residual network module, and extracting image characteristics of the preprocessed rain image;
as shown in fig. 2, in this embodiment, it is preferable that 1 convolutional layer, 5 convolutional residual blocks composed of short-hop connections, and a long-hop connection constitute a residual network module, which is used to perform image feature extraction on the preprocessed rain image; the structure of the convolution residual block is shown in fig. 3, and the convolution calculation formula is as follows:
x L+1 ＝x L +F(x L +W L )，
wherein x is L+1 Is the convolution result of the L +1 th convolutional layer, x L As a result of convolution of the L-th convolutional layer, W L Is the weight of the L-th convolutional layer, F (x) L +W L ) Representing the residual part.
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block; in the embodiment, 3 downsampling and 3 channel attention modules SE _ Block are preferably selected to form an encoder network, which is used for extracting the rainprint features in the image features step by step; 3 times of upsampling and 3 channel attention modules SE _ Block form a decoder network; the system is used for fusing the rainprint features extracted by the encoder network step by step to obtain fused rainprint information; the upsampling is specifically composed of a convolution layer with the size of 3x3 and the step length of 2; the downsampling is specifically composed of a transposed convolutional layer with the size of 3x3 and the step length of 2; the channel attention module SE _ Block consists of a global pooling layer, a full connection layer, a ReLU activation layer, a full connection layer and a Sigmoid layer.
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter the decoder network after being processed by a channel attention module SE _ Block and three convolution layers, and the output characteristic information of the decoder network on other layers and the corresponding layers of the encoder network are superposed and fused in a jump connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
and (2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and the rain print information output by the coding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model.
And 3, step 3: training the attention residual error neural network model built in the step 2 by adopting the preprocessed rainy image obtained in the step 1, constraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain a trained attention residual error neural network model, namely a rain-removing neural network model;
the expression of the above loss function is:
Loss＝Loss Smooth_L1 +λLoss SSIM ，
wherein Loss Smooth_L1 Represents a smooth mean absolute error loss function expressed as follows:
wherein, y i Showing the ith rainless image of the input, x i The i Zhang Youyu image, f (x), representing the input i ) The output of the i Zhang Youyu image after the attention residual error neural network model processing is shown;
Loss SSIM representing the structural similarity loss function, the expression is as follows:
Loss SSIM ＝1-SSIM(Y,F(x))，
wherein, Y and x represent the input image without rain and the image with rain respectively, F (x) represents the output of the input image with rain x after being processed by the attention residual error neural network model;
SSIM denotes structural similarity, expressed as follows:
SSIM＝[g(Y,F(x))] α ·[c(Y,F(x))] β ·[s(Y,F(x))] γ ，
where g, c, and s represent brightness, contrast, and texture, respectively, and α, β, and γ represent a brightness coefficient, a contrast coefficient, and a texture coefficient, respectively.
And 4, step 4: and (3) taking the rain image to be processed as the input of the trained network model in the step (3), carrying out rain removing treatment on the image by using a rain removing neural network model, and outputting a clean image after rain removing.
Example 2:
referring to fig. 4, a schematic diagram of a deep learning model of the method of the present invention;
the method for removing rain from a residual single image based on an attention mechanism is the same as that in embodiment 1, wherein the attention residual neural network model constructed in the step 2 specifically comprises the following image processing processes:
4.1, inputting a rain image, outputting a characteristic F1 through first convolutional layer processing, wherein the number of channels is 32, outputting a characteristic F2 through 5 convolutional residual block processing consisting of short jump connection on the characteristic F1, and the number of channels is 32;
4.2, the output characteristic F1 is connected with the output characteristic F2 through long jump, and the output characteristic F3 is output;
4.3, outputting the characteristic F3, and outputting the characteristic F4 after the processing of a channel attention module SE _ Block;
4.4, outputting the characteristic F4 through a downsampling convolution layer to perform downsampling processing for the first time, and outputting the characteristic F5, wherein the number of characteristic channels is 64;
4.5, outputting the characteristic F5, processing by a channel attention module SE _ Block, and outputting a characteristic F6;
4.6, outputting the characteristic F6, performing second downsampling processing on the downsampled convolutional layer, and outputting a characteristic F7, wherein the number of characteristic channels is 128;
4.7, outputting the characteristic F7, processing by a channel attention module SE _ Block, and outputting a characteristic F8;
4.6, outputting the characteristic F8, performing third downsampling processing on the downsampled convolutional layer to output a characteristic F9, wherein the number of characteristic channels is 256;
4.8, outputting the characteristic F9, processing by a channel attention module SE _ Block, and outputting a characteristic F10;
4.9, outputting a characteristic F11 after the output characteristic F10 is processed by three convolution layers;
4.10, outputting the characteristic F11, processing by a channel attention module SE _ Block, and outputting a characteristic F12;
4.11, performing first upsampling processing on the output characteristic F12 through an upsampling convolutional layer to output a characteristic F13, wherein the number of characteristic channels is 128;
4.12, performing channel connection on the output characteristic F7 and the output characteristic F13, and processing the output characteristic F14 through a fifth convolution layer;
4.13, outputting the characteristic F14 through a channel attention module SE _ Block to output a characteristic F15;
4.14, performing second upsampling processing on the output characteristic F15 through the upsampling convolutional layer to output a characteristic F16, wherein the number of characteristic channels is 64;
4.15, performing channel connection on the output characteristic F5 and the output characteristic F16, and processing the output characteristic F17 through a sixth convolution layer;
4.16, outputting the characteristic F17 through a channel attention module SE _ Block to output a characteristic F18;
4.17, outputting the characteristic F18 through an upsampling convolutional layer to perform upsampling processing for the third time, and outputting a characteristic F19, wherein the number of characteristic channels is 32;
4.18, channel connection is carried out on the output characteristic F3 and the output characteristic F19, and the output characteristic F20 is processed through a seventh convolution layer;
4.19, processing the output characteristic F20 by an eighth convolution layer to output a characteristic F21, wherein the number of characteristic channels is 3;
4.20, subtracting the output characteristic F21 from the input image of the step 4.1, namely the rained image, and finally outputting a clear image after rain removal.
Example 3:
the method for removing rain from the residual single image based on the attention mechanism is the same as that of the channel attention module SE _ Block described in the embodiment 1-2 and the step (2.2.1), and is shown in FIG. 5; the specific steps of processing the image features by a channel attention module SE _ Block comprise:
5.1, outputting the weight of each channel after the image features of the input image sequentially pass through a global pooling layer, a complete connection layer, a ReLU activation layer, a complete connection layer and a Sigmoid layer;
and 5.2, multiplying each channel weight output in the step 5.1 with the input image characteristic in the step 5.1 channel by channel, and outputting the processed image characteristic.
The invention extracts more detailed image characteristics by utilizing the residual error network and the coding and decoding network structure, fully obtains the rainprint information of the rain image, and further obtains more detailed information of the rainprint in the rain image by adopting an attention mechanism, thereby effectively obtaining a clean clear image without rain and providing more convenience for subsequent high-level computer vision tasks.
The invention has not been described in detail in part of its common general knowledge to those skilled in the art.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (6)
1. A residual single image rain removing method based on an attention mechanism is characterized by comprising the following specific steps:
(1) Preprocessing an input image to obtain a preprocessed rainy image;
(2) Building an attention residual error neural network model:
(2.1) adopting a convolution layer, a convolution residual block formed by short jump connection and a residual network module formed by long jump connection, and extracting image characteristics of the preprocessed rain image;
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block;
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter a decoder network after being processed by a channel attention module SE _ Block and a convolutional layer, and the output characteristic information of decoder networks on other layers and the corresponding layers of the encoder network are overlapped and fused in a hopping connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
(2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and rain print information output by the encoding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model;
(3) Training an attention residual error neural network model by adopting the preprocessed rainy image, restraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain a rain-removing neural network model;
the loss function is specifically:
Loss＝Loss Smooth_L1 +λLoss SSIM ，
wherein Loss Smooth_L1 Represents a smooth mean absolute error loss function, expressed as follows:
wherein, y i Showing the ith rainless image of the input, x i An i Zhang Youyu image representing input, and f (xi) represents output of an i Zhang Youyu image after being processed by an attention residual error neural network model;
Loss SSIM representing a structural similarity loss function, the expression is as follows:
Loss SSIM ＝1-SSIM(Y,F(x))，
wherein, Y and x represent the input image without rain and the image with rain respectively, F (x) represents the output of the input image with rain x after being processed by the attention residual error neural network model;
SSIM denotes structural similarity, expressed as follows:
SSIM＝[g(Y,F(x))] α ·[c(Y,F(x))] β ·[s(Y,F(x))] γ ，
wherein g, c and s respectively represent brightness, contrast and structure, and alpha, beta and gamma respectively represent brightness coefficient, contrast coefficient and structure coefficient;
(4) And taking the rain image to be processed as the input of a rain removing neural network model, carrying out rain removing processing on the image by using the rain removing neural network model, and outputting a clean image after rain removing.
2. The method of claim 1, wherein: the pretreatment in the step (1) is specifically as follows: the pixel values of the input image are normalized to [0,1], and the pixel size is clipped to 256 × 3.
3. The method of claim 1, wherein: and (3) the residual error network module in the step (2.1) is formed by 1 convolution layer, 5 convolution residual error blocks formed by short jump connection and 1 long jump connection.
4. The method of claim 3, wherein: the convolution residual block formula is as follows:
x L+1 ＝x L +F(x L +W L )，
wherein x is L+1 Is the convolution result of the L +1 th convolutional layer, x L As a result of convolution of the L-th convolutional layer, W L Is the weight of the L-th convolutional layer, F (x) L +W L ) Representing the residual part.
5. The method of claim 1, wherein: the encoder network in the step (2.2.1) consists of 3 times of downsampling and 3 channel attention modules SE _ Block and is used for extracting the rain print features in the image features step by step; the decoder network consists of 3 times of upsampling and 3 channel attention modules SE _ Block and is used for fusing the rainprint features extracted by the encoder network step by step to obtain rainprint information.
6. The method of claim 5, wherein: the upsampling is specifically composed of a convolution layer with the size of 3x3 and the step length of 2; the downsampling is specifically composed of a transposed convolutional layer with the size of 3x3 and the step length of 2;
the channel attention module SE _ Block consists of a global pooling layer, a full connectivity layer, a ReLU activation layer, a full connectivity layer and a Sigmoid layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110244785.7A CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110244785.7A CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112907479A CN112907479A (en) | 2021-06-04 |
| CN112907479B true CN112907479B (en) | 2023-04-18 |
Family
ID=76107761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110244785.7A Active CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112907479B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113436101B (en) * | 2021-06-28 | 2023-11-10 | 杭州电子科技大学 | Method for removing rain by Dragon lattice tower module based on efficient channel attention mechanism |
| CN113450278B (en) * | 2021-06-30 | 2022-02-18 | 中国矿业大学 | Image rain removing method based on cross-domain collaborative learning |
| CN113450288B (en) * | 2021-08-04 | 2022-09-06 | 广东工业大学 | Single image rain removing method and system based on deep convolutional neural network and storage medium |
| CN113673590B (en) * | 2021-08-13 | 2022-12-23 | 广东工业大学 | Rain removing method, system and medium based on multi-scale hourglass dense connection network |
| CN113658074B (en) * | 2021-08-16 | 2023-07-28 | 福州大学 | Single image raindrop removing method based on LAB color space multi-scale fusion network |
| CN113706420B (en) * | 2021-10-20 | 2022-02-18 | 常州微亿智造科技有限公司 | Rain line removing device and rain line removing method in industrial detection |
| CN114387436B (en) * | 2021-12-28 | 2022-10-25 | 北京安德医智科技有限公司 | Wall coronary artery detection method and device, electronic device and storage medium |
| CN114998138B (en) * | 2022-06-01 | 2024-05-28 | 北京理工大学 | High dynamic range image artifact removal method based on attention mechanism |
| CN115345801B (en) * | 2022-10-18 | 2023-03-24 | 中科方寸知微(南京)科技有限公司 | Image compression and filter removal method and system based on image denoising idea |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447918A (en) * | 2018-11-02 | 2019-03-08 | 北京交通大学 | Removing rain based on single image method based on attention mechanism |
| AU2020100196A4 (en) * | 2020-02-08 | 2020-03-19 | Juwei Guan | A method of removing rain from single image based on detail supplement |
| CN112070690A (en) * | 2020-08-25 | 2020-12-11 | 西安理工大学 | Single image rain removing method based on convolutional neural network double-branch attention generation |
-
2021
- 2021-03-05 CN CN202110244785.7A patent/CN112907479B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109447918A (en) * | 2018-11-02 | 2019-03-08 | 北京交通大学 | Removing rain based on single image method based on attention mechanism |
| AU2020100196A4 (en) * | 2020-02-08 | 2020-03-19 | Juwei Guan | A method of removing rain from single image based on detail supplement |
| CN112070690A (en) * | 2020-08-25 | 2020-12-11 | 西安理工大学 | Single image rain removing method based on convolutional neural network double-branch attention generation |
Non-Patent Citations (4)
| Title |
|---|
| Removing rain from single images via a deep detail network;Xueyang Fu et al.;《2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)》;20171109;第1715-1723页 * |
| Residual Squeeze-and-Excitation Network for Fast Image Deraining;Jun Fu et al.;《arXiv[eess.IV]》;20200601;第1-7页 * |
| 基于深度学习的非局部注意力增强网络图像去雨算法研究;盖杉 等;《电子学报》;20201015;第48卷(第10期);第1899-1908页 * |
| 注意力残差网络的单图像去雨方法研究;徐爱生 等;《小型微型计算机***》;20200529;第41卷(第6期);第1281-1285页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112907479A (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112907479B (en) | Residual single image rain removing method based on attention mechanism | |
| CN108510451B (en) | Method for reconstructing license plate based on double-layer convolutional neural network | |
| CN110009580B (en) | Single-picture bidirectional rain removing method based on picture block rain drop concentration | |
| CN110517203B (en) | Defogging method based on reference image reconstruction | |
| CN110378849B (en) | Image defogging and rain removing method based on depth residual error network | |
| CN109993804A (en) | A kind of road scene defogging method generating confrontation network based on condition | |
| CN108269244B (en) | Image defogging system based on deep learning and prior constraint | |
| CN110544213A (en) | Image defogging method based on global and local feature fusion | |
| CN111681180B (en) | Priori-driven deep learning image defogging method | |
| CN112508960A (en) | Low-precision image semantic segmentation method based on improved attention mechanism | |
| Fan et al. | Multiscale cross-connected dehazing network with scene depth fusion | |
| CN113160286A (en) | Near-infrared and visible light image fusion method based on convolutional neural network | |
| CN110766640B (en) | Image defogging method based on depth semantic segmentation | |
| Yan et al. | MMP-net: a multi-scale feature multiple parallel fusion network for single image haze removal | |
| Qian et al. | CIASM-Net: a novel convolutional neural network for dehazing image | |
| CN112164010A (en) | Multi-scale fusion convolution neural network image defogging method | |
| Pham et al. | Low-light image enhancement for autonomous driving systems using DriveRetinex-Net | |
| CN111598793A (en) | Method and system for defogging image of power transmission line and storage medium | |
| CN107301625B (en) | Image defogging method based on brightness fusion network | |
| CN111161159B (en) | Image defogging method and device based on combination of priori knowledge and deep learning | |
| CN117058019A (en) | Pyramid enhancement network-based target detection method under low illumination | |
| Hua et al. | Iterative residual network for image dehazing | |
| Wu et al. | MSDB-based CNN architecture for image dehazing in driverless cars | |
| CN117764988B (en) | Road crack detection method and system based on heteronuclear convolution multi-receptive field network | |
| Unnikrishnan et al. | Non-Local Retinex Based Dehazing and Low Light Enhancement of Images. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |