CN115689958A - Synthetic radar image denoising method based on deep learning - Google Patents

Abstract

The invention discloses a synthetic radar image denoising method based on deep learning, which comprises the following steps: s1, preprocessing an initial noise image; s2, inputting the preprocessed image into the constructed MALNet model, and processing the image characteristics; s3, converting the processed image characteristics into floating point number information of 0-1 through a Sigmod activation function, and subtracting the floating point number information from the initial noise image to complete image reconstruction; and S4, comparing the reconstructed image with the initial image without noise, calculating a Loss value through a Loss function, performing Adam optimization on the weight parameters of the MALNet model through back propagation, and finally storing the trained MALNet model parameters. The invention has good robustness, and makes the network structure achieve good balance in controlling noise reduction and detail balance.

Description

Synthetic radar image denoising method based on deep learning

Technical Field

The invention relates to an image denoising method, in particular to a synthetic radar image denoising method based on deep learning.

Background

Synthetic Aperture Radar (SAR) is a sensor for capturing microwaves, and the principle is to establish an image through waveform reflection, so that the problem that the traditional optical remote sensing radar is influenced by environments such as weather, air impurities and the like when acquiring the image is solved. The synthetic aperture radar is mostly applied to Change Detection (CD), the Change detection is to dynamically acquire image information for a certain target, and the Change detection comprises three steps of image preprocessing, generation of a difference map and analysis and calculation of the difference map, and is applied to natural disaster estimation, resource management and allocation, land topography feature measurement and the like. However, speckle noise inherent in SAR images can degrade the performance of their change detection during the transform detection process. Therefore, the image denoising method has become a basic method for preprocessing in the change detection. How to recover a clean image from a noisy SAR image is a problem which needs to be solved urgently. The image denoising algorithm is not only applied to the synthetic aperture radar image, but also relates to the fields of industry, medicine, life consumption, engineering and the like, and is quite wide in application.

Image denoising is one of the basic aspects of enhancing image processing. At present, denoising in a synthetic aperture radar image is mainly divided into two directions, namely image filtering processing and incoherent multi-view processing. The image filtering process mainly includes noise filtering by filters, such as median filtering, mean filtering, BM3D, CSF, WNNM, TNRD and wavelet denoising, which is essentially a Maximum A Posteriori (MAP) optimization problem, and although these algorithms achieve good performance in removing synthetic noise, the denoising effect is poor in an actual environment due to a certain deviation between their assumed noise and actual noise in nature. The other one is incoherent multi-view denoising, which is based on the principle that an image is decomposed into a plurality of doppler bandwidths, each doppler bandwidth uses a different synthetic aperture, and finally, the decomposed images are superposed.

Disclosure of Invention

The invention aims to: the invention aims to provide a synthetic radar image denoising method based on deep learning, which realizes balance on controlling noise reduction and detail balance.

The technical scheme is as follows: the synthetic radar image denoising method comprises the following steps:

s1, preprocessing an initial noise image;

s2, inputting the preprocessed image into the constructed MALNet model, and processing the image characteristics;

s3, converting the processed image characteristics into floating point number information of 0-1 through a Sigmod activation function, and subtracting the floating point number information from the initial noise image to complete image reconstruction;

and S4, comparing the reconstructed image with the initial image without noise, calculating a Loss value through a Loss function, performing Adam optimization on the weight parameters of the MALNet model through back propagation, and finally storing the trained MALNet model parameters.

Further, in step S2, the MALNet model includes an irregular convolution kernel, an attention module, and a dense cascade block, where the irregular convolution kernel, the attention module, and the dense cascade block are connected in sequence, and the processing steps of the image features are as follows:

s21, sequentially passing the image with noise through an asymmetric convolution kernel and an attention module to obtain image basic features;

and S22, automatically learning residual error information of the image by the image basic features through the dense cascade block, and feeding back the residual error information to the last layer of output layer.

Further, in step S2, the irregular convolution kernel includes conv3 × 3, conv3 × 1, conv1 × 3 and conv5 × 5, the noisy image is subjected to preliminary feature extraction of the image through conv3 × 3, conv3 × 1, conv1 × 3 and conv5 × 5, and finally the preliminary features of the image are fusion-spliced and expressed as:

F(x)＝Cat[ReLU(conv(5*5))+ReLU(conv(3*1))+ReLU(conv(1*3))+ReLU(conv(3*3))]

in the formula, after each convolution kernel, the image values are filtered to the range of [0, ∞ ] through the ReLU activation function.

Further, in step S2, the dense cascade block includes multiple cascade block layers, each of which includes Conv3x3, reLU and an average pooling layer, which are connected in sequence, the noisy image sequentially passes through the cascade block layer of each stage to perform feature enhancement, and the output of each stage is used as the input of the next stage;

when the network is in forward propagation, the output of each convolution module is used as the input of the next convolution module and the next convolution module, and the global receptive field of the image is obtained through the final generalized average pooling.

Further, in step S4, a given mean square error is used to train the MALNet model, and the Loss function is expressed as:

MALNet[y(a,b)]＝Rim(a,b)＝y(a,b)-n(σ,δ)

where N is the number of all samples, y (a, b) represents the input noise image, and N (σ, δ) represents the noise with variance δ and mean σ.

Compared with the prior art, the invention has the following remarkable effects:

aiming at the synthetic aperture radar SAR noise image, the convolutional neural network model MALNEt is adopted, the model uses an end-to-end architecture, an independent subnet or manual intervention is not needed, compared with other denoising algorithms, the convolutional neural network model MALNEt not only can generate a better result, but also has good robustness, and the network achieves good balance in noise reduction control and detail balance; the denoising results of the images at different noise levels further show that the convolutional neural network model MALNet can provide a perceptually satisfactory denoising result.

Drawings

FIG. 1 is a diagram of an asymmetric convolution kernel of the present invention;

FIG. 2 is a block diagram of an attention module of the present invention;

FIG. 3 is a schematic diagram of a dense concatenation block;

fig. 4 is a block diagram of the MALNet model of the present invention;

FIG. 5 (a) is a diagram of the effect of a noise-free image;

FIG. 5 (b) is a noise image effect diagram;

FIG. 5 (c) is a WNNM denoising effect graph;

FIG. 5 (D) is a denoising effect diagram of SAR-BM 3D;

FIG. 5 (e) is a de-noising effect graph of SAR-CNN;

fig. 5 (f) is a diagram of the noise removal effect of the MALNet according to the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and the detailed description.

Aiming at the SAR image denoising problem, the invention provides a Multi-scale Attention-based dense cascade convolution neural network (MALNet) denoising algorithm, which mainly applies the idea of Multi-scale irregular convolution kernels and Attention, wherein the Multi-scale irregular convolution kernels have better image receptive field compared with a single convolution kernel, namely the Multi-scale irregular convolution kernels can collect image information from different scales so as to extract more detailed image characteristics. And then, dimension splicing is carried out on convolution kernels with different scales in the network, and an attention mechanism is introduced into the spliced feature diagram for carrying out attention division on the features, so that the whole model has good enhancement capability on the main features of the image. A Dense Cascade layer (DCB) is adopted in the middle of the network to further strengthen the characteristics, and sigmoid is used as an activation function during output, so that the gradient of the model is prevented from being saturated and disappeared. And finally, restoring and reconstructing the image through network difference reduction to realize the denoising function.

As shown in fig. 4, the denoising network MALNet according to the present invention mainly includes a multi-scale attention mechanism module, a Dense Cascade Block DCB (sense Cascade Block). The network of the invention is an end-to-end structure, and the denoising mechanism is that firstly, an image with noise is subjected to initial asymmetric convolution kernel and Attention Module (CBAM conditional Block attachment Module) to obtain the basic characteristics of the image, secondly, residual information from a main learning image is fed back through a Dense Cascade Block (DCB), and finally, the residual information is fed back to the output characteristics of the last layer, and the image is reconstructed after denoising.

The following is a description of each module:

asymmetric convolution: the regular convolution structures such as 3*3, 4*4 and 5*5 are used in most neural networks today. The regular convolution kernel structure has good effect in feature extraction, but has a defect in the aspect of extracting some image edge information and image blur information. The asymmetric convolution structure was the first network structure proposed by Chunwei Tian in 2021 and is very effective in restoring noisy low resolution images, and the asymmetric convolution can be model compressed and accelerated compared to the existing square convolution, and previous work shows that the standard d × d convolution can be decomposed into 1 × d and d × 1 convolutions to reduce the number of parameters. The theory is quite simple, if the rank of the two-dimensional convolution kernel is 1, the operation can be equivalently converted into a series of one-dimensional convolutions. However, since the kernel learned down in the deep network has a distribution eigenvalue, its internal rank is higher than in reality, and therefore applying the transform directly to the convolution kernel results in significant information loss. On the other hand, asymmetric convolution is also widely used for network structure design, for example, in addition-v 3, 7*7 convolution is replaced by 1*7 convolution and 7*1 convolution. The semantic segmentation ENet network also adopts the method to design an efficient semantic segmentation network, and the precision is slightly reduced, but the parameter quantity is reduced by 33%. Thus, asymmetric convolution has advantages over square convolution in terms of rate and accuracy. The present invention employs two asymmetric convolution modules, conv3 × 3, conv3 × 1, conv1 × 3, conv5 × 5, as shown in fig. 1. The four convolution kernels are used for carrying out preliminary feature extraction on the images, and finally, the images are subjected to fusion splicing (Concatenate, cat), and the network of the four convolution kernels can be expressed as:

F(x)＝Cat[ReLU(conv(5*5))+ReLU(conv(3*1))+ReLU(conv(1*3))+ReLU(conv(3*3))] (1)

in the formula, after each convolution kernel, a ReLU activation function is needed to filter the image numerical value to the range of [0, ∞ ] so as to prevent the image from generating gradient explosion and disappearance when the image reversely propagates.

An attention mechanism is as follows: the attention mechanism is essentially a mechanism that autonomously learns a set of weighting coefficients through the network and emphasizes the interested region in a dynamic weighting manner, and simultaneously suppresses irrelevant background regions. The current mainstream attention mechanism is three types: channel attention, spatial attention, and self-attention. The channel attention refers to the fact that weight coefficients of different channels of a current model are built, the best model channel is captured through learning, and weight systems of the rest channels are automatically reduced, so that important features are strengthened and non-important features are restrained. The representation of the channel attention is mainly SEnet. Spatial attention refers to improving the feature expression of key regions, and essentially transforms spatial information in an original picture into another space through a spatial transformation module, retains the key information, generates a weight mask (mask) for each position and weights the output, thereby enhancing a specific target region of interest and weakening irrelevant background regions. The invention introduces the attention mechanism of the current comparison mainstream, namely CBAM, the structure of which is shown as figure 2, and the invention collects the attention mechanism of space and channel, and the effect is better.

Dense cascade block: the dense cascade network is a network for automatically training and automatically adding hidden units, and has the advantages of high learning rate, self-definition of network neurons and depth, back propagation effectiveness and the like. The invention designs a network structure of a dense cascade layer, which is inspired by Resnet, and as shown in figure 3, the structure forms the dense cascade layer by a series of convolution modules, when the network forwards propagates, the output of each convolution module is used as the input of the next convolution module and the next convolution module, and then the global receptive field of the image is obtained by the last generalized average pooling, so that the network can better learn the whole information of the image.

The denoising network MALNet of the present invention is shown in FIG. 4, and the MALNet mainly comprises a multi-scale irregular convolution kernel, an attention module, and a Cascade Block DCB (sense Cascade Block). The network of the invention is designed in an end-to-end structure, namely, the input is a noise graph and the output is a clean graph. The denoising mechanism of the invention is that firstly, an image with noise is obtained through an initial asymmetric convolution kernel and an attention module to obtain the basic characteristics of the image, secondly, residual error information of the image is automatically learned through a cascade block (DCB), and finally, the residual error information is fed back to the output characteristics of the last layer, and the image is reconstructed after denoising.

Denoising the image according to equation (2):

y(a,b)＝n(σ,δ)×x(a,b) (2)

where y (a, b) represents an input noise image, and x (a, b) represents an image without noise; n (σ, δ) represents noise with variance δ and mean σ.

After the input noise Image passes through the network MALNet, a Residual Image (Rim) is output, which is expressed as:

MALNet[y(a,b)]＝Rim(a,b)＝y(a,b)-n(σ,δ) (3)

then, a given Mean Square Error (MSE) is used to train the denoising network, whose Loss function is expressed as:

(4) Where w and b are the learned Weights (Weights, w) and biases (Bias, b) in the network, and N is the number of all samples, which is the set of all learned parameters in the network.

The method uses a training set of 400 manufactured images added with noise, during training, firstly, a noise parameter is selected as sigma, then, an initial denoising result and an original image are respectively cut into image blocks with the size of 256 × 256 at the same position of the image, the image blocks are used as the input of a network to train the proposed MALNET network, each image is cut into 40x 40 windowing to train, the used data enhancement modes are random turning, translation, scaling, rotation and mirroring, the dimension of an epoch of the training is set to be 150, a Batchm is set to be a normalization setting, a momentum is set to be 0.95, the dimension of a Batchsize is set to be 64, an Adam optimizer is used to optimize the MALNET, the learning rate is dynamically adjusted according to Norepo =30, 60 and 90, the initial setting is 0.0001, and the rest settings are default settings. By iteratively adjusting parameters such as epoch, learning rate, batch size, and the number of subsequent DCB cycles, the resulting model is the best performing model obtained on the validation set during training. During testing, the network can adapt to noise images with any size, the image is subjected to Gaussian noise additive processing by adopting Python, and a model algorithm is realized by using a Pythrch. The detailed steps are as follows:

a. first, data is preprocessed by converting an image with 256 × 256 input pixels into grayscale images (with grayscale values of 0-255), and dividing each image into pane images with 40x 40 pixels. Secondly, noise is added to each pane image one by one, the noise level is mean value sigma, and the variance is 0 Gaussian noise, and each pane image is randomly turned, translated, rotated, mirrored and scaled to achieve the effect of data enhancement. Finally, all panes are restored to a pending image with a pixel size of 256 × 256.

b. The image is sent to a designed MALNet model, firstly, the image is convoluted and extracted by four parallel convolution kernels (Conv 5 x 5, conv3x 1, conv1 x3, conv3x 3), and then the image value is filtered to be in a non-negative range by a ReLU activation function. Secondly, a set of weighting coefficients is independently learned through the network through a CBAM Attention mechanism (Attention), and a mechanism for emphasizing interested areas and simultaneously suppressing irrelevant background areas in a dynamic weighting mode is adopted. Finally, as shown in fig. 4, the four parallel image features are overlapped in channel dimension (Concat), and the four residual images are overlapped and fused.

c. And then, carrying out information enhancement on the image characteristics through a six-layer DCB module, wherein the working principle of the DCB module is as follows: firstly, the three calculation layers of the first stage are passed, and the calculation layers are respectively: conv3x3, reLU, average pooling (Average Pool), and feature enhancement in the second and third stages, and taking the output of each stage as the input value sum of the next stage, completing the construction of the dense cascade network. And finally, converting the image numerical value into a floating point number between 0 and 1 by a Sigmod activation function to complete the construction of the whole DCB module.

d. And c, after the numerical matrix obtained after the image is subjected to step c, after twelve layers of DCB characteristic iterative enhancement, and after Resnet, the numerical matrix is connected and added with the front network residual error.

e. And (5) converting the floating point number information into 0-1 by a Sigmod activation function again, and subtracting the initial noise image to complete the reconstruction of a clean image.

f. The image reconstructed by the MALNEt is compared with the original image without noise, the Loss value is calculated through a Loss function, and Adam optimization is carried out on the weight parameters of the MALNEt model through back propagation.

g. And storing the trained parameter model, and finishing the input and output operation of the image through a test network, namely performing image denoising end to end.

And c, circularly repeating the steps a-g, and finishing all image denoising tasks by the MALNet model.

Qualitative and quantitative experiments are carried out in the invention to evaluate and demonstrate the performance of the proposed MALNet model in denoising. WNNM, SAR-BM3D and SAR-CNN algorithms are compared with the method proposed by us. And visually observing the clear condition and the complete sign of the denoised image. For fair comparison, default settings provided by authors in three algorithm documents are used for comparison, peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values are used as objective evaluation indexes, the PSNR and the SSIM are respectively calculated and used as error metrics, and other denoising methods are compared to test the denoising effect of the model. Suppose the original clean image X is of size NxM, N is the image length, M is the image height, i is the image abscissa, j is the image ordinate, and Y is the denoised image. PSNR is expressed as:

where fmax denotes the maximum intensity of the input image, there are 256 possible gray values for some 8-bit gray scale images, so fmax =256. The PSNR is used to measure the denoising effect of the model, and other indexes can be used to evaluate the difference between the original image and the denoised image.

SSIM is a quality assessment used to measure the similarity between two images. Let X and Y be two non-negative image signals for the calculation of SSIM, expressed as:

wherein, mu _x And mu _y Denotes the average intensity, σ, of X and Y, respectively _x And σ _y The standard deviation of X and Y is shown. Sigma _xy Is the covariance of the images x and y, c ₁ And c ₂ Is a constant value. Local parameter mu _x 、μ _y 、σ _x 、σ _y And σ _xy In an 8 x 8 square window and calculated in a pixel-shifted manner.

The invention compares three denoising algorithms, selects an airplane as a verification image, for example, as shown in fig. 5 (a) - (f) for denoising effect diagrams of airplane images, the visual effect comparison of denoising results of different algorithms when the current noise level is sigma =30 is shown, it can be obviously observed that the WNNM denoising effect diagram has many defects which are not completely removed, the texture loss is quite serious, the noise removal of the whole image is not clean, and the visual effect is fuzzy. The SAR-BM3D retains some details, but the body of the airplane is fuzzy, most edge information of the tail part is erased, and the wave edge information in the coast image is erased more seriously. The information of the runway edge in the SAR-CNN denoising effect graph is not well reserved, the placed objects beside the runway edge are fuzzy, although the wings of the airplane are recovered, the whole airplane at the lowest part is still far from the reference image, the recovered small objects are fuzzy, in another scene contrast graph, the picture presents the situation of more noise points, the wrinkles of some texture lines are serious, the tearing feeling of the picture is strong, and the denoising effect is general.

In the denoising evaluation indexes in table 1, PSNR and SSIM results of different methods on different images are respectively represented, rows in table 1 are PSNR and SSIM values generated by different algorithms when noise parameters are σ =20, 30, 40, and 50, columns are PSNR and SSIM values of the same method at different noise levels, respectively, and the method includes: the SAR-BM3D model with the denoising parameter being sigma, the SAR-CNN model with the denoising parameter being sigma, the WNNM model with the denoising parameter being sigma and the MALNet of the invention.

TABLE 1 De-noising evaluation index Table

From table 1, it can be calculated that the average PSNR value of the proposed MALNet is higher than SAR-BM3D by about 9.25dB, higher than SAR-CNN by about 0.75dB, and higher than WNNM by about 14.45dB, and the PSNR values obtained by the rest models of the present invention are higher than the compared values of the rest models, especially when the noise parameter is 20, the method of the present invention is higher than the compared SAR-CNN algorithm by 2.56dB at maximum. In terms of structural similarity, it can be seen that the structural similarity of the MALNet is mostly the highest value in the compared methods, and is slightly lower than the SAR-CNN only when the noise parameter is 40, and the obtained average structural similarity is also the highest. Therefore, by integrating two objective evaluation indexes of PSNR and SSIM, the network provided by the invention is superior to the compared method in denoising performance.

Claims (5)

1. A synthetic radar image denoising method based on deep learning is characterized by comprising the following steps:

s1, preprocessing an initial noise image;

s2, inputting the preprocessed image into the constructed MALNet model, and processing the image characteristics;

s3, converting the processed image characteristics into floating point number information of 0-1 through a Sigmod activation function, and subtracting the floating point number information from the initial noise image to complete image reconstruction;

and S4, comparing the reconstructed image with the initial image without noise, calculating a Loss value through a Loss function, performing Adam optimization on the weight parameters of the MALNet model through back propagation, and finally storing the trained MALNet model parameters.

2. The deep learning-based synthetic radar image denoising method according to claim 1, wherein in step S2, the MALNet model comprises an irregular convolution kernel, an attention module and a dense cascade block, the irregular convolution kernel, the attention module and the dense cascade block are connected in sequence, and the processing steps for the image features are as follows:

s21, sequentially passing the image with noise through an asymmetric convolution kernel and an attention module to obtain image basic features;

and S22, automatically learning residual error information of the image by the image basic features through the dense cascade block, and feeding back the residual error information to the last layer of output layer.

3. The method for denoising the synthetic radar image based on the deep learning of claim 2, wherein in step S2, the irregular convolution kernel includes conv3 × 3, conv3 × 1, conv1 × 3 and conv5, the noisy image is respectively subjected to primary feature extraction of the image through conv3 × 3, conv3 × 1, conv1 × 3 and conv5, and finally the primary features of the image are fusion-spliced and expressed as:

F(x)＝Cat[ReLU(conv(5*5))+ReLU(conv(3*1))+ReLU(conv(1*3))+ReLU(conv(3*3))]

in the formula, after each convolution kernel, the image values are filtered to the range of [0, ∞ ] through the ReLU activation function.

4. The deep learning-based synthetic radar image denoising method according to claim 2, wherein in step S2, the dense cascade block comprises multiple cascade block layers, each polar cascade block layer comprises a Conv3x3, a ReLU activation function and an average pooling layer which are connected in sequence, the noisy image is subjected to feature enhancement sequentially through the cascade block layer of each stage, and the output of each stage is used as the input of the next stage;

when the network is in forward transmission, the output of each convolution module is used as the input of the next convolution module and the next convolution module, and the global receptive field of the image is obtained through the average pooling layer.

5. The deep learning-based synthetic radar image denoising method according to claim 2, wherein in step S4, a given mean square error is adopted to train a MALNet model, and a Loss function of the model is expressed as:

MALNet[y(a,b)]＝Rim(a,b)＝y(a,b)-n(σ,δ)

where N is the number of all samples, y (a, b) represents the input noise image, and N (σ, δ) represents the noise with variance δ and mean σ.

Images