CN109242798B - Poisson denoising method based on three-segment sub-network representation - Google Patents

Poisson denoising method based on three-segment sub-network representation Download PDF

Info

Publication number
CN109242798B
CN109242798B CN201811078556.7A CN201811078556A CN109242798B CN 109242798 B CN109242798 B CN 109242798B CN 201811078556 A CN201811078556 A CN 201811078556A CN 109242798 B CN109242798 B CN 109242798B
Authority
CN
China
Prior art keywords
network
conv
image
sub
poisson
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811078556.7A
Other languages
Chinese (zh)
Other versions
CN109242798A (en
Inventor
刘且根
李冠玉
张凤芹
张明辉
王玉皞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanchang University
Original Assignee
Nanchang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanchang University filed Critical Nanchang University
Priority to CN201811078556.7A priority Critical patent/CN109242798B/en
Publication of CN109242798A publication Critical patent/CN109242798A/en
Application granted granted Critical
Publication of CN109242798B publication Critical patent/CN109242798B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a Poisson denoising method based on three-segment sub-network representation, which comprises the following steps: step (a): after convolution processing is carried out on input images with all pixel values being 1, the input images are added with noisy images which are processed through two convolution operators to obtain an intermediate processing image 1; step (b): processing the intermediate processing image 1 by a resnet module without a batch normalization layer to obtain an intermediate processing image 2; step (c): adding the two convolutions of the step (b) with a map with a pixel value of 1 to obtain a final result; the invention inherits and expands the structure and the advantages of the traditional variance stabilization transformation scheme, and utilizes three sub-networks to process nonlinear learning mapping by a network design and supervised learning method, so that the network widening greatly improves the denoising performance; compared with the traditional iterative denoising method, the method has better denoising performance, and particularly has more excellent image recovery effect under the condition of low signal-to-noise ratio.

Description

Poisson denoising method based on three-segment sub-network representation
Technical Field
The invention relates to the technical field of optoelectronic devices, in particular to a Poisson denoising method based on three-segment sub-network representation.
Background
Image denoising, which is a fundamental problem in image processing and aims at estimating an ideal image from a noisy observed image, is generally an ill-posed inverse problem, which has been intensively studied by a large amount of literature, but mainly aims at additive white gaussian noise, whereas in photon counting imaging systems, such as CCD solid-state photodetector arrays, astronomical imaging, computed X-ray imaging (CR), fluorescence confocal microscopy, and the like, the acquired image is often contaminated by quantum noise.
Since light has quantum special effect, the number of quanta reaching the surface of the photodetector has statistical fluctuation, so that image monitoring has granularity, the granularity causes the reduction of image contrast and the hiding of image detail information, and the measurement uncertainty caused by the light quanta becomes the Poisson noise of the image. In the case of photon confinement, poisson noise is typically generated. Signal degradation caused by poisson noise is a common phenomenon in biomedical imaging, night vision, and astronomy applications. Therefore, poisson noise is particularly important for subsequent processing such as classification and recognition of images. However, unlike additive gaussian noise, poisson noise is a multiplicative noise, does not satisfy the simple additive principle and has signal-dependent noise strength and variance. The noise signal depends on the image itself, i.e. the features related to the signal. In particular, assuming that f is an observed noise contaminated image obeying a poisson distribution, the dispersion probability is:
Figure BDA0001799901210000011
where the noise-free image u represents the mean of the distribution and i represents the pixel value. Typically, the signal-to-noise ratio (i.e., peak) per pixel is
Figure BDA0001799901210000012
Therefore, in the observed image, a lower signal strength means a stronger noise. The noise power in an image is usually measured by the maximum value of the image, called the peak.
Statistically, pixels with large brightness are more disturbed, and thus removing poisson noise is a difficult task. One classical method for removing poisson noise is to perform variance stabilizing transformation, such as Anscombe, HaarFisz, CVS transformation, etc., on observed data in a space domain or a transformation domain (such as wavelet), and each transformed data is approximately gaussian distribution with the same variance, thereby converting into a general gaussian denoising problem. Various algorithms such as wiener filtering and wavelet threshold shrinkage can be applied to the problem, and finally, final de-noising data can be obtained through inverse VS transformation. However, only when the number of photons gradually increases, the data after VS transformation gradually tends to gaussian distribution, and is not suitable for low photon number situations, such as X-Ray and Gamma Ray data, and meanwhile, the VS transformation is a nonlinear transformation, which is not beneficial to analyzing and optimizing the performance of the denoising algorithm.
The practical application needs a more effective Poisson denoising method, does not need variance stabilizing transformation preprocessing, and can directly analyze Poisson data. The wavelet transform coefficient of the noisy image is subjected to threshold operation adaptive to the statistical characteristics of Poisson noise, so that stable variance transformation can be avoided. For this reason, Kolaczyk modifies the wavelet coefficient threshold strategy originally used for gaussian noise with respect to the statistical properties of poisson noise, but is still not very effective in low photon count situations. Then, Kolaczyk proposes another algorithm, and a false detection rate can be preset through a statistical hypothesis-testing (hypothesis-testing) strategy, so that the consistency (importance) of the Haar wavelet coefficients is judged, and corresponding threshold operation is carried out. The method is further expanded to biorthogonal Haar wavelets, and smoother de-noised images can be formed. Another class of methods is to process poisson noise under a multi-scale Bayesian framework. The Bayesian method has the advantages that the prior knowledge about an ideal image can be combined in the denoising process, and meanwhile, the denoising problem can be simplified by utilizing multi-scale analysis, so that the Bayesian method is approved by scholars.
The above methods are all desirable, but have the general disadvantage of not handling image recovery well at low signal-to-noise ratios.
Disclosure of Invention
The invention aims to provide a Poisson denoising method based on three-segment sub-network representation, so as to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme: a Poisson denoising method based on three-segment sub-network representation comprises the following steps:
step (a): after convolution processing is carried out on input images with all pixel values being 1, the input images are added with noisy images which are processed through two convolution operators to obtain an intermediate processing image 1;
step (b): processing the intermediate processing image 1 by a resnet module without a batch normalization layer to obtain an intermediate processing image 2;
step (c): and (c) adding the two convolutions to the graph with the pixel value of 1 to obtain the final result.
Further, the step (a) is:
let the output of the input noisy image f after passing through a convolution operator
Figure BDA0001799901210000032
Comprises the following steps:
Figure BDA0001799901210000031
where w is the weight to be learned, b is the bias, d1,d2The filter numbers which are input and output respectively have great influence on the experimental result by learning weight and bias under the training of the network; the output g of subnetwork 1 can thus be expressed approximately as:
g=Conv(1)+Conv(Conv(f))
where Conv (1) is an output of an image having all pixel values of 1 after passing through one convolution layer, and Conv (f)) is an output of a noisy image after passing through two consecutive convolution layers. The subnetwork 1 mainly simulates ansscomb transformation, converts random variables with poisson distribution into random variables with approximate standard gaussian distribution, and facilitates subsequent further processing.
Further, the step (b) is:
due to the strong nonlinearity of poisson noise, the forward transformation of the first step and the denoising of the second step are not accurate and can only be approximately realized. Thus, the design of subnet 2 is not to remove pure gaussian noise, but rather an approximate gaussian noise removal. Therefore, a more flexible operation operator is needed to assist the whole process, so that a batch normalization layer is not used in the residual learning network of the sub-network II, and the idea is experimentally verified in other networks;
the sub-network 2 utilizes the idea of residual learning, and the feature mapping of the input g of the network is F (g), which is expressed as follows:
F(g)=w2σ(w1g)
where f (g) is a complex function composed of convolution operator and ReLU activation function, σ represents nonlinear function ReLU, and the ReLU unit is the activation layer of the network and is an essential part of the network, and is often set after the convolution layer. The output of subnetwork 2 is h (g), which is expressed as a function of:
H(g)=F(g,{wi})+g
the residual function is easier to optimize, the degradation problem of the deep network can be solved, and the number of network layers can be greatly deepened.
Further, the step (c) is:
the sub-network 3 simulates the inverse transformation of the Anscombe transformation, so that the inverse transformation returns stable variance and denoised data to the original range, and corresponds to the sub-network 1;
output=Conv(1)+Conv(Conv(H(g)))
wherein, Conv (H (g)) is the result of two continuous convolution layers on the output of the sub-network 2, such continuous convolution operation not only can improve the recovery effect of the image, but also can make the training process more stable.
Compared with the prior art, the invention has the beneficial effects that:
the invention combines a variance stable transformation structure with a convolution neural network and provides a novel Poisson denoising non-iterative algorithm. The rationality and the strength of the variance stability transformation scheme are researched by utilizing a joint learning strategy and a two-stage progressive learning strategy. Inheriting and expanding the structure and the advantages of a traditional variance stabilizing transformation scheme, and processing nonlinear learning mapping by three sub-networks through a network design and supervised learning method. The widening of the network greatly improves the denoising performance. Compared with the traditional iterative denoising method, the method has better denoising performance, and particularly has more excellent image recovery effect under the condition of low signal-to-noise ratio.
Drawings
FIG. 1 is a flow chart of the algorithm steps of the present invention;
FIG. 2 is a diagram of a training framework of the present invention;
FIG. 3 is a graph of the denoising result of the present invention at a peak value of 0.1; (a) is an original image; (b) is a noise map; (c) (d) (e) Poisson-NL, NLPCA and the reconstruction result map of the algorithm respectively;
FIG. 4 is a graph of the denoising result of the present invention at peak value 2; (a) is an original image; (b) is a noise map; (c) (d) (e) are Poisson-NL, NLPCA and the reconstructed result of the algorithm.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The embodiments described herein are only for explaining the technical solution of the present invention and are not limited to the present invention.
The specific steps of the method of the present invention are described below with reference to FIG. 1.
Step (a): based on the variance-stabilized transformation known as f is an observed image containing poisson noise, assuming that there is a transformation Φ that can make Φ f sparse and obey poisson distribution, the description that can be approximated by taylor's formula is:
Figure BDA0001799901210000051
wherein, O (Φ f)i) The remainder of the taylor formula, i.e., the deviation of the actual function value from the polynomial, is expressed. Intuitively, if we use convolution operators to model the phi transform, then giCan be approximately represented as the sum of some convolution operations. Therefore, we approximate the nonlinear square root operator with a convolution operation. For stabilization purposes in subnetwork 1, we generalize the above equation using convolutional layers and summation operations in approximation, namely:
Figure BDA0001799901210000052
specifically, the output of the input image after a convolution operator
Figure BDA0001799901210000053
Comprises the following steps:
Figure BDA0001799901210000054
wherein w is the theory of essentialsWeight of learning, b is offset, d1,d2The number of filters, input and output respectively, learning weights and biases under the training of the network has a great influence on the experimental result.
Conv (1) is the output of an image with all pixel values 1 after a convolution operation, and Conv (f) is the output of an input image containing Poisson noise after a convolution operation. The image with the pixel value of 1 and the input image containing the Poisson noise are respectively added together after convolution operation, and the processing of the sub-network 1 is a key different from the traditional Poisson denoising network. As is known, the conventional forward ansscomb transform can only accurately process poisson data with a large peak value, but the recovery effect is not good in the case of a small peak value, and the phi transform trained in the invention exists definitely and can well process input signals with any peak value. It is worth mentioning that as the peak decreases, the stabilization process becomes less accurate, requiring a more complex structure. Therefore, in the case of low snr with a peak value less than 20, the subnetwork 1 uses two consecutive convolutional layers (i.e. Conv (f)) to improve the performance, which not only makes the image recovery effect more outstanding, but also makes the whole training process more stable, i.e.:
g=Conv(1)+Conv(Conv(f))
the subnetwork 1 mainly simulates ansscomb transformation, converts random variables with poisson distribution into random variables with approximate standard gaussian distribution, and facilitates subsequent further processing.
Step (b): let the feature mapping of input g of subnetwork 2 be f (g), which has the following expression:
F(g)=w2σ(w1g)
wherein, f (g) is a complex function composed of a convolution operator and a ReLU activation function, w is a weight to be learned, σ represents a nonlinear function ReLU, and a ReLU unit is an activation layer of a network and is an indispensable part of the network, and is often set after the convolution layer, and a model after realizing sparseness through the ReLU can better mine relevant features and fit training data. The output of subnetwork 2 is h (g), which is expressed as a function of:
H(g)=F(g,{wi})+g
the residual function is easier to optimize, the degradation problem of the deep network can be solved, and the number of network layers can be greatly deepened.
Due to the strong nonlinearity of poisson noise, after the forward transformation of the sub-network 1, the output data is not purely gaussian-like but approximately gaussian-like with the same variance, so that a more flexible operator is needed to be used in the part of the sub-network 2 to assist the whole process. Therefore, unlike some existing algorithms for gaussian denoising using residual learning, the residual block of the subnetwork 2 only sets the convolution layer and the ReLU layer, and does not adopt the batch normalization operation, and some reliable experiments also prove that the batch normalization layer is not necessary.
In the network training process, the parameters of each layer in the subnetwork 2 are trained together with the parameters of the subnetwork 1 and the subnetwork 3, and experiments prove that the denoising effect of the joint training is better than the denoising effect of the separated two-step training.
Step (c): the sub-network 3 simulates the inverse transformation of the Anscombe transformation, so that the inverse transformation returns stable variance and denoised data to the original range, and corresponds to the sub-network 1;
output=Conv(1)+Conv(Conv(H(g)))
wherein, Conv (h (g)) is the result of passing the output of the sub-network 2 through two continuous convolution layers, such continuous convolution operation not only can improve the image recovery effect, but also can make the training process more stable.
In recent years, there has been a trend that a network of expanding the filter size and the number of channels is very advantageous for the image recovery task, and therefore the number of basic channels is set to 128 in each of three-segment sub-networks, which not only broadens the network but also improves the training efficiency.
At this point, the training of the network is all over. The technical scheme of the invention adopts different peak factors to evaluate the performance of the proposed method. The standard values of the various parameters in the experimental process are respectively set as follows: the convolution kernel size is 7 multiplied by 7, the number of filters is 128, the test image size is 256 multiplied by 256, and the sliding step size14, network depth 5 layers, momentum 0.9, learning rate initially set to 0.1, weight decay 10-4The model is trained by using a Caffe framework and is realized by an NVIDIA Titan X GPU. The quality of the reconstructed image is measured by using the peak signal-to-noise ratio (PSNR).
By combining the results and analysis, the poisson denoising method based on three-segment sub-network representation provided by the invention can ensure that poisson noise can be well removed at each peak value, has a more obvious effect especially under the condition of low peak value, has low algorithm complexity, is suitable for a deep neural network, and can be better applied in practice.
The foregoing merely represents preferred embodiments of the invention, which are described in some detail and detail, and therefore should not be construed as limiting the scope of the invention. It should be noted that, for those skilled in the art, various changes, modifications and substitutions can be made without departing from the spirit of the present invention, and these are all within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (4)

1. A Poisson denoising method based on three-segment sub-network representation is characterized in that: the method comprises the following steps:
step (a): after convolution processing is carried out on input images with all pixel values being 1, the input images are added with noisy images which are processed through two convolution operators to obtain an intermediate processing image 1;
step (b): processing the intermediate processing image 1 by a resnet module without a batch normalization layer to obtain an intermediate processing image 2;
step (c): and (c) adding the two convolutions to the graph with the pixel value of 1 to obtain the final result.
2. The poisson denoising method based on the three-segment sub-network representation as claimed in claim 1, wherein: the step (a) is as follows:
setting an input noisy image f to be calculated through a convolutionOutput after son
Figure FDA0003005428620000011
Comprises the following steps:
Figure FDA0003005428620000012
where w is the weight to be learned, b is the bias, d1,d2The filter numbers which are input and output respectively have great influence on the experimental result by learning weight and bias under the training of the network; the output g of subnetwork 1 is thus expressed approximately as:
g=Conv(1)+Conv(Conv(f))
wherein, Conv (1) is the output of the image with all pixel values of 1 after passing through one convolution layer, and Conv (f)) is the output of the image with noise after passing through two continuous convolution layers; the subnetwork 1 mainly simulates ansscomb transformation, converts random variables with poisson distribution into random variables with approximate standard gaussian distribution, and facilitates subsequent further processing.
3. The poisson denoising method based on the three-segment sub-network representation as claimed in claim 1, wherein: the step (b) is as follows:
due to strong nonlinearity of Poisson noise, the forward transformation of the first step and the denoising of the second step are not accurate and can only be approximately realized; therefore, the design of subnetwork 2 is not to remove pure gaussian noise, but an approximate gaussian noise removal; therefore, a more flexible operation operator is needed to assist the whole process, so that a batch normalization layer is not used in the residual learning network of the sub-network 2, and the idea is experimentally verified in other networks;
the sub-network 2 utilizes the idea of residual learning, and the feature mapping of the input g of the network is F (g), which is expressed as follows:
F(g)=w2σ(w1g)
wherein, F (g) is a composite function composed of convolution operator and ReLU activation function, sigma represents nonlinear function ReLU, ReLU unit is the activation layer of network and is the essential part of network, and is set after convolution layer; the output of subnetwork 2 is h (g), which is expressed as a function of:
H(g)=F(g,{wi})+g
the residual function is easier to optimize and can solve the degradation problem of the deep network, so that the number of network layers is greatly deepened.
4. The poisson denoising method based on the three-segment sub-network representation as claimed in claim 1, wherein: the step (c) is as follows:
the sub-network 3 simulates the inverse transformation of the Anscombe transformation, so that the inverse transformation returns stable variance and denoised data to the original range, and corresponds to the sub-network 1;
output=Conv(1)+Conv(Conv(H(g)))
wherein, Conv (H (g)) is the result of two continuous convolution layers on the output of the sub-network 2, such continuous convolution operation not only can improve the recovery effect of the image, but also can make the training process more stable.
CN201811078556.7A 2018-09-14 2018-09-14 Poisson denoising method based on three-segment sub-network representation Active CN109242798B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811078556.7A CN109242798B (en) 2018-09-14 2018-09-14 Poisson denoising method based on three-segment sub-network representation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811078556.7A CN109242798B (en) 2018-09-14 2018-09-14 Poisson denoising method based on three-segment sub-network representation

Publications (2)

Publication Number Publication Date
CN109242798A CN109242798A (en) 2019-01-18
CN109242798B true CN109242798B (en) 2021-07-20

Family

ID=65058526

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811078556.7A Active CN109242798B (en) 2018-09-14 2018-09-14 Poisson denoising method based on three-segment sub-network representation

Country Status (1)

Country Link
CN (1) CN109242798B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978778B (en) * 2019-03-06 2020-11-13 浙江工业大学 Convolutional neural network medical CT image denoising method based on residual learning
CN113111720B (en) * 2021-03-17 2024-06-11 浙江工业大学 Electromagnetic modulation signal denoising method and system based on deep learning

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875361A (en) * 2017-02-17 2017-06-20 深圳市唯特视科技有限公司 A kind of method that poisson noise is removed based on depth convolutional neural networks
CN108280811A (en) * 2018-01-23 2018-07-13 哈尔滨工业大学深圳研究生院 A kind of image de-noising method and system based on neural network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875361A (en) * 2017-02-17 2017-06-20 深圳市唯特视科技有限公司 A kind of method that poisson noise is removed based on depth convolutional neural networks
CN108280811A (en) * 2018-01-23 2018-07-13 哈尔滨工业大学深圳研究生院 A kind of image de-noising method and system based on neural network

Also Published As

Publication number Publication date
CN109242798A (en) 2019-01-18

Similar Documents

Publication Publication Date Title
Gupta et al. Image denoising with linear and non-linear filters: A review
CN110047055B (en) Infrared image detail enhancement and denoising method
Hoshyar et al. Comparing the performance of various filters on skin cancer images
Ravishankar et al. A survey on noise reduction techniques in medical images
CN109242798B (en) Poisson denoising method based on three-segment sub-network representation
Thakur et al. Agsdnet: Attention and gradient-based sar denoising network
Gopinathan et al. Wavelet and FFT Based Image Denoising Using Non-Linear Filters.
CN112750090A (en) Underwater image denoising method and system for improving wavelet threshold
Villar-Corrales et al. Deep learning architectural designs for super-resolution of noisy images
Jagatheeswari et al. A novel approach for contrast enhancement based on histogram equalization followed by median filter
Biradar et al. Speckle noise reduction using hybrid TMAV based fuzzy filter
Santosh et al. Efficient techniques for denoising of speckle and highly corrupted impulse noise images
Anju et al. Satellite image denoising using shearlet transform
Iqbal et al. Reduction of speckle noise in medical images using stationary wavelet transform and fuzzy logic
Cheng et al. Fast iterative adaptive reconstruction in low-dose CT imaging
Teku et al. Contrast Enhanced Low-light Visible and Infrared Image Fusion.
Nigam et al. A comparative study of thresholding techniques for image denoising
Ni et al. Speckle suppression for sar images based on adaptive shrinkage in contourlet domain
Charde A review on image denoising using wavelet transform and Median filter over AWGN channel
Sahu et al. Comparison of digital image denoising method using various Transforms on Satellite Imagery
Khan et al. Image denoising based on adaptive wavelet thresholding by using various shrinkage methods under different noise condition
Chen et al. Multi-stage directional median filter
Huang et al. High quality infrared images with novel algorithm for multi-noises removal
Patil et al. An approach to image denoising using autoencoders and spatial filters for Gaussian noise
Prasad et al. A Systematic Review of Noise Types, Denoising Methods, and Evaluation Metrics in Images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant