CN113724146A - Single-pixel imaging method based on plug-and-play prior - Google Patents

Single-pixel imaging method based on plug-and-play prior Download PDF

Info

Publication number
CN113724146A
CN113724146A CN202110795952.7A CN202110795952A CN113724146A CN 113724146 A CN113724146 A CN 113724146A CN 202110795952 A CN202110795952 A CN 202110795952A CN 113724146 A CN113724146 A CN 113724146A
Authority
CN
China
Prior art keywords
pixel
noise
under
image
undersampling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110795952.7A
Other languages
Chinese (zh)
Other versions
CN113724146B (en
Inventor
张军
田烨
付莹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202110795952.7A priority Critical patent/CN113724146B/en
Publication of CN113724146A publication Critical patent/CN113724146A/en
Application granted granted Critical
Publication of CN113724146B publication Critical patent/CN113724146B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a single-pixel imaging method based on plug-and-play prior, relates to a single-pixel imaging method under an undersampling condition based on plug-and-play prior, and belongs to the field of computational photography. The method combines the advantages of a model-based method and a depth learning-based method, implicitly replaces the prior of a modern image noise reduction algorithm into an imaging method, and improves the imaging quality of a single-pixel image under the undersampling condition. The invention provides a noise-free single-pixel imaging method under an undersampling condition and a single-pixel imaging method considering noise under the undersampling condition under the same concept of plug-and-play. The noise-free single-pixel imaging method under the undersampling condition can realize the reconstruction of a high-quality single-pixel image under the undersampling condition under the condition of not considering noise. The single-pixel imaging method considering noise under the undersampling condition can realize high-quality single-pixel image reconstruction under the undersampling condition under the condition of considering noise, and has robustness to the noise.

Description

Single-pixel imaging method based on plug-and-play prior
Technical Field
The invention relates to a single-pixel imaging method and a single-pixel imaging system, in particular to a single-pixel imaging method and a single-pixel imaging system under an undersampling condition based on plug-and-play prior, and belongs to the field of computational photography.
Background
The single-pixel imaging technology is a novel computational imaging technology, the technology utilizes structured light to modulate a target image, a single-pixel detector without spatial resolution capability obtains a one-dimensional measuring light signal, and an image is reconstructed according to the correlation between the structured light and the one-dimensional measuring light signal. Compared with the traditional two-dimensional pixel detector imaging method, the cost of the image sensor is reduced, the image measurement times can be reduced by combining a compressed sensing algorithm, and the image acquisition speed is improved. In recent years, the technology is increasingly applied to the fields of underwater imaging, three-dimensional imaging, remote sensing, target tracking, terahertz imaging and the like.
The current single-pixel imaging method includes two types, a model-based single-pixel imaging method and a Deep Learning (DL) -based single-pixel imaging method. Model-based single pixel imaging methods include Linear correlation methods (Linear correlation methods), Alternating projection methods (AP), and Compressive sensing methods (CS). Among them, the compressive sensing method is the most commonly used algorithm among model-based single-pixel algorithms. The compressive sensing method adds the prior knowledge of human about the image as a constraint to the design of the single-pixel image reconstruction algorithm. Under undersampling conditions, the compressed sensing method has proven to be the most effective method in model-based single-pixel imaging methods. Because the compressive sensing method is limited by the expressiveness of human manual creation prior, the reconstruction quality of the single-pixel image is rapidly reduced along with the reduction of the sampling rate, and the compressive sensing method is difficult to recover the single-pixel image under the condition of high noise.
In recent years, with the development of deep learning technology, a single-pixel imaging method based on deep learning appears, and the process of single-pixel imaging and restoration is simulated by training an end-to-end neural network. Although the single-pixel imaging method based on the deep learning improves the single-pixel imaging quality at an extremely low sampling rate, the effect of the single-pixel imaging method based on the deep learning is not obviously changed along with the increase of the sampling rate, and is even inferior to that of the single-pixel imaging method based on the model. Furthermore, the single-pixel imaging method based on deep learning has poor universality, and the trained network is limited by specific application. When the sampling rate of a single pixel, the image size and the imaging environment change, a new network needs to be retrained to adapt to a new task. In addition, the deep learning-based single-pixel imaging method is not robust to noise.
Therefore, from the demand of improving the quality of single-pixel imaging recovery under the undersampling condition, a single-pixel imaging method with good universality and robustness to noise is urgently needed.
Disclosure of Invention
The invention discloses a single-pixel imaging method based on plug-and-play prior, which aims to solve the technical problems that: the advantages of a model-based method and a depth learning-based method are combined, a modern image denoising algorithm is implicitly replaced into an imaging method as a priori, and the quality of single-pixel image imaging under an undersampling condition is improved. Under the same concept of plug and play, the invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition.
The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which is based on a Generalized Alternating Projection Algorithm (GAP), models a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity term and a prior term, combines Plug-and-Play (PnP), and regards the prior term sub-problem as an image noise reduction problem to be replaced by a modern image noise reduction Algorithm (such as a convolutional neural network with a noise reduction function in deep learning). By alternately and iteratively solving the two subproblems, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized without considering noise.
The invention discloses a single-pixel imaging Method considering noise under an undersampling condition, which is based on an Alternating Direction multiplier Algorithm (ADMM) and models a single-pixel image reconstruction problem as an optimization problem. Different from a noise-free single-pixel imaging method under an undersampling condition, the modeling of the optimization problem comprises a noise model. The optimization problem solving process is decomposed into two subproblems and a dual variable updating step according to fidelity items, prior items and dual variables, and the subproblems of the prior items are regarded as image noise reduction problems by combining Plug-and-Play (PnP) and replaced by modern image noise reduction algorithms (such as convolutional neural networks with noise reduction function in deep learning). By alternately and iteratively solving the two subproblems and the dual variable, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the robustness to the noise is realized.
The purpose of the invention is realized by the following technical scheme:
the invention discloses a noise-free single-pixel imaging method under an undersampling condition, which comprises the following steps of:
step 101: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized alternating projection Algorithm (GAP);
the noise-free single-pixel image imaging optimization model under the undersampling condition in step 102 is the optimization model described by the formula (1):
Figure BDA0003162767430000021
wherein, therein
Figure BDA0003162767430000022
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000023
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000024
For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
and (3) decomposing the optimization problem, namely solving the subproblems about the fidelity terms according to the framework of the generalized alternative projection algorithm. In particular, at known k time information vkOn the premise of (1), the solution is vkEuclidean projection on a linear manifold, as shown in equation (2):
xk+1＝vk+AT(AAT)-1(b-Avk) (2)
step 104: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
the alternative to the prior term optimization sub-problem solution described in step 104 is shown in equation (3):
vk+1＝Dσ(xk+1) (3)
wherein DσFor the image noise reducer, a convolutional neural network having a noise reduction function in deep learning may be employed. The selected convolutional neural network is an FFDNet network, a UNet network, an IRCNN network or a DRUNet network, but is not limited to the convolutional neural network. In order to obtain a higher quality single pixel reconstruction image, the convolutional neural network preferably adopts a DRUNet network.
Step 105: by alternately iterating the solution of the fidelity sub-problem and the replaced modern image noise reduction algorithm, high-quality single-pixel imaging under the condition of no noise under sampling can be realized without considering the noise.
And (3) alternately iterating the solution of the fidelity item-related subproblem and the replaced modern image noise reduction algorithm, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
The invention discloses a single-pixel imaging method considering noise under an undersampling condition, which comprises the following steps of:
step 201: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction Multiplier Algorithm (ADMM);
the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 selects the optimization model described in formula (4):
Figure BDA0003162767430000031
wherein, therein
Figure BDA0003162767430000032
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000033
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000034
For m measurements received by a single pixel detector. According to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
Figure BDA0003162767430000035
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
the optimization problem decomposition in step 203 is shown as formula (6):
Figure BDA0003162767430000036
Figure BDA0003162767430000037
Figure BDA0003162767430000041
wherein
Figure BDA0003162767430000042
Regarding the solution of the fidelity term sub-problem (6-1), according to the alternative direction multiplier algorithm ADMM, the solution can be:
xk+1＝(ATA+ρI)-1(ATb+ρvk) (7)
step 204: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
the prior term optimization sub-problem replacement described in step 204 is shown in equation (8):
Figure BDA0003162767430000043
wherein DσFor the image noise reducer, a convolutional neural network having a noise reduction function in deep learning may be employed. The selected convolutional neural network is an FFDNet network, a UNet network, an IRCNN network or a DRUNet network, but is not limited to the convolutional neural network. In order to obtain a higher quality single pixel reconstruction image, the convolutional neural network preferably adopts a DRUNet network.
Step 205: by alternately iterating the solution of the fidelity sub-problem, the replaced modern image denoising algorithm and the dual variable update, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering the noise, and the robustness to the noise is realized.
And (3) updating alternate iteration, namely, iterating formulas (7), (8) and (6-3) until the difference value of x and v is within a preset threshold range, to obtain a reconstructed single-pixel image, namely, realizing the high-quality single-pixel imaging under the condition of undersampling and considering noise.
Has the advantages that:
1. the single-pixel imaging method based on plug-and-play prior disclosed by the invention combines the advantages of a model-based optimization method and a learning-based deep learning method, and replaces the modern image denoising algorithm as prior implicitly into the imaging method, thereby improving the quality of single-pixel image imaging under the undersampling condition. Under the same concept of plug and play, the invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition.
2. The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which is based on a Generalized Alternating Projection Algorithm (GAP), models a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity term and a prior term, combines Plug-and-Play (PnP), and regards the prior term sub-problem as an image noise reduction problem to be replaced by a modern image noise reduction Algorithm (such as a convolutional neural network with a noise reduction function in deep learning). By alternately and iteratively solving the two subproblems, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized without considering noise.
3. The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which combines Plug-and-Play (PnP) and can implicitly insert a modern image noise reduction algorithm (such as a convolutional neural network with a noise reduction function in deep learning) into an optimization algorithm as a priori. The neural network with the noise reduction function can be trained in advance, when the size of a single-pixel image and the sampling rate are changed, the network does not need to be retrained, and the universality is good.
4. The invention discloses a single-pixel imaging Method considering noise under an undersampling condition, which is based on an Alternating Direction Multiplier Algorithm (ADMM) and models a single-pixel image reconstruction problem as an optimization problem. Different from a noise-free single-pixel imaging method under an undersampling condition, the modeling of the optimization problem comprises a noise model. The optimization problem solving process is decomposed into two subproblems and a dual variable updating step according to fidelity items, prior items and dual variables, and the subproblems of the prior items are regarded as image noise reduction problems by combining Plug-and-Play (PnP) and replaced by modern image noise reduction algorithms (such as convolutional neural networks with noise reduction function in deep learning). By alternately and iteratively solving the two subproblems and the dual variable, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the robustness to the noise is realized.
5. The single-pixel imaging method considering noise under the undersampling condition, disclosed by the invention, combines Plug-and-Play (PnP) and can implicitly insert a modern image noise reduction algorithm (such as a convolutional neural network with a noise reduction function in deep learning) into an optimization algorithm as a priori. The neural network with the noise reduction function can be trained in advance, when the size of a single-pixel image, the sampling rate and the environmental noise are changed, the network does not need to be retrained, and the universality is good.
Drawings
FIG. 1 is a schematic diagram of an optical path of a single-pixel imaging method based on plug-and-play apriori according to the method of the present invention.
Fig. 2 is a flow chart of a noise-free single-pixel imaging method under the undersampling condition according to the method of the present invention.
FIG. 3 is a flow chart of a single pixel imaging method under undersampling conditions that accounts for noise in accordance with the method of the present invention.
FIG. 4 is a comparison graph of simulation results of single pixel imaging with the method of the present invention and other methods at different sampling rates without noise.
FIG. 5 is a comparison graph of simulation results of single pixel imaging with the method of the present invention and other methods at a fixed sampling rate and different noise levels.
Fig. 6 is a comparison graph of the results of single-pixel imaging in real experiments of the method of the present invention and other methods, in which fig. 6(a) is the result of single-pixel imaging of the "Ghost" target image (64 × 64) under different sampling rates and different algorithms, and fig. 6(b) is the result of single-pixel imaging of the "SPI" target image (64 × 64) under different sampling rates and different algorithms.
Detailed Description
For a better understanding of the objects and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples. However, the present invention is not limited to the following examples.
Example 1:
as shown in fig. 2, the noise-free single-pixel imaging method under the under-sampling condition disclosed in this embodiment includes the following steps:
step 101: the built single-pixel imaging light path is as shown in fig. 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are coded and modulated and then are smaller than the Nyquist sampling theorem;
specifically, the projector projects structured light patterns with random gray scales to a target image according to a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulated structured light patterns are used as a one-dimensional measured value of a single pixel. Wherein the relationship between the target image, the structured light pattern, and the single pixel measurement is:
Ax＝b (1)
wherein, therein
Figure BDA0003162767430000061
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000062
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000063
For m measurements received by a single pixel detector. In the under-sampling case, the number m of the received measured values is lower than the Nyquist sampling theorem, soThe pixel imaging problem is highly ill-conditioned and irreversible.
Step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized Alternating Projection Algorithm (GAP);
specifically, single pixel imaging modeling is an optimization problem as follows:
Figure BDA0003162767430000064
where f (x) is a fidelity term, depending on the single pixel imaging model, to ensure that the relationship between the reconstructed single pixel image and the measured values received by the single pixel detector conforms to the single pixel imaging principle. g (x) is a prior term that contains some prior information of the image to be reconstructed. Combining with the generalized alternative projection algorithm, introducing an auxiliary parameter v to convert the above unconstrained optimization problem into a constrained optimization problem, i.e. the noise-free single-pixel reconstruction optimization model under the undersampling condition described in step 102, as shown in formula (3):
Figure BDA0003162767430000065
wherein, therein
Figure BDA0003162767430000066
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000067
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000068
For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
the optimization questions of step 103And (4) problem decomposition, solving the subproblems about the fidelity terms according to the framework of the generalized alternative projection algorithm. In particular, at known k time information vkOn the premise of v iskEuclidean projection on a linear manifold, as shown in equation (4):
xk+1＝vk+AT(AAT)-1(b-Avk) (4)
step 104: training the DRUNet network by using a natural image with noise to enable the DRUNet network to have the noise reduction capability;
specifically, the DRUNet network is trained on a large data set consisting of a DIV2K data set, a Flick2K data set, a Wateloo expansion data set and a BSD data set, so that the DRUNet network has the noise reduction capability.
Step 105: combining Plug-and-Play (PnP), regarding the optimization sub-problem about the prior item as an image denoising problem, and replacing the solution of the optimization sub-problem about the prior item with the DRUNet network trained in the step 104;
the alternative to the prior term optimization sub-problem solution described in step 105 is shown in equation (5):
vk+1＝Dσ(xk+1) (5)
wherein DσFor the image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 106: by alternately iterating the solution of the fidelity sub-problem and the replaced convolutional neural network, high-quality single-pixel imaging under the condition of no noise under the undersampling condition can be realized without considering the noise.
And (3) alternately iterating the solution of the fidelity item-related subproblem and the convolutional neural network after replacement, namely iterating the formulas (4) and (5) until the algorithm converges, namely the difference value of x and v is in a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
Example 2:
as shown in fig. 3, the single-pixel imaging method considering noise under the under-sampling condition disclosed in this embodiment includes the following steps:
step 201: the built single-pixel imaging light path is as shown in fig. 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are coded and modulated and then are smaller than the Nyquist sampling theorem;
specifically, the projector projects structured light patterns with random gray scales to a target image according to a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulated structured light patterns are used as a one-dimensional measured value of a single pixel. Wherein the relationship between the target image, the structured light pattern, and the single pixel measurement is:
Ax＝b (6)
wherein, therein
Figure BDA0003162767430000071
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000072
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000073
For m measurements received by a single pixel detector. In the undersampling case, the number of measurements received, m, is lower than the nyquist sampling theorem, so the single-pixel imaging problem is highly ill-conditioned and irreversible.
Step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction multiplier Algorithm (ADMM);
specifically, single pixel imaging modeling is an optimization problem as follows:
Figure BDA0003162767430000074
where f (x) is a fidelity term, depending on the single pixel imaging model, to ensure that the relationship between the reconstructed single pixel image and the measured values received by the single pixel detector conforms to the single pixel imaging principle. g (x) is a prior term that contains some prior information of the image to be reconstructed. By combining with the alternative direction multiplier algorithm, introducing an auxiliary parameter v to convert the above unconstrained optimization problem into a constrained optimization problem, that is, the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 is as shown in formula (8):
Figure BDA0003162767430000075
wherein, therein
Figure BDA0003162767430000076
And n is the number of pixels of the image to be restored.
Figure BDA0003162767430000077
The structured light patterns are random gray scale, and m is the number of the structured light patterns.
Figure BDA0003162767430000078
For m measurements received by a single pixel detector. According to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
Figure BDA0003162767430000079
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
the optimization problem decomposition in step 203 is shown as formula (10):
Figure BDA0003162767430000081
Figure BDA0003162767430000082
Figure BDA0003162767430000083
wherein
Figure BDA0003162767430000084
Regarding the solution of the fidelity term sub-problem (10-1), according to the alternative direction multiplier algorithm ADMM, the solution can be:
xk+1＝(ATA+ρI)-1(ATb+ρvk) (11)
step 204: training the DRUNet network by using a natural image with noise to enable the DRUNet network to have the noise reduction capability;
specifically, the DRUNet network is trained on a large data set consisting of a DIV2K data set, a Flick2K data set, a Wateloo expansion data set and a BSD data set, so that the DRUNet network has the noise reduction capability.
Step 205: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image denoising problem, and replacing the solution of the optimization sub-problem of the prior item with the DRUNet network trained in step 204;
the alternative to the prior term optimization sub-problem solution described in step 205 is shown in equation (12):
Figure BDA0003162767430000085
wherein DσFor the image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 206: by alternately iterating the solution of the fidelity sub-problem, the convolutional neural network after replacement and the dual variable update, the reconstruction of a high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the noise robustness is realized.
And (3) updating alternate iteration, namely iterating equations (11), (12) and (10-3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, to the solution of the fidelity sub-problem, the convolutional neural network after replacement and the dual variable, so as to obtain a reconstructed single-pixel image, namely to realize the high-quality single-pixel imaging under the condition of undersampling and considering noise.
The effect of the embodiment is verified from both simulation and experiment.
1. Simulation result
In order to verify the accuracy of the noise-free single-pixel imaging method on image reconstruction under the undersampling condition, the image in the Set12 data Set is Set as a target image, and under the condition that the measurement noise is not considered, the reconstruction results of the noise-free single-pixel imaging method under the undersampling condition and the noise-free single-pixel imaging method under the undersampling condition, which are provided by the invention, under different sampling rates are compared with the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates. In order to quantitatively measure the effect of different single-pixel imaging methods, Peak signal to noise ratio (PSNR) and Structural Similarity (SSIM) are used to measure the spatial quality and visual effect of the image reconstruction result. For each method, the peak signal-to-noise ratio PSNR and the structural similarity SSIM of all reconstructed images in the data set are averaged. The image size was set to 64 x 64 and the sampling rate was set from 0.2 to 0.9.
TABLE 1 Single Pixel reconstruction results (average PSNR) on Set12 dataset
Figure BDA0003162767430000091
Table 2 single pixel reconstruction results (average SSIM) on Set12 dataset
Figure BDA0003162767430000092
As can be seen from tables 1 and 2, the proposed noise-free single-pixel imaging method under the undersampling condition, i.e., the GAP-prunet algorithm, is significantly better than other algorithms in terms of spatial quality and visual effect. To more clearly contrast the differences between the different single pixel imaging methods, the "Cameraman" image in the Set12 dataset was reconstructed at different sampling rates in different ways as shown in FIG. 4.
In order to verify the accuracy of the single-pixel imaging method considering noise under the undersampling condition on image reconstruction, images in a Set12 data Set are Set as target images, and under the condition of considering different measurement noise levels, the reconstruction results of the single-pixel imaging method considering noise under the undersampling condition and the noise under the undersampling condition, which are provided by the invention, under a fixed sampling rate are compared with the reconstruction results of the single-pixel imaging method based on a model and the single-pixel imaging method based on learning under the fixed sampling rate. In order to quantitatively measure the effect of different single-pixel imaging methods, the spatial quality and the visual effect of an image reconstruction result are measured by using Peak signal to noise ratio (PSNR) and Structural Similarity (SSIM). For each method, the peak signal-to-noise ratio PSNR and the structural similarity SSIM of all reconstructed images in the data set are averaged. The image size was set to 64 x 64 and the sampling rate was set to 0.5.
TABLE 3 Single Pixel reconstruction results (average PSNR) on Set12 data Set
Figure BDA0003162767430000101
Table 4 single pixel reconstruction results (average SSIM) on Set12 dataset
Figure BDA0003162767430000102
Figure BDA0003162767430000111
As can be seen from tables 3 and 4, the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly superior to other algorithms in terms of spatial quality and visual effect. To more clearly contrast the differences between the different single pixel imaging methods, the "Cameraman" image in the Set12 dataset was reconstructed under different methods at different noise levels as shown in FIG. 5.
2. Results of the experiment
In order to verify the performance of the noise-free single-pixel imaging method under the undersampling condition and the noise-considered single-pixel imaging method under the undersampling condition, which are provided by the invention, on the real single-pixel measurement value, an optical path is constructed according to the optical path schematic diagram of the single-pixel imaging method in fig. 1, wherein the model of the projector is Panasonic (X416C XGA), and the model of the single-pixel detector is Thorlabs (PDA100a 2). The reconstruction results of the noise-free single-pixel imaging method under the undersampling condition and the noise-considered single-pixel imaging method under the undersampling condition, which are provided by the invention, under different sampling rates are in a pair form as shown in fig. 6 with the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates.
Fig. 6(a) shows the result of single-pixel imaging of the "Ghost" target image (64 × 64) under different sampling rates and different algorithms, and it can be seen from fig. 6(a) that the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly better than other algorithms in terms of spatial quality and visual effect due to the fact that non-negligible noise exists in the actual measurement. Fig. 6(b) is a single-pixel imaging result of the "SPI" target image (64 × 64) under different sampling rates and different algorithms, and further proves that the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly superior to other algorithms in terms of spatial quality and visual effect.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. The noiseless single-pixel imaging method under the undersampling condition is characterized in that: comprises the following steps of (a) carrying out,
step 101: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized Alternating Projection Algorithm (GAP);
step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
step 104: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
step 105: by alternately iterating the solution of the fidelity sub-problem and the replaced modern image noise reduction algorithm, high-quality single-pixel imaging under the condition of no noise under sampling can be realized without considering the noise.
2. A method of noiseless single pixel imaging under undersampling as claimed in claim 1 wherein: the noise-free single-pixel image imaging optimization model under the undersampling condition in step 102 is the optimization model described by the formula (1):
Figure FDA0003162767420000011
wherein, therein
Figure FDA0003162767420000012
The image is a target image, and n is the number of pixels of the image to be restored;
Figure FDA0003162767420000013
the structured light patterns are random gray scales, and m is the number of the structured light patterns;
Figure FDA0003162767420000014
for m measurements received by a single pixel detector.
3. A method of noiseless single pixel imaging under undersampling as claimed in claim 2, characterized in that: 103, decomposing the optimization problem, and solving a subproblem about the fidelity term according to a frame of a generalized alternative projection algorithm; at known time k information vkOn the premise of (1), the solution is vkEuclidean projection on a linear manifold, as shown in equation (2):
xk+1＝vk+AT(AAT)-1(b-Avk) (2)
4. a method of noiseless single pixel imaging under undersampling conditions as claimed in claim 1 or 2, characterized by: the alternative to the prior term optimization sub-problem solution described in step 104 is shown in equation (3):
vk+1＝Dσ(xk+1) (3)
wherein DσA convolution neural network with a noise reduction function in deep learning is adopted as an image noise reducer.
5. A method of noiseless single pixel imaging under undersampling conditions as claimed in claim 1 or 2, characterized by: and (3) alternately iterating the solution of the fidelity item-related subproblem and the replaced modern image noise reduction algorithm, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
6. The single-pixel imaging method considering noise under the undersampling condition is characterized in that: comprises the following steps of (a) carrying out,
step 201: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction Multiplier Algorithm (ADMM);
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
step 204: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
step 205: by alternately iterating the solution of the fidelity sub-problem, the replaced modern image denoising algorithm and the dual variable update, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering the noise, and the robustness to the noise is realized.
7. A method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 6, wherein: the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 selects the optimization model described in formula (4):
Figure FDA0003162767420000021
wherein, therein
Figure FDA0003162767420000022
The image is a target image, and n is the number of pixels of the image to be restored;
Figure FDA0003162767420000023
is a structured light pattern of random gray scale,m is the number of structured light patterns;
Figure FDA0003162767420000024
m measured values received by a single-pixel detector; according to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
Figure FDA0003162767420000025
8. a method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 7, wherein: the optimization problem decomposition in step 203 is shown as formula (6):
Figure FDA0003162767420000026
Figure FDA0003162767420000027
Figure FDA0003162767420000028
wherein
Figure FDA0003162767420000029
Regarding the solution of the fidelity item subproblem (6-1), according to the alternative direction multiplier algorithm ADMM, the solution is:
xk+1＝(ATA+pI)-1(ATb+pvk) (7)
9. a method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 8, wherein: the prior term optimization sub-problem replacement described in step 204 is shown in equation (8):
Figure FDA00031627674200000210
wherein DσA convolution neural network with a noise reduction function in deep learning is adopted as an image noise reducer.
10. A method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 9, wherein: and (3) updating alternate iteration, namely, iterating formulas (7), (8) and (6-3) until the difference value of x and v is within a preset threshold range, to obtain a reconstructed single-pixel image, namely, realizing the high-quality single-pixel imaging under the condition of undersampling and considering noise.
CN202110795952.7A 2021-07-14 2021-07-14 Single-pixel imaging method based on plug-and-play prior Active CN113724146B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110795952.7A CN113724146B (en) 2021-07-14 2021-07-14 Single-pixel imaging method based on plug-and-play prior

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110795952.7A CN113724146B (en) 2021-07-14 2021-07-14 Single-pixel imaging method based on plug-and-play prior

Publications (2)

Publication Number Publication Date
CN113724146A true CN113724146A (en) 2021-11-30
CN113724146B CN113724146B (en) 2024-06-04

Family

ID=78673287

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110795952.7A Active CN113724146B (en) 2021-07-14 2021-07-14 Single-pixel imaging method based on plug-and-play prior

Country Status (1)

Country Link
CN (1) CN113724146B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117408875A (en) * 2023-12-13 2024-01-16 阿尔玻科技有限公司 Method, storage medium and apparatus for reconstructing compressed ultrafast photographic images
CN117876837A (en) * 2024-03-11 2024-04-12 北京理工大学 Near infrared single-pixel imaging method and system based on depth expansion network

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020069399A1 (en) * 1999-08-16 2002-06-06 Z-Force Corporation System of reusable software parts and methods of use
US20200011951A1 (en) * 2018-07-06 2020-01-09 The Board Of Trustees Of The Leland Stanford Junior University Deep Learning based Methods to Accelerate Multi-Spectral Imaging
CN111259991A (en) * 2020-02-28 2020-06-09 南京理工大学 Under-sampling single-pixel imaging target identification method in noise environment
US20200284883A1 (en) * 2019-03-08 2020-09-10 Osram Gmbh Component for a lidar sensor system, lidar sensor system, lidar sensor device, method for a lidar sensor system and method for a lidar sensor device
CN112203068A (en) * 2020-09-30 2021-01-08 中国科学院长春光学精密机械与物理研究所 Single-pixel imaging method, system, device and medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020069399A1 (en) * 1999-08-16 2002-06-06 Z-Force Corporation System of reusable software parts and methods of use
US20200011951A1 (en) * 2018-07-06 2020-01-09 The Board Of Trustees Of The Leland Stanford Junior University Deep Learning based Methods to Accelerate Multi-Spectral Imaging
US20200284883A1 (en) * 2019-03-08 2020-09-10 Osram Gmbh Component for a lidar sensor system, lidar sensor system, lidar sensor device, method for a lidar sensor system and method for a lidar sensor device
CN111259991A (en) * 2020-02-28 2020-06-09 南京理工大学 Under-sampling single-pixel imaging target identification method in noise environment
CN112203068A (en) * 2020-09-30 2021-01-08 中国科学院长春光学精密机械与物理研究所 Single-pixel imaging method, system, device and medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
徐树奎;涂丹;李国辉;张军;: "计算摄影综述", 计算机应用研究, no. 11, 15 November 2010 (2010-11-15) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117408875A (en) * 2023-12-13 2024-01-16 阿尔玻科技有限公司 Method, storage medium and apparatus for reconstructing compressed ultrafast photographic images
CN117876837A (en) * 2024-03-11 2024-04-12 北京理工大学 Near infrared single-pixel imaging method and system based on depth expansion network
CN117876837B (en) * 2024-03-11 2024-06-07 北京理工大学 Near infrared single-pixel imaging method and system based on depth expansion network

Also Published As

Publication number Publication date
CN113724146B (en) 2024-06-04

Similar Documents

Publication Publication Date Title
Gehrig et al. Combining events and frames using recurrent asynchronous multimodal networks for monocular depth prediction
US10593021B1 (en) Motion deblurring using neural network architectures
CN104137150B (en) Motion compensation in range finding imaging
CN113724146B (en) Single-pixel imaging method based on plug-and-play prior
CN101373135B (en) Method and apparatus for measuring three-dimensional curved surface based on instantaneous accidental light irradiation
CN113837938A (en) Super-resolution method for reconstructing potential image based on dynamic vision sensor
CN109945802A (en) A kind of structural light three-dimensional measurement method
CN104243837A (en) Vibration detection and remote sensing image recovery method based on single-exposure video reconstruction
CN113487739A (en) Three-dimensional reconstruction method and device, electronic equipment and storage medium
Nakashima et al. Learning to drop points for lidar scan synthesis
Wang et al. Underwater self-supervised monocular depth estimation and its application in image enhancement
CN113643220B (en) Physical information driven infrared image turbulence effect suppression method and system
CN114757842A (en) Color calculation ghost imaging method based on plug-and-play generalized alternative projection algorithm
CN112468791A (en) Light intensity measurement iterative imaging method based on single-pixel detection
CN117036442A (en) Robust monocular depth completion method, system and storage medium
CN110365966B (en) Video quality evaluation method and device based on window
CN117132704A (en) Three-dimensional reconstruction method of dynamic structured light, system and computing equipment thereof
CN110926611A (en) Noise suppression method applied to compressed sensing spectral imaging system
CN113240610B (en) Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism
CN103115673B (en) Acquisition method for high signal-to-noise ratio variable illumination and light field reuse
CN105869162A (en) Active detection imaging data fusion method based on three-edge confidence measure
CN118247418B (en) Method for reconstructing nerve radiation field by using small quantity of blurred images
Mao et al. Time-of-flight camera via a single-pixel correlation image sensor
CN117710416B (en) Fluid light flow velocity measurement method and system for complex illumination change scene
Seo et al. A study on selective dehazing system based on haze detection network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant