CN113724146A - Single-pixel imaging method based on plug-and-play prior - Google Patents
Single-pixel imaging method based on plug-and-play prior Download PDFInfo
- Publication number
- CN113724146A CN113724146A CN202110795952.7A CN202110795952A CN113724146A CN 113724146 A CN113724146 A CN 113724146A CN 202110795952 A CN202110795952 A CN 202110795952A CN 113724146 A CN113724146 A CN 113724146A
- Authority
- CN
- China
- Prior art keywords
- pixel
- noise
- under
- image
- undersampling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 136
- 238000000034 method Methods 0.000 claims abstract description 42
- 230000009467 reduction Effects 0.000 claims abstract description 42
- 238000005457 optimization Methods 0.000 claims description 68
- 238000005070 sampling Methods 0.000 claims description 37
- 238000013135 deep learning Methods 0.000 claims description 21
- 230000009977 dual effect Effects 0.000 claims description 16
- 238000005259 measurement Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 9
- 239000003638 chemical reducing agent Substances 0.000 claims description 6
- 238000013528 artificial neural network Methods 0.000 claims description 5
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 241001270131 Agaricus moelleri Species 0.000 claims description 3
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 description 22
- 230000000007 visual effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000002427 irreversible effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a single-pixel imaging method based on plug-and-play prior, relates to a single-pixel imaging method under an undersampling condition based on plug-and-play prior, and belongs to the field of computational photography. The method combines the advantages of a model-based method and a depth learning-based method, implicitly replaces the prior of a modern image noise reduction algorithm into an imaging method, and improves the imaging quality of a single-pixel image under the undersampling condition. The invention provides a noise-free single-pixel imaging method under an undersampling condition and a single-pixel imaging method considering noise under the undersampling condition under the same concept of plug-and-play. The noise-free single-pixel imaging method under the undersampling condition can realize the reconstruction of a high-quality single-pixel image under the undersampling condition under the condition of not considering noise. The single-pixel imaging method considering noise under the undersampling condition can realize high-quality single-pixel image reconstruction under the undersampling condition under the condition of considering noise, and has robustness to the noise.
Description
Technical Field
The invention relates to a single-pixel imaging method and a single-pixel imaging system, in particular to a single-pixel imaging method and a single-pixel imaging system under an undersampling condition based on plug-and-play prior, and belongs to the field of computational photography.
Background
The single-pixel imaging technology is a novel computational imaging technology, the technology utilizes structured light to modulate a target image, a single-pixel detector without spatial resolution capability obtains a one-dimensional measuring light signal, and an image is reconstructed according to the correlation between the structured light and the one-dimensional measuring light signal. Compared with the traditional two-dimensional pixel detector imaging method, the cost of the image sensor is reduced, the image measurement times can be reduced by combining a compressed sensing algorithm, and the image acquisition speed is improved. In recent years, the technology is increasingly applied to the fields of underwater imaging, three-dimensional imaging, remote sensing, target tracking, terahertz imaging and the like.
The current single-pixel imaging method includes two types, a model-based single-pixel imaging method and a Deep Learning (DL) -based single-pixel imaging method. Model-based single pixel imaging methods include Linear correlation methods (Linear correlation methods), Alternating projection methods (AP), and Compressive sensing methods (CS). Among them, the compressive sensing method is the most commonly used algorithm among model-based single-pixel algorithms. The compressive sensing method adds the prior knowledge of human about the image as a constraint to the design of the single-pixel image reconstruction algorithm. Under undersampling conditions, the compressed sensing method has proven to be the most effective method in model-based single-pixel imaging methods. Because the compressive sensing method is limited by the expressiveness of human manual creation prior, the reconstruction quality of the single-pixel image is rapidly reduced along with the reduction of the sampling rate, and the compressive sensing method is difficult to recover the single-pixel image under the condition of high noise.
In recent years, with the development of deep learning technology, a single-pixel imaging method based on deep learning appears, and the process of single-pixel imaging and restoration is simulated by training an end-to-end neural network. Although the single-pixel imaging method based on the deep learning improves the single-pixel imaging quality at an extremely low sampling rate, the effect of the single-pixel imaging method based on the deep learning is not obviously changed along with the increase of the sampling rate, and is even inferior to that of the single-pixel imaging method based on the model. Furthermore, the single-pixel imaging method based on deep learning has poor universality, and the trained network is limited by specific application. When the sampling rate of a single pixel, the image size and the imaging environment change, a new network needs to be retrained to adapt to a new task. In addition, the deep learning-based single-pixel imaging method is not robust to noise.
Therefore, from the demand of improving the quality of single-pixel imaging recovery under the undersampling condition, a single-pixel imaging method with good universality and robustness to noise is urgently needed.
Disclosure of Invention
The invention discloses a single-pixel imaging method based on plug-and-play prior, which aims to solve the technical problems that: the advantages of a model-based method and a depth learning-based method are combined, a modern image denoising algorithm is implicitly replaced into an imaging method as a priori, and the quality of single-pixel image imaging under an undersampling condition is improved. Under the same concept of plug and play, the invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition.
The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which is based on a Generalized Alternating Projection Algorithm (GAP), models a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity term and a prior term, combines Plug-and-Play (PnP), and regards the prior term sub-problem as an image noise reduction problem to be replaced by a modern image noise reduction Algorithm (such as a convolutional neural network with a noise reduction function in deep learning). By alternately and iteratively solving the two subproblems, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized without considering noise.
The invention discloses a single-pixel imaging Method considering noise under an undersampling condition, which is based on an Alternating Direction multiplier Algorithm (ADMM) and models a single-pixel image reconstruction problem as an optimization problem. Different from a noise-free single-pixel imaging method under an undersampling condition, the modeling of the optimization problem comprises a noise model. The optimization problem solving process is decomposed into two subproblems and a dual variable updating step according to fidelity items, prior items and dual variables, and the subproblems of the prior items are regarded as image noise reduction problems by combining Plug-and-Play (PnP) and replaced by modern image noise reduction algorithms (such as convolutional neural networks with noise reduction function in deep learning). By alternately and iteratively solving the two subproblems and the dual variable, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the robustness to the noise is realized.
The purpose of the invention is realized by the following technical scheme:
the invention discloses a noise-free single-pixel imaging method under an undersampling condition, which comprises the following steps of:
step 101: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized alternating projection Algorithm (GAP);
the noise-free single-pixel image imaging optimization model under the undersampling condition in step 102 is the optimization model described by the formula (1):
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
and (3) decomposing the optimization problem, namely solving the subproblems about the fidelity terms according to the framework of the generalized alternative projection algorithm. In particular, at known k time information vkOn the premise of (1), the solution is vkEuclidean projection on a linear manifold, as shown in equation (2):
xk+1=vk+AT(AAT)-1(b-Avk) (2)
step 104: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
the alternative to the prior term optimization sub-problem solution described in step 104 is shown in equation (3):
vk+1=Dσ(xk+1) (3)
wherein DσFor the image noise reducer, a convolutional neural network having a noise reduction function in deep learning may be employed. The selected convolutional neural network is an FFDNet network, a UNet network, an IRCNN network or a DRUNet network, but is not limited to the convolutional neural network. In order to obtain a higher quality single pixel reconstruction image, the convolutional neural network preferably adopts a DRUNet network.
Step 105: by alternately iterating the solution of the fidelity sub-problem and the replaced modern image noise reduction algorithm, high-quality single-pixel imaging under the condition of no noise under sampling can be realized without considering the noise.
And (3) alternately iterating the solution of the fidelity item-related subproblem and the replaced modern image noise reduction algorithm, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
The invention discloses a single-pixel imaging method considering noise under an undersampling condition, which comprises the following steps of:
step 201: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction Multiplier Algorithm (ADMM);
the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 selects the optimization model described in formula (4):
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector. According to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
the optimization problem decomposition in step 203 is shown as formula (6):
whereinRegarding the solution of the fidelity term sub-problem (6-1), according to the alternative direction multiplier algorithm ADMM, the solution can be:
xk+1=(ATA+ρI)-1(ATb+ρvk) (7)
step 204: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
the prior term optimization sub-problem replacement described in step 204 is shown in equation (8):
wherein DσFor the image noise reducer, a convolutional neural network having a noise reduction function in deep learning may be employed. The selected convolutional neural network is an FFDNet network, a UNet network, an IRCNN network or a DRUNet network, but is not limited to the convolutional neural network. In order to obtain a higher quality single pixel reconstruction image, the convolutional neural network preferably adopts a DRUNet network.
Step 205: by alternately iterating the solution of the fidelity sub-problem, the replaced modern image denoising algorithm and the dual variable update, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering the noise, and the robustness to the noise is realized.
And (3) updating alternate iteration, namely, iterating formulas (7), (8) and (6-3) until the difference value of x and v is within a preset threshold range, to obtain a reconstructed single-pixel image, namely, realizing the high-quality single-pixel imaging under the condition of undersampling and considering noise.
Has the advantages that:
1. the single-pixel imaging method based on plug-and-play prior disclosed by the invention combines the advantages of a model-based optimization method and a learning-based deep learning method, and replaces the modern image denoising algorithm as prior implicitly into the imaging method, thereby improving the quality of single-pixel image imaging under the undersampling condition. Under the same concept of plug and play, the invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition.
2. The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which is based on a Generalized Alternating Projection Algorithm (GAP), models a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity term and a prior term, combines Plug-and-Play (PnP), and regards the prior term sub-problem as an image noise reduction problem to be replaced by a modern image noise reduction Algorithm (such as a convolutional neural network with a noise reduction function in deep learning). By alternately and iteratively solving the two subproblems, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized without considering noise.
3. The invention discloses a noise-free single-pixel imaging method under an undersampling condition, which combines Plug-and-Play (PnP) and can implicitly insert a modern image noise reduction algorithm (such as a convolutional neural network with a noise reduction function in deep learning) into an optimization algorithm as a priori. The neural network with the noise reduction function can be trained in advance, when the size of a single-pixel image and the sampling rate are changed, the network does not need to be retrained, and the universality is good.
4. The invention discloses a single-pixel imaging Method considering noise under an undersampling condition, which is based on an Alternating Direction Multiplier Algorithm (ADMM) and models a single-pixel image reconstruction problem as an optimization problem. Different from a noise-free single-pixel imaging method under an undersampling condition, the modeling of the optimization problem comprises a noise model. The optimization problem solving process is decomposed into two subproblems and a dual variable updating step according to fidelity items, prior items and dual variables, and the subproblems of the prior items are regarded as image noise reduction problems by combining Plug-and-Play (PnP) and replaced by modern image noise reduction algorithms (such as convolutional neural networks with noise reduction function in deep learning). By alternately and iteratively solving the two subproblems and the dual variable, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the robustness to the noise is realized.
5. The single-pixel imaging method considering noise under the undersampling condition, disclosed by the invention, combines Plug-and-Play (PnP) and can implicitly insert a modern image noise reduction algorithm (such as a convolutional neural network with a noise reduction function in deep learning) into an optimization algorithm as a priori. The neural network with the noise reduction function can be trained in advance, when the size of a single-pixel image, the sampling rate and the environmental noise are changed, the network does not need to be retrained, and the universality is good.
Drawings
FIG. 1 is a schematic diagram of an optical path of a single-pixel imaging method based on plug-and-play apriori according to the method of the present invention.
Fig. 2 is a flow chart of a noise-free single-pixel imaging method under the undersampling condition according to the method of the present invention.
FIG. 3 is a flow chart of a single pixel imaging method under undersampling conditions that accounts for noise in accordance with the method of the present invention.
FIG. 4 is a comparison graph of simulation results of single pixel imaging with the method of the present invention and other methods at different sampling rates without noise.
FIG. 5 is a comparison graph of simulation results of single pixel imaging with the method of the present invention and other methods at a fixed sampling rate and different noise levels.
Fig. 6 is a comparison graph of the results of single-pixel imaging in real experiments of the method of the present invention and other methods, in which fig. 6(a) is the result of single-pixel imaging of the "Ghost" target image (64 × 64) under different sampling rates and different algorithms, and fig. 6(b) is the result of single-pixel imaging of the "SPI" target image (64 × 64) under different sampling rates and different algorithms.
Detailed Description
For a better understanding of the objects and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings and examples. However, the present invention is not limited to the following examples.
Example 1:
as shown in fig. 2, the noise-free single-pixel imaging method under the under-sampling condition disclosed in this embodiment includes the following steps:
step 101: the built single-pixel imaging light path is as shown in fig. 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are coded and modulated and then are smaller than the Nyquist sampling theorem;
specifically, the projector projects structured light patterns with random gray scales to a target image according to a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulated structured light patterns are used as a one-dimensional measured value of a single pixel. Wherein the relationship between the target image, the structured light pattern, and the single pixel measurement is:
Ax=b (1)
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector. In the under-sampling case, the number m of the received measured values is lower than the Nyquist sampling theorem, soThe pixel imaging problem is highly ill-conditioned and irreversible.
Step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized Alternating Projection Algorithm (GAP);
specifically, single pixel imaging modeling is an optimization problem as follows:
where f (x) is a fidelity term, depending on the single pixel imaging model, to ensure that the relationship between the reconstructed single pixel image and the measured values received by the single pixel detector conforms to the single pixel imaging principle. g (x) is a prior term that contains some prior information of the image to be reconstructed. Combining with the generalized alternative projection algorithm, introducing an auxiliary parameter v to convert the above unconstrained optimization problem into a constrained optimization problem, i.e. the noise-free single-pixel reconstruction optimization model under the undersampling condition described in step 102, as shown in formula (3):
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
the optimization questions of step 103And (4) problem decomposition, solving the subproblems about the fidelity terms according to the framework of the generalized alternative projection algorithm. In particular, at known k time information vkOn the premise of v iskEuclidean projection on a linear manifold, as shown in equation (4):
xk+1=vk+AT(AAT)-1(b-Avk) (4)
step 104: training the DRUNet network by using a natural image with noise to enable the DRUNet network to have the noise reduction capability;
specifically, the DRUNet network is trained on a large data set consisting of a DIV2K data set, a Flick2K data set, a Wateloo expansion data set and a BSD data set, so that the DRUNet network has the noise reduction capability.
Step 105: combining Plug-and-Play (PnP), regarding the optimization sub-problem about the prior item as an image denoising problem, and replacing the solution of the optimization sub-problem about the prior item with the DRUNet network trained in the step 104;
the alternative to the prior term optimization sub-problem solution described in step 105 is shown in equation (5):
vk+1=Dσ(xk+1) (5)
wherein DσFor the image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 106: by alternately iterating the solution of the fidelity sub-problem and the replaced convolutional neural network, high-quality single-pixel imaging under the condition of no noise under the undersampling condition can be realized without considering the noise.
And (3) alternately iterating the solution of the fidelity item-related subproblem and the convolutional neural network after replacement, namely iterating the formulas (4) and (5) until the algorithm converges, namely the difference value of x and v is in a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
Example 2:
as shown in fig. 3, the single-pixel imaging method considering noise under the under-sampling condition disclosed in this embodiment includes the following steps:
step 201: the built single-pixel imaging light path is as shown in fig. 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are coded and modulated and then are smaller than the Nyquist sampling theorem;
specifically, the projector projects structured light patterns with random gray scales to a target image according to a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulated structured light patterns are used as a one-dimensional measured value of a single pixel. Wherein the relationship between the target image, the structured light pattern, and the single pixel measurement is:
Ax=b (6)
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector. In the undersampling case, the number of measurements received, m, is lower than the nyquist sampling theorem, so the single-pixel imaging problem is highly ill-conditioned and irreversible.
Step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction multiplier Algorithm (ADMM);
specifically, single pixel imaging modeling is an optimization problem as follows:
where f (x) is a fidelity term, depending on the single pixel imaging model, to ensure that the relationship between the reconstructed single pixel image and the measured values received by the single pixel detector conforms to the single pixel imaging principle. g (x) is a prior term that contains some prior information of the image to be reconstructed. By combining with the alternative direction multiplier algorithm, introducing an auxiliary parameter v to convert the above unconstrained optimization problem into a constrained optimization problem, that is, the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 is as shown in formula (8):
wherein, thereinAnd n is the number of pixels of the image to be restored.The structured light patterns are random gray scale, and m is the number of the structured light patterns.For m measurements received by a single pixel detector. According to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
the optimization problem decomposition in step 203 is shown as formula (10):
whereinRegarding the solution of the fidelity term sub-problem (10-1), according to the alternative direction multiplier algorithm ADMM, the solution can be:
xk+1=(ATA+ρI)-1(ATb+ρvk) (11)
step 204: training the DRUNet network by using a natural image with noise to enable the DRUNet network to have the noise reduction capability;
specifically, the DRUNet network is trained on a large data set consisting of a DIV2K data set, a Flick2K data set, a Wateloo expansion data set and a BSD data set, so that the DRUNet network has the noise reduction capability.
Step 205: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image denoising problem, and replacing the solution of the optimization sub-problem of the prior item with the DRUNet network trained in step 204;
the alternative to the prior term optimization sub-problem solution described in step 205 is shown in equation (12):
wherein DσFor the image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 206: by alternately iterating the solution of the fidelity sub-problem, the convolutional neural network after replacement and the dual variable update, the reconstruction of a high-quality single-pixel image under the undersampling condition can be realized under the condition of considering noise, and the noise robustness is realized.
And (3) updating alternate iteration, namely iterating equations (11), (12) and (10-3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, to the solution of the fidelity sub-problem, the convolutional neural network after replacement and the dual variable, so as to obtain a reconstructed single-pixel image, namely to realize the high-quality single-pixel imaging under the condition of undersampling and considering noise.
The effect of the embodiment is verified from both simulation and experiment.
1. Simulation result
In order to verify the accuracy of the noise-free single-pixel imaging method on image reconstruction under the undersampling condition, the image in the Set12 data Set is Set as a target image, and under the condition that the measurement noise is not considered, the reconstruction results of the noise-free single-pixel imaging method under the undersampling condition and the noise-free single-pixel imaging method under the undersampling condition, which are provided by the invention, under different sampling rates are compared with the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates. In order to quantitatively measure the effect of different single-pixel imaging methods, Peak signal to noise ratio (PSNR) and Structural Similarity (SSIM) are used to measure the spatial quality and visual effect of the image reconstruction result. For each method, the peak signal-to-noise ratio PSNR and the structural similarity SSIM of all reconstructed images in the data set are averaged. The image size was set to 64 x 64 and the sampling rate was set from 0.2 to 0.9.
TABLE 1 Single Pixel reconstruction results (average PSNR) on Set12 dataset
Table 2 single pixel reconstruction results (average SSIM) on Set12 dataset
As can be seen from tables 1 and 2, the proposed noise-free single-pixel imaging method under the undersampling condition, i.e., the GAP-prunet algorithm, is significantly better than other algorithms in terms of spatial quality and visual effect. To more clearly contrast the differences between the different single pixel imaging methods, the "Cameraman" image in the Set12 dataset was reconstructed at different sampling rates in different ways as shown in FIG. 4.
In order to verify the accuracy of the single-pixel imaging method considering noise under the undersampling condition on image reconstruction, images in a Set12 data Set are Set as target images, and under the condition of considering different measurement noise levels, the reconstruction results of the single-pixel imaging method considering noise under the undersampling condition and the noise under the undersampling condition, which are provided by the invention, under a fixed sampling rate are compared with the reconstruction results of the single-pixel imaging method based on a model and the single-pixel imaging method based on learning under the fixed sampling rate. In order to quantitatively measure the effect of different single-pixel imaging methods, the spatial quality and the visual effect of an image reconstruction result are measured by using Peak signal to noise ratio (PSNR) and Structural Similarity (SSIM). For each method, the peak signal-to-noise ratio PSNR and the structural similarity SSIM of all reconstructed images in the data set are averaged. The image size was set to 64 x 64 and the sampling rate was set to 0.5.
TABLE 3 Single Pixel reconstruction results (average PSNR) on Set12 data Set
Table 4 single pixel reconstruction results (average SSIM) on Set12 dataset
As can be seen from tables 3 and 4, the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly superior to other algorithms in terms of spatial quality and visual effect. To more clearly contrast the differences between the different single pixel imaging methods, the "Cameraman" image in the Set12 dataset was reconstructed under different methods at different noise levels as shown in FIG. 5.
2. Results of the experiment
In order to verify the performance of the noise-free single-pixel imaging method under the undersampling condition and the noise-considered single-pixel imaging method under the undersampling condition, which are provided by the invention, on the real single-pixel measurement value, an optical path is constructed according to the optical path schematic diagram of the single-pixel imaging method in fig. 1, wherein the model of the projector is Panasonic (X416C XGA), and the model of the single-pixel detector is Thorlabs (PDA100a 2). The reconstruction results of the noise-free single-pixel imaging method under the undersampling condition and the noise-considered single-pixel imaging method under the undersampling condition, which are provided by the invention, under different sampling rates are in a pair form as shown in fig. 6 with the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates.
Fig. 6(a) shows the result of single-pixel imaging of the "Ghost" target image (64 × 64) under different sampling rates and different algorithms, and it can be seen from fig. 6(a) that the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly better than other algorithms in terms of spatial quality and visual effect due to the fact that non-negligible noise exists in the actual measurement. Fig. 6(b) is a single-pixel imaging result of the "SPI" target image (64 × 64) under different sampling rates and different algorithms, and further proves that the proposed single-pixel imaging method considering noise under the undersampling condition, i.e., the ADMM-DRUNet algorithm, is significantly superior to other algorithms in terms of spatial quality and visual effect.
The above detailed description is intended to illustrate the objects, aspects and advantages of the present invention, and it should be understood that the above detailed description is only exemplary of the present invention and is not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. The noiseless single-pixel imaging method under the undersampling condition is characterized in that: comprises the following steps of (a) carrying out,
step 101: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 102: establishing a noise-free single-pixel image imaging optimization model under an undersampling condition based on a Generalized Alternating Projection Algorithm (GAP);
step 103: decomposing the optimization problem into two sub-problems according to a fidelity term and a prior term by combining a generalized alternative projection algorithm GAP;
step 104: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
step 105: by alternately iterating the solution of the fidelity sub-problem and the replaced modern image noise reduction algorithm, high-quality single-pixel imaging under the condition of no noise under sampling can be realized without considering the noise.
2. A method of noiseless single pixel imaging under undersampling as claimed in claim 1 wherein: the noise-free single-pixel image imaging optimization model under the undersampling condition in step 102 is the optimization model described by the formula (1):
3. A method of noiseless single pixel imaging under undersampling as claimed in claim 2, characterized in that: 103, decomposing the optimization problem, and solving a subproblem about the fidelity term according to a frame of a generalized alternative projection algorithm; at known time k information vkOn the premise of (1), the solution is vkEuclidean projection on a linear manifold, as shown in equation (2):
xk+1=vk+AT(AAT)-1(b-Avk) (2)
4. a method of noiseless single pixel imaging under undersampling conditions as claimed in claim 1 or 2, characterized by: the alternative to the prior term optimization sub-problem solution described in step 104 is shown in equation (3):
vk+1=Dσ(xk+1) (3)
wherein DσA convolution neural network with a noise reduction function in deep learning is adopted as an image noise reducer.
5. A method of noiseless single pixel imaging under undersampling conditions as claimed in claim 1 or 2, characterized by: and (3) alternately iterating the solution of the fidelity item-related subproblem and the replaced modern image noise reduction algorithm, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value of x and v is within a preset threshold range, so as to obtain a reconstructed single-pixel image, namely the high-quality single-pixel imaging under the under-sampling noise-free condition is realized.
6. The single-pixel imaging method considering noise under the undersampling condition is characterized in that: comprises the following steps of (a) carrying out,
step 201: a light path is built according to a single-pixel imaging principle, and a single-pixel detector is adopted to collect one-dimensional optical signals smaller than the Nyquist sampling theorem;
step 202: establishing a single-pixel image imaging optimization model considering noise under an undersampling condition based on an Alternating Direction Multiplier Algorithm (ADMM);
step 203: decomposing the optimization problem into two subproblems and a dual variable updating step by combining an alternating direction multiplier algorithm according to a fidelity term, a prior term and the dual variable;
step 204: combining Plug-and-Play (PnP), regarding an optimization sub-problem about a prior item as an image noise reduction problem, and replacing the solution of the optimization sub-problem about the prior item with a modern image noise reduction algorithm;
step 205: by alternately iterating the solution of the fidelity sub-problem, the replaced modern image denoising algorithm and the dual variable update, the reconstruction of the high-quality single-pixel image under the undersampling condition can be realized under the condition of considering the noise, and the robustness to the noise is realized.
7. A method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 6, wherein: the single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202 selects the optimization model described in formula (4):
wherein, thereinThe image is a target image, and n is the number of pixels of the image to be restored;is a structured light pattern of random gray scale,m is the number of structured light patterns;m measured values received by a single-pixel detector; according to an alternating direction multiplier algorithm ADMM, the solution of the optimization equation needs to be converted into an augmented Lagrange form:
8. a method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 7, wherein: the optimization problem decomposition in step 203 is shown as formula (6):
whereinRegarding the solution of the fidelity item subproblem (6-1), according to the alternative direction multiplier algorithm ADMM, the solution is:
xk+1=(ATA+pI)-1(ATb+pvk) (7)
9. a method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 8, wherein: the prior term optimization sub-problem replacement described in step 204 is shown in equation (8):
wherein DσA convolution neural network with a noise reduction function in deep learning is adopted as an image noise reducer.
10. A method of noise-aware single-pixel imaging under undersampling conditions as defined in claim 9, wherein: and (3) updating alternate iteration, namely, iterating formulas (7), (8) and (6-3) until the difference value of x and v is within a preset threshold range, to obtain a reconstructed single-pixel image, namely, realizing the high-quality single-pixel imaging under the condition of undersampling and considering noise.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110795952.7A CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110795952.7A CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Publications (2)
Publication Number | Publication Date |
---|---|
CN113724146A true CN113724146A (en) | 2021-11-30 |
CN113724146B CN113724146B (en) | 2024-06-04 |
Family
ID=78673287
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110795952.7A Active CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113724146B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117408875A (en) * | 2023-12-13 | 2024-01-16 | 阿尔玻科技有限公司 | Method, storage medium and apparatus for reconstructing compressed ultrafast photographic images |
CN117876837A (en) * | 2024-03-11 | 2024-04-12 | 北京理工大学 | Near infrared single-pixel imaging method and system based on depth expansion network |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020069399A1 (en) * | 1999-08-16 | 2002-06-06 | Z-Force Corporation | System of reusable software parts and methods of use |
US20200011951A1 (en) * | 2018-07-06 | 2020-01-09 | The Board Of Trustees Of The Leland Stanford Junior University | Deep Learning based Methods to Accelerate Multi-Spectral Imaging |
CN111259991A (en) * | 2020-02-28 | 2020-06-09 | 南京理工大学 | Under-sampling single-pixel imaging target identification method in noise environment |
US20200284883A1 (en) * | 2019-03-08 | 2020-09-10 | Osram Gmbh | Component for a lidar sensor system, lidar sensor system, lidar sensor device, method for a lidar sensor system and method for a lidar sensor device |
CN112203068A (en) * | 2020-09-30 | 2021-01-08 | 中国科学院长春光学精密机械与物理研究所 | Single-pixel imaging method, system, device and medium |
-
2021
- 2021-07-14 CN CN202110795952.7A patent/CN113724146B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020069399A1 (en) * | 1999-08-16 | 2002-06-06 | Z-Force Corporation | System of reusable software parts and methods of use |
US20200011951A1 (en) * | 2018-07-06 | 2020-01-09 | The Board Of Trustees Of The Leland Stanford Junior University | Deep Learning based Methods to Accelerate Multi-Spectral Imaging |
US20200284883A1 (en) * | 2019-03-08 | 2020-09-10 | Osram Gmbh | Component for a lidar sensor system, lidar sensor system, lidar sensor device, method for a lidar sensor system and method for a lidar sensor device |
CN111259991A (en) * | 2020-02-28 | 2020-06-09 | 南京理工大学 | Under-sampling single-pixel imaging target identification method in noise environment |
CN112203068A (en) * | 2020-09-30 | 2021-01-08 | 中国科学院长春光学精密机械与物理研究所 | Single-pixel imaging method, system, device and medium |
Non-Patent Citations (1)
Title |
---|
徐树奎;涂丹;李国辉;张军;: "计算摄影综述", 计算机应用研究, no. 11, 15 November 2010 (2010-11-15) * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117408875A (en) * | 2023-12-13 | 2024-01-16 | 阿尔玻科技有限公司 | Method, storage medium and apparatus for reconstructing compressed ultrafast photographic images |
CN117876837A (en) * | 2024-03-11 | 2024-04-12 | 北京理工大学 | Near infrared single-pixel imaging method and system based on depth expansion network |
CN117876837B (en) * | 2024-03-11 | 2024-06-07 | 北京理工大学 | Near infrared single-pixel imaging method and system based on depth expansion network |
Also Published As
Publication number | Publication date |
---|---|
CN113724146B (en) | 2024-06-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Gehrig et al. | Combining events and frames using recurrent asynchronous multimodal networks for monocular depth prediction | |
US10593021B1 (en) | Motion deblurring using neural network architectures | |
CN104137150B (en) | Motion compensation in range finding imaging | |
CN113724146B (en) | Single-pixel imaging method based on plug-and-play prior | |
CN101373135B (en) | Method and apparatus for measuring three-dimensional curved surface based on instantaneous accidental light irradiation | |
CN113837938A (en) | Super-resolution method for reconstructing potential image based on dynamic vision sensor | |
CN109945802A (en) | A kind of structural light three-dimensional measurement method | |
CN104243837A (en) | Vibration detection and remote sensing image recovery method based on single-exposure video reconstruction | |
CN113487739A (en) | Three-dimensional reconstruction method and device, electronic equipment and storage medium | |
Nakashima et al. | Learning to drop points for lidar scan synthesis | |
Wang et al. | Underwater self-supervised monocular depth estimation and its application in image enhancement | |
CN113643220B (en) | Physical information driven infrared image turbulence effect suppression method and system | |
CN114757842A (en) | Color calculation ghost imaging method based on plug-and-play generalized alternative projection algorithm | |
CN112468791A (en) | Light intensity measurement iterative imaging method based on single-pixel detection | |
CN117036442A (en) | Robust monocular depth completion method, system and storage medium | |
CN110365966B (en) | Video quality evaluation method and device based on window | |
CN117132704A (en) | Three-dimensional reconstruction method of dynamic structured light, system and computing equipment thereof | |
CN110926611A (en) | Noise suppression method applied to compressed sensing spectral imaging system | |
CN113240610B (en) | Double-channel ghost imaging reconstruction method and system based on human eye imitation mechanism | |
CN103115673B (en) | Acquisition method for high signal-to-noise ratio variable illumination and light field reuse | |
CN105869162A (en) | Active detection imaging data fusion method based on three-edge confidence measure | |
CN118247418B (en) | Method for reconstructing nerve radiation field by using small quantity of blurred images | |
Mao et al. | Time-of-flight camera via a single-pixel correlation image sensor | |
CN117710416B (en) | Fluid light flow velocity measurement method and system for complex illumination change scene | |
Seo et al. | A study on selective dehazing system based on haze detection network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |