CN113724146B - Single-pixel imaging method based on plug-and-play prior - Google Patents
Single-pixel imaging method based on plug-and-play prior Download PDFInfo
- Publication number
- CN113724146B CN113724146B CN202110795952.7A CN202110795952A CN113724146B CN 113724146 B CN113724146 B CN 113724146B CN 202110795952 A CN202110795952 A CN 202110795952A CN 113724146 B CN113724146 B CN 113724146B
- Authority
- CN
- China
- Prior art keywords
- noise
- pixel
- under
- undersampling
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 claims abstract description 52
- 230000009467 reduction Effects 0.000 claims abstract description 47
- 238000005070 sampling Methods 0.000 claims abstract description 32
- 238000005457 optimization Methods 0.000 claims description 68
- 238000013135 deep learning Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 18
- 230000009977 dual effect Effects 0.000 claims description 18
- 238000005259 measurement Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 13
- 239000003638 chemical reducing agent Substances 0.000 claims description 6
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims 1
- 238000013528 artificial neural network Methods 0.000 description 7
- 230000000007 visual effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000002427 irreversible effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a single-pixel imaging method based on a plug-and-play prior, relates to a single-pixel imaging method based on a plug-and-play prior under-sampling condition, and belongs to the field of computational photography. The invention combines the advantages of a model-based method and a depth learning-based method, implicitly replaces the modern image noise reduction algorithm as prior to the imaging method, and improves the imaging quality of single-pixel images under the undersampling condition. The invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition under the same concept of plug and play. The noise-free single-pixel imaging method under the undersampling condition can realize high-quality single-pixel image reconstruction under the undersampling condition without considering noise. The single-pixel imaging method taking noise into consideration under the undersampling condition can realize high-quality single-pixel image reconstruction under the undersampling condition under the condition of taking noise into consideration, and has robustness to noise.
Description
Technical Field
The invention relates to a single-pixel imaging method and a system, in particular to a single-pixel imaging method and a system under the undersampling condition based on plug-and-play priori, belonging to the field of computational photography.
Background
The single-pixel imaging technology is a novel computational imaging technology, which modulates a target image by using structured light, obtains a one-dimensional measurement light signal by a single-pixel detector without spatial resolution, and reconstructs an image according to the correlation between the structured light and the one-dimensional measurement light signal. Compared with the traditional two-dimensional pixel detector imaging method, the cost of the image sensor is reduced, and the image measurement times can be reduced by combining a compressed sensing algorithm, so that the image acquisition speed is improved. In recent years, the technology is increasingly applied to the fields of underwater imaging, three-dimensional imaging, remote sensing, target tracking, terahertz imaging and the like.
Current single-pixel imaging methods include two types, model-based single-pixel imaging methods and depth learning (DEEP LEARNING, DL) -based single-pixel imaging methods. Model-based single-pixel imaging methods include a linear correlation method (Linear correlation methods), an alternating projection method (ALTERNATING PROJECTION METHODS, AP), and a compressed sensing method (Compressive sensing methods, CS). Among them, the compressed sensing method is the most commonly used algorithm among model-based single-pixel algorithms. Compressed sensing methods add a priori knowledge of the image by humans as constraints to the design of single pixel image reconstruction algorithms. Under undersampling conditions, the compressed sensing method has proven to be the most effective method of the model-based single-pixel imaging method. Since the compressed sensing method is limited by the expressive nature of human manual construction prior, the quality of single-pixel image reconstruction is rapidly reduced as the sampling rate is reduced, and the compressed sensing method is difficult to recover the single-pixel image under the condition of high noise.
In recent years, with the development of deep learning technology, a single-pixel imaging method based on deep learning appears, and a single-pixel imaging and restoring process is simulated by training an end-to-end neural network. Although the single-pixel imaging method based on the deep learning improves the single-pixel imaging quality at an extremely low sampling rate, as the sampling rate increases, the effect of the single-pixel imaging method based on the deep learning does not change significantly, even less than that of the single-pixel imaging method based on the model. And the single-pixel imaging method based on deep learning is poor in universality, and the trained network is limited to specific application. When the sampling rate, image size, imaging environment of a single pixel change, a new network needs to be retrained to accommodate the new task. In addition, the depth learning single pixel imaging based method is not robust to noise.
Therefore, from the need for improving the quality of single-pixel imaging recovery in the case of undersampling, there is an urgent need for a single-pixel imaging method that is highly versatile and robust to noise.
Disclosure of Invention
The invention discloses a single-pixel imaging method based on plug and play priori, which aims to solve the technical problems that: and combining the advantages of a model-based method and a depth learning-based method, implicitly replacing a modern image noise reduction algorithm as a priori into an imaging method, and improving the quality of single-pixel image imaging under the undersampling condition. The invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition under the same concept of plug and play.
The invention discloses a single-pixel imaging method without noise under undersampling condition, which is based on a generalized alternating projection algorithm (Generalized Alternating Projection Algorithm, GAP) to model a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity item and an priori item, plug-and-Play (PnP) is combined, the priori item sub-problems are regarded as image noise reduction problems, and modern image noise reduction algorithms (such as a convolutional neural network with noise reduction function in deep learning) are replaced. By alternately and iteratively solving the two sub-problems, high-quality single-pixel image reconstruction under the undersampling condition can be realized without considering noise.
According to the single-pixel imaging method considering noise under the undersampling condition, the single-pixel image reconstruction problem is modeled as an optimization problem based on an alternating direction multiplier algorithm (ALTERNATING DIRECTION METHOD ofMultiplierAlgorithm, ADMM). Unlike the noise-free single-pixel imaging method under undersampling conditions, the modeling of the optimization problem includes a noise model. The optimization problem solving process is divided into two sub-problems and a dual variable updating step according to fidelity terms, prior terms and dual variables, and the prior term sub-problems are regarded as image noise reduction problems and are replaced by modern image noise reduction algorithms (for example, a convolutional neural network with a noise reduction function in deep learning) by combining Plug-and-Play prior (PnP). By alternately and iteratively solving the two sub-problems and the dual variable, the high-quality single-pixel image reconstruction under the undersampling condition can be realized under the condition of taking noise into consideration, and the noise is robust.
The aim of the invention is achieved by the following technical scheme:
the invention discloses a single-pixel imaging method without noise under undersampling condition, which comprises the following steps:
Step 101: constructing an optical path according to a single-pixel imaging principle, and acquiring a one-dimensional optical signal smaller than the Nyquist sampling theorem by adopting a single-pixel detector;
Step 102: establishing a noise-free single-pixel image imaging optimization model under the undersampling condition based on a generalized alternating projection algorithm (Generalized AlternatingProjection Algorithm, GAP);
The noise-free single-pixel image imaging optimization model under the undersampling condition in the step 102 is an optimization model shown in a formula (1):
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to the fidelity terms and the prior terms by combining a generalized alternating projection algorithm GAP;
the optimization problem described in step 103 is solved, and the sub-problem about the fidelity term is solved according to the framework of the generalized alternating projection algorithm. Specifically, on the premise that k time information v k is known, the solution is to obtain the euclidean projection of v k on the linear manifold, as shown in formula (2):
xk+1＝vk+AT(AAT)-1(b-Avk) (2)
step 104: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with a modern image noise reduction algorithm;
the alternatives described in step 104 with respect to the a priori term optimization sub-problem solution are shown in equation (3):
vk+1＝Dσ(xk+1) (3)
Wherein D σ is an image noise reducer, and a convolutional neural network with a noise reduction function in deep learning can be adopted. The convolutional neural network is FFDNet, UNet, IRCNN or DRUNet, but is not limited thereto. In order to obtain a higher quality single-pixel reconstructed image, the convolutional neural network preferably employs a DRUNet network.
Step 105: by alternately iterating the modern image denoising algorithm after the solution and replacement of the related fidelity term sub-problem, high-quality single-pixel imaging under the undersampling noiseless condition can be realized without considering noise.
The method comprises the steps of alternately iterating a modern image noise reduction algorithm after the solution and replacement of related fidelity term sub-problems, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value between x and v is in a preset threshold range, so as to obtain a reconstructed single-pixel image, namely high-quality single-pixel imaging under the condition of undersampling and noiselessness.
The invention discloses a single-pixel imaging method considering noise under undersampling condition, which comprises the following steps:
Step 201: constructing an optical path according to a single-pixel imaging principle, and acquiring a one-dimensional optical signal smaller than the Nyquist sampling theorem by adopting a single-pixel detector;
Step 202: establishing a single-pixel image imaging optimization model taking noise into consideration under the undersampling condition based on an alternating direction multiplier sub-algorithm (ALTERNATING DIRECTION METHOD OF MULTIPLIER ALGORITHM, ADMM);
The single-pixel image imaging optimization model taking noise into consideration under the undersampling condition in step 202 is an optimization model described in formula (4):
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector. According to the alternating direction multiplier algorithm ADMM, the above-mentioned optimization equation solution needs to be converted into an augmented lagrangian form:
Step 203: decomposing the optimization problem into two sub-problems and a dual variable updating step according to the fidelity term, the prior term and the dual variable by combining the alternating direction multiplier algorithm;
the optimization problem described in step 203 is decomposed as shown in equation (6):
Wherein the method comprises the steps of Regarding the solution of the fidelity sub-problem (6-1), according to the alternating direction multiplier algorithm ADMM, it can be solved as:
xk+1＝(ATA+ρI)-1(ATb+ρvk) (7)
Step 204: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with a modern image noise reduction algorithm;
the a priori term optimization sub-problem replacement described in step 204 is as shown in equation (8):
Wherein D σ is an image noise reducer, and a convolutional neural network with a noise reduction function in deep learning can be adopted. The convolutional neural network is FFDNet, UNet, IRCNN or DRUNet, but is not limited thereto. In order to obtain a higher quality single-pixel reconstructed image, the convolutional neural network preferably employs a DRUNet network.
Step 205: through the solution to the related fidelity term sub-problem, the modern image noise reduction algorithm after replacement and the dual variable update alternate iteration, the high-quality single-pixel image reconstruction under the undersampling condition can be realized under the condition of taking noise into consideration, and the noise robustness is realized.
The reconstructed single-pixel image is obtained by alternately iterating the solution and the replaced modern image noise reduction algorithm of the related fidelity term sub-problem and the update of the dual variable, namely iterating the formulas (7), (8) and (6-3) until the difference value of x and v is in a preset threshold range, namely realizing high-quality single-pixel imaging under the condition of undersampling and noise consideration.
The beneficial effects are that:
1. The single-pixel imaging method based on the plug-and-play prior combines the advantages of a model-based optimization method and a learning-based deep learning method, implicitly replaces a modern image noise reduction algorithm as the prior into the imaging method, and improves the quality of single-pixel image imaging under the undersampling condition. The invention provides a noise-free single-pixel imaging method under the undersampling condition and a single-pixel imaging method considering noise under the undersampling condition under the same concept of plug and play.
2. The invention discloses a single-pixel imaging method without noise under undersampling condition, which is based on a generalized alternating projection algorithm (Generalized Alternating Projection Algorithm, GAP) to model a single-pixel image reconstruction problem as an optimization problem, wherein the optimization problem solving process is decomposed into two sub-problems according to a fidelity item and an priori item, plug-and-Play (PnP) is combined, the priori item sub-problems are regarded as image noise reduction problems, and modern image noise reduction algorithms (such as a convolutional neural network with noise reduction function in deep learning) are replaced. By alternately and iteratively solving the two sub-problems, high-quality single-pixel image reconstruction under the undersampling condition can be realized without considering noise.
3. The noise-free single-pixel imaging method under the undersampling condition disclosed by the invention can be combined with Plug-and-Play (PnP), and can be used for implicitly inserting a modern image noise reduction algorithm (for example, a convolutional neural network with a noise reduction function in deep learning) as a priori into an optimization algorithm. The neural network with the noise reduction function can be trained in advance, and when the size and the sampling rate of a single-pixel image are changed, the network does not need to be trained again, so that the universality is good.
4. The single-pixel imaging method taking noise into consideration under the undersampling condition disclosed by the invention models a single-pixel image reconstruction problem as an optimization problem based on an alternating direction multiplier algorithm (ALTERNATING DIRECTION METHOD OF MULTIPLIER ALGORITHM, ADMM). Unlike the noise-free single-pixel imaging method under undersampling conditions, the modeling of the optimization problem includes a noise model. The optimization problem solving process is divided into two sub-problems and a dual variable updating step according to fidelity terms, prior terms and dual variables, and the prior term sub-problems are regarded as image noise reduction problems and are replaced by modern image noise reduction algorithms (for example, a convolutional neural network with a noise reduction function in deep learning) by combining Plug-and-Play prior (PnP). By alternately and iteratively solving the two sub-problems and the dual variable, the high-quality single-pixel image reconstruction under the undersampling condition can be realized under the condition of taking noise into consideration, and the noise is robust.
5. The single-pixel imaging method taking noise into consideration under the undersampling condition disclosed by the invention combines Plug-and-Play (PnP), and can implicitly insert a modern image noise reduction algorithm (for example, a convolutional neural network with a noise reduction function in deep learning) as a priori into an optimization algorithm. The neural network with the noise reduction function can be trained in advance, and when the size of a single-pixel image, the sampling rate and the environmental noise are changed, the network does not need to be trained again, so that the universality is good.
Drawings
Fig. 1 is a schematic diagram of an optical path of a single pixel imaging method based on a plug and play prior according to the method of the present invention.
Fig. 2 is a flow chart of a single pixel imaging method without noise under undersampling conditions in accordance with the method of the present invention.
Fig. 3 is a flow chart of a single pixel imaging method that accounts for noise under undersampling conditions in accordance with the method of the present invention.
FIG. 4 is a comparison of simulation results for single pixel imaging with the method of the present invention and other methods without noise at different sampling rates.
FIG. 5 is a comparison of single pixel imaging simulation results for the method of the present invention and other methods at different noise levels at a fixed sampling rate.
Fig. 6 is a comparison of results of single-pixel imaging in real experiments of the method and other methods according to the present invention, wherein fig. 6 (a) is a single-pixel imaging result of a "Ghost" target image (64×64) under different sampling rates and different algorithms, and fig. 6 (b) is a single-pixel imaging result of a "SPI" target image (64×64) under different sampling rates and different algorithms.
Detailed Description
For a better description of the objects and advantages of the present invention, the following description will be given with reference to the accompanying drawings and examples. The present invention is not limited to the following examples.
Example 1:
as shown in fig. 2, the noise-free single-pixel imaging method under the undersampling condition disclosed in this embodiment includes the following steps:
step 101: the built single-pixel imaging light path is shown in figure 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are less than the Nyquist sampling theorem after coded modulation;
Specifically, the projector projects the random gray structured light patterns onto the target image in a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulation of each structured light pattern as a one-dimensional measurement value of a single pixel. The relation among the target image, the structured light pattern and the single-pixel measured value is as follows:
Ax＝b (1)
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector. In the case of undersampling, the number of measurements received m is below the nyquist sampling theorem, so the single-pixel imaging problem is highly ill-conditioned and irreversible.
Step 102: establishing a noise-free single-pixel image imaging optimization model under the undersampling condition based on a generalized alternating projection algorithm (Generalized Alternating Projection Algorithm, GAP);
Specifically, single pixel imaging modeling is an optimization problem of:
Where f (x) is a fidelity term, depending on the single-pixel imaging model, ensures that the relationship between the reconstructed single-pixel image and the measured values received by the single-pixel detector conforms to the single-pixel imaging principle. g (x) is an a priori term that contains some a priori information of the image to be reconstructed. Combining with a generalized alternate projection algorithm, introducing an auxiliary parameter v to convert the unconstrained optimization problem into a constrained optimization problem, namely, a noiseless single-pixel reconstruction optimization model under the undersampling condition described in step 102, as shown in a formula (3):
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector.
Step 103: decomposing the optimization problem into two sub-problems according to the fidelity terms and the prior terms by combining a generalized alternating projection algorithm GAP;
The optimization problem described in step 103 is solved, and the sub-problem about the fidelity term is solved according to the framework of the generalized alternating projection algorithm. Specifically, on the premise that k time information v k is known, the euclidean projection of v k on the linear manifold is as shown in formula (4):
xk+1＝vk+AT(AAT)-1(b-Avk) (4)
Step 104: training DRUNet the network by using the natural image with noise to enable the DRUNet network to have noise reduction capability;
Specifically, the DRUNet network is trained on a large dataset consisting of a DIV2K dataset, a click 2K dataset, a Waterloo Exploration dataset and a BSD dataset, so that the DRUNet network has noise reduction capability.
Step 105: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with a DRUNet network trained in step 104;
The alternatives described in step 105 with respect to the a priori term optimization sub-problem solution are shown in equation (5):
vk+1＝Dσ(xk+1) (5)
Wherein D σ is an image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 106: by alternately iterating the convolution neural network after the solution and replacement of the related fidelity term sub-problem, high-quality single-pixel imaging under the undersampling noiseless condition can be realized without considering noise.
And (3) alternately iterating the convolution neural network after the solution and replacement of the related fidelity term sub-problem, namely iterating the formulas (4) and (5) until the algorithm converges, namely that the difference value of x and v is in a preset threshold range, so as to obtain a reconstructed single-pixel image, namely realizing high-quality single-pixel imaging under the under-sampling noiseless condition.
Example 2:
as shown in fig. 3, the single-pixel imaging method taking noise into consideration under the undersampling condition disclosed in this embodiment includes the following steps:
Step 201: the built single-pixel imaging light path is shown in figure 1, and a single-pixel detector is adopted to collect one-dimensional optical signals which are less than the Nyquist sampling theorem after coded modulation;
Specifically, the projector projects the random gray structured light patterns onto the target image in a certain sequence, and the single-pixel detector synchronously receives the light intensity reflected by the image after the modulation of each structured light pattern as a one-dimensional measurement value of a single pixel. The relation among the target image, the structured light pattern and the single-pixel measured value is as follows:
Ax＝b (6)
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector. In the case of undersampling, the number of measurements received m is below the nyquist sampling theorem, so the single-pixel imaging problem is highly ill-conditioned and irreversible.
Step 202: establishing a single-pixel image imaging optimization model taking noise into consideration under the undersampling condition based on an alternating direction multiplier sub-algorithm (ALTERNATING DIRECTION METHOD OFMULTIPLIER ALGORITHM, ADMM);
Specifically, single pixel imaging modeling is an optimization problem of:
Where f (x) is a fidelity term, depending on the single-pixel imaging model, ensures that the relationship between the reconstructed single-pixel image and the measured values received by the single-pixel detector conforms to the single-pixel imaging principle. g (x) is an a priori term that contains some a priori information of the image to be reconstructed. The auxiliary parameter v is introduced to convert the above unconstrained optimization problem into a constrained optimization problem by combining an alternate direction multiplier algorithm, namely a single-pixel image imaging optimization model considering noise under the undersampling condition described in step 202, as shown in formula (8):
Wherein, therein For the target image, n is the number of pixels of the image to be restored. /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns. /(I)For m measurements received by a single pixel detector. According to the alternating direction multiplier algorithm ADMM, the above-mentioned optimization equation solution needs to be converted into an augmented lagrangian form:
Step 203: decomposing the optimization problem into two sub-problems and a dual variable updating step according to the fidelity term, the prior term and the dual variable by combining the alternating direction multiplier algorithm;
the optimization problem described in step 203 is decomposed as shown in formula (10):
Wherein the method comprises the steps of Regarding the solution of the fidelity sub-problem (10-1), according to the alternating direction multiplier algorithm ADMM, it can be solved as:
xk+1＝(ATA+ρI)-1(ATb+ρvk) (11)
step 204: training DRUNet the network by using the natural image with noise to enable the DRUNet network to have noise reduction capability;
Specifically, the DRUNet network is trained on a large dataset consisting of a DIV2K dataset, a click 2K dataset, a Waterloo Exploration dataset and a BSD dataset, so that the DRUNet network has noise reduction capability.
Step 205: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with the DRUNet network trained in step 204;
The alternatives described in step 205 with respect to the a priori term optimization sub-problem solution are shown in equation (12):
Wherein D σ is an image noise reducer, a convolutional neural network with a noise reduction function in deep learning is adopted, and the convolutional neural network adopts a DRUNet network.
Step 206: by alternately iterating the solution, the replaced convolution neural network and the dual variable update of the related fidelity term sub-problem, the high-quality single-pixel image reconstruction under the undersampling condition can be realized under the condition of considering noise, and the robustness to the noise is realized.
The reconstructed single-pixel image is obtained by alternately iterating the solution and the replaced convolution neural network of the related fidelity term sub-problem and the update of the dual variable, namely iterating formulas (11), (12) and (10-3) until the algorithm converges, namely the difference value of x and v is in a preset threshold range, namely the high-quality single-pixel imaging under the condition of undersampling and noise consideration is realized.
The effect of the examples was verified from both simulation and experiment.
1. Simulation results
In order to verify the accuracy of the noiseless single-pixel imaging method on image reconstruction under the undersampling condition, the Set12 dataset image is Set as a target image, and under the condition of not considering measurement noise, the reconstruction results of the noiseless single-pixel imaging method under the undersampling condition and the single-pixel imaging method considering noise under the undersampling condition under different sampling rates are compared with the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates. To quantitatively measure the effect of different single pixel imaging methods, peak signal to noise ratio (PSNR) and structural similarity (Structural similarity, SSIM) are used to measure the spatial quality and visual effect of the image reconstruction results. For each method, the peak signal-to-noise ratio PSNR and structural similarity SSIM of all reconstructed images in the dataset are averaged. The image size is set to 64 x 64 and the sampling rate is set from 0.2 to 0.9.
Table 1 Single Pixel reconstruction results (average PSNR) on Set12 dataset
Table 2 Single Pixel reconstruction results on Set12 dataset (average SSIM)
It can be seen from tables 1 and 2 that the proposed noise-free single-pixel imaging method under undersampling conditions, that is, the GAP-DRUNet algorithm, is significantly superior to other algorithms in terms of spatial quality and visual effect. To more clearly contrast the distinction between different single-pixel imaging methods, the "CAMERAMAN" images in the Set12 dataset reconstruct images at different sampling rates and different methods as shown in FIG. 4.
In order to verify the accuracy of a single-pixel imaging method considering noise under an undersampling condition on image reconstruction, setting an image in a Set12 dataset as a target image, and comparing a reconstruction result of the noiseless single-pixel imaging method under the undersampling condition and the single-pixel imaging method considering noise under the undersampling condition under the fixed sampling rate with a reconstruction result of a model-based single-pixel imaging method and a reconstruction result of the learning-based single-pixel imaging method under the fixed sampling rate under the condition of considering different measurement noise levels. To quantitatively measure the effect of different single pixel imaging methods, peak signal-to-noise ratio (PSNR) and structural similarity (Structural similarity, SSIM) are used to measure the spatial quality and visual effect of the image reconstruction results. For each method, the peak signal-to-noise ratio PSNR and structural similarity SSIM of all reconstructed images in the dataset are averaged. The image size is set to 64 x 64 and the sampling rate is set to 0.5.
Table 3 Single Pixel reconstruction results (average PSNR) on the Set12 dataset
Table 4 Single Pixel reconstruction results (average SSIM) on the set12 dataset
It can be seen from tables 3 and 4 that the proposed single pixel imaging method, i.e. ADMM-DRUNet algorithm, taking noise into account under-sampling conditions is significantly better than other algorithms in terms of spatial quality and visual effect. To more clearly contrast the distinction between different single-pixel imaging methods, the "CAMERAMAN" images in the Set12 dataset reconstruct images at different noise levels and different methods as shown in FIG. 5.
2. Experimental results
To verify the performance of the noise-free single-pixel imaging method under undersampling conditions and the single-pixel imaging method taking noise into account under undersampling conditions proposed by the present invention on a real single-pixel measured value, the optical path is constructed according to the optical path schematic diagram of the single-pixel imaging method in fig. 1 to acquire the single-pixel measured value, wherein the model of the projector is Panasonic (X416C XGA), and the model of the single-pixel detector is Thorlabs (PDA 100 A2). The reconstruction results of the noiseless single-pixel imaging method under the undersampling condition and the single-pixel imaging method taking noise into consideration under the undersampling condition under different sampling rates are shown in a pair of the reconstruction results of the model-based single-pixel imaging method and the learning-based single-pixel imaging method under different sampling rates, such as shown in fig. 6.
Fig. 6 (a) shows the single-pixel imaging result of the "Ghost" target image (64×64) under different sampling rates and different algorithms, and as can be seen from fig. 6 (a), the proposed single-pixel imaging method taking noise into consideration under the undersampling condition, that is, the ADMM-DRUNet algorithm is obviously superior to other algorithms in terms of spatial quality and visual effect due to the fact that the noise exists in the actual measurement. Fig. 6 (b) shows the single-pixel imaging result of the "SPI" target image (64×64) under the conditions of different sampling rates and different algorithms, and further demonstrates that the spatial quality and visual effect of the single-pixel imaging method taking noise into consideration under the proposed undersampling condition, i.e. the ADMM-DRUNet algorithm, are obviously superior to those of other algorithms.
While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.
Claims (10)
1. The single-pixel imaging method without noise under the undersampling condition is characterized in that: comprises the steps of,
Step 101: constructing an optical path according to a single-pixel imaging principle, and acquiring a one-dimensional optical signal smaller than the Nyquist sampling theorem by adopting a single-pixel detector;
Step 102: establishing a noise-free single-pixel image imaging optimization model under the undersampling condition based on a generalized alternating projection algorithm (Generalized Alternating Projection Algorithm, GAP);
Step 103: decomposing the optimization problem into two sub-problems according to the fidelity terms and the prior terms by combining a generalized alternating projection algorithm GAP;
step 104: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with a modern image noise reduction algorithm;
step 105: by alternately iterating the modern image denoising algorithm after the solution and replacement of the related fidelity term sub-problem, high-quality single-pixel imaging under the undersampling noiseless condition can be realized without considering noise.
2. The method for single pixel imaging without noise under undersampling conditions of claim 1, wherein: the noise-free single-pixel image imaging optimization model under the undersampling condition in the step 102 is an optimization model shown in a formula (1):
Wherein, therein N is the number of pixels of the image to be restored; /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns; /(I)For m measurements received by a single pixel detector.
3. A single pixel imaging method without noise under undersampling conditions as in claim 2, wherein: the optimization problem decomposition in step 103 is carried out, and the sub-problems related to the fidelity terms are solved according to the framework of the generalized alternating projection algorithm; on the premise that k time information v k is known, solving into Euclidean projection of v k on a linear manifold, as shown in formula (2):
xk+1＝vk+AT(AAT)-1(b-Avk) (2)。
4. A single pixel imaging method without noise under undersampling conditions as in claim 1 or 2, wherein: the alternatives described in step 104 with respect to the a priori term optimization sub-problem solution are shown in equation (3):
vk+1＝Dσ(xk+1) (3)
wherein D σ is an image noise reducer, and a convolutional neural network with a noise reduction function in deep learning is adopted.
5. A single pixel imaging method without noise under undersampling conditions as in claim 1 or 2, wherein: the method comprises the steps of alternately iterating a modern image noise reduction algorithm after the solution and replacement of related fidelity term sub-problems, namely iterating the formulas (2) and (3) until the algorithm converges, namely the difference value between x and v is in a preset threshold range, so as to obtain a reconstructed single-pixel image, namely high-quality single-pixel imaging under the condition of undersampling and noiselessness.
6. The single pixel imaging method taking noise into consideration under the undersampling condition is characterized in that: comprises the steps of,
Step 201: constructing an optical path according to a single-pixel imaging principle, and acquiring a one-dimensional optical signal smaller than the Nyquist sampling theorem by adopting a single-pixel detector;
Step 202: establishing a single-pixel image imaging optimization model taking noise into consideration under the undersampling condition based on an alternating direction multiplier sub-algorithm (ALTERNATING DIRECTION METHOD OF MULTIPLIER ALGORITHM, ADMM);
Step 203: decomposing the optimization problem into two sub-problems and a dual variable updating step according to the fidelity term, the prior term and the dual variable by combining the alternating direction multiplier algorithm;
Step 204: combining Plug-and-Play (PnP), regarding the optimization sub-problem of the prior item as an image noise reduction problem, and replacing the optimization sub-problem solution of the prior item with a modern image noise reduction algorithm;
Step 205: through the solution to the related fidelity term sub-problem, the modern image noise reduction algorithm after replacement and the dual variable update alternate iteration, the high-quality single-pixel image reconstruction under the undersampling condition can be realized under the condition of taking noise into consideration, and the noise robustness is realized.
7. The method of single pixel imaging taking noise into account under undersampling conditions as in claim 6, wherein: the single-pixel image imaging optimization model taking noise into consideration under the undersampling condition in step 202 is an optimization model described in formula (4):
Wherein, therein N is the number of pixels of the image to be restored; /(I)The structured light patterns are random gray scale, and m is the number of the structured light patterns; /(I)M measurements received for a single pixel detector; according to the alternating direction multiplier algorithm ADMM, the above-mentioned optimization equation solution needs to be converted into an augmented lagrangian form:
。
8. The method of single pixel imaging taking noise into account under undersampling conditions as in claim 7, wherein: the optimization problem described in step 203 is decomposed as shown in equation (6):
Wherein the method comprises the steps of Regarding the solution of the fidelity sub-problem (6-1), according to the alternating direction multiplier algorithm ADMM, the solution is:
xk+1＝(ATA+pI)-1(ATb+pvk) (7)。
9. the method of single pixel imaging taking noise into account under undersampling conditions as in claim 8, wherein: the a priori term optimization sub-problem replacement described in step 204 is as shown in equation (8):
wherein D σ is an image noise reducer, and a convolutional neural network with a noise reduction function in deep learning is adopted.
10. The method of single pixel imaging taking noise into account under undersampling conditions as in claim 9, wherein: the reconstructed single-pixel image is obtained by alternately iterating the solution and the replaced modern image noise reduction algorithm of the related fidelity term sub-problem and the update of the dual variable, namely iterating the formulas (7), (8) and (6-3) until the difference value of x and v is in a preset threshold range, namely realizing high-quality single-pixel imaging under the condition of undersampling and noise consideration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795952.7A CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795952.7A CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113724146A CN113724146A (en) | 2021-11-30 |
| CN113724146B true CN113724146B (en) | 2024-06-04 |
Family
ID=78673287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110795952.7A Active CN113724146B (en) | 2021-07-14 | 2021-07-14 | Single-pixel imaging method based on plug-and-play prior |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113724146B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117408875A (en) * | 2023-12-13 | 2024-01-16 | 阿尔玻科技有限公司 | Method, storage medium and apparatus for reconstructing compressed ultrafast photographic images |
| CN117876837B (en) * | 2024-03-11 | 2024-06-07 | 北京理工大学 | Near infrared single-pixel imaging method and system based on depth expansion network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259991A (en) * | 2020-02-28 | 2020-06-09 | 南京理工大学 | Under-sampling single-pixel imaging target identification method in noise environment |
| CN112203068A (en) * | 2020-09-30 | 2021-01-08 | 中国科学院长春光学精密机械与物理研究所 | Single-pixel imaging method, system, device and medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU6915400A (en) * | 1999-08-16 | 2001-03-19 | Z-Force Corporation | System of reusable software parts and methods of use |
| US11125844B2 (en) * | 2018-07-06 | 2021-09-21 | The Board Of Trustees Of The Leland Stanford Junior University | Deep learning based methods to accelerate multi-spectral imaging |
| US11726184B2 (en) * | 2019-03-08 | 2023-08-15 | Leddartech Inc. | Component for a LIDAR sensor system, LIDAR sensor system, LIDAR sensor device, method for a LIDAR sensor system and method for a LIDAR sensor device |
-
2021
- 2021-07-14 CN CN202110795952.7A patent/CN113724146B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259991A (en) * | 2020-02-28 | 2020-06-09 | 南京理工大学 | Under-sampling single-pixel imaging target identification method in noise environment |
| CN112203068A (en) * | 2020-09-30 | 2021-01-08 | 中国科学院长春光学精密机械与物理研究所 | Single-pixel imaging method, system, device and medium |
Non-Patent Citations (1)
| Title |
|---|
| 计算摄影综述;徐树奎;涂丹;李国辉;张军;;计算机应用研究;20101115(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113724146A (en) | 2021-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10593021B1 (en) | Motion deblurring using neural network architectures | |
| CN113724146B (en) | Single-pixel imaging method based on plug-and-play prior | |
| CN111340867A (en) | Depth estimation method and device for image frame, electronic equipment and storage medium | |
| CN113379601A (en) | Real world image super-resolution method and system based on degradation variational self-encoder | |
| CN113487739A (en) | Three-dimensional reconstruction method and device, electronic equipment and storage medium | |
| CN104243837A (en) | Vibration detection and remote sensing image recovery method based on single-exposure video reconstruction | |
| Xie et al. | S3im: Stochastic structural similarity and its unreasonable effectiveness for neural fields | |
| Wang et al. | Underwater self-supervised monocular depth estimation and its application in image enhancement | |
| Nakashima et al. | Learning to drop points for lidar scan synthesis | |
| Jia et al. | Depth measurement based on a convolutional neural network and structured light | |
| Zhu et al. | Wavelet based deep learning for depth estimation from single fringe pattern of fringe projection profilometry | |
| Zhao et al. | Sptr: Structure-preserving transformer for unsupervised indoor depth completion | |
| CN117036442A (en) | Robust monocular depth completion method, system and storage medium | |
| CN112785629A (en) | Aurora motion characterization method based on unsupervised deep optical flow network | |
| CN117152733A (en) | Geological material identification method, system and readable storage medium | |
| CN117132704A (en) | Three-dimensional reconstruction method of dynamic structured light, system and computing equipment thereof | |
| Hyneman et al. | Human face project | |
| Wu et al. | SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer | |
| Azzarelli et al. | Towards a robust framework for nerf evaluation | |
| Zhang et al. | Deep external and internal learning for noisy compressive sensing | |
| CN105160664A (en) | Low-rank model based compressed sensing video reconstruction method | |
| Whittle et al. | A deep learning approach to no-reference image quality assessment for monte carlo rendered images | |
| Seo et al. | A study on selective dehazing system based on haze detection network | |
| Dias et al. | Change Detection in Moving-Camera Videos Using a Shift-Invariant Dissimilarity Metric | |
| Teng et al. | Unsupervised learning method for underwater concrete crack image enhancement and augmentation based on cross domain translation strategy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |