CN116773018A - Space spectrum combined image reconstruction method and system for calculating spectrum imaging - Google Patents
Space spectrum combined image reconstruction method and system for calculating spectrum imaging Download PDFInfo
- Publication number
- CN116773018A CN116773018A CN202310738127.2A CN202310738127A CN116773018A CN 116773018 A CN116773018 A CN 116773018A CN 202310738127 A CN202310738127 A CN 202310738127A CN 116773018 A CN116773018 A CN 116773018A
- Authority
- CN
- China
- Prior art keywords
- spectrum
- spatial
- image
- spectral
- attention mechanism
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001228 spectrum Methods 0.000 title claims abstract description 176
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000003384 imaging method Methods 0.000 title claims abstract description 26
- 230000007246 mechanism Effects 0.000 claims abstract description 85
- 230000003595 spectral effect Effects 0.000 claims abstract description 73
- 238000012545 processing Methods 0.000 claims abstract description 28
- 235000001466 Ribes nigrum Nutrition 0.000 claims abstract description 26
- 241001312569 Ribes nigrum Species 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000005070 sampling Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 27
- 238000000701 chemical imaging Methods 0.000 claims description 14
- 239000006185 dispersion Substances 0.000 claims description 11
- 230000004927 fusion Effects 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 7
- 238000012549 training Methods 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000002601 radiography Methods 0.000 claims 2
- 238000002591 computed tomography Methods 0.000 claims 1
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 7
- 238000011156 evaluation Methods 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001575 pathological effect Effects 0.000 description 1
- 239000010453 quartz Substances 0.000 description 1
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/42—Absorption spectrometry; Double beam spectrometry; Flicker spectrometry; Reflection spectrometry
- G01J3/433—Modulation spectrometry; Derivative spectrometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/457—Correlation spectrometry, e.g. of the intensity
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J2003/283—Investigating the spectrum computer-interfaced
Landscapes
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Image Processing (AREA)
Abstract
The invention provides a spatial spectrum combined image reconstruction method and a spatial spectrum combined image reconstruction system for calculating spectrum imaging, comprising the following steps: obtaining a two-dimensional aliasing image of the target object according to the CASSI system; preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image; alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image to determine spatial characteristics; among these, the spatial attention mechanism includes: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics; alternately carrying out up-sampling, spectrum attention mechanism and global jump connection processing on the space characteristics to determine a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global hopping connection includes: the spectral features and the intermediate spatial features of the corresponding scale are added. The invention improves the accuracy of reconstructing hyperspectral images.
Description
Technical Field
The present invention relates to the field of spectral imaging. In particular to a spatial spectrum combined image reconstruction method and a spatial spectrum combined image reconstruction system for calculating spectrum imaging.
Background
At present, the spectrum imaging technology is widely applied to the fields of biomedicine, remote sensing, quality detection of printed matters and the like. The spectrum imager can simultaneously contain two-dimensional space information and one-dimensional spectrum information of a target object during shooting to generate a three-dimensional data cube, but is influenced by hardware equipment, and an optical system based on space scanning or spectrum scanning needs extremely long imaging time; moreover, the measured data cubes present problems in storage and transmission, resulting in low accuracy of the final imaging.
In the prior art, applying compressed sensing theory to compressed imaging techniques of spectral imaging systems can greatly ameliorate these problems, such as snapshot compressed spectral imaging techniques. The snapshot compressed spectral imaging refers to a compressed imaging system that maps a multi-channel hyperspectral image into one measurement. Currently, a variety of snapshot compressed spectral systems have been proposed, the most typical of which is coded aperture compressed spectral imaging (Coded Aperture Snapshot Spectral Imaging, CASSI). The CASSI system encodes an input hyperspectral image into a two-dimensional compressed image through a mask, and then restores three-dimensional or multidimensional information using an algorithm. The system has the advantages of high acquisition speed, simple structure, low cost and lower reconstruction quality, so that how to improve the quality of the reconstructed spectrum image is the key of wide application of the CASSI system.
Disclosure of Invention
The invention is based on the above-mentioned needs of the prior art, and the technical problem to be solved by the invention is to provide a spatial spectrum combined image reconstruction method and a spatial spectrum combined image reconstruction system for calculating spectrum imaging so as to improve the accuracy of the reconstructed spectrum image.
In order to solve the problems, the invention is realized by adopting the following technical scheme:
a method for reconstructing a spatial spectrum combined image of computational spectrum imaging comprises the following steps:
obtaining a two-dimensional aliasing image of the target object according to the CASSI system;
preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image;
alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image, and determining spatial features obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics;
alternately performing up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to determine a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global hopping connection includes: the spectral features and the intermediate spatial features of the corresponding scale are added.
Optionally, the preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image includes:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H,W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n The data is composed of 1+d (n-1) data to W+d (n-1) data of the two-dimensional aliasing image, i is not less than 1<C, i is a positive number.
Optionally, the fusing the spectrum global feature and the spectrum local feature obtained by performing attention mechanism operation on the input data according to spectrum dimension expansion to obtain a spectrum feature includes:
performing attention mechanism operation on input data to obtain spectrum global characteristics; dividing input data into blocks according to a preset size, and carrying out attention mechanism operation on each block of data to obtain spectrum local characteristics; and fusing the spectrum global features and the corresponding spectrum local features to obtain spectrum features.
Optionally, the attention mechanism operation includes:
expanding input data according to dimensions to obtain three second-order tensors;
performing linear operation on each tensor respectively;
Q＝Linear(X 2D )
K＝Linear(X 2D )
V＝Linear(X 2D )
wherein Linear (·) represents a Linear operation, Q represents a query vector, K represents a key vector, and V represents a value vector;
and processing a linear operation result based on the Softmax function to obtain an attention score value, wherein the formula is as follows:
wherein X is 2D Representing a two-dimensional aliased image, d k Representing the number of pixels.
Optionally, the fusing the spectral global feature and the corresponding spectral local feature includes:
and splicing the attention score values of the spectrum global features and the spectrum local features, and performing 1*1 convolution processing on the spliced results to obtain a fusion result.
Optionally, the method further comprises:
constructing a spatial-spectral joint attention model according to an alternating spatial attention module and a pooling layer, and an alternating first convolution layer, a spectral attention module and a second convolution layer, wherein the spatial attention module executes a spatial attention mechanism, the pooling layer performs a downsampling operation, the first convolution layer performs an upsampling operation, the spectral attention module executes a spectral attention mechanism, and the second convolution layer performs a global jump connection operation;
and training the spatial spectrum joint attention model by using a two-dimensional aliasing image corresponding to the known hyperspectral image.
Optionally, the method further comprises:
using a reverse gradient propagation algorithm, according to a loss function Updating the weight of the spatial spectrum joint attention neural network, wherein +_>Pixels of reconstructed hyperspectral image representing the ith position, x i The pixel of the known hyperspectral image representing the ith position, MSE (-) represents the mean square error, perceptual Loss represents the Perceptual Loss, and λ represents the weighting factor of the Perceptual Loss.
A computed-spectrum-imaged spatial-spectral-joint-image reconstruction system, comprising:
the acquisition module is used for acquiring a two-dimensional aliasing image of the target object according to the CASSI system;
the first processing module is used for preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image;
the second processing module is used for alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image and determining the spatial characteristics obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics;
the reconstruction module is used for alternately carrying out up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to obtain a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global jump connection processing includes: the spectral fusion features and the corresponding intermediate spatial features are added.
Optionally, the first processing module is configured to:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H，W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n Data from the n+d-th data to the w+d (lambda) th of the two-dimensional aliased image n -1) data formation, i < C, i being a positive number.
A computer readable storage medium having stored thereon a computer program, said computer readable storage medium having stored thereon a computed-spectrum-imaged spatial-spectral-joint-image reconstruction program, which, when executed by a processor, implements the steps of said computed-spectrum-imaged spatial-spectral-joint-image reconstruction method.
Compared with the prior art, the spatial spectrum combined image reconstruction method and the spatial spectrum combined image reconstruction system for calculating the spectral imaging can give consideration to both spatial global information and spectral global information in the reconstruction process through spatial attention coding and spectral attention decoding. In the spectrum attention module, the problem that the spectrum correlation caused by directly regarding all information of a spectrum band as a vector cannot be fully utilized is solved, and the problem that the reconstruction precision is not high caused by the fact that the space spectrum correlation of a hyperspectral data cube is excavated and utilized inadequately in the prior art is also solved.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present description, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flowchart of a method for reconstructing a spatial spectrum joint image for computed-spectrum imaging according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network structure of a method for reconstructing a spatial spectrum joint image of computed-spectrum imaging according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a spectrum attention module of a method for reconstructing a spatial spectrum combined image for computing spectral imaging according to an embodiment of the present invention;
fig. 4 is a schematic diagram showing a comparison between a reconstructed spectrum and a true spectrum of a spatial spectrum combined image reconstruction method for calculating spectral imaging according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
For the purpose of facilitating an understanding of the embodiments of the present invention, reference will now be made to the following description of specific embodiments, taken in conjunction with the accompanying drawings, which are not intended to limit the scope of the invention.
Example 1
The image reconstruction method is a key to the imaging system to obtain high quality images. For compressed hyperspectral imaging, the observed information quantity is far less than the information quantity to be reconstructed, so that the calculation and reconstruction process solves a pathological inversion problem, a traditional optimization algorithm generally utilizes priori information of hyperspectral images, one or more effective regular terms are modeled manually, and then the regular terms are used as punishment functions to reconstruct a target through iterative calculation. The reconstruction accuracy of such methods is mainly dependent on the validity of manually set regularization terms, which may lead to failure in the face of certain complex structured spectral image reconstructions. In addition, the method has the defects of low reconstruction speed and poor reconstruction quality, and severely limits the practicability of the compression imaging system. In the aspect of a deep learning method, a deep neural network is generally used for learning the mapping relation between observation information and a target image, and the reconstruction process is a simple forward calculation process and has the characteristic of high speed, so that the reconstruction speed can be greatly improved when the deep neural network is used for reconstructing a compressed imaging image. In addition, the learning capability of the prior features of different data can be improved by designing the structure of the neural network, and a high-quality reconstructed image is obtained. Therefore, a reconstruction method based on deep learning has become a currently mainstream reconstruction method.
In the prior art, the reconstruction image is generally a two-dimensional aliasing image and a mask obtained by a compression imaging technology to recover a hyperspectral image, and the specific process includes: firstly, a low-precision hyperspectral image with the same size as a target hyperspectral image is obtained from a two-dimensional aliasing image through a dispersion process in an inversion CASSI system as input, and then a convolutional neural network is used for learning the mapping relation between the low-precision image and the target image. The existing convolutional neural network comprises lambda-Net, TSA-Net, MST and the like, wherein the lambda-Net learns the spatial correlation of spectral image data by adding a self-attention mechanism module in a deep region of the U-Net network. The self-attention mechanism module obtains a space dimension attention matrix by carrying out one-dimensional expansion on a feature map of a network deep layer and calculating the correlation between pixels, and finally multiplies the attention matrix by the original feature map to output data with the same size as the original feature map. The TSA-Net divides the three-dimensional data cube into three dimensions of x, y and z in the decoding stage of the U-Net, attention matrixes in three dimensions are respectively modeled by convolution, the obtained attention matrixes are multiplied by the original feature graphs respectively, and finally the feature graphs with the convolved attention are output. Because of the limitations of convolutional neural networks in capturing long-range dependencies, these methods do not fully mine the spatial, spectral global information of the spectral images. The MST is integrally constructed by a mask guided spectrum attention module by adopting a U-shaped structure consisting of an encoder, a bottleneck and a decoder. The mask directs the spectral attention module to directly consider all information on each spectral band as a vector, then generates Q, K, V by using the fully connected layer, multiplies the attention matrix by V by Q, K, thereby obtaining the correlation between the vectors and obtaining global information of the spectral dimension. However, this approach models only the correlation of the optical dimensions, does not process spatial information separately, and the convolutional neural network-based approach cannot meet the requirement of fully utilizing global features due to the limitations of the convolution itself. In summary, these attention modes do not fully exploit the spatial dimension and the global features of the spectral dimension of the hyperspectral image, resulting in its poor reconstruction accuracy.
In the prior art, convolution attention is used for modeling the correlation of corresponding dimensions, and only modeling is performed on the correlation of single dimension, but global information of space dimension and global information of spectrum dimension cannot be considered. Therefore, none of these methods adequately mine the spatial, spectral global information of the spectral image, and in addition, the importance of local features is ignored when using attention to the spectral dimensions, which results in poor accuracy of the prior art reconstructed hyperspectral images.
Therefore, the embodiment of the invention provides a spatial spectrum combined image reconstruction method for calculating spectral imaging to improve the accuracy of reconstructed hyperspectral data, namely solving the problems of the prior art that the spatial spectrum correlation of a hyperspectral data cube is explored and utilized inadequately. The flow is shown in fig. 1, and comprises:
s1: a two-dimensional aliased image of the target object is obtained according to the CASSI system.
The CASSI system modulates signals of different wavelengths using a physical mask and a disperser, and mixes all the modulated signals to generate a two-dimensional aliased image. The CASSI system is divided into two types according to the number of dispersive elements, namely a single-dispersion coded aperture snapshot spectrometer and a double-dispersion coded aperture snapshot spectrometer, and the spectrometers of any type comprise an objective lens, a coded aperture, a relay lens, a dispersive element and an imaging lens, except for the number and the positions of all components. The coded aperture performs spatial coding, namely, spatial information of an object is modulated in a spatial dimension, and the spatial information can be realized by using a spatial light modulator or a quartz chromeplate mask, and the imaging in the last step is transmitted or blocked according to the spatial coding of the coded aperture. The dispersive element performs dispersion, namely, original light carrying object information is unfolded along a certain space dimension according to different wavelengths; the steps of dispersion and space coding are used for realizing the regulation and control of spectrum and space dimension, and the compressed spectrum imaging is completed.
In the present embodiment, it is assumed that a two-dimensional aliasing image of 256×310 in size of the target object is obtained
S2: and preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image.
In this embodiment, preprocessing refers to reconstructing a two-dimensional aliasing image received by the detector to obtain cube information to be processed by the dispersive element in the last step, that is, a state that different wavelengths of light rays of object information are not separated yet.
Preferably, the method specifically comprises the following steps:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H，W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n The data is composed of 1+d (n-1) data to w+d (n-1) data of the two-dimensional aliasing image, i is more than or equal to 1 and less than C, and i is a positive number.
For example, assuming a reconstruction spectral band number c=28, the reconstruction cube height H and width W are both 256, the step size d is 2, and λ is obtained according to the above formula 1 Consists of the first line data to the 256 th line data of the two-dimensional aliasing image, lambda 2 Consists of data from line 3 to line 258 of a two-dimensional aliased image 28 Consists of 55 th line data to 310 th line data of the two-dimensional aliasing image. Combining the data of each wave band to obtain a first hyperspectral image with the size of 256 multiplied by 28
S3: alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image, and determining spatial features obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: and performing attention mechanism operation on the input data according to the expansion of the space dimension to obtain intermediate space characteristics.
In this embodiment, the number of periods of alternation of the spatial attention mechanism and the downsampling step is greater than 2, and in the last alternation period, only the spatial attention mechanism appears, the spatial feature is obtained, and another alternation operation of the next step is performed on the spatial feature.
Preferably, the attention mechanism operation includes:
expanding input data according to dimensions to obtain three second-order tensors;
performing linear operation on each tensor respectively;
Q＝Linear(X 2D )
K＝Linear(X 2D )
V＝Linear(X 2D )
where Linear (·) represents a Linear operation, Q represents a query vector, K represents a key vector, and V represents a value vector.
And processing a linear operation result based on the Softmax function to obtain an attention score value, wherein the formula is as follows:
wherein X is 2D Representing a two-dimensional aliased image, d k Representing the number of pixels.
The key vectors are weighted and summed by a given query vector; the weight is obtained by calculating the similarity between the query vector and the key vector; the weighted sum is multiplied by the value vector to obtain the attention score value.
The spatial attention mechanism in the encoding stage is mainly to expand the first hyperspectral image tensor into a two-dimensional image matrix X in the spatial dimension 2D The image matrix is then split into Q, K, V using a linear layer, and the extracted Q, K, V is then subjected to attention score calculation and mask superposition. For example, the first hyperspectral image of 256×256×28 is unfolded according to the spatial dimension to obtain 256×256 1×28 vectors, and correlation between spatial features is obtained through calculation.
S4: alternately performing up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to determine a second hyperspectral image; wherein the spectral attention mechanism comprises: the spectrum global feature and the spectrum local feature obtained by processing the input data are fused to obtain the spectrum feature; the global hopping connection includes: the spectral features and the intermediate spatial features of the corresponding scale are added.
In this step, the alternating sequence is to perform up-sampling on the spatial features, then perform spectral attention mechanism operation on the up-sampled result, and then perform global jump connection. The number of the alternate occurrence cycles in the step is more than or equal to 2, the number of the cycles is positive, and the up-sampling frequency is consistent with the down-sampling frequency.
Preferably, the fusing the spectrum global feature and the spectrum local feature obtained by performing attention mechanism operation on the input data according to spectrum dimension expansion to obtain spectrum features includes:
performing attention mechanism operation on input data to obtain spectrum global characteristics; dividing input data into blocks according to a preset size, and carrying out attention mechanism operation on each block of data to obtain spectrum local characteristics; and fusing the spectrum global features and the corresponding spectrum local features to obtain spectrum features.
In the decoding stage, the spectrum attention mechanism is divided into two paths, and firstly, the spatial characteristic image tensor is unfolded into a two-dimensional image matrix X according to the spectrum dimension 2D Then dividing an image matrix into Q, K and V by using a linear layer, and then carrying out attention score calculation and mask superposition on the extracted Q, K and V to obtain a spectrum global feature; the other path carries out the block operation on the generated Q, K and V, and then carries out the spectrum attention operation to obtain the attention score diagram of each small block, thus obtaining the spectrum dimension characteristics of the local attention mechanism. Finally, the features obtained in the two paths are fused and output. The global jump connection adds the spatial global features of different scales generated in the spatial dimension of the encoding stage to the features of the same scale in the spectral dimension of the decoding stage.
For example, performing attention mechanism operation on 28 256×256 vectors obtained by expanding a 256×256×28 first hyperspectral image according to a spectrum dimension to obtain a spectrum global feature, partitioning each vector of the 28 256×256 vectors according to a size of 32×32, performing attention mechanism operation on the partitioned vector to obtain a spectrum local feature, and fusing the spectrum global feature and the spectrum local feature to obtain a spectrum feature.
Preferably, the fusing the spectral global feature and the corresponding spectral local feature includes:
and splicing the attention score values of the spectrum global features and the spectrum local features, and performing 1*1 convolution processing on the spliced results to obtain a fusion result.
And then, adding the spectral features and the corresponding intermediate space features with the same data size, repeatedly and alternately executing up-sampling, a spectral attention mechanism and global jump connection for a plurality of times on the added hyperspectral image, and determining a second hyperspectral image as a spectral image of the target object.
Further, the embodiment of the invention further comprises:
constructing a spatial-spectral joint attention model according to an alternating spatial attention module and a pooling layer, and an alternating first convolution layer, a spectral attention module and a second convolution layer, wherein the spatial attention module executes a spatial attention mechanism, the pooling layer performs a downsampling operation, the first convolution layer performs an upsampling operation, the spectral attention module executes a spectral attention mechanism, and the second convolution layer performs a global jump connection operation;
and training the spatial spectrum joint attention model by using a two-dimensional aliasing image corresponding to the known hyperspectral image.
The method comprises the following specific steps:
the first step: a spatial spectrum joint attention model ReconNet was constructed as shown in fig. 2.
In the spatial attention module, the Vision Transformer architecture is used to treat the spectrum of each pixel in the input data as a vector to determine the correlation between the pixels. One path of the spectrum attention module regards each wave band of input data as a vector, the correlation among spectrums is determined, the other path of the spectrum attention module segments the input data according to the size of 32 multiplied by 32, attention mechanism operation is carried out on each block, and then characteristic fusion is realized by carrying out 1 multiplied by 1 convolution on the spliced characteristics of the two paths of characteristics in the spectrum dimension, as shown in fig. 3. In this embodiment, the scale of the spatial attention module and the spectrum attention module is set to 2, the pooling layer performs downsampling by a convolution kernel with the scale of 2×2 to achieve the effects of halving the width and doubling the number of channels, the first convolution layer is composed of transposed convolutions with the step length of 2 to achieve upsampling of input data, and the scaling factor is the same as downsampling. The second convolution layer enhances the stability and information flow of the network model by adding spatial features of the same scale to spectral features. Wherein B in the figure represents the number of first hyperspectral images input to the model at a time, and H, W, and C represent the height, width, and spectral band number of the first hyperspectral images, respectively.
And secondly, constructing a training set.
In this embodiment, a CAVE data set is selected, spectral interpolation and clipping and splicing operations are performed on the CAVE data set, 28 wave bands are derived, wavelengths of the 28 wave bands are between 450nm and 650nm, 205 hyperspectral data cubes with the size of 1024×1024 are used as training sets, and data cubes with the size of 256×256 are randomly clipped from the 28 wave bands each time to be used as training labels.
Thirdly, the training label is encoded through a mask to generate a two-dimensional aliasing image which is expressed as Y 0 ∈N 256×310 Where 256 and 310 represent the height and width, respectively, of the two-dimensional aliased image.
Fourth, data preprocessing is carried out on the two-dimensional aliasing image, and a preprocessed first hyperspectral image X is obtained 0 ∈N 256 ×256×28 Where 28 represents the spectral dimension of the preprocessed image, which is input into the ReconNet model, the weights of the model are updated according to the loss function.
Preferably, a backward gradient propagation algorithm is used, according to the loss function Updating the weight of the spatial spectrum joint attention neural network, wherein +_>Pixels of reconstructed hyperspectral image representing the ith position, x i The pixel of the known hyperspectral image representing the ith position, MSE (-) represents mean square error, perceptual Loss represents Perceptual Loss, is used for guaranteeing consistency of deep semantic information of the reconstructed hyperspectral image and the truth image, and lambda represents a weighting coefficient of the Perceptual Loss.
According toThe weight of Reconnet is updated by the loss of Reconnet, and the embodiment sets the initial learning rate of the model network to be 4×10 -4 And adopting a learning rate attenuation strategy with 50 steps as a period and Gamma of 0.5, and using an Adam optimizer to perform gradient descent, wherein the iteration step length is 300 times. 10 scenes from the KAIST dataset were selected for testing the model.
And fifthly, putting a first hyperspectrum obtained by preprocessing the two-dimensional aliasing image obtained by the CASSI system into a trained ReconNet model to obtain a second hyperspectral image serving as a spectrum image of the target object.
Finally, the reconstructed second hyperspectral image is evaluated through three evaluation indexes, namely structural similarity (Structure Similarity Index Measure, SSIM), peak Signal-to-Noise Ratio (PSNR) and spectral angle mapping (Spectral Angle Mapper, SAM), wherein the SSIM and the PSNR are used for evaluating a spatial reconstruction structure, and the SAM is used for evaluating spectral reconstruction quality. The average evaluation results of the objective evaluation indexes of the 10 images shown in table 1 show that the embodiment of the invention can well finish reconstructing the coded and modulated hyperspectral image; as can be seen from the comparison of the randomly selected reconstructed spectrum with the real spectrum shown in fig. 4, the reconstructed spectrum substantially coincides with the real spectrum. Where group Truth in the figure represents the real spectrum and Reconstruction represents the reconstructed spectrum.
TABLE 1
| Evaluation index | PSNR | SSIM | SAM |
| Evaluation results | 39.26 | 0.976 | 3.19 |
The embodiment of the invention designs a network into a coding-decoding structure, uses a spatial attention mechanism for coding and uses a spectral attention mechanism for decoding. Specifically, in the encoder, a spatial attention mechanism is used to extract spatial global features in the two-dimensional aliasing image, and the spatial global features under different scales are acquired along with a downsampling operation. In the decoder, the reconstructed image is obtained by decoding the spatial global features in the spectral dimension using an attention mechanism, accompanied by an upsampling operation, taking full advantage of the spatial, spectral global information. The method comprises the steps that when attention is used for a spectrum dimension, the problem of importance of local features is ignored, and a spectrum attention module respectively performs attention operation of global features and local features of the spectrum dimension to decode and reconstruct the features obtained by an encoder, so that a reconstructed hyperspectral image is obtained. Specifically, a two-dimensional aliasing image of the same target and a high-resolution hyperspectral image of the corresponding target are acquired first. First aliased images from two dimensions by data preprocessingObtain initialization input +.>Then inputting H into the proposed network to fully utilize the space and spectrum global information, wherein the formula is as follows:
H′＝ReconNet(H)
wherein ReconNet is a supervised reconstruction network, H ′ Is a reconstructed hyperspectral image.
In the prior art, in a spectrum attention mechanism, the information of the whole wave band of a hyperspectral image is often directly regarded as a vector so as to calculate the correlation between spectrum wave bands, but the information of the whole wave band is regarded as a vector between the large variation of the information of the space region of the spectrum image, so that the spectrum correlation between the images cannot be fully explored. The embodiment of the invention utilizes the local characteristics in the spectrum attention module in the decoding stage, so that the problems of the development and the underutilization of the spatial spectrum correlation of the hyperspectral data cube are solved.
The embodiment of the invention designs a multidimensional data reconstruction algorithm on a two-dimensional aliasing image obtained from a compressed spectrum imaging system with a monochromatic scattered code aperture by utilizing a deep learning technology, realizes the function of reconstructing a hyperspectral image from compressed sampling, and can be further expanded to the field of video compression calculation imaging and the like.
The embodiment of the invention provides a spatial spectrum combined image reconstruction method for calculating spectrum imaging, which can give consideration to spatial global information and spectral global information in the reconstruction process through spatial attention coding and spectral attention decoding. In the spectrum attention module, the problem that the spectrum correlation caused by directly regarding all information of a spectrum band as a vector cannot be fully utilized is solved, and the problem that the reconstruction precision is not high caused by the fact that the space spectrum correlation of a hyperspectral data cube is excavated and utilized inadequately in the prior art is also solved.
Example 2
The embodiment of the invention provides a spatial spectrum combined image reconstruction system for calculating spectrum imaging, which comprises the following steps:
the acquisition module is used for acquiring a two-dimensional aliasing image of the target object according to the CASSI system;
the first processing module is used for preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image;
the second processing module is used for alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image and determining the spatial characteristics obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics;
the reconstruction module is used for alternately carrying out up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to obtain a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global jump connection processing includes: the spectral fusion features and the corresponding intermediate spatial features are added.
Preferably, the first processing module is configured to:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H，W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n Data from 1+d (n-1) data to W+d (n-1) data of the two-dimensional aliased imageThe composition is that i is more than or equal to 1 and less than C, i is a positive number.
Preferably, the reconstruction module is configured to:
performing attention mechanism operation on input data to obtain spectrum global characteristics; dividing input data into blocks according to a preset size, and carrying out attention mechanism operation on each block of data to obtain spectrum local characteristics; and fusing the spectrum global features and the corresponding spectrum local features to obtain spectrum features.
Preferably, the attention mechanism operation includes:
expanding input data according to dimensions to obtain three second-order tensors;
performing linear operation on each tensor respectively;
Q＝Linear(X 2D )
K＝Linear(X 2D )
V＝Linear(X 2D )
and processing a linear operation result based on the Softmax function to obtain an attention score value, wherein the formula is as follows:
wherein X is 2D Representing a two-dimensional aliased image, d k Representing the number of pixels.
Preferably, the reconstruction module is configured to:
and splicing the attention score values of the spectrum global features and the spectrum local features, and performing 1*1 convolution processing on the spliced results to obtain a fusion result.
A computer readable storage medium having stored thereon a computer program, said computer readable storage medium having stored thereon a computed-spectrum-imaged spatial-spectral-joint-image reconstruction program, which, when executed by a processor, implements the steps of said computed-spectrum-imaged spatial-spectral-joint-image reconstruction method.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. A method for reconstructing a spatial-spectral joint image of computed-spectral imaging, comprising:
obtaining a two-dimensional aliasing image of the target object according to the CASSI system;
preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image;
alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image, and determining spatial features obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics;
alternately performing up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to determine a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global hopping connection includes: the spectral features and the intermediate spatial features of the corresponding scale are added.
2. The method for reconstructing a spatial spectrum combined image for computed tomography according to claim 1, wherein said preprocessing said two-dimensional aliased image to obtain a first hyperspectral image comprises:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H,W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n The data is composed of 1+d (n-1) data to W+d (n-1) data of the two-dimensional aliasing image, i is not less than 1<C, i is a positive number.
3. The method for reconstructing a spatial spectrum combined image for calculating spectral imaging according to claim 1, wherein the fusing the spectral global features and the spectral local features obtained by performing attention mechanism operation on the input data according to spectral dimension expansion to obtain spectral features comprises:
performing attention mechanism operation on input data to obtain spectrum global characteristics; dividing input data into blocks according to a preset size, and carrying out attention mechanism operation on each block of data to obtain spectrum local characteristics; and fusing the spectrum global features and the corresponding spectrum local features to obtain spectrum features.
4. A method of spatial spectrum joint image reconstruction for computed-spectrum imaging according to claim 1, wherein the attention mechanism operation comprises:
expanding input data according to dimensions to obtain three second-order tensors;
performing linear operation on each tensor respectively;
Q＝Linear(X 2D )
K＝Linear(X 2D )
V＝Linear(X 2D )
wherein Linear (·) represents a Linear operation, Q represents a query vector, K represents a key vector, and V represents a value vector;
and processing a linear operation result based on the Softmax function to obtain an attention score value, wherein the formula is as follows:
wherein X is 2D Representing a two-dimensional aliased image, d k Representing the number of pixels.
5. A method of spatial spectral joint image reconstruction for computed radiography according to claim 3, wherein said fusing said spectral global features with corresponding spectral local features comprises:
and splicing the attention score values of the spectrum global features and the spectrum local features, and performing 1*1 convolution processing on the spliced results to obtain a fusion result.
6. A method of spatial spectrum joint image reconstruction for computed-spectrum imaging as in claim 1, further comprising:
constructing a spatial-spectral joint attention model according to an alternating spatial attention module and a pooling layer, and an alternating first convolution layer, a spectral attention module and a second convolution layer, wherein the spatial attention module executes a spatial attention mechanism, the pooling layer performs a downsampling operation, the first convolution layer performs an upsampling operation, the spectral attention module executes a spectral attention mechanism, and the second convolution layer performs a global jump connection operation;
and training the spatial spectrum joint attention model by using a two-dimensional aliasing image corresponding to the known hyperspectral image.
7. The method for spatial spectral joint image reconstruction for computed radiography according to claim 6, further comprising:
using a reverse gradient propagation algorithm, according to a loss function Updating the weight of the spatial spectrum joint attention neural network, wherein +_>Pixels of reconstructed hyperspectral image representing the ith position, x i The pixel of the known hyperspectral image representing the ith position, MSE (-) represents the mean square error, perceptual Loss represents the Perceptual Loss, and λ represents the weighting factor of the Perceptual Loss.
8. A computed-spectrum-imaged spatial-spectral-joint-image reconstruction system, comprising:
the acquisition module is used for acquiring a two-dimensional aliasing image of the target object according to the CASSI system;
the first processing module is used for preprocessing the two-dimensional aliasing image to obtain a first hyperspectral image;
the second processing module is used for alternately carrying out a spatial attention mechanism and downsampling on the first hyperspectral image and determining the spatial characteristics obtained by the spatial attention mechanism of the last alternate period; wherein the spatial attention mechanism comprises: performing attention mechanism operation on input data according to space dimension expansion to obtain intermediate space characteristics;
the reconstruction module is used for alternately carrying out up-sampling, spectrum attention mechanism and global jump connection processing on the spatial features to obtain a second hyperspectral image; wherein the spectral attention mechanism comprises: the method comprises the steps of fusing a spectrum global feature and a spectrum local feature which are obtained by performing attention mechanism operation on input data according to spectrum dimension expansion, and obtaining spectrum features; the global jump connection processing includes: the spectral fusion features and the corresponding intermediate spatial features are added.
9. The computed-spectral-imaging spatial-temporal-spectral-joint-image-reconstruction system of claim 8, wherein the first processing module is configured to:
determining the number C of reconstructed spectrum bands according to the number of wavelengths modulated by the CASSI system on the target object;
determining a height H and a width W of a reconstruction cube based on the rows of the two-dimensional aliased image;
using the formula H (H, W, n λ )＝Y(H,W-d(λ n -λ c ) Reconstructing a first hyperspectral image, wherein H (H, W, n) λ ) Represents a first hyperspectral image, d represents the step size of chromatic dispersion in a CASSI system, H, W, C represent the height, width and spectral band number of the hyperspectral image, lambda respectively n N-th band, lambda representing hyperspectral image c Indicating the total number of wave bands contained in the hyperspectral image; wherein lambda is n Data from the n+d-th data to the w+d (lambda) th of the two-dimensional aliased image n -1) data formation, 1.ltoreq.i<C, i is a positive number.
10. A computer readable storage medium having stored thereon a computer program having stored thereon a computed-spectrum imaged spatial-spectral-joint-image reconstruction program which, when executed by a processor, implements the steps of a computed-spectrum imaged spatial-spectral-joint-image reconstruction method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310738127.2A CN116773018A (en) | 2023-06-21 | 2023-06-21 | Space spectrum combined image reconstruction method and system for calculating spectrum imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310738127.2A CN116773018A (en) | 2023-06-21 | 2023-06-21 | Space spectrum combined image reconstruction method and system for calculating spectrum imaging |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116773018A true CN116773018A (en) | 2023-09-19 |
Family
ID=87992537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310738127.2A Pending CN116773018A (en) | 2023-06-21 | 2023-06-21 | Space spectrum combined image reconstruction method and system for calculating spectrum imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116773018A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117589086A (en) * | 2023-11-22 | 2024-02-23 | 西湖大学 | Spectrum three-dimensional imaging method, system and application based on fringe projection |
| CN118212536A (en) * | 2024-05-20 | 2024-06-18 | 南京理工大学 | Physical-guided super-resolution compression coding spectrum imaging method |
-
2023
- 2023-06-21 CN CN202310738127.2A patent/CN116773018A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117589086A (en) * | 2023-11-22 | 2024-02-23 | 西湖大学 | Spectrum three-dimensional imaging method, system and application based on fringe projection |
| CN118212536A (en) * | 2024-05-20 | 2024-06-18 | 南京理工大学 | Physical-guided super-resolution compression coding spectrum imaging method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Deng et al. | Machine learning in pansharpening: A benchmark, from shallow to deep networks | |
| Xie et al. | Hyperspectral image super-resolution using deep feature matrix factorization | |
| He et al. | A new pansharpening method based on spatial and spectral sparsity priors | |
| CN116773018A (en) | Space spectrum combined image reconstruction method and system for calculating spectrum imaging | |
| CN107525588B (en) | Rapid reconstruction method of dual-camera spectral imaging system based on GPU | |
| Chen et al. | Learning memory augmented cascading network for compressed sensing of images | |
| CN113160294B (en) | Image scene depth estimation method and device, terminal equipment and storage medium | |
| CN111260576A (en) | Hyperspectral unmixing algorithm based on de-noising three-dimensional convolution self-coding network | |
| CN108369728A (en) | The method and system that the measured value sensed is merged | |
| US11978146B2 (en) | Apparatus and method for reconstructing three-dimensional image | |
| CN114972803B (en) | Snapshot type spectrum imaging method and system based on joint optimization | |
| US20160173736A1 (en) | Method and System for Reconstructing Sampled Signals | |
| Vu et al. | Perception-enhanced image super-resolution via relativistic generative adversarial networks | |
| Thomas et al. | Comparing distances for quality assessment of fused images | |
| Gelb et al. | Reducing effects of bad data using variance based joint sparsity recovery | |
| CN117079105B (en) | Remote sensing image spatial spectrum fusion method and device, electronic equipment and storage medium | |
| Zhang et al. | ESSAformer: Efficient transformer for hyperspectral image super-resolution | |
| CN114331849B (en) | Cross-mode nuclear magnetic resonance hyper-resolution network and image super-resolution method | |
| Zhang et al. | Polarization image demosaicking via nonlocal sparse tensor factorization | |
| CN116205962A (en) | Monocular depth estimation method and system based on complete context information | |
| CN114862976A (en) | Multispectral image reconstruction method for rotation diffraction | |
| Chen et al. | PDWN: Pyramid deformable warping network for video interpolation | |
| CN111882512B (en) | Image fusion method, device and equipment based on deep learning and storage medium | |
| Xu et al. | AACNet: Asymmetric attention convolution network for hyperspectral image dehazing | |
| Wang et al. | Local conditional neural fields for versatile and generalizable large-scale reconstructions in computational imaging |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |