CN113624691B - Spectral image super-resolution mapping method based on space-spectrum correlation - Google Patents
Spectral image super-resolution mapping method based on space-spectrum correlation Download PDFInfo
- Publication number
- CN113624691B CN113624691B CN202010379860.6A CN202010379860A CN113624691B CN 113624691 B CN113624691 B CN 113624691B CN 202010379860 A CN202010379860 A CN 202010379860A CN 113624691 B CN113624691 B CN 113624691B
- Authority
- CN
- China
- Prior art keywords
- sub
- pixel
- spectral
- correlation
- pixels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003595 spectral effect Effects 0.000 title claims abstract description 98
- 238000013507 mapping Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000001228 spectrum Methods 0.000 title claims abstract description 15
- 230000009021 linear effect Effects 0.000 claims abstract description 33
- 238000003384 imaging method Methods 0.000 claims abstract description 15
- 238000005457 optimization Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000002922 simulated annealing Methods 0.000 claims abstract description 6
- 230000006870 function Effects 0.000 claims description 15
- 230000005484 gravity Effects 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- YBJHBAHKTGYVGT-ZKWXMUAHSA-N (+)-Biotin Chemical compound N1C(=O)N[C@@H]2[C@H](CCCCC(=O)O)SC[C@@H]21 YBJHBAHKTGYVGT-ZKWXMUAHSA-N 0.000 claims description 4
- FEPMHVLSLDOMQC-UHFFFAOYSA-N virginiamycin-S1 Natural products CC1OC(=O)C(C=2C=CC=CC=2)NC(=O)C2CC(=O)CCN2C(=O)C(CC=2C=CC=CC=2)N(C)C(=O)C2CCCN2C(=O)C(CC)NC(=O)C1NC(=O)C1=NC=CC=C1O FEPMHVLSLDOMQC-UHFFFAOYSA-N 0.000 claims description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 230000002441 reversible effect Effects 0.000 claims description 3
- 230000009022 nonlinear effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 13
- 238000011156 evaluation Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000011158 quantitative evaluation Methods 0.000 description 2
- 238000000701 chemical imaging Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001932 seasonal effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000002553 single reaction monitoring Methods 0.000 description 1
- 238000013426 sirius red morphometry Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/255—Details, e.g. use of specially adapted sources, lighting or optical systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N2021/1765—Method using an image detector and processing of image signal
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N2021/1793—Remote sensing
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biochemistry (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Theoretical Computer Science (AREA)
- Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a spectral image super-resolution mapping method based on space-spectral correlation, which comprises the following steps: (1) Processing the original spectrum image through a hybrid space gravitation model based on a linear Euclidean distance to obtain space correlation; (2) Meanwhile, calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance to obtain spectral correlation; (3) Establishing a linear model to integrate the spatial correlation and the spectral correlation to obtain an optimization function with good linear and nonlinear properties; and (4) allocating the class label to each sub-pixel by using a class allocation method based on simulated annealing to obtain a super-resolution mapping result. The invention aims to overcome the influence of linear and nonlinear imaging conditions on super-resolution mapping through space-spectrum correlation, realize innovation of super-resolution mapping and provide technical support for wide application of spectral images.
Description
Technical Field
The invention relates to the technical field of remote sensing information processing, in particular to a spectral image super-resolution mapping method based on space-spectrum correlation.
Background
Land cover category mapping information is one of the important remote sensing data for ecosystem monitoring, population change and environmental protection. Spectral images (multispectral images and hyperspectral images) are often used to provide land cover category mapping information due to their rich spectral characteristics. However, due to the influence of imaging conditions, the resolution of the acquired spectral image is sometimes coarse, resulting in a large number of mixed pixels. These mixed pixels create challenges for obtaining accurate land cover mapping information. Although spectral unmixing can obtain an abundance image of the proportion of each land cover type from the spectral image, land cover type mapping information is not obtained. As a subsequent processing technique of spectral unmixing, super-resolution Mapping (SRM) may process mixed pixels at a sub-pixel level using land cover category Mapping information to obtain an SRM result. In SRM, each blended pixel is segmented into S × S sub-pixels according to a scaling factor S, and then a land cover category label is assigned to each sub-pixel to obtain an SRM result.
SRM results have been successfully applied in many areas such as water boundary extraction, building mapping, fire zone mapping and land cover change detection. There are two main types of SRM (initialization followed by optimization and soft-hard conversion) depending on the way the SRM results are obtained. In the initialization stage, labels are randomly allocated to sub-pixels, and then the position of each sub-pixel is gradually changed through an optimization algorithm to obtain a final SRM result. SRMs based on perimeter minimization, pixel swapping, neighborhood value, particle swarm optimization, and genetic algorithms all belong to this category. However, since this type of optimization process is complicated, a long processing time is generally required. Soft and hard translation is more popular due to its simple computational processing. Soft-hard conversion involves two steps (i.e., subpixel sharpening and class assignment). Firstly, obtaining a fine abundance image with a sub-pixel class proportion through sub-pixel sharpening. The sub-pixel sharpening is realized by super-resolution reconstruction, a back propagation neural network, a space attraction model, an index cokriging, a Hopfield neural network, target information and spectral characteristics. Labels are then assigned to all sub-pixels by class assignment according to these proportions. The class allocation method comprises simulated annealing, linear optimization, a highest score value, an object unit and a class unit. When appropriate assistance data is present, such as deep learning networks, multi-temporal images, panchromatic images, fine-scale information, and lidar data can be used to improve SRM results. In the study of SRM, it is generally believed that the reason for coarse spectral imaging is due to the influence of linear imaging conditions. Therefore, in most SRM simulation experiments, a simulated coarse spectral image as experimental data is obtained from a fine spectral image by linear down-sampling. However, some effects of imaging conditions, such as sensor variation, solar angle, and seasonal effects, also have non-linear behavior. These effects are also one of the main factors that contribute to the roughness of the spectral image. Conventional SRM methods tend to ignore the effects of non-linear imaging conditions and thus affect the final SRM result. Furthermore, although conventional SRM methods may utilize spectral characteristics of the spectral image to improve the final SRM result, these spectral characteristics are typically obtained from spectral unmixing results of the spectral image. Spectral unmixing errors can affect the accuracy of spectral characteristics.
In order to solve the problems, a Super-resolution Mapping method (SSC) Based on spatial-Spectral Correlation for Spectral image Super-resolution Mapping is proposed, wherein the SSC comprises spatial Correlation and Spectral Correlation. The mixed space gravitation model based on the linear Euclidean distance can effectively depict the space correlation on the pixel scale and the sub-pixel scale. Here, in order to overcome the influence of the linear imaging condition, spatial correlation is obtained using a hybrid spatial attraction. The Kullback-Leibler distance has proven to be a good non-linear estimator of the similarity of two Probability Density Functions (PDFs). In order to overcome the influence of nonlinear imaging conditions, the Kullback-Leibler distance is used for calculating the spectral characteristics, and a new spectral correlation is provided. The spatial correlation and the new spectral correlation are combined to obtain an optimization function with linear and nonlinear characteristics. And finally, according to the maximization of the optimization function, a class allocation method based on simulated annealing is adopted to allocate the label to each sub-pixel, and a final SRM result is obtained.
Disclosure of Invention
The purpose of the invention is as follows: the invention provides a novel spectral image super-resolution mapping method based on space-spectral correlation, which is used for establishing a more effective super-resolution mapping model, further overcoming linear and nonlinear imaging influences and improving the super-resolution mapping precision.
The technical scheme is as follows:
a spectral image super-resolution mapping based on space-spectral correlation comprises the following steps:
(1) Processing the original spectrum image through a hybrid space gravitation model based on a linear Euclidean distance to obtain space correlation; the method specifically comprises the following steps:
assume that the input spectral image Y has K pixels, N classes, and B spectral bands. Abundance image F n (N =1,2.., N) is obtained from the spectral image Y by spectral decomposition. F n Comprising a pixel P A (a =1,2,.. K) belongs to the nth class of ratio values. Assuming S as a scale factor, each pixel is divided into S × S sub-pixels p a (a＝1，2，...，KS 2 ). For spatial correlation, since the spatial correlation of pixel and sub-pixel scales is considered by calculating linear Euclidean distances, the hybrid spatial gravity model is used to generate nullsAnd the influence of linear imaging conditions is overcome by the cross correlation. In the hybrid spatial gravity model, p is a sub-pixel a Of the nth class
The calculation is as follows:
where delta is a weight parameter,
and
is for sub-pixel p a The spatial correlation of the nth class of the pixel scale and the sub-pixel scale can be obtained by equations (2) and (3):
wherein, F n (P J ) Is the central sub-pixel p a Is adjacent to the pixel P J A proportion belonging to the nth class. x is the number of aj Determining a sub-pixel p a And neighborhood sub-pixel p j Whether they belong to the same category n is given by equation (4). K I And K II Respectively the number of neighboring pixels and sub-pixels. Here 8 neighboring pixels or 8 neighboring sub-pixels are considered. d (p) a ，P J ) Is the central sub-pixel p a And adjacent pixel P J Linear euclidean distance between, d (p) a ，p j ) Is the central sub-pixel p a And adjacent sub-pixel p j Linear euclidean distance between them. Epsilon 1 And ε 2 Refers to digital model parameters.
According to the principle of spatial correlation maximization, C spa This can be obtained by equations (5) and (6):
(2) Calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance to obtain spectral correlation; the method comprises the following specific steps: aiming at the spectral correlation, the spectral correlation is obtained by calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance, and the influence of the nonlinear imaging condition is overcome. Firstly, an original spectral image Y is firstly up-sampled by using bicubic interpolation under a scale factor S, and an obtained up-sampled image has KS 2 A sub-pixel p a (a＝1，2，...，KS 2 ). Then, the center subpixel p a And neighborhood sub-pixel p j Between the Kullback-Leibler distance KLD (p) a ；p j ) Obtained from the formula (7).
KLD(p a ；p j )＝KL(p a ；p j )+KL(p j ；p a ) (7)
Wherein KL (p) a ；p j ) Is from the central sub-pixel p a To the adjacent sub-pixel p j Kullback-Leibler divergence. KL (p) j ；p a ) Is from adjacent sub-pixels p j To the central sub-pixel p a Kullback-Leibler divergence. KL (p) a ；p j ) Is defined as:
wherein
And
respectively, the central sub-pixel p a And adjacent sub-pixel p j Probability Density Function (PDF). Here all sub-pixels p are created directly using the spectral characteristics of the spectral image i Is/are as follows
Suppose that
Obeying a Gaussian distribution with sub-pixels p i The part of the 8 neighborhoods which are the centers is
Can be positioned as follows:
where B is the number of spectral bands. e and M are derived from
And calculating the sub-pixel value of the internal spectrum. e is an element of i B Is an average vector of
The M covariance matrix includes [ M b，c ]Wherein
M is a reversible symmetric matrix with positive determinant, | M | > 0.
The combinations (8) and (9) can be derived as:
KL(p j ；p a ) Also derived from the above process. For sub-pixel p a Of class n
The calculation is as follows:
wherein the number K of adjacent sub-pixels II Still set to 8.y is n (p a ；p j ) The central sub-pixel p is judged by an empirical threshold T a And adjacent sub-pixel p j Whether the same class n belongs to is defined as:
wherein, y n (p a ；p j ) =1 denotes a central subpixel p a And adjacent sub-pixel p j The PDFs of the spectral characteristics are similar, and the two sub-pixels are considered as the same class; conversely, y n (p a ；p j )＝0。
According to the Kullback-Leibler distance theory, the smaller the Kullback-Leibler distance between two sub-pixels, the higher the spectral correlation between the two sub-pixels. Thus, the spectral dependence C spe Is defined as:
(3) Will spatially correlate C spa And spectral correlation C spe And combining to obtain the optimization function F with good linear and nonlinear characteristics. The goal of the method presented herein is to maximize F, with the formula：
max F＝(1-θ)C spa +θC spe (14)
Wherein theta (0 & lttheta & lt 1) is used for balancing C spa And C spe The weight parameter of the influence in between.
And (3) distributing a category label to each sub-pixel according to the maximization of the function F by using a category distribution method based on simulated annealing to obtain a final super-resolution mapping result. In addition, the following constraint functions should be satisfied in the category assignment.
Where the first equation constraint means that each sub-pixel belongs to only one class. The second constraint ensures that the sub-pixel number of the class is fixed.
Has the advantages that: the method of the invention establishes a more effective super-resolution mapping model, further overcomes the linear and nonlinear imaging influence and improves the super-resolution mapping precision.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 (a) is a schematic diagram of an evaluation reference image of a ulissses multispectral remote sensing image dataset.
Fig. 2 (b) is a schematic diagram of a super-resolution mapping result of a Ulysses multispectral remote sensing image dataset based on a sub-pixel scale spatial gravitation model.
Fig. 2 (c) is a schematic diagram of a super-resolution mapping result of a Ulysses multispectral remote sensing image dataset based on space-spectrum interpolation.
Fig. 2 (d) is a schematic diagram of a super-resolution mapping result of a uyssses multispectral remote sensing image dataset based on a hybrid spatial gravity model.
Fig. 2 (e) is a schematic diagram of a super-resolution mapping result of a ulfosses multispectral remote sensing image dataset based on target spatial correlation.
Fig. 2 (f) is a schematic diagram of a super-resolution mapping result of a ulfosses multispectral remote sensing image dataset based on space-spectral correlation.
Fig. 3 (a) is a schematic diagram of an evaluation reference image of a KSC hyperspectral remote sensing image dataset.
Fig. 3 (b) is a schematic diagram of a super-resolution mapping result of the KSC hyperspectral remote sensing image dataset based on the sub-pixel scale spatial gravitation model.
Fig. 3 (c) is a schematic diagram of a super-resolution mapping result of the KSC hyperspectral remote sensing image data set based on space-spectrum interpolation.
FIG. 3 (d) is a schematic diagram of a super-resolution mapping result of the KSC hyperspectral remote sensing image data set based on the hybrid spatial gravitation model.
Fig. 3 (e) is a schematic diagram of a super-resolution mapping result of the KSC hyperspectral remote sensing image dataset based on the target spatial correlation.
Fig. 3 (f) is a schematic diagram of a super-resolution mapping result of the KSC hyperspectral remote sensing image data set based on spatial-spectral correlation.
Detailed Description
The invention is further elucidated with reference to the drawings and the embodiments.
The block diagram of the implementation of the method for Super-resolution Mapping (Super-resolution Mapping Based on Space-Spectrum Correlation for Spectral image, SSC) of the Spectral image Based on the spatial-Spectral Correlation is shown in fig. 1. As shown in FIG. 1, the super-resolution mapping method for the spectral image based on the spatial-spectral correlation of the present invention includes the following steps:
(1) Processing the original spectral image through a hybrid space gravitation model based on a linear Euclidean distance to obtain spatial correlation; the method specifically comprises the following steps:
assume that the input spectral image Y has K pixels, N classes, and B spectral bands. Abundance image F n (N =1,2.., N) is obtained from the spectral image Y by spectral decomposition. F n Includes a pixel P A (a =1,2,.. K) belongs to the nth class of ratio values. Assuming S as a scale factor, each pixel is divided into S × S sub-pixels p a (a＝1，2，...，KS 2 ). For spatial correlation, the spatial correlation at the pixel and sub-pixel scales is considered by calculating linear Euclidean distancesAnd the spatial correlation is generated by using the hybrid space gravitation model, so that the influence of linear imaging conditions is overcome. In the hybrid spatial gravity model, p is a sub-pixel a Of the nth class
The calculation is as follows:
where delta is a weight parameter,
and
is for sub-pixel p a The spatial correlation of the nth class of pixel scale and sub-pixel scale can be obtained by equations (2) and (3):
wherein, F n (P J ) Is the central sub-pixel p a Is adjacent to the pixel P J A proportion belonging to the nth class. x is the number of aj Determining sub-pixel p a And neighborhood sub-pixel p j Whether they belong to the same class n is given by equation (4). K I And K II Respectively the number of neighboring pixels and sub-pixels. Here 8 neighboring pixels or 8 neighboring sub-pixels are considered. d (p) a ，P J ) Is the central sub-pixel p a And adjacent pixel P J Linear euclidean distance between, d (p) a ，p j ) Is the central sub-pixel p a And adjacent sub-pixel p j Linear euclidean distance between. Epsilon 1 And ε 2 Refers to digital model parameters.
According to the principle of spatial correlation maximization, C spa This can be obtained by equations (5) and (6):
(2) Calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance to obtain spectral correlation; the method specifically comprises the following steps: aiming at the spectral correlation, the spectral correlation is obtained by calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance, and the influence of nonlinear imaging conditions is overcome. Firstly, an original spectral image Y is firstly up-sampled by using bicubic interpolation under a scale factor S, and an obtained up-sampled image has KS 2 A sub-pixel p a (a＝1，2，...，KS 2 ). Then, the center subpixel p a And neighborhood sub-pixel p j The Kullback-Leibler distance KLD (p) between a ；p j ) Obtained from equation (7).
KLD(p a ；p j )＝KL(p a ；p j )+KL(p j ；p a ) (7)
Wherein KL (p) a ；p j ) Is from the central sub-pixel p a To the adjacent sub-pixel p j Kullback-Leibler divergence. KL (p) j ；p a ) Is from adjacent sub-pixels p j To the central sub-pixel p a Kullback-Leibler divergence. KL (p) a ；p j ) Is defined as:
wherein
And
respectively, the central sub-pixel p a And adjacent sub-pixel p j Probability Density Function (PDF). Here all sub-pixels p are created directly using the spectral characteristics of the spectral image i Is/are as follows
Suppose that
Obeying a Gaussian distribution with sub-pixels p i The part of the 8 neighbourhood that is the centre is
Can be positioned as follows:
where B is the number of spectral bands. e and M are derived from
And calculating the sub-pixel value of the internal spectrum. e is an element of i B Is an average vector in which
The M covariance matrix includes [ M b，c ]Wherein
M is a reversible symmetric matrix with positive determinant, | M | > 0.
The combinations (8) and (9) can be derived as:
KL(p j ；p a ) Also derived from the above process. For sub-pixel p a The spectral correlation of the nth class
The calculation is as follows:
wherein the number of adjacent sub-pixels K II Still set to 8.y is n (p a ；p j ) The central sub-pixel p is judged by an empirical threshold T a And adjacent sub-pixel p j Whether the same class n belongs to is defined as:
wherein, y n (p a ；p j ) =1 denotes a central subpixel p a And adjacent sub-pixel p j The PDFs of the spectral characteristics between the two sub-pixels are similar, and the two sub-pixels are considered as the same class; conversely, y n (p a ；p j )＝0。
According to the Kullback-Leibler distance theory, the smaller the Kullback-Leibler distance between two sub-pixels, the higher the spectral correlation between the two sub-pixels. Thus, the spectral dependence C spe Is defined as:
(3) Will spatial correlation C spa And spectral correlation C spe And combining to obtain the optimization function F with good linear and nonlinear characteristics. The method proposed hereinThe goal of the method is to maximize F, the formula:
max F＝(1-θ)C spa +θC spe (14)
wherein theta (0 & lttheta & lt 1) is used for balancing C spa And C spe The weight parameter of the influence in between.
And (3) according to the maximization of the function F, the class allocation method based on simulated annealing allocates class labels to each sub-pixel to obtain a final super-resolution mapping result. In addition, the following constraint functions should be satisfied in the category assignment.
Where the first equation constraint means that each sub-pixel belongs to only one class. The second constraint ensures that the sub-pixel number of the class is fixed.
Fig. 2 is a super-resolution mapping result of a ulissses multispectral remote sensing image dataset. Wherein: a) evaluating a reference image, b) a super-resolution mapping method (SPSAM) based on a sub-pixel scale spatial gravity model, c) a super-resolution mapping method (SSI) based on space-spectrum interpolation, d) a super-resolution mapping method (MSAM) based on a hybrid spatial gravity model, e) a super-resolution mapping method (OSD) based on target spatial correlation, f) a super-resolution mapping method (SSC) based on spatial-spectral correlation.
FIG. 3 is a super-resolution mapping result of a KSC hyperspectral remote sensing image data set. Wherein: a) evaluating a reference image, b) a super-resolution mapping method (SPSAM) based on a sub-pixel scale spatial gravity model, c) a super-resolution mapping method (SSI) based on space-spectrum interpolation, d) a super-resolution mapping method (MSAM) based on a hybrid spatial gravity model, e) a super-resolution mapping method (OSD) based on target spatial correlation, f) a super-resolution mapping method (SSC) based on spatial-spectral correlation.
We demonstrate the efficiency of the proposed method by applying it to two different sets of spectral images. For quantitative evaluation, the original fine telemetered image is downsampled using S =6 and adjusted to 50% brightness, i.e., L =50%, to produce a simulated low resolution image. Quantitative evaluation was performed using the location accuracy and overall accuracy evaluation (OA) and (Kappa coefficient, kappa) for each category.
In the first set of experiments, the target is a Ulysses multispectral remote sensing image dataset, FIG. 2 is a super-resolution mapping result of a low-resolution multispectral remote sensing image, and it can be seen from the figure that the result of the SSC method is closest to the reference image, and the effect is better. Table 1 further verifies the superior super-resolution mapping advantage of the proposed method for the mapping accuracy, overall accuracy evaluation OA and Kappa, for each class of the various methods in the first set of experiments.
And selecting a hyperspectral remote sensing image from the KSC in the second group of experiments. Fig. 3 shows super-resolution mapping results of four methods. Table 2 shows the overall accuracy evaluation OA and Kappa for each of the methods in the second set of experiments. Similar to the first set of experimental conclusions, the proposed method SCC still has significant advantages in the sub-super-resolution mapping method.
TABLE 1 data analysis results (%)
TABLE 2 data analysis results (%)
Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to the details of the foregoing embodiments, and various equivalent changes (such as number, shape, position, etc.) may be made within the technical scope of the present invention, and all such equivalent changes are within the protection scope of the present invention.
Claims (1)
1. A spectral image super-resolution mapping method based on space-spectrum correlation is characterized in that: the method comprises the following steps:
(1) Processing the original spectral image through a hybrid space gravitation model based on a linear Euclidean distance to obtain spatial correlation; the method comprises the following specific steps:
assume that the input spectral image Y has K pixels, N classes, and B spectral bands; abundance image F n (N =1,2,.., N) is obtained from the spectral image Y by spectral decomposition; f n Includes a pixel P A (a =1,2,.. K) a ratio value belonging to the nth class; let S be a scale factor, each pixel is divided into S × S sub-pixels p a (a＝1，2，...，KS 2 )；
Aiming at the spatial correlation, the linear Euclidean distance is calculated to consider the spatial correlation of the pixel and the sub-pixel scale, and the mixed spatial gravity model is used for generating the spatial correlation, so that the influence of the linear imaging condition is overcome; in the hybrid spatial gravity model, p is a sub-pixel a Of the nth class
The calculation is as follows:
where delta is a weight parameter,
and
is for sub-pixel p a The spatial correlation of the nth class of pixel scale and sub-pixel scale can be obtained by equations (2) and (3):
wherein, F n (P J ) Is the central sub-pixel p a Of adjacent pixel P J A proportion belonging to the nth class; x is the number of aj Determining sub-pixel p a And adjacent sub-pixel p j Whether the same class n belongs to is given by formula (4); k is I And K II The number of adjacent pixels and sub-pixels, respectively; here consider 8 neighboring pixels or 8 neighboring sub-pixels; d (p) a ，P J ) Is the central sub-pixel p a And adjacent pixel P J Linear euclidean distance between, d (p) a ，p j ) Is the central sub-pixel p a And adjacent sub-pixel p j Linear euclidean distance between; epsilon 1 And ε 2 Refers to a digital model parameter;
according to the principle of spatial correlation maximization, C spa This can be obtained by equations (5) and (6):
(2) Calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance to obtain spectral correlation; the method comprises the following specific steps:
aiming at the spectral correlation, the spectral correlation is obtained by calculating the spectral characteristics of the spectral image by utilizing the nonlinear Kullback-Leibler distance, so that the influence of nonlinear imaging conditions is overcome; firstly, an original spectral image Y is firstly up-sampled by using bicubic interpolation under a scale factor S, and then an obtained up-sampled imageHas KS 2 A sub-pixel p a (a＝1，2，...，KS 2 ) (ii) a Then, the center subpixel p a With adjacent sub-pixel p j Between the Kullback-Leibler distance KLD (p) a ；p j ) Obtained by the formula (7);
KLD(p a ；p j )＝KL(p a ；p j )+KL(p j ；p a ) (7)
wherein KL (p) a ；p j ) Is from the central sub-pixel p a To the adjacent sub-pixel p j The Kullback-Leibler divergence; KL (p) j ；p a ) Is from adjacent sub-pixels p j To the central sub-pixel p a The Kullback-Leibler divergence; KL (p) a ；p j ) Is defined as:
wherein
And
respectively, the central sub-pixel p a And adjacent sub-pixel p j The probability density function PDF of (1); here, the spectral characteristics of the spectral image are directly used to create all the sub-pixels p i Is/are as follows
Suppose that
Obeying a Gaussian distribution to sub-pixel p i The central 8 adjacent parts are
Can be positioned as follows:
wherein B is the number of spectral bands; e and M are derived from
Calculating the sub-pixel value of the internal spectrum;
is an average vector in which
M covariance matrix comprising
Wherein
M is a reversible symmetric matrix with positive determinant, | M | > 0;
the combinations (8) and (9) can be derived as:
KL(p j ；p a ) Also derived from the above process; for sub-pixel p a Of class n
The calculation is as follows:
wherein the number of adjacent sub-pixels K II Still set to 8; y is n (p a ；p j ) Is judged by an empirical threshold TCentral sub-pixel p a And adjacent sub-pixel p j Whether the same class n belongs to is defined as:
wherein, y n (p a ；p j ) =1 denotes the central subpixel p a And adjacent sub-pixel p j The PDFs of the spectral characteristics are similar, and the two sub-pixels are considered as the same class; conversely, y n (p a ；p j )＝0；
According to the Kullback-Leibler distance theory, the smaller the Kullback-Leibler distance between two sub-pixels is, the higher the spectral correlation between the two sub-pixels is; thus, the spectral dependence C spe Is defined as:
(3) Will spatial correlation C spa And spectral correlation C spe Combining to obtain an optimization function F with good linear and nonlinear characteristics; the goal of the method presented herein is to maximize F, with the formula:
max F＝(1-θ)C spa +θC spe (14)
wherein theta (0 & lttheta & lt 1) is used for balancing C spa And C spe A weight parameter of the influence between;
based on the class allocation method of simulated annealing, allocating a class label to each sub-pixel according to the maximization of the function F to obtain a final super-resolution mapping result; in addition, the following constraint functions should be satisfied when the categories are allocated:
where the first equation constraint means that each sub-pixel belongs to only one class, the second constraint ensures that the number of sub-pixels for a class is fixed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010379860.6A CN113624691B (en) | 2020-05-07 | 2020-05-07 | Spectral image super-resolution mapping method based on space-spectrum correlation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010379860.6A CN113624691B (en) | 2020-05-07 | 2020-05-07 | Spectral image super-resolution mapping method based on space-spectrum correlation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113624691A CN113624691A (en) | 2021-11-09 |
| CN113624691B true CN113624691B (en) | 2022-10-04 |
Family
ID=78377011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010379860.6A Active CN113624691B (en) | 2020-05-07 | 2020-05-07 | Spectral image super-resolution mapping method based on space-spectrum correlation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113624691B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117522687B (en) * | 2023-11-03 | 2024-05-14 | 西安电子科技大学 | Super-resolution reconstruction method of hyperspectral image based on particle dynamics |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682441A (en) * | 2012-03-01 | 2012-09-19 | 清华大学 | Hyperspectral image super-resolution reconstruction method based on subpixel mapping |
| CN109727195A (en) * | 2018-12-25 | 2019-05-07 | 成都元点智库科技有限公司 | A kind of image super-resolution reconstructing method |
| CN110084747A (en) * | 2019-03-15 | 2019-08-02 | 南京航空航天大学 | Spatial attraction model sub-pixed mapping localization method under being supported based on panchromatic sharpening technique |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2952216B1 (en) * | 2009-10-29 | 2011-12-30 | Galderma Res & Dev | METHOD AND APPARATUS FOR ANALYZING HYPER-SPECTRAL IMAGES |
| WO2018134729A1 (en) * | 2017-01-18 | 2018-07-26 | Technion Research & Development Foundation Ltd. | Sparsity-based ultrasound super-resolution imaging |
| TWI624807B (en) * | 2017-09-27 | 2018-05-21 | Taichung Veterans General Hosital | Iterative analysis of medical images |
| CN108509380A (en) * | 2018-03-27 | 2018-09-07 | 北京科技大学 | A kind of high spectrum image solution mixing method |
| CN109035154B (en) * | 2018-06-07 | 2021-08-20 | 复旦大学 | Hyperspectral image nonlinear unmixing method considering spectral variability |
| CN110070518B (en) * | 2019-03-15 | 2023-05-23 | 南京航空航天大学 | Hyperspectral image super-resolution mapping method based on dual-path support |
| CN110210300B (en) * | 2019-04-26 | 2023-05-26 | 南京航空航天大学 | Urban construction sub-pixel positioning method integrating multispectral image space-spectrum information |
-
2020
- 2020-05-07 CN CN202010379860.6A patent/CN113624691B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682441A (en) * | 2012-03-01 | 2012-09-19 | 清华大学 | Hyperspectral image super-resolution reconstruction method based on subpixel mapping |
| CN109727195A (en) * | 2018-12-25 | 2019-05-07 | 成都元点智库科技有限公司 | A kind of image super-resolution reconstructing method |
| CN110084747A (en) * | 2019-03-15 | 2019-08-02 | 南京航空航天大学 | Spatial attraction model sub-pixed mapping localization method under being supported based on panchromatic sharpening technique |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113624691A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108038445B (en) | SAR automatic target identification method based on multi-view deep learning framework | |
| Wang et al. | Allocating classes for soft-then-hard subpixel mapping algorithms in units of class | |
| Wang et al. | Land cover change detection at subpixel resolution with a Hopfield neural network | |
| Xiang et al. | Superpixel generating algorithm based on pixel intensity and location similarity for SAR image classification | |
| CN110070518B (en) | Hyperspectral image super-resolution mapping method based on dual-path support | |
| Wang et al. | Indicator cokriging-based subpixel mapping without prior spatial structure information | |
| Wang et al. | Fast subpixel mapping algorithms for subpixel resolution change detection | |
| CN107067405B (en) | Remote sensing image segmentation method based on scale optimization | |
| Shi et al. | Adaptive multi-scale deep neural networks with perceptual loss for panchromatic and multispectral images classification | |
| CN110728197B (en) | Single-tree-level tree species identification method based on deep learning | |
| Chen et al. | Hybrid constraints of pure and mixed pixels for soft-then-hard super-resolution mapping with multiple shifted images | |
| CN111259906A (en) | Method for generating and resisting remote sensing image target segmentation under condition containing multilevel channel attention | |
| Ling et al. | Superresolution land cover mapping using spatial regularization | |
| CN111738055B (en) | Multi-category text detection system and bill form detection method based on same | |
| CN102446278B (en) | Multitemporal remote sensing image-based subpixel positioning method | |
| Ma et al. | A super-resolution convolutional-neural-network-based approach for subpixel mapping of hyperspectral images | |
| CN111461213A (en) | Training method of target detection model and target rapid detection method | |
| CN109215003B (en) | Image fusion method and device | |
| CN111414931A (en) | Multi-branch multi-scale small target detection method based on image depth | |
| Liu et al. | A comparison of sub-pixel mapping methods for coastal areas | |
| CN111583330B (en) | Multi-scale space-time Markov remote sensing image sub-pixel positioning method and system | |
| CN110298883B (en) | Remote sensing image sub-pixel positioning method based on extended random walk algorithm | |
| CN113624691B (en) | Spectral image super-resolution mapping method based on space-spectrum correlation | |
| CN110210300B (en) | Urban construction sub-pixel positioning method integrating multispectral image space-spectrum information | |
| CN115810149A (en) | High-resolution remote sensing image building extraction method based on superpixel and image convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |