CN113409193B - Super-resolution reconstruction method and device for hyperspectral image - Google Patents
Super-resolution reconstruction method and device for hyperspectral image Download PDFInfo
- Publication number
- CN113409193B CN113409193B CN202110682864.6A CN202110682864A CN113409193B CN 113409193 B CN113409193 B CN 113409193B CN 202110682864 A CN202110682864 A CN 202110682864A CN 113409193 B CN113409193 B CN 113409193B
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- rgb color
- color image
- xyz
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000001228 spectrum Methods 0.000 claims abstract description 136
- 230000003595 spectral effect Effects 0.000 claims abstract description 23
- 238000004590 computer program Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000000701 chemical imaging Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 12
- 239000011159 matrix material Substances 0.000 description 9
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003707 image sharpening Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
- G06T2207/10036—Multispectral image; Hyperspectral image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a super-resolution reconstruction method and device of a hyperspectral image, which relate to the technical field of spectral imaging and comprise the steps of carrying out neighborhood matching on a low-resolution RGB color image obtained by converting a low-resolution hyperspectral image and a first high-resolution RGB color image, and determining corresponding similar spectral information of each matching point on a target high-resolution hyperspectral image; according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and similar spectrum information obtained by converting the first high-resolution RGB color image and the second high-resolution RGB color image, the real spectrum information of each matching point in the target high-resolution hyperspectral image is determined, the similar spectrum is matched in a neighborhood matching mode, the space structure information of the original hyperspectral image is reserved, the high-resolution hyperspectral image can be quickly reconstructed, and the reconstruction accuracy is high.
Description
Technical Field
The invention relates to the technical field of spectral imaging, in particular to a super-resolution reconstruction method and device of a hyperspectral image.
Background
The spectrum imaging technology can simultaneously acquire two-dimensional space image information and one-dimensional spectrum information, and is widely applied to the fields of remote sensing, biomedicine and the like. However, the current spectrometer cannot acquire a spectrum image with higher spatial resolution, and when spectrum information is acquired, the spatial resolution and the spectrum resolution are always balanced, so that the high spatial resolution cannot be achieved while the high spectrum resolution is not considered. Therefore, in order to obtain an image having a relatively high spatial resolution and a relatively high spectral resolution, the spatial super-resolution of the hyperspectral image is becoming a popular research direction.
The current method for enhancing the spatial resolution of the hyperspectral image involves a step of converting a three-dimensional spectral image into a two-dimensional matrix, and the method damages the spatial structure information of the original hyperspectral image and has high requirements and high dependence on camera application functions.
Disclosure of Invention
The invention aims to provide a super-resolution reconstruction method and device of a hyperspectral image, which are used for matching similar spectrums in a neighborhood matching mode, so that the spatial structure information of the original hyperspectral image is reserved; the novel polynomial correction model is provided for correcting similar spectrums, and can reconstruct high-resolution hyperspectral images rapidly and has higher reconstruction accuracy.
In a first aspect, an embodiment of the present invention provides a super-resolution reconstruction method for a hyperspectral image, including:
acquiring a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object, wherein the first high-resolution RGB color image is acquired under a first light source, and the second high-resolution RGB color image is acquired under a second light source;
performing neighborhood matching on the low-resolution high-spectrum image according to the low-resolution RGB color image obtained by converting the low-resolution high-spectrum image under the first light source condition, and determining corresponding similar spectrum information of each matching point on a target high-resolution high-spectrum image, wherein the target high-resolution high-spectrum image is obtained by performing super-resolution reconstruction on the low-resolution high-spectrum image;
converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space respectively to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
and determining the real spectrum information of each matching point in the target high-resolution high-spectrum image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum information.
With reference to the first aspect, the embodiment of the present invention provides a first possible implementation manner of the first aspect, where the method further includes:
and correcting the two ends of the real spectrum information of each matching point according to the trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image to obtain the corrected spectrum information of each matching point in the target high-resolution high-spectrum image.
With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the step of correcting, according to a trend of similar spectral information of each pixel point in the first high-resolution RGB color image, two ends of real spectral information of each matching point includes:
and migrating the two-end wave band trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image to two ends of the real spectrum information of each matching point.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the step of matching the low resolution RGB color image obtained by converting the low resolution hyperspectral image with the first high resolution RGB color image to determine corresponding similar spectral information of each matching point on the target high resolution hyperspectral image includes:
converting the low-resolution hyperspectral image into an RGB space under a first light source to obtain a low-resolution RGB color image;
matching each pixel point on a first high-resolution RGB color image acquired under the first light source with a corresponding feature point in the corresponding field of the low-resolution RGB color image;
determining corresponding position points of the low-resolution hyperspectral image according to the feature points successfully matched, and extracting spectral information of each corresponding position point;
and taking the spectrum information extracted from the corresponding position point of the low-resolution hyperspectral image as similar spectrum information of the target high-resolution hyperspectral.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of converting the first high resolution RGB color image and the second high resolution RGB color image into XYZ space to obtain a first high resolution XYZ image and a second high resolution XYZ image includes:
converting the RGB value of each pixel point in the first high-resolution RGB color image into the XYZ value to obtain a first high-resolution XYZ image;
and converting the RGB value of each pixel point in the second high-resolution RGB color image into the XYZ value to obtain a second high-resolution XYZ image.
With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the step of determining, according to a fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image, and the similar spectral information, real spectral information of each matching point in the target high-resolution hyperspectral image includes:
performing fifth-order polynomial fitting based on XYZ values of the first high-resolution XYZ image, XYZ values of the second high-resolution XYZ image, and the similar spectral information;
determining a functional relationship between the similar spectrum information and the real spectrum information based on a fitting result;
and reconstructing real spectrum information of the target high-resolution hyperspectral through the functional relation and the similar spectrum information.
With reference to the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the method further includes:
and reconstructing the target high-resolution hyperspectral image based on the real spectrum information or the corrected spectrum information of each matching point in the target high-resolution hyperspectral image.
In a second aspect, an embodiment of the present invention further provides a super-resolution reconstruction apparatus for a hyperspectral image, including:
the acquisition module acquires a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object, wherein the first high-resolution RGB color image is acquired under a first light source, and the second high-resolution RGB color image is acquired under a second light source;
the matching module is used for carrying out neighborhood matching on the low-resolution RGB color image obtained by converting the low-resolution hyperspectral image under the condition of a first light source and the first high-resolution RGB color image, and determining corresponding similar spectrum information of each matching point on a target high-resolution hyperspectral image, wherein the target high-resolution hyperspectral image is obtained by carrying out super-resolution reconstruction on the low-resolution hyperspectral image;
the conversion module is used for respectively converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
the determining module is used for determining the real spectrum information of each matching point in the target high-resolution high-spectrum image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum information.
In a third aspect, an embodiment provides an electronic device, including a memory, a processor, where the memory stores a computer program executable on the processor, and where the processor implements the steps of the method according to any of the foregoing embodiments when the computer program is executed.
In a fourth aspect, embodiments provide a machine-readable storage medium storing machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the steps of the method of any of the preceding embodiments.
The embodiment of the invention provides a super-resolution reconstruction method and device for a hyperspectral image, which are used for converting a low-resolution hyperspectral image to be reconstructed into a low-resolution RGB color image, matching the low-resolution hyperspectral image with a first high-resolution RGB color image under a first light source, matching each pixel point on the first high-resolution RGB color image under the first light source with a corresponding position point, obtaining similar spectrum information of the corresponding position point on a target high-resolution hyperspectral image, carrying out XYZ space conversion on the first high-resolution RGB color image and a second high-resolution RGB color image, fitting according to the converted XYZ image and the similar spectrum information, obtaining a relation function of the similar spectrum information and real spectrum information, further determining the real spectrum information, retaining the space structure information of the original hyperspectral image, and being capable of rapidly reconstructing the high-resolution hyperspectral image and having higher reconstruction precision.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a super-resolution reconstruction method of a hyperspectral image according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of comparing a reconstructed spectrum with a real spectrum according to an embodiment of the present invention;
fig. 3 is a schematic functional block diagram of a super-resolution reconstruction device for hyperspectral images according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a hardware architecture of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The current method is to extract the end member spectrum of the low resolution spectrum image by a feature extraction method to obtain a spectrum base vector; and decomposing the high-resolution image based on regularization to obtain a coefficient matrix. And finally realizing the super resolution of the low resolution spectrum image through the combination of the coefficient matrix and the spectrum base vector.
However, in practical application, in the process of converting the three-dimensional spectrum image into a two-dimensional matrix, the spatial structure information of the original image is destroyed by matrix decomposition and synthesis; furthermore, the accuracy of this matrix factorization method is highly dependent on the acquisition of the camera's corresponding function and spectral sensitivity function.
Based on the above, the super-resolution reconstruction method and device for the hyperspectral image provided by the embodiment of the invention can match similar spectrums in a neighborhood matching mode, retain the spatial structure information of the original hyperspectral image, can reconstruct the hyperspectral image rapidly and have higher reconstruction precision.
In order to facilitate understanding of the present embodiment, first, a method for reconstructing a hyperspectral image according to the present embodiment of the present invention is described in detail, and the present application proposes a method for enhancing spatial resolution of a low-resolution hyperspectral image by neighborhood matching, fitting, and trend migration using one low-resolution hyperspectral image and two corresponding color images from different light sources.
Fig. 1 is a flowchart of a super-resolution reconstruction method of a hyperspectral image according to an embodiment of the present invention.
Referring to fig. 1, the method includes the steps of:
step S102, a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object are acquired, wherein the first high-resolution RGB color image is acquired under a first light source, the second high-resolution RGB color image is acquired under a second light source, the first high-resolution RGB color image corresponds to first light source information, and the second high-resolution RGB color image corresponds to second light source information.
Step S104, carrying out neighborhood matching on the low-resolution RGB color image obtained by converting the low-resolution hyperspectral image under the condition of a first light source and the first high-resolution RGB color image, and determining corresponding similar spectrum information of each matching point on a target high-resolution hyperspectral image, wherein the target high-resolution hyperspectral image is obtained by carrying out super-resolution reconstruction on the low-resolution hyperspectral image;
step S106, converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space respectively to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
step S108, determining the real spectrum information of each matching point in the target high-resolution hyperspectral image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum information, wherein the matching point is a successfully matched position point.
In a preferred embodiment of practical application, a low-resolution high-spectrum image to be reconstructed is converted into a low-resolution RGB color image, the low-resolution RGB color image is matched with a first high-resolution RGB color image under a first light source, each pixel point on the first high-resolution RGB color image under the first light source is matched with a corresponding position point, similar spectrum information of a corresponding position point on a target high-resolution high-spectrum image is obtained, then the first high-resolution RGB color image and a second high-resolution RGB color image are subjected to XYZ space conversion, a relation function between the similar spectrum information and real spectrum information is obtained according to fitting of the converted XYZ image and the similar spectrum information, further real spectrum information is determined, spatial structure information of an original high-resolution high-spectrum image is reserved, and the high-resolution high-spectrum image can be quickly reconstructed and has high reconstruction precision.
In some embodiments, the inventors have found that CIE color matching functions are insensitive to short and long waves, and have found that final corrected spectra are obtained by re-correcting both ends of the resulting real spectrum according to the trend of similar spectra. This is done for each point on the first high resolution RGB color image, resulting in a reconstructed high resolution hyperspectral image. Illustratively, the above method further comprises:
step 1.1), correcting the two ends of the real spectrum information of each matching point according to the trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image, and obtaining the corrected spectrum information of each matching point in the target high-resolution high-spectrum image.
Wherein, when the step 1.1) is concretely implemented, the method comprises the following steps: and (3) migrating the two-end wave band trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image to two ends of the real spectrum information of each matching point.
It should be noted that, the present application shifts the trend of the bands at the two ends of the similar spectrum to the real spectrum, so as to correct the phenomenon of larger error caused by insensitivity of the color matching function to the bands at the two ends, and finally obtain the reconstructed high-resolution hyperspectral image.
In some embodiments, the step S104 may be further implemented by the following steps, which specifically include:
step 2.1), converting the low-resolution hyperspectral image into an RGB space under a first light source to obtain a low-resolution RGB color image;
step 2.2), matching each pixel point on the first high-resolution RGB color image acquired under the first light source with a corresponding feature point on a corresponding neighborhood of the low-resolution RGB color image;
step 2.3), determining corresponding position points of the low-resolution hyperspectral image according to each successfully matched characteristic point, and extracting spectral information of each corresponding position point;
and 2.4), taking the spectrum information extracted from the corresponding position point of the low-resolution hyperspectral image as similar spectrum information of the target high-resolution hyperspectral.
Here, the low-resolution RGB color image is matched with each pixel point on the first high-resolution RGB image obtained by shooting in a neighborhood matching manner to perform matching of similar RGB values, the position of the matched feature point on the low-resolution RGB image corresponds to the position of the low-resolution hyperspectral image, and the spectral information of the low-resolution RGB color image is extracted from the position of the low-resolution hyperspectral image to be used as the similar spectral information of the high-resolution hyperspectral image at the position.
It can be understood that the pixel points on the first high-resolution RGB image, the feature points on the low-resolution RGB color image, the corresponding position points of the low-resolution hyperspectral image, and the corresponding position points of the high-resolution hyperspectral image correspond sequentially in the above order.
In some embodiments, step S106 further comprises the steps of:
step 3.1), converting the RGB value of each pixel point in the first high-resolution RGB color image into XYZ value to obtain the first high-resolution XYZ image.
And 3.2), converting the RGB value of each pixel point in the second high-resolution RGB color image into XYZ values to obtain a second high-resolution XYZ image.
In some embodiments, step S108 further comprises:
step 4.1), performing fifth-order polynomial fitting based on the XYZ values of the first high-resolution XYZ image, the XYZ values of the second high-resolution XYZ image and the matched similar spectrum;
step 4.2), determining a functional relation between the similar spectrum information and the real spectrum information based on the fitting result;
and 4.3) reconstructing the real spectrum information of the target resolution hyperspectral through the functional relation and the similar spectrum information.
The pixel points on the high-resolution RGB color image are converted into color coordinates XYZ values in the XYZ space through color management, and an association function relationship exists between the real spectrum and the similar spectrum, so that the function relationship between the real spectrum and the similar spectrum is solved in a fitting mode through a computing model of XYZ, and the reconstruction of the real spectrum is carried out by utilizing the function relationship and the similar spectrum. The XYZ calculation model is a polynomial correction model, and the XYZ values of the two images and the corresponding similar spectrum information matched with the two images are input to the XYZ calculation model for fitting.
In some embodiments, the method provided by the embodiment of the present invention further includes:
and 5.3) reconstructing the target high-resolution hyperspectral image based on the real spectrum information or the corrected spectrum information of each matching point in the target high-resolution hyperspectral image.
In some embodiments, the high resolution RGB color image dimension from CAVE data set named "Balloon" is 512 x 3, which corresponds to a 400nm to 700nm low resolution hyperspectral image dimension of 64 x 31, illumination conditions: d65 light source and a light source; a 10 field of view.
In the super-resolution reconstruction process of the hyperspectral image, firstly, light source information is acquired, and a low-resolution hyperspectral image, an RGB color image under an A light source and an RGB color image under a D65 light source are acquired. The low resolution hyperspectral image is converted to RGB space under the D65 light source to obtain a low resolution RGB color image. For each pixel point P on the high resolution RGB color image at D65 D65i (i is pixel position), for example, pixel P1 is in the high resolution RGB color map with pixel position (142, 444) and RGB value of [0.385 0.397 0.659 ]] T Finding a feature point p1 (36, 111) whose position corresponds to that on the low-resolution RGB color image,RGB value [0.251 0.251 0.496 ]] T 。
As an alternative embodiment, matching the RGB value of each pixel point with the RGB value corresponding to the found corresponding feature point in the preset domain range with the similar RGB value, and determining the similar RGB value of the feature point. For example, the preset domain range is 8×8.
Illustratively, taking all RGB values within 8×8 of the feature point p1 point neighborhood to match points on the high resolution RGB image, the following formula is:
where Δr, Δg, Δb are the difference between the RGB values of each pixel point on the high resolution RGB color image and the RGB values on the low resolution neighborhood of the corresponding feature point of its pixel point, respectively. Point p where the minimum ΔE is obtained 1 ′[0.394 0.402 0.697] T As similar RGB values, a point corresponding to the feature point on the low resolution hyperspectral image is found, and the spectral information is extracted as a similar spectrum r'.
Pixel point P of two high-resolution RGB color images under different light sources D65 [0.385 0.397 0.659] T And P A [0.311 0.150 0.113] T Conversion to XYZ values T by color management D65 [41.968 41.316 69.169] T And T A [45.223 39.946 21.973] T Then, fifth order polynomial fitting is performed. The polynomial assuming that the true spectrum r and the similar spectrum r' exist at the wavelength λ is as follows:
wherein a is i Is a polynomial coefficient, i is an order. The one-dimensional vector XYZ values can be regarded as the reduced dimensions of the one-dimensional vector spectral data, and the following computational model exists between the XYZ values x, y, z and the spectrum r:
wherein the matrix
Is a CIE 10 ° field of view color matching function.
Thus, substitution of r in formula (3) by formula (2) yields the following relationship:
wherein, the liquid crystal display device comprises a liquid crystal display device,
is a combination of a i Is a coefficient matrix of (a). The weight matrix A can be obtained by solving the inversion operation:
[-289.51 2710.13-10040.0 18464.9-16861.8 6118.21] T the weights are brought into equation (2) to achieve spectral reconstruction. On the basis, the two-end trends of the reconstructed spectrum are revised again to solve the error caused by insensitivity of the color matching function to the two-end wave bands, the two-end trends of the similar spectrum r' are migrated to the reconstructed spectrum, and the derivatives of the similar spectrum curves at the 410nm end point and the 670nm end point are calculated firstly, wherein the derivatives have the following formula:
the derivative is then transferred to 410nm and 670nm of the reconstructed spectrum, completing the trend transfer, and finally obtaining the fully reconstructed spectrum.
For all pixel points P on high resolution RGB color image i And performing the above operation to finally obtain a reconstructed high-resolution hyperspectral image.
In some alternative embodiments, the reconstructed target hyperspectral image may be evaluated by three evaluation indices, peak signal-to-noise ratio (PSNR), spectral Angle Mapping (SAM), relatively dimensionless global Error (ERGAS). Wherein, table 1 is the evaluation result of PSNR, ERGAS, SAM, and the corrected spectrum of this point is compared with the real spectrum, as shown in fig. 2.
Table 1 evaluation results of super resolution hyperspectral image
| PSNR | ERGAS | SAM | |
| Balloon | 51.426 | 0.373 | 1.552 |
In some embodiments, as shown in fig. 3, an embodiment of the present invention further provides a super-resolution reconstruction apparatus for hyperspectral image, including:
the acquisition module acquires a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object, wherein the first high-resolution RGB color image is acquired under a first light source, and the second high-resolution RGB color image is acquired under a second light source;
the matching module is used for carrying out neighborhood matching on the low-resolution RGB color image obtained by converting the low-resolution hyperspectral image under the condition of a first light source and the first high-resolution RGB color image, and determining corresponding similar spectrum information of each matching point on a target high-resolution hyperspectral image, wherein the target high-resolution hyperspectral image is obtained by carrying out super-resolution reconstruction on the low-resolution hyperspectral image;
the conversion module is used for respectively converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
and the determining module is used for determining the real spectrum information of each matching point in the target high-resolution high-spectrum image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum.
In the embodiment of the present invention, the electronic device may be, but is not limited to, a personal computer (Personal Computer, PC), a notebook computer, a monitoring device, a server, and other computer devices with analysis and processing capabilities.
As an exemplary embodiment, referring to fig. 4, an electronic device 110 includes a communication interface 111, a processor 112, a memory 113, and a bus 114, the processor 112, the communication interface 111, and the memory 113 being connected by the bus 114; the memory 113 is used for storing a computer program supporting the processor 112 to execute the image sharpening method, and the processor 112 is configured to execute the program stored in the memory 113.
The machine-readable storage medium referred to herein may be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, or the like. For example, a machine-readable storage medium may be: RAM (Radom Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., hard drive), any type of storage disk (e.g., optical disk, dvd, etc.), or a similar storage medium, or a combination thereof.
The non-volatile medium may be a non-volatile memory, a flash memory, a storage drive (e.g., hard drive), any type of storage disk (e.g., optical disk, dvd, etc.), or a similar non-volatile storage medium, or a combination thereof.
It can be understood that the specific operation method of each functional module in this embodiment may refer to the detailed description of the corresponding steps in the above method embodiment, and the detailed description is not repeated here.
The computer readable storage medium provided by the embodiments of the present invention stores a computer program, where the computer program code may implement the method described in any of the foregoing embodiments when executed, and the specific implementation may refer to the method embodiment and will not be described herein.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described system and apparatus may refer to corresponding procedures in the foregoing method embodiments, which are not described herein again.
In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. The super-resolution reconstruction method of the hyperspectral image is characterized by comprising the following steps of:
acquiring a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object, wherein the first high-resolution RGB color image is acquired under a first light source, and the second high-resolution RGB color image is acquired under a second light source;
performing neighborhood matching on the low-resolution high-spectrum image according to the low-resolution RGB color image obtained by converting the low-resolution high-spectrum image under the first light source condition, and determining corresponding similar spectrum information of each matching point on a target high-resolution high-spectrum image, wherein the target high-resolution high-spectrum image is obtained by performing super-resolution reconstruction on the low-resolution high-spectrum image;
converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space respectively to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
and determining the real spectrum information of each matching point in the target high-resolution high-spectrum image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum information.
2. The method according to claim 1, wherein the method further comprises:
and correcting the two ends of the real spectrum information of each matching point according to the trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image to obtain the corrected spectrum information of each matching point in the target high-resolution high-spectrum image.
3. The method according to claim 2, wherein the step of correcting both ends of the true spectrum information of each matching point according to the trend of the similar spectrum information of each pixel point in the first high resolution RGB color image comprises:
and migrating the two-end wave band trend of the similar spectrum information of each pixel point in the first high-resolution RGB color image to two ends of the real spectrum information of each matching point.
4. The method of claim 1, wherein the step of matching the low resolution RGB color image converted from the low resolution hyperspectral image with the first high resolution RGB color image to determine corresponding similar spectral information for each matching point on the target hyperspectral image comprises:
converting the low-resolution hyperspectral image into an RGB space under a first light source to obtain a low-resolution RGB color image;
matching each pixel point on a first high-resolution RGB color image acquired under the first light source with a corresponding feature point on a corresponding neighborhood of the low-resolution RGB color image;
determining corresponding position points of the low-resolution hyperspectral image according to the feature points successfully matched, and extracting spectral information of each corresponding position point;
and taking the spectrum information extracted from the corresponding position point of the low-resolution hyperspectral image as similar spectrum information of the target high-resolution hyperspectral.
5. The method according to claim 1, wherein the step of converting the first high resolution RGB color image and the second high resolution RGB color image to XYZ space, respectively, to obtain a first high resolution XYZ image and a second high resolution XYZ image, comprises:
converting the RGB value of each pixel point in the first high-resolution RGB color image into the XYZ value to obtain a first high-resolution XYZ image;
and converting the RGB value of each pixel point in the second high-resolution RGB color image into the XYZ value to obtain a second high-resolution XYZ image.
6. The method according to claim 1, wherein the step of determining the true spectral information of each matching point in the target high resolution hyperspectral image based on the fitting result of the first high resolution XYZ image, the second high resolution XYZ image and the similar spectral information comprises:
performing fifth-order polynomial fitting based on XYZ values of the first high-resolution XYZ image, XYZ values of the second high-resolution XYZ image, and the similar spectral information;
determining a functional relationship between the similar spectrum information and the real spectrum information based on a fitting result;
and reconstructing real spectrum information of the target high-resolution hyperspectral through the functional relation and the similar spectrum information.
7. The method according to claim 1 or 2, characterized in that the method further comprises:
and reconstructing the target high-resolution hyperspectral image based on the real spectrum information or the corrected spectrum information of each matching point in the target high-resolution hyperspectral image.
8. A super-resolution reconstruction device for hyperspectral images, comprising:
the acquisition module acquires a low-resolution hyperspectral image, a first high-resolution RGB color image and a second high-resolution RGB color image aiming at the same target object, wherein the first high-resolution RGB color image is acquired under a first light source, and the second high-resolution RGB color image is acquired under a second light source;
the matching module is used for carrying out neighborhood matching on the low-resolution RGB color image obtained by converting the low-resolution hyperspectral image under the condition of a first light source and the first high-resolution RGB color image, and determining corresponding similar spectrum information of each matching point on a target high-resolution hyperspectral image, wherein the target high-resolution hyperspectral image is obtained by carrying out super-resolution reconstruction on the low-resolution hyperspectral image;
the conversion module is used for respectively converting the first high-resolution RGB color image and the second high-resolution RGB color image into an XYZ space to obtain a first high-resolution XYZ image and a second high-resolution XYZ image;
the determining module is used for determining the real spectrum information of each matching point in the target high-resolution high-spectrum image according to the fitting result of the first high-resolution XYZ image, the second high-resolution XYZ image and the similar spectrum information.
9. An electronic device comprising a memory, a processor and a program stored on the memory and capable of running on the processor, the processor implementing the method of any one of claims 1 to 7 when executing the program.
10. A computer readable storage medium, characterized in that the computer program is stored in the readable storage medium, which computer program, when executed, implements the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110682864.6A CN113409193B (en) | 2021-06-18 | 2021-06-18 | Super-resolution reconstruction method and device for hyperspectral image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110682864.6A CN113409193B (en) | 2021-06-18 | 2021-06-18 | Super-resolution reconstruction method and device for hyperspectral image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113409193A CN113409193A (en) | 2021-09-17 |
| CN113409193B true CN113409193B (en) | 2023-07-04 |
Family
ID=77681753
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110682864.6A Active CN113409193B (en) | 2021-06-18 | 2021-06-18 | Super-resolution reconstruction method and device for hyperspectral image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113409193B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113792710A (en) * | 2021-11-15 | 2021-12-14 | 滨州学院 | Spectrum reconstruction method and device and electronic equipment |
| CN117252875B (en) * | 2023-11-17 | 2024-02-09 | 山东大学 | Medical image processing method, system, medium and equipment based on hyperspectral image |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682441A (en) * | 2012-03-01 | 2012-09-19 | 清华大学 | Hyperspectral image super-resolution reconstruction method based on subpixel mapping |
| CN105957003A (en) * | 2016-04-25 | 2016-09-21 | 四川大学 | Porous medium super dimensional reconstruction method based on learning |
| CN106157249A (en) * | 2016-08-01 | 2016-11-23 | 西安电子科技大学 | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood |
| CN107301372A (en) * | 2017-05-11 | 2017-10-27 | 中国科学院西安光学精密机械研究所 | High spectrum image super-resolution method based on transfer learning |
| CN108520495A (en) * | 2018-03-15 | 2018-09-11 | 西北工业大学 | Hyperspectral image super-resolution reconstruction method based on cluster manifold priori |
| CN108537728A (en) * | 2018-03-05 | 2018-09-14 | 中国地质大学（武汉） | High spectrum image super-resolution forming method and system based on spectrum fidelity |
| CN108648256A (en) * | 2018-05-17 | 2018-10-12 | 四川大学 | A kind of gray scale core three-dimensional method for reconstructing based on super dimension |
| CN110223383A (en) * | 2019-06-17 | 2019-09-10 | 重庆大学 | A kind of plant three-dimensional reconstruction method and system based on depth map repairing |
| CN111861885A (en) * | 2020-07-15 | 2020-10-30 | 中国人民解放***箭军工程大学 | Super-pixel sparse representation method for hyperspectral super-resolution reconstruction |
| CN112907495A (en) * | 2021-02-18 | 2021-06-04 | 北京印刷学院 | Spatial resolution enhancement method and device for hyperspectral image and electronic equipment |
-
2021
- 2021-06-18 CN CN202110682864.6A patent/CN113409193B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102682441A (en) * | 2012-03-01 | 2012-09-19 | 清华大学 | Hyperspectral image super-resolution reconstruction method based on subpixel mapping |
| CN105957003A (en) * | 2016-04-25 | 2016-09-21 | 四川大学 | Porous medium super dimensional reconstruction method based on learning |
| CN106157249A (en) * | 2016-08-01 | 2016-11-23 | 西安电子科技大学 | Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood |
| CN107301372A (en) * | 2017-05-11 | 2017-10-27 | 中国科学院西安光学精密机械研究所 | High spectrum image super-resolution method based on transfer learning |
| CN108537728A (en) * | 2018-03-05 | 2018-09-14 | 中国地质大学（武汉） | High spectrum image super-resolution forming method and system based on spectrum fidelity |
| CN108520495A (en) * | 2018-03-15 | 2018-09-11 | 西北工业大学 | Hyperspectral image super-resolution reconstruction method based on cluster manifold priori |
| CN108648256A (en) * | 2018-05-17 | 2018-10-12 | 四川大学 | A kind of gray scale core three-dimensional method for reconstructing based on super dimension |
| CN110223383A (en) * | 2019-06-17 | 2019-09-10 | 重庆大学 | A kind of plant three-dimensional reconstruction method and system based on depth map repairing |
| CN111861885A (en) * | 2020-07-15 | 2020-10-30 | 中国人民解放***箭军工程大学 | Super-pixel sparse representation method for hyperspectral super-resolution reconstruction |
| CN112907495A (en) * | 2021-02-18 | 2021-06-04 | 北京印刷学院 | Spatial resolution enhancement method and device for hyperspectral image and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| Denoising of Diffusion MRI Data via Graph Framelet Matching in x-q Space;G.Chen;《IEEE Transactions on Medical Imaging》;第38卷(第12期);2838-2848 * |
| 基于学习的单幅彩色图像超分辨率重建;尧潞阳;《北京印刷学院学报》;第23卷(第4期);72-75 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113409193A (en) | 2021-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113409193B (en) | Super-resolution reconstruction method and device for hyperspectral image | |
| US8942512B2 (en) | Methods and systems for processing a first image with reference to a second image | |
| CN110501072B (en) | Reconstruction method of snapshot type spectral imaging system based on tensor low-rank constraint | |
| US8385687B1 (en) | Methods for determining a transformation between images | |
| Kotwal et al. | A novel approach to quantitative evaluation of hyperspectral image fusion techniques | |
| JP4185075B2 (en) | Green coverage map creation device, green coverage map creation method, and program. | |
| CN111174912B (en) | Snapshot type dispersion ambiguity-resolving hyperspectral imaging method | |
| US20130132044A1 (en) | Lens Modeling | |
| US20130129201A1 (en) | Method for Pan-Sharpening Panchromatic and Multispectral Images Using Wavelet Dictionaries | |
| CN111380836A (en) | Black and odorous water positioning method and device based on remote sensing image and related equipment | |
| Lanaras et al. | Advances in hyperspectral and multispectral image fusion and spectral unmixing | |
| Wu et al. | Reconstruction of spectral color information using weighted principal component analysis | |
| AU2018101634A4 (en) | System and method for processing and merging images | |
| CN109389646B (en) | Method for carrying out radiometric calibration on color camera by utilizing multispectral image | |
| Meng et al. | Improving the spatial resolution of hyperspectral image using panchromatic and multispectral images: An integrated method | |
| CN110706188B (en) | Image fusion method and device, electronic equipment and storage medium | |
| CN114972122B (en) | Hyperspectral remote sensing image bad pixel restoration method and device, electronic equipment and medium | |
| CN113627357B (en) | High-spatial-high-spectral-resolution intrinsic decomposition method and system for remote sensing image | |
| Gerçek et al. | Automated Co‐Registration of Satellite Images through Luminance Transformation | |
| CN116109535A (en) | Image fusion method, device and computer readable storage medium | |
| Scharr et al. | Spatio-spectral deconvolution for high resolution spectral imaging with an application to the estimation of sun-induced fluorescence | |
| Mishra et al. | Spatial enhancement of SWIR band from Resourcesat-2A by preserving spectral details for accurate mapping of water bodies | |
| Stamatopoulos | Orientation and calibration of long focal length cameras in digital close-range photogrammetry | |
| Jiang et al. | Fusiform multi-scale pixel self-attention network for hyperspectral images reconstruction from a single RGB image | |
| Lloyd et al. | Efficient destriping of remote sensing images using an oriented super-Gaussian filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |