CN108288256B - Multispectral mosaic image restoration method - Google Patents
Multispectral mosaic image restoration method Download PDFInfo
- Publication number
- CN108288256B CN108288256B CN201810097052.3A CN201810097052A CN108288256B CN 108288256 B CN108288256 B CN 108288256B CN 201810097052 A CN201810097052 A CN 201810097052A CN 108288256 B CN108288256 B CN 108288256B
- Authority
- CN
- China
- Prior art keywords
- image
- mosaic
- spectral
- spectrum
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000003595 spectral effect Effects 0.000 claims abstract description 68
- 238000001228 spectrum Methods 0.000 claims abstract description 35
- 239000011159 matrix material Substances 0.000 claims abstract description 6
- 238000005070 sampling Methods 0.000 claims abstract description 5
- 230000005540 biological transmission Effects 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 description 11
- 238000003384 imaging method Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 3
- 239000007888 film coating Substances 0.000 description 3
- 238000009501 film coating Methods 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000000701 chemical imaging Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention relates to a multispectral mosaic image restoration method, which can restore a multispectral mosaic video image shot by a film-coated video spectrometer into a complete multispectral image with high spatial resolution and high spectral resolution, and solves the application limitation caused by real-time transmission of the multispectral image, and the method comprises the following steps: 1) determining a mosaic template according to the number of spectral bands contained in a pixel matrix block of an original multispectral mosaic image S; 2) extracting a monospectrum image S1; 3) down-sampling the single-spectral-segment image S1 to obtain an image S2; 4) up-sampling the image S2 by 2 times to obtain an image S3: 5) performing interpolation operation on the image S3; 6) the restoration condition of the single spectrum segment image S1 is judged, and all the single spectrum segment image restoration is completed.
Description
Technical Field
The invention relates to a multispectral image processing technology, in particular to a multispectral mosaic image restoration method.
Background
Super-resolution reconstruction and spectral reconstruction research and exploration in multispectral image processing are always important contents for application development of multispectral image technology. The multispectral image is a high-resolution remote sensing image integrating image graphics and spectroscopy, and the spatial resolution and the spectral resolution of the multispectral image are higher than those of a common remote sensing image. Compared with a common digital image, the multispectral image has spatial information and spectral information of ground objects, and geometric shape information and a spectral characteristic curve of a target can be obtained simultaneously. However, due to the defects of large amount of spectral bands, high spatial resolution, large data amount, high information redundancy, low data transmission rate and the like of the multispectral image, the multispectral image has application limitation in the fields of real-time monitoring, moving target tracking and the like. Therefore, how to reduce the information redundancy degree to achieve the real-time transmission of multispectral image data and the multispectral video imaging technology under the condition that the spatial resolution and the spectral resolution of the multispectral image are not changed is a new research hotspot, and numerous scientific researchers are also involved in the research.
As far as multispectral image data are concerned, the common data characteristics are mainly as follows:
1. the method has high spatial resolution and low spectral resolution, and is suitable for handheld multispectral imaging with less ground object types and high requirement on spatial resolution, for example: multispectral camera imaging techniques.
2. The space resolution is low, and spectral resolution is high, is applicable to ground object kind more, and the big and regular machine-carried multispectral formation of image of target geometry, for example: farmland crop detection, forest monitoring and the like.
3. The space resolution is high, and spectral resolution is high, and the hyperspectral imaging technique that also is applicable to aerospace scientific research field, for example: the hyperspectral image shot by the satellite-borne hyperspectral imager is mainly applied to atmosphere monitoring and the like.
The three multi/hyperspectral images have the inevitable defects of high data information redundancy, low data transmission rate and the like. At present, the multi-spectrogram images are mainly applied to static imaging scenes and are not suitable for video imaging and moving object shooting. Therefore, the multispectral image acquisition mode in the mosaic form is applied to a multispectral/hyperspectral image by researchers, and the image is composed of a plurality of pixel matrix blocks by adding a filter film in front of an array detector, as shown in fig. 1.
Similar to a color digital image imaging mode, different pixel points in a pixel matrix block respond to different spectral bands, each pixel point in an image only responds to information of a certain spectral band, and an output multispectral image is similar to a digital mosaic image. The imaging mode can ensure that the spatial resolution and the spectral resolution of an output image are unchanged, greatly reduces the information redundancy, transmits image data in real time, and can perform multispectral image video shooting.
However, the image data shot by the film-coated multispectral video imager cannot be directly applied to analysis processing, because the imaging technology compresses multidimensional multispectral image data into a two-dimensional data format, the output image has a mosaic effect, the outline in the image is fuzzy, and the detail loss of an object is serious. The spatial resolution of the single-spectrum image is reduced along with the increase of the number of the spectrum sections, the spectrum information missing from each pixel point is increased along with the increase of the number of the spectrum sections, and the defect is more obvious particularly under the condition of large number of the spectrum sections. Therefore, the two-dimensional multispectral image data needs to be restored, the spatial resolution of a single spectral segment image of the multispectral image is obviously enhanced, and all spectral information of each pixel is reconstructed, so that a complete multispectral image is restored.
Disclosure of Invention
In order to solve the problems in the background art, the invention provides a multispectral mosaic image reconstruction and restoration method, which can restore a multispectral mosaic video image shot by a film-coated video spectrometer into a complete multispectral image with high spatial resolution and high spectral resolution, and solves the application limitation caused by real-time transmission of the multispectral image.
Introduction of the principle contents of the present invention:
a multispectral mosaic image restoration method includes the steps of extracting a single-spectrum-segment image from a multispectral mosaic image by using mosaic template information, calculating pixel values of missing pixels in the single-spectrum-segment image by using a Taylor series estimation method, and reconstructing space and spectrum information of the single-spectrum-segment image to enable the spatial resolution of the single-spectrum-segment image to be close to the resolution of a detector as far as possible. Because the calculation of the value of the missing pixel is carried out independently for a single image, the interference of adjacent spectral band information is avoided, and the spectral error of the reconstructed complete multispectral image is smaller than that of other methods.
The Taylor series estimation method is used for estimating missing pixel values approximately by utilizing first and second derivatives of known pixel values around the missing pixel points and utilizing a Taylor series formula, and the edge information of the image can be greatly reserved by applying the calculation mode. By enlarging the calculation range of the known pixel points in the calculation process, the loss of image detail information caused by serious image sparsity can be effectively reduced.
The reconstruction of the complete multispectral image is based on the space and spectrum information of the single-spectrum-segment image, the extracted original single-spectrum-segment image is zoomed, the estimation operation of the missing pixel value of the zoomed image is completed, and the zooming and estimation operation processes are repeated until the space resolution of the reconstructed single-spectrum-segment image reaches the resolution of the original two-dimensional image. And restoring all the reconstructed single spectral band images into complete multispectral images, namely restoring two-dimensional image data into multi-dimensional data.
The method carries out multi-dimensional reconstruction on two-dimensional multispectral image data through a scaling process and a Taylor series estimation method so as to restore a complete multispectral image, so that the multispectral image data can still keep high spatial resolution and spectral resolution under a real-time transmission condition, complete multispectral image video acquisition and transmission are realized, and the spectral technology is applied to tracking and detecting a moving target.
The specific technical scheme of the invention is as follows:
a multispectral mosaic image restoration method comprises the following steps:
1) setting the number of the spectral segments as N, the size of the mosaic template is M × M, wherein M is a positive integer, M is M x M N, and N is not less than 4;
2) extracting a monospectrum image S1;
3) downsampling the single-spectrum-segment image S1, namely deleting all null pixels in the image S1 to obtain an image S2;
4) up-sampling the image S2 by 2 times to obtain an image S3:
5) performing interpolation operation on the image S3;
5.1) first interpolation;
performing interpolation operation on the intersection point of each column of empty pixels of each newly inserted line of the image S3;
5.2) second interpolation;
and then, carrying out interpolation operation on the residual null pixels of the image S3 to obtain an image S4: the residual null pixels are positioned at the intersection point of the connecting line of two adjacent known pixel points in the same column and the connecting line of two adjacent known pixel points in the same row;
6) judging the restoration condition of the single spectrum segment image S1, and completing the restoration of all the single spectrum segment images, which comprises the following steps:
setting the times of executing the steps 4) to 5) in the restoration process of each single-spectrum image as P, wherein P is more than or equal to 0;
when N is an even number, and when
The extracted single spectral band image S1 is considered to be restored, and the step 2-5) is repeated until all the single spectral band images of the original multispectral mosaic image are restored;
when N is an even number, and when
Continuing to repeat the steps 4) to 5) until
Then, the step 2-5) is carried out on the residual single spectral band images until all the single spectral band images of the original multispectral mosaic image are completely restored;
when N is an odd number, and when
And is
Repeating the steps 4) to 5) until the condition is satisfied
And (3) inserting a row of empty pixels into the obtained image S4 every 2 × P rows and 2 × P columns in the image S4 to enable the size of the image to be equal to that of the single-spectrum-segment image S1, then carrying out estimation operation on the unknown pixels in the processed image by adopting the mean value of the adjacent pixels, considering that the extracted single-spectrum-segment image S1 is restored, and repeating the step 2-5) until all the single-spectrum-segment images of the original multispectral mosaic image are restored.
The step 2) is specifically implemented as follows:
extracting a single spectral band image S1 from the original multispectral mosaic image S according to the arrangement sequence of the spectral bands in the mosaic template;
the method comprises the following steps: traversing an original multispectral mosaic image S by adopting a sliding window, wherein the size of the sliding window is equal to that of a mosaic template, only one pixel point in the sliding window is set to be 1, the rest pixel points are set to be 0, and the position where the pixel point is set to be 1 is the same as the position where the same spectral band information pixel is located in the mosaic template;
each cell in the single-spectrum image S1 represents an image element, each image element responds to a piece of spectral information, and the spectral information of all the image elements in each pixel matrix block responds differently.
The step 3) is specifically carried out as follows:
and deleting the pixel with the pixel point set as 0 in the single-spectrum image S1, and reducing the spatial resolution of the single-spectrum image S1 to 1/N of the original multispectral mosaic image S, thereby obtaining an image S2.
The step 4) is specifically carried out as follows:
a row and a column of null pixels are inserted in every other row and column in the image S2, resulting in an image S3 in which the spatial resolution of the image S3 is 4 times that of the image S2.
The specific process of interpolation operation in the steps 5.1) and 5.2) is as follows:
a1, setting P point as unknown pixel point, four known pixel neighborhood N around P point1，N2，N3，N4Each neighborhood has 4 known pixels, and the first-order gradient operator and the second-order gradient operator are used for calculating the first-order derivative and the second-order derivative of the known pixels in each neighborhood;
a2, calculating the average value of the first and second derivatives of four known pixels in each neighborhood;
the mean of the first derivatives of the four neighborhoods in the window is
Mean of the second derivative of
A3, calculating Taylor series approximation values of the P point in the window in four neighborhood directions:
a4, calculating the weight coefficient omega of each neighborhoodi：
(i ═ 1,2,3,4), (k ═ 1,2,3, 4); ρ is a constant, making the weight coefficients ω of the four neighborhoodsiThe sum is 1;
a5, calculating an estimated value I (P) of the P point pixel:
the invention has the beneficial effects that:
1. the invention provides a method for restoring a multispectral mosaic image and reconstructing a complete multispectral image based on a mosaic multispectral image shot by a film-coated video spectrometer by utilizing a film-coated mosaic template and a Taylor series approximate estimation method. The image restoration method can perform spatial super-resolution reconstruction and pixel missing spectrum reconstruction of a single spectral band image in a data preprocessing stage, and realize complete reconstruction of a multispectral image video.
2. The method solves the problem that the spatial resolution of the single-spectrum image is reduced due to the increase of the number of the spectrum segments of the multispectral mosaic image; the problems of insufficient spectral information and incomplete image data in a multispectral mosaic image are solved; the method solves the problems of large spectrum classification error caused by low spatial resolution of a single spectrum segment image in a multispectral mosaic image and serious spectrum information loss; the spatial resolution of the reconstructed complete multispectral image is made as close to the detector resolution as possible.
3. As the number of the spectral bands coated by the spectrometer is increased, the spatial resolution of the extracted single spectral band image is reduced, and the spectral band information with single pixel missing is increased, the single spectral band image signals are seriously sparse, the edge of the reconstructed image is blurred, and the spectral classification error is increased. Therefore, the method expands the range of the known pixel domain around the pixel point to be estimated in the single-spectrum image, calculates the approximate value of the Taylor series by utilizing the first-order and second-order derivatives of all known pixel points in the neighborhood of the pixel point to be estimated, and calculates the pixel value of the pixel point to be interpolated in the spectrum through a weighting formula. The edge information in the original image can be greatly preserved by expanding the neighborhood range of the known pixels and calculating the first-order and second-order derivatives, so that the spatial resolution of the reconstructed image is enhanced, and the spectral error rate is reduced.
Drawings
FIG. 1 is an original multi-spectral mosaic image taken with a film-coated video spectrometer in an embodiment of the present invention;
FIG. 2 is a schematic view of a coated mosaic template used in the embodiment of the present invention
FIG. 3 is a single spectral band image extracted in an embodiment of the present invention;
FIG. 4 illustrates a step of reconstructing a single spectral band image according to an embodiment of the present invention;
FIG. 5(a) is a schematic diagram illustrating a distribution of interpolation points in a first interpolation process according to an embodiment of the present invention;
FIG. 5(b) is a schematic diagram illustrating the distribution of all interpolation points during the first interpolation process according to the embodiment of the present invention;
FIG. 5(c) is a diagram illustrating a distribution of interpolation points during the second interpolation process according to an embodiment of the present invention;
FIG. 5(d) is a schematic diagram illustrating the distribution of all interpolation points during the second interpolation process according to the embodiment of the present invention;
FIG. 6 is a Taylor series estimation method for pixels in non-same row and non-same column as known pixels in the embodiment of the present invention;
FIG. 7 is a Taylor series estimation method for pixels in the same row and different rows or in the same row and different columns as known pixels in the embodiment of the present invention;
fig. 8(a) is a photographed original two-dimensional multispectral mosaic image;
FIG. 8(b) is a partial enlarged view of FIG. 8 (a);
FIG. 8(c) is a partial enlarged view of FIG. 8 (b);
FIG. 9(a) is a single spectral band image after being restored by the method of the present invention;
FIG. 9(b) is a partial enlarged view of FIG. 9 (a);
fig. 9(c) is a partial enlarged view of fig. 9 (b).
Detailed Description
As shown in fig. 1, a film-coating video spectrometer is used to collect an original multispectral mosaic image S, and imaging lenses of 9, 16 and 25 bands can be selected according to the type of film coating of the lens, in this embodiment, a 25 band film coating lens is selected. Fig. 2 shows a mosaic template (also referred to as a pixel matrix) with 25 spectral bands, and an image taken by a video spectrometer can be regarded as being composed of a plurality of mosaic pixel matrices, each pixel in the pixel matrices only responds to information of a specific spectral band, and each pixel responds to information of different spectral bands.
As shown in fig. 2, a single spectral band image S1 is extracted from the original multispectral mosaic image S according to the sequence of spectral bands in the mosaic template, the original multispectral mosaic image S is traversed by a sliding window, the size of the sliding window is equal to that of the mosaic template, only one point in the sliding window is set to be 1, the rest points are set to be 0, and the point position of the set point to be 1 is the same as the position of the same spectral band information pixel in the mosaic template. For example, if the 3 rd spectral pixel of the 25-spectral mosaic template in fig. 2 is located at position (1,3), the size of the sliding window used for extracting the 3 rd spectral image is 5 × 5, and only the position (1,3) in the window is located at 1, and the rest of the points are located at 0.
It can be seen from fig. 3 that the spatial resolution of the extracted single spectral band image S1 is severely reduced, only 1/25 of the spatial resolution of the original image. In order to improve the spatial resolution of the single-spectral-band images, the spatial resolution of all the single-spectral-band images is made as close as possible to the resolution of the detector, and the spectral information of the reconstructed images is made as close as possible to the spectral characteristics of the real object, so in this example the reconstruction is performed using the steps as shown in fig. 4.
The first step is as follows: extracting a single-spectrum image S1, and then downsampling the single-spectrum image to obtain an image S2; deleting 1/25 image elements without values in the image, wherein the spatial resolution of the image can be reduced to the original image;
then, performing 2 times of upsampling on the downsampled image S2, and inserting null pixels in every other row and column in the image to obtain an image S3; as a second step in the process of fig. 3, the distribution of known pixel points in the image is made to resemble a checkerboard.
The second step is that: first interpolation
The image S3 after down sampling is interpolated by taylor series, the position of the pixel of the first interpolation is shown in fig. 5(a) and (b) (i.e. the position of the intersection of each column of empty pixels in each row newly inserted into the image S3), the white dot represents the known pixel, the gray dot represents the pixel to be inserted in this step, and the pixel to be inserted and the known pixel are in the same column and row. The pixel calculation process at the gray circle point is shown in fig. 6, a sliding window with the size of 7 × 7 is adopted to traverse the whole image, and taylor series approximate estimation calculation is performed on the central pixel in the window, such as the point P in fig. 6. The calculation process is as follows:
a1, selecting four known pixel neighborhood N around P point1，N2，N3，N4And calculating the first and second derivatives of the known pixels in each neighborhood by using a first gradient operator and a second gradient operator, such as: n is a radical of2Within the neighborhood, four known pixels q21，q22，q23，q24Has a first derivative of I2′(q21)，I2′(q22)，I2′(q23)，I2′(q24) Second derivative is I2″(q21)，I2″(q22)，I2″(q23)，I2″(q24)；
A2, calculating the average value of the first and second derivatives of four known pixels in each neighborhood, such as: n is a radical of2Within neighborhood, first derivative mean
Mean of second derivative
The mean of the first derivatives of the four neighborhoods in the window is
Mean of the second derivative of
A3, calculating Taylor series approximation values of the P point in the window in four neighborhood directions: ,
a4, calculating the weight coefficient omega of each neighborhoodi：
(i ═ 1,2,3,4), (k ═ 1,2,3, 4); ρ is a constant, making the weight coefficients ω of the four neighborhoodsiThe sum is 1;
a5, calculating an estimated value I (P) of the P point pixel:
the third step: second interpolation
The remaining unknown pixels in the image after the second step (as shown in fig. 5 (b)) are evaluated, and the remaining unknown pixels are in different columns or different rows in the same column with the known pixels (the black dots in fig. 5(c) represent the pixels evaluated in this step, i.e., the intersection positions of the connecting lines of two adjacent known pixel points in the same column and the connecting lines of two adjacent known pixel points in the same row). The neighborhood partition used in this step is shown in FIG. 7, using four neighborhoods N in FIG. 71，N2，N3，N4The known pixel information is repeated to calculate the image in the second stepThe remaining unknown pel in (b) is calculated at the value of the spectrum, and the result is shown in fig. 5(d), and the black dot represents the pel inserted in the third step.
The fourth step: performing 2 times of upsampling on the image after the third step is completed, and then repeating the calculation processes of the second step and the third step on the image until the size of the image is equal to or close to that of the original two-dimensional image;
if the number of the spectral bands in the original two-dimensional mosaic image is even, the resolution of the restored image can be as close to that of the detector as possible only by performing restoration calculation according to the four steps. If the number of the spectral bands in the original two-dimensional mosaic image is odd, the following process is required after the fourth step is completed:
A. and inserting a row of blank pixels into the image processed in the fourth step every other even rows and even columns, for example, if the 25-spectrum mosaic multispectral image adopted in the invention is restored, inserting a row of blank pixels into the image after the fourth step every other four rows and four columns, so that the size of the image is equal to that of the original two-dimensional mosaic image.
B. And carrying out estimation operation on the unknown pixel in the processed image by adopting the mean value of the adjacent pixels.
The above steps are directed to a method for restoring a single spectral band image, and all the above calculation processes need to be performed on each extracted single spectral band image in order to restore a complete multispectral image. A 25-spectral multispectral mosaic image was taken using a film-coated video spectrometer, as shown in fig. 8(a), 8(b), and 8 (c). Fig. 9(a), 9(b), and 9(c) show images processed by the restoration method of the present invention. As can be seen from the effect comparison in the figure, the spatial resolution of the processed single-spectrum image is remarkably improved, and the outline and detail information of the image are enhanced.
For the restoration method, a mode of combining downsampling and upsampling can ensure that the known pixel point value and position are not influenced by interpolation, so that the calculation result is more accurate, and the possibility of distortion of the graph is reduced. And by utilizing a neighborhood expanding Taylor series interpolation method, the first-order derivative and the second-order derivative are calculated, so that the image edge information can be greatly reserved, and the spectrum classification error is reduced.
Claims (5)
1. A multispectral mosaic image restoration method is characterized by comprising the following steps:
1) setting the number of the spectral segments as N, the size of the mosaic template is M × M, wherein M is a positive integer, M is M x M N, and N is not less than 4;
2) extracting a monospectrum image S1;
3) downsampling the single-spectrum-segment image S1, namely deleting all null pixels in the image S1 to obtain an image S2;
4) up-sampling the image S2 by 2 times to obtain an image S3:
5) performing interpolation operation on the image S3;
5.1) first interpolation;
performing interpolation operation on the intersection point of each column of empty pixels of each newly inserted line of the image S3;
5.2) second interpolation;
and then, carrying out interpolation operation on the residual null pixels of the image S3 to obtain an image S4: the residual null pixels are positioned at the intersection point of the connecting line of two adjacent known pixel points in the same column and the connecting line of two adjacent known pixel points in the same row;
6) judging the restoration condition of the single spectrum segment image S1, and completing the restoration of all the single spectrum segment images, which comprises the following steps:
setting the times of executing the steps 4) to 5) in the restoration process of each single-spectrum image as P, wherein P is more than or equal to 0;
when N is an even number, and when
The extracted single spectral band image S1 is considered to be restored, and the step 2-5) is repeated until all the single spectral band images of the original multispectral mosaic image are restored;
when N is an even number, and when
Continuing to repeat the steps 4) to 5) until
Then, the step 2-5) is carried out on the residual single spectral band images until all the single spectral band images of the original multispectral mosaic image are completely restored;
when N is an odd number, and when
And is
Repeating the steps 4) to 5) until the condition is satisfied
And (3) inserting a row of empty pixels into the obtained image S4 every 2 × P rows and 2 × P columns in the image S4 to enable the size of the image to be equal to that of the single-spectrum-segment image S1, then carrying out estimation operation on the unknown pixels in the processed image by adopting the mean value of the adjacent pixels, considering that the extracted single-spectrum-segment image S1 is restored, and repeating the step 2-5) until all the single-spectrum-segment images of the original multispectral mosaic image are restored.
2. The multi-spectral mosaic image restoration method according to claim 1, wherein: the step 2) comprises the following specific steps:
extracting a single spectral band image S1 from the original multispectral mosaic image S according to the arrangement sequence of the spectral bands in the mosaic template;
the method comprises the following steps: traversing an original multispectral mosaic image S by adopting a sliding window, wherein the size of the sliding window is equal to that of a mosaic template, only one pixel point in the sliding window is set to be 1, the rest pixel points are set to be 0, and the position where the pixel point is set to be 1 is the same as the position where the same spectral band information pixel is located in the mosaic template;
each cell in the single-spectrum image S1 represents an image element, each image element responds to a piece of spectral information, and the spectral information of all the image elements in each pixel matrix block responds differently.
3. The multi-spectral mosaic image restoration method according to claim 1, wherein: the step 3) comprises the following specific steps:
and deleting the pixel with the pixel point set as 0 in the single-spectrum image S1, and reducing the spatial resolution of the single-spectrum image S1 to 1/N of the original multispectral mosaic image S, thereby obtaining an image S2.
4. The multi-spectral mosaic image restoration method according to claim 1, wherein: the step 4) comprises the following specific steps:
a row and a column of null pixels are inserted in every other row and column in the image S2, resulting in an image S3 in which the spatial resolution of the image S3 is 4 times that of the image S2.
5. The multi-spectral mosaic image restoration method according to claim 1, wherein: the specific process of interpolation operation in the steps 5.1) and 5.2) is as follows:
a1, setting P point as unknown pixel point, four known pixel neighborhood N around P point1，N2，N3，N4Each neighborhood has 4 known pixels, and the first-order gradient operator and the second-order gradient operator are used for calculating the first-order derivative and the second-order derivative of the known pixels in each neighborhood;
a2, calculating the average value of the first and second derivatives of four known pixels in each neighborhood;
the mean of the first derivatives of the four neighborhoods in the window is
Mean of the second derivative of
A3, calculating P points in the window in four neighborhood directionsTaylor series approximation:
a4, calculating the weight coefficient omega of each neighborhoodi：
(i ═ 1,2,3,4), (k ═ 1,2,3, 4); ρ is a constant, making the weight coefficients ω of the four neighborhoodsiThe sum is 1;
a5, calculating an estimated value I (P) of the P point pixel:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097052.3A CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810097052.3A CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108288256A CN108288256A (en) | 2018-07-17 |
| CN108288256B true CN108288256B (en) | 2020-07-31 |
Family
ID=62836170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810097052.3A Active CN108288256B (en) | 2018-01-31 | 2018-01-31 | Multispectral mosaic image restoration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108288256B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146787B (en) * | 2018-08-15 | 2022-09-06 | 北京理工大学 | Real-time reconstruction method of dual-camera spectral imaging system based on interpolation |
| CN109029380B (en) * | 2018-09-14 | 2019-12-03 | 中国科学院西安光学精密机械研究所 | Stereo visual system and its calibration distance measuring method based on film coated type multispectral camera |
| WO2021003594A1 (en) * | 2019-07-05 | 2021-01-14 | Baidu.Com Times Technology (Beijing) Co., Ltd. | Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation |
| CN110458766B (en) * | 2019-07-11 | 2023-08-25 | 天津大学 | Snapshot hyperspectral image demosaicing method |
| CN110579279B (en) * | 2019-09-19 | 2021-08-06 | 西安理工大学 | Design method of nine-spectral-band multispectral imaging system of single sensor |
| CN112288008B (en) * | 2020-10-29 | 2022-03-01 | 四川九洲电器集团有限责任公司 | Mosaic multispectral image disguised target detection method based on deep learning |
| CN112489042A (en) * | 2020-12-21 | 2021-03-12 | 大连工业大学 | Metal product printing defect and surface damage detection method based on super-resolution reconstruction |
| CN114397255B (en) * | 2021-11-12 | 2023-09-01 | 中国科学院西安光学精密机械研究所 | Wide-spectrum high-resolution video spectrum imaging system and method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221128A (en) * | 2007-04-18 | 2008-07-16 | 中国科学院自动化研究所 | Multi-optical spectrum reconstruction method based on self-adapting finite element |
| CN103793883A (en) * | 2013-12-11 | 2014-05-14 | 北京工业大学 | Principal component analysis-based imaging spectral image super resolution restoration method |
| CN105469360A (en) * | 2015-12-25 | 2016-04-06 | 西北工业大学 | Non local joint sparse representation based hyperspectral image super-resolution reconstruction method |
| CN106780423A (en) * | 2017-01-12 | 2017-05-31 | 清华大学 | It is a kind of based on a small number of wave bands partial image high and the low point of high-quality spectrum reconstruction method of high spectrum image |
-
2018
- 2018-01-31 CN CN201810097052.3A patent/CN108288256B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101221128A (en) * | 2007-04-18 | 2008-07-16 | 中国科学院自动化研究所 | Multi-optical spectrum reconstruction method based on self-adapting finite element |
| CN103793883A (en) * | 2013-12-11 | 2014-05-14 | 北京工业大学 | Principal component analysis-based imaging spectral image super resolution restoration method |
| CN105469360A (en) * | 2015-12-25 | 2016-04-06 | 西北工业大学 | Non local joint sparse representation based hyperspectral image super-resolution reconstruction method |
| CN106780423A (en) * | 2017-01-12 | 2017-05-31 | 清华大学 | It is a kind of based on a small number of wave bands partial image high and the low point of high-quality spectrum reconstruction method of high spectrum image |
Non-Patent Citations (1)
| Title |
|---|
| "Taylor series-based generic demosaicking algorithm for multispectral image";Jiatong Han 等;《Proc. SPIE 10462, AOPC 2017: Optical Sensing and Imaging Technology and Applications》;20171024;第1-9页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108288256A (en) | 2018-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108288256B (en) | Multispectral mosaic image restoration method | |
| Shen et al. | Super-resolution reconstruction algorithm to MODIS remote sensing images | |
| CN112805744B (en) | System and method for demosaicing multispectral images | |
| CN114119444B (en) | Multi-source remote sensing image fusion method based on deep neural network | |
| CN109146787B (en) | Real-time reconstruction method of dual-camera spectral imaging system based on interpolation | |
| Valsesia et al. | Permutation invariance and uncertainty in multitemporal image super-resolution | |
| KR101975472B1 (en) | System and method for generating continuous digital zooming image using super-resolution | |
| CN110544212B (en) | Convolutional neural network hyperspectral image sharpening method based on hierarchical feature fusion | |
| CN110579279B (en) | Design method of nine-spectral-band multispectral imaging system of single sensor | |
| CN113673590A (en) | Rain removing method, system and medium based on multi-scale hourglass dense connection network | |
| CN108961163A (en) | A kind of high-resolution satellite image super-resolution reconstruction method | |
| Liu et al. | An efficient residual learning neural network for hyperspectral image superresolution | |
| CN111932452B (en) | Infrared image convolution neural network super-resolution method based on visible image enhancement | |
| CN111696043A (en) | Hyperspectral image super-resolution reconstruction algorithm of three-dimensional FSRCNN | |
| CN114841856A (en) | Image super-pixel reconstruction method of dense connection network based on depth residual channel space attention | |
| CN115578255A (en) | Super-resolution reconstruction method based on inter-frame sub-pixel block matching | |
| CN106780423B (en) | Spectral reconstruction method based on minority-band high-resolution image | |
| CN116740340A (en) | Calculation spectrum imaging error correction method based on deep learning | |
| Aburaed et al. | 3d expansion of srcnn for spatial enhancement of hyperspectral remote sensing images | |
| CN109859153B (en) | Multispectral image fusion method based on adaptive spectrum-spatial gradient sparse regularization | |
| CN104574338A (en) | Remote sensing image super-resolution reconstruction method based on multi-angle linear array CCD sensors | |
| Ibrahim et al. | 3DRRDB: Super resolution of multiple remote sensing images using 3D residual in residual dense blocks | |
| CN114463196A (en) | Image correction method based on deep learning | |
| Aburaed et al. | Sisr of hyperspectral remote sensing imagery using 3d encoder-decoder runet architecture | |
| CN113674154B (en) | Single image super-resolution reconstruction method and system based on generation countermeasure network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |