CN113962943B - Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network - Google Patents

Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network Download PDF

Info

Publication number
CN113962943B
CN113962943B CN202111172046.8A CN202111172046A CN113962943B CN 113962943 B CN113962943 B CN 113962943B CN 202111172046 A CN202111172046 A CN 202111172046A CN 113962943 B CN113962943 B CN 113962943B
Authority
CN
China
Prior art keywords
pixels
hyperspectral
spectrum
pixel
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111172046.8A
Other languages
Chinese (zh)
Other versions
CN113962943A (en
Inventor
詹天明
徐超
宋博
吴泽彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NANJING AUDIT UNIVERSITY
Original Assignee
NANJING AUDIT UNIVERSITY
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NANJING AUDIT UNIVERSITY filed Critical NANJING AUDIT UNIVERSITY
Priority to CN202111172046.8A priority Critical patent/CN113962943B/en
Publication of CN113962943A publication Critical patent/CN113962943A/en
Application granted granted Critical
Publication of CN113962943B publication Critical patent/CN113962943B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G06T2207/10036—Multispectral image; Hyperspectral image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

The invention discloses a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network, which comprises the steps of obtaining hyperspectral images of two time points in the same area and forming a sample image; selecting 5% of all pixels in the sample image as training pixels, and marking the pixels with change labels according to the change condition in the training pixels to obtain invariant pixels; the invention adopts a space spectrum combination strategy of completely separating and fusing a spectrum and a space, respectively pursuing better feature extraction effect on the spectrum and the space, then performing bidirectional reconstruction on the spectrum of an invariant pixel at two time points by adopting a coding network, taking a reconstruction error as a new spectrum feature source, further inhibiting the influence of noise on the spectrum, then screening partial wave bands based on a wave band selection algorithm to participate in space feature extraction, and performing feature enhancement by using an initial feature on the basis of a residual error framework to improve the feature extraction effect.

Description

Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network
Technical Field
The invention relates to the technical field of image processing, in particular to a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network.
Background
With the development of remote sensing technology, change detection based on remote sensing image data has important application in the fields of city development, terrain analysis and resource analysis; the hyperspectral imaging technology is an important product of the development of the remote sensing technology and is also an important data source of a remote sensing image; the abundant spectral information of the hyperspectral image provides spectral features finer than those of the traditional multispectral image, but simultaneously, high dimensionality and a large amount of redundancy are brought, and the challenge is brought to the feature extraction task.
At present, a spatial-spectral combination strategy of integrating spectral information and spatial information into feature extraction is adopted in multiple researches to detect changes of hyperspectral images; however, an important fact is that it is difficult to find a feature space such that both spectral and spatial features can be optimally expressed; in addition, under the influence of factors such as noise, the spectrums of unchanged pixels at two time points have certain difference; in addition, due to the extremely high resolution of hyperspectrum, the information of adjacent bands is usually highly similar, and it is noted that the spatial information of a part of bands is largely destroyed, which means that the spatial information of all bands is not required; therefore, a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network needs to be designed.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, better and effectively solve the existing problems, and provides a hyperspectral change detection method based on a bidirectional reconstruction coding network and a reinforced residual error network.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network comprises the following steps,
the method comprises the following steps that (A), hyperspectral images of two time points in the same area are obtained, and a sample image is formed;
selecting 5% of all pixels in the sample image as training pixels, and marking the pixels with change labels according to the change condition in the training pixels to obtain invariant pixels;
step (C), a bidirectional coding network is constructed by using invariant pixels in the training pixels to reconstruct the spectrum, a reconstruction error and a variable spectrum vector are obtained, and then the reconstruction error and the variable spectrum vector are fused into comprehensive spectrum input;
step (D), comprehensively inputting the spectrum into a one-dimensional convolution neural network to extract the spectral characteristics;
step (E), filtering a wave band with spatial information in a sample image by using an Optimal Clustering frame algorithm, forming a hyperspectral variable image, and then taking out the spectrum of a training pixel and a neighborhood pixel from the hyperspectral variable image after the wave band filtering to form a change tensor;
step (F), inputting the change tensor into the reinforced two-dimensional residual convolution neural network to extract spatial features;
and (G) fusing the spectral features and the spatial features, inputting the fused spectral features and spatial features into a full-connection network to obtain a classification result of the pixels, and generating a change detection result graph.
The hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network comprises the step (A) of acquiring hyperspectral images of two time points in the same area and forming a sample image, wherein the hyperspectral images of the same area are acquired at two different time points
Figure BDA0003293610800000031
And &>
Figure BDA0003293610800000032
Figure BDA0003293610800000033
Wherein->
Figure BDA0003293610800000034
Representing the real number domain, h and w are height and width, and c is the number of bands.
In the hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network, in the step (B), 5% of all pixels in a sample image are selected as training pixels, and change labels are marked on the pixels according to the change condition in the training pixels to obtain invariant pixels, the method comprises the following specific steps,
step (B1), selecting 5% of all pixels in the sample image as training pixels, and setting a set U = {1, 2., hw } to represent subscripts of each pixel, wherein a subset pi = { Tr, va, te } of a power set of U is a random division of U and respectively represents a training set, a verification set and a test set, wherein | Tr | =0.05hw, | Va | =0.025h;
step (B2), according to the change condition in the training pixel, a change label is marked on the pixel, and the real change label of the ith pixel is marked as y true (i) Where i ∈ Tr, y true (i) =1 denotes that the i-th pixel is a change pixel, y true (i) =0 indicates that the ith pixel is an invariant pixel.
The hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network comprises the following specific steps of (C) reconstructing a spectrum by utilizing a bidirectional coding network constructed by invariant pixels in training pixels, obtaining a reconstruction error and a change spectrum vector, and then fusing the reconstruction error and the change spectrum vector into a comprehensive spectrum input,
step (C1), a bidirectional coding network is constructed by utilizing the invariant pixels in the training pixels to reconstruct the spectrum, and the spectrum corresponding to the ith pixel in the two hyperspectral images is set as
Figure BDA0003293610800000035
And &>
Figure BDA0003293610800000036
Using invariant pixel construction X in Tr (1) And X (2) Between two-way reconstruction mapping f 1 、g 1 、f 2 And g 2 As shown in the formula (1),
Figure BDA0003293610800000037
wherein the content of the first and second substances,
Figure BDA0003293610800000041
represents->
Figure BDA0003293610800000042
In the mapping f k Coding of k ∈ {1,2}, j ∈ { Tr and y ∈ true (j)＝0，/>
Figure BDA0003293610800000043
S =120;
step (C2), obtaining a reconstruction error, and solving the following optimization problem by using a coding network to obtain f 1 ，g 1 ，f 2 ，g 2 As shown in the formula (2) and the formula (3),
Figure BDA0003293610800000044
Figure BDA0003293610800000045
step (C3) of obtaining a variation spectrum vector and calculating a spectrum variation vector of the ith pixel
Figure BDA0003293610800000046
As shown in the formula (4), the,
Figure BDA0003293610800000047
/>
and (C4) fusing the reconstruction error and the changed spectral vector into a spectral comprehensive input, and calculating the spectral comprehensive input spe _ input i ，
Figure BDA0003293610800000048
The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network comprises the step (D) of comprehensively inputting the spectrum into a one-dimensional convolutional neural network to extract the spectral characteristics, wherein the spe _ input is specifically used i Inputting the data into 3 layers of one-dimensional convolution kernel with the size of 3 to obtain a spectrum feature vector
Figure BDA0003293610800000049
And spe is the dimension of the spectral feature vector, and spe =120.
In the hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network, step (E), a wave band with spatial information is screened in a sample image by applying an Optimal Clustering frame algorithm, a hyperspectral change image is formed, then the spectrum of a training pixel and a neighborhood pixel is taken out from the hyperspectral change image after the wave band screening to form a change tensor, the specific steps are as follows,
step (E1) of calculating a hyperspectral variation image
Figure BDA00032936108000000410
As shown in the formula (6),
X (D) ＝|X (1) -X (2) | (6)；
step (E2), using the Optimal Clusting Framework algorithm to X (D) C space wave bands are sorted, d wave bands are selected according to sorting results to obtain a hyperspectral change image with the selected wave bands
Figure BDA0003293610800000051
Figure BDA0003293610800000052
Wherein d =24;
step (E3), for the ith pixel, from
Figure BDA0003293610800000053
Extracting the combination of the change spectrum vector of the pixel and the change spectrum vectors of the pixels in the neighborhood into change tensor>
Figure BDA0003293610800000054
Where b is the spatial dimension of the variation tensor, b =3.
The hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network comprises the step (F) of inputting a change tensor into a reinforced two-dimensional residual error convolutional neural network to extract spatial features, wherein the spatial features are specifically extracted by inputting the change tensor into the reinforced two-dimensional residual error convolutional neural network
Figure BDA0003293610800000055
Inputting the signals into a two-dimensional residual convolutional neural network consisting of 8 enhanced residual convolutional blocks to obtain a spatial feature vector->
Figure BDA0003293610800000056
Where spa is the dimension of the spatial feature vector, spa =12 × d =288, and let the output of the ith layer of the network be u l Then u is l+1 As shown in the formula (7),
Figure BDA0003293610800000057
wherein, therein
Figure BDA0003293610800000058
Represents the convolution operation of the l-th layer.
In the hyperspectral change detection method based on the bidirectional reconstruction coding network and the reinforced residual error network, step (G), the spectral features and the spatial features are fused and then input into the full-connection network to obtain the classification result of the pixels and generate a change detection result graph, the specific steps are as follows,
step (G1) of calculating a spectral space vector
Figure BDA0003293610800000059
As shown in the formula (8),
Figure BDA00032936108000000510
step (G2) of converting ssv i Inputting the data into a full-connection network for class II classification;
a step (G3) of generating a change detection result map from the classification result of each pixel
Figure BDA00032936108000000511
Wherein the changed pixels are white and the unchanged pixels are black. />
The invention has the beneficial effects that: the invention relates to a hyperspectral change detection method based on a bidirectional reconstruction coding network and a reinforced residual error network, which comprises the steps of firstly adopting a spatial spectrum combination strategy of completely separating and then fusing a spectrum and a space, respectively pursuing better characteristic extraction effects on the spectrum and the space, then adopting the coding network to perform bidirectional reconstruction on the spectrum of an invariant pixel at two time points, taking a reconstruction error as a new spectrum characteristic source, further inhibiting the influence of noise on the spectrum, then screening partial wave bands based on a wave band selection algorithm to participate in spatial characteristic extraction, and performing characteristic enhancement by using initial characteristics on the basis of a residual error framework to improve the characteristic extraction effect.
Drawings
FIG. 1 is a flow chart of a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network of the invention;
FIG. 2 is a schematic view of a first time point hyperspectral image of the present invention;
FIG. 3 is a second time point high spectral image of the present invention;
FIG. 4 is a schematic diagram of the results of the change detection of the present invention;
fig. 5 is a ground truth diagram of the actual change area of the invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings.
As shown in fig. 1, a hyperspectral change detection method based on a bidirectional reconstruction coding network and an enhanced residual error network of the invention comprises the following steps,
step (A), acquiring hyperspectral images of two time points in the same area, and forming a sample image, wherein the hyperspectral images of the same area are acquired at two different time points
Figure BDA0003293610800000061
And &>
Figure BDA0003293610800000062
Wherein
Figure BDA0003293610800000063
Representing the real number domain, h and w are height and width, and c is the number of bands.
Step (B), selecting 5% of all pixels in the sample image as training pixels, marking the pixels with change labels according to the change condition in the training pixels to obtain invariant pixels, which comprises the following steps,
step (B1), selecting 5% of all pixels in the sample image as training pixels, and setting a set U = {1, 2., hw } to represent subscripts of each pixel, wherein a subset pi = { Tr, va, te } of a power set of U is a random division of U and respectively represents a training set, a verification set and a test set, wherein | Tr | =0.05hw, | Va | =0.025h;
step (B2), according to the change condition in the training pixel, a change label is marked on the pixel, and the real change label of the ith pixel is marked as y true (i) Where i ∈ Tr, y true (i) =1 denotes that the i-th pixel is a changed pixel, y true (i) =0 indicates that the i-th pixel is an invariant pixel.
Step (C), a bidirectional coding network is constructed by utilizing the invariant pixels in the training pixels to reconstruct the spectrum, a reconstruction error and a variable spectrum vector are obtained, then the reconstruction error and the variable spectrum vector are fused into a spectrum comprehensive input, the specific steps are as follows,
step (C1), a bidirectional coding network is constructed by utilizing the invariant pixels in the training pixels to reconstruct the spectrum, and the spectrum corresponding to the ith pixel in the two hyperspectral images is set as
Figure BDA0003293610800000071
And &>
Figure BDA0003293610800000072
Using invariant pixel construction X in Tr (1) And X (2) Bidirectional reconstruction mapping f between 1 、g 1 、f 2 And g 2 Based on the formula (1), is selected>
Figure BDA0003293610800000073
Wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003293610800000074
represents->
Figure BDA0003293610800000075
In the mapping f k Coding of k ∈ {1,2}, j ∈ Tr and y true (j)＝0，/>
Figure BDA0003293610800000076
S =120;
step (C2), obtaining a reconstruction error, and solving the following optimization problem by using a coding network to obtain f 1 ,g 1 ，f 2 ,g 2 As shown in the formula (2) and the formula (3),
Figure BDA0003293610800000077
Figure BDA0003293610800000078
step (C3) of obtaining a variation spectrum vector and calculating a spectrum variation vector of the ith pixel
Figure BDA0003293610800000079
As shown in the formula (4),
Figure BDA0003293610800000081
and (C4) fusing the reconstruction error and the changed spectral vector into a spectral comprehensive input, and calculating the spectral comprehensive input spe _ input i ，
Figure BDA0003293610800000082
Step (D), the spectrum is comprehensively input into a one-dimensional convolution neural network to extract the spectral characteristics, and spe _ input is specifically used i Inputting the data into 3 layers of one-dimensional convolution kernel with the size of 3 to obtain a spectrum feature vector
Figure BDA0003293610800000083
And spe is the dimension of the spectral feature vector, and spe =120.
Step (E), using an Optimal Clustering Framework algorithm to screen a wave band with spatial information in a sample image, and forming a hyperspectral variable image, then taking out the spectrum of a training pixel and a neighborhood pixel from the hyperspectral variable image after the wave band screening to form a variable tensor, the concrete steps are as follows,
step (E1) of calculating a hyperspectral variation image
Figure BDA0003293610800000084
As shown in the formula (6),
X (D) ＝|X (1) -X (2) | (6)；
step (E2), using the Optimal Clustering Framework algorithm to X (D) C spatial wave bands are sorted, d wave bands are selected according to the sorting result to obtain a hyperspectral variable image after the wave bands are selected
Figure BDA0003293610800000085
Figure BDA0003293610800000086
Wherein d =24；
Step (E3), for the ith pixel, from
Figure BDA0003293610800000087
Extracting the combination of the change spectrum vector of the pixel and the change spectrum vectors of the pixels in the neighborhood into change tensor>
Figure BDA0003293610800000088
Where b is the spatial dimension of the change tensor, b =3.
Step (F), inputting the change tensor into the reinforced two-dimensional residual convolution neural network to extract spatial features, wherein the spatial features are specifically extracted
Figure BDA0003293610800000089
Inputting the space characteristic vector into a two-dimensional residual convolution neural network of 8 enhanced residual convolution blocks to obtain a space characteristic vector
Figure BDA00032936108000000810
Where spa is the dimension of the spatial feature vector, spa =12 × d =288, and let the output of the ith layer of the network be u l Then u is l+1 As shown in the formula (7),
Figure BDA00032936108000000811
wherein, therein
Figure BDA0003293610800000091
Represents the convolution operation of the l-th layer.
Step (G), the spectral feature and the spatial feature are fused and input into a full-connection network to obtain the classification result of the pixel, and a change detection result graph is generated, the specific steps are as follows,
step (G1) of calculating a spectral space vector
Figure BDA0003293610800000092
As shown in the formula (8), the,
Figure BDA0003293610800000093
step (G2), ssv i Inputting the data into a full-connection network for class II classification;
a step (G3) of generating a change detection result map based on the classification result of each pixel
Figure BDA0003293610800000094
Wherein the changed pixels are white and the unchanged pixels are black.
The following describes the use effect of the hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network, and a specific embodiment of the invention is shown in fig. 2 to 4, which can show the hyperspectral change detection effect, wherein fig. 2 and 3 are hyperspectral images of two different time phases in the same area, fig. 4 is a final change detection result diagram of the method, and fig. 5 is a result diagram of actual change, and as can be seen from the detection result and the result diagram of the actual change, the method can effectively detect the change area.
In summary, according to the hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network, firstly, a spatial-spectral combination strategy that spectra and spaces are completely separated and then fused is adopted, better feature extraction effects are sought on the spectra and the spaces respectively, then, bidirectional reconstruction is performed on the spectra of invariant pixels at two time points by adopting the coding network, reconstruction errors serve as new spectral feature sources, the influence of noise on the spectra is further suppressed, then, partial wave bands are screened based on a wave band selection algorithm to participate in spatial feature extraction, feature enhancement is performed by using initial features on the basis of a residual error framework to improve the feature extraction effect, and the hyperspectral image change detection method is suitable for a hyperspectral image change detection task.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (8)

1. A hyperspectral change detection method based on a bidirectional reconstruction coding network and a reinforced residual error network is characterized by comprising the following steps: comprises the following steps of (a) carrying out,
the method comprises the following steps that (A), hyperspectral images of two time points in the same area are obtained, and a sample image is formed;
selecting 5% of all pixels in the sample image as training pixels, and marking the pixels with change labels according to the change condition in the training pixels to obtain invariant pixels;
step (C), a bidirectional coding network is constructed by using invariant pixels in the training pixels to reconstruct the spectrum, a reconstruction error and a variable spectrum vector are obtained, and then the reconstruction error and the variable spectrum vector are fused into comprehensive spectrum input;
step (D), comprehensively inputting the spectrum into a one-dimensional convolution neural network to extract the spectral characteristics;
step (E), filtering a wave band with spatial information in a sample image by using an Optimal Clustering frame algorithm, forming a hyperspectral variable image, and then taking out the spectrum of a training pixel and a neighborhood pixel from the hyperspectral variable image after the wave band filtering to form a change tensor;
step (F), inputting the change tensor into the reinforced two-dimensional residual convolution neural network to extract spatial features;
and (G) fusing the spectral characteristics and the spatial characteristics, inputting the fused spectral characteristics and spatial characteristics into a full-connection network to obtain a classification result of the pixels, and generating a change detection result graph.
2. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 1 is characterized in that: step (A), acquiring hyperspectral images of two time points in the same area, and forming a sample image, wherein the hyperspectral images are specifically acquired at two different timesPoint-collected hyperspectral images of the same area
Figure FDA0003293610790000011
And
Figure FDA0003293610790000012
wherein->
Figure FDA0003293610790000013
Representing the real number domain, h and w are height and width, and c is the number of bands.
3. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 1 is characterized in that: step (B), selecting 5% of all pixels in the sample image as training pixels, and marking the pixels with change labels according to the change condition in the training pixels to obtain invariant pixels, which comprises the following steps,
step (B1), selecting 5% of all pixels in the sample image as training pixels, and setting a set U = {1, 2., hw } to represent subscripts of each pixel, wherein a subset pi = { Tr, va, te } of a power set of U is a random division of U and respectively represents a training set, a verification set and a test set, wherein | Tr | =0.05hw, | Va | =0.025h;
step (B2), according to the change condition in the training pixel, a change label is marked on the pixel, and the real change label of the ith pixel is marked as y true (i) Where i ∈ Tr, y true (i) =1 denotes that the i-th pixel is a changed pixel, y true (i) =0 indicates that the i-th pixel is an invariant pixel.
4. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 3 is characterized in that: step (C), a bidirectional coding network is constructed by utilizing the invariant pixels in the training pixels to reconstruct the spectrum, a reconstruction error and a variable spectrum vector are obtained, then the reconstruction error and the variable spectrum vector are fused into a spectrum comprehensive input, the specific steps are as follows,
step (C1), a bidirectional coding network is constructed by utilizing the invariant pixels in the training pixels to reconstruct the spectrum, and the spectrum corresponding to the ith pixel in the two hyperspectral images is set as
Figure FDA0003293610790000021
Using invariant pixel construction X in Tr (1) And X (2) Bidirectional reconstruction mapping f between 1 、g 1 、f 2 And g 2 As shown in the formula (1),
Figure FDA0003293610790000022
wherein the content of the first and second substances,
Figure FDA0003293610790000023
represents->
Figure FDA0003293610790000024
In the mapping f k Coding of k ∈ {1,2}, j ∈ { Tr and y ∈ true (j)＝0，/>
Figure FDA0003293610790000025
S =120;
step (C2), obtaining a reconstruction error, and solving the following optimization problem by using a coding network to obtain f 1 ，g 1 ,f 2 ,g 2 As shown in the formula (2) and the formula (3),
Figure FDA0003293610790000031
Figure FDA0003293610790000032
step (C3) of obtaining a variation spectral vector and calculating the ith pixelVector of spectral variation
Figure FDA0003293610790000033
As shown in the formula (4),
Figure FDA0003293610790000034
and (C4) fusing the reconstruction error and the changed spectral vector into a spectral comprehensive input, and calculating the spectral comprehensive input spe _ input i ，
Figure FDA0003293610790000035
5. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 4 is characterized in that: step (D), the spectrum is comprehensively input into a one-dimensional convolution neural network to extract the spectral characteristics, and spe _ input is specifically used i Inputting the data into 3 layers of one-dimensional convolution kernel with the size of 3 to obtain a spectral feature vector
Figure FDA0003293610790000036
And spe is the dimension of the spectral feature vector, and spe =120.
6. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 5 is characterized in that: step (E), using an Optimal Clustering Framework algorithm to screen wave bands with spatial information in a sample image, and forming a hyperspectral variable image, then taking out the spectrum of a training pixel and a neighborhood pixel from the hyperspectral variable image after wave band screening to form a change tensor, the concrete steps are as follows,
step (E1) of calculating a hyperspectral variation image
Figure FDA0003293610790000038
As shown in the formula (6),
X (D) ＝|X (1) -X (2) | (6)；
step (E2), using the Optimal Clustering Framework algorithm to X (D) C spatial wave bands are sorted, d wave bands are selected according to the sorting result to obtain a hyperspectral variable image after the wave bands are selected
Figure FDA0003293610790000037
Figure FDA0003293610790000041
Wherein d =24;
step (E3), for the ith pixel, from
Figure FDA0003293610790000042
Extracting the combination of the change spectrum vector of the pixel and the change spectrum vector of the pixel adjacent to the pixel into change tensor->
Figure FDA0003293610790000043
Where b is the spatial dimension of the variation tensor, b =3.
7. The method for detecting hyperspectral change based on the bidirectional reconstructed coding network and the reinforced residual error network according to claim 6 is characterized in that: step (F), inputting the change tensor into the reinforced two-dimensional residual convolution neural network to extract spatial features, wherein the spatial features are specifically extracted
Figure FDA0003293610790000044
Inputting the signals into a two-dimensional residual convolutional neural network consisting of 8 enhanced residual convolutional blocks to obtain a spatial feature vector->
Figure FDA0003293610790000045
Where spa is the dimension of the spatial feature vector, spa =12 × d =288, and let the output of the ith layer of the network be u l Then u is l+1 Is shown asIs shown in a formula (7),
Figure FDA0003293610790000046
/>
wherein, wherein
Figure FDA0003293610790000047
Represents the convolution operation of the l-th layer.
8. The hyperspectral change detection method based on the bidirectional reconstruction coding network and the enhanced residual error network according to claim 7 is characterized in that: step (G), the spectral feature and the spatial feature are fused and input into a full-connection network to obtain the classification result of the pixel, and a change detection result graph is generated, the specific steps are as follows,
step (G1) of calculating a spectral space vector
Figure FDA0003293610790000048
As shown in the formula (8),
Figure FDA0003293610790000049
step (G2) of converting ssv i Inputting the data into a full-connection network for classification of two types;
a step (G3) of generating a change detection result map from the classification result of each pixel
Figure FDA00032936107900000410
Wherein the changed pixels are white and the unchanged pixels are black. />
CN202111172046.8A 2021-10-08 2021-10-08 Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network Active CN113962943B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111172046.8A CN113962943B (en) 2021-10-08 2021-10-08 Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111172046.8A CN113962943B (en) 2021-10-08 2021-10-08 Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network

Publications (2)

Publication Number Publication Date
CN113962943A CN113962943A (en) 2022-01-21
CN113962943B true CN113962943B (en) 2023-04-07

Family

ID=79463626

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111172046.8A Active CN113962943B (en) 2021-10-08 2021-10-08 Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network

Country Status (1)

Country Link
CN (1) CN113962943B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115240072B (en) * 2022-08-04 2024-03-12 南京审计大学 Hyperspectral multi-class change detection method based on multidirectional multi-scale spectrum-space residual convolution neural network

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634137A (en) * 2020-12-28 2021-04-09 西安电子科技大学 Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017066825A1 (en) * 2015-10-19 2017-04-27 National Ict Australia Limited Spectral reconstruction

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634137A (en) * 2020-12-28 2021-04-09 西安电子科技大学 Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
面向图像场景转换的改进型生成对抗网络;肖进胜等;《软件学报》;20210930;全文 *

Also Published As

Publication number Publication date
CN113962943A (en) 2022-01-21

Similar Documents

Publication Publication Date Title
Liu et al. Remote sensing image fusion based on two-stream fusion network
Javan et al. A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery
CN111539316B (en) High-resolution remote sensing image change detection method based on dual-attention twin network
CN111127374B (en) Pan-sharing method based on multi-scale dense network
CN110660038B (en) Multispectral image and full-color image fusion method based on generation countermeasure network
Liebel et al. Single-image super resolution for multispectral remote sensing data using convolutional neural networks
Pan Cloud removal for remote sensing imagery via spatial attention generative adversarial network
CN109727207B (en) Hyperspectral image sharpening method based on spectrum prediction residual convolution neural network
CN111080629A (en) Method for detecting image splicing tampering
Guan et al. Multistage dual-attention guided fusion network for hyperspectral pansharpening
CN109636769A (en) EO-1 hyperion and Multispectral Image Fusion Methods based on the intensive residual error network of two-way
CN110544212B (en) Convolutional neural network hyperspectral image sharpening method based on hierarchical feature fusion
CN111914909B (en) Hyperspectral change detection method based on space-spectrum combined three-direction convolution network
CN114266957B (en) Hyperspectral image super-resolution restoration method based on multi-degradation mode data augmentation
Castrodad et al. Discriminative sparse representations in hyperspectral imagery
Xue et al. Sparse graph regularization for robust crop mapping using hyperspectral remotely sensed imagery with very few in situ data
CN115205590A (en) Hyperspectral image classification method based on complementary integration Transformer network
CN113962943B (en) Hyperspectral change detection method based on bidirectional reconstruction coding network and reinforced residual error network
Li et al. DnSwin: Toward real-world denoising via a continuous Wavelet Sliding Transformer
Yan et al. Md³net: Integrating model-driven and data-driven approaches for pansharpening
CN114863173A (en) Land resource audit-oriented self-interaction high-attention spectrum image classification method
CN113421198A (en) Hyperspectral image denoising method based on subspace non-local low-rank tensor decomposition
CN109460788B (en) Hyperspectral image classification method based on low-rank-sparse information combination network
Ma et al. Hyperspectral image denoising based on low-rank representation and superpixel segmentation
CN114511470B (en) Attention mechanism-based double-branch panchromatic sharpening method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant