CN114155443A - Hyperspectral image classification method based on multi-receptive-field attention network - Google Patents

Hyperspectral image classification method based on multi-receptive-field attention network Download PDF

Info

Publication number
CN114155443A
CN114155443A CN202111153710.4A CN202111153710A CN114155443A CN 114155443 A CN114155443 A CN 114155443A CN 202111153710 A CN202111153710 A CN 202111153710A CN 114155443 A CN114155443 A CN 114155443A
Authority
CN
China
Prior art keywords
attention
node
features
receptive
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111153710.4A
Other languages
Chinese (zh)
Inventor
丁遥
张志利
赵晓枫
蔡伟
阳能军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rocket Force University of Engineering of PLA
Original Assignee
Rocket Force University of Engineering of PLA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rocket Force University of Engineering of PLA filed Critical Rocket Force University of Engineering of PLA
Priority to CN202111153710.4A priority Critical patent/CN114155443A/en
Publication of CN114155443A publication Critical patent/CN114155443A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a hyperspectral image classification method of a multi-sensor wildcard attention network. Accurately dividing an original hyperspectral image into self-adaptive local areas by using an unsupervised principal component analysis method and simple linear iterative clustering; the spectral characteristics of the pixels are extracted by adopting a double-layer convolutional neural network, so that the number of nodes needing to be calculated is reduced, and the noise of an original hyperspectral image is suppressed; constructing a multi-sensing-field map based on a superpixel cube; extracting local node features and edge features by using a multi-feature attention module and learning important coefficients of the two features by using a graph attention network; and finally, interpreting the image features by using cross entropy loss to obtain a label of each pixel, so that the hyperspectral image is correctly classified. Compared with the prior art, the hyperspectral features can be automatically extracted and classified, and the classification accuracy rate reaches over 93%.

Description

Hyperspectral image classification method based on multi-receptive-field attention network
Technical Field
The invention relates to the technical field of geographical remote sensing, in particular to a hyperspectral image classification method of a multi-sensor wilderness attention network.
Background
The hyperspectral image (HSI) is a three-dimensional image taken by an aerospace vehicle equipped with a hyperspectral imager. Each pixel in the image contains hundreds of different bands of reflectance information, which makes it suitable for many practical applications, such as military target detection, mineral exploration, and agricultural production. The goal of hyperspectral image classification is to classify each pixel in an image into a specific label based on sample features, and these pixels can be represented by different image colors. In recent years, hyperspectral image classification has increasingly become a research hotspot. However, label deficiency, high data dimensionality, spectral similarity, pixel blending, etc. present a significant challenge to hyperspectral image classification.
In the early stages, various machine learning classification methods are applied to HSI classification, such as Support Vector Machine (SVM), k-nearest neighbor (KNN), naive bayes, decision trees, and Extreme Learning Machine (ELM). It is worth mentioning that SVM is one of the most representative and effective machine learning classification methods. With the development of support vector machine methods, support vector machine methods using some kernel transformation techniques, such as multi-kernel learning and Kernel Support Vector Machine (KSVM), are widely used. Meanwhile, sparse representation based classifiers (SRC) have also attracted more and more attention, and a large number of SRC classifier based algorithms have been proposed. However, the above methods are all based on the spectral characteristics of HSI, do not consider the spatial information of HSI, and the classification performance is not ideal. In order to fully utilize spatial information, some classifiers based on spectral-spatial information, such as superpixel segmentation, morphological segmentation and graph construction, have also been studied. However, conventional machine learning classification methods rely heavily on expertise and are unable to learn deep features from HSI, such as the information disclosed in "Spectral-spatial classification of hyperspectral images with a super-pixel-based discrete search model," IEEE trans. geosci. remove series, vol.53, No.8, pp.4186-4201, aug.2015), published by l.fang, s.li, x.kang, and j.a. Benediktsson.
With the development of deep learning and the progress of Artificial Intelligence (AI), deep learning methods have been widely applied in the fields of natural language processing, computer vision, intelligent decision making, and the like. Inspired by the successful application of deep learning, many deep learning classifiers have been developed for HSI classification with excellent performance, such as Stacked Automatic Encoders (SAEs), Recurrent Neural Networks (RNN), and Convolutional Neural Networks (CNN). Compared with a machine learning method, the deep learning classifier can automatically learn deep complex features of the HSI. Among the above deep learning methods, CNN shows a bright foreground and is widely used. In addition, as deep learning progresses, various CNN-based methods are proposed, from one-dimensional CNN to three-dimensional CNN, from single CNN to hybrid CNN, from shallow CNN to deep CNN. However, early CNN methods only learned local spatial information and could not extract deep spatio-spectral features of HSI, such as disclosed in "CNN-based multi-layer spatial-spectral fusion and sample amplification with local and non-local constraints for hyper-spectral image classification," IEEE J.Sel.topics application.earth observer.Remote Sens., vol.12, No.4, pp.1299-1313, Apr.2019, published by J.Feng et al. To overcome these drawbacks, different forms of CNN methods are beginning to emerge. Recently, some advanced CNN techniques such as spectral-spatial attention networks and dynamic convolution kernels have been explored to improve CNN-based approaches. However, CNNs require a large number of training tags to train the network, which contradicts the tag deficiency in HSI and these advanced CNN deep learning models are designed for euclidean data, the inherent correlation between adjacent land covers is often neglected, making the above methods unable to learn the relationship between labeled and unlabeled data, such as that disclosed in "a CNN with scale association and transformed method for hyperspectral identification," trans.
Therefore, the convolutional neural network in both the traditional machine learning method and the deep learning faces certain limitation in the hyperspectral classification.
Disclosure of Invention
In order to overcome the problem that the convolutional neural network faces limitation in hyperspectral classification in the traditional machine learning method and the deep learning method, the invention aims to provide a novel hyperspectral image classification method of a multi-scale receptive field map attention neural network.
The concept and technical solution of the present invention will now be described as follows:
the basic concept of the invention is that in a convolutional neural network, a superpixel segmentation algorithm is adopted to refine local spatial features of original HSI, two layers of one-dimensional CNNs are provided to extract superpixels and spectral features capable of being automatically learned, edges of a graph and local semantic features of a learning graph are introduced into a graph attention network (GAT), a multi-scale receptive field GAT is provided for extracting local-global adjacent node features and edge features, and finally, a graph attention network and a softMax function are utilized to perform multi-receiving feature fusion and pixel label prediction.
The invention relates to a technical solution of a hyperspectral image multi-scale classification method combining a graph structure and a convolutional neural network, which has a superpixel cube structure multi-receptive-field graph and a convolutional neural network and is characterized in that: accurately partitioning the original HSI into adaptive local regions (superpixels) by using a supervised-free Principal Component Analysis (PCA) method and Simple Linear Iterative Clustering (SLIC); the double-layer 1D CNN is adopted to extract the spectral characteristics of the pixels, so that the number of nodes to be calculated is reduced, and the noise of the original HSI is suppressed; constructing a multi-sensing-field map based on a superpixel cube; extracting local node features and edge features by using a multi-feature attention module and learning important coefficients of the two features by using a graph attention network; and fusing each receptive field node-edge characteristic by utilizing a characteristic fusion attention module to provide the characteristics of the classification nodes. Finally, the image features are explained by using cross entropy loss to obtain the label of each pixel, and the method specifically comprises the following steps:
step 1: constructing a spectral-spatial conversion module
Given HSI cube IB＝{x1,x2,…,xmIs composed of
Figure BDA0003287895020000031
One pixel, B wave bands, wherein
Figure BDA0003287895020000032
H respectively represents the spatial width and height of the hyperspectral image; in order to improve the calculation efficiency, the Principal Component Analysis (PCA) method is used for reducing the dimension, and a first principal component is selected to generate a dimension reduction image
Figure BDA0003287895020000033
Having m pixels and b wavebands, where b < B, r denotes dimensionality reduction; then, the SLIC is adopted to divide the pixels into superpixels, and the local superpixel HSI can be expressed by a mathematical expression as:
Figure BDA0003287895020000034
wherein
Figure BDA0003287895020000035
Is represented by comprising niA super-pixel of a single pixel,
Figure BDA0003287895020000036
k is the total number of superpixels; the pixels in the super pixel have strong spectral spatial correlation; in the method of the invention, the superpixels are taken as nodes of the graph; by controlling the number K of the super pixels, the scale of the graph can be controlled, and the calculation complexity of the algorithm is reduced;
step 2: constructing a frequency domain conversion module:
step 2.1: in order to extract the discriminative and robust spectral features for the proposed method, a spectral transformer is provided, whose framework is shown in fig. 1; two layers of 1 x 1CNN kernels are used to extract the spectral values of a single pixel in each band, and in the proposed method, the spatial position p0The spectral feature vector of the pixel can be written as:
Figure BDA0003287895020000037
wherein p is0(X, y) is the spatial position of the pixel in the HSI, Xi(p0) Representing the spatial position p of the pixel in the ith spectrum0The spectral value of (d);
step 2.2: output characteristics of convolution layer in the b-th spectral band
Figure BDA0003287895020000038
Is composed of
Figure BDA0003287895020000039
Wherein
Figure BDA00032878950200000310
And
Figure BDA00032878950200000311
trainable weights (1 × 1 convolution kernels) and bias, respectively, σ (·) is the activation function, i.e., ReLU;
step 2.3: in order to display the corresponding relation between the pixels and the super pixels, an incidence matrix is constructed
Figure BDA00032878950200000312
In particular, M can be calculated as
Figure BDA0003287895020000041
Wherein xiIs IB"flattening" (HSI) denotes a flattening operation of the hyperspectral image in the spatial dimension. As described in equation (4), a mapping between spatial pixels and superpixels can be achieved
Step 2.4: finally, the average spectral feature of each superpixel is taken as a node feature vector, and the graph node feature can be expressed mathematically as
Figure BDA0003287895020000042
Wherein HiIs the ith node feature vector, NiRepresenting the number of pixels contained in the super-pixel,
Figure BDA0003287895020000043
is the spectral feature vector of the pixel as shown in equation (2);
and step 3: constructing a multi-feature attention Module
The attention mechanism and the multiple features are very important for extracting the features of the graph, and the features of the neighbor nodes and the edges are extracted through the multi-feature attention module; the multi-feature attention can be divided into three parts, namely neighbor node attention, edge attention and feature fusion attention; the specific details of these three sections are as follows:
step 3.1: neighbor node attention
In order to aggregate information of neighbor nodes, the module adopts a multilayer graph attention mechanism, neighbor node attention coefficients of classified nodes can be obtained, and the output of the first convolution layer can be calculated as
Figure BDA0003287895020000044
After multilayer convolution, niCan be converted into a form that
Figure BDA0003287895020000045
Where ← represents the assignment symbol, i is the ith hop neighbor of node n, K represents the number of neighbor nodes in the ith hop neighbor of node n,
Figure BDA0003287895020000046
represents nikThe importance coefficient of (a); n isiIs a neighbor node characteristic of the central node n, n according to equation (6)iCan be decomposed into a linear sum of the characteristics of the neighboring nodes;
step 3.2: attention at the same time
Edges contain rich semantic information. However, most graph attention networks only focus on the aggregation of adjacent nodes, and lack the extraction of edge features; inspired by the calculation edge in the graph convolution, the step introduces the Gaussian distance aijTo define relationships between nodes, i.e.
Figure BDA0003287895020000051
Wherein h isiAnd hjThe spectral characteristics (calculated by equation (4)) of the nodes i, j are represented, and the euclidean distance between the two nodes is represented as | | | hi-hj||2,
Figure BDA0003287895020000052
Represents hjThe t hop neighbor node set of (1), gamma represents that the experience set is 0.2;
similar to equation (5), the output of the first convolutional layer in side attention can be expressed as
Figure BDA0003287895020000053
Wherein
Figure BDA0003287895020000054
Represents the output of the l-1 convolutional layer in edge attention,
Figure BDA0003287895020000055
learning attention coefficient, a, representing side attentioniCan be converted into
Figure BDA0003287895020000056
Wherein a isiIs the ith edge-hopping feature of the center node n,
Figure BDA0003287895020000057
denotes aikThe importance coefficient of (a);
step 3.3: feature fusion attention
In order to comprehensively utilize the edge and node characteristics and provide characteristic fusion attention, the characteristics extracted by the attention module are adaptively fused, and the mathematical expression of n can be expressed as
Figure BDA0003287895020000058
Wherein
Figure BDA0003287895020000059
And
Figure BDA00032878950200000510
respectively representing the important coefficients of adjacent nodes and edges;
as described above, centroid node n may ultimately be represented as
Figure BDA00032878950200000511
Wherein alpha isiAnd betaiRespectively represent nikAnd aikThe weight coefficient of (2).
And 4, step 4: construction of multiple receptor fields Module
In order to extract the global information of HSI, a multi-branch receptive field aggregation mechanism is designed; the mechanism of multi-receptor field design is shown in FIG. 1; the multi-receptive-field module can avoid the negative influence of the low-order jumping points and learn the remote characteristic information of the HSI; in branch s, the receptive field for node n may be formed
Ri(n)＝Ri-1(n)∪R1 (12)
Wherein R is1(n) is a set of 1-hop neighbors of node n, R0(n) n; the characteristics of the central node n in branch i can finally be represented as
Figure BDA0003287895020000061
Wherein i is the ith hop neighbor of node n;
and 5: feature fusion attention and decision module
And finally, adopting graphic attention to fuse the multi-scale receptive field characteristics. The output characteristic O of the central node n can be calculated as
O＝σ(∑i∈Sei·WTni) (14)
Wherein eiSignificant coefficients representing features in branch i, S is the number of multi-scale degrees, W is the trainable weight, σ is the activation function, i.e., LeakyReLU; to determine the label of each pixel, the output features O are classified using a softmax classifier, i.e.
Figure BDA0003287895020000062
Step 6: loss function and model training
Penalizing differences between the network output and the original markup tags using a cross-entropy function, i.e.
Figure BDA0003287895020000063
Wherein y isGIs a set of tags; c denotes the number of classes, YzfIs a training label matrix; end-to-end training is adopted, and Adam is adopted to update the network parameters of the invention;
in the invention, three hyper-parameters are set, namely the number of the hyper-pixels K, the number L of convolution layers of MFaM, the number T of iteration times and the learning rate lr; the optimal hyper-parameter settings are shown in table 1.
This compares beneficial effect with prior art and is: a one-dimensional CNN is provided to learn the spectral characteristics of the superpixels; secondly, graph edges are introduced into the GAT, a feature fusion mechanism based on the GAT is researched, and the features of the classification nodes are represented by combining neighbor node information and edge information; and thirdly, a multi-scale receptive field mechanism is provided, multi-scale local semantic features are extracted, and local global spatial context information is learned. The hyperspectral image classification method can automatically extract hyperspectral features and complete classification. The classification accuracy reaches more than 93 percent.
Drawings
FIG. 1: the invention relates to a hyperspectral image classification method based on a multi-receptive-field attention network.
The experiment of the invention is that the invention is trained on a pytorch 1.8 by using a GeForce GTX 1080Ti 11G GPU and a computer of a 3.70G Intel i9-10900K CPU. Firstly, performing superpixel segmentation on an original hyperspectral image, and extracting the spectral feature of each superpixel block; then constructing a multi-sensing-field map with the super pixels as nodes; then extracting graph nodes and edge features by using a multi-feature attention module; and finally, interpreting the image characteristics by using cross entropy loss to obtain a label of each pixel, and predicting the node.
Detailed Description
The embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.
The method of the invention comprises four parts: the system comprises a spectrum space transformation module for HSI preprocessing, a multi-sensing-field graph construction module for spectrum space feature extraction, a multi-feature attention module, a feature fusion attention and decision module. The process of the method of the invention is realized as follows: first, the original HSI is accurately divided into adaptive local regions (superpixels) by using unsupervised Principal Component Analysis (PCA) method and Simple Linear Iterative Clustering (SLIC); the spectral characteristics of the pixels are extracted by adopting the double-layer 1D CNN, so that the number of nodes needing to be calculated is reduced, and the noise of the original HSI is suppressed; constructing a multi-sensor field map based on a super-pixel cube; extracting local node features and edge features by using a multi-feature attention module and learning important coefficients of the two features by using a graphical attention network; and fusing each receptive field node-edge characteristic by utilizing a characteristic fusion attention module to provide the characteristic of the classification node. Finally, the image features are interpreted by using cross entropy loss to obtain labels of each pixel, and as shown above, the HSI cube can be divided into superpixels by a space transformer. However, PCA dimensionality reduction loses spectral information of HSI, i.e., superpixels cannot adequately extract spectral features of HSI. The general approach is to extract the spectral value of each pixel directly from the original HSI and then calculate the spectral average of the pixels in the superpixel. The method is simple and intuitive, but cannot suppress and eliminate the noise of the original HSI through network training.
TABLE 1 hyper-parameter settings for different datasets
Figure BDA0003287895020000071
And (4) classification results:
the experiment was conducted on the invention in a pytorch 1.8, using a GeForce GTX 1080Ti 11G GPU and a 3.70G Intel i9-10900K CPU computer. The method is used for classifying the data sets of Pavia University, Salinas and Houston 2013, overall classification accuracy (OA), average classification accuracy (AA) and Kappa coefficient (Kappa) are used as measuring indexes, 10 times of test operation are carried out for each time, and the average value is taken to obtain the result (OA: the proportion correctly classified in all samples; AA: the average value of the proportion correctly classified in each class; Kappa: the consistency of correctly classified data is expressed.)
TABLE 2 results of three standard data set experiments
Pavia University Salinas Houston 2013
OA(％) 98.62±0.76 98.65±0.28 94.87±0.76
AA(％) 98.38±0.92 99.05±0.15 95.70±0.52
Kappa 0.98 0.98 0.94
The results of the three standard data sets in the table 2 show that the method has good classification effect on each data set, has good adaptability to different data sets, and achieves high classification precision.

Claims (7)

1. A hyperspectral image classification method based on a multi-receptive-field attention network is provided with a multi-receptive-field image and a convolutional neural network which are constructed by a superpixel cube, and is characterized in that: accurately partitioning an original hyperspectral image (HSI) into adaptive local regions by using an unsupervised Principal Component Analysis (PCA) method and Simple Linear Iterative Clustering (SLIC); the spectral characteristics of the pixels are extracted by adopting a double-layer 1D Convolutional Neural Network (CNN), so that the number of nodes needing to be calculated is reduced, and the noise of the original HSI is suppressed; constructing a multi-sensing-field map based on a superpixel cube; extracting local node features and edge features by using a multi-feature attention module and learning important coefficients of the two features by using a graph attention network; fusing each receptive field node-edge characteristic by utilizing a characteristic fusion attention module, giving out the characteristic of a classification node, and finally, interpreting the image characteristic by utilizing cross entropy loss to obtain a label of each pixel, wherein the method specifically comprises the following steps:
step 1: constructing a spectral-spatial conversion module
Step 2: constructing a frequency domain conversion module:
and step 3: constructing a multi-feature attention Module
And 4, step 4: construction of multiple receptor fields Module
And 5: feature fusion attention and decision module
Step 6: loss functions and model training.
2. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the step 1 of constructing the spectrum space conversion module specifically comprises the following steps:
given HSI cube IB＝{x1,x2,…,xmIs composed of
Figure FDA0003287895010000011
One pixel, B wave bands, wherein
Figure FDA0003287895010000012
H respectively represents the spatial width and height of the hyperspectral image; in order to improve the calculation efficiency, the Principal Component Analysis (PCA) method is used for reducing the dimension, and a first principal component is selected to generate a dimension reduction image
Figure FDA0003287895010000013
Having m pixels and b bands, where b<<B. r represents the dimensionality reduction; then, the SLIC is adopted to divide the pixels into superpixels, and the local superpixel HSI can be expressed by a mathematical expression as:
Figure FDA0003287895010000014
wherein
Figure FDA0003287895010000015
Is represented by comprising niA super-pixel of a single pixel,
Figure FDA0003287895010000016
k is the total number of superpixels; the pixels in the super pixel have strong spectral spatial correlation; in the method of the invention, the superpixels are taken as nodes of the graph; by controlling the number of the super pixels K, the scale of the graph can be controlled, and the calculation complexity of the algorithm is reduced.
3. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the step 2 of constructing the frequency domain conversion module specifically comprises the following steps:
step 2.1: in order to extract discriminative and robust spectral features for the proposed method, a spectral transformer is provided that employs two layers of 1 x 1CNN kernels to extract spectral values of a single pixel in each band, and in the proposed method, a spatial position p0The spectral feature vector of the pixel can be written as:
Figure RE-FDA0003484350750000029
wherein p is0(X, y) is the spatial position of the pixel in the HSI, Xi(p0) Representing the spatial position p of the pixel in the ith spectrum0The spectral value of (d);
step 2.2: output characteristics of convolution layer in the b-th spectral band
Figure RE-FDA0003484350750000021
Is composed of
Figure RE-FDA0003484350750000022
Wherein
Figure RE-FDA0003484350750000023
And
Figure RE-FDA0003484350750000024
trainable weights (1 × 1 convolution kernels) and bias, respectively, σ (·) is the activation function, i.e., ReLU;
step 2.3: in order to display the corresponding relation between the pixels and the super pixels, an incidence matrix is constructed
Figure RE-FDA0003484350750000025
In particular, M can be calculated as
Figure RE-FDA0003484350750000026
Wherein xiIs IB"flattening" (HSI) denotes a flattening operation of the hyperspectral image in the spatial dimension, as described in equation (4), a mapping between spatial pixels and superpixels can be achieved;
step 2.4: finally, the average spectral feature of each superpixel is used as a node feature vector, and the graph node feature can be expressed mathematically as
Figure RE-FDA0003484350750000027
Wherein HiIs the ith node feature vector, NiRepresenting the number of pixels contained in the super-pixel,
Figure RE-FDA0003484350750000028
is the spectral feature vector of the pixel as shown in equation (5).
4. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the multi-feature attention can be divided into three parts, namely neighbor node attention, side attention and feature fusion attention; the specific details of these three sections are as follows:
step 3.1: neighbor node attention
In order to aggregate information of neighbor nodes, the module adopts a multilayer graph attention mechanism, neighbor node attention coefficients of classified nodes can be obtained, and the output of the first convolution layer can be calculated as
Figure FDA0003287895010000031
After multilayer convolution, niCan be converted into a form that
Figure FDA0003287895010000032
Where ← represents the assignment symbol, i is the ith hop neighbor of node n, K represents the number of neighbor nodes in the ith hop neighbor of node n,
Figure FDA0003287895010000033
represents nikThe importance coefficient of (a); n isiIs a neighbor node characteristic of the central node n, n according to equation (7)iCan be decomposed into linear sums of adjacent node features;
step 3.2: attention at the same time
Edges contain rich semantic information. However, most graph attention networks only focus on the aggregation of adjacent nodes, and lack the extraction of edge features; inspired by the calculation edge in the graph convolution, the step introduces the Gaussian distance aijTo define relationships between nodes, i.e.
Figure FDA0003287895010000034
Wherein h isiAnd hjThe spectral characteristics of the nodes i, j are represented (calculated by equation (5)), and the euclidean distance between two nodes is represented as | | hi-hj||2,
Figure FDA0003287895010000035
Represents hjThe t hop neighbor node set of (1), gamma represents that the experience set is 0.2;
similar to equation (6), the output of the first convolutional layer in side attention can be expressed as
Figure FDA0003287895010000036
Wherein
Figure FDA0003287895010000037
Represents the output of the l-1 convolutional layer in edge attention,
Figure FDA0003287895010000038
learning attention coefficient, a, representing side attentioniCan be converted into
Figure FDA0003287895010000039
Wherein a isiIs the ith edge-hopping feature of the center node n,
Figure FDA00032878950100000310
denotes aikThe importance coefficient of (a);
step 3.3: feature fusion attention
In order to comprehensively utilize edge and node features and provide feature fusion attention, the features extracted by the attention module are adaptively fused, and the mathematical expression of n can be expressed as
Figure FDA00032878950100000311
Wherein
Figure FDA0003287895010000041
And
Figure FDA0003287895010000042
respectively representing the important coefficients of adjacent nodes and edges;
as described above, centroid node n may ultimately be represented as
Figure FDA0003287895010000043
Wherein alpha isiAnd betaiRespectively represent nikAnd aikThe weight coefficient of (2).
5. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the specific steps of constructing the multi-receptive-field module in the step 4 are as follows:
in order to extract the global information of HSI, a multi-branch receptive field aggregation mechanism is designed; the multi-receptive-field module can avoid the negative influence of the low-order jumping points and learn the remote characteristic information of the HSI; in branch s, the receptive field for node n may be formed
Ri(n)＝Ri-1(n)∪R1 (13)
Wherein R is1(n) is a set of 1-hop neighbors of node n, R0(n) n; the characteristics of the central node n in branch i can be finally expressed as
Figure FDA0003287895010000044
Where i is the ith hop neighbor of node n.
6. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the feature fusion attention and decision module described in step 5 is specifically as follows:
and finally, adopting graphic attention to fuse the multi-scale receptive field characteristics. The output characteristic O of the central node n can be calculated as
O＝σ(∑i∈Sei·WTni) (15)
Wherein eiSignificant coefficients representing features in branch i, S is the number of multi-scale degrees, W is the trainable weight, σ is the activation function, i.e., LeakyReLU; to determine the label of each pixel, the output features O are classified using a softmax classifier, i.e.
Figure FDA0003287895010000045
7. The hyperspectral image classification based on the multi-receptive-field attention network of claim 1, wherein: the "loss function and model training" described in step 6 is specifically as follows:
penalizing differences between the network output and the original markup tags using a cross-entropy function, i.e.
Figure FDA0003287895010000046
Wherein y isGIs a set of tags; c denotes the number of classes, YzfIs a training label matrix; end-to-end training is adopted, and Adam is adopted to update the network parameters of the invention.
CN202111153710.4A 2021-09-29 2021-09-29 Hyperspectral image classification method based on multi-receptive-field attention network Pending CN114155443A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111153710.4A CN114155443A (en) 2021-09-29 2021-09-29 Hyperspectral image classification method based on multi-receptive-field attention network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111153710.4A CN114155443A (en) 2021-09-29 2021-09-29 Hyperspectral image classification method based on multi-receptive-field attention network

Publications (1)

Publication Number Publication Date
CN114155443A true CN114155443A (en) 2022-03-08

Family

ID=80462272

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111153710.4A Pending CN114155443A (en) 2021-09-29 2021-09-29 Hyperspectral image classification method based on multi-receptive-field attention network

Country Status (1)

Country Link
CN (1) CN114155443A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758203A (en) * 2022-03-31 2022-07-15 长江三峡技术经济发展有限公司 Residual dense visual transformation method and system for hyperspectral image classification
CN115908908A (en) * 2022-11-14 2023-04-04 北京卫星信息工程研究所 Remote sensing image gathering type target identification method and device based on graph attention network
CN116824282A (en) * 2023-08-31 2023-09-29 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention
CN117422932A (en) * 2023-11-17 2024-01-19 中国矿业大学 Hyperspectral image classification method based on multi-mode enhanced graph attention network
CN117893839A (en) * 2024-03-15 2024-04-16 华东交通大学 Multi-label classification method and system based on graph attention mechanism

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758203A (en) * 2022-03-31 2022-07-15 长江三峡技术经济发展有限公司 Residual dense visual transformation method and system for hyperspectral image classification
CN114758203B (en) * 2022-03-31 2023-01-10 长江三峡技术经济发展有限公司 Residual intensive visual transformation method and system for hyperspectral image classification
CN115908908A (en) * 2022-11-14 2023-04-04 北京卫星信息工程研究所 Remote sensing image gathering type target identification method and device based on graph attention network
CN115908908B (en) * 2022-11-14 2023-09-15 北京卫星信息工程研究所 Remote sensing image aggregation type target recognition method and device based on graph attention network
CN116824282A (en) * 2023-08-31 2023-09-29 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention
CN116824282B (en) * 2023-08-31 2023-11-10 中国石油大学(华东) Semi-supervised hyperspectral image classification method integrating multi-jump graph convolution and attention
CN117422932A (en) * 2023-11-17 2024-01-19 中国矿业大学 Hyperspectral image classification method based on multi-mode enhanced graph attention network
CN117422932B (en) * 2023-11-17 2024-05-28 中国矿业大学 Hyperspectral image classification method based on multi-mode enhanced graph attention network
CN117893839A (en) * 2024-03-15 2024-04-16 华东交通大学 Multi-label classification method and system based on graph attention mechanism
CN117893839B (en) * 2024-03-15 2024-06-07 华东交通大学 Multi-label classification method and system based on graph attention mechanism

Similar Documents

Publication Publication Date Title
Sellars et al. Superpixel contracted graph-based learning for hyperspectral image classification
Ding et al. Multiscale graph sample and aggregate network with context-aware learning for hyperspectral image classification
Ding et al. Semi-supervised locality preserving dense graph neural network with ARMA filters and context-aware learning for hyperspectral image classification
Wang et al. Adaptive spectral–spatial multiscale contextual feature extraction for hyperspectral image classification
Ghaderizadeh et al. Hyperspectral image classification using a hybrid 3D-2D convolutional neural networks
Song et al. A survey of remote sensing image classification based on CNNs
Ding et al. AF2GNN: Graph convolution with adaptive filters and aggregator fusion for hyperspectral image classification
Bhatti et al. MFFCG–Multi feature fusion for hyperspectral image classification using graph attention network
Lorenzo et al. Hyperspectral band selection using attention-based convolutional neural networks
Heydari et al. Meta-analysis of deep neural networks in remote sensing: A comparative study of mono-temporal classification to support vector machines
CN111695467B (en) Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion
CN114155443A (en) Hyperspectral image classification method based on multi-receptive-field attention network
Datta et al. Hyperspectral image classification: Potentials, challenges, and future directions
Kong et al. Spectral–spatial feature extraction for HSI classification based on supervised hypergraph and sample expanded CNN
Liu et al. Deep multigrained cascade forest for hyperspectral image classification
Vo et al. Semantic image segmentation using fully convolutional neural networks with multi-scale images and multi-scale dilated convolutions
Amorim et al. Semi-supervised learning with convolutional neural networks for UAV images automatic recognition
Cao et al. Deep convolutional networks with superpixel segmentation for hyperspectral image classification
Zuo et al. FSL-EGNN: Edge-labeling graph neural network for hyperspectral image few-shot classification
Chen et al. Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network
Mahmoud et al. Pseudoinverse learning autoencoder with DCGAN for plant diseases classification
Fırat et al. Spatial-spectral classification of hyperspectral remote sensing images using 3D CNN based LeNet-5 architecture
Aydemir et al. Semisupervised hyperspectral image classification using deep features
Rajendran et al. Hyperspectral image classification model using squeeze and excitation network with deep learning
Jiang et al. Hyperspectral image classification with transfer learning and Markov random fields

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination