CN111160273A - Hyperspectral image space spectrum combined classification method and device - Google Patents
Hyperspectral image space spectrum combined classification method and device Download PDFInfo
- Publication number
- CN111160273A CN111160273A CN201911403750.2A CN201911403750A CN111160273A CN 111160273 A CN111160273 A CN 111160273A CN 201911403750 A CN201911403750 A CN 201911403750A CN 111160273 A CN111160273 A CN 111160273A
- Authority
- CN
- China
- Prior art keywords
- features
- hyperspectral image
- spatial
- spectral
- sample set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a hyperspectral image space spectrum combined classification method and a hyperspectral image space spectrum combined classification device, wherein the method comprises the following steps: normalizing the input hyperspectral image data, and obtaining a training sample set S for extracting spectral features after first preprocessing1And obtaining a training sample set S for extracting spatial features through second preprocessing2Respectively extracting spectral information and multi-scale spatial information of the hyperspectral image; connecting the extracted spectral features and multi-scale spatial features, and inputting the spectral features and the multi-scale spatial features into a deformable convolution neural network to extract high-level features; and inputting the extracted high-level features into a full link network and a softmax classifier for classification. The invention further uses a deformable convolution networkAnd the spatial spectrum features of higher levels are extracted, so that the depth features of the hyperspectral images are automatically extracted under the condition of small samples, and the classification precision of the hyperspectral images is improved.
Description
Technical Field
The invention relates to a hyperspectral image classification method, in particular to a hyperspectral image space spectrum combined classification method based on a deformable convolutional neural network, and belongs to the technical field of remote sensing information processing.
Background
According to the detection sensing mechanism of the hyperspectral sensor, the hyperspectral data contains abundant space and spectral information of ground objects, and the hyperspectral sensor has the characteristics of high spectral resolution, multiple imaging wave bands, large information amount and the like. Therefore, the hyperspectral image is widely applied to the fields of military affairs, agriculture, medicine, mining and the like. The classification and identification of the ground objects in the hyperspectral image are important means for assisting researchers to quickly understand information contained in the hyperspectral image.
Although spectral resolution is high for hyperspectral images, spatial resolution is still low relative to multispectral, panchromatic, and other types of images. In this scenario, a large number of urban ground object pixels are mixed pixels. Due to the fact that different types of ground objects in urban areas are mixed together in high density, the intra-class difference of the same ground object is large, the inter-class difference of different ground objects is small, and the factors bring great challenges to high-spectrum classification of the urban areas.
Under the background, a hyperspectral image space spectrum joint classification method based on a deformable convolutional neural network is researched. Aiming at the problem of intra-class difference of hyperspectral images, performing intra-class clustering on each class of samples by using a k-means algorithm to select representative training samples; a multi-scale space-spectrum combined network and a deformable convolution network are designed in a hyperspectral image classification task, and the spatial spectrum features of the hyperspectral image are extracted autonomously, so that rich information acquired through hyperspectral remote sensing is fully utilized, and the method has important significance for practical application.
Disclosure of Invention
The invention aims to provide a hyperspectral image space spectrum joint classification method based on a deformable convolutional neural network. Aiming at the characteristic of large intra-class difference of hyperspectral images of urban areas, the method firstly carries out intra-class clustering on samples of different classes and selects representative training samples; and then, the spectral information of the hyperspectral image and the multi-scale spatial information are fully combined, and the network performance and the classification precision of the hyperspectral image are improved by utilizing the characteristic that the deformable convolutional neural network can adapt to geometric deformation.
In order to achieve the purpose, the invention adopts the following technical scheme:
according to a first aspect of the invention, a hyperspectral image space spectrum joint classification method comprises the following steps:
normalizing the input hyperspectral image data;
respectively carrying out first preprocessing on the normalized hyperspectral images to obtain a training sample set S1 for extracting spectral features and carrying out second preprocessing to obtain a training sample set S2 for extracting spatial features;
extracting spectral features from the sample set S1 and spatial features from the sample set S2 through a multi-scale spatial-spectral joint network;
connecting the extracted spectral features and multi-scale spatial features, and inputting the spectral features and the multi-scale spatial features into a deformable convolutional neural network capable of adapting to geometric deformation to extract high-level features; and
and inputting the extracted high-level features into a full-connection network classifier through a variable convolution network for classification so as to obtain a hyperspectral image space spectrum joint classification result based on a variable convolution neural network.
Further, the step of normalizing the input hyperspectral image data comprises performing normalization by the following formula:
wherein y (i, j, k) represents a pixel with position coordinates (i, j) in the kth wave band in the hyperspectral image, and max (y)k) And min (y)k) Respectively representing the maximum value and the minimum value of the three-dimensional hyperspectral image in the kth wave band.
Further, the step of performing a first preprocessing on the normalized hyperspectral image to obtain a training sample set S1 for extracting spectral features includes:
performing intra-class clustering on the spectrums contained in each class of samples in the hyperspectral image by using a k-means clustering algorithm, wherein the clustering number is 3;
selecting 20% of each clusterThe sample stores its label as training sample label GtrThe rest is used as a test sample label Gte；
Extracting with tape label GtrThe data block within the 11 × 11 × L neighborhood range as the center is taken as a training sample set S1 for extracting spectral features.
Further, the step of performing a second preprocessing to obtain a training sample set S2 for extracting spatial features includes: performing dimensionality reduction on the original hyperspectral image by using a Principal Component Analysis (PCA) method, and extracting a labeled G from the dimensionality-reduced imagetrThe data blocks within the 11 × 11 × D neighborhood range as the center are used as the training sample set S2 for extracting the spatial features.
Further, the multi-scale space-spectrum combined network comprises a spectral feature extraction network and a spatial feature extraction network, the spectral feature extraction network structure comprises a 1 × 1 convolutional layer, a Normalization layer Batch Normalization and an excitation function ReLU, the spatial feature extraction network structure comprises a 3 × 3 convolutional layer and a 5 × 5 convolutional layer which are connected in parallel, and the Normalization layer Batch Normalization and the excitation function ReLU are respectively arranged behind the convolutional layers, wherein a training sample set S1 is input into the spectral feature extraction network to extract spectral features, and a training sample set S2 is input into the spatial feature extraction network to obtain multi-scale spatial features.
Further, the convolutional layers included in the deformable convolutional network from the input to the output sequentially include a first two-dimensional convolutional layer, a first deformable convolutional layer, a second two-dimensional convolutional layer and a global average pooling layer.
Further, the convolution kernel size of each of the two-dimensional convolution layer and the deformable convolution layer is 3 × 3, and the step length is 1.
Further, the number of convolution kernels is in turn: 64, 128, 256, 512.
Further, each of the two-dimensional convolution layer and the deformable convolution layer is normalized by a Batch Normalization layer, and the ReLU is used as an excitation function.
Further, the fully-connected network classifier includes a fully-connected layer and a softmax classification layer.
According to a second aspect of the present invention, a hyperspectral image space spectrum joint classification device comprises:
a memory having computer instructions stored therein;
a processor in data connection with a memory, which executes the computer instructions to perform the method for spatio-spectral joint classification of hyperspectral images based on a deformable convolutional neural network according to any of claims 1 to 8.
According to a third aspect of the present invention, a hyperspectral image space spectrum joint classification device comprises:
the normalization processing module is used for performing normalization processing on the input hyperspectral image data;
the training sample set generating module is used for respectively carrying out first preprocessing on the hyperspectral images after the normalization processing to obtain a training sample set S1 for extracting spectral features and carrying out second preprocessing to obtain a training sample set S2 for extracting spatial features;
a first feature extraction module, configured to extract spectral features from the training sample set S1 and spatial features from the training sample set S2 through a multi-scale spatial-spectral joint network;
the second characteristic extraction module is used for connecting the extracted spectral characteristics with the multi-scale spatial characteristics and inputting the connected spectral characteristics and the multi-scale spatial characteristics into a deformable convolution neural network capable of adapting to geometric deformation so as to extract high-level characteristics; and
and the full-connection network classifier is used for obtaining a hyperspectral image space spectrum combined classification result based on the deformable convolution neural network according to the input high-level features extracted by the deformable convolution network.
The hyperspectral image space spectrum joint classification method based on the deformable convolutional neural network has the advantages that:
in the invention, when selecting samples, a k-means clustering algorithm is adopted to perform intra-class clustering on each class of samples and select representative samples aiming at the problem of intra-class difference of hyperspectral images. The training samples selected in the mode can better cover the sample space than the training samples selected randomly;
the multi-scale space-spectrum combined network designed by the invention simultaneously utilizes the spectral characteristics and multi-scale spatial characteristics of the hyperspectral image, and can extract more abundant and perfect characteristic information;
the deformable convolution network adopted by the invention can adaptively extract the ground feature characteristics with different sizes and shapes, and can effectively improve the classification precision of the hyperspectral image.
Drawings
FIG. 1 is a flow chart of a hyperspectral image space-spectrum joint classification method provided by the invention;
FIG. 2 is a schematic diagram of a network for extracting spectral features according to the present invention;
FIG. 3 is a schematic diagram of a spatial feature extraction network according to the present invention;
fig. 4 is a schematic diagram of a deformable convolutional neural network module according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a flow chart of the hyperspectral image space-spectrum joint classification method provided by the invention includes the following contents:
step one, normalizing input hyperspectral image data.
In the embodiment of the invention, three-dimensional hyperspectral image data with the size of M multiplied by N multiplied by L is firstly normalized. In the present invention, the step of normalizing the input hyperspectral image data includes performing normalization by using a normalization formula, where the normalization formula is:
wherein y (i, j, k) represents a pixel with a position coordinate of (i, j) in the kth wave band in the hyperspectral image, and max (y)k) And min (y)k) Respectively representing the maximum value and the minimum value of the three-dimensional hyperspectral image in the kth wave band
And step two, respectively carrying out first preprocessing on the normalized hyperspectral images to obtain a training sample set S1 for extracting spectral features and carrying out second preprocessing to obtain a training sample set S2 for extracting spatial features.
In this embodiment, the normalized hyperspectral images are respectively processed in a dual-channel manner to obtain corresponding sample sets. Specifically, the training sample set S1 with spectral features extracted after the first pre-processing and the training sample set S2 with spatial features extracted after the second pre-processing are obtained.
In the present invention, as shown in fig. 2, the step of subjecting the normalized hyperspectral image to a first preprocessing to obtain a training sample set S1 for extracting spectral features includes:
performing intra-class clustering on the spectrums contained in each class of samples in the hyperspectral image by using a k-means clustering algorithm, wherein the clustering number is 3;
selecting 20% of samples from each cluster and storing labels of the samples, wherein the labels are used as training sample labels GtrThe rest is used as a test sample label Gte；
Extracting with tape label GtrThe data block within the 11 × 11 × L neighborhood range as the center is taken as a training sample set S1 for extracting spectral features.
In the embodiment of the invention, the third dimension of the original image is the wave band number L, the invention utilizes principal component analysis to reduce the dimension of the original image, the first D principal components are selected, and the obtained channel number of the third dimension after dimension reduction is D.
As shown in fig. 3, the step of performing a second preprocessing to obtain a training sample set S2 for extracting spatial features includes:
performing dimensionality reduction on the original hyperspectral image by using a Principal Component Analysis (PCA) method, and extracting a labeled G from the dimensionality-reduced imagetrThe data blocks within the 11 × 11 × D neighborhood range as the center are used as the training sample set S2 for extracting the spatial features.
And step three, extracting spectral features from the sample set S1 and extracting spatial features from the sample set S2 through a multi-scale space-spectrum joint network.
In an embodiment of the present invention, the multi-scale space-spectrum combination network includes a spectral feature extraction network and a spatial feature extraction network, and the spectral feature extraction network structure includes a 1 × 1 convolution layer, a normalization layer BatchNormalization, and an excitation function ReLU.
The spatial feature extraction network structure comprises a 3 × 3 convolutional layer and a 5 × 5 convolutional layer which are connected in parallel, and a Batch Normalization layer and an excitation function ReLU are respectively normalized after the convolutional layers.
Wherein the training sample set S1 is input to the spectral feature extraction network to extract spectral features, and the training sample set S2 is input to the spatial feature extraction network to obtain multi-scale spatial features.
And step four, connecting the extracted spectral features and multi-scale spatial features, and inputting the spectral features and the multi-scale spatial features into a deformable convolution neural network capable of adapting to geometric deformation to extract high-level features.
According to an embodiment of the present invention, as shown in fig. 4, the deformable convolutional neural network includes a first two-dimensional convolutional layer, a first deformable convolutional layer, a second two-dimensional convolutional layer, and a global average pooling layer. The convolution kernel size of each of the two-dimensional convolution layer and the deformable convolution layer is 3 x 3, and the step length is 1. The number of convolution kernels is as follows: 64, 128, 256, 512. After each of the two-dimensional convolution layer and the deformable convolution layer, a Batch Normalization layer is used for Normalization, and ReLU is used as an excitation function.
Here, the high-level feature merely indicates the high-level of the feature with respect to a primary feature (e.g., a feature at a pixel level) of the original image after being processed by the artificial neural network, and is not intended to accurately describe the feature. However, generally speaking, through neural network processing, the neural network tends to be more hierarchical and abstract as it goes deeper.
And fifthly, inputting the extracted high-grade features into a full-connection network classifier through a variable convolution network for classification so as to obtain a hyperspectral image space spectrum combined classification result based on the variable convolution neural network.
The fully-connected network classifier includes a fully-connected layer and a softmax classification layer.
According to another embodiment of the invention, there is also provided a hyperspectral image spatial spectrum joint classification device based on a deformable convolutional neural network, which is used for executing the classification method, and the device includes:
the normalization processing module is used for performing normalization processing on the input hyperspectral image data;
the training sample set generating module is used for respectively carrying out first preprocessing on the hyperspectral images after the normalization processing to obtain a training sample set S1 for extracting spectral features and carrying out second preprocessing to obtain a training sample set S2 for extracting spatial features;
a first feature extraction module, configured to extract spectral features from the training sample set S1 and spatial features from the training sample set S2 through a multi-scale spatial-spectral joint network;
the second characteristic extraction module is used for connecting the extracted spectral characteristics with the multi-scale spatial characteristics and inputting the connected spectral characteristics and the multi-scale spatial characteristics into a deformable convolution neural network capable of adapting to geometric deformation so as to extract high-level characteristics; and
and the full-connection network classifier is used for obtaining a hyperspectral image space spectrum combined classification result based on the deformable convolution neural network according to the input high-level features extracted by the deformable convolution network.
According to another embodiment of the invention, there is also provided a hyperspectral image spatial spectrum joint classification device based on a deformable convolutional neural network, which includes: a memory having computer instructions stored therein; and the processor is in data connection with the memory and executes the computer instructions so as to execute the hyperspectral image space spectrum joint classification method based on the deformable convolutional neural network.
The above detailed description is made on the hyperspectral image spatial spectrum joint classification method based on the deformable convolutional neural network provided by the present invention, but it is obvious that the scope of the present invention is not limited thereto. Various modifications of the invention are within the scope of the invention without departing from the scope of protection as defined in the appended claims.
It will be evident to those skilled in the art that the embodiments of the present invention are not limited to the details of the foregoing illustrative embodiments, and that the embodiments of the present invention are capable of being embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the embodiments being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Several units, modules or means recited in the system, apparatus or terminal claims may also be implemented by one and the same unit, module or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the embodiments of the present invention and not for limiting, and although the embodiments of the present invention are described in detail with reference to the above preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions can be made on the technical solutions of the embodiments of the present invention without departing from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A hyperspectral image space spectrum joint classification method is characterized by comprising the following steps:
normalizing the input hyperspectral image data;
respectively carrying out first preprocessing on the hyperspectral images after the normalization processing to obtain a training sample set S for extracting spectral features1And performing a second pre-processing to obtain a training sample set S for extracting spatial features2；
Using a multi-scale spatial-spectral joint network to extract the training sample set S1To extract spectral features and from the training sample set S2Extracting spatial features;
connecting the extracted spectral features and multi-scale spatial features, and inputting the spectral features and the multi-scale spatial features into a deformable convolutional neural network capable of adapting to geometric deformation to extract high-level features; and
and inputting the extracted high-level features into a full-connection network classifier through a variable convolution network for classification so as to obtain a hyperspectral image space spectrum joint classification result based on a variable convolution neural network.
2. The hyperspectral image space spectrum joint classification method according to claim 1, wherein the hyperspectral image after the normalization processing is subjected to a first preprocessing to obtain a training sample set S for extracting spectral features1Comprises the following steps:
performing intra-class clustering on the spectrum contained in each class of sample in the hyperspectral image by using a k-means clustering algorithm;
selecting a part of samples from each cluster to store labels thereof, wherein the labels are used as training sample labels GtrThe rest is used as a test sample label Gte；
Extracting with tape label GtrData blocks in 11 multiplied by L neighborhood range as a center are taken as a training sample set S for extracting spectral characteristics1。
3. The hyperspectral image space spectrum joint classification method according to claim 2, wherein the training sample set S subjected to the second preprocessing is used for extracting spatial features2Comprises the following steps: performing dimensionality reduction on the original hyperspectral image by using a Principal Component Analysis (PCA) method, and extracting a labeled G from the dimensionality-reduced imagetrData blocks in 11 multiplied by D neighborhood range as a central training sample set S for extracting spatial features2。
4. The hyperspectral image space spectrum joint classification method according to any one of claims 1 to 3, wherein the multi-scale space-spectrum joint network comprises a spectral feature extraction network and a spatial feature extraction network;
the spectral feature extraction network structure comprises a 1 × 1 convolution layer, a Normalization layer Batch Normalization and an excitation function ReLU;
the spatial feature extraction network structure comprises a 3 × 3 convolutional layer and a 5 × 5 convolutional layer which are connected in parallel, and a Batch Normalization layer and an excitation function ReLU are respectively normalized after the convolutional layers;
wherein, training sample set S1Inputting the data into a spectral feature extraction network to extract spectral features, training a sample set S2And inputting the spatial features into a spatial feature extraction network to obtain multi-scale spatial features.
5. The hyperspectral image spatial spectrum joint classification method according to claim 1, wherein the deformable convolutional neural network comprises a first two-dimensional convolutional layer, a first deformable convolutional layer, a second two-dimensional convolutional layer and a global average pooling layer which are connected in sequence.
6. The hyperspectral image spatial spectrum joint classification method according to claim 5, wherein the convolution kernel size of each two-dimensional convolution layer and each deformable convolution layer is 3 x 3, and the step length is 1.
7. The hyperspectral image space spectrum joint classification method according to claim 5 is characterized in that the number of convolution kernels sequentially is as follows: 64, 128, 256, 512.
8. The hyperspectral image spatial spectrum joint classification method according to claim 5, wherein each two-dimensional convolution layer and each deformable convolution layer are normalized by a Batch Normalization layer, and ReLU is used as an excitation function.
9. A hyperspectral image and space spectrum combined classification device is characterized by comprising:
a memory having computer instructions stored therein;
a processor in data connection with a memory, which executes the computer instructions to perform the method for spatio-spectral joint classification of hyperspectral images based on a deformable convolutional neural network according to any of claims 1 to 8.
10. A hyperspectral image and space spectrum combined classification device is characterized by comprising:
the normalization processing module is used for performing normalization processing on the input hyperspectral image data;
the training sample set generating module is used for respectively carrying out first preprocessing on the hyperspectral images after the normalization processing to obtain a training sample set S1 for extracting spectral features and carrying out second preprocessing to obtain a training sample set S2 for extracting spatial features;
a first feature extraction module, configured to extract spectral features from the training sample set S1 and spatial features from the training sample set S2 through a multi-scale spatial-spectral joint network;
the second characteristic extraction module is used for connecting the extracted spectral characteristics with the multi-scale spatial characteristics and inputting the connected spectral characteristics and the multi-scale spatial characteristics into a deformable convolution neural network capable of adapting to geometric deformation so as to extract high-level characteristics; and
and the full-connection network classifier is used for obtaining a hyperspectral image space spectrum combined classification result based on the deformable convolution neural network according to the input high-level features extracted by the deformable convolution network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911403750.2A CN111160273B (en) | 2019-12-31 | 2019-12-31 | Hyperspectral image spatial spectrum joint classification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911403750.2A CN111160273B (en) | 2019-12-31 | 2019-12-31 | Hyperspectral image spatial spectrum joint classification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111160273A true CN111160273A (en) | 2020-05-15 |
| CN111160273B CN111160273B (en) | 2023-05-09 |
Family
ID=70559954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911403750.2A Active CN111160273B (en) | 2019-12-31 | 2019-12-31 | Hyperspectral image spatial spectrum joint classification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111160273B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739001A (en) * | 2020-06-17 | 2020-10-02 | 东华大学 | Product surface defect detection model and detection method based on deformable convolution |
| CN111881797A (en) * | 2020-07-20 | 2020-11-03 | 北京理工大学 | Method, device, equipment and storage medium for finely classifying vegetation on coastal wetland |
| CN112634137A (en) * | 2020-12-28 | 2021-04-09 | 西安电子科技大学 | Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features |
| CN112699856A (en) * | 2021-03-24 | 2021-04-23 | 成都新希望金融信息有限公司 | Face ornament identification method and device, electronic equipment and storage medium |
| CN112818794A (en) * | 2021-01-25 | 2021-05-18 | 哈尔滨工业大学 | Hyperspectral remote sensing image generation method based on progressive space-spectrum combined depth network |
| CN113111834A (en) * | 2021-04-23 | 2021-07-13 | 大连海事大学 | Attention-based global local spectral weight waveband selection method |
| CN113435265A (en) * | 2021-06-09 | 2021-09-24 | 深圳大学 | Hyperspectral image classification method and device, electronic equipment and storage medium |
| CN113723469A (en) * | 2021-08-09 | 2021-11-30 | 中国人民解放军军事科学院国防工程研究院 | Interpretable hyperspectral image classification method and device based on space-spectrum combined network |
| CN114842264A (en) * | 2022-05-17 | 2022-08-02 | 北京理工大学 | Hyperspectral image classification method based on multi-scale spatial spectral feature joint learning |
| WO2023125456A1 (en) * | 2021-12-28 | 2023-07-06 | 苏州大学 | Multi-level variational autoencoder-based hyperspectral image feature extraction method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845381A (en) * | 2017-01-16 | 2017-06-13 | 西北工业大学 | Sky based on binary channels convolutional neural networks composes united hyperspectral image classification method |
| CN106997380A (en) * | 2017-03-21 | 2017-08-01 | 北京工业大学 | Imaging spectrum safe retrieving method based on DCGAN depth networks |
| CN108537238A (en) * | 2018-04-13 | 2018-09-14 | 崔植源 | A kind of classification of remote-sensing images and search method |
| CN108985360A (en) * | 2018-06-29 | 2018-12-11 | 西安电子科技大学 | Hyperspectral classification method based on expanding morphology and Active Learning |
| CN110009015A (en) * | 2019-03-25 | 2019-07-12 | 西北工业大学 | EO-1 hyperion small sample classification method based on lightweight network and semi-supervised clustering |
| RU2695963C1 (en) * | 2018-11-28 | 2019-07-29 | Владислав Анатольевич Третьяков | Method of recognizing images of biochemical ground objects with fine spectral differences based on hyperspectral data |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
-
2019
- 2019-12-31 CN CN201911403750.2A patent/CN111160273B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845381A (en) * | 2017-01-16 | 2017-06-13 | 西北工业大学 | Sky based on binary channels convolutional neural networks composes united hyperspectral image classification method |
| CN106997380A (en) * | 2017-03-21 | 2017-08-01 | 北京工业大学 | Imaging spectrum safe retrieving method based on DCGAN depth networks |
| CN108537238A (en) * | 2018-04-13 | 2018-09-14 | 崔植源 | A kind of classification of remote-sensing images and search method |
| CN108985360A (en) * | 2018-06-29 | 2018-12-11 | 西安电子科技大学 | Hyperspectral classification method based on expanding morphology and Active Learning |
| RU2695963C1 (en) * | 2018-11-28 | 2019-07-29 | Владислав Анатольевич Третьяков | Method of recognizing images of biochemical ground objects with fine spectral differences based on hyperspectral data |
| CN110009015A (en) * | 2019-03-25 | 2019-07-12 | 西北工业大学 | EO-1 hyperion small sample classification method based on lightweight network and semi-supervised clustering |
| CN110321963A (en) * | 2019-07-09 | 2019-10-11 | 西安电子科技大学 | Based on the hyperspectral image classification method for merging multiple dimensioned multidimensional sky spectrum signature |
Non-Patent Citations (4)
| Title |
|---|
| AMIT MUKHERJEE等: "Interest Points for Hyperspectral Image Data" * |
| 付琼莹;余旭初;张鹏强;薛志祥;: "结合极限学习机的高光谱影像聚类算法" * |
| 张文豪: "基于特征学习和深度学习的高光谱影像分类" * |
| 张琳;: "基于a-BvSBEM主动学习的高光谱图像分类" * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111739001A (en) * | 2020-06-17 | 2020-10-02 | 东华大学 | Product surface defect detection model and detection method based on deformable convolution |
| CN111881797A (en) * | 2020-07-20 | 2020-11-03 | 北京理工大学 | Method, device, equipment and storage medium for finely classifying vegetation on coastal wetland |
| CN112634137A (en) * | 2020-12-28 | 2021-04-09 | 西安电子科技大学 | Hyperspectral and full-color image fusion method based on AE extraction of multi-scale spatial spectrum features |
| CN112634137B (en) * | 2020-12-28 | 2024-03-05 | 西安电子科技大学 | Hyperspectral and panchromatic image fusion method for extracting multiscale spatial spectrum features based on AE |
| CN112818794A (en) * | 2021-01-25 | 2021-05-18 | 哈尔滨工业大学 | Hyperspectral remote sensing image generation method based on progressive space-spectrum combined depth network |
| CN112699856A (en) * | 2021-03-24 | 2021-04-23 | 成都新希望金融信息有限公司 | Face ornament identification method and device, electronic equipment and storage medium |
| CN113111834A (en) * | 2021-04-23 | 2021-07-13 | 大连海事大学 | Attention-based global local spectral weight waveband selection method |
| CN113111834B (en) * | 2021-04-23 | 2024-04-05 | 大连海事大学 | Global local spectrum weight band selection method based on attention |
| CN113435265A (en) * | 2021-06-09 | 2021-09-24 | 深圳大学 | Hyperspectral image classification method and device, electronic equipment and storage medium |
| CN113723469A (en) * | 2021-08-09 | 2021-11-30 | 中国人民解放军军事科学院国防工程研究院 | Interpretable hyperspectral image classification method and device based on space-spectrum combined network |
| WO2023125456A1 (en) * | 2021-12-28 | 2023-07-06 | 苏州大学 | Multi-level variational autoencoder-based hyperspectral image feature extraction method |
| CN114842264A (en) * | 2022-05-17 | 2022-08-02 | 北京理工大学 | Hyperspectral image classification method based on multi-scale spatial spectral feature joint learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111160273B (en) | 2023-05-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111160273B (en) | Hyperspectral image spatial spectrum joint classification method and device | |
| Xiong et al. | Material based object tracking in hyperspectral videos | |
| Wu et al. | ORSIm detector: A novel object detection framework in optical remote sensing imagery using spatial-frequency channel features | |
| Jia et al. | Collaborative representation-based multiscale superpixel fusion for hyperspectral image classification | |
| Xie et al. | Hyperspectral image super-resolution using deep feature matrix factorization | |
| Jia et al. | Gabor feature-based collaborative representation for hyperspectral imagery classification | |
| Rao et al. | A residual convolutional neural network for pan-shaprening | |
| CN108009559B (en) | Hyperspectral data classification method based on space-spectrum combined information | |
| Sirmacek et al. | Urban-area and building detection using SIFT keypoints and graph theory | |
| US6674915B1 (en) | Descriptors adjustment when using steerable pyramid to extract features for content based search | |
| CN107145836B (en) | Hyperspectral image classification method based on stacked boundary identification self-encoder | |
| CN109615008B (en) | Hyperspectral image classification method and system based on stack width learning | |
| CN107239759B (en) | High-spatial-resolution remote sensing image transfer learning method based on depth features | |
| Su et al. | A saliency-based band selection approach for hyperspectral imagery inspired by scale selection | |
| Jia et al. | Flexible Gabor-based superpixel-level unsupervised LDA for hyperspectral image classification | |
| CN112580480B (en) | Hyperspectral remote sensing image classification method and device | |
| CN113705641B (en) | Hyperspectral image classification method based on rich context network | |
| CN109034213B (en) | Hyperspectral image classification method and system based on correlation entropy principle | |
| Fang et al. | Aerial-BiSeNet: A real-time semantic segmentation network for high resolution aerial imagery | |
| CN115205590A (en) | Hyperspectral image classification method based on complementary integration Transformer network | |
| Liu et al. | Region-based relaxed multiple kernel collaborative representation for hyperspectral image classification | |
| CN111639697B (en) | Hyperspectral image classification method based on non-repeated sampling and prototype network | |
| CN107203779A (en) | The EO-1 hyperion dimension reduction method kept based on empty spectrum information | |
| Nyasaka et al. | Learning hyperspectral feature extraction and classification with resnext network | |
| Patil et al. | Expression invariant face recognition using semidecimated DWT, Patch-LDSMT, feature and score level fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |