CN113822207B - Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium - Google Patents
Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113822207B CN113822207B CN202111137984.4A CN202111137984A CN113822207B CN 113822207 B CN113822207 B CN 113822207B CN 202111137984 A CN202111137984 A CN 202111137984A CN 113822207 B CN113822207 B CN 113822207B
- Authority
- CN
- China
- Prior art keywords
- convolution
- channel
- layer
- module
- remote sensing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 86
- 238000001228 spectrum Methods 0.000 claims abstract description 116
- 238000012549 training Methods 0.000 claims abstract description 29
- 230000008569 process Effects 0.000 claims abstract description 24
- 230000009467 reduction Effects 0.000 claims abstract description 23
- 230000007246 mechanism Effects 0.000 claims abstract description 22
- 239000010410 layer Substances 0.000 claims description 143
- 230000006870 function Effects 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 21
- 230000004913 activation Effects 0.000 claims description 19
- 238000011176 pooling Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 239000002356 single layer Substances 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 13
- 230000003595 spectral effect Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000001965 increasing effect Effects 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 230000017105 transposition Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a hyperspectral remote sensing image identification method, a hyperspectral remote sensing image identification device, electronic equipment and a readable storage medium. The method comprises the steps of training an image recognition model in advance, and inputting the hyperspectral remote sensing image to be recognized into the image recognition model to obtain an image recognition result. The network input combination module of the image recognition model combines the neighborhood characteristics and spectrum dimensions of the input image, and selects a network input image block; the multi-scale bi-pass convolution module carries out fixed kernel convolution and multi-scale cavity group convolution operation on the network input image block by using the first channel and the second channel respectively; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation, so that the recognition precision of a hyperspectral image is effectively improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a method and apparatus for identifying hyperspectral remote sensing images, an electronic device, and a readable storage medium.
Background
With the development of hyperspectral sensors, the spectrum information of remote sensing images is more and more abundant, and the remote sensing technology enters a brand new stage by virtue of a plurality of fingerprint characteristics. The hyperspectral image has spectrum sampling interval up to nanometer level, and the obtainable pixel has hundred-band spectrum information, and is widely applied to the fields of ground object classification and identification, quantitative inversion, target detection, ecological environment monitoring and the like. In the field of ground object classification, the hyperspectral image classification is to assign each pixel to a unique class label according to the spectral features and the spatial features contained in the images of the hyperspectral image classification, which is the most active research field in the hyperspectral remote sensing technology at present and brings about wide attention to the remote sensing field. The traditional hyperspectral image recognition method classifies based on spectrum information, and the input information is one-dimensional information. For example: support vector machine, classification method of random forest, K nearest neighbor, etc. In recent years, spatial features and spectral features are increasingly applied to hyperspectral image recognition classification.
With the successful application of deep learning in various fields, those skilled in the art introduce deep learning into hyperspectral image recognition and achieve good classification performance. Such as the deep boltzmann machine, noise reduction self-coding, and deep convolutional neural networks. The deep convolution neural network can automatically extract effective space and high-dimensional characteristics of spectrum, and common convolution neural networks comprise one-dimensional convolution, two-dimensional convolution and three-dimensional convolution. Convolutional networks are generally operated on the basis of blocks (patch), and in hyperspectral remote sensing image classification, the size setting of the blocks can cause distortion of classification results. The problems of gradient disappearance and the like of a network model are caused by poor capability of space perception details and excessive increase of network depth, and meanwhile, the model training is insufficient and the recognition precision is low due to complex network structure and limited model training samples.
Disclosure of Invention
The application provides a hyperspectral remote sensing image identification method, a hyperspectral remote sensing image identification device, electronic equipment and a readable storage medium, and the identification accuracy of hyperspectral remote sensing images is effectively improved.
In order to solve the technical problems, the embodiment of the invention provides the following technical scheme:
In one aspect, the embodiment of the invention provides a hyperspectral remote sensing image identification method, which comprises the following steps:
pre-training an image recognition model; the image recognition model comprises a network input combination module, a multi-scale double-pass convolution module, a convolution characteristic mixing module and a compressed expansion spatial spectrum attention mechanism module;
inputting the hyperspectral remote sensing image to be identified into the image identification model to obtain an image identification result of the hyperspectral remote sensing image to be identified;
the network input combination module combines the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified, and selects a network input image block from the combination result; the multi-scale bi-pass convolution module carries out fixed kernel convolution operation on the network input image block by using a first channel, and carries out multi-scale cavity group convolution processing on the network input image block by using a second channel; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation.
Optionally, the multi-scale double-pass convolution module is formed by a first convolution layer and a second convolution layer in a double-channel mode;
the convolution kernel of the first convolution layer is fixed in size; the second convolution layer is formed by at least three layers of convolutions in a serial manner.
Optionally, the second convolution layer includes a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
Optionally, the mixing the multi-channel features in different convolution processes includes:
acquiring first channel characteristic information output by the first channel and second channel characteristic information output by the second channel;
combining the first channel characteristic information and the second channel characteristic information according to the channel dimension;
equally dividing the channels of the merging result into two groups, and arranging the groups according to the equipotential sequence; the co-located channels are extracted and reordered.
Optionally, the process of learning the target features in the spectrum dimension and the space dimension respectively, adaptively adjusting the spatial spectrum feature map, and conveying the finally obtained target spatial spectrum feature map to the discrimination convolution layer and the full connection layer for performing the dimension reduction operation includes:
Performing channel dimension operation on the space characteristics by using a neural network formed by a global pooling layer and a multi-layer perceptron, inputting the space characteristics to the full-connection layer through a RELU activation function layer, and outputting the space characteristics through a sigmoid activation function layer;
the spatial spectrum features are subjected to channel dimension pooling treatment to obtain a single channel, and spatial attention is generated through single-layer convolution operation and sigmoid activation function layer treatment;
determining a spatial spectrum attention according to the spatial attention and the channel attention;
and obtaining a channel dimension scaling characteristic according to the spatial spectrum attention and the spatial spectrum characteristic.
In another aspect, an embodiment of the present invention provides a hyperspectral remote sensing image recognition device, including:
the model pre-training module is used for pre-training the image recognition model; the image recognition model comprises a network input combination module, a multi-scale double-pass convolution module, a convolution characteristic mixing module and a compressed expansion spatial spectrum attention mechanism module;
the image recognition module is used for inputting the hyperspectral remote sensing image to be recognized into the image recognition model to obtain an image recognition result of the hyperspectral remote sensing image to be recognized; the network input combination module combines the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified, and selects a network input image block from the combination result; the multi-scale bi-pass convolution module carries out fixed kernel convolution operation on the network input image block by using a first channel, and carries out multi-scale cavity group convolution processing on the network input image block by using a second channel; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation.
Optionally, the multi-scale double-pass convolution module is formed by a first convolution layer and a second convolution layer in a double-channel mode;
the convolution kernel of the first convolution layer is fixed in size; the second convolution layer is formed by at least three layers of convolutions in a serial manner.
Optionally, the second convolution layer includes a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
The embodiment of the invention also provides electronic equipment, which comprises a processor, wherein the processor is used for realizing the steps of the hyperspectral remote sensing image identification method according to any one of the previous claims when executing the computer program stored in the memory.
The embodiment of the invention finally provides a readable storage medium, wherein a computer program is stored on the readable storage medium, and the computer program realizes the steps of the hyperspectral remote sensing image identification method according to any one of the previous claims when being executed by a processor.
The technical scheme provided by the application has the advantages that the spatial spectrum neighborhood data are adopted to combine network input, and the network input is input into the double-channel extracted image block formed by the fixed kernel convolution layer and the series-connection variant convolution layer to perform multi-scale feature separation type learning. When the spatial spectrum features are extracted by different scale cavity convolution combination modes, the receptive field of the convolution kernel can be improved simultaneously under the conditions of not increasing the number of network parameters and not reducing the data features, so that the problem of poor space detail perception capability is effectively solved, deep spatial spectrum features with stronger expression capability are extracted, and the image recognition accuracy is further improved. The method has the advantages that the method is combined with a channel space attention mechanism efficiently, the expressive force of specific image features is enhanced from two aspects of channels and space domains, more representative features are learned, and the recognition accuracy of hyperspectral remote sensing images is further improved. Furthermore, the whole image recognition model has a simple structure and high data processing speed, is beneficial to improving the image recognition and classification efficiency, and is beneficial to further promoting the wide application of hyperspectral image classification based on a deep learning method.
In addition, the embodiment of the invention also provides a corresponding implementation device, electronic equipment and a readable storage medium for the hyperspectral remote sensing image identification method, so that the method has more practicability, and the device, the electronic equipment and the readable storage medium have corresponding advantages.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the related art, the drawings that are required to be used in the embodiments or the description of the related art will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
Fig. 1 is a schematic flow chart of a hyperspectral remote sensing image recognition method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of another method for identifying hyperspectral remote sensing images according to the embodiment of the present invention;
fig. 3 is a schematic structural diagram of a network input combination module in an exemplary application scenario according to an embodiment of the present invention;
Fig. 4 is a schematic structural diagram of a multi-scale two-way convolution module according to an embodiment of the present invention in an exemplary application scenario;
FIG. 5 is a schematic structural diagram of a convolution feature mixing module according to an embodiment of the present disclosure in an exemplary application scenario;
fig. 6 is a schematic structural diagram of a compressed and expanded spatial spectrum attention mechanism module in an exemplary application scenario according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of an image recognition model in an exemplary application scenario according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of one illustrative example of an Indian pins dataset provided by an embodiment of the invention;
fig. 9 is a real ground object diagram corresponding to fig. 8 provided by the embodiment of the present invention;
FIG. 10 is a block diagram of a hyperspectral image recognition device according to an embodiment of the present invention;
fig. 11 is a block diagram of an embodiment of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of this application and in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.
Having described the technical solutions of embodiments of the present invention, various non-limiting implementations of the present application are described in detail below.
Referring first to fig. 1, fig. 1 is a schematic flow chart of a hyperspectral remote sensing image recognition method according to an embodiment of the present invention, where the embodiment of the present invention may include the following:
s101: the image recognition model is trained in advance.
The image recognition model of the embodiment can comprise a network input combination module, a multi-scale double-pass convolution module, a convolution feature mixing module and a compressed expansion spatial spectrum attention mechanism module. The network input combination module is a network input organization stage, and is based on an input training set data block, and firstly carries out overall dimension reduction, and carries out spectrum dimension reduction by utilizing a random neighborhood embedding method, wherein the dimension reduction is carried out to the spectrum dimension to obtain initial data, namely, the pixel neighborhood characteristics and the spectrum dimension after dimension reduction are combined to form a training basic block. Specifically, after the neighborhood characteristics and spectrum dimensions of the input image are combined with each other, a network input image block is selected through a sliding block window. The multi-scale bi-pass convolution module is designed into a dual-channel network input mode, one channel is subjected to fixed-core convolution, and the other channel is subjected to multi-scale cavity group convolution processing so as to learn multi-scale spatial spectrum characteristic expression. The spatial spectrum features of the embodiment refer to the spatial features and the spectral features of the input remote sensing image. The multi-scale bi-pass convolution module takes an input training set data block as a training unit, learns the spatial spectrum feature basic expression by using a fixed kernel convolution operation, and realizes the multi-scale spatial spectrum feature expression by using a serial convolution operation with different void ratios. The convolution feature mixing module is a feature combination mode, the repeated mixing process is designed, the multi-channel features of different convolution processes are mixed, namely deep features of double-channel learning are combined, corresponding positions of outputs of two channels are aligned by using a tensor transposition method, features at the same positions are extracted and combined, and then the combined new spatial spectrum feature tensor is transposed back to an initial shape, so that the convolution expression feature learning capacity of each stage is enhanced. For example, the multi-scale bi-pass convolution module may be formed by a 2-dimensional fixed-kernel convolution layer and a 2-dimensional 3-layer serial-variant convolution layer in a two-channel manner, and the convolution feature mixing module may form new combination features by combining channel dimensions of two branches, mix and combine deep features learned by two-channel, iterate three times by combining separate convolutions and convolutions, and finally output separate features. The compression expansion spatial spectrum attention mechanism module finishes importance weight learning and spatial attention of the channel, learns meaningful features in two dimensions of a spectrum dimension and a spatial dimension, sequentially deduces attention weight, and self-adaptively adjusts a spatial spectrum feature map to obtain a final spatial spectrum feature map. The model training process may include: as shown in fig. 2, first, the acquired original hyperspectral image sample data are normalized and data-divided, and these sample data are divided into a training set, a verification set, and a test set. Wherein, can randomly divide out a small amount of label data as training set and verification set from hyperspectral image, remaining label data is as test set, and the proportion of training set and test set can be set to 2:1:7. and training the image-based recognition model by using a small amount of label data, and finally carrying out classification recognition by combining the trained image recognition model. In a deep learning spatial spectrum combined network, input data is firstly subjected to dimension reduction based on a random neighborhood embedding (t-Distributed Stochastic Neighbor Embedding) method to form a new combination characteristic, and the new combination characteristic is transmitted to a multi-scale bi-pass convolution module to obtain a feature map for spatial spectrum feature extraction; and then, performing high-order feature mixing through a convolution feature mixing module to obtain a final spatial feature map, and finally, correcting the feature map through a compressed and expanded spatial spectrum attention mechanism module. Of course, the image recognition model of the present embodiment may inevitably further include a classification recognition module, where the classification recognition module is formed by a full-connection layer performing the dimension reduction operation and a classification function, for example, the classification function may be a softmax classification function, and the classification recognition module splices the final spatial feature map and the spectral feature map, fuses all feature information by using full connection, obtains the final spectral spatial feature, and transmits the final spectral spatial feature to the softmax function for target classification. Wherein the number of softmax function units is the total number of the corresponding categories. That is, the spatial spectrum features extracted by each network module of the network model are deeply transmitted to the full-connection layer, and the full-connection layer fuses all the features to 1 dimension and then transmits to the softmax function for classification.
S102: and inputting the hyperspectral remote sensing image to be identified into an image identification model to obtain an image identification result of the hyperspectral remote sensing image to be identified.
In order to improve the image recognition accuracy, the acquired hyperspectral remote sensing image to be recognized can be preprocessed, such as denoising, smoothing and the like. And then inputting the hyperspectral remote sensing image to be identified into an image identification model obtained by training in the previous step, combining the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified by a network input combination module of the image identification model, selecting network input image blocks from the combination result, inputting the network input image blocks into a multi-scale bi-pass convolution module, performing fixed kernel convolution operation on the network input image blocks by the multi-scale bi-pass convolution module by using a first channel, and performing multi-scale cavity group convolution processing on the network input image blocks by using a second channel. The convolution feature mixing module mixes the multi-channel features of the multi-scale double-pass convolution module in different convolution processes. The compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation, classifies the target spatial spectrum feature map according to a classification recognition function such as a softmax function of an image recognition model, and outputs a classification prediction corresponding recognition result, for example, a classification result map can be obtained.
In the technical scheme provided by the embodiment of the invention, the network input is combined by adopting the spatial spectrum neighborhood data, and the network input is input into a double-channel extracted image block formed by a fixed kernel convolution layer and a series-connection variant convolution layer for multi-scale feature separation type learning. When the spatial spectrum features are extracted by different scale cavity convolution combination modes, the receptive field of the convolution kernel can be improved simultaneously under the conditions of not increasing the number of network parameters and not reducing the data features, so that the problem of poor space detail perception capability is effectively solved, deep spatial spectrum features with stronger expression capability are extracted, and the image recognition accuracy is further improved. The method has the advantages that the method is combined with a channel space attention mechanism efficiently, the expressive force of specific image features is enhanced from two aspects of channels and space domains, more representative features are learned, and the recognition accuracy of hyperspectral remote sensing images is further improved. Furthermore, the whole image recognition model has a simple structure and high data processing speed, is beneficial to improving the image recognition and classification efficiency, and is beneficial to further promoting the wide application of hyperspectral image classification based on a deep learning method.
It should be noted that, in the present application, the steps may be executed simultaneously or in a certain preset order as long as the steps conform to the logic order, and fig. 1-2 are only schematic, and do not represent only such an execution order.
In the above embodiment, the structure of the network input combination module is not limited, and in this embodiment, an optional processing manner of the input data by the network input combination module is provided, which may include the following steps:
the network input combination module is constructed by taking a single image block of network input combination as an input unit, and as shown in fig. 3, the process of combining the initial spatial spectrum characteristic network input basic blocks by the network input combination module is as follows:
the method comprises the steps of taking a hyperspectral image block as an input unit, performing overall dimension reduction by a network input combination module, performing spectrum dimension reduction by using a random neighborhood embedding method, extracting the first 3 dimensions of each pixel after dimension reduction, merging the dimension reduction of each pixel in a given field into a one-dimensional vector on the basis, and adding the one-dimensional vector to the spectrum dimension to form a new combination characteristic. And simultaneously, selecting a neighborhood data block with a fixed size taking each pixel as a center as a network input block for training and prediction. If the hyperspectral image is referred to as HSI (X, Y, S), where X is the image length, Y is the image width, and S is the image spectral dimension. And (3) reducing the dimension of the HSI by using a random neighborhood embedding method to obtain HSI_down (X, Y, S_down), wherein S_down takes the first 3 dimensions after the dimension reduction. And carrying out sliding window processing on the whole image, merging all dimensionality reduction features of each pixel around the central neighborhood into a 1-dimensional vector, and splicing the 1-dimensional vector to the corresponding pixel spectrum feature vector to form HSI_new (X, Y, S_new), wherein the dimension of S_new is the synthesized feature dimension. Window clipping is carried out on HSI_new (X, Y, S_new) according to step length 1, and the window clipping is input into a network.
In the foregoing embodiment, the structure of the multi-scale bi-pass convolution module is not limited, and an alternative structural implementation of the multi-scale bi-pass convolution module is provided in this embodiment, referring to fig. 4, which may include:
the multi-scale double-pass convolution module can be formed by a first convolution layer and a second convolution layer in a double-channel mode; the convolution kernel size of the first convolution layer is fixed; the second convolution layer is formed by at least three convolutions in a series fashion. The second convolution layer comprises a first sub-convolution layer, a second sub-convolution layer and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
In this embodiment, a network input image block of a hyperspectral image to be identified is taken as an input unit, the multi-scale bi-pass convolution module is formed by a 2-dimensional fixed-core convolution layer and a 2-dimensional serial-variant convolution layer in a dual-channel mode, the fixed-core convolution layer is formed by convolution kernels with the same size, the serial-variant convolution layer is formed by combining at least three layers of convolution in a serial mode, wherein the convolution kernels of the three layers of convolution layers are consistent in size, and incremental cavity data rate is adopted, for example, the cavity convolution rate of a first layer is 1, the cavity convolution rate of a second layer is 3, and the cavity convolution rate of a third layer is 5. Based on the structure shown in fig. 4, the process of extracting the spatial spectrum features by the multi-scale bi-pass convolution module may be as follows:
(1) The first partial convolution layer of the parallel layers is a 2-dimensional fixed-kernel convolution layer.
(2) The second part of the parallel layers is a 2-dimensional serial variable convolution layer, at least three convolution extraction layers with holes are set, the convolution kernel sizes of all the layers are kept consistent, and the hole rate is increased in sequence.
(3) And each layer of network in the convolution module is added with a BN layer, meanwhile, a ReLU activation function is adopted, and the network adopts a padding mode with equal input and output sizes.
The embodiment is not limited to how to mix the multi-channel features in different convolution processes, and this embodiment also provides an alternative implementation, which may include:
acquiring first channel characteristic information output by a first channel and second channel characteristic information output by a second channel; combining the first channel characteristic information and the second channel characteristic information according to the channel dimension; equally dividing the channels of the merging result into two groups, and arranging the groups according to the equipotential sequence; the co-located channels are extracted and reordered.
In this embodiment, the functional module for implementing mixing of the multi-channel features in different convolution processes is a convolution feature mixing module, and the working process of the convolution feature mixing module is as follows in combination with the illustration of fig. 5: the method comprises the steps of forming new combined features by combining channel dimensions of two branches, combining deep features of two-channel learning, carrying out corresponding position alignment on outputs of the two channels by using a tensor transposition method, extracting and combining features at the same position, reversely transposed the combined new empty spectrum feature tensor back to an initial shape, enhancing the learning capacity of network features, selecting 1/2 of each time for mixing a convolution layer, and realizing full mixing of the empty spectrum features by a plurality of repeated modes. Based on the structure of the convolution layer of the convolution feature mixing module shown in fig. 5, the process of extracting the spatial spectrum feature map by the convolution feature mixing module can be as follows:
The characteristics of the multi-scale two-way convolution module output may be referred to as: f1 And (c, x, y) and F2 (c, x, y), performing feature dimension transposition on the two channel feature outputs to form a new tensor F_t (2, c, x, y), combining in the first dimension to form a tensor F_t (c, 2, x, y), finally combining the 2 nd dimension of the tensor to generate a final mixed feature tensor F (2 c, x, y), and conveying the F (2 c, x, y) to a subsequent compressed and expanded spatial spectrum attention mechanism module for processing by a convolution feature mixing module.
The above-mentioned method how to learn the target features in the spectrum dimension and the space dimension respectively and adaptively adjust the spatial spectrum feature map is not limited, and this application further provides an optional implementation manner, which may include:
performing channel dimension operation on the space characteristics by using a neural network formed by a global pooling layer and a multi-layer perceptron, inputting the space characteristics to a full-connection layer through a RELU activation function layer, and outputting the space characteristics through a sigmoid activation function layer; the spatial spectrum characteristics are subjected to channel dimension pooling treatment to obtain a single channel, and the single channel is subjected to single-layer convolution operation and sigmoid activation function layer treatment to generate spatial attention; determining spatial spectrum attention according to the spatial attention and the channel attention; and obtaining a channel dimension scaling characteristic according to the spatial spectrum attention and the spatial spectrum characteristic.
In this embodiment, the procedure of adjusting the spatial spectrum feature map by the compressed and expanded spatial spectrum attention mechanism module is as follows: and taking the learned empty spectrum characteristic diagram as a unit, firstly performing channel dimension operation, inputting the operation into a fully connected network through a RELU activation function through a neural network formed by global pooling and multi-layer perceptrons, and then performing sigmoid activation output to complete importance weight learning of the channel. And simultaneously, aiming at the input spatial spectrum characteristics, obtaining a single channel through channel dimension pooling, generating spatial attention through single-layer convolution operation and sigmoid activation, multiplying the spatial attention with channel attention points to form spatial spectrum attention, and multiplying the result by the original input characteristics to obtain scaling characteristics of the new channel dimension. The compressed and expanded spatial spectrum attention mechanism module and the structure thereof can be shown in fig. 6, and the process based on the extraction of the spatial spectrum feature map is as follows:
(1) And taking the mixed spatial spectrum characteristic diagram as a unit, firstly performing channel dimension operation, inputting the channel dimension operation into a fully connected network through a RELU activation function through a neural network formed by global pooling and a multi-layer perceptron through a global average pool, and then performing sigmoid activation output to complete importance weight learning of the channel. (2) And simultaneously, taking the mixed spatial spectrum feature map as a unit, carrying out global average pooling to obtain a two-dimensional spatial feature description map, generating spatial attention through single-layer convolution operation and sigmoid activation, multiplying the spatial attention with channel attention points to form spatial spectrum attention, and multiplying the result by original input features to obtain scaling features of new channel dimensions.
In order to verify the validity of the technical solution of the present application, the present application further provides a verification embodiment, which may include:
in this example, all training sample data were randomly divided into training data, validation data, and test data by 20%, 10%, and 70%. Based on the network structure shown in fig. 7 as an image recognition model, an Adam optimization method is adopted to optimize the image recognition model. The training number of the network is 1000 times, and the training number of each batch is set to be 60. The learning rate was set to 0.001. Training strategies that degrade the learning rate are also employed, with a 10% reduction in learning rate every 250 times. According to the parameters, training the network structure based on training sample data to obtain a trained image recognition model.
In order to verify that the image recognition model of the method can effectively improve the recognition accuracy of the hyperspectral remote sensing image, the image recognition model is used for classifying and recognizing the hyperspectral image of the real IndianPicnes, as shown in fig. 8 and 9, the image size is 145 rows by 145 columns by 224 wave bands, the corresponding water vapor wave bands and low signal to noise ratio wave bands are removed, and the remaining 200 available waves are classified. The technical scheme provided by the embodiment is adopted to identify and classify the hyperspectral remote sensing images according to the following steps:
First, for the indianpins hyperspectral dataset, 4 reference algorithms SVM, KNN, 1D-CNN, 2D-CNN were chosen for comparison in order to verify the validity of the algorithm.
Next, the classification result of each algorithm in the experiment was compared with the ground truth value, and the overall recognition accuracy (OverallAccuracy, OA), the average classification accuracy (AverageAccuracy, AA), and the Kappa coefficient were adopted as performance evaluation indexes. The calculation results are shown in Table 1:
table 1 identification performance contrast information of hyperspectral remote sensing image identification method
As can be seen from table 1, each performance evaluation index of the image recognition model provided by the technical scheme of the application is significantly higher than that of the other 4 reference algorithms.
As can be seen from the above, the present embodiment continuously learns the spatial spectrum features for the hyperspectral image database by adopting the dual-channel separation type multi-scale convolution and convolution mixed operation, constructs a spatial spectrum attention mechanism, enhances the representation capability of the features in the specific region, and learns more representative features when facing a large amount of redundant information. Compared with the prior art, the method has higher classification precision and capability, effectively solves the problems of poor space perception detail capability and network model performance degradation existing in the classification method of the convolutional neural network, and further improves the classification precision.
The embodiment of the invention also provides a corresponding device for the hyperspectral remote sensing image recognition method, so that the method has more practicability. Wherein the device may be described separately from the functional module and the hardware. The hyperspectral remote sensing image recognition device provided by the embodiment of the invention is introduced below, and the hyperspectral remote sensing image recognition device described below and the hyperspectral remote sensing image recognition method described above can be correspondingly referred to each other.
Based on the angles of the functional modules, referring to fig. 10, fig. 10 is a block diagram of a hyperspectral remote sensing image recognition device provided by an embodiment of the present invention under a specific implementation manner, where the device may include:
a model pre-training module 101 for pre-training an image recognition model; the image recognition model comprises a network input combination module, a multi-scale double-pass convolution module, a convolution characteristic mixing module and a compressed expansion spatial spectrum attention mechanism module.
The image recognition module 1002 is configured to input the hyperspectral remote sensing image to be recognized into an image recognition model, and obtain an image recognition result of the hyperspectral remote sensing image to be recognized. The network input combination module combines the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified, and selects a network input image block from the combination result; the multi-scale bi-pass convolution module carries out fixed kernel convolution operation on the network input image block by using a first channel, and carries out multi-scale cavity group convolution processing on the network input image block by using a second channel; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation.
Optionally, in some implementations of this embodiment, the multi-scale two-way convolution module is configured by a first convolution layer and a second convolution layer in a dual-channel form; the convolution kernel size of the first convolution layer is fixed; the second convolution layer is formed by at least three convolutions in a series fashion.
As an alternative implementation of this embodiment, the second convolution layer includes a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
Alternatively, in other implementations of the present embodiment, the image recognition module 1002 may be configured to: acquiring first channel characteristic information output by a first channel and second channel characteristic information output by a second channel; combining the first channel characteristic information and the second channel characteristic information according to the channel dimension; equally dividing the channels of the merging result into two groups, and arranging the groups according to the equipotential sequence; the co-located channels are extracted and reordered.
Optionally, in some other implementations of this embodiment, the image identifying module 1002 may be further configured to: performing channel dimension operation on the space characteristics by using a neural network formed by a global pooling layer and a multi-layer perceptron, inputting the space characteristics to a full-connection layer through a RELU activation function layer, and outputting the space characteristics through a sigmoid activation function layer; the spatial spectrum characteristics are subjected to channel dimension pooling treatment to obtain a single channel, and the single channel is subjected to single-layer convolution operation and sigmoid activation function layer treatment to generate spatial attention; determining spatial spectrum attention according to the spatial attention and the channel attention; and obtaining a channel dimension scaling characteristic according to the spatial spectrum attention and the spatial spectrum characteristic.
The functions of each functional module of the hyperspectral remote sensing image recognition device according to the embodiment of the present invention can be specifically implemented according to the method in the embodiment of the method, and the specific implementation process thereof can refer to the related description of the embodiment of the method, which is not repeated here.
From the above, the embodiment of the invention can effectively improve the recognition accuracy of the hyperspectral remote sensing image.
The hyperspectral remote sensing image recognition device is described from the perspective of a functional module, and further, the application also provides electronic equipment, which is described from the perspective of hardware. Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 11, the electronic device comprises a memory 110 for storing a computer program; a processor 111 for implementing the steps of the hyperspectral remote sensing image recognition method as mentioned in any of the embodiments above when executing a computer program.
Processor 111 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and processor 111 may also be a controller, microcontroller, microprocessor, or other data processing chip, among others. The processor 111 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 111 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 111 may be integrated with a GPU (Graphics Processing Unit, image processor) for taking care of rendering and drawing of content that the display screen is required to display. In some embodiments, the processor 111 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.
Memory 110 may include one or more computer-readable storage media, which may be non-transitory. Memory 110 may also include high-speed random access memory as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. Memory 110 may be an internal storage unit of an electronic device, such as a hard disk of a server, in some embodiments. The memory 110 may also be an external storage device of the electronic device, such as a plug-in hard disk provided on a server, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. in other embodiments. Further, the memory 110 may also include both internal storage units and external storage devices of the electronic device. The memory 110 may be used to store not only application software installed in an electronic device, but also various types of data, such as: code of a program that executes the vulnerability processing method, or the like, may also be used to temporarily store data that has been output or is to be output. In this embodiment, the memory 110 is at least configured to store a computer program 1101, where the computer program is loaded and executed by the processor 111 to implement the steps of the hyperspectral remote sensing image identification method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 110 may further include an operating system 1102, data 1103, and the like, and the storage manner may be transient storage or permanent storage. The operating system 1102 may include Windows, unix, linux, among other things. The data 1103 may include, but is not limited to, data corresponding to the identification result of the hyperspectral remote sensing image, and the like.
In some embodiments, the electronic device may further include a display 112, an input/output interface 113, a communication interface 114, or referred to as a network interface, a power supply 115, and a communication bus 116. Among other things, the display 112, an input output interface 113 such as a Keyboard (Keyboard) belongs to a user interface, which may alternatively include a standard wired interface, a wireless interface, etc. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface. The communication interface 114 may optionally include a wired interface and/or a wireless interface, such as a WI-FI interface, a bluetooth interface, etc., typically used to establish a communication connection between an electronic device and other electronic devices. The communication bus 116 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 11, but not only one bus or one type of bus.
Those skilled in the art will appreciate that the configuration shown in fig. 11 is not limiting of the electronic device and may include more or fewer components than shown, for example, may also include sensors 117 to perform various functions.
The functions of each functional module of the electronic device according to the embodiment of the present invention may be specifically implemented according to the method in the embodiment of the method, and the specific implementation process may refer to the related description of the embodiment of the method, which is not repeated herein.
From the above, the embodiment of the invention can effectively improve the recognition accuracy of the hyperspectral remote sensing image.
It will be appreciated that if the hyperspectral remote sensing image recognition method in the above embodiments is implemented in the form of a software functional unit and sold or used as a separate product, it may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution contributing to the prior art, or in a software product stored in a storage medium, performing all or part of the steps of the methods of the various embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), an electrically erasable programmable ROM, registers, a hard disk, a multimedia card, a card-type Memory (e.g., SD or DX Memory, etc.), a magnetic Memory, a removable disk, a CD-ROM, a magnetic disk, or an optical disk, etc., that can store program code.
Based on this, an embodiment of the present invention further provides a readable storage medium storing a computer program, where the computer program when executed by a processor performs the steps of the hyperspectral remote sensing image identification method according to any one of the embodiments above.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the hardware including the device and the electronic equipment disclosed in the embodiments, the description is relatively simple because the hardware includes the device and the electronic equipment corresponding to the method disclosed in the embodiments, and relevant places refer to the description of the method.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The above describes in detail a method, a device, an electronic device and a readable storage medium for identifying hyperspectral remote sensing images. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present invention, and such improvements and modifications fall within the scope of the claims of the present application.
Claims (8)
1. The hyperspectral remote sensing image identification method is characterized by comprising the following steps of:
pre-training an image recognition model; the image recognition model comprises a network input combination module, a multi-scale double-pass convolution module, a convolution characteristic mixing module and a compressed expansion spatial spectrum attention mechanism module;
inputting the hyperspectral remote sensing image to be identified into the image identification model to obtain an image identification result of the hyperspectral remote sensing image to be identified;
the network input combination module combines the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified, and selects a network input image block from the combination result; the multi-scale bi-pass convolution module carries out fixed kernel convolution operation on the network input image block by using a first channel, and carries out multi-scale cavity group convolution processing on the network input image block by using a second channel; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation;
The multi-scale double-pass convolution module is formed by a first convolution layer and a second convolution layer in a double-channel mode;
the convolution kernel of the first convolution layer is fixed in size; the second convolution layer is formed by at least three layers of convolutions in a serial manner.
2. The method of claim 1, wherein the second convolution layer comprises a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
3. The method of claim 1, wherein the mixing of multi-channel features in different convolution processes comprises:
acquiring first channel characteristic information output by the first channel and second channel characteristic information output by the second channel;
combining the first channel characteristic information and the second channel characteristic information according to the channel dimension;
equally dividing the channels of the merging result into two groups, and arranging the groups according to the equipotential sequence; the co-located channels are extracted and reordered.
4. The hyperspectral remote sensing image recognition method as claimed in any one of claims 1 to 3, wherein the process of learning target features in a spectrum dimension and a space dimension, adaptively adjusting a spatial spectrum feature map based on attention weights, and conveying the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation comprises the following steps:
performing channel dimension operation on the space characteristics by using a neural network formed by a global pooling layer and a multi-layer perceptron, inputting the space characteristics to the full-connection layer through a RELU activation function layer, and outputting the space characteristics through a sigmoid activation function layer;
the spatial spectrum features are subjected to channel dimension pooling treatment to obtain a single channel, and spatial attention is generated through single-layer convolution operation and sigmoid activation function layer treatment;
determining a spatial spectrum attention according to the spatial attention and the channel attention;
and obtaining a channel dimension scaling characteristic according to the spatial spectrum attention and the spatial spectrum characteristic.
5. A hyperspectral remote sensing image recognition device, comprising:
the model pre-training module is used for pre-training the image recognition model; the image recognition model comprises a network input combination module, a multi-scale double-pass convolution module, a convolution characteristic mixing module and a compressed expansion spatial spectrum attention mechanism module;
The image recognition module is used for inputting the hyperspectral remote sensing image to be recognized into the image recognition model to obtain an image recognition result of the hyperspectral remote sensing image to be recognized; the network input combination module combines the neighborhood characteristics and spectrum dimensions of the hyperspectral remote sensing image to be identified, and selects a network input image block from the combination result; the multi-scale bi-pass convolution module carries out fixed kernel convolution operation on the network input image block by using a first channel, and carries out multi-scale cavity group convolution processing on the network input image block by using a second channel; the convolution characteristic mixing module mixes the multi-channel characteristics in different convolution processes; the compressed and expanded spatial spectrum attention mechanism module learns target features in a spectrum dimension and a space dimension, adaptively adjusts a spatial spectrum feature map based on attention weight, and transmits the finally obtained target spatial spectrum feature map to a discrimination convolution layer and a full connection layer for dimension reduction operation;
the multi-scale double-pass convolution module is formed by a first convolution layer and a second convolution layer in a double-channel mode;
the convolution kernel of the first convolution layer is fixed in size; the second convolution layer is formed by at least three layers of convolutions in a serial manner.
6. The hyperspectral remote sensing image recognition device of claim 5, wherein the second convolution layer comprises a first sub-convolution layer, a second sub-convolution layer, and a third sub-convolution layer; the convolution kernel sizes of the first sub-convolution layer, the second sub-convolution layer and the third sub-convolution layer are the same, and incremental cavity convolution rate is adopted.
7. An electronic device comprising a processor and a memory, the processor being configured to implement the steps of the hyperspectral remote sensing image identification method as claimed in any one of claims 1 to 4 when executing a computer program stored in the memory.
8. A readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the hyperspectral remote sensing image identification method as claimed in any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137984.4A CN113822207B (en) | 2021-09-27 | 2021-09-27 | Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137984.4A CN113822207B (en) | 2021-09-27 | 2021-09-27 | Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822207A CN113822207A (en) | 2021-12-21 |
| CN113822207B true CN113822207B (en) | 2024-02-02 |
Family
ID=78921367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111137984.4A Active CN113822207B (en) | 2021-09-27 | 2021-09-27 | Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822207B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022748B (en) * | 2022-01-06 | 2022-04-08 | 深圳市海清视讯科技有限公司 | Target identification method, device, equipment and storage medium |
| CN116051896B (en) * | 2023-01-28 | 2023-06-20 | 西南交通大学 | Hyperspectral image classification method of lightweight mixed tensor neural network |
| CN116630700A (en) * | 2023-05-22 | 2023-08-22 | 齐鲁工业大学(山东省科学院) | Remote sensing image classification method based on introduction channel-space attention mechanism |
| CN116773534B (en) * | 2023-08-15 | 2024-03-05 | 宁德思客琦智能装备有限公司 | Detection method and device, electronic equipment and computer readable medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110148103A (en) * | 2019-04-29 | 2019-08-20 | 中国科学院西安光学精密机械研究所 | EO-1 hyperion and Multispectral Image Fusion Methods, computer readable storage medium, electronic equipment based on combined optimization |
| CN110705457A (en) * | 2019-09-29 | 2020-01-17 | 核工业北京地质研究院 | Remote sensing image building change detection method |
| CN111626267A (en) * | 2019-09-17 | 2020-09-04 | 山东科技大学 | Hyperspectral remote sensing image classification method using void convolution |
| CN111914907A (en) * | 2020-07-13 | 2020-11-10 | 河海大学 | Hyperspectral image classification method based on deep learning space-spectrum combined network |
| KR102197297B1 (en) * | 2019-09-27 | 2020-12-31 | 서울대학교산학협력단 | Change detection method using recurrent 3-dimensional fully convolutional network for hyperspectral image |
| CN112906662A (en) * | 2021-04-02 | 2021-06-04 | 海南长光卫星信息技术有限公司 | Method, device and equipment for detecting change of remote sensing image and storage medium |
| CN113011499A (en) * | 2021-03-22 | 2021-06-22 | 安徽大学 | Hyperspectral remote sensing image classification method based on double-attention machine system |
-
2021
- 2021-09-27 CN CN202111137984.4A patent/CN113822207B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110148103A (en) * | 2019-04-29 | 2019-08-20 | 中国科学院西安光学精密机械研究所 | EO-1 hyperion and Multispectral Image Fusion Methods, computer readable storage medium, electronic equipment based on combined optimization |
| CN111626267A (en) * | 2019-09-17 | 2020-09-04 | 山东科技大学 | Hyperspectral remote sensing image classification method using void convolution |
| KR102197297B1 (en) * | 2019-09-27 | 2020-12-31 | 서울대학교산학협력단 | Change detection method using recurrent 3-dimensional fully convolutional network for hyperspectral image |
| CN110705457A (en) * | 2019-09-29 | 2020-01-17 | 核工业北京地质研究院 | Remote sensing image building change detection method |
| CN111914907A (en) * | 2020-07-13 | 2020-11-10 | 河海大学 | Hyperspectral image classification method based on deep learning space-spectrum combined network |
| CN113011499A (en) * | 2021-03-22 | 2021-06-22 | 安徽大学 | Hyperspectral remote sensing image classification method based on double-attention machine system |
| CN112906662A (en) * | 2021-04-02 | 2021-06-04 | 海南长光卫星信息技术有限公司 | Method, device and equipment for detecting change of remote sensing image and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| Tri_training for remote sensing classification based on multi-scale homogeneity;J. Zhu, K. Tan and Q. Du;《2016 IEEE International Geoscience and Remote Sensing Symposium》;全文 * |
| 基于卷积神经网络的遥感图像降噪;潘凯;侯亮;;现代信息科技(第12期);全文 * |
| 多尺度特征融合空洞卷积 ResNet遥感图像建筑物分割;徐胜军;欧阳朴衍;郭学源;Taha Muthar Khan;段中兴;;光学精密工程(第07期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822207A (en) | 2021-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113822207B (en) | Hyperspectral remote sensing image identification method and device, electronic equipment and storage medium | |
| CN113822209B (en) | Hyperspectral image recognition method and device, electronic equipment and readable storage medium | |
| CN107767408B (en) | Image processing method, processing device and processing equipment | |
| CN111401516B (en) | Searching method for neural network channel parameters and related equipment | |
| CN109522942B (en) | Image classification method and device, terminal equipment and storage medium | |
| CN111695467B (en) | Spatial spectrum full convolution hyperspectral image classification method based on super-pixel sample expansion | |
| CN113159147B (en) | Image recognition method and device based on neural network and electronic equipment | |
| CN102324047B (en) | Hyper-spectral image ground object recognition method based on sparse kernel representation (SKR) | |
| CN107958230B (en) | Facial expression recognition method and device | |
| CN107145836B (en) | Hyperspectral image classification method based on stacked boundary identification self-encoder | |
| CN105138993A (en) | Method and device for building face recognition model | |
| CN110874604A (en) | Model training method and terminal equipment | |
| CN101853392A (en) | Remote sensing hyperspectral image band selection method based on conditional mutual information | |
| CN110007959A (en) | Hard-wired stratification mantissa bit length for deep neural network selects | |
| CN111401339A (en) | Method and device for identifying age of person in face image and electronic equipment | |
| CN107016359A (en) | A kind of fast face recognition method being distributed under complex environment based on t | |
| CN113705749A (en) | Two-dimensional code identification method, device and equipment based on deep learning and storage medium | |
| CN114494777A (en) | Hyperspectral image classification method and system based on 3D CutMix-transform | |
| CN110956116B (en) | Face image gender identification model and method based on convolutional neural network | |
| US20230186613A1 (en) | Sample Classification Method and Apparatus, Electronic Device and Storage Medium | |
| CN110795993A (en) | Method and device for constructing model, terminal equipment and medium | |
| CN115713669A (en) | Image classification method and device based on inter-class relation, storage medium and terminal | |
| CN111383716B (en) | Screening method, screening device, screening computer device and screening storage medium | |
| CN112052881B (en) | Hyperspectral image classification model device based on multi-scale near-end feature splicing | |
| CN114118411A (en) | Training method of image recognition network, image recognition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |