CN109255334B

CN109255334B - Remote sensing image ground feature classification method based on deep learning semantic segmentation network

Info

Publication number: CN109255334B
Application number: CN201811130333.0A
Authority: CN
Inventors: 楚博策; 帅通; 高峰; 王士成; 陈金勇
Original assignee: CETC 54 Research Institute
Current assignee: CETC 54 Research Institute
Priority date: 2018-09-27
Filing date: 2018-09-27
Publication date: 2021-12-07
Anticipated expiration: 2038-09-27
Also published as: CN109255334A

Abstract

The invention discloses a remote sensing image ground feature classification method based on a deep learning semantic segmentation network. The invention designs a method for constructing a multi-scale feature map group by using texture and structural features as the basis, the feature map group and an original image are combined to be used as the input of a deep learning network, besides, the invention designs an improved network structure of a full convolution network according to a deplab algorithm, parameter training is carried out through convolution and deconvolution, finally, overlapping segmentation is carried out on wide remote sensing images, and after classification, combination is carried out to obtain the final wide remote sensing image ground feature classification result. The method can efficiently and rapidly realize the pixel-level classification of various ground objects of the high-resolution remote sensing image, simplifies the complex process of the traditional classification method and realizes good segmentation and classification effects.

Description

Remote sensing image ground feature classification method based on deep learning semantic segmentation network

Technical Field

The invention belongs to the technical field of intelligent classification of remote sensing images, and particularly relates to a remote sensing ground object classification method based on a full convolution semantic segmentation network under the ground object interpretation requirement.

Background

The remote sensing image land feature classification is widely applied to various military and civil applications such as land survey, satellite film law enforcement, regional investigation and the like at present, obtains a better application effect and has a larger market development potential. With the gradual increase of satellite load and data volume, in remote sensing high-precision classification research, particularly when large-area (national or global scale) ground surface classification is faced, the traditional manual calibration method is difficult to support explosive growth tasks and demand workload, so that how to research how to adopt an artificial intelligence method to realize intelligent automatic processing of remote sensing images is important work with profound significance.

The existing method for classifying the ground features at present,

(1) most of the traditional methods mainly adopt a segmentation method such as superpixel to segment the remote sensing image into a plurality of regions, then traditional characteristics such as morphology and texture are extracted from the regions, and finally classifiers such as SVM are adopted to classify and combine the regions according to the characteristics to form classification results.

(2) In recent years, deep learning research on ground feature classification mainly focuses on initially segmenting an original image by methods such as superpixels and classifying segmented block images by neural networks such as CNN and DBN, so that the purpose of pixel-level ground feature classification is achieved.

(3) The invention innovatively provides a multi-scale feature description map of image textures and structures, the multi-scale feature description map and an original 3-dimensional image are combined to generate a high-dimensional image, the feature description power of the image textures and the like is enhanced, then a deep learning model-full convolution depth network applied to a semantic segmentation task is improved and applied to terrain classification of a high-resolution remote sensing image, the integration of segmentation and classification processes is realized, and a good classification effect is obtained.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a remote sensing image ground feature classification method based on a deep learning semantic segmentation network, so that the classification precision and efficiency are further improved, and the classification error is reduced.

The purpose of the invention is realized as follows:

a remote sensing image surface feature classification method based on a deep learning semantic segmentation network comprises the following steps:

(1) collecting high-resolution visible light remote sensing images with different loads, carrying out pixel-by-pixel labeling on ground objects in each image, forming a binary image by labeled data, and packaging an original remote sensing image and a corresponding labeled image to form a training set and a test set;

(2) extracting two-dimensional entropy, roughness and contrast texture features of the original remote sensing images in the training set by adopting a multi-scale window to form a multi-scale feature map group; extracting the edges of the ground objects from the original remote sensing images in the training set by adopting a Canny operator to form a structural characteristic diagram;

(3) constructing a deep learning full convolution semantic segmentation model based on the idea of deep Lab;

(4) combining the multi-scale feature map group, the structural feature map and the original remote sensing image generated in the step (2) to form an input map group, and performing model training as the input of the deep learning full convolution semantic segmentation model in the step (3) to finally obtain a model with stable parameters;

(5) and (4) segmenting the original remote sensing images to be classified in the test set, classifying the segmented images through the trained model with stable parameters in the step (4), combining the classification results together to generate a wide detection result, and when the detection results in the overlapped area are contradictory, retaining the result of the non-background pixels classified in the overlapped area to obtain a final combination result.

The full convolution semantic segmentation model in the step (3) is specifically as follows:

the model is divided into a downward section and an upward section, wherein the downward passage changes the original 13 layers of convolution layers of the VGGnet into 6 layers according to the classification intensity, and a 16 multiplied by 16 dimensional characteristic heat map is obtained as a 7 th layer after 6 layers of convolution and pooling are carried out on an input layer; in an upward path, performing interpolation up-sampling on the 7 th layer of deconvolution layer to restore the size of the 7 th layer to be the same as that of the 6 th layer, performing up-sampling on the 7 th layer, and fusing the 5 th layer of porous convolution to generate an 8 th layer; and (3) performing up-sampling on the 8 th layer, fusing the 6 th layer porous convolution to generate a 9 th layer, and performing size transformation and reduction on the output of the 9 th layer to the size of the original remote sensing image to obtain a final classification result.

Compared with the background technology, the invention has the following advantages:

1. the invention provides a multi-scale feature map group to replace the input of a pure RGB image, enhance the feature representation force and guide the feature extraction direction of a neural network;

2. the deep learning full convolution neural network is adopted, the end-to-end ground feature classification task can be realized, and error accumulation caused by multi-step composition in the traditional method is replaced.

3. The method adopts a Deeplab network structure to be applied to the remote sensing ground object classification direction. The porous convolution can effectively solve the problem of insufficient receptive field of the remote sensing large-scale ground object, and meanwhile, the boundary can be effectively optimized by adopting the conditional random field CRF, so that the edge classification effect is extracted.

Drawings

FIG. 1 is a flow chart of the present invention.

FIG. 2 is a texture feature map of the present invention.

FIG. 3 is a schematic diagram of the construction of a multi-scale feature map set according to the present invention.

Fig. 4 is a structural feature diagram of the present invention.

Fig. 5 is a diagram of a full convolutional network design in accordance with the present invention.

Fig. 6 is a diagram of the improved deep network design of the present invention.

Fig. 7 is a diagram of the deep network architecture of the present invention.

Fig. 8 is a narrow high resolution image of the present invention.

FIG. 9 is a graph of the accuracy of the classification method of the present invention compared to other methods.

FIG. 10 is a comparison graph of the classification effect of the method of the present invention and other methods.

FIG. 11 is a diagram illustrating the effect of classifying wide images according to the present invention.

Detailed Description

The following description of the embodiments of the present invention is provided in order to better understand the present invention for those skilled in the art with reference to the accompanying drawings. It is to be expressly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the subject matter of the present invention.

FIG. 1 is a schematic block diagram of a remote sensing image surface feature classification method based on a deep semantic segmentation network according to a specific implementation of the present invention.

In this embodiment, the method for classifying the remote sensing image surface features based on the deep learning semantic segmentation network shown in fig. 1 includes the following steps:

1. data preparation

The data preparation comprises the steps of collecting and marking images, wherein remote sensing images of high-resolution visible light with different loads are collected, then pixel-by-pixel marking is carried out on ground features in each image, the marked data form a binary image, wherein 0 gray represents background pixels, 1-6 gray represents 6 types of ground features such as buildings, grasslands and the like, and the remote sensing images and the corresponding marked images are packaged to form a training set and a testing set;

2. feature map set extraction

The feature map group extraction comprises extraction of a multi-scale texture feature map group and a structural feature map. Considering that different ground objects have insignificant differences among some texture features (such as directionality and the like) and have larger differences among some texture features (such as concentration, complexity, brightness variation and the like), the invention adopts two-dimensional entropy, roughness and contrast as texture features, and the extraction effect graph is shown in fig. 2. In addition, considering that the change of the size of the window of the texture features can affect the description capability of the texture features to the whole situation and the local situation, the invention adopts a multi-scale window to extract the features to form a multi-scale feature map group, and ensures the comprehensiveness of the feature description capability, as shown in fig. 3. In the aspect of structural characteristics, the structure is not characterized, but the edges of the ground objects in the image are extracted by adopting a Canny operator to highlight the information proportion of the structural information in the network input image set, as shown in fig. 4, the structural characteristics can be well prevented from being covered by redundant information in the deep learning training process, and abstract structural characteristics can be better extracted in the subsequent deep network characteristic training and extracting process.

3. Model training

The deep learning full convolution semantic segmentation model is designed by referring to the idea of deep Lab, deep Lab is a derivative improved network of a full convolution network, and a network design diagram is shown in FIG. 5. The network model can be divided into two sections, corresponding to two sections of downward (adopting porous convolution to extract characteristics and gradually down-sample and simultaneously extract semantic characteristics) and upward (gradually up-sample characteristics to recover detailed information) paths in deep Lab. In which the downward path is modified based on VGGnet and the network structure is shown in fig. 6. Performing porous convolution operation on a multi-channel image and a feature map group, obtaining a feature heat map (16 multiplied by 16 dimensional image) as a 7 th layer after 6 layers of convolution and pooling, performing up-sampling on a 7 th layer of deconvolution layer to restore the size of the 7 th layer to be the same as that of the 6 th layer, converting a low-resolution image into high resolution by an interpolation method, performing up-sampling on the 7 th layer, and fusing the up-sampling with the output of the 5 th layer after porous convolution to generate an 8 th layer; the 8 th layer is upsampled and then fused with the 6 th layer of porous convolution to generate a 9 th layer, namely a final classification result, as shown in fig. 7. Calculating the loss of the classification of the multi-classification regression model softmax pixel by pixel according to the classification result to obtain the calculated loss value of each target, sequencing the loss values, selecting the first B (empirical value) targets with the largest loss values as the sample of the difficult case, feeding the loss values of the sample of the difficult case back to the full convolution neural network model, and updating the parameters of the full convolution neural network model by using a random gradient descent method. And for each remote sensing ground feature classification labeling image, continuously updating parameters of the regional full convolution neural network according to the training process, thereby obtaining a full convolution neural network model of ground feature classification, and using the model in a subsequent ground feature classification task.

4. Ground object classification

The method comprises the steps of firstly segmenting an original wide remote sensing image as shown in figure 8, assuming that the resolution is X, setting the size L of a narrow image, using 0.5X L of the length of the shorter side of the narrow image as the number of overlapped pixels by adopting an overlapping segmentation method, then classifying the segmented image through a depth model, and finally merging the classification results together to produce a wide detection result. When the overlap region detection results produce contradiction, the result classified as non-background pixels therein is retained as the final merged result as shown in fig. 9.

In order to verify the effectiveness of the method, firstly, a data set manufactured by the user is used for training a model, and then the comparison verification of the human body target classification effect is carried out on the basis of the acquired remote sensing image under the complex scene. In the embodiment, a Tensorflow framework is selected to realize a Deeplab network architecture, model parameters are initialized and trained based on the number of data sets and the classification of the ground feature classification tasks, and finally a model for ground feature classification is obtained

The invention realizes end-to-end remote sensing ground feature classification, and adopts a kappa coefficient and an intersection ratio IOU to carry out measurement indexes, wherein the kappa coefficient represents the discrete comprehensive evaluation of the pixel ratio and the number of pixels classified into a ground feature in other ground feature categories, and the intersection ratio IOU represents the ratio of the total number of pixels correctly classified into the ground feature category to the sum of the number of pixels classified into the ground feature category and the total number of pixels of the ground feature. The final kappa coefficient of the invention is 83%, the merging ratio IOU is 81%, and the performance is greatly improved compared with other deep learning methods such as VGG (kappa coefficient 75%%, merging ratio IOU is 71%), rescet 50(kappa coefficient 81%, merging ratio IOU is 78%), rescet 101(kappa coefficient 77%, merging ratio IOU is 72%), rescet 152(kappa coefficient 79%, merging ratio IOU is 75%), as shown in FIG. 10. A comparison graph of the classification effect of specific different classification model structures is shown in fig. 11.

Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be made apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all matters of the invention which utilize the inventive concepts are protected.

Claims

1. A remote sensing image surface feature classification method based on a deep learning semantic segmentation network is characterized by comprising the following steps:

2. The remote sensing image terrain classification method based on the deep learning semantic segmentation network as claimed in claim 1, wherein the full convolution semantic segmentation model in the step (3) is specifically as follows: