CN112560544A - Method and system for identifying ground object of remote sensing image and computer readable storage medium - Google Patents
Method and system for identifying ground object of remote sensing image and computer readable storage medium Download PDFInfo
- Publication number
- CN112560544A CN112560544A CN201910854251.9A CN201910854251A CN112560544A CN 112560544 A CN112560544 A CN 112560544A CN 201910854251 A CN201910854251 A CN 201910854251A CN 112560544 A CN112560544 A CN 112560544A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- sensing image
- image
- feature
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method, a system and a computer readable storage medium for identifying a ground feature of a remote sensing image, wherein the method comprises the following steps: collecting an original sample remote sensing image for training; carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image; constructing a multi-scale dense convolutional network; training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image; and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features. The invention adopts a dense hierarchical connection mode of the DenseNet to reconstruct the network, deepens the network depth and enhances the transmission of characteristic information; and a multi-scale feature conversion layer is constructed, and image feature information extracted by the convolution kernel is enriched, so that the network can achieve the purpose of high accuracy rate for identifying the ground features of the remote sensing image.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a method and a system for identifying ground features of remote sensing images and a computer readable storage medium.
Background
Feature identification of a remotely sensed image refers to the process of linking each pixel in the image to a feature class label. This process is image semantic segmentation, but it can be considered as a pixel-level image classification. Semantic segmentation is a typical computer vision problem that involves taking some raw data (e.g., a flat image) as input and converting them into a mask with highlighted regions of interest, where each pixel in the image is assigned a category ID according to the object of interest to which it belongs. The ground feature identification of the remote sensing image is a key technology of a geographic information system, and has very important functions in various fields such as land planning, disaster prevention and control, unmanned aerial vehicles, satellites and resource monitoring.
The traditional remote sensing image ground object identification method mainly comprises a threshold value method, a boundary detection method, a region method and the like. The implementation principle of these methods is different, but basically utilizes the low-level semantics of the image, including information such as color, texture and shape of image pixels, but the actual segmentation effect when encountering complex scenes is not ideal.
Currently, deep learning is mainly used for performing related research, including Full Convolutional Network (FCN) introducing a coding-decoding structure, but FCN has problems such as semantic information loss and lacks of research on correlation between pixels; the SegNet of the unpoiling operation ensures the integrity of high-frequency information, but ignores the information between adjacent pixels in the lower-resolution feature image; and the DeconvNet containing a complete connection layer has a larger network model and is more difficult to train.
Disclosure of Invention
In order to solve at least one technical problem, the invention provides a remote sensing image surface feature identification method, a system and a computer readable storage medium.
In order to achieve the above object, a first aspect of the present invention provides a method for identifying a feature in a remote sensing image, the method comprising:
collecting an original sample remote sensing image for training;
carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
constructing a multi-scale dense convolutional network;
training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
In this scheme, carry out data enhancement to the original sample remote sensing image who gathers and handle, specifically include:
and enhancing the original sample remote sensing image by adopting a data enhancement algorithm, wherein the data enhancement algorithm comprises any one or more of a cutting algorithm, a translation algorithm, a turning algorithm, a rotation algorithm, a noise addition algorithm, a scaling algorithm and a filtering algorithm.
In the scheme, the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer and a plurality of dense connecting blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image and enriching high-level semantic information of the image processed by the network;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the feature image information output by each layer to be repeatedly utilized so as to strengthen the transmission of the feature image information and deepen the network structure.
In the scheme, the method for identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network specifically comprises the following steps:
inputting the remote sensing image to be identified into the multi-scale dense convolution network;
carrying out multi-scale feature extraction on the remote sensing image to be identified by a multi-scale feature conversion layer;
generating a feature image with semantic information through a plurality of down-sampling layers of an encoder structure, and repeatedly utilizing feature image information output by each layer by adopting a dense connecting block in the period;
mapping the characteristic image output by the encoder structure back to the size of the remote sensing image to be identified through a plurality of upper sampling layers of the encoder structure so as to carry out pixel-by-pixel classification, and using a dense connecting block to repeatedly utilize the characteristic image information output by each layer;
and segmenting the ground objects in the remote sensing image to be identified according to the pixel-by-pixel classification result, and identifying.
In the scheme, the multi-scale feature conversion layer comprises a plurality of convolution layers with convolution kernels of different sizes, a plurality of batch normalization layers and a Concat connection layer,
the plurality of convolution layers are respectively used for extracting semantic information of a plurality of images;
the batch normalization layers are used for performing normalization processing on input data of each layer in the neural network training process;
and the Concat connecting layer is used for connecting the image semantic information extracted by the convolution kernels with different sizes.
Further, the multi-scale feature conversion layer comprises convolution layers of four convolution kernels with different sizes, wherein the four convolution kernels with different sizes are respectively 1 × 1 convolution kernel, 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel, wherein 1 × 1 convolution kernel is used for retaining original image information, and 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel are used for extracting various image semantic information.
In the scheme, any two layers in the dense connecting blocks are connected by a direct feature diagram, the feature output of any layer can be directly connected to all subsequent layers, the three-dimensional features of each layer in the previous (l-1) layer of the l-th layer in the dense connecting blocks are used as input, and the size of the three-dimensional features extracted by the l-th layer is as follows: k is a radical ofl＝Fl([k0，k1，…，kl-1]) Wherein [ k ]0，k1，…，kl-1]The three-dimensional feature maps representing the layers are densely connected and the spatial size of any feature map is the same; function FlThe representation is a set of batch normalization, activation functions, and convolution operations.
The second aspect of the present invention further provides a remote sensing image feature identification system, including: the remote sensing image ground feature identification method program comprises a memory and a processor, wherein the memory comprises the remote sensing image ground feature identification method program, and the remote sensing image ground feature identification method program realizes the following steps when being executed by the processor:
collecting an original sample remote sensing image for training;
carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
constructing a multi-scale dense convolutional network;
training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
In the scheme, the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer and a plurality of dense connecting blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image and enriching high-level semantic information of the image processed by the network;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the feature image information output by each layer to be repeatedly utilized so as to strengthen the transmission of the feature image information and deepen the network structure.
The third aspect of the present invention further provides a computer readable storage medium, where the computer readable storage medium includes a remote sensing image feature identification method program, and when the remote sensing image feature identification method program is executed by a processor, the steps of the remote sensing image feature identification method are implemented.
The invention provides a remote sensing image surface feature identification method and system based on a multi-scale dense convolution network and a computer readable storage medium. The method is structurally expanded on the basis of a network of an encoder-decoder structure, firstly, the network is reconstructed by means of a dense hierarchical connection mode of a DenseNet network, the network depth is deepened, the transmission of characteristic information is enhanced, and image characteristic information extracted by a convolution kernel is greatly enriched by constructing a multi-scale characteristic conversion layer, so that the network achieves the purpose of high accuracy rate of remote sensing image ground object identification.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of a method for recognizing a ground object in a remote sensing image according to the invention;
FIG. 2 is an architectural diagram of a multi-scale dense convolutional network of the present invention;
FIG. 3 is a flow chart of a method for identifying surface features in remote sensing images by using a multi-scale dense convolutional network according to the present invention;
FIG. 4 shows an architecture diagram of a dense joint block of the present invention;
FIG. 5 shows an architecture diagram of a multi-scale feature conversion layer of the present invention;
FIG. 6 is a graph illustrating an example effect of the multi-scale dense convolutional network on the identification of surface features of remote sensing images;
fig. 7 shows a block diagram of a remote sensing image ground object recognition system of the invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
FIG. 1 shows a flow chart of a method for identifying a ground object in a remote sensing image according to the invention.
As shown in fig. 1, a first aspect of the present invention provides a method for identifying a surface feature in a remote sensing image, where the method includes:
s102, collecting an original sample remote sensing image for training;
s104, performing data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
s106, constructing a multi-scale dense convolution network;
s108, training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and S110, after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
It should be noted that the technical solution of the present invention can be implemented in a terminal device such as a PC, a mobile phone, a PAD, and the like.
The land feature may be any one or more of a building, a road, a sports ground, a tree, a vehicle, a pedestrian, a river, and a lake. But is not limited thereto.
According to the embodiment of the invention, the training of the multi-scale dense convolution network is carried out by combining the original sample remote sensing image and the enhanced sample remote sensing image, and the training specifically comprises the following steps:
training a multi-scale dense convolution network by using the original sample remote sensing image and the enhanced sample remote sensing image, modifying parameters among layers in the multi-scale dense convolution network according to errors between a training output result and label information, and reducing the errors;
and stopping training the multi-scale dense convolution network when the error meets the preset requirement.
The method includes the steps of acquiring an original sample remote sensing image, acquiring an enhanced sample remote sensing image, storing the original sample remote sensing image and the enhanced sample remote sensing image in a sample library, storing label information corresponding to the original sample remote sensing image and the enhanced sample remote sensing image in the sample library, training a multi-scale dense convolution network according to the original sample remote sensing image, the enhanced sample remote sensing image and the corresponding label information, and optimizing network prediction accuracy.
According to the embodiment of the invention, the data enhancement processing is carried out on the acquired remote sensing image of the original sample, and the method specifically comprises the following steps:
and enhancing the original sample remote sensing image by adopting a data enhancement algorithm.
It should be noted that the data enhancement algorithm may include any one or more of a clipping algorithm, a translation algorithm, a flipping algorithm, a rotation algorithm, a noise adding algorithm, a scaling algorithm, and a filtering algorithm. But is not limited thereto.
The method can be understood that the data enhancement technology is utilized to carry out data amplification on the remote sensing satellite image, so that enough data volume can be provided for network training, and the over-fitting phenomenon in the network training is avoided.
FIG. 2 shows an architectural diagram of a multi-scale dense convolutional network of the present invention.
As shown in fig. 2, the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer, and a plurality of densely connected blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image and enriching high-level semantic information of the image processed by the network;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the feature image information output by each layer to be repeatedly utilized so as to strengthen the transmission of the feature image information and deepen the network structure.
It should be noted that the encoder structure is configured to extract features from an input image, the decoding structure is configured to decode a result image from the features, and a last layer of the decoder structure is a softmax layer, which is used to classify pixels.
FIG. 3 shows a flow chart of the method for identifying the surface features in the remote sensing image by the multi-scale dense convolution network.
As shown in fig. 3, identifying the surface feature in the remote sensing image to be identified by the multi-scale dense convolution network specifically includes:
s302, inputting the remote sensing image to be identified into the multi-scale dense convolution network;
s304, performing multi-scale feature extraction on the remote sensing image to be identified by a multi-scale feature conversion layer;
s306, generating a feature image with semantic information through a plurality of downsampling layers of an encoder structure, and repeatedly utilizing feature image information output by each layer by adopting a dense connecting block in the period;
s308, mapping the characteristic images output by the encoder structure back to the size of the remote sensing image to be identified through a plurality of up-sampling layers of the encoder structure so as to carry out pixel-by-pixel classification, and repeatedly utilizing the characteristic image information output by each layer by adopting dense connecting blocks in the period;
and S310, segmenting the ground feature in the remote sensing image to be identified according to the pixel-by-pixel classification result, and identifying.
It should be noted that, in the process of encoding by the encoder structure and decoding by the decoder structure, dense connection blocks are adopted to make the feature image information output by each layer be reused, so as to enhance the transmission of the feature image information.
According to an embodiment of the present invention, the multi-scale feature conversion layer comprises a plurality of convolution layers of convolution kernels of different sizes, a plurality of batch normalization layers and a Concat connection layer,
the plurality of convolution layers are respectively used for extracting semantic information of a plurality of images;
the batch normalization layers are used for performing normalization processing on input data of each layer in the neural network training process;
and the Concat connecting layer is used for connecting the image semantic information extracted by the convolution kernels with different sizes.
It can be understood that a Batch Normalization layer (BN) can be used for stabilizing a network structure and reducing the risk of network overfitting; the function of the Concat connection layer is to splice two or more feature images in the channel or num dimension, so that image feature information can be enriched.
As shown in fig. 4, the multi-scale feature conversion layer includes convolution layers of four convolution kernels with different sizes, namely 1 × 1 convolution kernel, 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel, four BN layers and one Concat connection layer, wherein 1 × 1 convolution kernel is used for retaining original image information, and 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel are used for extracting various image semantic information.
It should be noted that 3 × 3 convolution kernels, 5 × 5 convolution kernels, and 7 × 7 convolution kernels can be used to extract high-level semantic information of various images. In other embodiments, the multi-scale feature conversion layer includes two, three, four convolution kernels of different sizes. But is not limited thereto.
According to the embodiment of the invention, the dense connection block can be used multiple times in the whole network structure (as shown in fig. 2), any two layers in the dense connection block have direct feature map connection, the feature output of any layer can be directly connected to all the subsequent layers, the three-dimensional feature of each layer in the previous (l-1) layer in the first layer in the dense connection block is used as an input, and then the size of the three-dimensional feature extracted by the first layer is as follows: k is a radical ofl＝Fl([k0，k1，…，kl-1]) Wherein [ k ]0，k1，…，kl-1]The three-dimensional feature maps representing the layers are densely connected and the spatial size of any feature map is the same; function FlThe representation is a set of batch normalization, activation functions, and convolution operations.
It can be understood that the transmission of the features can be enhanced by introducing a dense connection block structure, so that the quantity of parameters required by the network is reduced, the calculated quantity is reduced, the problem of gradient disappearance in network training can be effectively solved, the network structure can be deepened, effective high-level semantic information of images can be extracted, and the optimal recognition performance of the network can be realized.
As shown in FIG. 5, assume that the input is a remote sensing image X0A neural network passing through an L layer, wherein the non-linear transformation of the i layer is denoted as Hi(*)，Hi() may be an accumulation of various function operations, such as BN, ReLU, Pooling or Conv, etc. The characteristic output of the i-th layer is denoted Xi. The input to the i-th layer is related not only to the output of the i-1 layer, but also to the outputs of all previous layers, and is noted as: xl＝Hl([X0，X1，…，Xl-1]) Wherein]Represents splicing (ligation), i.e.X0To Xl-1All of the layersThe output characteristic maps are combined together by channels. The nonlinear transformation H used here is a combination of BN, ReLU, and Conv (3 × 3). The features of each layer are connected by means of a connection, the l-th layer has l inputs when passing through the l layers, so that a network of l layers has a total of
And (4) connecting. By adopting DenseNet, the invention can reduce network parameters, make the network narrower, and is beneficial to the transmission of characteristic image information and the reduction of the situation of gradient disappearance during the backward propagation.
According to an embodiment of the present invention, with the encoder and decoder structures, the unfilled convolutional layer, ReLu active layer, and max-pooling layer are used in the encoder and decoder structures, and since the convolution is performed using the Concat connection layer series profile, the number of convolutional filters is doubled after each downsampling.
It should be noted that specific structures and parameters of the entire network, such as the sizes and the number of convolution kernels in the multi-scale feature conversion layer, the number of dense connection layers, the number of convolution layers inside the connection layers, the down-sampling size, the number of output layer units, and the like, can be specifically adjusted in the actual scene application so that the network has a better recognition and segmentation effect.
FIG. 6 is a graph illustrating an example effect of the multi-scale dense convolutional network on the identification of the surface features of the remote sensing image.
As shown in FIG. 6, compared with the traditional remote sensing image ground object identification method based on the convolutional network, the remote sensing image ground object identification method based on the multi-scale dense convolutional network can obtain the high-level semantic features of the image with better performance and more abundant information, thereby achieving better identification effect.
Fig. 7 shows a block diagram of a remote sensing image ground object recognition system of the invention.
As shown in fig. 7, the second aspect of the present invention further provides a remote sensing image feature recognition system 7, which includes: a memory 71 and a processor 72, wherein the memory includes a remote sensing image feature identification method program, and the remote sensing image feature identification method program realizes the following steps when executed by the processor:
collecting an original sample remote sensing image for training;
carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
constructing a multi-scale dense convolutional network;
training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
It should be noted that the system of the present invention can be operated in a terminal device such as a PC, a mobile phone, a PAD, etc.
It should be noted that the Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, and so on. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It should be noted that the system may further include a display, and the display may be referred to as a display screen or a display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch panel, or the like. The display is used for displaying information processed in the system, recognition results and a work interface for displaying visualization.
According to the embodiment of the invention, the data enhancement processing is carried out on the acquired remote sensing image of the original sample, and the method specifically comprises the following steps:
and enhancing the original sample remote sensing image by adopting a data enhancement algorithm, wherein the data enhancement algorithm comprises any one or more of a cutting algorithm, a translation algorithm, a turning algorithm, a rotation algorithm, a noise addition algorithm, a scaling algorithm and a filtering algorithm.
According to an embodiment of the invention, the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer, and a plurality of densely connected blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image and enriching high-level semantic information of the image processed by the network;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the feature image information output by each layer to be repeatedly utilized so as to strengthen the transmission of the feature image information and deepen the network structure.
According to the embodiment of the invention, the identification of the ground features in the remote sensing image to be identified through the multi-scale dense convolution network specifically comprises the following steps:
inputting the remote sensing image to be identified into the multi-scale dense convolution network;
carrying out multi-scale feature extraction on the remote sensing image to be identified by a multi-scale feature conversion layer;
generating a feature image with semantic information through a plurality of down-sampling layers of an encoder structure, and repeatedly utilizing feature image information output by each layer by adopting a dense connecting block in the period;
mapping the characteristic image output by the encoder structure back to the size of the remote sensing image to be identified through a plurality of upper sampling layers of the encoder structure so as to carry out pixel-by-pixel classification, and using a dense connecting block to repeatedly utilize the characteristic image information output by each layer;
and segmenting the ground objects in the remote sensing image to be identified according to the pixel-by-pixel classification result, and identifying.
According to an embodiment of the present invention, the multi-scale feature conversion layer comprises a plurality of convolution layers of convolution kernels of different sizes, a plurality of batch normalization layers and a Concat connection layer,
the plurality of convolution layers are respectively used for extracting semantic information of a plurality of images;
the batch normalization layers are used for performing normalization processing on input data of each layer in the neural network training process;
and the Concat connecting layer is used for connecting the image semantic information extracted by the convolution kernels with different sizes.
Further, the multi-scale feature conversion layer comprises convolution layers of four convolution kernels with different sizes, wherein the four convolution kernels with different sizes are respectively 1 × 1 convolution kernel, 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel, wherein 1 × 1 convolution kernel is used for retaining original image information, and 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel are used for extracting various image semantic information.
According to the embodiment of the invention, any two layers in the dense connecting block are directly connected by the feature map, the feature output of any layer can be directly connected to all the subsequent layers, the three-dimensional features of each layer in the previous (l-1) layer of the l-th layer in the dense connecting block are used as input, and the size of the three-dimensional features extracted by the l-th layer is as follows: k is a radical ofl＝Fl([k0，k1，…，kl-1]) Wherein [ k ]0，k1，…，kl-1]The three-dimensional feature maps representing the layers are densely connected and the spatial size of any feature map is the same; function FlThe representation is a set of batch normalization, activation functions, and convolution operations.
The third aspect of the present invention further provides a computer readable storage medium, where the computer readable storage medium includes a remote sensing image feature identification method program, and when the remote sensing image feature identification method program is executed by a processor, the steps of the remote sensing image feature identification method are implemented.
The invention provides a remote sensing image surface feature identification method and system based on a multi-scale dense convolution network and a computer readable storage medium. The method is structurally expanded on the basis of a network of an encoder-decoder structure, firstly, the network is reconstructed by means of a dense hierarchical connection mode of a DenseNet network, the network depth is deepened, the transmission of characteristic information is enhanced, and image characteristic information extracted by a convolution kernel is greatly enriched by constructing a multi-scale characteristic conversion layer, so that the network achieves the purpose of high accuracy rate of remote sensing image ground object identification.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A remote sensing image surface feature identification method is characterized by comprising the following steps:
collecting an original sample remote sensing image for training;
carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
constructing a multi-scale dense convolutional network;
training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
2. The method for recognizing the ground features of the remote sensing image according to claim 1, wherein data enhancement processing is performed on the acquired original sample remote sensing image, and specifically comprises the following steps:
and enhancing the original sample remote sensing image by adopting a data enhancement algorithm, wherein the data enhancement algorithm comprises any one or more of a cutting algorithm, a translation algorithm, a turning algorithm, a rotation algorithm, a noise addition algorithm, a scaling algorithm and a filtering algorithm.
3. The remote sensing image surface feature identification method according to claim 1, wherein the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer and a plurality of dense connection blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the characteristic image information output by each layer to be repeatedly used.
4. The method for recognizing the surface features of the remote sensing image according to claim 3, wherein the recognizing the surface features in the remote sensing image to be recognized through the multi-scale dense convolution network specifically comprises:
inputting the remote sensing image to be identified into the multi-scale dense convolution network;
carrying out multi-scale feature extraction on the remote sensing image to be identified by a multi-scale feature conversion layer;
generating a feature image with semantic information through a plurality of down-sampling layers of an encoder structure, and repeatedly utilizing feature image information output by each layer by adopting a dense connecting block in the period;
mapping the characteristic image output by the encoder structure back to the size of the remote sensing image to be identified through a plurality of upper sampling layers of the encoder structure so as to carry out pixel-by-pixel classification, and using a dense connecting block to repeatedly utilize the characteristic image information output by each layer;
and segmenting the ground objects in the remote sensing image to be identified according to the pixel-by-pixel classification result, and identifying.
5. The remote sensing image surface feature identification method according to claim 3, wherein the multi-scale feature conversion layer comprises convolution layers of convolution kernels with different sizes, a plurality of batch normalization layers and a Concat connection layer,
the plurality of convolution layers are respectively used for extracting semantic information of a plurality of images;
the batch normalization layers are used for performing normalization processing on input data of each layer in the neural network training process;
and the Concat connecting layer is used for connecting the image semantic information extracted by the convolution kernels with different sizes.
6. The method for identifying the remote sensing image ground features according to claim 5, wherein the multi-scale feature conversion layer comprises convolution layers of four convolution kernels with different sizes, wherein the four convolution kernels with different sizes are 1 × 1 convolution kernel, 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel respectively, wherein the 1 × 1 convolution kernel is used for retaining original image information, and the 3 × 3 convolution kernel, 5 × 5 convolution kernel and 7 × 7 convolution kernel are used for extracting various image semantic information.
7. A remote sensing image surface feature identification method according to claim 3, characterized in that any two layers in the dense connection block are connected by a direct feature map, the feature output of any layer can be directly connected to all subsequent layers, the three-dimensional features of each layer in the previous (l-1) layer of the l-th layer in the dense connection block are used as input, and then the size of the three-dimensional features extracted from the l-th layer is as follows: k is a radical ofl＝Fl([k0，k1，…，kl-1]) Wherein [ k ]0，k1，…，kl-1]The three-dimensional feature maps representing the layers are densely connected and the spatial size of any feature map is the same; function FlThe representation is a set of batch normalization, activation functions, and convolution operations.
8. A remote sensing image feature recognition system is characterized by comprising: the remote sensing image ground feature identification method program comprises a memory and a processor, wherein the memory comprises the remote sensing image ground feature identification method program, and the remote sensing image ground feature identification method program realizes the following steps when being executed by the processor:
collecting an original sample remote sensing image for training;
carrying out data enhancement processing on the acquired original sample remote sensing image to obtain an enhanced sample remote sensing image;
constructing a multi-scale dense convolutional network;
training the multi-scale dense convolution network by combining the original sample remote sensing image and the enhanced sample remote sensing image;
and after the multi-scale dense convolution network training is finished, identifying the ground features in the remote sensing image to be identified through the multi-scale dense convolution network, and marking the identified ground features.
9. The remote sensing image surface feature identification system of claim 8, wherein the multi-scale dense convolutional network comprises an encoder structure, a decoder structure, a multi-scale feature conversion layer and a plurality of dense connection blocks;
an encoder structure comprising a plurality of downsampling layers for generating a feature image having semantic information;
a decoder structure comprising a plurality of upsampled layers for mapping the low resolution feature image output by the encoder structure back to the size of the input image for pixel-by-pixel classification;
the multi-scale feature conversion layer is used for carrying out multi-scale feature extraction on the remote sensing image;
and the dense connecting block is formed by a dense hierarchical network connection mode of DenseNet, and the hierarchical connection mode enables the characteristic image information output by each layer to be repeatedly used.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a remote sensing image feature identification method program, which when executed by a processor, implements the steps of a remote sensing image feature identification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910854251.9A CN112560544A (en) | 2019-09-10 | 2019-09-10 | Method and system for identifying ground object of remote sensing image and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910854251.9A CN112560544A (en) | 2019-09-10 | 2019-09-10 | Method and system for identifying ground object of remote sensing image and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112560544A true CN112560544A (en) | 2021-03-26 |
Family
ID=75028870
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910854251.9A Pending CN112560544A (en) | 2019-09-10 | 2019-09-10 | Method and system for identifying ground object of remote sensing image and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112560544A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113792640A (en) * | 2021-09-07 | 2021-12-14 | 海南大学 | DenseNet-based ocean remote sensing image noise identification method |
| CN113902018A (en) * | 2021-10-12 | 2022-01-07 | 深圳壹账通智能科技有限公司 | Image sample generation method and device, computer readable medium and electronic equipment |
| CN115810016A (en) * | 2023-02-13 | 2023-03-17 | 四川大学 | Lung infection CXR image automatic identification method, system, storage medium and terminal |
-
2019
- 2019-09-10 CN CN201910854251.9A patent/CN112560544A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113792640A (en) * | 2021-09-07 | 2021-12-14 | 海南大学 | DenseNet-based ocean remote sensing image noise identification method |
| CN113792640B (en) * | 2021-09-07 | 2023-07-14 | 海南大学 | Ocean remote sensing image noise identification method based on DenseNet |
| CN113902018A (en) * | 2021-10-12 | 2022-01-07 | 深圳壹账通智能科技有限公司 | Image sample generation method and device, computer readable medium and electronic equipment |
| CN115810016A (en) * | 2023-02-13 | 2023-03-17 | 四川大学 | Lung infection CXR image automatic identification method, system, storage medium and terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111915592B (en) | Remote sensing image cloud detection method based on deep learning | |
| CN109871798B (en) | Remote sensing image building extraction method based on convolutional neural network | |
| WO2019136623A1 (en) | Apparatus and method for semantic segmentation with convolutional neural network | |
| CN110889449A (en) | Edge-enhanced multi-scale remote sensing image building semantic feature extraction method | |
| CN112560544A (en) | Method and system for identifying ground object of remote sensing image and computer readable storage medium | |
| CN111860233B (en) | SAR image complex building extraction method and system based on attention network selection | |
| CN113468996B (en) | Camouflage object detection method based on edge refinement | |
| CN114187450A (en) | Remote sensing image semantic segmentation method based on deep learning | |
| CN110717921B (en) | Full convolution neural network semantic segmentation method of improved coding and decoding structure | |
| CN116453121B (en) | Training method and device for lane line recognition model | |
| CN110751111A (en) | Road extraction method and system based on high-order spatial information global automatic perception | |
| CN114972378A (en) | Brain tumor MRI image segmentation method based on mask attention mechanism | |
| CN114022408A (en) | Remote sensing image cloud detection method based on multi-scale convolution neural network | |
| CN116912257B (en) | Concrete pavement crack identification method based on deep learning and storage medium | |
| CN116205962A (en) | Monocular depth estimation method and system based on complete context information | |
| CN117788296B (en) | Infrared remote sensing image super-resolution reconstruction method based on heterogeneous combined depth network | |
| CN116452930A (en) | Multispectral image fusion method and multispectral image fusion system based on frequency domain enhancement in degradation environment | |
| CN102254306B (en) | Real-time image defogging method based on image simplified hierachical model | |
| Gao | A method for face image inpainting based on generative adversarial networks | |
| Liu et al. | AiOENet: All-in-one low-visibility enhancement to improve visual perception for intelligent marine vehicles under severe weather conditions | |
| CN114581762A (en) | Road extraction method based on multi-scale bar pooling and pyramid pooling | |
| CN116543165B (en) | Remote sensing image fruit tree segmentation method based on dual-channel composite depth network | |
| CN117115563A (en) | Remote sensing land coverage classification method and system based on regional semantic perception | |
| CN111274936B (en) | Multispectral image ground object classification method, system, medium and terminal | |
| CN117058367A (en) | Semantic segmentation method and device for high-resolution remote sensing image building |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |