CN114913434B - High-resolution remote sensing image change detection method based on global relation reasoning - Google Patents

High-resolution remote sensing image change detection method based on global relation reasoning Download PDF

Info

Publication number
CN114913434B
CN114913434B CN202210622122.9A CN202210622122A CN114913434B CN 114913434 B CN114913434 B CN 114913434B CN 202210622122 A CN202210622122 A CN 202210622122A CN 114913434 B CN114913434 B CN 114913434B
Authority
CN
China
Prior art keywords
network
change detection
features
remote sensing
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210622122.9A
Other languages
Chinese (zh)
Other versions
CN114913434A (en
Inventor
梁漪
张成坤
韩敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dalian University of Technology
Original Assignee
Dalian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dalian University of Technology filed Critical Dalian University of Technology
Priority to CN202210622122.9A priority Critical patent/CN114913434B/en
Publication of CN114913434A publication Critical patent/CN114913434A/en
Application granted granted Critical
Publication of CN114913434B publication Critical patent/CN114913434B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

A high-resolution remote sensing image change detection method based on global relation reasoning comprises the following steps: firstly, extracting multi-scale characteristics of a double-phase remote sensing image through a pre-trained encoder; secondly, using a global relationship reasoning module to respectively conduct relationship reasoning among the areas of each scale feature; and finally, establishing a decoder for multi-scale feature fusion, and generating a final change detection result through a semantic segmentation head. In addition to the local information acquired by the stacked convolution layers, the global semantic information which can represent the internal relation between the changed objects is fully considered, and in addition, the method is a network structure of the encoder-decoder, can realize detail information recovery, effectively weaken the interference of background noise and reduce the phenomenon of false detection. The method and the device fully utilize the multi-scale information and the global semantic information of the remote sensing image to generate the division characteristics with resolution effect, effectively improve the change detection precision and have wide application prospect.

Description

High-resolution remote sensing image change detection method based on global relation reasoning
Technical Field
The invention belongs to the technical field of remote sensing image processing, and relates to a high-resolution remote sensing image change detection method based on global relation reasoning. The method can be used for detecting the building change in two different simultaneous optical remote sensing images, and can be applied to land utilization, coverage, urban planning and the like.
Background
The primary objective of the remote sensing image change detection task is to make a decision on the change state of image pixels or objects by comparing time-spaced bi-temporal or multi-temporal images, so that important theoretical basis can be provided for upper management and decision of tasks such as urban space layout, land utilization detection, water body monitoring, disaster monitoring and the like. In recent years, with the rapid development of satellite sensor technology, the resolution of remote sensing images has been developed from tens and hundreds of meters in the past to the current meter and sub-meter levels, and the remote sensing images have a trend of high resolution. The high-resolution remote sensing images bring more technical challenges to the detection of the surface change, contain more surface information, have finer texture details of the ground features, have more complex topological relations among the ground features, and greatly increase the uncertainty of a change detection result. In addition, unlike the imaging mode of the common optical image, the remote sensing image is obtained from a satellite or an unmanned aerial vehicle at a high altitude to overlook the ground, and the process is easily influenced by factors such as time, light, environment and the like, and can cause interference such as shape distortion, overexposure, shadow and the like, so that the change detection difficulty is further increased.
With the development of big data and computing resources, deep learning technology is gradually introduced into the field of change detection and analysis. The deep network can learn abstract representation of data layer by layer, the nonlinear learning capability of the deep network is easier to process complex pattern recognition, and a practical solution is provided for high-resolution remote sensing image change detection. In the image field, the smallest learning unit of a depth network is a 2-dimensional convolution. By means of the translation invariance of convolution and the characteristic of being capable of smoothing noise, the spatial hierarchical structure of the mode can be learned by stacking multiple layers of convolution, and the extracted multi-level features contain rich local details. Therefore, various convolutional neural networks are designed for coping with the remote sensing image change detection task in many domestic and foreign researches.
Rodrigo CayeDaudt et al in paper "DaudtR,Bertrand LS,Alexandre B.Fully Convolutional Siamese Networks for Change Detection[C]//IEEEInternational Conference onImage Processing, ICIP 2018,Athens,Greece,October 7-10,2018:4063–4067." propose 3 full convolution (Fully Convolutional, FC) network structures, FC-EF, FC-Siam-Conc and FC-Siam-Diff, respectively. The FC-EF comprises an encoder and a decoder which can extract 4 scale features, and the double-time-phase images are spliced together and then input into a network, so that the FC-EF belongs to a single-input single-output structure; the FC-Siam-Conc comprises a twin encoder and a decoder, the twin-phase images are respectively input into the twin encoder, and the extracted channels with the same scale characteristic are connected and processed and then input into the decoder, so that the FC-Siam-Conc belongs to a double-input single-output structure; the FC-Siam-Diff also comprises a twin encoder and a decoder, wherein the double-phase images are respectively input into the twin encoder, and the difference value is calculated for the extracted same scale characteristic and then input into the decoder. Further, rodrigo CayeDaudt et al in paper "DaudtR, Bertrand LS,Alexandre B,et al.Multitask learning for large-scale semantic change detection[J]. Computer Vision and Image Understanding,2019,187:102783." modified the FC-EF network by adding a residual block at the tail of each layer of the FC-EF encoder to facilitate training of the network (i.e., FC-EF-Res).
Chen et al in paper "Chen H,Shi Z W.A Spatial-Temporal Attention-Based Method and a NewDataset for Remote SensingImage Change Detection[J].Remote Sensing,2020,12(10):1662." proposed a spatiotemporal attention network (Spatial-temporal Attention Network, STANet) that extracts image features based on ResNet convolutional networks and uses an attention mechanism to adapt the region of interest of the image of interest. Shi et al in paper "ShiQ,Liu M X,Li S C,et al.ADeeply Supervised Attention Metric-BasedNetwork and an Open Aerial Image Dataset forRemote Sensing Change Detection[J].IEEETransactions on Geoscience and Remote Sensing,2022,60:1–16." proposed a deep supervised attention metric Network (Deeply Supervised Attention Metric-based Network, DSAMNet) that also extracted image features based on ResNet convolutional networks and generated discriminative features using spatial attention and channel attention mechanisms.
However, the depth network of stacked convolution layers is limited to a small area for fine detail feature extraction, subject to the constraints of the convolution kernel receptive field. And the context linkage between the multi-scale features is not tight, meaning information is not fully utilized yet. It is well known that under the same scenario, the semantic properties of variant objects are related, i.e. the variant object bodies are consistent. If the semantic information can be better utilized, one or a small number of change objects are determined, and the rest change objects are further distinguished through semantic relations, so that the detection precision and the detection efficiency can be improved. Aiming at the defects of the prior art, a change detection method comprehensively utilizing global semantic information and local information is needed at present.
Disclosure of Invention
The optical remote sensing image has the advantages of wide scene coverage, complex background, shadow noise, overexposure, cloud layer shielding and the like caused by an imaging mode. Detecting some local changes in complex paired optical telemetry images remains an open and challenging task. Aiming at the problem that a plurality of change objects are difficult to accurately detect only through local detail information in a high-resolution remote sensing image, the invention provides an effective change detection method by considering global semantic information of the image and the inherent relation between the objects. The invention provides a high-resolution remote sensing image change detection method based on global relation reasoning, which can improve the change detection precision, aiming at the defect that the prior art focuses on the local detail information of the optical remote sensing image more and does not fully utilize global semantic information to carry out change detection.
In order to solve the problems, the invention adopts the following technical scheme:
A high-resolution remote sensing image change detection method based on global relation reasoning. The method not only obtains local information in the stacked convolution layers, but also fully considers global semantic information capable of representing internal relations between changed objects, and particularly adopts a global relation reasoning module to model semantic relations between different objects or areas on a remote sensing image feature map. In addition, the method is a network structure of the encoder-decoder, can realize detail information recovery, effectively weakens the interference of background noise, and reduces the phenomenon of false detection. The specific high-resolution remote sensing image change detection method based on global relation reasoning comprises the following steps:
first, sample division and data preprocessing.
1.1 Sample division). A pair of high-resolution remote sensing images covering the same ground surface at different moments { t 1,t2 } and corresponding actual labels are taken as one sample, and the collected samples are divided into a training set, a verification set and a test set;
1.2 Data preprocessing). The preprocessing process is divided into image clipping and standardization. Taking one sample as an example, cutting the image and the actual label into 256×256, and respectively normalizing the cut image channel by channel to obtain the processed image at time t 1 And image at time t 2/>Wherein C, H, W respectively represents the channel number, height and width of the image.
And secondly, constructing a change detection network architecture based on global relation reasoning.
2.1 A) constructing an encoder. Constructing twin encoders based on ResNet network for extracting respectivelyAndFeature maps at 4 scales, resulting in multi-scale features/>And
2.2 A global relationship inference module (Global Reasoning, gloRe) is constructed. Multi-scale features extracted from twin encodersAnd/>Respectively, to 4 GloRe. Each GloRe is connected in series with a convolution block with a core of 3x3, and the enhancement features/>And/>
2.3 A decoder is constructed. The main function of the decoder is the fusion of multi-scale enhancement features and the generation of difference features, with the following sub-steps:
2.3.1 Multi-scale enhanced feature fusion). And gradually approaching the upper-level scale features by the minimum scale features. I.e. And/>Generation and/> by fusion unitFusion features of equal size/>And/>Generation and/> by fusion unitFusion features of equal size/>And so on, finally generate and/>Fusion features of equal size/>Each fusion unit has the same structure to fuse/>And/>For example, the fusion process is described as follows:
1) Features to be characterized Input to an up-sampling layer using bilinear interpolation, then input to a convolution layer with a kernel of 3×3, and obtain the feature/>, by activating the function ReLU layerI.e./>
2) Features to be characterizedInput into a convolution layer with 3×3 kernel, and get features by activating function ReLU layerI.e./>
3) Merging in a channel cascadeAnd/>And is sent to a convolution block with a kernel of 1 multiplied by 1 to obtain fusion characteristics/>I.e./>
2.3.2 A difference feature is generated. Fused featuresAnd/>The difference is input into a kernel for obtaining difference characteristics F diff of a 3X 3 convolution block, namely/>
2.4 Construction of a change detection Head). Is formed by serially connecting an upsampling layer of bilinear interpolation and a convolution block and is used for generating a change detection graphThe method comprises the following substeps:
2.4.1 Up-sampling (scale factor 2, bilinear interpolation) the difference feature F diff, obtaining an updated feature F diff(1) by a convolution block (conv3×3+bn+relu);
2.4.2 Up-sampling (scaling factor 2, bilinear interpolation) the feature F diff(1), obtaining the feature F diff(2) by convolution block (conv3×3+bn+relu);
2.4.3 Feature F diff(2) is output by the convolutional layer (Conv1×1, output channel 1) and Sigmoid layer
Third, construct the loss function.
The Loss function is constructed as a combination of Dice Loss (Dice Loss) and binary cross entropy Loss (Binary CrossEntropy Loss, BCE Loss).
Fourth, network training and verification.
4.1 A) the base setting. Parameters required to initialize the network training process include iteration round (Epoch), batch size (Batchsize), initial learning rate (LEARNING RATE, LR). A learning rate update strategy is set, such as linear decay, exponential decay, fixed step decay, etc. The change detection network weights are updated using an adaptive moment estimation optimizer (Adaptive Moment Estimation, adam), setting its first and second moment attenuation coefficients β 1, β 2.
4.2 Network training). The network one-time training process corresponds to one Epoch, and comprises the following substeps:
4.2.1 Inputting a Batchsize-sized training sample into a change detection network based on global relation reasoning to obtain a change detection result;
4.2.2 Calculating the Dice Loss and the BCE Loss, transmitting the sum of the Dice Loss and the BCE Loss to an Adam optimizer to update the weight of the change detection network, and repeating the step 4.2.1) until the training of the network on all training samples is completed, thereby obtaining a trained change detection network under one Epoch.
4.3 Network authentication). Inputting the verification set to the trained network to obtain a change detection result of the verification set, and calculating an evaluation index of the network according to the change detection result and the actual sample label.
Fifth, the network training and verification process is repeated.
And (3) completing the network training of the Epoch times, and selecting the optimal verification result as the final network.
Compared with the prior art, the invention has the beneficial effects that:
(1) From the perspective of practicality and operability of the model, the invention constructs an end-to-end remote sensing image change detection network, which can acquire local information through an encoder and global semantic information through a global relation reasoning module, fully utilizes the local information and the semantic information to carry out change detection on the image, and overcomes the problems of missing and missed detection of a change object caused by low global modeling efficiency of the traditional convolutional neural network, so that the invention has the advantage of high detection precision.
(2) The detection network constructed by the invention uses decoding characteristics instead of encoding characteristics to calculate the change information, so that complex background and imaging interference in the image can be effectively avoided. And in consideration of the complementary effect of the multi-scale features, the multi-scale coding features are fused in the decoder, and can guide the detail recovery of the change object to resist the problem of the scale difference of the change object, so that the identification of the change object by the method is more accurate.
(3) The encoder constructed by the invention fuses the multi-scale characteristics of each image first and then calculates the difference characteristics, rather than calculating each layer of difference characteristics first and then fusing the multi-scale characteristics, can effectively prevent more salt and pepper noise from being generated, and has excellent change detection visual effect.
Drawings
FIG. 1 is a flow chart of a high resolution remote sensing image change detection method based on global relationship reasoning;
FIG. 2 is a block diagram of a high resolution remote sensing image change detection network based on global relationship reasoning;
FIG. 3 is a block diagram of a decoder;
FIG. 4 is a graph of network training loss variation;
FIG. 5 is a graph showing the change in the evaluation index of the network authentication process F1;
FIG. 6 (a) is a real tag of LEVIR test set;
FIG. 6 (b) is a graph showing the detection result of the change in FC-EF;
FIG. 6 (c) is a graph showing the detection result of the change in FC-EF-Conc;
FIG. 6 (d) is a graph showing the detection result of the change in FC-EF-Diff;
FIG. 6 (e) is a graph of the change detection result of the FC-EF-Res;
FIG. 6 (f) is a graph of the change detection result of STANet;
FIG. 6 (g) is a graph of the change detection result of DSAMNet;
Fig. 6 (h) is a graph of the change detection result of the present invention.
Detailed Description
In order to make the solution to the problems of the method, the method scheme adopted and the effect of the method achieved by the invention more clear, the invention is further described in detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. It should be further noted that, for convenience of description, only some, but not all of the matters related to the present invention are shown in the accompanying drawings.
1. Data and operating environment.
This example is illustrated using LEVIR change detection public data sets containing building changes. The hardware platform of the simulation experiment of this example is: CPU is Intel (R) Core (TM) i7-8700, main frequency is 3.2GHz, memory 16GB, GPU is NVIDIA GTX 1070, memory 8G. The software platform of the simulation experiment of this example is: python 3.8.
2. The implementation steps.
(1) Data acquisition and data preprocessing.
(1A) And (5) data acquisition. LEVIR remote sensing images acquired the changes in 2002-2018 of various buildings (villas or high-rise apartments) in 20 areas of multiple cities in texas, usa. The dataset contains 637 remote sensing images with resolution of 0.5 m/pixel and corresponding real labels, and has total of 3 channels of red, green and blue, and size 1024×1024. The dataset was represented at 7:1:2, respectively comprising 445, 64 and 128 pairs of remote sensing images.
(1B) And (5) preprocessing data. The preprocessing process is divided into image clipping and standardization. Because of the limitation of the GPU, the experiment cannot be directly trained on the whole image, so all samples are cut to 256×256 size, and the cut images are normalized channel by channel according to equation (1).
Wherein,For a training sample taken at time t 1、t2,/>Calculating the mean value of all images under red, green and blue channels at the moment, and calculating LEVIR datasets channel by channel, wherein the mean value is as followsCalculated/>, as the variance of all images under red, green, blue channels at time-
(2) The training dataset of LEVIR is input to a change detection network based on global relational reasoning for forward propagation.
FIG. 2 shows a global relationship reasoning-based high-resolution remote sensing image change detection network structure for a pair LEVIR of preprocessed imagesAnd/>For example, the forward propagation process of the network is as follows:
(2a) The multi-scale features are acquired by an encoder. To speed training, resNet18 pre-trained on the ImageNet dataset (containing 120 more than ten thousand images, for a total of 1000 categories) was used for encoding. ResNet18 separate extraction of And/>Features on 4 scales, i.e./>And/>Wherein/>And/>
(2B) The multi-scale enhanced features are obtained through a global relationship inference module (GloRe). GloRe essentially is that the local regions on the feature map are treated as nodes, and the relationships between the nodes represent the relationships between the local regions on the feature map. The present example sets the node numbers (N) of 4 GloRe to 128, 64, 32, and 16, respectively. Multi-scale features extracted by twin encoders And/>Enhancement features/> are output by GloRe and corresponding convolution blocks with a kernel of 3×3 (conv3×3+bn+relu), respectivelyAnd/>
(2C) The difference feature F diff is acquired by the decoder. Referring to fig. 3, the decoding process is as follows:
s2c.1 multiscale enhancement features are obtained by fusion units Equal-sized fusion featuresAnd gradually approaching the upper-level scale features by the minimum scale features. Specifically,/>And/>Feature generation by fusion unit FF 1 >And/>Feature generation by fusion unit FF 2 >And/>Feature generation by fusion unit FF 3 >The fusion units FF 1、FF2 and FF 3 have the same structure.
S2c.2 fusion feature extractionAnd/>Difference/>Input to the kernel to get difference features/>, for the 3 x 3 convolution block
(2D) Difference feature through change detection Head output change detection graph
(3) Calculating a predicted output according to (2)Loss from the genuine label Y.
Where L is the loss function, L Dice is DiceLoss, and L BCE is BCELoss. DiceLoss addresses the problem of class imbalance, expressed as shown in equation (3).
In the method, in the process of the invention,And Y ij represents the predicted value and the true value of the index position of the i row and the j column, respectively. Epsilon is a smoothing constant, preventing denominator from being 0, and epsilon=10 -5 is set in this example. BCELoss is used for calculating the matching degree between the prediction change detection result and the real label, and the expression is shown in the expression (4).
(4) And (5) performing iterative training and verification on the network.
(4A) And (5) setting parameters. The network training procedure set epoch=50, battsize=16, lr=0.001. The learning rate updating strategy adopts linear decay, specifically, the first 25 Epoch learning rates are kept unchanged, and the second 25 Epoch learning rates are linearly decayed to 0. Parameter β 1＝0.9,β2 = 0.999 for Adam optimizer.
(4B) And (5) performing iterative training on the network. The network one-time training process corresponds to one Epoch, and comprises the following substeps:
S4b.1, inputting a Batchsize training sample into a change detection network based on global relation reasoning to perform forward propagation, so as to obtain a change detection result;
And S4b.2, calculating corresponding loss, updating the network weight by using an Adam optimizer, and repeating the step S4b.1 until the training of the network in all training samples is completed, so as to obtain a well-trained change detection network under one Epoch.
(4C) And (5) network verification. And inputting the verification set to the network trained at the present time to obtain a change detection result of the verification set, and calculating an evaluation index F1 score according to the change detection result and the actual sample label (formula (6)). And repeating the network training and verification process to complete the network training of the Epoch times. From the rapid decay and stabilization of training loss of fig. 4, and the evaluation of the present example validation set F1 of fig. 5, the training network of round 50 can be determined to be the optimal network.
(5) And (5) network testing. The test set is input to the optimal network and the following evaluation index is calculated.
The Accuracy (Acc) measures the proximity of the predicted value to the true value, i.e., correctly predicts the ratio of the (un) variation category, and the expression is shown in equation (5).
Wherein TP, TN, FP, FN is true, false, and false, respectively.
The F1 score can be regarded as a harmonic mean of the precision and recall, expressed as shown in equation (6).
The average intersection ratio (Mean Intersection over Union, MIoU) is the ratio of the intersection and union of the two sets of the true value and the predicted value calculated, and the expression is shown in the expression (7).
Kappa coefficients (Kappa Coefficient, KC) measure the agreement of model predictions with actual class, differing from Acc in that Kappa coefficients can bias samples with class imbalance as shown in expression (8).
(6) Results were compared and analyzed.
Comparative analysis was performed with the present invention using 6 prior art techniques, all of which performed as a LEVIR test set as shown in Table 1. The invention refers to a remote sensing image change detection method based on global relation reasoning, FC-EF, FC-Siam-Conc, FC-Siam-Diff and FC-EF-Res refer to 4 full convolution change detection networks proposed by Daudt et al, STANet refers to a space-time attention network proposed by Chen et al, and DSAMNet refers to a depth supervision attention measurement network proposed by Shi et al.
TABLE 1 Performance evaluation Table of the present invention and existing remote sensing image Change detection model
As can be seen from the combination of the table 1, the invention obtains excellent detection results, and all indexes are improved more than other technologies, so that the invention can obtain higher change detection precision. In addition, fig. 6 shows the visualization result of the present invention on one of the test samples compared to the prior art, and it can be seen that the present invention can obtain a cleaner visual effect, and the changing object has a clearer edge and a better internal consistency.
Finally, it should be noted that: the above examples are only for the purpose of illustrating the embodiments of the present application, and it should be understood that the examples are only for the purpose of illustrating the present application and not for the purpose of limiting the scope of the present application, and that it is possible for those skilled in the art, after having read the present application, to make several variations and modifications without departing from the spirit of the application, and to make various equivalent modifications of the present application fall within the scope of the present application as defined in the appended claims.

Claims (3)

1. The high-resolution remote sensing image change detection method based on global relation reasoning is characterized by comprising the following steps of:
firstly, dividing a sample and preprocessing data;
1.1 Sample division;
1.2 Data preprocessing to obtain an image at time t 1 after processing And the image at time t 2 Wherein C, H, W represents the channel number, height and width of the image respectively;
secondly, constructing a change detection network architecture based on global relation reasoning;
2.1 -constructing an encoder; constructing twin encoders based on ResNet network for extracting respectively And/>Feature maps at 4 scales, resulting in multi-scale features/>And
2.2 A global relationship inference module GloRe is constructed; multi-scale features extracted from twin encodersAnd(. Epsilon. { t 1,t2 }) respectively input to 4 pieces of GloRe; each GloRe is connected in series with a convolution block with a core of 3 x 3, and the enhancement features/>And/>
2.3 A decoder is built for fusion of multi-scale enhancement features and generation of difference features, with the sub-steps of:
2.3.1 Multi-scale enhanced feature fusion; gradually converging the minimum scale feature to the previous level scale feature; i.e. AndGeneration and/> by fusion unitFusion features of equal size/>And/>Generation and/> by fusion unitFusion features of equal size/>And so on, finally generate and/>Fusion features of equal size/>Each fusion unit has the same structure to fuse/>And/>Illustratively, the fusion process is described as follows:
1) Features to be characterized Input to an up-sampling layer using bilinear interpolation, then input to a convolution layer with a kernel of 3×3, and obtain the feature/>, by activating the function ReLU layerI.e./>
2) Features to be characterizedInput into a convolution layer with 3×3 kernel, and get the feature/>, by activating function ReLU layerI.e.
3) Merging in a channel cascadeAnd/>And is fed to a kernel 1 x1 convolution block to obtain fusion characteristicsI.e./>
2.3.2 Generating a difference feature; fused featuresAnd/>The difference is input into a kernel for obtaining difference characteristics F diff of a 3X 3 convolution block, namely/>
2.4 Constructing a change detection Head; is formed by serially connecting an upsampling layer of bilinear interpolation and a convolution block and is used for generating a change detection graphThe method comprises the following substeps:
2.4.1 Up-sampling the difference feature F diff to obtain an updated feature F diff(1) through a convolution block;
2.4.2 Up-sampling the characteristic F diff(1), and obtaining the characteristic F diff(2) through a convolution block;
2.4.3 Feature F diff(2) output through convolutional and Sigmoid layers
Thirdly, constructing a loss function;
Constructing a Loss function in a combination mode of Dice Loss and binary cross entropy Loss BCE Loss;
fourthly, training and verifying the network;
4.1 Basic setting, including initializing parameters required by a network training process, setting a learning rate updating strategy, updating a change detection network weight by using an adaptive moment estimation optimizer Adam, and setting a first moment attenuation coefficient beta 1 and a second moment attenuation coefficient beta 2;
4.2 Network training; the network one-time training process corresponds to one Epoch, and comprises the following substeps:
4.2.1 Inputting a Batchsize-sized training sample into a change detection network based on global relation reasoning to obtain a change detection result;
4.2.2 Calculating the Dice Loss and the BCE Loss, transmitting the sum of the Dice Loss and the BCE Loss to an Adam optimizer to update the weight of the change detection network, and repeating the step 4.2.1) until the training of the network on all training samples is completed, so as to obtain a trained change detection network under one Epoch;
4.3 Network authentication; inputting the verification set to a trained network to obtain a change detection result of the verification set, and calculating an evaluation index of the network according to the change detection result and an actual sample label;
fifth, repeating the network training and verification process;
And (3) completing the network training of the Epoch times, and selecting the optimal verification result as the final network.
2. The method for detecting the change of the high-resolution remote sensing image based on the global relation reasoning according to claim 1, wherein in the step 1.1), the sample division is specifically as follows: a pair of high-resolution remote sensing images covering the same ground surface at different moments { t 1,t2 } and corresponding actual labels are taken as one sample, and the collected samples are divided into a training set, a verification set and a test set.
3. The method for detecting the change of the high-resolution remote sensing image based on the global relation reasoning according to claim 1, wherein in the step 1.2), the data preprocessing process is divided into image clipping and standardization.
CN202210622122.9A 2022-06-02 2022-06-02 High-resolution remote sensing image change detection method based on global relation reasoning Active CN114913434B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210622122.9A CN114913434B (en) 2022-06-02 2022-06-02 High-resolution remote sensing image change detection method based on global relation reasoning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210622122.9A CN114913434B (en) 2022-06-02 2022-06-02 High-resolution remote sensing image change detection method based on global relation reasoning

Publications (2)

Publication Number Publication Date
CN114913434A CN114913434A (en) 2022-08-16
CN114913434B true CN114913434B (en) 2024-06-11

Family

ID=82771619

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210622122.9A Active CN114913434B (en) 2022-06-02 2022-06-02 High-resolution remote sensing image change detection method based on global relation reasoning

Country Status (1)

Country Link
CN (1) CN114913434B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115526886B (en) * 2022-10-26 2023-05-26 中国铁路设计集团有限公司 Optical satellite image pixel level change detection method based on multi-scale feature fusion
CN117319610B (en) * 2023-11-28 2024-01-30 松立控股集团股份有限公司 Smart city road monitoring method based on high-order panoramic camera region enhancement

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063569A (en) * 2018-07-04 2018-12-21 北京航空航天大学 A kind of semantic class change detecting method based on remote sensing image
AU2020103901A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Image Semantic Segmentation Method Based on Deep Full Convolutional Network and Conditional Random Field
CN113420662A (en) * 2021-06-23 2021-09-21 西安电子科技大学 Remote sensing image change detection method based on twin multi-scale difference feature fusion
CN113706482A (en) * 2021-08-16 2021-11-26 武汉大学 High-resolution remote sensing image change detection method
CN113936217A (en) * 2021-10-25 2022-01-14 华中师范大学 Priori semantic knowledge guided high-resolution remote sensing image weakly supervised building change detection method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109063569A (en) * 2018-07-04 2018-12-21 北京航空航天大学 A kind of semantic class change detecting method based on remote sensing image
AU2020103901A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Image Semantic Segmentation Method Based on Deep Full Convolutional Network and Conditional Random Field
CN113420662A (en) * 2021-06-23 2021-09-21 西安电子科技大学 Remote sensing image change detection method based on twin multi-scale difference feature fusion
CN113706482A (en) * 2021-08-16 2021-11-26 武汉大学 High-resolution remote sensing image change detection method
CN113936217A (en) * 2021-10-25 2022-01-14 华中师范大学 Priori semantic knowledge guided high-resolution remote sensing image weakly supervised building change detection method

Also Published As

Publication number Publication date
CN114913434A (en) 2022-08-16

Similar Documents

Publication Publication Date Title
CN111738124B (en) Remote sensing image cloud detection method based on Gabor transformation and attention
CN112668494A (en) Small sample change detection method based on multi-scale feature extraction
CN114913434B (en) High-resolution remote sensing image change detection method based on global relation reasoning
CN110705457A (en) Remote sensing image building change detection method
Alsabhan et al. Automatic building extraction on satellite images using Unet and ResNet50
CN109871875B (en) Building change detection method based on deep learning
CN110533631A (en) SAR image change detection based on the twin network of pyramid pondization
Zhang et al. A GANs-based deep learning framework for automatic subsurface object recognition from ground penetrating radar data
Chen et al. Unsupervised multimodal change detection based on structural relationship graph representation learning
CN113033520A (en) Tree nematode disease wood identification method and system based on deep learning
CN109635726B (en) Landslide identification method based on combination of symmetric deep network and multi-scale pooling
CN114283285A (en) Cross consistency self-training remote sensing image semantic segmentation network training method and device
CN113988147B (en) Multi-label classification method and device for remote sensing image scene based on graph network, and multi-label retrieval method and device
Liu et al. Survey of road extraction methods in remote sensing images based on deep learning
CN114037891A (en) High-resolution remote sensing image building extraction method and device based on U-shaped attention control network
CN115311502A (en) Remote sensing image small sample scene classification method based on multi-scale double-flow architecture
Shang et al. Spatiotemporal reflectance fusion using a generative adversarial network
CN114332075A (en) Rapid structural defect identification and classification method based on lightweight deep learning model
Seydi et al. BDD-Net+: A building damage detection framework based on modified coat-net
CN117788810A (en) Learning system for unsupervised semantic segmentation
CN114511787A (en) Neural network-based remote sensing image ground feature information generation method and system
CN116580243A (en) Cross-domain remote sensing scene classification method for mask image modeling guide domain adaptation
CN115829996A (en) Unsupervised synthetic aperture radar image change detection method based on depth feature map
Tahraoui et al. Land change detection in sentinel-2 images using ir-mad and deep neural network
CN112507826B (en) End-to-end ecological variation monitoring method, terminal, computer equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant