CN111339827A - SAR image change detection method based on multi-region convolutional neural network - Google Patents

SAR image change detection method based on multi-region convolutional neural network Download PDF

Info

Publication number
CN111339827A
CN111339827A CN202010056678.7A CN202010056678A CN111339827A CN 111339827 A CN111339827 A CN 111339827A CN 202010056678 A CN202010056678 A CN 202010056678A CN 111339827 A CN111339827 A CN 111339827A
Authority
CN
China
Prior art keywords
layer
data set
region
training
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010056678.7A
Other languages
Chinese (zh)
Inventor
高峰
吕越
董军宇
张珊
杨冰冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ocean University of China
Original Assignee
Ocean University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ocean University of China filed Critical Ocean University of China
Priority to CN202010056678.7A priority Critical patent/CN111339827A/en
Publication of CN111339827A publication Critical patent/CN111339827A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G06F18/2193—Validation; Performance evaluation; Active pattern learning techniques based on specific statistical tests
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Probability & Statistics with Applications (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

A SAR image change detection method based on a multi-region convolutional neural network comprises the following steps: carrying out difference analysis on two SAR images in the same geographical position and different time phases to obtain a difference image; pre-classifying the differential image to obtain a constructed training data set and a constructed test data set; sending the sample training data set into a proposed multi-region convolutional neural network for training; and using the trained network for a test set to test so as to obtain a change detection result of the whole same-place multi-temporal SAR image. According to the method, the number of the data sets is doubled by adding Gaussian noise when the data sets are constructed, the diversity of samples is enriched, and the over-fitting problem is solved; meanwhile, the method uses an attention mechanism for channels and spaces to improve the performance of the network, improves the robustness of SAR image change detection on noise, and has strong generalization capability.

Description

SAR image change detection method based on multi-region convolutional neural network
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a Synthetic Aperture Radar (SAR) image change detection method based on a multi-region convolutional neural network.
Background
Image change detection techniques aim to detect changes that occur between images of the same scene at different time periods. Image change detection techniques rely primarily on changes in radiation values or local texture. These variations may be due to real variations in the surface coverage or due to illumination angle, atmospheric conditions, sensor accuracy. The ground humidity and other conditions. The basic premise of change detection is that the change in radiation value or local texture caused by the change in the object itself is separable from the change caused by some random factor. With the development of satellite remote sensing technology, in recent years, the acquisition amount of remote sensing data is increased by geometric multiples, and the synthetic aperture radar has all-weather and all-day imaging capability and can not be influenced by weather conditions, so that the synthetic aperture radar becomes a current research hotspot. In the 60 s of the 20 th century, more and more satellites carrying high-resolution synthetic aperture radars have been launched in countries around the world, such as: radarsat-2, TerrasAR-X, and the like. In 2016, 8 months, China successfully launches a high-resolution third satellite, can obtain an SAR image with the resolution of 1 meter, and has clear imaging, distinct layers and rich information. The synthetic aperture radar has the characteristics of all weather and all time, and can conveniently obtain images of the same area in different time periods, so that SAR image change detection is an important application field of the SAR image change detection. SAR image change detection is one of the main applications of remote sensing technology, and obtains the change information required by people according to the difference between images through comparative analysis of images in different periods. SAR image change detection has been applied to many aspects, such as monitoring of regions with harsh natural conditions, such as tropical rainforests, deserts and the like, where manual detection is difficult, to understand the change of ecological environment; monitoring the farmland and analyzing the growth condition of crops; monitoring military targets, and knowing information such as force deployment, military maneuver and the like; the urban environment is monitored, urban layout is reasonably planned, land use management and standardization are carried out, illegal land occupation and illegal buildings are monitored, and the like. With the development of remote sensing observation technology and SAR imaging technology, the capability of acquiring multi-band, multi-polarization and multi-period SAR images is greatly improved, and a rich information source can be provided for the research of the dynamic process problems. However, SAR images have a large amount of speckle noise, and current methods often have difficulty accurately detecting changing regions in the images.
In recent years, many studies on change detection of SAR images have been made, and the methods can be mainly classified into unsupervised methods and supervised methods according to whether a priori knowledge is used. (1) The accuracy of the unsupervised method greatly depends on the data distribution of the image, if the data distribution is reasonable, the traditional threshold segmentation and clustering method can obtain a better result, but the unsupervised method has poor noise robustness and adaptability. (2) The supervised learning method can often obtain more effective results, such as learning models of a limited boltzmann machine, an extreme learning machine, a convolutional neural network and the like, but the supervised learning method needs a large number of label samples to be used for model training, and is difficult to obtain excellent performance under the conditions of poor label quality and insufficient quantity, and in addition, the generalization capability of the model is greatly influenced due to the influence of noise. In a word, when the change detection is performed on the multi-time SAR image, the current method is easily affected by noise, and accurate change information is difficult to obtain.
Disclosure of Invention
The embodiment of the invention provides a remote sensing image change detection method based on a multi-region convolutional neural network, so as to improve the accuracy and performance of SAR image change detection. The technical scheme of the embodiment of the invention is realized as follows:
the SAR image change detection method based on the multi-region convolutional neural network comprises the following steps:
carrying out difference analysis on two multi-temporal SAR images at the same place to obtain a difference image;
pre-classifying the difference images to obtain a training data set and a test data set for constructing a model;
gaussian noise is added to the obtained training data set, training samples are enriched, and the problem of model overfitting is solved;
using the obtained training data set for training the multi-region convolutional neural network;
and (4) using the test data set for testing the multi-region convolutional neural network so as to obtain a change detection result of the whole image.
The method comprises the following specific steps:
(1) carrying out difference analysis on two multi-temporal SAR images at the same place to obtain a difference image:
performing difference analysis on the two multi-temporal SAR images by using a log ratio to obtain a differential image of the multi-temporal SAR images;
the calculation process of the differential image comprises the following steps:
IDI＝|logI1-logI2|
wherein, I1And I2Respectively representing two multi-temporal SAR images, IDIThe method comprises the steps that differential images of two multi-temporal SAR images are represented, | · | is absolute value operation, and log represents logarithm operation with 10 as a base;
(2) for differential image IDIPre-classifying to construct a training data set and a test data set;
(2.1) pre-classifying the difference images by using a multilayer fuzzy C mean value clustering algorithm to obtain a pseudo label matrix;
(2.2) extracting the spatial positions marked as 0 and 1 in the pseudo label matrix, taking neighborhood pixels of L × L around pixel points corresponding to the spatial positions marked as 0 and 1 in the differential image obtained in the step 1 as a training data set, wherein the value of L is an odd number not less than 3 (the optimal value is 11), and the number of samples in the obtained training data set is marked as T1Each sample of the training data set is a block of pixels of size L × L;
(2.3) extracting the spatial position marked as 0.5 in the pseudo label matrix, taking neighborhood pixels of L × L around a pixel point corresponding to the spatial position marked as 0.5 in the differential image obtained in the step 1 as a test data set, wherein the value of L is an odd number not less than 3 (the optimal value is 11),the number of samples in the obtained test data set is recorded as T2Each sample of the test data set is a block of pixels of size L × L;
the method is characterized by further comprising the following steps:
(2.4) respectively carrying out region segmentation on each sample of the training data set and the test data set according to an upper mode, a lower mode, a left mode and a right mode, namely respectively taking an L × k region which is positioned above the pixel block and is larger than 1/2 pixel blocks, taking an L × k region which is positioned below the pixel block and is larger than 1/2 pixel blocks, taking an L × L region which is positioned at the left side of the pixel block and is larger than 1/2 pixel blocks and taking an L × L region which is positioned at the right side of the pixel block and is larger than 1/2 pixel blocks, namely the two regions which are vertically segmented are the same in size, the two regions which are horizontally segmented are the same in size, the values of k and L depend on the dimension L of the pixel blocks, and [ L/2] is]≤k＜L，[L/2]≤l＜L；[L/2]Watch (A) Show greater thanL/2Minimum integer of (2)
Wherein k and l optimal values of 11 for the upper and lower regions are 11 and 7, respectively, and k and l optimal values of 11 for the left and right regions are 7 and 11, respectively;
(3) gaussian noise is added to the training data set and the test data set, training samples are enriched, and the problem of model overfitting is relieved;
(4) and (3) applying the training data set obtained in the step (3) after the Gaussian noise is added to the training of the multi-region convolutional neural network:
(4.1) constructing a multi-region convolutional neural network by using Tensorflow, wherein the constructed network consists of five branch networks, each branch respectively processes five different regions of the same pixel block, namely the upper region, the lower region, the left region, the right region and the global region, and the five branch networks of the multi-region convolutional neural network have the following structures: input layer → low-layer convolution layer → middle-layer convolution layer → high-layer convolution layer → full-connected layer; the input layer is the training sample obtained in the step 3; each branch network adopts the operation process of the following steps 4.2 to 4.5;
(4.2) for each branch network, its lower convolution layer extracts the lower features F of the input layerL：
The low-layer convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nLWherein m is 3, nLIs a power exponent of 2 of minimum 128;
low level feature FLThe calculation process of (2) is as follows:
FL＝σ(XW1+b1)
wherein X represents the input data of the lower convolutional layer, the data is one of the five branches of the training set obtained in step 3 after adding Gaussian noise, σ represents the ReLU activation function, and W1And b1Representing weights and offsets for the lower convolutional layers;
(4.3) extracting the middle layer feature F of the input data through the middle layer convolution layer of the branched networkM：
The middle layer convolution layer has convolution kernel size of m × m and convolution kernel number of nMWherein m is 3, nMIs a power exponent of 2 of minimum 128;
middle layer characteristic FMThe calculation process of (2) is as follows:
FM＝σ(FLW2+b2)
wherein, FMDenotes the middle level characteristics, σ denotes the ReLU activation function, W2And b2Representing weights and offsets of the middle convolutional layer;
(4.4) extracting the high-level features F of the input data through the high-level convolution layer of the branched networkH：
The high-level convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nHWherein m is 1, nHIs a power exponent of 2 of minimum 128;
high level feature FHThe calculation process of (2) is as follows:
FH＝σ(FMW3+b3)
wherein, FHRepresenting high level features, σ represents the ReLU activation function, W3And b3Weights and biases representing higher convolutional layers;
(4.5) construction of the dimension matching function G1And G2Thus through the dimension matching function G1And G2For low layer characteristic FLMiddle layer characteristic FMAnd high layer characteristics FHThe characteristic fusion is carried out, and the characteristic fusion is carried out,
wherein the dimension matches the function G1Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 5 and n is a power exponent of 2 of 64, and a dimension matching function G2Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 3 and n is a power exponent of 2 of minimum 64;
obtaining a fused feature map Fm，FmSize is FL、FM、FHThe size after stacking is calculated as:
Fm＝G1(FL)+G2(FM)+FH
(4.6) the number of samples of the training data set added with the Gaussian noise obtained in the step 3 is T1After 4.1-4.5 operations are respectively carried out on the upper, lower, left, right and global five-class regions of each training sample of the training data set, the fusion feature maps of the upper, lower, left, right and global five-class regions of each training sample are output and are respectively marked as FUp、FBottom、FLeft、FRight、FFull；
(4.7) feature F for the upper regionUpLower region characteristic FBottom, left region feature FLeftRight region feature FRightAnd performing feature information fusion on the global features FFull to obtain a fusion feature map F of the input dataA：
The fusion characteristic diagram FAThe calculation process of (2) is as follows:
firstly, obtaining a fusion feature map F of the five types of regionsmCarrying out dimension conversion to obtain a one-dimensional vector, wherein the process is as follows:
FT＝T(Fm)
wherein, FTRepresenting a feature diagram after being converted into a one-dimensional vector, wherein T (-) represents a dimension conversion function;
(4.8) converting the one-dimensional feature F of each directionTU、FTB、FTL、FTR、FTFAccording to the splicing, the spliced characteristic diagram is marked as F, and then the F is subjected to two-layer full-connection operation to obtain R:
R＝δ(Wfc2(σ(Wfc1F)))
wherein Wfc1Denotes a first layer fully connected operation, Wfc2Representing the second layer full join operation, delta representing the Softmax function, sigma representing the ReLU activation function, and a vector with dimension 2 × 1 for R after full join
Figure BDA0002373133540000051
Wherein a represents the probability that a training sample belongs to the invariant class and b represents the probability that a training sample belongs to the variant class, based on the vector
Figure BDA0002373133540000052
To output a prediction tag of the ith sample
Figure BDA0002373133540000053
i＝1,2,3,...,T1；
When a > b
Figure BDA0002373133540000054
Is equal to the class to which a belongs i.e
Figure BDA0002373133540000055
Is 0, when a < b
Figure BDA0002373133540000056
Is equal to the class to which b belongs i.e
Figure BDA0002373133540000057
Is 1;
(4.9) calculating a cross entropy loss function of the multi-region convolutional neural network, wherein the calculation process of the loss function is as follows:
Figure BDA0002373133540000058
wherein, yiFor training of step 2.2Real tag, y, of the ith sample in the training dataseti1 denotes that the label of the input data is 1, i.e. the position pixel is changed, yi0 means that the label of the input data is 0, i.e. the position pixel is unchanged,
Figure BDA0002373133540000059
a prediction tag, T, representing the ith sample1For the number of samples of the training data set, i is the sample of the training, i ═ 1,2,31Log denotes base 10 logarithmic operation;
then optimizing parameters of the multi-region convolutional neural network by using a Stochastic Gradient Descent (SGD) algorithm;
(5) inputting the test data set in the step 2.3 into the optimized multi-region convolutional neural network, and obtaining T related to the test data set according to the process from the step 4.2 to the step 4.82A prediction tag;
(6) combining the training data set in step 2.2 and T obtained in step 52And (4) obtaining a change result graph of the place in the step 1 by the prediction label.
According to the remote sensing image change detection method based on the multi-region convolutional neural network, the images are processed through the difference analysis and the independence analysis, and the characteristics of high classification precision and low noise sensitivity of a multi-region convolutional neural network classifier are utilized. The remote sensing image change detection method based on the multi-region convolutional neural network has the following advantages:
1. and carrying out difference analysis on the multi-temporal SAR images to obtain difference images of the two images, and carrying out pre-classification by using an FCM fuzzy clustering algorithm to obtain a pseudo tag matrix. The differential image can effectively inhibit noise interference and improve the performance of the pre-classification algorithm.
2. The multi-region convolutional neural network utilizes the residual block to continuously improve the representativeness of extracted features by fusing the low-layer, middle-layer and high-layer feature features of different regions, thereby obtaining more robust feature representation and improving the precision of a change detection method.
3. The data enhancement means can increase the number of samples of the training network on the basis of limited samples, prevent the overfitting problem and improve the generalization capability of the network.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a diagram illustrating an image processing method according to the present invention;
FIG. 3 is a schematic diagram of a data set partitioning method according to the present invention;
FIG. 4 is a diagram illustrating a multi-region convolutional neural network structure according to the present invention;
FIG. 5 is a schematic diagram of a regional feature extractor of the present invention;
FIG. 6 is a schematic diagram of input data according to the present invention;
FIG. 7 is a graph comparing the effects of the method of the embodiment with those of the prior art.
To clearly illustrate the structure of embodiments of the present invention, certain dimensions, structures and devices are shown in the drawings, which are for illustrative purposes only and are not intended to limit the invention to the particular dimensions, structures, devices and environments, which may be adjusted or modified by one of ordinary skill in the art according to particular needs and are still included in the scope of the appended claims.
Detailed Description
In the following description, various aspects of the invention will be described, but it will be apparent to those skilled in the art that the invention may be practiced with only some or all of the structures or processes of the invention. Specific numbers, configurations and sequences are set forth in order to provide clarity of explanation, but it will be apparent that the invention may be practiced without these specific details. In other instances, well-known features have not been set forth in detail in order not to obscure the invention.
Referring to fig. 1, the method comprises the following specific steps:
step 1: carrying out difference analysis on two multi-temporal SAR images at the same place to obtain a difference image:
performing difference analysis on the two multi-temporal SAR images by using a log ratio to obtain a differential image of the multi-temporal SAR images;
the calculation process of the differential image comprises the following steps:
IDI＝|logI1-logI2|
wherein, I1And I2Respectively representing two multi-temporal SAR images, IDIThe method comprises the steps that differential images of two multi-temporal SAR images are represented, | · | is absolute value operation, and log represents logarithm operation with 10 as a base;
step 2: for differential image IDIPre-classifying to construct a training data set and a test data set;
step 2.1: pre-classifying the difference image by using a multilayer fuzzy C-means clustering algorithm to obtain a pseudo tag matrix;
step 2.2, extracting the spatial positions marked as 0 and 1 in the pseudo label matrix, taking neighborhood pixels of L × L around pixel points corresponding to the spatial positions marked as 0 and 1 in the differential image obtained in the step 1 as a training data set, wherein the value of L is an odd number not less than 3 (the optimal value is 11), and the number of samples in the obtained training data set is marked as T1Each sample of the training data set is a block of pixels of size L × L;
step 2.3, extracting the spatial position marked as 0.5 in the pseudo label matrix, taking neighborhood pixels of L × L around a pixel point corresponding to the spatial position marked as 0.5 in the differential image obtained in the step 1 as a test data set, wherein the value of L is an odd number not less than 3 (the optimal value is 11), and the number of samples in the obtained test data set is marked as T2Each sample of the test data set is a block of pixels of size L × L;
the method is characterized by further comprising the following steps:
step 2.4, performing region segmentation on each sample of the training data set and the test data set respectively according to an upper mode, a lower mode, a left mode and a right mode, namely respectively taking an L × k region which is positioned above the pixel block and is larger than 1/2 pixel blocks, taking an L × k region which is positioned below the pixel block and is larger than 1/2 pixel blocks, and taking L × which is positioned at the left side of the pixel block and is larger than 1/2 pixel blocksThe L area is an L × L area which is positioned on the right side of the pixel block and is larger than 1/2 pixel blocks, namely the two areas divided up and down are the same in size, the two areas divided left and right are the same in size, the values of k and L depend on the size L of the pixel block, and [ L/2]]≤k＜L，[L/2]≤l＜L；[L/2]Means greater thanL/2Minimum integer of (2)
Wherein k and l optimal values of 11 for the upper and lower regions are 11 and 7, respectively, and k and l optimal values of 11 for the left and right regions are 7 and 11, respectively;
and step 3: gaussian noise is added to the training data set and the test data set, training samples are enriched, and the problem of model overfitting is relieved;
and 4, step 4: and (3) applying the training data set obtained in the step (3) after the Gaussian noise is added to the training of the multi-region convolutional neural network:
step 4.1: utilize Tensorflow to construct multizone convolutional neural network, the network of constructing comprises five branch networks, and every branch handles five different areas of upper and lower, left and right and global of same pixel block respectively, and five branch networks's of multizone convolutional neural network structure is: input layer → low-layer convolution layer → middle-layer convolution layer → high-layer convolution layer → full-connected layer; the input layer is the training sample obtained in the step 3; each branch network adopts the operation process of the following steps 4.2 to 4.5;
step 4.2: for each branch network, its lower convolution layer extracts the lower features F of the input layerL：
The low-layer convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nLWherein m is 3, nLIs a power exponent of 2 of minimum 128;
low level feature FLThe calculation process of (2) is as follows:
FL＝σ(XW1+b1)
wherein X represents the input data of the lower convolutional layer, the data is one of the five branches of the training set obtained in step 3 after adding Gaussian noise, σ represents the ReLU activation function, and W1And b1Weight sum representing lower convolutional layerBiasing;
step 4.3: extracting middle layer feature F of input data through middle layer convolution layer of the branch networkM：
The middle layer convolution layer has convolution kernel size of m × m and convolution kernel number of nMWherein m is 3, nMIs a power exponent of 2 of minimum 128;
middle layer characteristic FMThe calculation process of (2) is as follows:
FM＝σ(FLW2+b2)
wherein, FMDenotes the middle level characteristics, σ denotes the ReLU activation function, W2And b2Representing weights and offsets of the middle convolutional layer;
step 4.4: extracting high-level features F of input data through high-level convolutional layers of the branched networkH：
The high-level convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nHWherein m is 1, nHIs a power exponent of 2 of minimum 128;
high level feature FHThe calculation process of (2) is as follows:
FH＝σ(FMW3+b3)
wherein, FHRepresenting high level features, σ represents the ReLU activation function, W3And b3Weights and biases representing higher convolutional layers;
step 4.5: constructing a dimension matching function G1And G2Thus through the dimension matching function G1And G2For low layer characteristic FLMiddle layer characteristic FMAnd high layer characteristics FHThe characteristic fusion is carried out, and the characteristic fusion is carried out,
wherein the dimension matches the function G1Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 5 and n is a power exponent of 2 of 64, and a dimension matching function G2Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 3 and n is a power exponent of 2 of minimum 64;
obtaining a fused feature map Fm，FmSize is FL、FM、FHThe size after stacking is calculated as:
Fm＝G1(FL)+G2(FM)+FH
step 4.6: the number of samples of the training data set added with the Gaussian noise obtained in the step 3 is T1After 4.1-4.5 operations are respectively carried out on the upper, lower, left, right and global five-class regions of each training sample of the training data set, the fusion feature maps of the upper, lower, left, right and global five-class regions of each training sample are output and are respectively marked as FUp、FBottom、FLeft、FRight、FFull；
Step 4.7: for the upper region feature FUpLower region characteristic FBottomLeft region feature FLeftRight region feature FRightAnd global feature FFullCarrying out feature information fusion to obtain a fusion feature map F of input dataA：
The fusion characteristic diagram FAThe calculation process of (2) is as follows:
firstly, obtaining a fusion feature map F of the five types of regionsmCarrying out dimension conversion to obtain a one-dimensional vector, wherein the process is as follows:
FT＝T(Fm)
wherein, FTRepresenting a feature diagram after being converted into a one-dimensional vector, wherein T (-) represents a dimension conversion function;
step 4.8: one-dimensional features F of each directionTU、FTB、FTL、FTR、FTFAccording to the splicing, the spliced characteristic diagram is marked as F, and then the F is subjected to two-layer full-connection operation to obtain R:
R＝δ(Wfc2(σ(Wfc1F)))
wherein Wfc1Denotes a first layer fully connected operation, Wfc2Representing the full-join operation of the second layer, delta representing a Softmax function, sigma representing a ReLU activation function, and the dimension of R after full-join beingVector of 2 × 1
Figure BDA0002373133540000091
Wherein a represents the probability that a training sample belongs to the invariant class and b represents the probability that a training sample belongs to the variant class, based on the vector
Figure BDA0002373133540000092
To output a prediction tag of the ith sample
Figure BDA0002373133540000093
i＝1,2,3,...,T1；
When a > b
Figure BDA0002373133540000094
Is equal to the class to which a belongs i.e
Figure BDA0002373133540000095
Is 0, when a < b
Figure BDA0002373133540000096
Is equal to the class to which b belongs i.e
Figure BDA0002373133540000097
Is 1;
step 4.9: calculating a cross entropy loss function of the multi-region convolutional neural network, wherein the calculation process of the loss function is as follows:
Figure BDA0002373133540000098
wherein, yiIs the true label, y, of the ith sample in the training dataset of step 2.2i1 denotes that the label of the input data is 1, i.e. the position pixel is changed, yi0 means that the label of the input data is 0, i.e. the position pixel is unchanged,
Figure BDA0002373133540000099
a prediction tag, T, representing the ith sample1As samples of a training data setThe number i is the sample of the first training, i 1,2,31Log denotes base 10 logarithmic operation;
then optimizing parameters of the multi-region convolutional neural network by using a Stochastic Gradient Descent (SGD) algorithm;
and 5: inputting the test data set in the step 2.3 into the optimized multi-region convolutional neural network, and obtaining T related to the test data set according to the process from the step 4.2 to the step 4.82A prediction tag;
step 6 combining the training data set of step 2.2 and T obtained in step 52And (4) obtaining a change result graph of the place in the step 1 by the prediction label.
The effect of the present invention is further explained by combining simulation experiments as follows:
the simulation experiment of the invention is carried out under the hardware environment of Intel Xeon E5-2620, NVIDIA TITAN XP, the memory 16GB and the software environment of Ubuntu 16.04.6, Keras and Matlab2016a, and the experimental objects are three groups of multi-temporal SAR images Ottawa dataset, Farmland dataset and Rome dataset, the Ottawa dataset is shot by Radarsat satellites in Wttawa region in 1997 5 months and 8 months respectively, and is 350 × 290 pixels in size, such as the first line of FIG. 7. the Farmland dataset is shot by Radardast satellites in Huanghe region in 2008 6 months and 2009 6 months in 2008, is 306 × 291 pixels in size, such as the second line of FIG. 7. the Rome dataset is shot in Roman region in 2003 6 months 27 days and 2003 4 months 3 months in 2003, and is 256 in size, such as the third line of FIG. 7. the simulation experiment data of the invention is shown in FIG. 7(C) which is a real simulation detection graph of real detection image change.
The results of the comparison of the method of the present invention with the prior art more advanced change detection method are shown in FIG. 8. The method of Principal Component Analysis and K-means Clustering (hereinafter abbreviated PCAKM) in comparative experiments is set forth in the article "unused change detection in satellite image using Principal Component Analysis and K-means Clustering"; the Extreme Learning Machine (hereinafter abbreviated as ELM) method is proposed in the article "Change detection from synthetic apparatus based on neighboring branched-based ratio and Extreme Learning Machine"; the PCANet method is proposed in the article "Automatic change detection in synthetic opacity based on PCANet"; the Deep neural networks (hereinafter abbreviated as DBN) method is proposed in the article "Change detection in synthetic aperture images based on Deep neural networks". As shown in fig. 8, although the input image has strong noise, the method of the present invention can still obtain the variation information in the multi-temporal SAR image, and has better robustness to the noise.
As shown in the first four columns of fig. 8, other methods are easily affected by noise interference, and it is difficult to accurately express change information, especially, a subtle change area exists in the Rome data set, and the method can still accurately identify and eliminate noise.
The invention uses the classification accuracy (PCC) and the Kappa Coefficient (KC) to compare with the method on objective indexes, and the calculation method is as follows:
Figure BDA0002373133540000101
Figure BDA0002373133540000102
where N is the total number of pixels, OE ═ FP + FN is the total number of errors, FP is the number of false detections, indicating the number of pixels in the reference map that have not changed but have been detected as changed in the final change map, FN is the number of missed detections, indicating the number of pixels in the reference map that have changed but have not changed in the final change map, PRE indicates the number and proportional relationship of false detections and missed detections, PRE ═ TP [ (TP + FP-FN) × TP + (TN + FN-FP) × TN ]/(N × N), where TP is the number of pixels that have actually changed and TN is the number of pixels that have actually not changed.
TABLE 1 Experimental results of Change detection method of Ottawa data set
Method of producing a composite material PCC(％) KC(％)
PKAKM 97.57 90.73
ELM 98.17 93.15
PCANet 98.13 92.99
DBN 98.33 93.76
The method of the invention 98.71 95.11
TABLE 2 Experimental results of the change detection method of the Farmland data set
Method of producing a composite material PCC(％) KC(％)
PKAKM 94.03 62.92
ELM 97.70 76.05
PCANet 96.14 71.55
DBN 98.62 87.49
The method of the invention 99.64 96.87
TABLE 3 Change detection method for Rome dataset Experimental results
Method of producing a composite material PCC(％) KC(％)
PKAKM 96.70 86.56
ELM 97.70 89.87
PCANet 97.25 89.05
DBN 96.99 87.08
The method of the invention 97.96 91.89
The method based on the multi-region convolutional neural network is mainly specially provided for improving the analysis and understanding of the multi-temporal remote sensing image. However, obviously, the method is also suitable for analyzing the images shot by common imaging equipment such as a digital camera, and the obtained beneficial effects are similar.
The method for detecting the change of the remote sensing image based on the multi-region convolutional neural network provided by the invention is described in detail above, but obviously, the specific implementation form of the invention is not limited to this. It will be apparent to those skilled in the art that various obvious changes may be made therein without departing from the scope of the invention as defined in the appended claims.

Claims (3)

1. A SAR image change detection method based on a multi-region convolutional neural network comprises the following steps:
step 1: carrying out difference analysis on two multi-temporal SAR images at the same place to obtain a difference image:
performing difference analysis on the two multi-temporal SAR images by using a log ratio to obtain a differential image of the multi-temporal SAR images;
the calculation process of the differential image comprises the following steps:
IDI＝|logI1-logI2|
wherein, I1And I2Respectively representing two multi-temporal SAR images, IDIThe method comprises the steps that differential images of two multi-temporal SAR images are represented, | · | is absolute value operation, and log represents logarithm operation with 10 as a base;
step 2: for differential image IDIPre-classifying to construct a training data set and a test data set;
step 2.1: pre-classifying the difference image by using a multilayer fuzzy C-means clustering algorithm to obtain a pseudo tag matrix;
step 2.2, extracting the spatial positions marked as 0 and 1 in the pseudo label matrix, taking neighborhood pixels of L × L around pixel points corresponding to the spatial positions marked as 0 and 1 in the differential image obtained in the step 1 as a training data set, wherein the value of L is an odd number not less than 3, and the number of samples in the obtained training data set is marked as T1Each sample of the training data set is a block of pixels of size L × L;
step 2.3, extracting the spatial position marked as 0.5 in the pseudo label matrix, taking neighborhood pixels of L × L around a pixel point corresponding to the spatial position marked as 0.5 in the differential image obtained in the step 1 as a test data set, wherein the value of L is an odd number not less than 3, and the number of samples in the obtained test data set is marked as T2Each sample of the test data set is a block of pixels of size L × L;
the method is characterized by further comprising the following steps:
step 2.4, performing region segmentation on each sample of the training data set and the test data set respectively in an upper mode, a lower mode, a left mode and a right mode, namely respectively taking an L × k region which is positioned above the pixel block and is larger than 1/2 pixel blocks, taking an L × k region which is positioned below the pixel block and is larger than 1/2 pixel blocks, taking an L × L region which is positioned on the left side of the pixel block and is larger than 1/2 pixel blocks, taking an L × L region which is positioned on the right side of the pixel block and is larger than 1/2 pixel blocks, namely the two regions which are segmented up and down are the same in size, the two regions which are segmented left and right are the same in size, the values of k and L depend on the dimension L of the pixel blocks, and [ L/2] is more than or equal to k < L, and [ L/2] is more than or equal to L < L, and [ L/2] represents;
and step 3: gaussian noise is added to the training data set and the test data set, training samples are enriched, and the problem of model overfitting is relieved;
and 4, step 4: and (3) applying the training data set obtained in the step (3) after the Gaussian noise is added to the training of the multi-region convolutional neural network:
step 4.1: utilize Tensorflow to construct multizone convolutional neural network, the network of constructing comprises five branch networks, and every branch handles five different areas of upper and lower, left and right and global of same pixel block respectively, and five branch networks's of multizone convolutional neural network structure is: input layer → low-layer convolution layer → middle-layer convolution layer → high-layer convolution layer → full-connected layer; the input layer is the training sample obtained in the step 3; each branch network adopts the operation process of the following steps 4.2 to 4.5;
step 4.2: for each branch network, its lower convolution layer extracts the lower features F of the input layerL：
The low-layer convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nLWherein m is 3, nLIs a power exponent of 2 of minimum 128;
low level feature FLThe calculation process of (2) is as follows:
FL＝σ(XW1+b1)
wherein X represents the input data of the lower convolutional layer, the data is one of the five branches of the training set obtained in step 3 after adding Gaussian noise, σ represents the ReLU activation function, and W1And b1Representing weights and offsets for the lower convolutional layers;
step 4.3: a middle layer convolution layer through the branched networkTaking the middle layer features F of the input dataM：
The middle layer convolution layer has convolution kernel size of m × m and convolution kernel number of nMWherein m is 3, nMIs a power exponent of 2 of minimum 128;
middle layer characteristic FMThe calculation process of (2) is as follows:
FM＝σ(FLW2+b2)
wherein, FMDenotes the middle level characteristics, σ denotes the ReLU activation function, W2And b2Representing weights and offsets of the middle convolutional layer;
step 4.4: extracting high-level features F of input data through high-level convolutional layers of the branched networkH：
The high-level convolutional layer has the convolutional kernel size of m × m and the convolutional kernel number of nHWherein m is 1, nHIs a power exponent of 2 of minimum 128;
high level feature FHThe calculation process of (2) is as follows:
FH＝σ(FMW3+b3)
wherein, FHRepresenting high level features, σ represents the ReLU activation function, W3And b3Weights and biases representing higher convolutional layers;
step 4.5: constructing a dimension matching function G1And G2Thus through the dimension matching function G1And G2For low layer characteristic FLMiddle layer characteristic FMAnd high layer characteristics FHThe characteristic fusion is carried out, and the characteristic fusion is carried out,
wherein the dimension matches the function G1Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 5 and n is a power exponent of 2 of 64, and a dimension matching function G2Is a convolution layer with convolution kernel size of m × m and convolution kernel number of n, where m is 3 and n is a power exponent of 2 of minimum 64;
obtaining a fused feature map Fm，FmSize is FL、FM、FHThe size after stacking is calculated as:
Fm＝G1(FL)+G2(FM)+FH
step 4.6: the number of samples of the training data set added with the Gaussian noise obtained in the step 3 is T1After 4.1-4.5 operations are respectively carried out on the upper, lower, left, right and global five-class regions of each training sample of the training data set, the fusion feature maps of the upper, lower, left, right and global five-class regions of each training sample are output and are respectively marked as FUp、FBottom、FLeft、FRight、FFull；
Step 4.7: for the upper region feature FUpLower region characteristic FBottomLeft region feature FLeftRight region feature FRightAnd global feature FFullCarrying out feature information fusion to obtain a fusion feature map F of input dataA：
The fusion characteristic diagram FAThe calculation process of (2) is as follows:
firstly, obtaining a fusion feature map F of the five types of regionsmCarrying out dimension conversion to obtain a one-dimensional vector, wherein the process is as follows:
FT＝T(Fm)
wherein, FTRepresenting a feature diagram after being converted into a one-dimensional vector, wherein T (-) represents a dimension conversion function;
step 4.8: one-dimensional features F of each directionTU、FTB、FTL、FTR、FTFAccording to the splicing, the spliced characteristic diagram is marked as F, and then the F is subjected to two-layer full-connection operation to obtain R:
R＝δ(Wfc2(σ(Wfc1F)))
wherein Wfc1Denotes a first layer fully connected operation, Wfc2Representing the second layer full join operation, delta representing the Softmax function, sigma representing the ReLU activation function, and a vector with dimension 2 × 1 for R after full join
Figure FDA0002373133530000031
Wherein a represents the probability that a training sample belongs to the invariant class and b represents the probability that a training sample belongs to the variant class, based on the vector
Figure FDA0002373133530000032
To output a prediction tag of the ith sample
Figure FDA0002373133530000033
When a > b
Figure FDA0002373133530000038
Is equal to the class to which a belongs i.e
Figure FDA0002373133530000034
Is 0, when a < b
Figure FDA0002373133530000035
Is equal to the class to which b belongs i.e
Figure FDA0002373133530000036
Is 1;
step 4.9: calculating a cross entropy loss function of the multi-region convolutional neural network, wherein the calculation process of the loss function is as follows:
Figure FDA0002373133530000037
wherein, yiIs the true label, y, of the ith sample in the training dataset of step 2.2i1 denotes that the label of the input data is 1, i.e. the position pixel is changed, yi0 means that the label of the input data is 0, i.e. the position pixel is unchanged,
Figure FDA0002373133530000041
a prediction tag, T, representing the ith sample1For the number of samples of the training data set, i is the number of samples of the training, i＝1,2,3,...,T1Log denotes base 10 logarithmic operation;
then optimizing parameters of the multi-region convolutional neural network by using a Stochastic Gradient Descent (SGD) algorithm;
and 5: inputting the test data set in the step 2.3 into the optimized multi-region convolutional neural network, and obtaining T related to the test data set according to the process from the step 4.2 to the step 4.82A prediction tag;
step 6 combining the training data set of step 2.2 and T obtained in step 52And (4) obtaining a change result graph of the place in the step 1 by the prediction label.
2. The method for detecting the SAR image variation based on the multi-region convolutional neural network as claimed in claim 1, wherein in the step 2.2 and the step 2.3, the optimal value of L is 11.
3. The method for detecting SAR image variation based on multi-region convolutional neural network of claim 1, wherein in the step 2.4, k and l optimal values of 11 for the upper region and the lower region are 11 and 7 respectively, and k and l optimal values of 11 for the left region and the right region are 7 and 11 respectively.
CN202010056678.7A 2020-01-18 2020-01-18 SAR image change detection method based on multi-region convolutional neural network Pending CN111339827A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010056678.7A CN111339827A (en) 2020-01-18 2020-01-18 SAR image change detection method based on multi-region convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010056678.7A CN111339827A (en) 2020-01-18 2020-01-18 SAR image change detection method based on multi-region convolutional neural network

Publications (1)

Publication Number Publication Date
CN111339827A true CN111339827A (en) 2020-06-26

Family

ID=71185185

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010056678.7A Pending CN111339827A (en) 2020-01-18 2020-01-18 SAR image change detection method based on multi-region convolutional neural network

Country Status (1)

Country Link
CN (1) CN111339827A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914909A (en) * 2020-07-15 2020-11-10 南京审计大学 Hyperspectral change detection method based on space-spectrum combined three-direction convolution network
CN112241682A (en) * 2020-09-14 2021-01-19 同济大学 End-to-end pedestrian searching method based on blocking and multi-layer information fusion
CN112612316A (en) * 2020-12-18 2021-04-06 北京首钢自动化信息技术有限公司 Heating and ventilation equipment control method and device
CN112686871A (en) * 2020-12-31 2021-04-20 杭州电子科技大学 SAR image change detection method based on improved logarithmic ratio operator and Gabor _ ELM
CN112734695A (en) * 2020-12-23 2021-04-30 中国海洋大学 SAR image change detection method based on regional enhancement convolutional neural network
CN112990046A (en) * 2021-03-25 2021-06-18 北京百度网讯科技有限公司 Difference information acquisition method, related device and computer program product
CN112990391A (en) * 2021-05-20 2021-06-18 四川大学 Feature fusion based defect classification and identification system of convolutional neural network
CN113255451A (en) * 2021-04-25 2021-08-13 西北工业大学 Method and device for detecting change of remote sensing image, electronic equipment and storage medium
CN113537020A (en) * 2021-07-07 2021-10-22 中国人民解放军国防科技大学 Complex SAR image target identification method based on improved neural network

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105005774A (en) * 2015-07-28 2015-10-28 中国科学院自动化研究所 Face relative relation recognition method based on convolutional neural network and device thereof
CN107563422A (en) * 2017-08-23 2018-01-09 西安电子科技大学 A kind of polarization SAR sorting technique based on semi-supervised convolutional neural networks
CN109344693A (en) * 2018-08-13 2019-02-15 华南理工大学 A kind of face multizone fusion expression recognition method based on deep learning
CN110069994A (en) * 2019-03-18 2019-07-30 中国科学院自动化研究所 Face character identifying system, method based on face multizone
CN110659591A (en) * 2019-09-07 2020-01-07 中国海洋大学 SAR image change detection method based on twin network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105005774A (en) * 2015-07-28 2015-10-28 中国科学院自动化研究所 Face relative relation recognition method based on convolutional neural network and device thereof
CN107563422A (en) * 2017-08-23 2018-01-09 西安电子科技大学 A kind of polarization SAR sorting technique based on semi-supervised convolutional neural networks
CN109344693A (en) * 2018-08-13 2019-02-15 华南理工大学 A kind of face multizone fusion expression recognition method based on deep learning
CN110069994A (en) * 2019-03-18 2019-07-30 中国科学院自动化研究所 Face character identifying system, method based on face multizone
CN110659591A (en) * 2019-09-07 2020-01-07 中国海洋大学 SAR image change detection method based on twin network

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914909B (en) * 2020-07-15 2021-01-12 南京审计大学 Hyperspectral change detection method based on space-spectrum combined three-direction convolution network
CN111914909A (en) * 2020-07-15 2020-11-10 南京审计大学 Hyperspectral change detection method based on space-spectrum combined three-direction convolution network
CN112241682A (en) * 2020-09-14 2021-01-19 同济大学 End-to-end pedestrian searching method based on blocking and multi-layer information fusion
CN112241682B (en) * 2020-09-14 2022-05-10 同济大学 End-to-end pedestrian searching method based on blocking and multi-layer information fusion
CN112612316A (en) * 2020-12-18 2021-04-06 北京首钢自动化信息技术有限公司 Heating and ventilation equipment control method and device
CN112612316B (en) * 2020-12-18 2022-05-20 北京首钢自动化信息技术有限公司 Heating and ventilation equipment control method and device
CN112734695B (en) * 2020-12-23 2022-03-22 中国海洋大学 SAR image change detection method based on regional enhancement convolutional neural network
CN112734695A (en) * 2020-12-23 2021-04-30 中国海洋大学 SAR image change detection method based on regional enhancement convolutional neural network
CN112686871A (en) * 2020-12-31 2021-04-20 杭州电子科技大学 SAR image change detection method based on improved logarithmic ratio operator and Gabor _ ELM
CN112686871B (en) * 2020-12-31 2024-04-26 杭州电子科技大学 SAR image change detection method based on improved logarithmic comparison operator and Gabor_ELM
CN112990046A (en) * 2021-03-25 2021-06-18 北京百度网讯科技有限公司 Difference information acquisition method, related device and computer program product
CN112990046B (en) * 2021-03-25 2023-08-04 北京百度网讯科技有限公司 Differential information acquisition method, related device and computer program product
CN113255451A (en) * 2021-04-25 2021-08-13 西北工业大学 Method and device for detecting change of remote sensing image, electronic equipment and storage medium
CN113255451B (en) * 2021-04-25 2023-04-07 西北工业大学 Method and device for detecting change of remote sensing image, electronic equipment and storage medium
CN112990391A (en) * 2021-05-20 2021-06-18 四川大学 Feature fusion based defect classification and identification system of convolutional neural network
CN113537020A (en) * 2021-07-07 2021-10-22 中国人民解放军国防科技大学 Complex SAR image target identification method based on improved neural network

Similar Documents

Publication Publication Date Title
CN111339827A (en) SAR image change detection method based on multi-region convolutional neural network
CN110659591B (en) SAR image change detection method based on twin network
CN111738124B (en) Remote sensing image cloud detection method based on Gabor transformation and attention
CN109558859B (en) Mining area distribution information extraction method and system based on DInSAR and DCNN
CN112712535B (en) Mask-RCNN landslide segmentation method based on simulation difficult sample
CN110135354B (en) Change detection method based on live-action three-dimensional model
CN110008948B (en) Hyperspectral image target detection method based on variational self-coding network
CN103226826B (en) Based on the method for detecting change of remote sensing image of local entropy visual attention model
CN107230197B (en) Tropical cyclone objective strength determination method based on satellite cloud image and RVM
CN110598613B (en) Expressway agglomerate fog monitoring method
CN109816707A (en) A kind of field of opencast mining information extracting method based on high-resolution satellite image
CN108171119B (en) SAR image change detection method based on residual error network
CN109829426B (en) Railway construction temporary building monitoring method and system based on high-resolution remote sensing image
CN111008664A (en) Hyperspectral sea ice detection method based on space-spectrum combined characteristics
Xu et al. Feature-based constraint deep CNN method for mapping rainfall-induced landslides in remote regions with mountainous terrain: An application to Brazil
CN107967454B (en) Double-path convolution neural network remote sensing classification method considering spatial neighborhood relationship
CN112734695B (en) SAR image change detection method based on regional enhancement convolutional neural network
CN109145993B (en) SAR image classification method based on multi-feature and non-negative automatic encoder
CN113469097B (en) Multi-camera real-time detection method for water surface floaters based on SSD network
CN114612315A (en) High-resolution image missing region reconstruction method based on multi-task learning
CN106971402B (en) SAR image change detection method based on optical assistance
CN111275680B (en) SAR image change detection method based on Gabor convolution network
CN112270285A (en) SAR image change detection method based on sparse representation and capsule network
Albalooshi et al. Deep belief active contours (DBAC) with its application to oil spill segmentation from remotely sensed sea surface imagery
Heidary et al. Urban change detection by fully convolutional siamese concatenate network with attention

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200626