CN117808807A - Optical satellite remote sensing image instance level change detection method - Google Patents
Optical satellite remote sensing image instance level change detection method Download PDFInfo
- Publication number
- CN117808807A CN117808807A CN202410228524.XA CN202410228524A CN117808807A CN 117808807 A CN117808807 A CN 117808807A CN 202410228524 A CN202410228524 A CN 202410228524A CN 117808807 A CN117808807 A CN 117808807A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- satellite remote
- optical satellite
- sensing image
- pair
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003287 optical effect Effects 0.000 title claims abstract description 175
- 238000001514 detection method Methods 0.000 title claims abstract description 121
- 230000008859 change Effects 0.000 title claims abstract description 53
- 238000012360 testing method Methods 0.000 claims abstract description 67
- 230000005540 biological transmission Effects 0.000 claims abstract description 13
- 239000011159 matrix material Substances 0.000 claims description 38
- 238000000034 method Methods 0.000 claims description 34
- 238000012549 training Methods 0.000 claims description 31
- 238000000605 extraction Methods 0.000 claims description 23
- 238000013135 deep learning Methods 0.000 claims description 20
- 238000013528 artificial neural network Methods 0.000 claims description 14
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 230000006870 function Effects 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000004590 computer program Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009469 supplementation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an optical satellite remote sensing image instance level change detection method, which comprises the following steps: inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair; performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair; based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image. The invention can realize instance-level change detection in the rough aligned optical satellite remote sensing image pair.
Description
Technical Field
The invention mainly relates to the technical field of optical satellite remote sensing change detection, in particular to an optical satellite remote sensing image instance level change detection method.
Background
In recent years, with the high-speed development of space satellite technology, the change detection technology based on optical satellite remote sensing images has wide application in the fields of national defense and military. The method is mainly applied to battlefield situation awareness, military deployment monitoring and the like.
However, the existing method mainly detects the pixel-level change of the image, and has large calculation amount and high requirement on calculation equipment. In addition, the existing method needs to carry out fine registration on the image pairs in advance and then carry out change detection, and the mode is complex in process.
Disclosure of Invention
Aiming at the technical problems existing in the prior art, the invention provides an optical satellite remote sensing image instance level change detection method.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
in one aspect, the invention provides a method for detecting example level change of an optical satellite remote sensing image, which comprises the following steps:
(S1) inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
(S2) performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
and (S3) based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image.
The step (S3) includes:
the target detection results of the two images in the test optical satellite remote sensing image pair are respectively set asAnd->Wherein->Respectively represent the 1 st to the 1 st of the first image in the test optical satellite remote sensing image pair>Target information of individual targets->Respectively represent the 1 st to the 1 st in the second image in the test optical satellite remote sensing image pair>Target information of individual targets->;
Calculating the distance of the target detection results of the two images from element pair to obtain an initial distance matrix;
Using a given distance thresholdSupplementing the initial distance matrix->Obtaining distance square matrix +.>Wherein->Representing a real number;
square matrix of distanceInputting the element pairs into an optimal transmission problem solving algorithm to obtain successfully matched element pairs, and recording the output matched pairs as +.>Wherein the number of matched pairs is the size of the square matrix +.>,/>Representing the +.f in the first image>The target detection result and the first +.>The target detection results are successfully matched and become the +.>Pairs of matching pairs;
For successful match firstMatching pair->There are three situations:
if both elements are background elements, i.eAnd->Then there is no need to pay attention to the matching pair;
if neither element is a background element, i.eAnd->Then the target detection result forming the matching pair is considered to have no change in the two images of the test optical satellite remote sensing image pair;
if one of the two elements is a background element, i.eOr->Then the target detection result constituting the matched pair is considered to have a change in the two images of the test optical satellite remote sensing image pair, when +.>Testing the first image of the optical satellite remote sensing image pair>The target detection results do not correspond to the second image in the test optical satellite remote sensing image pair, when +.>Testing the second image of the optical satellite remote sensing image pair>The target detection results do not correspond to the first image in the test optical satellite remote sensing image pair.
Further, in the step (S2), the test optical satellite remote sensing image pair is input into a trained optical satellite remote sensing image pair target detection model to perform target detection, so as to obtain target detection results of each image in the test optical satellite remote sensing image pair.
Further, in the (S2) of the present invention, the training method of the optical satellite remote sensing image on the target detection model includes:
acquiring an optical satellite remote sensing image pair for training, wherein the optical satellite remote sensing image pair is an imaging result of an optical satellite in different periods of the same region, marking an interested target in the optical satellite remote sensing image pair, and marking information comprises the position and scale information of the target;
setting up a deep learning network for instance level change detection, wherein the deep learning network comprises a feature extraction module, a space-time attention module and a target detection module, and the feature extraction module is used for extracting a depth feature map of each image from an input optical satellite remote sensing image pair to obtain a depth feature map pair of the optical satellite remote sensing image pair; the space-time attention module is used for modeling the correlation of the depth feature map pair to each pixel in space and time and outputting a depth feature map after each image in the optical satellite remote sensing image pair is enhanced; the target detection module is used for extracting target information in the depth feature map after each image is enhanced, and the target information comprises the position and scale information of a target;
and training the deep learning network by using the optical satellite remote sensing image pair as training data, so that the deep learning network achieves a good fitting effect on the training data, and a target detection model of the trained optical satellite remote sensing image pair is obtained.
Further, the annotation information further comprises rotation information of the target in the image, and the target information extracted by the target detection module further comprises rotation information of the target in the image.
Furthermore, the invention utilizes the optical satellite remote sensing image pair to train the deep learning network based on the random gradient descent method.
Further, when the deep learning network is trained, conditions for training convergence are as follows: ending training when the training times reach the appointed times; alternatively, training is terminated when the trained penalty function falls below a specified threshold and continues to fall below the threshold.
Further, the feature extraction module comprises two feature extraction networks sharing weights, two optical satellite remote sensing images in the input optical satellite remote sensing image pair are respectively and correspondingly input into one feature extraction network, and depth feature maps of the optical satellite remote sensing images are obtained through extraction.
Furthermore, the target detection module comprises two neural networks sharing weight, depth feature maps obtained by enhancing two images in the optical satellite remote sensing image pair are respectively and correspondingly input into one neural network, and target information in each optical satellite remote sensing image is extracted.
In another aspect, the present invention provides an optical satellite remote sensing image change detection apparatus, including:
the first module is used for inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
the second module is used for carrying out target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
and the third module is used for establishing a corresponding relation of target detection results in the two images by utilizing an optimal transmission theory based on the target detection results of each image in the test optical satellite remote sensing image pair, calculating to obtain a final change result and finishing the change detection of the optical satellite remote sensing image.
In another aspect, the present invention provides a computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image.
In another aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image.
Compared with the prior art, the invention has the technical effects that:
the invention establishes the corresponding relation of the two image detection results by utilizing the optimal transmission theory, and calculates the final change result. Compared with the pixel level change detection, the invention can reduce a large amount of computation, and does not need to carry out fine registration on the image pairs in advance.
Furthermore, the invention provides an optical satellite remote sensing image pair target detection model and a training method thereof, which utilize a deep neural network to extract multi-scale features of a time sequence image pair, use a time-space attention module to capture time-space dependencies of different scale features, and use a target detection module to detect an interested target in an image.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the structures shown in these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of an example level change detection method for an optical satellite remote sensing image according to an embodiment;
FIG. 2 is a schematic diagram of a deep learning network in one embodiment;
FIG. 3 is a block diagram of a spatiotemporal attention module used in an embodiment;
FIG. 4 is a schematic diagram of a distance matrix after threshold supplementation in one embodiment.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a flowchart of an embodiment provides a method for detecting an example level change of an optical satellite remote sensing image, which includes the following steps:
(S1) inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
(S2) performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
and (S3) based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image.
It will be appreciated that there are many methods for detecting targets disclosed in the prior art, and those skilled in the art may choose to implement the target detection in step (S2), and the method for detecting targets is not particularly limited in the present invention.
In one embodiment, a target detection method is provided, that is, the test optical satellite remote sensing image pair is input into a trained optical satellite remote sensing image pair target detection model to perform target detection, and a target detection result of each image in the test optical satellite remote sensing image pair is obtained.
In one embodiment, a method for training an optical satellite remote sensing image on a target detection model is provided, including:
acquiring an optical satellite remote sensing image pair for training, wherein the optical satellite remote sensing image pair is an imaging result of an optical satellite in different periods of the same region, marking an interested target in the optical satellite remote sensing image pair, and marking information comprises the position and scale information of the target as a training sample;
setting up a deep learning network for instance level change detection, wherein the deep learning network comprises a feature extraction module, a space-time attention module and a target detection module, and the feature extraction module is used for extracting a depth feature map of each image from an input optical satellite remote sensing image pair to obtain a depth feature map pair of the optical satellite remote sensing image pair; the space-time attention module is used for modeling the correlation of the depth feature map pair to each pixel in space and time and outputting a depth feature map after each image in the optical satellite remote sensing image pair is enhanced; the target detection module is used for extracting target information in the depth feature map after each image is enhanced, and the target information comprises the position and scale information of a target;
and training the deep learning network by using a training sample, so that the deep learning network achieves a good fitting effect on training data, and a target detection model of the trained optical satellite remote sensing image is obtained.
In the training method of the optical satellite remote sensing image to the target detection model, the optical satellite remote sensing image pair is an imaging result of the optical satellite to the same region in different periods, and is based on the same length and width. The optical satellite remote sensing image is marked with the target of interest, and the marked information comprises the position and scale information of the target and can also comprise the rotation information of the target in the image. Each training sample consists of the labeling information in the two images in the optical satellite remote sensing image pair and the optical satellite remote sensing image pair.
In an embodiment, referring to fig. 2, which is a schematic structural diagram of a deep learning network, the built deep learning network includes a feature extraction module, a space-time attention module and a target detection module, where the feature extraction module is configured to extract a depth feature map of each image from an input optical satellite remote sensing image pair, so as to obtain a depth feature map pair of the optical satellite remote sensing image pair; the space-time attention module is used for modeling the correlation of the depth feature map pair to each pixel in space and time and outputting a depth feature map after each image in the optical satellite remote sensing image pair is enhanced; the target detection module is used for extracting target information in the depth feature map after each image is enhanced, wherein the target information comprises the position and scale information of the target and can also comprise the rotation information of the target in the image.
In one example of the invention, the deep neural network is trained using a random gradient descent method based on the training samples.
The feature extraction module can adopt a publicly available neural network structure to output depth feature maps with different scales and the same channel number. In the training and testing process of the neural network, a feature extraction module adopts a twin neural network structure, an optical satellite remote sensing image pair in a training sample is used as a batch, and is input into the feature extraction module to extract a depth feature map of each image in the optical satellite remote sensing image pair.
The space-time attention module is used for modeling the correlation of the optical satellite remote sensing image pair corresponding depth characteristic map pair on the space and time pixel by pixel, and outputting the enhanced depth characteristic map pair.
The input of the target detection module is an enhanced depth characteristic map pair, the output optical satellite remote sensing image pair is used for outputting target information of an interested target in two images, and the target information comprises position and scale information of the target and can also comprise rotation information of the target in the images. The target detection module adopts a publicly available neural network structure. In the training and testing process of the neural network, the target detection module adopts a twin neural network structure, takes a pair of enhanced depth feature maps as a batch, inputs the enhanced depth feature maps into the target detection module, and extracts target information of an interested target in each image.
Referring to fig. 2, the feature extraction module includes two feature extraction networks sharing weights, two optical satellite remote sensing images in the input optical satellite remote sensing image pair are respectively input into one feature extraction network correspondingly, and depth feature maps of each optical satellite remote sensing image are obtained by extraction. Feature maps of the image pairs extracted by the depth feature extraction module are respectively recorded asAnd->Wherein->、/>And->Is the number, length and width of channels of the depth profile map.
Referring to fig. 3, a structure diagram of a spatiotemporal attention module used in an embodiment, the implementation method of the spatiotemporal attention module includes:
depth feature map of two optical satellite remote sensing images in optical satellite remote sensing image pairAnd->Stacking a stack feature map tensor +.>;
Inputting the stacked feature map tensor into an attention submodule, wherein the input stacked feature map tensor is converted into three feature map tensorsTensor of the three feature maps>、And->Respectively reconstruct into a queryMatrix->A key matrix->And a value matrix->. The reconstruction method transforms the multi-dimensional feature map into a two-dimensional matrix, wherein the dimensions of the features remain unchanged. In the present embodiment, the feature map tensor +.>、/>And->The dimension of (2) is 4, the characteristic dimension is kept unchanged during reconstruction, the latter three dimensions are flattened (specific reference can be made to a PyTorch and other deep learning frames) to obtain a reconstructed matrix with the shape of +.>WhereinI.e. the number of elements in the feature map.
Map attention mapDefined as a similarity matrix. Element in similarity matrix ∈>Is->Personal bond and->Similarity between queries.
Generating an attention map by applying a softmax function based on the key matrix and the query matrixWherein the key matrix->Query matrix->,/>Wherein->Representing real number,/->Representing the number of depth feature map elements, ++>The number of channels of the depth feature map;
matrix of pass valuesAttention map->Is calculated as a reconstruction matrix by matrix multiplication>;
The matrix is to be reconstructedAnd stack feature map tensor->Adding to obtain updated feature map tensor,/>、/>And->The number, length and width of channels of the depth feature map;
tensor the updated characteristic mapSplit into two feature maps->And->,/>And->And the depth feature maps are respectively obtained after the two images in the optical satellite remote sensing image pair are enhanced.
Referring to fig. 2, in the embodiment, the target detection module includes two neural networks sharing weights, and depth feature maps obtained by enhancing two images in the optical satellite remote sensing image pair are respectively input into one neural network correspondingly, so as to extract target information in each optical satellite remote sensing image.
In an embodiment, a method for detecting an instance-level change of an optical satellite remote sensing image is provided, wherein the step (S3) is implemented by the following steps:
(1) The target detection results of the two images in the test optical satellite remote sensing image pair are respectively set asAnd->Wherein->Respectively represent the 1 st to the 1 st of the first image in the test optical satellite remote sensing image pair>Target information of individual targets->Respectively represent the 1 st to the 1 st in the second image in the test optical satellite remote sensing image pair>Target information of individual targets->;
(2) Calculating the distance of the target detection results of the two images from element pair to obtain an initial distance matrix;
(3) Using a given distance thresholdSupplementing the initial distance matrix->Obtaining distance square matrix +.>;
(4) Square matrix of distanceInputting the element pairs into an optimal transmission problem solving algorithm to obtain successfully matched element pairs, and recording the output matched pairs as +.>Wherein the number of matched pairs is the size of the square matrix +.>,/>Representing the +.f in the first image>The target detection result and the first +.>The target detection results are successfully matched and become the +.>Matching pair->;
(5) For successful match firstMatching pair->There are three situations:
if both elements are background elements, i.eAnd->Then there is no need to pay attention to the matching pair;
if neither element is a background element, i.eAnd->Then the target detection result forming the matching pair is considered to have no change in the two images of the test optical satellite remote sensing image pair;
if one of the two elements is a background element, i.eOr->Then the target detection result constituting the matched pair is considered to have a change in the two images of the test optical satellite remote sensing image pair, when +.>Testing the first image of the optical satellite remote sensing image pair>The target detection results do not correspond to the second image in the test optical satellite remote sensing image pair, when +.>Testing the second image of the optical satellite remote sensing image pair>The target detection results do not correspond to the first image in the test optical satellite remote sensing image pair.
In the above step (2), ioU or GIoU may be used for the horizontal frame (including no rotation information); KFIOU or SheWIOU etc. can be used for the rotation box (containing rotation information). Taking GIoU as an example, the initial distance matrix obtained is recorded asThe method comprises the following steps:
;
in the step (3), a distance threshold value is usedSupplementing the initial distance matrix->Obtaining a distance square matrix. As shown in FIG. 4, in one embodiment, a distance matrix after threshold is supplemented, wherein the distance matrix +.>Front of (2)Go, front->Column is the initial distance matrix->The values of the remaining elements are +.>。
;
The supplemented element is used as a background matching pair, namely the distance between the target detection result and the background is。
In the step (4), the distance matrix is obtainedInputting into an optimal transmission problem solving algorithm, such as Skinhorn algorithm, to obtain successfully matched element pairs, and recording the output matched pairs as +.>Wherein the number of matched pairs is the size of the square matrix +.>。
In an embodiment of the present invention, an optical satellite remote sensing image change detection device is provided, including:
the first module is used for inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
the second module is used for carrying out target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
and the third module is used for establishing a corresponding relation of target detection results in the two images by utilizing an optimal transmission theory based on the target detection results of each image in the test optical satellite remote sensing image pair, calculating to obtain a final change result and finishing the change detection of the optical satellite remote sensing image.
The implementation method of each module and the construction of the model can be the method described in any of the foregoing embodiments, which is not described herein.
In another aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the steps of the optical satellite remote sensing image instance level change detection method provided in any of the foregoing embodiments when executing the computer program. The computer device may be a server. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing sample data. The network interface of the computer device is used for communicating with an external terminal through a network connection.
In another aspect, the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the optical satellite remote sensing image instance level change detection method provided in any of the above embodiments.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The invention is not a matter of the known technology.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The method for detecting the instance level change of the optical satellite remote sensing image is characterized by comprising the following steps of:
(S1) inputting two optical satellite images of the same area at different times as a test optical satellite remote sensing image pair;
(S2) performing target detection on the test optical satellite remote sensing image pair to obtain target detection results of each image in the test optical satellite remote sensing image pair;
and (S3) based on the target detection results of each image in the test optical satellite remote sensing image pair, establishing a corresponding relation of the target detection results in the two images by utilizing an optimal transmission theory, and calculating to obtain a final change result to finish the change detection of the optical satellite remote sensing image.
2. The method for detecting the instance-level change of the optical satellite remote sensing image according to claim 1, wherein the step (S3) comprises:
the target detection results of the two images in the test optical satellite remote sensing image pair are respectively set asAndwherein->Respectively represent the 1 st to the 1 st of the first image in the test optical satellite remote sensing image pair>Target information of individual targets->Respectively represent the 1 st to the 1 st in the second image in the test optical satellite remote sensing image pair>Target information of individual targets->;
Calculating the distance of the target detection results of the two images from element pair to obtain an initial distance matrix;
Using a given distance thresholdSupplementing the initial distance matrix->Obtaining distance square matrix +.>Wherein->Representing a real number;
square matrix of distanceInputting the element pairs into an optimal transmission problem solving algorithm to obtain successfully matched element pairs, and recording the output matched pairs as +.>Wherein the number of matched pairs is the size of the square matrix +.>,/>Representing the +.f in the first image>The target detection result and the first +.>The target detection results are successfully matched and become the +.>Matching pair->;
For successful match firstMatching pair->There are three situations:
if both elements are background elements, i.eAnd->Then there is no need to pay attention to the matching pair;
if neither element is a background element, i.eAnd->Then the target detection result forming the matching pair is considered to have no change in the two images of the test optical satellite remote sensing image pair;
if one of the two elements is a background element, i.eOr->Then the target detection result constituting the matched pair is considered to have a change in the two images of the test optical satellite remote sensing image pair, when +.>Testing the first image of the optical satellite remote sensing image pair>The target detection results do not correspond to the second image in the test optical satellite remote sensing image pair, when +.>Testing the second image of the optical satellite remote sensing image pair>The target detection results do not correspond to the first image in the test optical satellite remote sensing image pair.
3. The method for detecting the instance level change of the optical satellite remote sensing image according to claim 1, wherein in the step (S2), the test optical satellite remote sensing image pair is input into a trained optical satellite remote sensing image pair target detection model to perform target detection, so as to obtain a target detection result of each image in the test optical satellite remote sensing image pair.
4. The method for detecting the instance-level change of an optical satellite remote sensing image according to claim 3, wherein in the step (S2), the method for training the object detection model by the optical satellite remote sensing image comprises the following steps:
acquiring an optical satellite remote sensing image pair for training, wherein the optical satellite remote sensing image pair is an imaging result of an optical satellite in different periods of the same region, marking an interested target in the optical satellite remote sensing image pair, and marking information comprises the position and scale information of the target;
setting up a deep learning network for instance level change detection, wherein the deep learning network comprises a feature extraction module, a space-time attention module and a target detection module, and the feature extraction module is used for extracting a depth feature map of each image from an input optical satellite remote sensing image pair to obtain a depth feature map pair of the optical satellite remote sensing image pair; the space-time attention module is used for modeling the correlation of the depth feature map pair to each pixel in space and time and outputting a depth feature map after each image in the optical satellite remote sensing image pair is enhanced; the target detection module is used for extracting target information in the depth feature map after each image is enhanced, and the target information comprises the position and scale information of a target;
and training the deep learning network by using the optical satellite remote sensing image pair as training data, so that the deep learning network achieves a good fitting effect on the training data, and a target detection model of the trained optical satellite remote sensing image pair is obtained.
5. The method of claim 4, wherein the annotation information further comprises rotation information of the object in the image, and the object information extracted by the object detection module further comprises rotation information of the object in the image.
6. The method for detecting the instance level change of the optical satellite remote sensing image according to claim 4, wherein the deep learning network is trained by utilizing the optical satellite remote sensing image pair based on a random gradient descent method.
7. The method for detecting the instance level change of the optical satellite remote sensing image according to claim 4, wherein when the deep learning network is trained, conditions for training convergence are as follows: ending training when the training times reach the appointed times; alternatively, training is terminated when the trained penalty function falls below a specified threshold and continues to fall below the threshold.
8. The method for detecting the instance level change of the optical satellite remote sensing image according to claim 4, claim 5, claim 6 or claim 7, wherein the feature extraction module comprises two feature extraction networks sharing weights, and two optical satellite remote sensing images in the input optical satellite remote sensing image pair are respectively input into one feature extraction network correspondingly, so as to extract a depth feature map of each optical satellite remote sensing image.
9. The method for detecting the instance-level variation of the optical satellite remote sensing image according to claim 8, wherein the method for realizing the time-space attention module comprises the following steps:
overlapping depth feature maps of two optical satellite remote sensing images in an optical satellite remote sensing image pair into a stacked feature map tensor;
Inputting the stacked feature map tensor into an attention submodule, converting the input stacked feature map tensor into three feature map tensors in the attention submodule, and reconstructing the three feature map tensors into a key matrix, a query matrix and a value matrix respectively;
generating an attention map by applying a softmax function based on the key matrix and the query matrixWherein the key matrix->Query matrix->,/>Wherein->Representing real number,/->Representing the number of depth feature map elements, ++>The number of channels of the depth feature map;
matrix of pass valuesAttention map->Is calculated as a reconstruction matrix by matrix multiplication> ;
The matrix is to be reconstructedAnd stack feature map tensor->Adding to obtain updated feature map tensor +.>C, H and W are the number, length and width of channels of the depth profile map;
tensor the updated characteristic mapSplit into two feature maps->And->,/>And->And the depth feature maps are respectively obtained after the two images in the optical satellite remote sensing image pair are enhanced.
10. The method for detecting the instance level change of the optical satellite remote sensing image according to claim 4, claim 5, claim 6, claim 7 or claim 9, wherein the target detection module comprises two neural networks sharing weights, and the depth feature maps enhanced by the two images in the optical satellite remote sensing image pair are respectively input into one neural network correspondingly, so as to extract target information in each optical satellite remote sensing image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410228524.XA CN117808807B (en) | 2024-02-29 | 2024-02-29 | Optical satellite remote sensing image instance level change detection method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410228524.XA CN117808807B (en) | 2024-02-29 | 2024-02-29 | Optical satellite remote sensing image instance level change detection method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117808807A true CN117808807A (en) | 2024-04-02 |
CN117808807B CN117808807B (en) | 2024-05-14 |
Family
ID=90432040
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410228524.XA Active CN117808807B (en) | 2024-02-29 | 2024-02-29 | Optical satellite remote sensing image instance level change detection method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117808807B (en) |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112489096A (en) * | 2020-11-26 | 2021-03-12 | 西北工业大学 | Remote sensing image change detection method under low registration precision based on graph matching model |
CN112633114A (en) * | 2020-12-17 | 2021-04-09 | 深圳中科保泰科技有限公司 | Unmanned aerial vehicle inspection intelligent early warning method and device for building change event |
CN114926511A (en) * | 2022-05-11 | 2022-08-19 | 西安电子科技大学 | High-resolution remote sensing image change detection method based on self-supervision learning |
CN114973028A (en) * | 2022-05-17 | 2022-08-30 | 中国电子科技集团公司第十研究所 | Aerial video image real-time change detection method and system |
US20220358334A1 (en) * | 2021-05-10 | 2022-11-10 | Qingdao Technological University | Assembly body change detection method, device and medium based on attention mechanism |
WO2023017663A1 (en) * | 2021-08-13 | 2023-02-16 | Mitsubishi Electric Corporation | Systems and methods for image processing based on optimal transport and epipolar geometry |
CN115937546A (en) * | 2022-11-30 | 2023-04-07 | 北京百度网讯科技有限公司 | Image matching method, three-dimensional image reconstruction method, image matching device, three-dimensional image reconstruction device, electronic apparatus, and medium |
CN116403007A (en) * | 2023-04-12 | 2023-07-07 | 北京卫星信息工程研究所 | Remote sensing image change detection method based on target vector |
CN117093848A (en) * | 2023-07-20 | 2023-11-21 | 北京航空航天大学 | Attention mechanism-based double-wall structure feature extraction method |
-
2024
- 2024-02-29 CN CN202410228524.XA patent/CN117808807B/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112489096A (en) * | 2020-11-26 | 2021-03-12 | 西北工业大学 | Remote sensing image change detection method under low registration precision based on graph matching model |
CN112633114A (en) * | 2020-12-17 | 2021-04-09 | 深圳中科保泰科技有限公司 | Unmanned aerial vehicle inspection intelligent early warning method and device for building change event |
US20220358334A1 (en) * | 2021-05-10 | 2022-11-10 | Qingdao Technological University | Assembly body change detection method, device and medium based on attention mechanism |
WO2023017663A1 (en) * | 2021-08-13 | 2023-02-16 | Mitsubishi Electric Corporation | Systems and methods for image processing based on optimal transport and epipolar geometry |
CN114926511A (en) * | 2022-05-11 | 2022-08-19 | 西安电子科技大学 | High-resolution remote sensing image change detection method based on self-supervision learning |
CN114973028A (en) * | 2022-05-17 | 2022-08-30 | 中国电子科技集团公司第十研究所 | Aerial video image real-time change detection method and system |
CN115937546A (en) * | 2022-11-30 | 2023-04-07 | 北京百度网讯科技有限公司 | Image matching method, three-dimensional image reconstruction method, image matching device, three-dimensional image reconstruction device, electronic apparatus, and medium |
CN116403007A (en) * | 2023-04-12 | 2023-07-07 | 北京卫星信息工程研究所 | Remote sensing image change detection method based on target vector |
CN117093848A (en) * | 2023-07-20 | 2023-11-21 | 北京航空航天大学 | Attention mechanism-based double-wall structure feature extraction method |
Non-Patent Citations (3)
Title |
---|
MARCO FIORUCCI, PETER NAYLOR, MAKOTO YAMADA: "Optimal Transport for Change Detection on Lidar Point Clouds", IGARSS 2023 - 2023 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM, 20 October 2023 (2023-10-20), pages 982 - 985 * |
NIKITA PRONKO: "Change Point Detection with Optimal Transport and Geometric Discrepancy", HTTPS://DOI.ORG/10.48550/ARXIV.1707.08658, 26 July 2017 (2017-07-26), pages 1 - 26 * |
刘占红;刘一超;薛峰;李慧;: "遥感图像变化检测综述", 微型机与应用, no. 15, 10 August 2013 (2013-08-10), pages 7 - 10 * |
Also Published As
Publication number | Publication date |
---|---|
CN117808807B (en) | 2024-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110837846B (en) | Image recognition model construction method, image recognition method and device | |
CN110929622A (en) | Video classification method, model training method, device, equipment and storage medium | |
Chen et al. | Convolutional neural network based dem super resolution | |
WO2019136591A1 (en) | Salient object detection method and system for weak supervision-based spatio-temporal cascade neural network | |
CN108154133B (en) | Face portrait-photo recognition method based on asymmetric joint learning | |
CN106156777A (en) | Textual image detection method and device | |
CN115019370A (en) | Depth counterfeit video detection method based on double fine-grained artifacts | |
CN114419406A (en) | Image change detection method, training method, device and computer equipment | |
CN111242840A (en) | Handwritten character generation method, apparatus, computer device and storage medium | |
CN112580480A (en) | Hyperspectral remote sensing image classification method and device | |
CN111008570B (en) | Video understanding method based on compression-excitation pseudo-three-dimensional network | |
CN113095158A (en) | Handwriting generation method and device based on countermeasure generation network | |
Zhang et al. | Collaborative weighted multi-view feature extraction | |
Li et al. | Enhanced bird detection from low-resolution aerial image using deep neural networks | |
CN117808807B (en) | Optical satellite remote sensing image instance level change detection method | |
CN117152823A (en) | Multi-task age estimation method based on dynamic cavity convolution pyramid attention | |
CN116612272A (en) | Intelligent digital detection system for image processing and detection method thereof | |
Xue et al. | A subspace learning-based method for JPEG mismatched steganalysis | |
CN115861540A (en) | Three-dimensional reconstruction method, device and equipment for two-dimensional face and storage medium | |
CN113343810B (en) | Pedestrian re-recognition model training and recognition method and device based on time sequence diversity and correlation | |
Liu et al. | Towards misregistration-tolerant change detection using deep learning techniques with object-based image analysis | |
Li et al. | Refined division features based on Transformer for semantic image segmentation | |
CN107958229B (en) | Face recognition method, device and equipment based on neighbor keeping low-rank representation | |
CN112307980A (en) | Image identification method based on incomplete multi-view clustering | |
Li et al. | Thick Cloud Removal for Multitemporal Remote Sensing Images: When Tensor Ring Decomposition Meets Gradient Domain Fidelity |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |