CN111563408A - High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning - Google Patents
High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning Download PDFInfo
- Publication number
- CN111563408A CN111563408A CN202010157591.9A CN202010157591A CN111563408A CN 111563408 A CN111563408 A CN 111563408A CN 202010157591 A CN202010157591 A CN 202010157591A CN 111563408 A CN111563408 A CN 111563408A
- Authority
- CN
- China
- Prior art keywords
- perception
- layer
- feature
- landslide
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000008447 perception Effects 0.000 title claims abstract description 76
- 238000001514 detection method Methods 0.000 title claims abstract description 36
- 230000000750 progressive effect Effects 0.000 title claims abstract description 15
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000013507 mapping Methods 0.000 claims abstract description 35
- 238000010606 normalization Methods 0.000 claims abstract description 17
- 238000010586 diagram Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims abstract description 10
- 239000000969 carrier Substances 0.000 claims abstract description 6
- 230000002708 enhancing effect Effects 0.000 claims abstract description 5
- 230000016776 visual perception Effects 0.000 claims description 26
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 16
- 241000282414 Homo sapiens Species 0.000 claims description 15
- 230000000007 visual effect Effects 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 10
- 230000015572 biosynthetic process Effects 0.000 claims description 9
- 238000003786 synthesis reaction Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 8
- 230000003595 spectral effect Effects 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 5
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 230000004456 color vision Effects 0.000 claims description 4
- 230000005284 excitation Effects 0.000 claims description 4
- 239000013589 supplement Substances 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 3
- 230000000452 restraining effect Effects 0.000 claims description 2
- 230000007547 defect Effects 0.000 abstract description 4
- 210000004556 brain Anatomy 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 238000011160 research Methods 0.000 description 6
- 238000010276 construction Methods 0.000 description 4
- 230000004438 eyesight Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000001629 suppression Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000001788 irregular Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000012821 model calculation Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000011434 tangent normalization method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a high-resolution image landslide automatic detection method for multi-level perception characteristic progressive self-learning, which comprises the following steps: 1. dividing feature levels based on perception depth by utilizing a high-resolution image of a landslide region, and supporting feature normalization mapping of perception level progression; 2. establishing a scale normalization model with gradually enhanced features, mapping each level of perception feature elements with spatial scales and dimensions as carriers, and generating a multilevel feature map with highly organized semantic information; 3. constructing a progressive self-learning regional network which is restrained by a comprehensive multi-level characteristic diagram, and generating a landslide target-oriented detection network through end-to-end training; d) inputting target high-resolution image data to be analyzed to a detection network, performing targeted detection on the landslide target from the angle of gradually enhancing image characteristics, and finally outputting landslide target image representation. The method overcomes the defect of single understanding of the existing complex scene image, and enhances the correlation capability among the characteristics to enable the detection result to be more accurate.
Description
Technical Field
The invention belongs to the technical field of geospatial data processing, and particularly relates to a high-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning.
Background
Because landslide disasters bring influences such as universality, loss, threat and the like, the effective and rapid identification of landslide disasters is a problem which is urgently needed to be solved at present. The field survey cannot meet the requirement for landslide disaster identification, and the identification by using an optical satellite remote sensing technology becomes the mainstream. The method comprises the steps of identifying a target, extracting key information and a target area based on an object-oriented classification method, and combining visual interpretation to obtain detailed information of the landslide and improve the success rate of the landslide interpretation of the remote sensing image.
Research results show that the problems of large feature difference of landslide on images, representation difference under different scales and the like are faced by using the traditional object-oriented identification technology, so that non-landslide objects are not completely removed, the landslide is excessively extracted, meanwhile, detailed features of small landslide on the images are easily ignored, and the requirements are met by combining manual interpretation.
In order to adapt to the requirements and development of the remote sensing data and cloud computing era, the computer field and the GIS field are combined to become the current research direction. By establishing a relatively complex network structure, a large-scale high-resolution remote sensing image data set is built, deep excavation is carried out on a large amount of effective information in the image, application such as trend prediction is further carried out, and automatic analysis and understanding of the remote sensing image are achieved.
In recent years, with the appearance of large-scale data and the development of deep neural networks, raw data is used as input, and an End-to-End (End-to-End) learning process is possible, so that the method is an important research direction in the field of machine learning. A model capable of automatically learning features is designed based on a convolutional neural network according to a detection target, and representativeness of essential features of original data is obtained through learning of large-scale data. The model after repeated iterative training can extract rich internal information, and classification and detection are facilitated.
However, for a long time, the combination of remote sensing image interpretation and computer vision always shows that the computer vision technology is too high in ratio, the application of the remote sensing technology is less, and the lack of professional understanding of remote sensing image data cannot break through the existing association method. Most researches focus on the construction of a deep learning network, more network layers are pursued, but the self property of characteristic data and the correlation between knowledge and the data are ignored, and bottlenecks are met in the aspects of precision, accuracy and the like.
When remote sensing data is analyzed and mined, the problem that a large number of non-standard features cannot be effectively utilized is faced, so that a machine cannot accurately learn image features, accurate judgment cannot be given for massive data to be predicted, universality is not provided, and finally the situation that result features are meaningless or learning is not stopped is caused.
Disclosure of Invention
The invention aims to provide a multi-level perception characteristic gradual self-learning method facing landslide disasters based on a high-resolution remote sensing image, aiming at a region with multiple landslide disasters which is difficult to monitor on site. The method can realize the perception level progression from human brain understanding to machine realization of the monitored remote sensing image to be analyzed, overcome the defect of single understanding of the complex scene image in the existing method, and increase the correlation capability among the characteristics, thereby obtaining an accurate landslide detection result.
In order to achieve the purpose, the multilayer perception feature progressive self-learning method facing to landslide disasters based on the high-resolution remote sensing image comprises the following steps:
and 4, inputting target high-resolution image data to be analyzed to a detection network, performing targeted detection on the landslide target from the angle of gradually enhancing image characteristics, and finally outputting landslide target image representation.
Preferably, step 1 comprises:
extracting the data features of each image to be processed into a uniform feature item expression sample set; unifying vectors and matrixes to form basic feature items by using a basic data structure, and taking each feature item as metadata; generating three layers of feature perception sets by metadata through algorithms with different complexity; wherein,
in a visual perception layer, processing is carried out based on metadata to obtain a shallow visual characteristic of a two-dimensional single channel of image data to be processed, wherein the shallow visual characteristic is obtained by integrating one or more of an original gray matrix, a gray co-occurrence matrix and a color synthesis matrix;
processing the instrument sensing layer based on metadata to obtain the mass constraint characteristics of the image data to be processed, wherein the mass constraint characteristics are obtained by integrating spectral characteristic vectors and/or shape characteristic vectors;
and in the algorithm perception layer, the boundary characteristics and the depth semantic characteristics of the visual perception layer are mined on the basis of the characteristics extracted in the visual perception layer and the instrument perception layer.
Preferably, step 2 comprises:
step 2.1, unifying different scale characteristics: analyzing the input characteristic structure by adopting a data normalization method and carrying out unified mapping;
step 2.2, constructing a top-down path: adopting a top-down characteristic supplement enhancement idea in a characteristic pyramid network, gradually participating gradually deepened perception level characteristics from a lower layer network to a higher layer network, adding different sized convolution layers in the network for deep characteristic extraction, and obtaining three semantic characteristics of a shallow layer, a middle layer and a high layer;
step 2.3, obtaining multilayer characteristics by pooling: and (3) obtaining a multi-level feature map fusing the semantic features of each layer by utilizing the three layers of semantic features generated in the step (2.2) and performing spatial pyramid pooling operation.
Preferably, step 3 comprises:
step 3.1, inputting a multi-level feature map, carrying out specific dimension transformation, and enabling the feature map to be shared for subsequent extraction of the region candidate frame and connection of a full connection layer for output;
step 3.2, extracting a region candidate frame: generating a series of anchor points by using the RPN idea and the characteristic diagram, judging whether the anchor points belong to a target region or not by using an output layer excitation function, and then restraining and correcting the anchor points by using frame regression and a non-maximum value to obtain an accurate prediction region;
and 3.3, returning the prediction region mapping generated in the step 3.2 to a feature map, performing 1 × 1 convolution processing, then sending the prediction region mapping to an interested region pool, obtaining a feature mapping with a fixed size, using the feature mapping to a neural network layer of a target detection task, and finally generating a landslide target-oriented detection network through end-to-end training.
The invention takes the perception hierarchy progression from human brain understanding to machine realization facing to the remote sensing image as a core, combines the typical characteristics of the remote sensing image and the associated depth mining characteristics of dependence image data and knowledge, overcomes the defect of single understanding of the complex scene image in the prior art, enhances the association capability between the characteristics, and supplements the research method for realizing the rapid automatic detection of the natural disasters to a certain extent; in the process, the hierarchy division of the image features based on the perception depth breaks through the limitation that a large amount of auxiliary data is used in the traditional image interpretation, and the image feature understanding from the human brain to the machine is standardized; the spatial scale and the dimension transformation are adopted as carriers for unifying the features of different organization forms, so that the complexity of information association of multi-level features in time, space and attributes is reduced, and the speed of model calculation and learning is obviously increased; the method can support the area detection of various irregular landslide targets, expand the space range of the landslide disaster monitoring system and improve the time efficiency, and is also beneficial to developing a perception constraint automatic detection method of an image data source for targets with different characteristics.
Drawings
FIG. 1 is a flow chart of the method steps of the present invention.
FIG. 2 is a schematic diagram of the feature hierarchy based on perceptual level partitioning in the present invention.
FIG. 3 is a network schematic diagram of normalized mapping hierarchy features in the present invention.
FIG. 4 shows the construction of a regionalized network and progressive enhancement constraint detection in the present invention.
FIG. 5 is a schematic diagram of high resolution image data of a target to be analyzed according to an embodiment of the present invention.
FIG. 6 is a schematic representation of the landslide target image obtained after the data in FIG. 5 is processed by the method of the present invention (identified landslide is indicated by a box).
Detailed Description
The technical scheme of the invention is further explained in detail by combining the attached drawings:
as shown in FIG. 1, the multilevel perceptual feature gradual self-learning method facing landslide disasters based on the high-resolution remote sensing image comprises the following steps 1-4.
As shown in fig. 1, the utilizing of the high-resolution image of the landslide region means that a high-resolution image sample set of the landslide region is obtained first, and according to the processing requirements of the existing image feature extraction algorithms (these algorithms are the prior art, and are not explained herein), the data features of each image to be processed are extracted as a uniform feature term expression sample set; the unified vector and the matrix are used as basic data structures to form basic characteristic items, and each characteristic item is used as metadata. The high-resolution image sample set of the landslide area can be an existing network resource. The feature hierarchy division based on the perception depth is a feature hierarchy division standard of a perception level provided by depending on association of image data and knowledge, and three layers of feature perception sets are generated by metadata through algorithms with different complexities according to the hierarchical processing requirements during specific execution.
The division of the feature hierarchy based on perceptual level is explained in connection with fig. 2. The present application is based on perceptual depth-partitioning from human brain understanding to machine implementation into three levels: a visual perception layer, an instrument perception layer, and an algorithm perception layer. The complexity of these three levels goes from low to high.
And in the visual perception layer, processing based on metadata to obtain a shallow visual characteristic of a two-dimensional single channel of the image data to be processed, wherein the shallow visual characteristic is obtained by synthesizing one or more of an original gray matrix, a gray co-occurrence matrix and a color synthesis matrix. In the human brain understanding level, the visual perception layer takes the color perception of the image by human eyes as the consideration of the layer. The two-dimensional single-channel shallow visual characteristic comprises an original gray level matrix, a gray level co-occurrence matrix and a color synthesis matrix. The original gray matrix records the gray value of each pixel in the remote sensing image, the gray value is determined by dividing the white and the black into the number of grades according to the logarithmic relationship, and the gray value belongs to the first-level visual perception, namely the visual perception, of the unprocessed remote sensing image by human beings. The gray level co-occurrence matrix is a texture of an image formed by repeated appearance of pixel gray levels in a spatial position, describes joint distribution of two pixel gray levels with a certain spatial position relation, can present the texture on the image, has a good auxiliary effect on visual interpretation of ground objects, and belongs to the first-level visual perception of human beings on roughly processed remote sensing images. The color synthesis matrix is a characteristic generated by utilizing natural true color synthesis in the field of remote sensing data processing, the true color synthesis means that the colors of ground objects on a synthesized color image are close to or consistent with the colors of actual ground objects, and a general method is R/G/B synthesis corresponding to red, green and blue of a multispectral image, and belongs to the first-level visual perception of human beings on a purposefully processed remote sensing image. In order to facilitate calculation and represent the visual characteristics of the data to be processed in multiple angles, the embodiment performs matrix addition on the original gray level matrix, the gray level co-occurrence matrix and the color synthesis matrix to represent the shallow visual characteristics.
And processing the image data to be processed based on the metadata to obtain the mass constraint characteristics of the image data to be processed in an instrument perception layer, wherein the mass constraint characteristics are integrated with the spectral characteristic vector and/or the shape characteristic vector. The instrument perception layer considers the data characteristic perception recorded by the remote sensing instrument and performs more complex calculation to obtain the qualitative constraint characteristic; the method comprises the following steps: spectral feature vectors and shape feature vectors. The spectral characteristics refer to that the satellite remote sensing data is collected with related data in each wave band of the full spectrum before being synthesized by the high-definition picture, and the characteristics of the target ground object are obtained by analyzing according to the full spectrum characteristics of the ground object, so that the method is a main mode in the field of remote sensing data processing. For landslide type identification, the most important parameters are the rock characteristics of the landslide surface and the surrounding vegetation characteristics. According to the ratio operation among the wave bands, not only can the target spectral characteristics be extracted, but also the landslide characteristics in the image can be highlighted, the landslide category can be extracted or the bare land range can be estimated as a result, the heterogeneous characteristics of different ground objects can be extracted, and the landslide and non-landslide regions can be more accurately segmented; the shape feature, namely the shape heterogeneity criterion, is calculated through two measurement values of landscape ecology, namely smoothness heterogeneity and compactness heterogeneity, and the feature is used as landslide detection measurement to restrict the adaptive learning of irregular landslide shapes. Useful information is obtained based on the relatively complex observation of the human brain by instruments as a hierarchical criterion for the above features. In this embodiment, the qualitative constraint feature is expressed by performing vector multiplication on the obtained spectral feature vector and shape feature vector.
And in the algorithm perception layer, the boundary characteristics and the depth semantic characteristics of the visual perception layer are mined on the basis of the characteristics extracted in the visual perception layer and the instrument perception layer. And the algorithm perception layer carries out depth perception on the basis of the characteristics as consideration, and the boundary characteristics and the depth semantic characteristics of the algorithm perception layer are mined. Based on an image depth information acquisition technology and a spatial big data mining technology in the field of computer vision, a machine can learn the depth expression characteristics of a landslide target through basic extracted characteristics and a series of algorithms, wherein the depth expression characteristics comprise target boundary characteristics and depth semantic characteristics. The target boundary features are extracted by utilizing an edge detection technology in computer vision, and the principle of edge detection is that a feature map is obtained after an original picture is subjected to convolution operation through a convolution kernel with a specific structure, and the feature map can just reflect the edges of the image; the deep semantic features are relations between implicit image data and knowledge which are not clearly shown are extracted from a large number of data samples, a machine can find and learn useful features in the implicit image data, and quick information matching is achieved after multiple training.
And 2, carrying out normalized mapping on the hierarchical characteristic scale. And establishing a scale normalization mapping model for gradually enhancing the features based on the three-layer perception feature set, mapping each level of perception feature elements with spatial scale and dimensionality as carriers, and generating a multi-level feature map with highly organized semantic information.
In the step, the spatial characteristics presented by the data structure of each layer of characteristics are analyzed in sequence, wherein the spatial characteristics comprise dimensions and dimensionality; the perception layer-by-layer progression is used as a criterion of a Feature mapping network layer, a normalization mapping model with multiple hidden layers is constructed by utilizing a Feature Pyramid Network (FPN) idea, and the purpose is to enable participated features of each layer level to constrain semantic information from shallow to deep. The characteristic pyramid network mainly comprises a top-down path for generating different dimensionality characteristics, namely reflecting image characteristics of a target object in a shallow network and a high-level network, and a bottom-up framework realizes the characteristic enhancement layer by amplifying the proportion of an upper-level characteristic diagram and then adding the upper-level characteristic diagram by using vector special multiplication and other methods; the improved mapping model based on the feature pyramid network uses a top-down path to creatively embed progressive hierarchical features into the network path, so as to generate a multilevel feature map with highly organized semantic information.
As shown in fig. 3, the construction of the hierarchical feature scale normalization mapping model includes the following steps:
and 2.1, unifying the characteristics of different scales. A data Normalization universal method is adopted, and comprises an Inverse tangent Normalization method (Inverse Normalization) which is divided according to a basic structure and is suitable for fixed features, a kernel function fusion method (KernelFunction) and a support vector machine (Gaussian SVM) which is suitable for dynamic change feature fusion, wherein the support vector machine is combined with the Gaussian kernel function method (SVM), and an input feature structure is analyzed and uniformly mapped from a network layer. The normalization method comprises the following three steps:
step 2.1a, arctangent normalization: the method belongs to a characteristic scaling method, and data is scaled to fall into a small specific interval. The advantages are that it has very good symmetry and is insensitive to inputs over a certain range; the following formula is adopted:
step 2.1b, kernel function fusion method: the low-dimensional space is mapped to the high-dimensional space by using a kernel function, and the mapping can change two types of points which are linearly inseparable in the low-dimensional space into linearly separable points. Considering kernel functions
I.e. the square of the inner product, of which
And
are two points in two-dimensional space; adopting a mapping formula:
step 2.1c, gaussian kernel function method: expanding the data dimension to an infinite dimension by using a Gaussian kernel function mode to obtain a boundary, wherein the result is only related to the calculation of the distance between the x and the central point xn and is not related to other data; the following formula is adopted:
and 2.2, constructing a top-down path. And (3) adopting a top-down characteristic supplement enhancement idea in the characteristic pyramid network, gradually participating gradually deepened perception level characteristics from a lower layer network to a higher layer network, adding convolution layers with different sizes into the network for deep characteristic extraction, and obtaining three semantic characteristics of a shallow layer, a middle layer and a high layer.
And 2.3, pooling to obtain multilayer characteristics. And obtaining a multi-level feature map fusing the semantic features of each layer by utilizing the three layers of semantic features generated in the last step and through Spatial Pyramid Pooling (SPP) operation. The spatial pyramid pooling is to perform pooling operation on features with different scales in the same input image by using a plurality of windows to obtain pooled features with the same length.
And 3, gradually building self-learning of the regional network. And taking the multi-level characteristic diagram as input, constructing a progressive self-learning regional network which is restrained by the comprehensive multi-level characteristic diagram, and generating a landslide target-oriented detection network through end-to-end training.
In step 3, constructing a regional Network facing landslide target detection based on a Region pro-social Network (RPN) to obtain a more accurate target position; the area generation network outputs a series of rectangular prediction areas (Object explosals) through specific convolution operation by using an input feature map, and maps the rectangular prediction areas back to an original image; End-to-End (End-to-End) training of the network, i.e., adjusting the error of the predicted result obtained from the input End (input data) to the output End compared with the actual result until the model converges or the expected effect is achieved, is performed to perform network-level tuning.
With reference to fig. 4, in step 3, the progressive self-learning construction of the regional network includes the following steps:
and 3.1, inputting a multi-level feature map, and performing specific dimension transformation (Reshape), wherein the feature map is shared for subsequent Region candidate box (Region poppesals) extraction and connecting a Fully Connected layer (FC) for output.
Step 3.2, the extraction of the region candidate frame described in step 3.1 is to generate a series of anchor points (anchors) from the feature map by using the RPN network concept, judge whether anchors belong to the target region by using the output layer excitation function (Softmax), and then correct anchors by using Bounding Box Regression (BBR) and Non-maximum suppression (NMS) to obtain an accurate prediction region. This step involves the following items:
step 3.2a, anchor point. An anchor point is a set of rectangles generated from the input feature map. Record the coordinates (x) of the upper left and lower right corner points of the rectangle1,y1,x2,y2) Each pixel in the figure generates a rectangle of 3 shapes with an aspect ratio of about width: height ∈ {1:1,1:2,2:1 }.
And 3.2b, outputting the excitation function of the layer. In the multi-classification scene, some inputs are mapped into real numbers between 0 and 1, the normalization guarantees that the sum is 1, and the influence of the characteristics on the probability is multiplicative in the use purpose; the following formula is adopted:
wherein i represents the ith element in the array, and the softmax value Si of the element is the ratio of the index of the element to the sum of the indexes of all the elements.
And 3.2c, frame regression. Four-dimensional vectors are typically used for bounding boxes
To indicate the center point coordinates and width and height of the frame, respectively. P represents the original prediction region and G represents the real region of the object. The purpose of bounding box regression is to be, for a given (P)x,Py,Pw,Ph) Find a mapping f such that
And is
First layer mapping:
the method is translation (Δ x, Δ y) and scaling (S)w,Sh) The following formula is adopted:
and second layer mapping:
calculated true translation (t)x,ty) And scaling (t)w,th) The following formula is adopted:
step 3.2d, non-maxima suppression. The non-maximum suppression is to solve the problem of repeated candidate frames, use a simple algorithm, obtain a suggestion list sorted according to scores and iterate the sorted list, discard and cross over (IoU) suggestions with a value greater than a certain predefined threshold, and propose a candidate frame with a higher score. In summary, the process of non-maxima suppression is an iterative-traversal-elimination process.
And 3.3, returning the prediction Region mapping generated in the step 3.2 to a feature map, performing convolution processing by 1 x 1, sending the feature map into a Region of Interest (ROI), obtaining a feature mapping with a fixed size, using the feature mapping to a neural network layer of a target detection task, and finally generating a detection network facing the landslide target through end-to-end training.
And 4, inputting target high-resolution image data to be analyzed to a detection network, performing targeted detection on the landslide target from the angle of gradually enhancing image characteristics, and finally outputting landslide target image representation.
Fig. 5 shows high resolution image data to be analyzed, in which there are several landslides. The image data is input to the landslide target-oriented detection network obtained by the processing in the above step 1-3, and the result shown in fig. 6 is output. As shown in fig. 6, the output results have identified a landslide area and are identified by a rectangular box.
It will be appreciated by those skilled in the art that the use of other shaped boxes to represent a landslide area can be readily varied based on the concepts of the present application.
In summary, the scheme of the application is based on the concept of perception level progression facing the understanding of remote sensing images from human brain to machine realization, and combines the typical characteristics of the remote sensing images and the characteristic of dependence on image data and knowledge association depth mining, so that the defect of single understanding of the complex scene images in the prior art is overcome, the association capability among the characteristics is enhanced, and the research method for realizing the rapid automatic detection of the natural disasters at present is supplemented to a certain extent; in the process, the hierarchy division of the image features based on the perception depth breaks through the limitation that a large amount of auxiliary data is used in the traditional image interpretation, and the image feature understanding from the human brain to the machine is standardized; the spatial scale and the dimension transformation are adopted as carriers for unifying the features of different organization forms, so that the complexity of information association of multi-level features in time, space and attributes is reduced, and the speed of model calculation and learning is obviously increased; the method can support the area detection of various irregular landslide targets, expand the space range of the landslide disaster monitoring system and improve the time efficiency, and is also beneficial to developing a perception constraint automatic detection method of an image data source for targets with different characteristics.
Claims (4)
1. The method for automatically detecting the landslide of the high-resolution image with multi-level perception characteristics and progressive self-learning is characterized by comprising the following steps of:
step 1, dividing three feature levels based on perceptual depth dependent image data and knowledge association by utilizing a high-resolution image of a landslide region: the method comprises the following steps that a visual perception layer, an instrument perception layer and an algorithm perception layer are used for obtaining three layers of perception feature sets which are used for supporting feature normalization mapping of perception level progression; the method comprises the steps that the color perception of human eyes to images is used as the layered representation of a visual perception layer, the data characteristic perception recorded by a remote sensing instrument is used as the layered standard of an instrument perception layer, and the depth expression characteristics of a landslide target are learned on the basis of the processing results of the visual perception layer and the instrument perception layer in an algorithm perception layer;
step 2, establishing a scale normalization mapping model for feature progressive enhancement based on the three-layer perception feature set, mapping each level perception feature element with spatial scale and dimension as carriers, and generating a multi-level feature map with highly organized semantic information;
step 3, taking the multilayer characteristic diagram as input, constructing a progressive self-learning regional network which is restrained by the comprehensive multilayer characteristic diagram, and generating a landslide target-oriented detection network through end-to-end training;
and 4, inputting target high-resolution image data to be analyzed to a detection network, performing targeted detection on the landslide target from the angle of gradually enhancing image characteristics, and finally outputting landslide target image representation.
2. The method of claim 1, wherein step 1 comprises:
extracting the data features of each image to be processed into a uniform feature item expression sample set; unifying vectors and matrixes to form basic feature items by using a basic data structure, and taking each feature item as metadata; generating three layers of feature perception sets by metadata through algorithms with different complexity; wherein,
in a visual perception layer, processing is carried out based on metadata to obtain a shallow visual characteristic of a two-dimensional single channel of image data to be processed, wherein the shallow visual characteristic is obtained by integrating one or more of an original gray matrix, a gray co-occurrence matrix and a color synthesis matrix;
processing the instrument sensing layer based on metadata to obtain the mass constraint characteristics of the image data to be processed, wherein the mass constraint characteristics are obtained by integrating spectral characteristic vectors and/or shape characteristic vectors;
and in the algorithm perception layer, the boundary characteristics and the depth semantic characteristics of the visual perception layer are mined on the basis of the characteristics extracted in the visual perception layer and the instrument perception layer.
3. The method of claim 1, wherein step 2 comprises:
step 2.1, unifying different scale characteristics: analyzing the input characteristic structure by adopting a data normalization method and carrying out unified mapping;
step 2.2, constructing a top-down path: adopting a top-down characteristic supplement enhancement idea in a characteristic pyramid network, gradually participating gradually deepened perception level characteristics from a lower layer network to a higher layer network, adding different sized convolution layers in the network for deep characteristic extraction, and obtaining three semantic characteristics of a shallow layer, a middle layer and a high layer;
step 2.3, obtaining multilayer characteristics by pooling: and (3) obtaining a multi-level feature map fusing the semantic features of each layer by utilizing the three layers of semantic features generated in the step (2.2) and performing spatial pyramid pooling operation.
4. The method of claim 1, wherein step 3 comprises:
step 3.1, inputting a multi-level feature map, carrying out specific dimension transformation, and enabling the feature map to be shared for subsequent extraction of the region candidate frame and connection of a full connection layer for output;
step 3.2, extracting a region candidate frame: generating a series of anchor points by using the RPN idea and the characteristic diagram, judging whether the anchor points belong to a target region or not by using an output layer excitation function, and then restraining and correcting the anchor points by using frame regression and a non-maximum value to obtain an accurate prediction region;
and 3.3, returning the prediction region mapping generated in the step 3.2 to a feature map, performing 1 × 1 convolution processing, then sending the prediction region mapping to an interested region pool, obtaining a feature mapping with a fixed size, using the feature mapping to a neural network layer of a target detection task, and finally generating a landslide target-oriented detection network through end-to-end training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010157591.9A CN111563408B (en) | 2020-03-09 | 2020-03-09 | High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010157591.9A CN111563408B (en) | 2020-03-09 | 2020-03-09 | High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111563408A true CN111563408A (en) | 2020-08-21 |
| CN111563408B CN111563408B (en) | 2021-04-16 |
Family
ID=72071437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010157591.9A Active CN111563408B (en) | 2020-03-09 | 2020-03-09 | High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111563408B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347960A (en) * | 2020-11-13 | 2021-02-09 | 成都理工大学 | Landslide positioning method |
| CN112964693A (en) * | 2021-02-19 | 2021-06-15 | 山东捷讯通信技术有限公司 | Raman spectrum band region segmentation method |
| CN114821288A (en) * | 2021-01-29 | 2022-07-29 | 中强光电股份有限公司 | Image identification method and unmanned aerial vehicle system |
| CN114969402A (en) * | 2022-07-25 | 2022-08-30 | 广州海普网络科技有限公司 | Vector geographic information acquisition method based on remote sensing image |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989322A (en) * | 2015-01-27 | 2016-10-05 | 同济大学 | High-resolution remote sensing image-based multi-index fusion landslide detection method |
| US20180045853A1 (en) * | 2016-08-15 | 2018-02-15 | U.S.A. As Represented By The Administrator Of The National Aeronautics And Space Administration | Landslide hazard assessment system and method |
-
2020
- 2020-03-09 CN CN202010157591.9A patent/CN111563408B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105989322A (en) * | 2015-01-27 | 2016-10-05 | 同济大学 | High-resolution remote sensing image-based multi-index fusion landslide detection method |
| US20180045853A1 (en) * | 2016-08-15 | 2018-02-15 | U.S.A. As Represented By The Administrator Of The National Aeronautics And Space Administration | Landslide hazard assessment system and method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112347960A (en) * | 2020-11-13 | 2021-02-09 | 成都理工大学 | Landslide positioning method |
| CN114821288A (en) * | 2021-01-29 | 2022-07-29 | 中强光电股份有限公司 | Image identification method and unmanned aerial vehicle system |
| CN112964693A (en) * | 2021-02-19 | 2021-06-15 | 山东捷讯通信技术有限公司 | Raman spectrum band region segmentation method |
| CN114969402A (en) * | 2022-07-25 | 2022-08-30 | 广州海普网络科技有限公司 | Vector geographic information acquisition method based on remote sensing image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111563408B (en) | 2021-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111898507B (en) | Deep learning method for predicting earth surface coverage category of label-free remote sensing image | |
| CN111563408B (en) | High-resolution image landslide automatic detection method with multi-level perception characteristics and progressive self-learning | |
| CN111986099B (en) | Tillage monitoring method and system based on convolutional neural network with residual error correction fused | |
| CN111080629B (en) | Method for detecting image splicing tampering | |
| CN108830285B (en) | Target detection method for reinforcement learning based on fast-RCNN | |
| CN109871902B (en) | SAR small sample identification method based on super-resolution countermeasure generation cascade network | |
| WO2018023734A1 (en) | Significance testing method for 3d image | |
| CN111985376A (en) | Remote sensing image ship contour extraction method based on deep learning | |
| CN112347895A (en) | Ship remote sensing target detection method based on boundary optimization neural network | |
| CN113160062B (en) | Infrared image target detection method, device, equipment and storage medium | |
| CN113609896A (en) | Object-level remote sensing change detection method and system based on dual-correlation attention | |
| CN111414954B (en) | Rock image retrieval method and system | |
| CN110738132B (en) | Target detection quality blind evaluation method with discriminant perception capability | |
| CN111027497A (en) | Weak and small target rapid detection method based on high-resolution optical remote sensing image | |
| CN111008664A (en) | Hyperspectral sea ice detection method based on space-spectrum combined characteristics | |
| Han et al. | Change guiding network: Incorporating change prior to guide change detection in remote sensing imagery | |
| CN113033315A (en) | Rare earth mining high-resolution image identification and positioning method | |
| CN116052095B (en) | Vehicle re-identification method for smart city panoramic video monitoring | |
| CN117456376A (en) | Remote sensing satellite image target detection method based on deep learning | |
| CN115953330B (en) | Texture optimization method, device, equipment and storage medium for virtual scene image | |
| Zhang et al. | Multilevel feature context semantic fusion network for cloud and cloud shadow segmentation | |
| CN115984712A (en) | Multi-scale feature-based remote sensing image small target detection method and system | |
| CN115482463A (en) | Method and system for identifying land cover of mine area of generated confrontation network | |
| CN114972335A (en) | Image classification method and device for industrial detection and computer equipment | |
| CN113012167A (en) | Combined segmentation method for cell nucleus and cytoplasm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A High Resolution Image Landslide Automatic Detection Method Based on Progressive Self learning of Multilevel Perception Features Effective date of registration: 20230625 Granted publication date: 20210416 Pledgee: Zhejiang Deqing Rural Commercial Bank Co.,Ltd. science and technology sub branch Pledgor: ZHEJIANG ZHONGHAIDA SPACE INFORMATION TECHNOLOGY Co.,Ltd. Registration number: Y2023980045468 |