CN116258967B - Urban illegal construction change detection method based on improved SNUNet-CD - Google Patents
Urban illegal construction change detection method based on improved SNUNet-CD Download PDFInfo
- Publication number
- CN116258967B CN116258967B CN202310511235.6A CN202310511235A CN116258967B CN 116258967 B CN116258967 B CN 116258967B CN 202310511235 A CN202310511235 A CN 202310511235A CN 116258967 B CN116258967 B CN 116258967B
- Authority
- CN
- China
- Prior art keywords
- data
- convolution kernel
- snunet
- change detection
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 72
- 230000008859 change Effects 0.000 title claims abstract description 63
- 238000010276 construction Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 20
- 238000005520 cutting process Methods 0.000 claims abstract description 14
- 230000003247 decreasing effect Effects 0.000 claims abstract description 4
- 230000004913 activation Effects 0.000 claims description 30
- 230000006870 function Effects 0.000 claims description 27
- 238000011176 pooling Methods 0.000 claims description 13
- 230000002159 abnormal effect Effects 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- PJOHVEQSYPOERL-SHEAVXILSA-N (e)-n-[(4r,4as,7ar,12br)-3-(cyclopropylmethyl)-9-hydroxy-7-oxo-2,4,5,6,7a,13-hexahydro-1h-4,12-methanobenzofuro[3,2-e]isoquinoline-4a-yl]-3-(4-methylphenyl)prop-2-enamide Chemical compound C1=CC(C)=CC=C1\C=C\C(=O)N[C@]1(CCC(=O)[C@@H]2O3)[C@H]4CC5=CC=C(O)C3=C5[C@]12CCN4CC1CC1 PJOHVEQSYPOERL-SHEAVXILSA-N 0.000 claims description 6
- 102100023126 Cell surface glycoprotein MUC18 Human genes 0.000 claims description 6
- 101000799969 Escherichia coli (strain K12) Alpha-2-macroglobulin Proteins 0.000 claims description 6
- 101000623903 Homo sapiens Cell surface glycoprotein MUC18 Proteins 0.000 claims description 6
- 210000002364 input neuron Anatomy 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 210000004205 output neuron Anatomy 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 230000009467 reduction Effects 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 230000006399 behavior Effects 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000010606 normalization Methods 0.000 description 6
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002349 favourable effect Effects 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000011835 investigation Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 101100322030 Drosophila melanogaster Abl gene Proteins 0.000 description 1
- 108700026244 Open Reading Frames Proteins 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000009430 construction management Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/60—Planning or developing urban green infrastructure
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
The invention provides an urban illegal construction change detection method based on improved SNUNet-CD, and relates to the technical field of electronic information; the method comprises the following steps: s10, collecting regional point cloud data and oblique photographic data in two periods, and generating color point cloud data; s20, rearranging the color point cloud data into a data structure to form three-dimensional synthesized data; s30, generating a plurality of data blocks as a data set; s40, constructing a change detection algorithm model applicable to the urban illegal building detection scene by taking the SNUNet-CD as a basic model; s50, cutting the processed two-stage three-dimensional synthesized data according to the image cutting sequence in the step S30 to generate a plurality of data pairs with serial numbers, and generating city building change detection results; s60, displaying the increasing and decreasing change condition of the building; the beneficial effects of the invention are as follows: the urban area can be subjected to detection of the illegal behaviors of the building based on the orthographic images and the point cloud data.
Description
Technical Field
The invention relates to the technical field of electronic information, in particular to an urban illegal building change detection method based on improved SNUNet-CD.
Background
With the rapid development of cities, a large number of buildings are put into construction, and the building area is greatly increased. The method has the advantages that a large amount of land is occupied for development or the original building design is changed for covering illegal buildings such as floors, due to the fact that human resources are limited, inspection means fall behind and the like, the illegal buildings and the transformation behaviors are difficult to effectively restrain, great economic loss is caused, and management difficulty is increased.
In the prior art, a remote sensing technology is generally adopted to regularly shoot map pieces at fixed points for screening, and in order to improve efficiency, a plurality of methods are proposed in the prior art for automatic matching detection, and abnormal positions are screened through coordinate matching comparison of two-period images and are used as judgment basis of illegal buildings.
The above-described method has the following general disadvantages: the acquisition of the high-altitude remote sensing image is difficult, the period is long, the timely illegal construction inspection is difficult, and the illegal construction is possibly blocked due to adverse factors such as cloud layers. The middle-low altitude remote sensing change detection only considers the image characteristic change of a two-dimensional plane, does not fully consider the images of materials and heights, is limited by the model images, and ensures that the investigation efficiency and accuracy are challenged greatly.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides an urban illegal construction change detection method based on improved SNUNet-CD, which can detect the illegal behaviors of buildings in urban areas based on orthographic images and point cloud data.
The technical scheme adopted for solving the technical problems is as follows: in a method for detecting urban construction violations based on improved SNUNet-CD, the improvement comprising the steps of:
s10, acquiring two-period regional point cloud data and oblique photographic data, and respectively constructing a point cloud model, an oblique model and an orthographic image;
according to the position matching information of the inclination model and the point cloud model, RGB information is mapped onto the point cloud data according to the space position by utilizing the inclination model, and color point cloud data are generated;
s20, rearranging the color point cloud data into a data structure to form three-dimensional synthesized data taking set size, RGB and point cloud intensity as channels;
s30, carrying out data annotation on the two-stage orthographic image by using Labelme to generate a MASK file, matching the two-stage three-dimensional synthesized data with the MASK file according to the position information, completing image cutting, and generating a plurality of data blocks as a data set;
s40, constructing a change detection algorithm model applicable to urban illegal building detection scenes by taking SNUNet-CD as a basic model, and training by inputting two-stage three-dimensional synthetic data into the change detection algorithm model, wherein MASK files are used as labels;
s50, cutting the processed two-stage three-dimensional synthesized data according to the sequence of image cutting in the step S30, generating a plurality of data pairs with serial numbers, inputting the data pairs into a change detection algorithm model one by one for detection, and re-splicing according to the serial numbers after the detection of the complete area is completed to generate city building change detection results;
and S60, mapping the urban building change detection result into the orthographic image of the second period according to the position information, so as to display the increasing and decreasing change condition of the building.
Further, in step S10, acquisition of multi-period regional point cloud data and oblique photography data is achieved by integrating the oblique camera and the laser lidar to the manned helicopter.
Further, in step S20, the set size includes a length, a width and a height of the data, where the length is X, the width is Y and the height is H.
Further, in step S30, the size of the data block is 1024×1024.
Further, in the step S40, the change detection algorithm model is an encoding-decoding structure, a twin network is adopted as an encoder, a dual-temporal image is used as an input, the dual-temporal image is respectively input into two branches of the twin network, and the two branches of the twin network have the characteristic of parameter sharing;
the two time-image images are subjected to layer-by-layer feature extraction through convolution filters with the same parameters, a feature map of the relative position is generated, and features of different layers are stacked according to channels so as to ensure the integrity of branch feature information.
Further, in the step S40, during the convolution downsampling process of the dual-temporal image data, features of two branches are fused, and the fused features are sequentially transmitted to the decoder through jump connection to compensate for deep position information loss of the decoder;
the three-dimensional synthesized data of two phases is input into a twin encoder network, and the downsampled output of each node has a sub-decoder to restore the original size; by means of a jump connection, the fine position features in the encoder will be transferred to the four sub-decoders, the shallow position information will be applied directly to the deep layer, maintaining fine granularity information.
Further, the change detection algorithm model comprises an ECAM module;
in the step S40, the four encoder outputs are stacked to form a whole, and after the MCAM module is combined to generate the channel attention feature, the multi-scale connection is performed to form an output combined with the attention feature increase;
the ECAM module is adopted, namely, the feature map is subjected to spatial pyramid maximum pooling, spatial pyramid average pooling and convolution operation respectively to generate three rows of features with the same length, the three rows of features are respectively input into the MLP to extract deep features, and after output results are added, a multi-level channel attention feature map is generated through activation of a MISS activation function.
Further, in step S40, a change detection algorithm model suitable for the urban illegal building detection scene is constructed, which includes: convolution Block in SNUNet-CD is improved to construct convolution blocks CB1 and CB2;
the convolution block CB1 comprises a first convolution kernel, a second convolution kernel, a batch standardization module BN and a third convolution kernel, wherein three-dimensional synthesized data sequentially passes through the first convolution kernel, the second convolution kernel and the batch standardization module BN, passes through the third convolution kernel and the batch standardization module BN after being processed by an activation function MISS, and passes through the activation function MISS again after being overlapped with the output of the second convolution kernel;
the convolution block CB2 comprises a third convolution kernel and a batch standardization module BN, three-dimensional synthesized data sequentially passes through the third convolution kernel and the batch standardization module BN, passes through the third convolution kernel and the batch standardization module BN after being processed by an activation function MISS, and is overlapped with the output of the third convolution kernel, and then passes through the activation function MISS again;
the first convolution kernel is a 1x1xC convolution kernel, namely a convolution kernel with the same channel number as the original data is adopted; the second convolution kernel is a convolution kernel of 5x5xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 5x 5; the third convolution kernel is a convolution kernel of 3x3xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 3x 3.
Further, in step S40, two-phase three-dimensional synthesized data is input into the change detection algorithm model for training, including the following steps:
s401, three-dimensional convolution operation, wherein the formula is as follows:
；
wherein phi is an activation function, i represents the number of layers of the neural network; j represents a convolution kernel sequence number; n is the number of three-dimensional data channels, P, Q, H is the length, width and height of the convolution kernel, m, P, Q and H respectively represent the current values of N, P, Q and H; x is the abscissa of the three-dimensional data, y is the ordinate of the three-dimensional data, and z is the height coordinate of the three-dimensional data; w represents a weight value, v represents an activation value, and b represents a bias value;
s402, a Mish activation function:
；
wherein ,；
compared with relu, the Mish activation function is favorable for keeping a small negative value, so that the network gradient flow is stabilized, and the gradient is prevented from disappearing;
s403, batch standardization:
；
wherein ,representing the value of the ith input node of the layer when the b-th sample of the current batch is input, is>Is thatThe length of the constructed row vector is batch size m, mu and sigma are the average value and standard deviation of the row, ϵ is an extremely small amount for preventing zero removal from being introduced, and gamma and beta are scale and shift parameters of the row;
s404, the model Loss function Loss consists of two parts, namely weighted cross entropyAnd overlap combined loss：
；
wherein The formula is:
；
h and W represent the height and width of the variation graph y ˆ, with the value of "class" being 0 or 1, corresponding to "unchanged" and "changed" pixels, respectively;
；
y ˆ calculates the overlap loss through the Softmax () layer, Y being the true value;
；
wherein ,referring to the kth output neuron, the denominator is the sum of all the input neuron indices and the numerator is the kth input neuron.
Further, the step S60 further includes:
s70, manually checking the abnormal position and recording the illegal construction condition of the abnormal area.
The beneficial effects of the invention are as follows: the method can detect the illegal behaviors of the building in the urban area based on the multi-period orthographic images and the point cloud data, greatly improves the efficiency and the accuracy of illegal detection, and reduces the labor and time cost of urban illegal construction management.
Drawings
Fig. 1 is a schematic diagram of a change detection algorithm model in a city violation change detection method based on improved SNUNet-CD according to the present invention.
Fig. 2 is a schematic diagram of a convolution block CB1 according to the present invention.
Fig. 3 is a schematic diagram of a convolution block CB2 in the present invention.
FIG. 4 is a schematic diagram of spatial pyramid pooling in accordance with the present invention.
Detailed Description
The invention will be further described with reference to the drawings and examples.
The conception, specific structure, and technical effects produced by the present invention will be clearly and completely described below with reference to the embodiments and the drawings to fully understand the objects, features, and effects of the present invention. It is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments, and that other embodiments obtained by those skilled in the art without inventive effort are within the scope of the present invention based on the embodiments of the present invention. In addition, all the coupling/connection relationships referred to in the patent are not direct connection of the single-finger members, but rather, it means that a better coupling structure can be formed by adding or subtracting coupling aids depending on the specific implementation. The technical features in the invention can be interactively combined on the premise of no contradiction and conflict.
Referring to fig. 1 to 4, the invention provides an improved SNUNet-CD-based urban illegal construction change detection method, in which a data source is an integrated inclined five-lens camera and laser Lidar device carried on a manned helicopter, inclined images and Lidar point clouds in different periods can be acquired through multiple times of flight data acquisition in urban areas, and a data processing technology is used for generating a live-action three-dimensional model and a point cloud model. And realizing data fusion aiming at the position matching information of the live-action three-dimensional model and the point cloud model, and carrying out fusion processing on the two heterogeneous data to generate the point cloud model with color characteristics.
And the detection of the illegal change is realized by adopting a three-dimensional SNUNet-CD change detection algorithm, color point cloud data in different periods are processed into two region synthesized data of five channels of R, G, B, point cloud intensity (Q) and height (H), the two synthesized data are respectively divided into a plurality of data with the same size and corresponding to each other, and the data are marked according to an actual investigation result to generate a data set. And then, constructing a SNunet-CD-based change detection algorithm model, learning the two-stage data and the annotation information, extracting the building abnormal position features of the two-stage images by the coding region, and mapping the abstract features into change detection region images by the decoding region. The change detection Mask is mapped to the corresponding position of the live-action three-dimensional image, change position information is provided for law enforcement personnel, and the second confirmation is carried out on site by combining a conventional illegal building inspection means, so that the illegal building detection efficiency can be greatly improved, the reliability of results is improved, and the city management cost is reduced.
To explain the technical solution of the present invention in more detail, the present invention provides a specific embodiment of a method for detecting urban illegal changes based on improved SNUNet-CD, specifically, the method includes the following steps:
s10, acquiring two-period regional point cloud data and oblique photographic data, respectively constructing a point cloud model, an oblique model and an orthographic image, and registering coordinates; according to the position matching information of the inclination model and the point cloud model, RGB information is mapped onto the point cloud data according to the space position by utilizing the inclination model, and color point cloud data are generated; in the embodiment, acquisition of multi-period regional point cloud data and oblique photography data is realized by integrating an oblique camera and a laser lidar to a manned helicopter;
s20, rearranging the color point cloud data into a data structure to form three-dimensional synthesized data taking set size, RGB and point cloud intensity as channels;
in this embodiment, in the step S20, the set size includes a length, a width and a height of the data, where the length is X, the width is Y and the height is H.
S30, carrying out data annotation on the two-stage orthographic image by using Labelme to generate a MASK file, matching the two-stage three-dimensional synthesized data with the MASK file according to the position information, completing image cutting, and generating a plurality of data blocks as a data set;
in the embodiment, the two-period three-dimensional synthesized data is marked with a change area according to the actual investigation condition, the non-change area is set to 0, the building information is increased to 1, and the building information is reduced to 2; in addition, when image cutting is performed, the image cutting is performed into a plurality of data blocks with the range of 1024 x 1024, and as a data set, a single sample of the data set includes: two-stage region three-dimensional synthetic data and a region MASK file; and, regard two-stage data and label as a sample set, divide the sample set, form training set, verify the proportion of the set, test set and be 8:1:1, a data set of 1;
s40, constructing a change detection algorithm model applicable to urban illegal building detection scenes by taking SNUNet-CD as a basic model, and training by inputting two-stage three-dimensional synthetic data into the change detection algorithm model, wherein MASK files are used as labels;
in the embodiment, SNUNet-CD is used as a basic model to perform data-specific and scene-adaptive optimization, so that a component generates a change detection algorithm model which is suitable for urban illegal building detection scenes. The SNUNet-CD is a change detection model, is an improved model based on a UNet++ model, is a semantic segmentation model based on a convolutional neural network, is added with multi-time phase input based on the UNet++ model, and is combined with an attention mechanism to realize the segmentation accuracy of image change positions.
The method is shown in combination with fig. 1, namely, a structural schematic diagram of a modified model aiming at a SNUNet-CD as a basic model in the invention, the modified model is called a change detection algorithm model, the change detection algorithm model is of an encoding-decoding structure, a twin network is adopted as an encoder, a double-time image is taken as an input and is respectively input into two branches of the twin network, and the two branches of the twin network have the characteristic of parameter sharing; the two time-image images are subjected to layer-by-layer feature extraction through convolution filters with the same parameters, a feature map of the relative position is generated, and features of different layers are stacked according to channels so as to ensure the integrity of branch feature information.
In the process of convoluting down-sampling the double-time image data, the features of the two branches are fused, and the fused features are sequentially transmitted to a decoder through jump connection to compensate the deep position information loss of the decoder; the three-dimensional synthesized data of two phases is input into a twin encoder network, and the downsampled output of each node has a sub-decoder to restore the original size; by means of a jump connection, the fine position features in the encoder will be transferred to the four sub-decoders, the shallow position information will be applied directly to the deep layer, maintaining fine granularity information.
In the invention, as shown in fig. 1 and 4, a CAM module is modified into an MCAM module by taking SNUNet-CD as a basic model, the modified point of the module is maximum pooling and average pooling, three parts are respectively spatial pyramid maximum pooling, spatial pyramid average pooling and convolution layers, the three parts are respectively input into an MLP for calculation, full-connection layer characteristics with equal length are generated, the three parts of characteristics are added and then input into a dash activation function for completing nonlinear conversion calculation, and an output result is taken as a multi-scale channel attention characteristic.
In this embodiment, referring to fig. 1, the downsampling finger scale is reduced, 2x2 max pooling and 2x2 average pooling are adopted, the upsampling finger scale is increased, the jump connection can implement the copy cutting operation, so that the left and right are consistent, then the splicing is performed, the weight sharing finger double-time phase convolution kernel parameters are consistent, and the stacking finger stacks the convolved channels.
With continued reference to FIG. 1, the change detection algorithm model includes an ECAM module (Integrated channel attention module); stacking the four encoder outputs to form a whole, generating channel attention characteristics by combining an MCAM module (i.e. a channel attention module), and performing multi-scale connection to form an output combined with the increase of the attention characteristics; the ECAM module adopts an MCAM module, namely the feature map is respectively input into the MLP to extract deep features after being subjected to maximum pooling and average pooling dimension reduction, and the channel attention feature map is generated through activation of the MISS activation function after output results are added.
In step S40, a change detection algorithm model suitable for the urban illegal building detection scene is constructed, including: the improvement to Convolution Block in SNUNet-CD is to construct convolution blocks CB1 and CB2 (convolution block 1 and convolution block 2, respectively, in fig. 1). Referring to fig. 2, a schematic diagram of a convolution block CB1 is shown, where the convolution block CB1 includes a first convolution kernel, a second convolution kernel, a batch normalization module BN, and a third convolution kernel, and three-dimensional synthesized data sequentially passes through the first convolution kernel, the second convolution kernel, and the batch normalization module BN, after being processed by an activation function MISS, passes through the third convolution kernel and the batch normalization module BN, and after being overlapped with an output of the second convolution kernel, passes through the processing of the activation function MISS again; referring to fig. 3, a schematic diagram of a convolution block CB2 is shown, where the convolution block CB2 includes a third convolution kernel and a batch normalization module BN, and three-dimensional synthesized data sequentially passes through the third convolution kernel and the batch normalization module BN, after being processed by an activation function MISS, passes through the third convolution kernel and the batch normalization module BN, and after being overlapped with an output of the third convolution kernel, passes through the processing of the activation function MISS again. In this embodiment, the first convolution kernel is a convolution kernel of 1x1xC, that is, a convolution kernel with the same number of channels as that of the original data is adopted; the second convolution kernel is a convolution kernel of 5x5xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 5x 5; the third convolution kernel is a convolution kernel of 3x3xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 3x 3.
In the above embodiment, in step S40, two-phase three-dimensional synthesized data is input into the change detection algorithm model for training, which includes the following steps:
s401, three-dimensional convolution operation, wherein the formula is as follows:
；
wherein phi is an activation function, i represents the number of layers of the neural network; j represents a convolution kernel sequence number; n is the number of three-dimensional data channels, P, Q, H is the length, width and height of the convolution kernel, m, P, Q and H respectively represent the current values of N, P, Q and H; x is the abscissa of the three-dimensional data, y is the ordinate of the three-dimensional data, and z is the height coordinate of the three-dimensional data; w represents a weight value, v represents an activation value, and b represents a bias value;
s402, a Mish activation function:
；
wherein ,；
compared with relu, the Mish activation function is favorable for keeping a small negative value, so that the network gradient flow is stabilized, and the gradient is prevented from disappearing;
s403, batch standardization:
；
wherein ,representing the value of the ith input node of the layer when the b-th sample of the current batch is input, is>Is thatThe length of the constructed row vector is batch size m, mu and sigma are the average value and standard deviation of the row, ϵ is an extremely small amount for preventing zero removal from being introduced, and gamma and beta are scale and shift parameters of the row;
s404, the model Loss function Loss consists of two parts, namely weighted cross entropyAnd overlap combined loss：
；
wherein The formula is:
；
h and W represent the height and width of the variation graph y ˆ, with the value of "class" being 0 or 1, corresponding to "unchanged" and "changed" pixels, respectively;
；
y ˆ calculates the overlap loss through the Softmax () layer, Y being the true value;
；
wherein ,referring to the kth output neuron, the denominator is the sum of all the input neuron indices and the numerator is the kth input neuron.
S50, cutting the processed two-stage three-dimensional synthesized data according to the sequence of image cutting in the step S30, generating a plurality of data pairs with serial numbers, inputting the data pairs into a change detection algorithm model one by one for detection, and re-splicing according to the serial numbers after the detection of the complete area is completed to generate city building change detection results;
s60, mapping the urban building change detection result into an orthographic image of the second period according to the position information, so as to display the increasing and decreasing change condition of the building;
s70, manually checking the abnormal position and recording the illegal construction condition of the abnormal area.
Based on the above, the invention provides a city illegal change detection method based on improved SNUNet-CD, and provides a data set construction method for multi-stage and multi-source data fusion, which can provide various optimized convolution blocks, can reduce the calculated amount and is light in weight; and a deep neural network model suitable for urban illegal change detection is constructed, so that the internal semantic information of the data can be refined, and the change detection precision is improved.
Therefore, by the urban illegal building change detection method based on the improved SNUNet-CD, the urban area can be subjected to the detection of the illegal building behaviors based on the multi-period orthographic images and the point cloud data, the efficiency and the accuracy of illegal detection are greatly improved, the labor and time cost of urban illegal building management are reduced, and firm, favorable and high-visualization analysis results are provided for law enforcement of related departments.
While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are included in the scope of the present invention as defined in the appended claims.
Claims (10)
1. The city illegal change detection method based on the improved SNUNet-CD is characterized by comprising the following steps of:
s10, acquiring two-period regional point cloud data and oblique photographic data, and respectively constructing a point cloud model, an oblique model and an orthographic image;
according to the position matching information of the inclination model and the point cloud model, RGB information is mapped onto the point cloud data according to the space position by utilizing the inclination model, and color point cloud data are generated;
s20, rearranging the color point cloud data into a data structure to form three-dimensional synthesized data taking set size, RGB and point cloud intensity as channels;
s30, carrying out data annotation on the two-stage orthographic image by using Labelme to generate a MASK file, matching the two-stage three-dimensional synthesized data with the MASK file according to the position information, completing image cutting, and generating a plurality of data blocks as a data set;
s40, constructing a change detection algorithm model applicable to urban illegal building detection scenes by taking SNUNet-CD as a basic model, and training by inputting two-stage three-dimensional synthetic data into the change detection algorithm model, wherein MASK files are used as labels;
s50, cutting the processed two-stage three-dimensional synthesized data according to the sequence of image cutting in the step S30, generating a plurality of data pairs with serial numbers, inputting the data pairs into a change detection algorithm model one by one for detection, and re-splicing according to the serial numbers after the detection of the complete area is completed to generate city building change detection results;
and S60, mapping the urban building change detection result into the orthographic image of the second period according to the position information, so as to display the increasing and decreasing change condition of the building.
2. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 1, wherein in step S10, acquisition of multi-period regional point cloud data and oblique photography data is achieved by integrating oblique cameras and laser lidar to a manned helicopter.
3. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 1, wherein in step S20, the set dimensions include length, width and height of the data, the length being X, the width being Y and the height being H.
4. The method for detecting urban traffic violation change based on modified SNUNet-CD according to claim 1, wherein in step S30, the size of the data block is 1024 x 1024.
5. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 1, wherein in step S40, the change detection algorithm model is an encoding-decoding structure, a twin network is adopted as an encoder, a dual-temporal image is adopted as an input, the images are respectively input to two branches of the twin network, and the two branches of the twin network have the characteristic of parameter sharing;
the two time-image images are subjected to layer-by-layer feature extraction through convolution filters with the same parameters, a feature map of the relative position is generated, and features of different layers are stacked according to channels so as to ensure the integrity of branch feature information.
6. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 5, wherein in step S40, during the process of convoluting down-sampling the dual-temporal image data, the features of the two branches are fused, and the fused features are sequentially transmitted to the decoder through jump connection, so as to compensate the deep position information loss of the decoder;
the three-dimensional synthesized data of two phases is input into a twin encoder network, and the downsampled output of each node has a sub-decoder to restore the original size; by means of a jump connection, the fine position features in the encoder will be transferred to the four sub-decoders, the shallow position information will be applied directly to the deep layer, maintaining fine granularity information.
7. The improved SNUNet-CD based urban violation change detection method according to claim 6, wherein the change detection algorithm model comprises an ECAM module;
in the step S40, the four encoder outputs are stacked to form a whole, and after the MCAM module is combined to generate the channel attention feature, the multi-scale connection is performed to form an output combined with the attention feature increase;
the ECAM module adopts an MCAM module, namely the feature map is respectively input into the MLP to extract deep features after being subjected to maximum pooling and average pooling dimension reduction, and the channel attention feature map is generated through activation of the MISS activation function after output results are added.
8. The method for detecting urban illegal building change based on improved SNUNet-CD according to claim 7, wherein the constructing a change detection algorithm model applicable to the urban illegal building detection scene in step S40 comprises: convolution Block in SNUNet-CD is improved to construct convolution blocks CB1 and CB2;
the convolution block CB1 comprises a first convolution kernel, a second convolution kernel, a batch standardization module BN and a third convolution kernel, wherein three-dimensional synthesized data sequentially passes through the first convolution kernel, the second convolution kernel and the batch standardization module BN, passes through the third convolution kernel and the batch standardization module BN after being processed by an activation function MISS, and passes through the activation function MISS again after being overlapped with the output of the second convolution kernel;
the convolution block CB2 comprises a third convolution kernel and a batch standardization module BN, three-dimensional synthesized data sequentially passes through the third convolution kernel and the batch standardization module BN, passes through the third convolution kernel and the batch standardization module BN after being processed by an activation function MISS, and is overlapped with the output of the third convolution kernel, and then passes through the activation function MISS again;
the first convolution kernel is a 1x1xC convolution kernel, namely a convolution kernel with the same channel number as the original data is adopted; the second convolution kernel is a convolution kernel of 5x5xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 5x 5; the third convolution kernel is a convolution kernel of 3x3xC, namely the convolution kernel which adopts the same channel number as the original data and has the length and width of 3x 3.
9. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 8, wherein in step S40, two-phase three-dimensional synthetic data are input into the change detection algorithm model for training, comprising the steps of:
s401, three-dimensional convolution operation, wherein the formula is as follows:
；
wherein phi is an activation function, i represents the number of layers of the neural network; j represents a convolution kernel sequence number; n is the number of three-dimensional data channels, P, Q, H is the length, width and height of the convolution kernel, m, P, Q and H respectively represent the current values of N, P, Q and H; x is the abscissa of the three-dimensional data, y is the ordinate of the three-dimensional data, and z is the height coordinate of the three-dimensional data; w represents a weight value, v represents an activation value, and b represents a bias value;
s402, a Mish activation function:
；
wherein ,；
s403, batch standardization:
；
wherein ,representing the value of the ith input node of the layer when the b-th sample of the current batch is input, is>Is thatThe length of the constructed row vector is batch size m, mu and sigma are the average value and standard deviation of the row, ϵ is an extremely small amount for preventing zero removal from being introduced, and gamma and beta are scale and shift parameters of the row;
s404, the model Loss function Loss consists of two parts, namely weighted cross entropyAnd overlap binding loss->：
；
wherein The formula is:
；
h and W represent the height and width of the variation graph y ˆ, with the value of "class" being 0 or 1, corresponding to "unchanged" and "changed" pixels, respectively;
；
y ˆ calculates the overlap loss through the Softmax () layer, Y being the true value;
；
wherein ,referring to the kth output neuron, the denominator is the sum of all the input neuron indices and the numerator is the kth input neuron.
10. The method for detecting urban illegal changes based on improved SNUNet-CD according to claim 1, wherein the step S60 further comprises:
s70, manually checking the abnormal position and recording the illegal construction condition of the abnormal area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310511235.6A CN116258967B (en) | 2023-05-09 | 2023-05-09 | Urban illegal construction change detection method based on improved SNUNet-CD |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310511235.6A CN116258967B (en) | 2023-05-09 | 2023-05-09 | Urban illegal construction change detection method based on improved SNUNet-CD |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116258967A CN116258967A (en) | 2023-06-13 |
| CN116258967B true CN116258967B (en) | 2023-08-04 |
Family
ID=86679605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310511235.6A Active CN116258967B (en) | 2023-05-09 | 2023-05-09 | Urban illegal construction change detection method based on improved SNUNet-CD |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116258967B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118134936A (en) * | 2024-05-09 | 2024-06-04 | 湖南省第二测绘院 | Method and device for detecting changes of illegal buildings through video images of iron towers |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093466A (en) * | 2013-01-21 | 2013-05-08 | 武汉大学 | Building three-dimensional change detection method based on LiDAR point cloud and image |
| CN106091972A (en) * | 2016-06-30 | 2016-11-09 | 河海大学 | A kind of building change detecting method based on moving window subpoint density |
| CN111323788A (en) * | 2020-01-19 | 2020-06-23 | 北京建筑大学 | Building change monitoring method and device and computer equipment |
| CN112819753A (en) * | 2021-01-12 | 2021-05-18 | 香港理工大学深圳研究院 | Building change detection method and device, intelligent terminal and storage medium |
| CN113762931A (en) * | 2021-09-10 | 2021-12-07 | 广州建通测绘地理信息技术股份有限公司 | Method for monitoring illegal building based on DSM and DOM |
| CN114332616A (en) * | 2021-12-29 | 2022-04-12 | 甘德强 | Building change detection method based on orthoimage and oblique photography data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10706185B2 (en) * | 2016-04-26 | 2020-07-07 | Arizona Board Of Regents On Behalf Of Arizona State University | Systems and methods for automated spatial change detection and control of buildings and construction sites using three-dimensional laser scanning data |
-
2023
- 2023-05-09 CN CN202310511235.6A patent/CN116258967B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093466A (en) * | 2013-01-21 | 2013-05-08 | 武汉大学 | Building three-dimensional change detection method based on LiDAR point cloud and image |
| CN106091972A (en) * | 2016-06-30 | 2016-11-09 | 河海大学 | A kind of building change detecting method based on moving window subpoint density |
| CN111323788A (en) * | 2020-01-19 | 2020-06-23 | 北京建筑大学 | Building change monitoring method and device and computer equipment |
| CN112819753A (en) * | 2021-01-12 | 2021-05-18 | 香港理工大学深圳研究院 | Building change detection method and device, intelligent terminal and storage medium |
| CN113762931A (en) * | 2021-09-10 | 2021-12-07 | 广州建通测绘地理信息技术股份有限公司 | Method for monitoring illegal building based on DSM and DOM |
| CN114332616A (en) * | 2021-12-29 | 2022-04-12 | 甘德强 | Building change detection method based on orthoimage and oblique photography data |
Non-Patent Citations (1)
| Title |
|---|
| "基于RS和GIS技术的城市违建治理动态监测研究与应用";黄蓉 等;《城市勘测》(第第6期期);第53-57页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116258967A (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800736B (en) | Road extraction method based on remote sensing image and deep learning | |
| CN114708585B (en) | Attention mechanism-based millimeter wave radar and vision fusion three-dimensional target detection method | |
| Liu et al. | Multiscale U-shaped CNN building instance extraction framework with edge constraint for high-spatial-resolution remote sensing imagery | |
| CN109934153B (en) | Building extraction method based on gating depth residual error optimization network | |
| CN105141889A (en) | Power transmission line intelligent patrol system based on image vision | |
| Dai et al. | Fast and accurate cable detection using CNN | |
| CN116258967B (en) | Urban illegal construction change detection method based on improved SNUNet-CD | |
| Xiao et al. | Treetop detection using convolutional neural networks trained through automatically generated pseudo labels | |
| CN113191204B (en) | Multi-scale blocking pedestrian detection method and system | |
| CN111797920A (en) | Remote sensing extraction method and system for depth network impervious surface with gate control feature fusion | |
| CN116403121A (en) | Remote sensing image water area segmentation method, system and equipment for multi-path fusion of water index and polarization information | |
| CN116994135A (en) | Ship target detection method based on vision and radar fusion | |
| CN113743358A (en) | Landscape visual feature recognition method based on all-dimensional acquisition and intelligent calculation | |
| CN116740516A (en) | Target detection method and system based on multi-scale fusion feature extraction | |
| CN114511788A (en) | Slope crack identification method, system, equipment and storage medium | |
| CN113610032A (en) | Building identification method and device based on remote sensing image | |
| Algamdi et al. | Instant Counting & Vehicle Detection during Hajj Using Drones | |
| CN113111740A (en) | Characteristic weaving method for remote sensing image target detection | |
| CN116543165B (en) | Remote sensing image fruit tree segmentation method based on dual-channel composite depth network | |
| CN114882490B (en) | Unlimited scene license plate detection and classification method based on point-guided positioning | |
| KR102416714B1 (en) | System and method for city-scale tree mapping using 3-channel images and multiple deep learning | |
| CN114550016B (en) | Unmanned aerial vehicle positioning method and system based on context information perception | |
| CN115424140A (en) | Satellite-borne mass image data distributed cooperative rapid high-precision processing system and method | |
| Häufel et al. | Evaluation of CNNs for land cover classification in high-resolution airborne images | |
| Karwowska et al. | MCWESRGAN: improving enhanced super-resolution generative adversarial network for satellite images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |