CN115346137A - High-standard farmland land mass vectorization extraction method based on multi-task learning - Google Patents
High-standard farmland land mass vectorization extraction method based on multi-task learning Download PDFInfo
- Publication number
- CN115346137A CN115346137A CN202211257905.8A CN202211257905A CN115346137A CN 115346137 A CN115346137 A CN 115346137A CN 202211257905 A CN202211257905 A CN 202211257905A CN 115346137 A CN115346137 A CN 115346137A
- Authority
- CN
- China
- Prior art keywords
- boundary
- module
- land
- parcel
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 37
- 230000011218 segmentation Effects 0.000 claims description 35
- 238000010586 diagram Methods 0.000 claims description 30
- 238000005070 sampling Methods 0.000 claims description 18
- 230000002708 enhancing effect Effects 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 9
- 230000004913 activation Effects 0.000 claims description 7
- 230000003993 interaction Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 150000001875 compounds Chemical class 0.000 claims description 5
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 238000012805 post-processing Methods 0.000 claims description 5
- 125000004432 carbon atom Chemical group C* 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 3
- 230000008447 perception Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 abstract description 4
- 230000003287 optical effect Effects 0.000 abstract description 2
- 238000003708 edge detection Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a high-standard farmland plot vectorization extraction method based on multi-task learning, which mainly comprises the following steps: designing an ultra-complete high-resolution dense multi-scale module to jointly extract shape variability features of the parcel objects and edge features related to parcel boundary categories, and designing a region-boundary-parcel decoupling multi-task module to jointly optimize parcel object extraction and parcel boundary extraction tasks; and designing a plot boundary-object interactive vectorization module to further optimize the adhesion phenomenon of the plot object result. The method can be suitable for optical remote sensing images under various spatial resolutions to be used for a high-standard farmland plot vectorization extraction task, and compared with the existing farmland plot extraction method, the method provided by the invention can process a vector result, has higher plot object extraction precision and smooth plot boundaries, and can meet application requirements of plot scale crop classification, yield estimation and the like.
Description
Technical Field
The invention belongs to the technical field of remote sensing image processing, and particularly relates to a high-standard farmland land parcel vectorization extraction method based on multi-task learning.
Background
Accurate farmland plot boundary information is important for farmers and agricultural managers to monitor crop growth in the field of precision agricultural services. However, the fully automatic extraction of the boundary of the farmland plot from the satellite image still faces a challenge. On the one hand, the land mass is fine and dense and has shape variability. The different plots have larger form difference and different sizes; on the other hand, as crops are mostly planted in the land parcels, the land parcels have larger difference of phenology between different land parcels, and the land parcels show space-time spectrum variability; in addition, from the visual effect, the plot is obscured by edge occlusion, most of the existing methods are difficult to completely identify plot objects, and the existing farmland plot data set is concentrated on a small-range single satellite image data source.
The current farmland plot extraction methods comprise methods based on edge detection, object segmentation and deep learning. The edge detection-based method mainly defines an edge detection operator manually, cannot automatically determine algorithm parameters, has very rough segmented boundaries and a saw-tooth phenomenon, and is difficult to apply to a region with dense plots. The object segmentation based method carries out edge connection or region segmentation through a region similarity criterion, and then carries out region classification through a sub-region result of segmentation, but the traditional methods can only segment plots and cannot extract farmland plots and boundaries at the same time. The deep learning method takes a land area or a land boundary as an identification object based on an edge or a region, tries the recognition object as a two-classification semantic segmentation task, trains two semantic segmentation networks, and is respectively used for detecting the land boundary and the land area, and finally outputs an example grid result of the land through post-processing. However, requirements cannot be met in subsequent applications, extraction results are rough, further post-processing is needed to obtain final segmentation results, a large number of parameters are needed for setting post-processing methods and effects, and the degree of automation is not high.
Disclosure of Invention
The invention provides a high-standard farmland land parcel vectorization extraction method based on region-boundary-land parcel decoupling multi-task learning, which has the following advantages: the method comprises the steps of designing an ultra-complete high-resolution dense multi-scale module to extract shape variability features of a parcel object and edge features related to parcel boundary classes. And secondly, designing a region-boundary-plot decoupling multitask module to jointly optimize the plot object extraction and the plot boundary extraction tasks, realizing multitask cooperative supervision of the plot boundary and the plot objects, and relieving the adhesion phenomenon among the plot objects. And thirdly, designing a plot boundary-object interactive vectorization module to further optimize the adhesion phenomenon of the plot object result, and generating a plot vectorization result which can be directly applied to subsequent agricultural remote sensing tasks. And fourthly, the method is applied to a plurality of domestic and foreign research areas, the optimal identification precision is obtained, and the method can be suitable for the optical satellite remote sensing images with medium and high resolution.
The invention provides a high-standard farmland plot vectorization extraction method based on multitask learning, which comprises the following steps:
step 1, performing cutting and normalization pretreatment on an input image;
step 2, extracting multi-scale features and edge detail features of the preprocessed image by using a multi-scale module;
step 3, obtaining a land area segmentation result, a land boundary prediction result and a land object segmentation result through a multitask module according to the multi-scale features and the edge detail features;
step 4, inputting the result output in the step 3 into a multi-task joint optimization loss function, and outputting the resultlossValue, back propagationlossUpdating network model parameters by values, wherein the network model packet is composed of a multi-scale module and a multi-task module;
step 5, inputting the remote sensing image into a network model, and outputting a block boundary and block object prediction result;
and 6, post-processing the prediction result to generate a vectorization block result, so as to realize accurate extraction of the block object and the block boundary.
Further, the specific implementation of step 1 includes:
for original input image large pictureXCutting with uniform sliding window to generate sample setXIs normalized band by band for each small pattern, byWhereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, firstiMean value of single band grid andithe variance of the grid of the individual single band,represents the normalized firstiNormalizing all the image pairs of the sample set by using the single-waveband raster data, and outputting the normalized sample set。
Further, the specific implementation of step 2 includes:
step 2.1, sample set after pretreatmentInput the methodkExtracting input image from network structure composed of serially connected dense feature extraction modulesX 0 Each module respectively outputs the dense featuresThe formula is as follows:
in the above formula, each dense feature extraction moduleDense i (. C) contains 3 dense feature convolution modulesConvdense 1 (·),Convdense 2 (·), Convdense 3 (·), X i-1 Represents the firstiIndividual dense feature extraction moduleDense i (ii) an input profile of the (v),Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of a first intensive characteristic convolution module and the output characteristic diagram of a second intensive characteristic convolution module in each intensive characteristic extraction module;
step 2.2, outputting the multi-stage characteristic diagram of 2.1Input to a decoder comprisingkAn upsampling module, wherein the formula is as follows:
in the above-mentioned formula, the compound has the following structure,represents the firstkThe up-sampling module outputs a characteristic map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,F interpolate (. Cndot.) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original dimensionWhen in use, willBilinear upsampled sumX n-1 The input features are superposed in the channel dimension and then are obtained by a 3 multiplied by 3 convolution moduleWhen it comes toAnd so on to finally obtainThen, it is named as a feature mapF dense ;
Step 2.3, the preprocessed sample setImage forming methodX 0 Is inputted intokExtracting high-resolution detail features of an original input image in a network structure consisting of a plurality of high-resolution feature extraction modules connected in series, wherein each module respectively corresponds to an output detail featureThe formula is as follows:
in the above formula, the first and second carbon atoms are,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original sizeX hk Is through the original inputX 0 TokThe high-resolution feature extraction module is used for obtaining the feature;
step 2.4, the details of step 2.3 are characterizedThe data is input into a network formed by the down-sampling modules, and the formula is as follows:
in the above formula, the first and second carbon atoms are,represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (-) represents a bilinear down-sampling of the input feature map for the spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,Cdenotes the superposition of input features in the channel dimension whenWhen in use, willX hn After bilinear down-sampling with the size of 0.5 timesX hn-1 The input features are superposed in the channel dimension and then are obtained by a 3 multiplied by 3 convolution moduleWhen it comes toAnd so on to finally obtainThen, it is named as a feature mapF hr 。
Further, the specific implementation of step 3 includes:
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, finally through 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region 。
Step 3.2, the step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
whereinConvblockRepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,F DH throughConvblockOutputting a location-aware field feature mapF pf ,F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic image as the row-column coordinates of the characteristic image, sampling the pixel value of the space grid, wherein the sampled characteristic image is a feature image of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a land object feature mapF po ;
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 feature mapF hr Inputting the data into a boundary maintaining module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, and finally integrates the class layer andSigmoidactivating function to output prediction result of block boundaryY boundary ;
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po The data are input to a land segmentation enhancement module, and the formula is as follows:
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generating initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passesOperation, i.e. subtracting the boundary prediction result output in step 3.3 from the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary characteristics of land blocks, enhancing the object characteristics of land blocks and outputting the enhanced land block characteristic diagramF sp Relieving the adhesion phenomenon before the land parcel, and finally obtaining the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidActivating the function to generate the final land object segmentation resultY rp 。
Further, the specific implementation of step 4 includes:
the segmentation result of the region of the multi-task segmentation result output by the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing land objectsSegmentation resultY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
wherein,respectively representing a real parcel area tag, a parcel boundary tag and a parcel object tag,L region , L ip , L rp are both binary cross-entropy loss functions,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,respectively a binary cross entropy loss function and a Dice loss function, for relieving the foreground and background imbalance phenomenon of the land parcel boundary, wherein the binary cross entropy loss function and the Dice loss function are used according to experience in the patentFor balancing the multitasking losses.
Further, the specific implementation of step 5 includes:
and storing the optimal precision result of the trained model on the verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
Further, the specific implementation of step 6 includes:
step 6.1, predicting the result of the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
whereinRespectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 x 3 size identity matrix, operated by morphological dilationFinally generating a land boundary enhancement resultY Db ;
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
wherein the operation is performed by boundary-object interactionP(,) generate the final parcel interaction resultY Mp ;
And 6.3, generating a final land mass vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
The method of the invention has the following remarkable effects: (1) Designing a complete high-resolution dense multi-scale module to jointly extract the shape variability features of the parcel objects and the edge features related to the parcel boundary categories; (2) Designing a region-boundary-plot decoupling multi-task module to jointly optimize plot object extraction and plot boundary extraction tasks, and jointly constraining prediction results among different tasks by modeling a spatial relationship among multi-task prediction results; (3) And designing a plot boundary-object interactive vectorization module to further optimize the blocking phenomenon of the plot object result.
Drawings
Fig. 1 is a remote sensing image input in step 1 according to the embodiment of the present invention.
Fig. 2 is a diagram of an ultra-complete high resolution dense multi-scale module network structure in step 2 according to an embodiment of the present invention.
Fig. 3 is a diagram of a zone-boundary-parcel decoupling multitask module network structure in step 3 according to the embodiment of the present invention.
Fig. 4 is a block boundary-object interactive vectorization module structure diagram in step 6 according to the embodiment of the present invention.
Fig. 5 shows the vectorized tile extraction result output in step 6 according to the embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
The invention provides a high-standard farmland plot vectorization extraction method based on multitask learning, which comprises the following steps of:
step 1, inputting an original remote sensing image large image, and performing sliding window clipping and normalization pretreatment on the original image as shown in fig. 1. The method further comprises the following steps:
for original input image large pictureXPerforming uniform sliding window clipping, with window size of 256 × 256 and step size of 256 × 256, generating a sample setXIs normalized band by band for each small pattern, byWhereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, secondiSingle band grid mean andithe variance of the grid of the individual single band,represents the normalizediNormalizing all image pairs of the sample set by using single-waveband raster data, and outputting the normalized sample set。
And 2, extracting the multi-scale features and the edge detail features by using an ultra-complete high-resolution dense multi-scale module, as shown in fig. 2. The method further comprises the following steps:
step 2.1, the preprocessed sample setInputting the image into a network structure composed of 5 serially connected dense feature extraction modules to extract an input imageX 0 The 5 modules respectively output the dense featuresThe formula is as follows:
in the above formula, each dense feature extraction moduleDense i (. 2) contains 3 dense feature convolution modulesConvdense 1 (·),Convdense 2 (·), Convdense 3 (·),X i-1 Represents the firstiIndividual dense feature extraction moduleDense i (ii) an input profile of the graph,Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of the first dense characteristic convolution module and the output characteristic diagram of the second dense characteristic convolution module in each dense characteristic extraction module.
Step 2.2, outputting the 2.1 multi-stage characteristic diagramInput to a decoder, which comprises 5 upsampling modules, the formula is as follows:
in the above formula, the first and second carbon atoms are,represents the firstkThe individual up-sampling modules output a profile map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,F interpolate (. To) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original sizeThen, the bilinear upsampled sumX 4 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution moduleWhen it comes toAnd so on, finally obtainIt is named as a feature mapF dense 。
Step 2.3, the preprocessed sample setMiddle imageX 0 Inputting the image data into a network structure consisting of 2 high-resolution feature extraction modules connected in series to extract high-resolution detail features of an original input image, wherein the 2 modules respectively correspond to output detail featuresThe formula is as follows:
in the above-mentioned formula, the compound has the following structure,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original sizeIs through the original inputX 0 The high-resolution feature extraction module obtains the high-resolution feature,is through inputAnd the high-resolution feature extraction module.
Step 2.4, the details of step 2.3 are characterizedThe data is input into a network formed by the down-sampling modules, the formula is as follows,
in the above-mentioned formula, the compound has the following structure,represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (. Cndot.) represents a bilinear down-sampling of the input feature map in a spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension whenWhen in use, willX h2 After bilinear down-sampling with the size of 0.5 timesX h1 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution moduleWhen is coming into contact withAnd so on, finally obtainName it as feature mapF hr 。
And 3, jointly extracting the plot boundary and the plot object by using the region-boundary-plot decoupling multitask module and the spectrum attention module, as shown in fig. 3. The method further comprises the following steps:
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, eventually by 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region 。
Step 3.2, the product of step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
whereinConvblockRepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,F DH through a processConvblockOutput location-aware field feature mapsF pf ,F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic image as the row-column coordinates of the characteristic image, sampling the pixel value of the space grid, wherein the sampled characteristic image is a feature image of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a feature map of the land objectF po 。
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 signatureF hr Inputting the data into a boundary keeping module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, and finally integrates the class layer andSigmoidactivating function to output prediction result of block boundaryY boundary 。
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po The data are input to a land segmentation enhancement module, and the formula is as follows:
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generates initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passesOperation, i.e. subtracting the block boundary prediction result output in step 3.3 by the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary features of land, enhancing the object features of land, and outputting the enhanced land feature mapF sp Relieving the adhesion phenomenon before the land parcel and finally generating the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidActivating the function to generate the final land object segmentation resultY rp 。
And 4, inputting the characteristic diagram finally output by the multitask module into the multitask joint optimization loss function, outputting a loss value, and reversely transmitting the loss value to update the network model parameters. The method further comprises the following steps:
the segmentation result of the region of the multi-task segmentation result output by the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing the land object segmentation resultY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
wherein,respectively representing a real parcel region label, a parcel boundary label and a parcel object label,L region , L ip , L rp are both binary cross-entropy loss functions that,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,respectively a binary cross entropy loss function and a Dice loss function, for relieving the foreground and background imbalance phenomenon of the land parcel boundary, wherein the binary cross entropy loss function and the Dice loss function are used according to experience in the patentFor balancing the multitasking losses.
Further, the specific implementation of step 5 includes:
and storing the optimal precision result of the trained model on the verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
Further, the specific implementation of step 6 includes:
step 6.1, predicting the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
whereinRespectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 x 3 size identity matrix, operated by morphological dilationFinally generating the land parcel boundaryEnhancing resultsY Db ;
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
wherein operations are performed through boundary-object interactionsP(,) generate the final parcel interaction resultY Mp ;
And 6.3, generating a final parcel vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (7)
1. A high-standard farmland plot vectorization extraction method based on multitask learning is characterized by comprising the following steps:
step 1, performing cutting and normalization pretreatment on an input image;
step 2, extracting multi-scale features and edge detail features of the preprocessed image by using a multi-scale module;
step 3, obtaining a land area segmentation result, a land boundary prediction result and a land object segmentation result through a multitask module according to the multi-scale features and the edge detail features;
step 4, inputting the result output in the step 3 into a multi-task joint optimization loss function, and outputtinglossValue, back propagationlossUpdating network model parameters, wherein the network model package consists of a multi-scale module and a multi-task module;
step 5, inputting the remote sensing image into a network model, and outputting a block boundary and block object prediction result;
and 6, post-processing the prediction result to generate a vectorization block result, so as to realize accurate extraction of the block object and the block boundary.
2. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 1 is as follows,
for original input image large pictureXCutting with uniform sliding window to generate sample setXNormalizing each small pattern band by band, passingWhereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, firstiMean value of single band grid andithe variance of the grid of the single band is,represents the normalizediNormalizing all image pairs of the sample set by using single-waveband raster data, and outputting the normalized sample set。
3. The high-standard farmland plot vectorization extraction method based on multitask learning according to claim 1, characterized in that: the implementation of said step 2 is as follows,
step 2.1, the preprocessed sample setInput devicekExtracting input image from network structure composed of series dense feature extraction modulesX 0 Of (2) a dense meshTarget characteristics, each module respectively corresponding to the output dense characteristicsThe formula is as follows:
in the above formula, each dense feature extraction moduleDense i (. C) contains 3 dense feature convolution modulesConvdense 1 (·),Convdense 2 (·), Convdense 3 (·), X i-1 Represents the firstiDense feature extraction moduleDense i (ii) an input profile of the (v),Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of a first intensive characteristic convolution module and the output characteristic diagram of a second intensive characteristic convolution module in each intensive characteristic extraction module;
step 2.2, outputting the 2.1 multi-stage characteristic diagramInput to a decoder comprisingkAn up-sampling module, the formula is as follows:
in the above formula, the first and second carbon atoms are,represents the firstkThe individual up-sampling modules output a profile map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,F interpolate (. Cndot.) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original dimensionk=At 1 time, willBilinear upsampled sumX n-1 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution moduleWhen it comes toAnd so on to finally obtainThen, it is named as a feature mapF dense ;
Step 2.3, the preprocessed sample setImage forming methodX 0 Is inputted intokExtracting high-resolution detail features of an original input image in a network structure consisting of serially connected high-resolution feature extraction modules, wherein each module respectively corresponds to output detail featuresThe formula is as follows:
in the above-mentioned formula, the compound has the following structure,Convblockrepresents a 3 × 3 convolutionA module consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original sizeX hk Is through the original inputX 0 To is thatkThe high-resolution characteristic extraction module obtains the characteristic;
step 2.4, the detail of step 2.3 is characterizedThe data is input into a network formed by the down-sampling modules, and the formula is as follows:
in the above-mentioned formula, the compound has the following structure,represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (. Cndot.) represents a bilinear down-sampling of the input feature map in a spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,Cdenotes the superposition of input features in the channel dimension whenWhen in use, willX hn Bilinear downsampling with size of 0.5 times andX hn-1 the input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution moduleWhen it comes toAnd so on to finally obtainThen, it is named as a feature mapF hr 。
4. The high-standard farmland plot vectorization extraction method based on multitask learning according to claim 3, characterized in that: the implementation of said step 3 is as follows,
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, eventually by 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region ;
Step 3.2, the step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
whereinConvblockRepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,F DH through a processConvblockOutput location-aware field feature mapsF pf ,F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic diagram as the row-column coordinates of the characteristic diagram, sampling the pixel value of the spatial grid, wherein the sampled characteristic diagram is a feature diagram of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a feature map of the land objectF po ;
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 signatureF hr Inputting the data into a boundary keeping module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, finally integrates class layers through 1 × 1 convolutionSigmoidActivating function to output prediction result of block boundaryY boundary ;
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po Input to the land segmentation enhancement module, the formula is as follows:
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generates initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passesOperation, i.e. subtracting the boundary prediction result output in step 3.3 from the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary features of land, enhancing the object features of land, and outputting the enhanced land feature mapF sp Relieving the adhesion phenomenon before the land parcel and finally generating the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidThe activation function generates the final block object segmentation resultY rp 。
5. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 4, characterized by that: the implementation of said step 4 is as follows,
the division result of the region of the multi-task division result output in the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing land object segmentation resultsY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
wherein,respectively representing a real parcel region label, a parcel boundary label and a parcel object label,L region , L ip , L rp are both binary cross-entropy loss functions,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,respectively binary cross entropy loss function and Dice loss function, for relieving foreground and background imbalance of land parcel boundary, λ 1 、λ 2 、λ 3 、λ bce 、λ dice Are smoothing parameters used to balance the multitasking penalty.
6. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 5 is as follows,
and storing the optimal precision result of the trained network model on a verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
7. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 6 is as follows,
step 6.1, predicting the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
whereinRespectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 × 3 unit matrix, and is expanded by morphologyFinally generating a land boundary enhancement resultY Db ;
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
wherein the operation is performed by boundary-object interactionP(-) generating the final parcel interaction resultY Mp ;
And 6.3, generating a final land mass vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211257905.8A CN115346137B (en) | 2022-10-14 | 2022-10-14 | High-standard farmland land mass vectorization extraction method based on multi-task learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211257905.8A CN115346137B (en) | 2022-10-14 | 2022-10-14 | High-standard farmland land mass vectorization extraction method based on multi-task learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115346137A true CN115346137A (en) | 2022-11-15 |
CN115346137B CN115346137B (en) | 2023-03-21 |
Family
ID=83957086
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211257905.8A Active CN115346137B (en) | 2022-10-14 | 2022-10-14 | High-standard farmland land mass vectorization extraction method based on multi-task learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115346137B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10482603B1 (en) * | 2019-06-25 | 2019-11-19 | Artificial Intelligence, Ltd. | Medical image segmentation using an integrated edge guidance module and object segmentation network |
CN112668421A (en) * | 2020-12-18 | 2021-04-16 | 武汉大学 | Attention mechanism-based rapid classification method for hyperspectral crops of unmanned aerial vehicle |
WO2022099303A1 (en) * | 2020-11-06 | 2022-05-12 | The Regents Of The University Of California | Machine learning techniques for tumor identification, classification, and grading |
CN114821315A (en) * | 2022-04-24 | 2022-07-29 | 福州大学 | Remote sensing image cultivated land plot extraction method combining edge detection and multitask learning |
-
2022
- 2022-10-14 CN CN202211257905.8A patent/CN115346137B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10482603B1 (en) * | 2019-06-25 | 2019-11-19 | Artificial Intelligence, Ltd. | Medical image segmentation using an integrated edge guidance module and object segmentation network |
WO2022099303A1 (en) * | 2020-11-06 | 2022-05-12 | The Regents Of The University Of California | Machine learning techniques for tumor identification, classification, and grading |
CN112668421A (en) * | 2020-12-18 | 2021-04-16 | 武汉大学 | Attention mechanism-based rapid classification method for hyperspectral crops of unmanned aerial vehicle |
CN114821315A (en) * | 2022-04-24 | 2022-07-29 | 福州大学 | Remote sensing image cultivated land plot extraction method combining edge detection and multitask learning |
Also Published As
Publication number | Publication date |
---|---|
CN115346137B (en) | 2023-03-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112052886B (en) | Intelligent human body action posture estimation method and device based on convolutional neural network | |
CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
CN110033002B (en) | License plate detection method based on multitask cascade convolution neural network | |
CN109726627B (en) | Neural network model training and universal ground wire detection method | |
CN110555458A (en) | Multi-band image feature level fusion method for generating countermeasure network based on attention mechanism | |
CN111738344B (en) | Rapid target detection method based on multi-scale fusion | |
CN111612017B (en) | Target detection method based on information enhancement | |
CN113627472B (en) | Intelligent garden leaf feeding pest identification method based on layered deep learning model | |
CN110458192B (en) | Hyperspectral remote sensing image classification method and system based on visual saliency | |
CN112132844A (en) | Recursive non-local self-attention image segmentation method based on lightweight | |
CN110210422B (en) | Ship ISAR image identification method based on optical image assistance | |
CN111476133B (en) | Unmanned driving-oriented foreground and background codec network target extraction method | |
CN111860537B (en) | Deep learning-based green citrus identification method, equipment and device | |
CN111428758A (en) | Improved remote sensing image scene classification method based on unsupervised characterization learning | |
CN115222946A (en) | Single-stage example image segmentation method and device and computer equipment | |
Mohmmad et al. | A survey machine learning based object detections in an image | |
CN110633640A (en) | Method for identifying complex scene by optimizing PointNet | |
CN112396036A (en) | Method for re-identifying blocked pedestrians by combining space transformation network and multi-scale feature extraction | |
CN115471675A (en) | Disguised object detection method based on frequency domain enhancement | |
CN114581789A (en) | Hyperspectral image classification method and system | |
CN112329647A (en) | Land use type identification method based on U-Net neural network | |
CN116758415A (en) | Lightweight pest identification method based on two-dimensional discrete wavelet transformation | |
CN115346137B (en) | High-standard farmland land mass vectorization extraction method based on multi-task learning | |
CN115965905A (en) | Crowd counting method and system based on multi-scale fusion convolutional network | |
Xie et al. | Pedestrian detection and location algorithm based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |