CN115346137A - High-standard farmland land mass vectorization extraction method based on multi-task learning - Google Patents

High-standard farmland land mass vectorization extraction method based on multi-task learning Download PDF

Info

Publication number
CN115346137A
CN115346137A CN202211257905.8A CN202211257905A CN115346137A CN 115346137 A CN115346137 A CN 115346137A CN 202211257905 A CN202211257905 A CN 202211257905A CN 115346137 A CN115346137 A CN 115346137A
Authority
CN
China
Prior art keywords
boundary
module
land
parcel
result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211257905.8A
Other languages
Chinese (zh)
Other versions
CN115346137B (en
Inventor
王心宇
潘洋
钟燕飞
张良培
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN202211257905.8A priority Critical patent/CN115346137B/en
Publication of CN115346137A publication Critical patent/CN115346137A/en
Application granted granted Critical
Publication of CN115346137B publication Critical patent/CN115346137B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a high-standard farmland plot vectorization extraction method based on multi-task learning, which mainly comprises the following steps: designing an ultra-complete high-resolution dense multi-scale module to jointly extract shape variability features of the parcel objects and edge features related to parcel boundary categories, and designing a region-boundary-parcel decoupling multi-task module to jointly optimize parcel object extraction and parcel boundary extraction tasks; and designing a plot boundary-object interactive vectorization module to further optimize the adhesion phenomenon of the plot object result. The method can be suitable for optical remote sensing images under various spatial resolutions to be used for a high-standard farmland plot vectorization extraction task, and compared with the existing farmland plot extraction method, the method provided by the invention can process a vector result, has higher plot object extraction precision and smooth plot boundaries, and can meet application requirements of plot scale crop classification, yield estimation and the like.

Description

High-standard farmland plot vectorization extraction method based on multitask learning
Technical Field
The invention belongs to the technical field of remote sensing image processing, and particularly relates to a high-standard farmland land parcel vectorization extraction method based on multi-task learning.
Background
Accurate farmland plot boundary information is important for farmers and agricultural managers to monitor crop growth in the field of precision agricultural services. However, the fully automatic extraction of the boundary of the farmland plot from the satellite image still faces a challenge. On the one hand, the land mass is fine and dense and has shape variability. The different plots have larger form difference and different sizes; on the other hand, as crops are mostly planted in the land parcels, the land parcels have larger difference of phenology between different land parcels, and the land parcels show space-time spectrum variability; in addition, from the visual effect, the plot is obscured by edge occlusion, most of the existing methods are difficult to completely identify plot objects, and the existing farmland plot data set is concentrated on a small-range single satellite image data source.
The current farmland plot extraction methods comprise methods based on edge detection, object segmentation and deep learning. The edge detection-based method mainly defines an edge detection operator manually, cannot automatically determine algorithm parameters, has very rough segmented boundaries and a saw-tooth phenomenon, and is difficult to apply to a region with dense plots. The object segmentation based method carries out edge connection or region segmentation through a region similarity criterion, and then carries out region classification through a sub-region result of segmentation, but the traditional methods can only segment plots and cannot extract farmland plots and boundaries at the same time. The deep learning method takes a land area or a land boundary as an identification object based on an edge or a region, tries the recognition object as a two-classification semantic segmentation task, trains two semantic segmentation networks, and is respectively used for detecting the land boundary and the land area, and finally outputs an example grid result of the land through post-processing. However, requirements cannot be met in subsequent applications, extraction results are rough, further post-processing is needed to obtain final segmentation results, a large number of parameters are needed for setting post-processing methods and effects, and the degree of automation is not high.
Disclosure of Invention
The invention provides a high-standard farmland land parcel vectorization extraction method based on region-boundary-land parcel decoupling multi-task learning, which has the following advantages: the method comprises the steps of designing an ultra-complete high-resolution dense multi-scale module to extract shape variability features of a parcel object and edge features related to parcel boundary classes. And secondly, designing a region-boundary-plot decoupling multitask module to jointly optimize the plot object extraction and the plot boundary extraction tasks, realizing multitask cooperative supervision of the plot boundary and the plot objects, and relieving the adhesion phenomenon among the plot objects. And thirdly, designing a plot boundary-object interactive vectorization module to further optimize the adhesion phenomenon of the plot object result, and generating a plot vectorization result which can be directly applied to subsequent agricultural remote sensing tasks. And fourthly, the method is applied to a plurality of domestic and foreign research areas, the optimal identification precision is obtained, and the method can be suitable for the optical satellite remote sensing images with medium and high resolution.
The invention provides a high-standard farmland plot vectorization extraction method based on multitask learning, which comprises the following steps:
step 1, performing cutting and normalization pretreatment on an input image;
step 2, extracting multi-scale features and edge detail features of the preprocessed image by using a multi-scale module;
step 3, obtaining a land area segmentation result, a land boundary prediction result and a land object segmentation result through a multitask module according to the multi-scale features and the edge detail features;
step 4, inputting the result output in the step 3 into a multi-task joint optimization loss function, and outputting the resultlossValue, back propagationlossUpdating network model parameters by values, wherein the network model packet is composed of a multi-scale module and a multi-task module;
step 5, inputting the remote sensing image into a network model, and outputting a block boundary and block object prediction result;
and 6, post-processing the prediction result to generate a vectorization block result, so as to realize accurate extraction of the block object and the block boundary.
Further, the specific implementation of step 1 includes:
for original input image large pictureXCutting with uniform sliding window to generate sample setXIs normalized band by band for each small pattern, by
Figure 640743DEST_PATH_IMAGE001
Whereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, firstiMean value of single band grid andithe variance of the grid of the individual single band,
Figure 823463DEST_PATH_IMAGE002
represents the normalized firstiNormalizing all the image pairs of the sample set by using the single-waveband raster data, and outputting the normalized sample set
Figure 432561DEST_PATH_IMAGE003
。
Further, the specific implementation of step 2 includes:
step 2.1, sample set after pretreatment
Figure 93350DEST_PATH_IMAGE004
Input the methodkExtracting input image from network structure composed of serially connected dense feature extraction modulesX 0 Each module respectively outputs the dense features
Figure 235618DEST_PATH_IMAGE005
The formula is as follows:
Figure 120397DEST_PATH_IMAGE006
Figure 13529DEST_PATH_IMAGE007
in the above formula, each dense feature extraction moduleDense i (. C) contains 3 dense feature convolution modulesConvdense 1 (·)，Convdense 2 (·), Convdense 3 (·), X i-1 Represents the firstiIndividual dense feature extraction moduleDense i (ii) an input profile of the (v),Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of a first intensive characteristic convolution module and the output characteristic diagram of a second intensive characteristic convolution module in each intensive characteristic extraction module;
step 2.2, outputting the multi-stage characteristic diagram of 2.1
Figure 478009DEST_PATH_IMAGE008
Input to a decoder comprisingkAn upsampling module, wherein the formula is as follows:
Figure 209204DEST_PATH_IMAGE009
in the above-mentioned formula, the compound has the following structure,
Figure 530464DEST_PATH_IMAGE010
represents the firstkThe up-sampling module outputs a characteristic map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,F interpolate (. Cndot.) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original dimension
Figure 51838DEST_PATH_IMAGE011
When in use, will
Figure 851166DEST_PATH_IMAGE012
Bilinear upsampled sumX n-1 The input features are superposed in the channel dimension and then are obtained by a 3 multiplied by 3 convolution module
Figure 702448DEST_PATH_IMAGE013
When it comes to
Figure 397871DEST_PATH_IMAGE014
And so on to finally obtain
Figure 701814DEST_PATH_IMAGE015
Then, it is named as a feature mapF dense ；
Step 2.3, the preprocessed sample set
Figure 806298DEST_PATH_IMAGE016
Image forming methodX 0 Is inputted intokExtracting high-resolution detail features of an original input image in a network structure consisting of a plurality of high-resolution feature extraction modules connected in series, wherein each module respectively corresponds to an output detail feature
Figure 715348DEST_PATH_IMAGE017
The formula is as follows:
Figure 378411DEST_PATH_IMAGE018
in the above formula, the first and second carbon atoms are,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original sizeX hk Is through the original inputX 0 TokThe high-resolution feature extraction module is used for obtaining the feature;
step 2.4, the details of step 2.3 are characterized
Figure 169649DEST_PATH_IMAGE019
The data is input into a network formed by the down-sampling modules, and the formula is as follows:
Figure 514043DEST_PATH_IMAGE020
in the above formula, the first and second carbon atoms are,
Figure 841381DEST_PATH_IMAGE021
represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (-) represents a bilinear down-sampling of the input feature map for the spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,Cdenotes the superposition of input features in the channel dimension when
Figure 878608DEST_PATH_IMAGE022
When in use, willX hn After bilinear down-sampling with the size of 0.5 timesX hn-1 The input features are superposed in the channel dimension and then are obtained by a 3 multiplied by 3 convolution module
Figure 157142DEST_PATH_IMAGE023
When it comes to
Figure 305227DEST_PATH_IMAGE024
And so on to finally obtain
Figure 720028DEST_PATH_IMAGE025
Then, it is named as a feature mapF hr 。
Further, the specific implementation of step 3 includes:
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
Figure 238076DEST_PATH_IMAGE026
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, finally through 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region 。
Step 3.2, the step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
Figure 738328DEST_PATH_IMAGE027
whereinConvblockRepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,F DH throughConvblockOutputting a location-aware field feature mapF pf ，F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic image as the row-column coordinates of the characteristic image, sampling the pixel value of the space grid, wherein the sampled characteristic image is a feature image of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a land object feature mapF po ；
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 feature mapF hr Inputting the data into a boundary maintaining module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
Figure 221262DEST_PATH_IMAGE028
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, and finally integrates the class layer andSigmoidactivating function to output prediction result of block boundaryY boundary ；
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po The data are input to a land segmentation enhancement module, and the formula is as follows:
Figure 21728DEST_PATH_IMAGE029
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generating initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passes
Figure 902221DEST_PATH_IMAGE030
Operation, i.e. subtracting the boundary prediction result output in step 3.3 from the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary characteristics of land blocks, enhancing the object characteristics of land blocks and outputting the enhanced land block characteristic diagramF sp Relieving the adhesion phenomenon before the land parcel, and finally obtaining the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidActivating the function to generate the final land object segmentation resultY rp 。
Further, the specific implementation of step 4 includes:
the segmentation result of the region of the multi-task segmentation result output by the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing land objectsSegmentation resultY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
Figure 889769DEST_PATH_IMAGE031
Figure 441973DEST_PATH_IMAGE032
wherein,
Figure 565787DEST_PATH_IMAGE033
respectively representing a real parcel area tag, a parcel boundary tag and a parcel object tag,L region , L ip , L rp are both binary cross-entropy loss functions,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,
Figure 115717DEST_PATH_IMAGE034
respectively a binary cross entropy loss function and a Dice loss function, for relieving the foreground and background imbalance phenomenon of the land parcel boundary, wherein the binary cross entropy loss function and the Dice loss function are used according to experience in the patent
Figure 357604DEST_PATH_IMAGE035
For balancing the multitasking losses.
Further, the specific implementation of step 5 includes:
and storing the optimal precision result of the trained model on the verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
Further, the specific implementation of step 6 includes:
step 6.1, predicting the result of the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
Figure 651182DEST_PATH_IMAGE036
wherein
Figure 160661DEST_PATH_IMAGE037
Respectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 x 3 size identity matrix, operated by morphological dilation
Figure 412651DEST_PATH_IMAGE038
Finally generating a land boundary enhancement resultY Db ；
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
Figure 843632DEST_PATH_IMAGE039
wherein the operation is performed by boundary-object interactionP(,) generate the final parcel interaction resultY Mp ；
And 6.3, generating a final land mass vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
The method of the invention has the following remarkable effects: (1) Designing a complete high-resolution dense multi-scale module to jointly extract the shape variability features of the parcel objects and the edge features related to the parcel boundary categories; (2) Designing a region-boundary-plot decoupling multi-task module to jointly optimize plot object extraction and plot boundary extraction tasks, and jointly constraining prediction results among different tasks by modeling a spatial relationship among multi-task prediction results; (3) And designing a plot boundary-object interactive vectorization module to further optimize the blocking phenomenon of the plot object result.
Drawings
Fig. 1 is a remote sensing image input in step 1 according to the embodiment of the present invention.
Fig. 2 is a diagram of an ultra-complete high resolution dense multi-scale module network structure in step 2 according to an embodiment of the present invention.
Fig. 3 is a diagram of a zone-boundary-parcel decoupling multitask module network structure in step 3 according to the embodiment of the present invention.
Fig. 4 is a block boundary-object interactive vectorization module structure diagram in step 6 according to the embodiment of the present invention.
Fig. 5 shows the vectorized tile extraction result output in step 6 according to the embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
The invention provides a high-standard farmland plot vectorization extraction method based on multitask learning, which comprises the following steps of:
step 1, inputting an original remote sensing image large image, and performing sliding window clipping and normalization pretreatment on the original image as shown in fig. 1. The method further comprises the following steps:
for original input image large pictureXPerforming uniform sliding window clipping, with window size of 256 × 256 and step size of 256 × 256, generating a sample setXIs normalized band by band for each small pattern, by
Figure 176787DEST_PATH_IMAGE040
Whereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, secondiSingle band grid mean andithe variance of the grid of the individual single band,
Figure 9614DEST_PATH_IMAGE041
represents the normalizediNormalizing all image pairs of the sample set by using single-waveband raster data, and outputting the normalized sample set
Figure 698084DEST_PATH_IMAGE042
。
And 2, extracting the multi-scale features and the edge detail features by using an ultra-complete high-resolution dense multi-scale module, as shown in fig. 2. The method further comprises the following steps:
step 2.1, the preprocessed sample set
Figure 147520DEST_PATH_IMAGE043
Inputting the image into a network structure composed of 5 serially connected dense feature extraction modules to extract an input imageX 0 The 5 modules respectively output the dense features
Figure 782901DEST_PATH_IMAGE044
The formula is as follows:
Figure 237278DEST_PATH_IMAGE045
Figure 362229DEST_PATH_IMAGE046
in the above formula, each dense feature extraction moduleDense i (. 2) contains 3 dense feature convolution modulesConvdense 1 (·)，Convdense 2 (·), Convdense 3 (·)，X i-1 Represents the firstiIndividual dense feature extraction moduleDense i (ii) an input profile of the graph,Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of the first dense characteristic convolution module and the output characteristic diagram of the second dense characteristic convolution module in each dense characteristic extraction module.
Step 2.2, outputting the 2.1 multi-stage characteristic diagram
Figure 236644DEST_PATH_IMAGE047
Input to a decoder, which comprises 5 upsampling modules, the formula is as follows:
Figure 472453DEST_PATH_IMAGE048
in the above formula, the first and second carbon atoms are,
Figure 811031DEST_PATH_IMAGE049
represents the firstkThe individual up-sampling modules output a profile map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,F interpolate (. To) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original size
Figure 546031DEST_PATH_IMAGE050
Then, the bilinear upsampled sumX 4 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution module
Figure 704480DEST_PATH_IMAGE051
When it comes to
Figure 743980DEST_PATH_IMAGE052
And so on, finally obtain
Figure 874747DEST_PATH_IMAGE053
It is named as a feature mapF dense 。
Step 2.3, the preprocessed sample set
Figure 75921DEST_PATH_IMAGE054
Middle imageX 0 Inputting the image data into a network structure consisting of 2 high-resolution feature extraction modules connected in series to extract high-resolution detail features of an original input image, wherein the 2 modules respectively correspond to output detail features
Figure 488710DEST_PATH_IMAGE055
The formula is as follows:
Figure 800743DEST_PATH_IMAGE056
in the above-mentioned formula, the compound has the following structure,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original size
Figure 51595DEST_PATH_IMAGE057
Is through the original inputX 0 The high-resolution feature extraction module obtains the high-resolution feature,
Figure 689250DEST_PATH_IMAGE058
is through input
Figure 291133DEST_PATH_IMAGE059
And the high-resolution feature extraction module.
Step 2.4, the details of step 2.3 are characterized
Figure 111583DEST_PATH_IMAGE060
The data is input into a network formed by the down-sampling modules, the formula is as follows,
Figure 482522DEST_PATH_IMAGE061
in the above-mentioned formula, the compound has the following structure,
Figure 291078DEST_PATH_IMAGE062
represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (. Cndot.) represents a bilinear down-sampling of the input feature map in a spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension when
Figure 645836DEST_PATH_IMAGE063
When in use, willX h2 After bilinear down-sampling with the size of 0.5 timesX h1 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution module
Figure 768513DEST_PATH_IMAGE064
When is coming into contact with
Figure 14862DEST_PATH_IMAGE065
And so on, finally obtain
Figure 197582DEST_PATH_IMAGE066
Name it as feature mapF hr 。
And 3, jointly extracting the plot boundary and the plot object by using the region-boundary-plot decoupling multitask module and the spectrum attention module, as shown in fig. 3. The method further comprises the following steps:
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
Figure 570794DEST_PATH_IMAGE067
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, eventually by 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region 。
Step 3.2, the product of step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
Figure 497162DEST_PATH_IMAGE068
whereinConvblockRepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,F DH through a processConvblockOutput location-aware field feature mapsF pf ，F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic image as the row-column coordinates of the characteristic image, sampling the pixel value of the space grid, wherein the sampled characteristic image is a feature image of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a feature map of the land objectF po 。
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 signatureF hr Inputting the data into a boundary keeping module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
Figure 311534DEST_PATH_IMAGE069
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, and finally integrates the class layer andSigmoidactivating function to output prediction result of block boundaryY boundary 。
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po The data are input to a land segmentation enhancement module, and the formula is as follows:
Figure 727472DEST_PATH_IMAGE070
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generates initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passes
Figure 558287DEST_PATH_IMAGE071
Operation, i.e. subtracting the block boundary prediction result output in step 3.3 by the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary features of land, enhancing the object features of land, and outputting the enhanced land feature mapF sp Relieving the adhesion phenomenon before the land parcel and finally generating the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidActivating the function to generate the final land object segmentation resultY rp 。
And 4, inputting the characteristic diagram finally output by the multitask module into the multitask joint optimization loss function, outputting a loss value, and reversely transmitting the loss value to update the network model parameters. The method further comprises the following steps:
the segmentation result of the region of the multi-task segmentation result output by the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing the land object segmentation resultY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
Figure 22767DEST_PATH_IMAGE072
Figure 753962DEST_PATH_IMAGE073
wherein,
Figure 75222DEST_PATH_IMAGE074
respectively representing a real parcel region label, a parcel boundary label and a parcel object label,L region , L ip , L rp are both binary cross-entropy loss functions that,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,
Figure 95131DEST_PATH_IMAGE075
respectively a binary cross entropy loss function and a Dice loss function, for relieving the foreground and background imbalance phenomenon of the land parcel boundary, wherein the binary cross entropy loss function and the Dice loss function are used according to experience in the patent
Figure 661504DEST_PATH_IMAGE076
For balancing the multitasking losses.
Further, the specific implementation of step 5 includes:
and storing the optimal precision result of the trained model on the verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
Further, the specific implementation of step 6 includes:
step 6.1, predicting the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
Figure 716047DEST_PATH_IMAGE077
wherein
Figure 411471DEST_PATH_IMAGE078
Respectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 x 3 size identity matrix, operated by morphological dilation
Figure 918675DEST_PATH_IMAGE079
Finally generating the land parcel boundaryEnhancing resultsY Db ；
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
Figure DEST_PATH_IMAGE080
wherein operations are performed through boundary-object interactionsP(,) generate the final parcel interaction resultY Mp ；
And 6.3, generating a final parcel vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (7)

1. A high-standard farmland plot vectorization extraction method based on multitask learning is characterized by comprising the following steps:
step 1, performing cutting and normalization pretreatment on an input image;
step 2, extracting multi-scale features and edge detail features of the preprocessed image by using a multi-scale module;
step 3, obtaining a land area segmentation result, a land boundary prediction result and a land object segmentation result through a multitask module according to the multi-scale features and the edge detail features;
step 4, inputting the result output in the step 3 into a multi-task joint optimization loss function, and outputtinglossValue, back propagationlossUpdating network model parameters, wherein the network model package consists of a multi-scale module and a multi-task module;
step 5, inputting the remote sensing image into a network model, and outputting a block boundary and block object prediction result;
and 6, post-processing the prediction result to generate a vectorization block result, so as to realize accurate extraction of the block object and the block boundary.
2. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 1 is as follows,
for original input image large pictureXCutting with uniform sliding window to generate sample setXNormalizing each small pattern band by band, passing
Figure 327466DEST_PATH_IMAGE001
Whereinb i ,mean i , std i Respectively representing the original remote-sensing imagesiSingle band raster data, firstiMean value of single band grid andithe variance of the grid of the single band is,
Figure 737981DEST_PATH_IMAGE002
represents the normalizediNormalizing all image pairs of the sample set by using single-waveband raster data, and outputting the normalized sample set
Figure 401044DEST_PATH_IMAGE003
。
3. The high-standard farmland plot vectorization extraction method based on multitask learning according to claim 1, characterized in that: the implementation of said step 2 is as follows,
step 2.1, the preprocessed sample set
Figure 395544DEST_PATH_IMAGE004
Input devicekExtracting input image from network structure composed of series dense feature extraction modulesX 0 Of (2) a dense meshTarget characteristics, each module respectively corresponding to the output dense characteristics
Figure 536676DEST_PATH_IMAGE005
The formula is as follows:
Figure 362549DEST_PATH_IMAGE006
Figure 432399DEST_PATH_IMAGE007
in the above formula, each dense feature extraction moduleDense i (. C) contains 3 dense feature convolution modulesConvdense 1 (·)，Convdense 2 (·), Convdense 3 (·), X i-1 Represents the firstiDense feature extraction moduleDense i (ii) an input profile of the (v),Convdense(. Cndot.) represents a dense feature convolution module, consisting of two convolution-batch normalization-activation modules,Crepresents the superposition of input features in the channel dimension,x 1 , x 2 respectively representing the output characteristic diagram of a first intensive characteristic convolution module and the output characteristic diagram of a second intensive characteristic convolution module in each intensive characteristic extraction module;
step 2.2, outputting the 2.1 multi-stage characteristic diagram
Figure 914196DEST_PATH_IMAGE008
Input to a decoder comprisingkAn up-sampling module, the formula is as follows:
Figure 124597DEST_PATH_IMAGE009
in the above formula, the first and second carbon atoms are,
Figure 742660DEST_PATH_IMAGE010
represents the firstkThe individual up-sampling modules output a profile map,Convblockrepresents a 3 x 3 convolution module, consisting of two convolution-batch normalization-activation modules,C(. Cndot.) represents the superposition of input features in the channel dimension,F interpolate (. Cndot.) represents a bilinear upsampling of the input feature map in a spatial dimension twice the original dimensionk=At 1 time, will
Figure 13105DEST_PATH_IMAGE011
Bilinear upsampled sumX n-1 The input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution module
Figure 14821DEST_PATH_IMAGE012
When it comes to
Figure 701017DEST_PATH_IMAGE013
And so on to finally obtain
Figure 501483DEST_PATH_IMAGE014
Then, it is named as a feature mapF dense ；
Step 2.3, the preprocessed sample set
Figure 614932DEST_PATH_IMAGE015
Image forming methodX 0 Is inputted intokExtracting high-resolution detail features of an original input image in a network structure consisting of serially connected high-resolution feature extraction modules, wherein each module respectively corresponds to output detail features
Figure 602480DEST_PATH_IMAGE016
The formula is as follows:
Figure 656149DEST_PATH_IMAGE017
in the above-mentioned formula, the compound has the following structure,Convblockrepresents a 3 × 3 convolutionA module consisting of two convolution-batch normalization-activation modules,F interpolate (. O) represents a bilinear upsampling of the input feature map for spatial dimensions twice the original sizeX hk Is through the original inputX 0 To is thatkThe high-resolution characteristic extraction module obtains the characteristic;
step 2.4, the detail of step 2.3 is characterized
Figure 514383DEST_PATH_IMAGE018
The data is input into a network formed by the down-sampling modules, and the formula is as follows:
Figure 64314DEST_PATH_IMAGE019
in the above-mentioned formula, the compound has the following structure,
Figure 804736DEST_PATH_IMAGE020
represents the firstkThe down-sampling module outputs a feature map,F interpolate_down (. Cndot.) represents a bilinear down-sampling of the input feature map in a spatial dimension of 0.5 times the original size,Convblockrepresents a 3 x 3 convolution module, consists of 2 convolution-batch normalization-activation modules,Cdenotes the superposition of input features in the channel dimension when
Figure 98315DEST_PATH_IMAGE021
When in use, willX hn Bilinear downsampling with size of 0.5 times andX hn-1 the input features are superposed in the channel dimension and then are obtained through a 3 multiplied by 3 convolution module
Figure 843679DEST_PATH_IMAGE022
When it comes to
Figure 830089DEST_PATH_IMAGE023
And so on to finally obtain
Figure 792229DEST_PATH_IMAGE024
Then, it is named as a feature mapF hr 。
4. The high-standard farmland plot vectorization extraction method based on multitask learning according to claim 3, characterized in that: the implementation of said step 3 is as follows,
step 3.1, outputting steps 2.2 and 2.4F dense AndF hr stacking in channel dimension, outputtingF DH Characteristic diagram ofF DH Inputting the result to a land region segmentation module and outputting the resultY region The formula is as follows:
Figure 623919DEST_PATH_IMAGE025
whereinConvblock 1 Represents a 3 × 3 convolution batch normalization and activation module, eventually by 1 × 1 convolution sumSigmoidActivating function, and finally outputting the result of block region segmentationY region ；
Step 3.2, the step 3.1F DH Inputting the characteristic graph into a boundary plot generation module to obtain a plot boundary characteristic graphF pb And land object feature mapF po The formula is as follows:
Figure 987904DEST_PATH_IMAGE026
whereinConvblockRepresents a 3 x 3 convolution module, consisting of 2 convolution-batch normalization-activation modules,F DH through a processConvblockOutput location-aware field feature mapsF pf ，F interpolate_size (·,Size(. DEG)) represents a dimension-specific bilinear interpolation of the input feature map by applying to the perceptual field feature mapF pf Bilinear interpolation toF DH The size of the space characteristic graph of the characteristic graph is sampled by gridgridsampleAnd operation, taking the pixel value of the perception field characteristic diagram as the row-column coordinates of the characteristic diagram, sampling the pixel value of the spatial grid, wherein the sampled characteristic diagram is a feature diagram of the boundary of the land parcelF pb And will beF DH Feature map minus parcel boundary feature mapF pb Obtaining a feature map of the land objectF po ；
Step 3.3, the characteristic diagram of the step 3.2 is processedF pb And step 2.4 signatureF hr Inputting the data into a boundary keeping module, and further learning the edge characteristics of the land parcel, wherein the formula is as follows:
Figure 879637DEST_PATH_IMAGE027
wherein,Crepresents the superposition of input features in the channel dimension,Convblock 1 represents a 3 × 3 convolution batch normalization and activation module, finally integrates class layers through 1 × 1 convolutionSigmoidActivating function to output prediction result of block boundaryY boundary ；
Step 3.4, the step 3.1F DH Feature map, boundary prediction result output in step 3.3Y boundary And the land object feature map in step 3.2F po Input to the land segmentation enhancement module, the formula is as follows:
Figure 299379DEST_PATH_IMAGE028
wherein, inputF DH Integrating the class layers by 1 x 1 convolution andSigmoidactivation function generates initial block object segmentation resultY ipo Then through the parcel boundary attention module, which passes
Figure 731498DEST_PATH_IMAGE029
Operation, i.e. subtracting the boundary prediction result output in step 3.3 from the identity matrixY boundary Characteristic diagram of rear and land objectF po Multiplying to suppress the boundary features of land, enhancing the object features of land, and outputting the enhanced land feature mapF sp Relieving the adhesion phenomenon before the land parcel and finally generating the characteristic diagramF sp With the initial block segmentation resultY ipo After stacking channel dimensions, 1 x 1 convolution sumSigmoidThe activation function generates the final block object segmentation resultY rp 。
5. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 4, characterized by that: the implementation of said step 4 is as follows,
the division result of the region of the multi-task division result output in the step 3Y region Block boundary prediction resultsY boundary Initial block object segmentation resultY ipo And enhancing land object segmentation resultsY rp And inputting the data into a multitask joint loss function, wherein the formula is as follows:
Figure 153252DEST_PATH_IMAGE030
Figure 278202DEST_PATH_IMAGE031
wherein,
Figure 152618DEST_PATH_IMAGE032
respectively representing a real parcel region label, a parcel boundary label and a parcel object label,L region , L ip , L rp are both binary cross-entropy loss functions,L bound is a combination of a binary cross entropy loss function and a Dice loss function, wherein,
Figure 857268DEST_PATH_IMAGE033
respectively binary cross entropy loss function and Dice loss function, for relieving foreground and background imbalance of land parcel boundary, λ 1 、λ 2 、λ 3 、λ bce 、λ dice Are smoothing parameters used to balance the multitasking penalty.
6. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 5 is as follows,
and storing the optimal precision result of the trained network model on a verification set, predicting the test set, and outputting a final block object prediction result and a block boundary prediction result.
7. The high-standard farmland parcel vectorization extraction method based on multitask learning according to claim 1, characterized by comprising the following steps: the implementation of said step 6 is as follows,
step 6.1, predicting the block boundaryY Pb Inputting a morphological boundary enhancement module to enhance the edge connectivity in the process of extracting the land parcel boundary, wherein the formula is as follows:
Figure 685592DEST_PATH_IMAGE034
wherein
Figure 184706DEST_PATH_IMAGE035
Respectively, a block boundary prediction result and a structural element, wherein the structural elementSIs a predefined 3 × 3 unit matrix, and is expanded by morphology
Figure 546418DEST_PATH_IMAGE036
Finally generating a land boundary enhancement resultY Db ；
Step 6.2, enhancing the result of the land parcel boundaryY Db And block object prediction resultsY Pp Inputting a boundary-object interaction module to relieve the adhesion phenomenon between the plots, wherein the formula is as follows:
Figure DEST_PATH_IMAGE037
wherein the operation is performed by boundary-object interactionP(-) generating the final parcel interaction resultY Mp ；
And 6.3, generating a final land mass vectorization result by the result in the step 6.2 through boundary smoothing and hole filling.
CN202211257905.8A 2022-10-14 2022-10-14 High-standard farmland land mass vectorization extraction method based on multi-task learning Active CN115346137B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211257905.8A CN115346137B (en) 2022-10-14 2022-10-14 High-standard farmland land mass vectorization extraction method based on multi-task learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211257905.8A CN115346137B (en) 2022-10-14 2022-10-14 High-standard farmland land mass vectorization extraction method based on multi-task learning

Publications (2)

Publication Number Publication Date
CN115346137A true CN115346137A (en) 2022-11-15
CN115346137B CN115346137B (en) 2023-03-21

Family

ID=83957086

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211257905.8A Active CN115346137B (en) 2022-10-14 2022-10-14 High-standard farmland land mass vectorization extraction method based on multi-task learning

Country Status (1)

Country Link
CN (1) CN115346137B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10482603B1 (en) * 2019-06-25 2019-11-19 Artificial Intelligence, Ltd. Medical image segmentation using an integrated edge guidance module and object segmentation network
CN112668421A (en) * 2020-12-18 2021-04-16 武汉大学 Attention mechanism-based rapid classification method for hyperspectral crops of unmanned aerial vehicle
WO2022099303A1 (en) * 2020-11-06 2022-05-12 The Regents Of The University Of California Machine learning techniques for tumor identification, classification, and grading
CN114821315A (en) * 2022-04-24 2022-07-29 福州大学 Remote sensing image cultivated land plot extraction method combining edge detection and multitask learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10482603B1 (en) * 2019-06-25 2019-11-19 Artificial Intelligence, Ltd. Medical image segmentation using an integrated edge guidance module and object segmentation network
WO2022099303A1 (en) * 2020-11-06 2022-05-12 The Regents Of The University Of California Machine learning techniques for tumor identification, classification, and grading
CN112668421A (en) * 2020-12-18 2021-04-16 武汉大学 Attention mechanism-based rapid classification method for hyperspectral crops of unmanned aerial vehicle
CN114821315A (en) * 2022-04-24 2022-07-29 福州大学 Remote sensing image cultivated land plot extraction method combining edge detection and multitask learning

Also Published As

Publication number Publication date
CN115346137B (en) 2023-03-21

Similar Documents

Publication Publication Date Title
CN112052886B (en) Intelligent human body action posture estimation method and device based on convolutional neural network
CN109584248B (en) Infrared target instance segmentation method based on feature fusion and dense connection network
CN110033002B (en) License plate detection method based on multitask cascade convolution neural network
CN109726627B (en) Neural network model training and universal ground wire detection method
CN110555458A (en) Multi-band image feature level fusion method for generating countermeasure network based on attention mechanism
CN111738344B (en) Rapid target detection method based on multi-scale fusion
CN111612017B (en) Target detection method based on information enhancement
CN113627472B (en) Intelligent garden leaf feeding pest identification method based on layered deep learning model
CN110458192B (en) Hyperspectral remote sensing image classification method and system based on visual saliency
CN112132844A (en) Recursive non-local self-attention image segmentation method based on lightweight
CN110210422B (en) Ship ISAR image identification method based on optical image assistance
CN111476133B (en) Unmanned driving-oriented foreground and background codec network target extraction method
CN111860537B (en) Deep learning-based green citrus identification method, equipment and device
CN111428758A (en) Improved remote sensing image scene classification method based on unsupervised characterization learning
CN115222946A (en) Single-stage example image segmentation method and device and computer equipment
Mohmmad et al. A survey machine learning based object detections in an image
CN110633640A (en) Method for identifying complex scene by optimizing PointNet
CN112396036A (en) Method for re-identifying blocked pedestrians by combining space transformation network and multi-scale feature extraction
CN115471675A (en) Disguised object detection method based on frequency domain enhancement
CN114581789A (en) Hyperspectral image classification method and system
CN112329647A (en) Land use type identification method based on U-Net neural network
CN116758415A (en) Lightweight pest identification method based on two-dimensional discrete wavelet transformation
CN115346137B (en) High-standard farmland land mass vectorization extraction method based on multi-task learning
CN115965905A (en) Crowd counting method and system based on multi-scale fusion convolutional network
Xie et al. Pedestrian detection and location algorithm based on deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant