CN117636172B - Target detection method and system for weak and small target of remote sensing image - Google Patents
Target detection method and system for weak and small target of remote sensing image Download PDFInfo
- Publication number
- CN117636172B CN117636172B CN202311669036.4A CN202311669036A CN117636172B CN 117636172 B CN117636172 B CN 117636172B CN 202311669036 A CN202311669036 A CN 202311669036A CN 117636172 B CN117636172 B CN 117636172B
- Authority
- CN
- China
- Prior art keywords
- boundary
- feature
- features
- texture
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 38
- 230000008447 perception Effects 0.000 claims abstract description 87
- 238000000605 extraction Methods 0.000 claims abstract description 50
- 238000000034 method Methods 0.000 claims abstract description 25
- 239000011159 matrix material Substances 0.000 claims description 54
- 230000004927 fusion Effects 0.000 claims description 40
- 230000004913 activation Effects 0.000 claims description 14
- 230000007246 mechanism Effects 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 10
- 230000002194 synthesizing effect Effects 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 5
- 238000001914 filtration Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 16
- 201000005625 Neuroleptic malignant syndrome Diseases 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 230000002708 enhancing effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000017525 heat dissipation Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004804 winding Methods 0.000 description 2
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000000875 corresponding effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of intelligent interpretation of remote sensing images, in particular to a method and a system for detecting perceived textures and boundary targets of weak and small targets of remote sensing images, wherein the method comprises the following steps: inputting the remote sensing image into a feature extraction network to extract basic features and then extracting texture perception features; inputting the remote sensing image into a boundary map extraction module to extract a binary boundary map, and extracting boundary perception characteristics; the boundary perception feature and the basic feature are fused through a boundary guiding feature module, and boundary guiding features are obtained; decoupling and outputting texture perception features and boundary guide features, inputting the texture perception features and the boundary guide features into a RCNN network with task decoupling for double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection; the method and the system can mine the information that the remote sensing weak and small target is difficult to highlight, and further improve the expression of the weak and small target, so that the detection performance of the weak and small target is improved.
Description
Technical Field
The invention relates to the technical fields of computer vision, deep learning and intelligent interpretation of remote sensing images, and particularly provides a method and a system for detecting perceived textures and boundary targets of weak and small targets of remote sensing images.
Background
In recent years, due to the large investment on civil and commercial satellites, the rapid development of imaging technology and other reasons, the quality of remote sensing images is improved, the acquisition cost of the images is reduced, and the method creates favorable conditions for target detection of the remote sensing images. However, a small target in a remote sensing image is always a difficult problem for the target detection task. These small and weak targets are widely distributed in the remote sensing image and have the following properties: (1) These targets have only a small pixel duty cycle in the remote sensing image; (2) The high frequency details of the target (e.g., texture details, boundary cues, colors, etc.) are not significant, the features vary widely, and are susceptible to interference from the then-current imaging conditions and complex contexts. However, small and weak targets often carry more important information, and effective detection of these targets is of great importance for real-world scenarios. In addition, in object detection, classification and regression are two highly correlated and contradictory tasks, most of which share the same features to perform classification and regression, but have respective information emphasis. The classification task tends to be more sensitive to the target boundary than the most semantic part of the features in the interior of the target, whereas the regression task is more sensitive to the target boundary. The use of coupled features does not yield optimal performance and the model requires extraction of the appropriate features for a particular task.
Disclosure of Invention
The invention provides a sensing texture and boundary feature target detection method for a weak and small target of a remote sensing image, which aims to solve the problems, and improves the information expression of the weak and small target by exploring high-frequency detail information such as key textures, boundaries and the like of the target, and enhances the features of the weak and small target, so that the detection precision of the weak and small target is improved. Meanwhile, the invention provides a decoupled detection head structure, which separates classification and positioning tasks, and a decoupled network can conduct targeted training according to specific tasks.
The invention provides a sensing texture and boundary target detection method for a weak and small target of a remote sensing image, which comprises the following steps:
Step 1: inputting the remote sensing image into a feature extraction network to extract basic features, inputting the basic features into a texture perception enhancement module, and extracting texture perception features;
Step 2: inputting a remote sensing image into a boundary map extraction module to extract a binary boundary map, inputting the boundary map into a boundary feature extraction module to extract boundary perception features; fusing the boundary perception feature and the basic feature through a boundary guiding feature module to obtain a boundary guiding feature;
step 3: and decoupling and outputting the texture perception feature and the boundary guide feature, inputting the texture perception feature and the boundary guide feature into a RCNN network with task decoupling for double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection.
Preferably, in the step 1, the basic feature is input into a texture perception enhancement module, and the extracting the texture perception feature includes: and inputting the basic features into a feature pyramid to obtain fusion features, inputting the fusion features into a texture perception enhancement module, and extracting texture perception features.
Preferably, the step 1 includes:
inputting the remote sensing image into a feature extraction network to extract basic features C i, and inputting the basic features into a feature pyramid to obtain fusion features
Wherein c represents the channel dimension of the fusion feature; h and w represent the length and width, respectively, of the fusion feature.
Preferably, the inputting the fusion feature into the texture perception enhancement module, and extracting the texture perception feature includes:
subjecting the fused feature map to 1×1 convolution of three parallel branches to integrate features and reduce the dimension to c 1 to obtain a feature f θ, F g, and characterizing f θ,/>Reduce the dimension to/>
Calculating the features of f θ and transposeThe covariance matrix sigma between the two pixels captures the correlation among pixels at different positions in a characteristic dimension space, and a calculation formula is as follows:
wherein, f θ is used for the treatment of the heat dissipation, F g represents the intermediate feature after convolution processing, (x, y) represents the position of the pixel point,/>Multiplying the representative elements;
introducing a spatial attention mechanism to strengthen the position of a target in the remote sensing image and highlight the pixel relationship in the target;
Carrying out global average pooling of channel dimensions on the input fusion feature f i, and adopting 3×3 to remove redundant noise;
The Sigmoid activation function is used for feature normalization to generate attention weight;
Converting weight dimensions to Generating an attention weight map M (x, y) and aligned with the channel dimension of the covariance matrix Σ, expressed as:
M(x,y)=Reshape(σ(Conv3×3(fgap(x,y))))
wherein GAP represents global average pooling operation, and sigma represents Sigmoid activation;
multiplying the attention weight map M (x, y) with the covariance matrix sigma (x, y) to obtain a target enhanced covariance matrix
The dimension of the covariance matrix of the target enhancement is calculated as
Regularizing the target enhanced covariance matrix and converting dimensions of the target enhanced covariance matrix to
Wherein the first dimension wh represents the number of covariance matrices of the target enhancement, which is equal to the feature size; the second dimension w represents the covariance matrix of the target enhancementIs a length of (2); the third dimension h represents the covariance matrix/>, of the target enhancementIs a width of (2);
wherein, as follows, the dot product of the matrix;
Integrating the relation matrix of pixels between different channels by adopting a 1X 1 convolution operation, and converting the channels into
By combiningMultiplying with f g, and restoring the characteristic channel to c by using 1×1 convolution to obtain texture characteristics;
Texture-aware features are obtained by adding the captured texture features to the fusion features f i.
Preferably, the obtaining texture-aware features by adding the captured texture features to the fusion features f i is achieved by the following formula:
Wherein, Representing matrix element addition.
Preferably, the inputting the remote sensing image into the boundary map extracting module for extracting the binary boundary map includes: and performing sliding extraction on the input remote sensing image by utilizing an edge extraction operator to generate a gradient map of the remote sensing image.
Preferably, the gradient map is subjected to gating filtering, a threshold value is set, and pixel values smaller than the threshold value are set to be zero, so that noise and useless texture information are removed.
Preferably, the inputting the boundary map into a boundary feature extraction module, and extracting the boundary perception feature includes:
extracting bottom features by adopting 3×3 convolution, BN, reLU and 3×3 convolution;
Reducing the dimension of the bottom layer feature by adopting 1×1 convolution, BN and ReLU operations, extracting edge information in the bottom layer feature by utilizing three parallel branches comprising 1×3 convolution, 3×1 convolution and 3×3 convolution, and summarizing and fusing the edge information to obtain potential edge features;
Recovering a characteristic channel by adopting 1X 1 convolution, BN and ReLU operation;
And transferring the bottom layer feature to be fused with the potential edge feature through jump connection, and generating boundary sensing features through 1X 1 convolution and Sigmoid activation.
Preferably, the fusing, by the boundary guiding feature module, the boundary sensing feature with the basic feature, and obtaining the boundary guiding feature includes:
adding the boundary sensing characteristic after downsampling and the basic characteristic to fuse the characteristic, adopting a jump connection structure and a3 multiplied by 3 convolution amplifying edge structure, and integrating the fused information;
And highlighting important feature channels in the fusion features by adopting a channel attention mechanism, and adding the important feature channels with the fusion features output by the feature pyramid to obtain boundary guiding features.
Preferably, the step 3 of decoupling the texture perception feature from the boundary guiding feature, inputting the decoupling prediction of the two branches into a RCNN network with task decoupling, obtaining a classification and positioning result, and completing the detection of the perceived texture and the boundary target includes:
inputting the texture perception feature generated in the step 1 to a classification branch in the task decoupling RCNN, and inputting the boundary guiding feature generated in the step 2 to a positioning branch in the task decoupling RCNN;
Setting different thresholds, and decoupling the classifying branch and the positioning branch to non-maximal inhibition NMS, so as to generate suggested areas with different densities;
decoupling sampling is carried out on the suggested area, and a decoupled region of interest is obtained; for the classification branches, adopting a fixed proportion to sample quantitative positive samples and negative samples; sampling a quantitative positive sample for the positioning branch;
and expanding the region of interest range of the classification branch, and synthesizing a high-quality region of interest for the positioning branch by utilizing Weiszfeld algorithm to assist in positioning prediction.
Preferably, synthesizing the upper left corner and the lower right corner of all the suggested areas and the true labeling areas allocated to each target, and searching the geometric center of the discrete sample points in an iterative re-weighted least square mode;
assuming the reference point to be fitted is x i, the algorithm attempts to find a center point y by iterating so that the distance between the point and the reference point is minimized, expressed as:
wherein, lll 2 represents the L2 norm, j represents the number of reference points, D represents the sum of the distances between the synthesized center point y and the reference point x, argmin represents the parameter value y, which minimizes the sum;
solving in continuous space, the above equation will take the minimum at the extremum according to the optimization theory. Thus, bias y:
solving the above method, and solving a central point as follows:
where k represents the number of iterations. The obtained central point is smaller than a specified value through continuous iteration, the point is the fitted geometric center, and the central point is not overlapped with any reference point;
And synthesizing a suggested area close to the real labeling frame through fitting the upper left corner and the lower right corner of the frame, training and positioning a branch network, accelerating the convergence of the model and improving the stability of the model.
The invention also provides a sensing texture and boundary target detection system for the weak and small targets of the remote sensing image, which comprises the following steps:
The feature extraction network is used for extracting basic features of the remote sensing image;
the texture perception enhancement module is used for extracting texture perception features through the basic features;
the boundary map extraction module is used for extracting a binary boundary map of the remote sensing image;
the boundary feature extraction module is used for extracting boundary perception features of the boundary map;
the boundary guiding feature module is used for fusing the boundary perception feature and the basic feature to obtain a boundary guiding feature;
and the decoupling transmission module is used for decoupling the texture perception feature and the boundary guide feature, outputting the texture perception feature and the boundary guide feature, inputting the texture perception feature and the boundary guide feature into a RCNN network with task decoupling for performing double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection.
Compared with the prior art, the invention has the following beneficial effects:
The invention provides a sensing texture and boundary feature target detection method for a weak and small target of a remote sensing image, and particularly provides a unique texture and boundary sensing network for solving the problem of the weak and small target in the remote sensing image; in order to mine deep information, a texture perception enhancement module and a boundary perception fusion module are provided, wherein the texture perception enhancement module can fully explore texture details in the target, the relation between features is built through a calculated covariance matrix, and meanwhile, the texture difference between the target and the background is highlighted; the latter introduces extra boundary clues to highlight the spatial position of the target, and the key features of the weak target are supplemented by exploring the high-frequency detail information such as the key textures, boundaries and the like of the target; the combination of the two modules enhances the perceptibility of the network to the weak and small targets. Meanwhile, in order to relieve task unfocused phenomenon caused by winding between classification and regression, RCNN for task decoupling is provided to independently train two branch networks, so that fine prediction on classification and positioning is realized. The detection method provided by the invention can mine the information that the remote sensing weak and small target is difficult to highlight, and further promote the expression of the weak and small target, so that the detection performance of the weak and small target is improved.
Drawings
FIG. 1 is a flow chart of a remote sensing image target detection method according to an embodiment of the present invention;
FIG. 2 is an exemplary diagram of a remote sensing image with a weak target present in accordance with an embodiment of the present invention;
FIG. 3 is an overall schematic diagram of an object detection network according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a texture perception enhancement module in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of boundary map extraction in a boundary aware fusion module according to an embodiment of the invention;
FIG. 6 is a schematic diagram of boundary-aware feature extraction in a boundary-aware fusion module according to an embodiment of the invention;
FIG. 7 is a schematic diagram of boundary-aware feature fusion in a boundary-aware fusion module according to an embodiment of the invention;
FIG. 8 is a task decoupling RCNN network schematic according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a proposed synthetic strategy according to an embodiment of the invention;
fig. 10 is an exemplary diagram of a result of detecting a weak target in a remote sensing image by using the target detection method according to the embodiment of the present invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, like modules are denoted by like reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, a detailed description thereof will not be repeated.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not to be construed as limiting the invention.
In a specific embodiment of the invention, a method for detecting a perceived texture and boundary feature target of a weak and small target of a remote sensing image is provided, which comprises the following steps:
step 1: inputting the remote sensing image into a feature extraction network to extract basic features, inputting the basic features into a texture perception enhancement module, further exploring the texture mode of a target of the remote sensing image, and extracting texture perception features;
in a specific embodiment, inputting the basic feature into a texture perception enhancement module, and extracting the texture perception feature includes: and inputting the basic features into a feature pyramid to obtain fusion features, inputting the fusion features into a texture perception enhancement module, and extracting texture perception features.
In a specific embodiment, the step 1 specifically includes: inputting the remote sensing image with weak and small targets into a feature extraction network to obtain basic features C i, and simultaneously inputting the basic features into a feature pyramid to obtain fused featuresWherein c represents the channel dimension of the fusion feature; h and w represent the length and width, respectively, of the fusion feature.
And inputting the fused features into a texture perception enhancement module to extract texture features. Specifically, first, the fused feature map is subjected to 1×1 convolution of three parallel branches to integrate features and reduce the dimension to c 1, to obtain a feature f θ,F g, and characterizing f θ,/>Reduce the dimension to/>Next, the features/>, of f θ and transpose are calculatedThe covariance matrix sigma between the two pixels is used for capturing the correlation between pixels at different positions in a characteristic dimension space, and a calculation formula is as follows:
wherein, f θ is used for the treatment of the heat dissipation, F g represents the intermediate feature after convolution processing, (x, y) represents the position of the pixel point,/>Representing the multiplication of the elements. Meanwhile, a spatial attention mechanism is introduced to strengthen the position of the target in the remote sensing image and highlight the pixel relationship in the target. Firstly, carrying out global average pooling of channel dimensions on an input fusion feature f i, and adopting 3×3 to remove redundant noise; then, the Sigmoid activation function is used for feature normalization to generate attention weight; finally, the weight dimension is converted to/>An attention weight map M (x, y) is generated and aligned with the channel dimension of the covariance matrix Σ, which can be expressed as:
M(x,y)=Reshape(σ(Conv3×3(fgap(x,y)))) (3)
wherein GAP represents global average pooling operation, and sigma represents Sigmoid activation; subsequently, the attention map M (x, y) is multiplied by the covariance matrix Σ (x, y) to obtain a target enhanced covariance matrix At this time, the dimension of the covariance matrix of the target enhancement calculated is/>Regularizing these values by using SoftMax, i.e., regularizing the entirety of the calculated target enhanced covariance matrix, and converting the dimensions of the target enhanced covariance matrix toThe covariance matrix of the target enhancement before processing is two-dimensional, and the covariance matrix of the target enhancement is converted into three-dimensional after conversion; wherein the first dimension wh represents the number of covariance matrices of the target enhancement, which is equal to the feature size; the second dimension w represents the covariance matrix/>, of the target enhancementIs a length of (2); the third dimension h represents the covariance matrix/>, of the target enhancementIs a width of (c). Thus,/>The meaning represented mathematically is the relationship between a pixel and all pixels at other locations. It can be written as:
wherein, as follows, the dot product of the matrix; at the same time, a 1X 1 convolution operation is adopted to integrate the relation matrix of pixels between different channels and convert the channels into At this point, a relationship map is generated that can magnify the similar locations of the homogeneous pixels and the target, which can be used to emphasize similar texture representations of the target at the global level, further enhancing valuable texture features; finally, by means of the method of the/>Multiplying with f g to obtain a valuable texture feature and restoring the feature channel to c using a1 x 1 convolution; by adding the captured texture features to the original features f i, texture-aware features are obtained. Specifically, the treatment process is as follows:
Wherein, Representing matrix element addition.
Step 2: inputting the remote sensing image into a boundary map extractor to extract a binary boundary map, and inputting the boundary map into a boundary perception feature extraction module to further extract boundary perception features of a target; and finally, fusing the extracted features with the basic features extracted in the step 1 to obtain boundary guiding features. The inputting the remote sensing image into the boundary map extracting module for extracting the binary boundary map comprises the following steps: performing sliding extraction on the input remote sensing image by utilizing an edge extraction operator to generate a gradient map of the remote sensing image; preferably, the gradient map is subjected to gating filtering, a threshold value is set, and pixel values smaller than the threshold value are set to be zero, so that noise and useless texture information are removed.
In a specific embodiment of the present invention, the step2 includes:
the proposed boundary sensing fusion module mainly comprises three parts: boundary map extraction, boundary perception feature extraction and boundary guidance feature fusion. And extracting the boundary map by using a boundary map extraction module. Specifically, the input image is subjected to sliding extraction by utilizing an edge extraction operator, and a gradient map of the image is generated. At this point, a lot of noise and useless detail texture may be contained in the gradient map. A hard threshold based gating filtering method is used to reject these disturbances. Specifically, by defining a set threshold, pixels in the image with pixel values smaller than the threshold are set to 0, so that edges of the image are preserved as much as possible.
Boundary perceptual features are extracted by a boundary feature extraction module, and specifically, for a generated boundary map, bottom features are extracted by a3×3 convolution layer, a BN layer, a ReLU layer, and a3×3 convolution layer. A bottleneck structure with asymmetric convolution extracts useful edge features and further suppresses noise. Specifically, first, a1×1 convolution, a BN layer, and a ReLU layer are used to reduce the dimension of a feature map, and three parallel branches respectively including a1×3 convolution, a3×1 convolution, and a3×3 convolution are used to extract potential edge features, and the extracted potential edge features are fused; then adding 1×1 convolution, BN layer, reLU layer to recover the characteristic channel; then, the underlying features are transferred through the jump connection to be fused with the extracted potential edge features, and the boundary sensing features are generated through 1×1 convolution and Sigmoid activation, and the features contain rich edge information.
And utilizing a boundary guidance feature module to fuse boundary features. Specifically, firstly, reducing the resolution of the boundary sensing features by using a downsampling operation to make the resolution of the boundary sensing features consistent with the basic features of the feature extraction network in the step 1, and fusing the boundary sensing features and the basic features in the form of element sums; then, adding the edge fusion feature generated in the previous step with the basic feature of the feature extraction network by using jump connection to amplify the edge structure in the basic feature, and integrating important information through a 3×3 convolution layer; and secondly, emphasizing important characteristic channels by using a channel attention mechanism, wherein the channel attention mechanism comprises a global average pooling layer, a1 multiplied by 1 convolution layer and a Sigmoid activation layer and is used for generating channel attention weights, multiplying important information by the edge fusion characteristics through jump connection, and finally adding the important information with the fusion characteristics output by the characteristic pyramid to generate boundary guiding characteristics.
Step 3: the model decouples the extracted texture perception features and boundary guide features, outputs the texture perception features and the boundary guide features, inputs the texture perception features and the boundary guide features into RCNN of task decoupling for double-branch decoupling prediction, and obtains classification and positioning results;
In a specific embodiment of the present invention, the step 3 includes:
And (3) independently inputting the texture perception features and the boundary guiding features generated in the step (1) and the step (2) into a task decoupling RCNN network to perform decoupling type and position prediction, inputting the texture perception features into a classification branch, and inputting the boundary guiding features into a positioning branch.
Specifically, the texture perception feature generated in the step 1 is input to a classification branch in the task decoupling RCNN, and the boundary guiding feature generated in the step 2 is input to a positioning branch in the task decoupling RCNN;
Setting different thresholds, and decoupling the classifying branch and the positioning branch to non-maximal inhibition NMS, so as to generate suggested areas with different densities;
decoupling sampling is carried out on the suggested area, and a decoupled region of interest is obtained; for the classification branches, adopting a fixed proportion to sample quantitative positive samples and negative samples; sampling a quantitative positive sample for the positioning branch;
and expanding the region of interest range of the classification branch, and synthesizing a high-quality region of interest for the positioning branch by utilizing Weiszfeld algorithm to assist in positioning prediction.
In a specific embodiment, firstly, decoupling NMSs of two branch networks by setting different thresholds to generate suggested areas with different densities; subsequently, the recommended region is decoupled and sampled to obtain a decoupled region of interest, and for the classification branches, a fixed ratio is used to sample quantitative positive and negative samples. For the positioning branch, only a quantitative positive sample is sampled; next, targeted enhancements are made to the two-branch network. Classification is aided by the introduction of more context information by expanding the range of the region of interest. The extended range requires trade-offs, too small a range is insufficient to introduce sufficient information, and too large a range will introduce excessive noise interference. For the positioning branch, a suggested synthesis strategy is adopted to synthesize a high-quality region of interest to assist positioning. The suggested synthesis strategy is based on Weiszfeld algorithm, synthesizes the upper left corner and the lower right corner of all suggested regions and real labeling regions allocated to each target, and searches the geometric center of the discrete sample points in the form of iterative re-weighted least square. Specifically, assuming the reference point to be fitted is x i, the algorithm attempts to find a center point y by iterating so that the distance between the point and the reference point is minimized.
Wherein, lll 2 represents the L 2 norm, j represents the number of reference points, D represents the sum of the distances between the synthesized center point y and the reference point x, argmin represents the parameter value y, which minimizes the sum. Solving in continuous space, the above equation will take the minimum at the extremum according to the optimization theory. Thus, bias y:
Solving the above equation, finding a center point which does not coincide with any reference point.
Where k represents the number of iterations. And (3) continuously iterating to obtain a central point which is the fitted geometric center, wherein D is smaller than a specified value. And synthesizing a suggested area close to the real labeling frame through fitting the upper left corner and the lower right corner of the frame, training and positioning a branch network, accelerating the convergence of the model and improving the stability of the model.
In a specific embodiment of the present invention, a system for detecting a perceived texture and a boundary target of a weak target of a remote sensing image is further provided, where the system for detecting a perceived texture and a boundary target of a weak target of a remote sensing image includes:
The feature extraction network is used for extracting basic features of the remote sensing image;
the texture perception enhancement module is used for extracting texture perception features through the basic features;
the boundary map extraction module is used for extracting a binary boundary map of the remote sensing image;
the boundary feature extraction module is used for extracting boundary perception features of the boundary map;
the boundary guiding feature module is used for fusing the boundary perception feature and the basic feature to obtain a boundary guiding feature;
and the decoupling transmission module is used for decoupling the texture perception feature and the boundary guide feature, outputting the texture perception feature and the boundary guide feature, inputting the texture perception feature and the boundary guide feature into a RCNN network with task decoupling for performing double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection.
Compared with the prior art, the invention discloses a unique texture and boundary sensing network to solve the problem of weak and small targets in remote sensing images. In order to mine deep information, a texture perception enhancement module and a boundary perception fusion module are provided. The former can fully explore the texture details inside the target, which builds the link between features through the calculated covariance matrix, while highlighting the texture differences between the target and the background. The latter introduces additional boundary cues to highlight the spatial position of the object. By exploring high-frequency detailed information such as textures, boundaries and the like of key targets, key features of weak and small targets are supplemented. The combination of the two modules enhances the perceptibility of the network to the weak and small targets. In order to relieve task unfocused phenomena caused by winding between classification and regression, a RCNN for task decoupling is provided to independently train two branch networks, so that fine prediction on classification and positioning is realized. The method provided by the invention can be used for mining the information of the remote sensing weak and small target which is difficult to be highlighted, and further improving the expression of the weak and small target, so as to improve the detection performance of the weak and small target.
The present invention will be further described with reference to specific examples and drawings.
In a specific embodiment of the invention, a flow of a sensing texture and boundary target detection method for a weak target of a remote sensing image is shown in fig. 1, a remote sensing image with the weak target is specifically input into a feature extraction network to extract basic features, the basic features are input into a texture sensing enhancement module to extract texture sensing features, the remote sensing image with the weak target is simultaneously input into a boundary map extraction module to extract a boundary map of the image, a boundary sensing fusion module is used for extracting boundary guiding features according to the boundary map, the texture sensing features and the boundary guiding features are independently input into a task decoupling RCNN network, a sampling strategy of decoupling is adopted to obtain a task-specific region of interest, a decoupling suggestion expansion strategy and a suggested synthesis strategy are used for enhancing training of a classification and regression network, and the trained network can effectively detect the weak target of the remote sensing image.
Fig. 2 is a remote sensing image of a weak and small target in an embodiment of the present invention, and it can be seen from the figure that the target of the image presents a small size due to a very high imaging height and an imaging view angle of a bird's eye view, and meanwhile, the information of the targets is greatly suppressed due to uneven illumination and shielding of a cloud layer, so that weak feature expression is presented. These targets are easily submerged in a broad background, resulting in the loss of critical information such as texture, boundaries, colors, etc., and in addition to further compression processing by the network, these weak targets are easily ignored by the network, causing a large range of missed detection. Aiming at the key problem of weak and small target feature deletion, the invention provides a key technology for excavating texture and boundary features, and effectively improves the detection accuracy of the weak and small target.
Fig. 3 is an overall schematic diagram of a remote sensing image-oriented weak and small target detection method according to a specific embodiment of the present invention, and as can be seen from the figure, the method provided by the embodiment of the present invention mainly includes three parts, namely, a first texture perception enhancement module: the method is responsible for exploring the texture mode of the image and extracting the texture perceived characteristics; second, a boundary perception fusion module: the method is in charge of fusing the extracted boundary sensing characteristics with basic characteristics and generating boundary guiding characteristics; third, task decoupling RCNN network: and the method is responsible for decoupling texture perception features and boundary guidance features, and realizes detection of weak and small targets of the remote sensing image by utilizing the decoupled features in combination with a decoupling sampling strategy, a decoupling suggestion expansion strategy and a suggestion synthesis strategy to train classification and regression network in a targeted manner.
Fig. 4 is a schematic diagram of a texture-aware enhancement module according to an embodiment of the present invention, specifically, taking as an example a fused feature f with a resolution of 256×100×100 output by a feature pyramid (i.e. the fused feature f i mentioned above, i being 3,4,5, and taking i=3 as an example here), first, the feature map integrates the 1×1 convolutionally that undergoes three parallel branches and reduces the dimension to 64, resulting in feature f θ,F g, and characterizing f θ,/>Reduce the dimension to/>Next, the features/>, of f θ and transpose are calculatedThe covariance matrix sigma between the two pixels is used for capturing the correlation between pixels at different positions in a characteristic dimension space, and a calculation formula is as follows:
where (x, y) represents the position of the pixel, Representing the multiplication of the elements. However, the above steps can only calculate the correlation of all pixels without difference, and cannot focus the pixel correlation inside the target. For this purpose, a spatial attention mechanism is introduced to strengthen the position of the object and highlight the pixel relationship inside the object. Firstly, carrying out global average pooling of channel dimension on an input feature f, and adopting 3×3 convolution to remove redundant noise; subsequently, the Sigmoid activation function is used for feature normalization to generate attention weights, and finally the weight dimensions are converted into/>An attention weight map is generated and aligned with the channel dimension of the covariance matrix Σ, which can be expressed as:
in this particular embodiment, the formula is predominantly clear of the c value compared to the original formula (2).
M(x,y)=Reshape(σ(Conv3×3(fgap(x,y)))) (3)
Wherein GAP represents global average pooling operation, and sigma represents Sigmoid activation; subsequently, the attention map M (x, y) is multiplied by the covariance matrix Sigma (x, y) to obtain a target-enhanced covariance matrixAt this time, the dimension of the covariance matrix of the target enhancement calculated is/> Regularizing these values by using SoftMax, i.e., regularizing the entirety of the calculated target enhanced covariance matrix, and converting the dimensions of the target enhanced covariance matrix to/>The covariance matrix of the target enhancement before processing is two-dimensional, and the covariance matrix of the target enhancement is converted into three-dimensional after conversion; wherein the first dimension 10000 represents the number of covariance matrices of the target enhancement, which is equal to the feature size; the second dimension 100 represents the covariance matrix/>, of the target enhancementIs a length of (2); the third dimension 100 represents the covariance matrix/>, of the target enhancementIs a width of (c). Thus,/>The meaning represented mathematically is the relationship between a pixel and all pixels at other locations. It can be written as:
wherein, as follows, the dot product of the matrix; at the same time, a 1X 1 convolution operation is adopted to integrate the relation matrix of pixels between different channels and convert the channels into At this time, a relationship map is obtained that can amplify the similar positions of the homogeneous pixels and the target, which can be used to emphasize the global similar texture representation of the target, further enhancing the beneficial texture features; finally, by means of the method of the/>Multiplied by f g to obtain a valuable texture feature and the feature channel is restored to 256 using a1 x 1 convolution. By adding the captured texture features to the original features f, texture-aware features are obtained. The treatment process is as follows:
Wherein, Representing matrix element addition.
Fig. 5 to fig. 7 are a schematic diagram of boundary map extraction, a schematic diagram of boundary sensing feature extraction, and a schematic diagram of boundary sensing feature fusion in a boundary sensing fusion module according to an embodiment of the invention. Specifically, assuming that an input image is I (x, y), as an alternative scheme, the embodiment of the invention takes a Sobel kernel as an example to extract a directional gradient map in a sliding manner in the horizontal and vertical directions of I (x, y); the gradient is the difference between adjacent pixels, and the pixel difference in the region where the gradient is severe is large, which is generally expressed as the edge inside the image. And superposing the horizontal gradient map and the vertical gradient map to obtain a boundary map G (x, y) of the image. However, there are many discrete noise points in G (x, y) and texture details of the disturbances, which are filtered using gating to obtain a fine boundary mapTo support extraction of boundary-aware features, which can be expressed as:
Wherein λ is a set threshold, which is set to 0.15 in the embodiment of the present invention. When (when) When the pixel value of (2) is smaller than 0.15 times the maximum gray value, it is set to 0. The boundary sensing feature is further extracted by a boundary feature extraction module, and the bottom layer feature f base is extracted by a3×3 convolution layer, a BN layer, a ReLU layer and a3×3 convolution layer. A bottleneck structure with asymmetric convolution, symmetric convolution, is used to extract useful edge features and further suppress noise. Specifically, firstly, a1×1 convolution layer, a BN layer and a ReLU layer are used for reducing the dimension of a feature map, and three parallel branches respectively comprising 1×3 convolution, 3×1 convolution and 3×3 convolution are utilized to extract and fuse edge features; then, adding a1×1 convolution layer, a BN layer and a ReLU layer to recover the characteristic channel; then, the basic feature f base is transferred through the jump connection to be fused with the fused edge feature, and the boundary perception feature B f is generated through the 1×1 convolution layer and the Sigmoid activation layer, wherein the feature contains rich edge information.
Bf=σ(Conv1×1(fbou)) (12)
Wherein Bottleneck represents the front to back two 1x1 convolution layers, BN layer, reLU layer, f 1×3,f3×1,f3×3 represents the edge features extracted by asymmetric and symmetric convolutions, respectively. The boundary guidance feature module is further utilized to fuse boundary features, as information compensation, the extracted edge perception feature B f is fused with the basic feature C i, i is epsilon {2,3,4,5}, effective emphasis of the edge is achieved, and the edge is integrated into the output P i of the FPN to further highlight edge expression of the feature. Specifically, the resolution of the boundary sensing feature is reduced by using a downsampling operation to be consistent with the basic feature of the feature extraction network, and the basic feature and the boundary sensing feature are fused in the form of element sum, which can be expressed as:
Wherein f i represents the fused feature and D represents the downsampling operation; then, adding the fused features with the basic features of the feature extraction network by using jump connection to amplify the edge structure in the features, and integrating important information by 3×3 convolution; secondly, a channel attention mechanism is utilized to emphasize important characteristic channels, the channel attention mechanism comprises a global average pooling layer, a1 multiplied by 1 convolution layer and a Sigmoid activation layer, and the generated channel attention weight is multiplied by the fused characteristic f i through jump connection to amplify the characteristics of the important channels; finally, the feature is added with the fusion feature P i of the feature pyramid to generate a feature of boundary guidance, which can be expressed as:
Fig. 8 is a schematic diagram of a task decoupling RCNN network, which includes two separate classification and positioning branches, in accordance with an embodiment of the present invention. After the task decoupling RCNN network obtains the decoupling characteristics, the task decoupling RCNN network firstly performs decoupling sampling to generate specific suggestions for different tasks so as to prevent information interaction between the tasks. Specifically, the task decoupling RCNN network first executes NMS with different thresholds to obtain different distribution of suggested regions. The embodiment of the invention sets the NMS threshold value of classification to 0.7, and the NMS threshold value of positioning to 0.85. After the suggested areas with different densities are obtained, decoupling sampling is carried out on the suggested areas, and the task-specific interested areas are obtained. For classification branches, the embodiment of the invention randomly samples 256 training samples and maintains 1:3, the positive and negative sample ratio enables the network to fully learn the foreground and background information. For locating branches, the embodiment of the invention randomly samples only 128 positive samples from the positive sample set to train the regression branches. Second, optimization is performed for a particular task. For the classification branch, a decoupled suggested expansion strategy is implemented to absorb more context information for the region of interest to assist in classification, and embodiments of the present invention expand the region of interest to a 1.3-fold range. For positioning branches, by suggesting a synthetic strategy, more high quality samples are added to the network to facilitate training of the positioning network. Finally, after the independent ROI alignment operation is performed to unify the sizes, the decoupled region of interest is input into an independent full-connection layer to realize the classification and positioning prediction of the target.
FIG. 9 is a schematic diagram of a proposed composition strategy according to an embodiment of the present invention, which is based on Weiszfeld algorithm, to compose all the proposed regions allocated to each target with the upper left corner and lower right corner of the true labeling region, and find the geometric center of the discrete sample points by iterative re-weighted least squares. Specifically, the embodiment of the invention assumes that there are 4 reference points x i, i e {1,2,3,4} to be fitted, and the algorithm iterates to find a center point y such that the distance between the point and the 4 reference points is minimized.
Wherein II 2 represents the L2 norm and argmin represents the parameter value y, which minimizes the sum. Solving in continuous space, the above equation will take the minimum at the extremum according to the optimization theory. Thus, bias y:
Solving the above equation, and solving a geometric median point, wherein the median point does not coincide with any reference point.
Where k represents the number of iterations. Through continuous iteration, the obtained median point is the fitted geometric median point, and the obtained median point is the geometric median point, so that the formula 15 is smaller than a specified value. And synthesizing a high-quality region of interest close to the real labeling frame by fitting the upper left corner and the lower right corner of the frame to train and position the branch network, accelerating the convergence of the model and improving the stability of the model.
Fig. 10 is an exemplary diagram of the detection result of a weak target in a remote sensing image by using the target detection method according to the embodiment of the present invention, and as shown in the drawing, the targets exhibit weak details due to the blockage of illumination and shadow, and by adding to their small size, so that the targets are difficult to be effectively predicted, especially for buildings, vessels, cable towers, self-constructed houses, etc. in the drawing. The remote sensing image target detection method for the weak targets can still effectively recall the weak targets, accurately identify the types and the positions of the targets, and fully demonstrate the detection capability of the method for the weak targets.
While embodiments of the present invention have been illustrated and described above, it will be appreciated that the above described embodiments are illustrative and should not be construed as limiting the invention. Variations, modifications, alternatives and variations of the above-described embodiments may be made by those of ordinary skill in the art within the scope of the present invention.
The above embodiments of the present invention do not limit the scope of the present invention. Any other corresponding changes and modifications made in accordance with the technical idea of the present invention shall be included in the scope of the claims of the present invention.
Claims (7)
1. The method for detecting the perceived texture and the boundary target of the weak target facing the remote sensing image is characterized by comprising the following steps of:
Step 1: inputting the remote sensing image into a feature extraction network to extract basic features, inputting the basic features into a texture perception enhancement module, and extracting texture perception features;
inputting the remote sensing image into a feature extraction network to extract basic features Inputting the basic features into a feature pyramid to obtain fusion features/>;
Wherein c represents the channel dimension of the fusion feature; h and w represent the length and width of the fusion feature, respectively;
subjecting the fused feature map to 1×1 convolution of three parallel branches to integrate features and reduce the dimension to c 1 to obtain processed features And will characterize/>Reduce the dimension to/>;
Calculation ofFeatures with transpose/>Covariance matrix/>Capturing correlation among pixels at different positions in a feature dimension space, wherein a calculation formula is as follows:
Wherein, Representing the intermediate features after convolution processing, (x, y) representing the position of the pixel point,/>Multiplying the representative elements;
introducing a spatial attention mechanism to strengthen the position of a target in the remote sensing image and highlight the pixel relationship in the target;
the fusion features to the input Global average pooling of channel dimensions is performed and 3 x 3 is employed to remove unwanted noise;
The Sigmoid activation function is used for feature normalization to generate attention weight;
Converting weight dimensions to Generating an attention weighting map/>And is associated with the covariance matrix/>Is expressed as:
Wherein GAP stands for global average pooling operation, Representative Sigmoid activation;
Attention weighting map And covariance matrix/>Multiplying to obtain a covariance matrix/>, which is enhanced by the target;
The dimension of the covariance matrix of the target enhancement is calculated as;
Regularizing the target enhanced covariance matrix and converting dimensions of the target enhanced covariance matrix to;
Wherein the first dimension wh represents the number of covariance matrices of the target enhancement, which is equal to the feature size; the second dimension w represents the covariance matrix of the target enhancementIs a length of (2); the third dimension h represents the covariance matrix/>, of the target enhancementIs a width of (2);
Wherein, Representing a matrix dot product;
Integrating the relation matrix of pixels between different channels by adopting a 1X 1 convolution operation, and converting the channels into ;
By combiningAnd/>Multiplying and restoring the characteristic channel to c by using 1×1 convolution to obtain texture characteristics;
By adding captured texture features to the fused features Obtaining texture perception characteristics;
Adding captured texture features to the fused features In the method, the texture perception characteristic is obtained through the following formula:
;
Wherein, Representing matrix element addition;
Step 2: inputting a remote sensing image into a boundary map extraction module to extract a binary boundary map, inputting the boundary map into a boundary feature extraction module to extract boundary perception features; fusing the boundary perception feature and the basic feature through a boundary guiding feature module to obtain a boundary guiding feature;
step 3: decoupling and outputting the texture perception feature and the boundary guiding feature, inputting the texture perception feature and the boundary guiding feature into a RCNN network with task decoupling for double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection;
inputting the texture perception feature generated in the step 1 to a classification branch in the task decoupling RCNN, and inputting the boundary guiding feature generated in the step 2 to a positioning branch in the task decoupling RCNN;
Setting different thresholds, and decoupling the classifying branch and the positioning branch to non-maximal inhibition NMS, so as to generate suggested areas with different densities;
decoupling sampling is carried out on the suggested area, and a decoupled region of interest is obtained; for the classification branches, adopting a fixed proportion to sample quantitative positive samples and negative samples; sampling a quantitative positive sample for the positioning branch;
and expanding the region of interest range of the classification branch, and synthesizing a high-quality region of interest for the positioning branch by utilizing Weiszfeld algorithm to assist in positioning prediction.
2. The method for detecting a perceived texture and a boundary target for a weak and small target of a remote sensing image according to claim 1, wherein the step 1 of inputting the basic feature into a texture perception enhancement module, the extracting texture perception features comprises: and inputting the basic features into a feature pyramid to obtain fusion features, inputting the fusion features into a texture perception enhancement module, and extracting texture perception features.
3. The method for detecting a perceived texture and a boundary target for a weak and small target of a remote sensing image according to claim 1, wherein the inputting the remote sensing image into the boundary map extraction module to extract a binary boundary map comprises: and performing sliding extraction on the input remote sensing image by utilizing an edge extraction operator to generate a gradient map of the remote sensing image.
4. The method for detecting the perceived texture and boundary target of the weak and small target of the remote sensing image according to claim 3, wherein the gradient map is subjected to gating filtering, a threshold value is set, and pixel values smaller than the threshold value are set to zero so as to remove noise and useless texture information.
5. The method for detecting a perceived texture and a boundary target for a weak and small target of a remote sensing image according to claim 1, wherein the inputting the boundary map into the boundary feature extraction module, the extracting boundary perceived features comprises:
extracting bottom features by adopting 3×3 convolution, BN, reLU and 3×3 convolution;
Reducing the dimension of the bottom layer feature by adopting 1×1 convolution, BN and ReLU operations, extracting edge information in the bottom layer feature by utilizing three parallel branches comprising 1×3 convolution, 3×1 convolution and 3×3 convolution, and summarizing and fusing the edge information to obtain potential edge features;
Recovering a characteristic channel by adopting 1X 1 convolution, BN and ReLU operation;
And transferring the bottom layer feature to be fused with the potential edge feature through jump connection, and generating boundary sensing features through 1X 1 convolution and Sigmoid activation.
6. The method for detecting a perceived texture and a boundary target for a weak and small target of a remote sensing image according to claim 1, wherein the fusing the boundary perceived feature and the basic feature by the boundary guiding feature module, and obtaining the boundary guiding feature comprises:
adding the boundary sensing characteristic after downsampling and the basic characteristic to fuse the characteristic, adopting a jump connection structure and a3 multiplied by 3 convolution amplifying edge structure, and integrating the fused information;
And highlighting important feature channels in the fusion features by adopting a channel attention mechanism, and adding the important feature channels with the fusion features output by the feature pyramid to obtain boundary guiding features.
7. The remote sensing image weak and small target-oriented sensing texture and boundary target detection system using the remote sensing image weak and small target-oriented sensing texture and boundary target detection method as claimed in claim 1, wherein the remote sensing image weak and small target-oriented sensing texture and boundary target detection system comprises:
The feature extraction network is used for extracting basic features of the remote sensing image;
the texture perception enhancement module is used for extracting texture perception features through the basic features;
the boundary map extraction module is used for extracting a binary boundary map of the remote sensing image;
the boundary feature extraction module is used for extracting boundary perception features of the boundary map;
the boundary guiding feature module is used for fusing the boundary perception feature and the basic feature to obtain a boundary guiding feature;
and the decoupling transmission module is used for decoupling the texture perception feature and the boundary guide feature, outputting the texture perception feature and the boundary guide feature, inputting the texture perception feature and the boundary guide feature into a RCNN network with task decoupling for performing double-branch decoupling prediction, obtaining classification and positioning results, and finishing perception texture and boundary target detection.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311669036.4A CN117636172B (en) | 2023-12-06 | 2023-12-06 | Target detection method and system for weak and small target of remote sensing image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311669036.4A CN117636172B (en) | 2023-12-06 | 2023-12-06 | Target detection method and system for weak and small target of remote sensing image |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117636172A CN117636172A (en) | 2024-03-01 |
CN117636172B true CN117636172B (en) | 2024-06-21 |
Family
ID=90031892
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311669036.4A Active CN117636172B (en) | 2023-12-06 | 2023-12-06 | Target detection method and system for weak and small target of remote sensing image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117636172B (en) |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110633661A (en) * | 2019-08-31 | 2019-12-31 | 南京理工大学 | Semantic segmentation fused remote sensing image target detection method |
CN112200045B (en) * | 2020-09-30 | 2024-03-19 | 华中科技大学 | Remote sensing image target detection model establishment method based on context enhancement and application |
CN112766184B (en) * | 2021-01-22 | 2024-04-16 | 东南大学 | Remote sensing target detection method based on multi-level feature selection convolutional neural network |
CN113723172A (en) * | 2021-06-11 | 2021-11-30 | 南京航空航天大学 | Fusion multi-level feature target detection method for weak and small targets of remote sensing images |
CN114120102A (en) * | 2021-11-03 | 2022-03-01 | 中国华能集团清洁能源技术研究院有限公司 | Boundary-optimized remote sensing image semantic segmentation method, device, equipment and medium |
CN115019201B (en) * | 2022-05-20 | 2024-03-05 | 西北工业大学 | Weak and small target detection method based on feature refinement depth network |
CN114972860A (en) * | 2022-05-23 | 2022-08-30 | 郑州轻工业大学 | Target detection method based on attention-enhanced bidirectional feature pyramid network |
CN115497005A (en) * | 2022-09-05 | 2022-12-20 | 重庆邮电大学 | YOLOV4 remote sensing target detection method integrating feature transfer and attention mechanism |
CN116051984B (en) * | 2022-12-20 | 2023-07-04 | 中国科学院空天信息创新研究院 | Weak and small target detection method based on Transformer |
CN116385896A (en) * | 2023-03-20 | 2023-07-04 | 西安电子科技大学 | Remote sensing small target detection method, system, equipment and medium based on fusion cascade attention mechanism |
CN116310850B (en) * | 2023-05-25 | 2023-08-15 | 南京信息工程大学 | Remote sensing image target detection method based on improved RetinaNet |
CN116704373A (en) * | 2023-06-21 | 2023-09-05 | 电子科技大学 | Remote sensing target detection method based on centrality and mutual exclusion constraint |
CN117036980A (en) * | 2023-08-31 | 2023-11-10 | 中国科学院长春光学精密机械与物理研究所 | Satellite remote sensing image small target detection method based on high-resolution characteristic self-attention |
-
2023
- 2023-12-06 CN CN202311669036.4A patent/CN117636172B/en active Active
Non-Patent Citations (2)
Title |
---|
Orientation-First Strategy With Angle Attention Module for Rotated Object Detection in Remote;Yuxi zhang et al.;《IEEE》;20220928;全文 * |
基于深度学习的单幅图片超分辨率重构研究进展;张宁等;《自动化学报》;20190624;全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN117636172A (en) | 2024-03-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110853026B (en) | Remote sensing image change detection method integrating deep learning and region segmentation | |
CN110782477A (en) | Moving target rapid detection method based on sequence image and computer vision system | |
CN113408492A (en) | Pedestrian re-identification method based on global-local feature dynamic alignment | |
CN110135438B (en) | Improved SURF algorithm based on gradient amplitude precomputation | |
CN111462027B (en) | Multi-focus image fusion method based on multi-scale gradient and matting | |
CN109376641B (en) | Moving vehicle detection method based on unmanned aerial vehicle aerial video | |
KR101928391B1 (en) | Method and apparatus for data fusion of multi spectral image and radar image | |
CN105303545A (en) | Fusion method for multispectral image and panchromatic satellite image | |
CN112785626A (en) | Twin network small target tracking method based on multi-scale feature fusion | |
CN115984323A (en) | Two-stage fusion RGBT tracking algorithm based on space-frequency domain equalization | |
CN112862748B (en) | Multi-dimensional domain feature combined SAR ship intelligent detection method | |
CN112967305A (en) | Image cloud background detection method under complex sky scene | |
CN117636172B (en) | Target detection method and system for weak and small target of remote sensing image | |
CN113763274A (en) | Multi-source image matching method combining local phase sharpness orientation description | |
CN113239828A (en) | Face recognition method and device based on TOF camera module | |
CN112734636A (en) | Fusion method of multi-source heterogeneous remote sensing images | |
Liu et al. | Multiscale registration of remote sensing image using robust SIFT features in Steerable-Domain | |
CN114120033A (en) | Hyperspectral image classification method, device, equipment and storage medium | |
CN113158942A (en) | Segmentation algorithm and device for detecting motion human behavior | |
Gu et al. | Aerial image and map synthesis using generative adversarial networks | |
Shakenov | Algorithms of background suppression in the problem of detection of point targets in images | |
Maiti et al. | UAVPal: A New Dataset for Semantic Segmentation in Complex Urban Landscape with Efficient Multi-scale Segmentation | |
CN113487487B (en) | Super-resolution reconstruction method and system for heterogeneous stereo image | |
CN110991313B (en) | Moving small target detection method and system based on background classification | |
Zhang et al. | Improved salient objects detection based on salient points |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |