CN110796028A - Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation - Google Patents

Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation Download PDF

Info

Publication number
CN110796028A
CN110796028A CN201910963413.2A CN201910963413A CN110796028A CN 110796028 A CN110796028 A CN 110796028A CN 201910963413 A CN201910963413 A CN 201910963413A CN 110796028 A CN110796028 A CN 110796028A
Authority
CN
China
Prior art keywords
feature
transformation
local
amplification
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910963413.2A
Other languages
Chinese (zh)
Other versions
CN110796028B (en
Inventor
肖志峰
钱林钧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201910963413.2A priority Critical patent/CN110796028B/en
Publication of CN110796028A publication Critical patent/CN110796028A/en
Application granted granted Critical
Publication of CN110796028B publication Critical patent/CN110796028B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Astronomy & Astrophysics (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides an unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation, which comprises the following steps: 1) constructing a feature extractor; 2) extracting features; 3) locally stretching and amplifying the branches for processing; 4) obtaining a region needing to be transformed; 5) constructing a transformation function; 6) local transformation and target regression; 7) the result is inverse transformed. The method can adaptively amplify partial areas of the convolutional network characteristic diagram, and effectively detect the small-scale targets in the unmanned aerial vehicle image.

Description

Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation
Technical Field
The invention belongs to the technical field of automatic target identification, and particularly relates to an automatic identification method for a small unmanned aerial vehicle image target.
Background
With the development of the unmanned aerial vehicle technology, unmanned aerial vehicles equipped with cameras have been widely deployed and applied to various industries such as agriculture, traffic, aerial photography and the like. Because the unmanned aerial vehicle has the characteristic of flying in the air, the image formed by the unmanned aerial vehicle has the characteristics that the visual angle is taken down or taken obliquely, and the target size is smaller. There are currently great advances in computer vision target detection, but current computer vision algorithms and data sets are designed and evaluated for laboratory settings with human-centered photographs of close-range natural scene objects taken horizontally. For the unmanned aerial vehicle image shot vertically, the interested objects are relatively small and the features are few, and generally, in the process of processing data, a target detection method based on deep learning easily loses target detail information along with deepening of a neural network, so that the detection positioning is inaccurate or cannot be detected. The FPN (feature pyramid network) network uses multi-scale features including high-resolution feature maps to predict targets, and has a certain effect on small target detection, but the detection effect is not ideal when the target scale is smaller than the down-sampling step of the feature map with the highest resolution.
Disclosure of Invention
Aiming at the problems of the existing unmanned aerial vehicle image small target detection method, the invention provides the unmanned aerial vehicle image small target detection method based on the local adaptive geometric transformation based on the deep convolutional neural network method, and the method can effectively detect the small-scale target in the unmanned aerial vehicle image.
The technical scheme of the invention is an unmanned aerial vehicle image small target detection method based on local adaptive geometric transformation, which mainly comprises the following steps:
1) construct feature extractor
Constructing a feature extractor with a self-adaptive local stretching amplification branch based on a deep full-convolution neural network;
2) extracting features
Performing feature extraction on an input image by using the deep fully-convolutional neural network feature extractor with the self-adaptive local stretching amplification branch constructed in the step 1), wherein the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the nth layer of feature extraction operation, xnIs passing through fnCalculating the obtained characteristics;
3) local stretching amplifying branch for processing
For the shallow feature x extracted in the step 2)mWherein m is<k, the down sampling step is S, the partial stretching amplification branch is used for processing, and x is obtained through the branch output result CmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jWhether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown;
4) obtaining the required transformation area
Thresholding C using a threshold t, Ci,jIf the two amplified external rectangles have an overlapping part, the external rectangle before amplification with smaller area is removed, the filtered rectangle before amplification forms a region set A needing amplification, each element in the set is in xmA rectangle on the feature layer coordinate system represents that the area needs to be enlarged;
5) constructing transformation functions
According to the size and the amplification factor of the rectangle in the set A, calculating a feature map after local amplification of the rectangular region and an original feature map xmMapping relation f and inverse transformation f of feature point coordinates-1；
6) Local transformation and object regression
According to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2Continuously sending the convolution layer to obtain the position and the class information of the detection frame;
7) result inverse transformation
Calculating the detection box at xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
The invention provides an unmanned aerial vehicle image small target detection system based on local adaptive geometric transformation, which comprises the following modules:
the feature extractor construction module: the method is used for constructing a feature extractor with a self-adaptive local stretching amplification branch on the basis of a deep full convolution neural network;
a feature extraction module: depth full volume with adaptive local stretch magnification branches for building with feature extractorThe product neural network feature extractor performs feature extraction on the input image, and the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the nth layer of feature extraction operation, xnIs passing through fnCalculating the obtained characteristics;
the local stretching and amplifying branch processing module comprises: extracting shallow layer feature x from feature extracting modulemWherein m is<k, the down sampling step is S, the partial stretching amplification branch is used for processing, and x is obtained through the branch output result CmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jWhether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown;
a transformation area acquisition module: for thresholding C using a threshold t, Ci,jIf the two amplified external rectangles have an overlapping part, the external rectangle before amplification with smaller area is removed, the filtered rectangle before amplification forms a region set A needing amplification, each element in the set is in xmA rectangle on the feature layer coordinate system represents that the area needs to be enlarged;
the transformation function building module: is used for calculating the feature map after the local amplification of the rectangular region and the original feature map x according to the size and the amplification factor of the rectangle in the set AmMapping relation f and inverse transformation f of feature point coordinates-1；
Local transformation and target regression module: according to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2Continuously sending the convolution layer to obtain the position and the class information of the detection frame;
a result inverse transformation module: for calculating the detection box at xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
Compared with the prior art, the invention has the following advantages and beneficial effects: on the premise of not greatly increasing the calculation amount of the detection network, the small-scale target detection performance of the network is increased.
Drawings
Fig. 1 is a flow chart of unmanned aerial vehicle image small target detection based on local adaptive geometric transformation.
Detailed Description
The technical solution of the present invention is further explained with reference to the drawings and the embodiments.
As shown in fig. 1, the present invention provides a method for detecting small targets in an image of an unmanned aerial vehicle based on local adaptive geometric transformation, which mainly comprises the following steps:
1) construct feature extractor
Constructing a feature extractor with an adaptive local stretching amplification branch based on a deep full convolution neural network, wherein, as shown in fig. 1, conv1, conv2, conv3, conv4 and conv5 are used as basic feature extraction networks, and the local stretching transformation branch input features are output results of a conv2 layer;
2) extracting features
Performing feature extraction on an input image by using the depth fully-convolutional neural network feature extractor with the adaptive local stretching amplification branch constructed in the step 1), wherein the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the n-th layer of feature extraction operation (the feature extraction mode is convolution, pooling and activation function), xnIs passing through fnAnd calculating the obtained characteristics.
3) Local stretching amplifying branch for processing
For the shallow feature x extracted in the step 2)m，(m<k) The down sampling step length is S, the partial stretching amplifying branch is used for processing, and the result C is output through the branch to obtain xmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jAnd whether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown.
3) Obtaining the required transformation area
Thresholding C using a threshold t, Ci,jIf the two enlarged external rectangles have an overlapped part, the enlarged external rectangle with smaller area is removed, the filtered enlarged external rectangle forms an area set A, each element in the set is in xmA rectangle on the feature layer coordinate system represents that this area needs to be enlarged. The flooding filling algorithm is the prior art, and the invention is not described.
4) Constructing transformation functions
According to the size and the amplification factor of the rectangle in the set A, calculating a feature map after local amplification of the rectangular region and an original feature map xmMapping relation f and inverse transformation f of feature point coordinates-1。
5) Local transformation and object regression
According to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2And continuously feeding the subsequent convolution layer to obtain the position and the class information of the detection frame.
6) Result inverse transformation
According to the detection frame and xm2The proportion relation of the characteristic diagram can obtain the result of the detection frame in xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
The utility model provides an unmanned aerial vehicle image small-object detection system based on local adaptive geometric transformation which characterized in that includes following module:
the feature extractor construction module: the method is used for constructing a feature extractor with a self-adaptive local stretching amplification branch on the basis of a deep full convolution neural network;
a feature extraction module: the method is used for extracting the features of an input image by utilizing a deep full convolution neural network feature extractor with an adaptive local stretching amplification branch constructed in a feature extractor construction module, and the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the nth layer of feature extraction operation, xnIs passing through fnCalculating the obtained characteristics;
the local stretching and amplifying branch processing module comprises: extracting shallow layer feature x from feature extracting modulemWherein m is<k, the down sampling step is S, the partial stretching amplification branch is used for processing, and x is obtained through the branch output result CmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jWhether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown;
a transformation area acquisition module: for thresholding C using a threshold t, Ci,jIf the two amplified external rectangles have an overlapping part, the external rectangle before amplification with smaller area is removed, the filtered rectangle before amplification forms a region set A needing amplification, each element in the set is in xmA rectangle on the feature layer coordinate system represents that the area needs to be enlarged;
the transformation function building module: is used for calculating the feature map after the local amplification of the rectangular region and the original feature map x according to the size and the amplification factor of the rectangle in the set AmMapping relation f and inverse transformation f of feature point coordinates-1；
Local transformation and eyeA standard regression module: according to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2Continuously sending the convolution layer to obtain the position and the class information of the detection frame;
a result inverse transformation module: for calculating the detection box at xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
The specific implementation manner and the steps of each module correspond, and the invention is not described.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (2)

1. An unmanned aerial vehicle image small target detection method based on local adaptive geometric transformation is characterized by comprising the following steps:
1) construct feature extractor
Constructing a feature extractor with a self-adaptive local stretching amplification branch based on a deep full-convolution neural network;
2) extracting features
Performing feature extraction on an input image by using the deep fully-convolutional neural network feature extractor with the self-adaptive local stretching amplification branch constructed in the step 1), wherein the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the nth layer of feature extraction operation, xnIs passing through fnCalculating the obtained characteristics;
3) local stretching amplifying branch for processing
For the shallow feature x extracted in the step 2)mWherein m is<k, down sampling step length of SProcessing using local stretch amplifier branch, and outputting result C to obtain xmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jWhether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown;
4) obtaining the required transformation area
Thresholding C using a threshold t, Ci,jIf the two amplified external rectangles have an overlapping part, the external rectangle before amplification with smaller area is removed, the filtered rectangle before amplification forms a region set A needing amplification, each element in the set is in xmA rectangle on the feature layer coordinate system represents that the area needs to be enlarged;
5) constructing transformation functions
According to the size and the amplification factor of the rectangle in the set A, calculating a feature map after local amplification of the rectangular region and an original feature map xmMapping relation f and inverse transformation f of feature point coordinates-1；
6) Local transformation and object regression
According to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2Continuously sending the convolution layer to obtain the position and the class information of the detection frame;
7) result inverse transformation
Calculating the detection box at xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
2. The utility model provides an unmanned aerial vehicle image small-object detection system based on local adaptive geometric transformation which characterized in that includes following module:
the feature extractor construction module: the method is used for constructing a feature extractor with a self-adaptive local stretching amplification branch on the basis of a deep full convolution neural network;
a feature extraction module: the method is used for extracting the features of an input image by utilizing a deep full convolution neural network feature extractor with an adaptive local stretching amplification branch constructed in a feature extractor construction module, and the mathematical description is Xn+1＝fn(xn) Wherein n belongs to {0,1,. k-1}, k is the number of layers of the deep full convolution neural network, fnFor the nth layer of feature extraction operation, xnIs passing through fnCalculating the obtained characteristics;
the local stretching and amplifying branch processing module comprises: extracting shallow layer feature x from feature extracting modulemWherein m is<k, the down sampling step is S, the partial stretching amplification branch is used for processing, and x is obtained through the branch output result CmWhether each position of the layer characteristic needs to be subjected to stretching transformation, wherein Ci,jWhether the characteristic point with the row position of i and the column position of j needs to be amplified or not is shown;
a transformation area acquisition module: for thresholding C using a threshold t, Ci,jIf the two amplified external rectangles have an overlapping part, the external rectangle before amplification with smaller area is removed, the filtered rectangle before amplification forms a region set A needing amplification, each element in the set is in xmA rectangle on the feature layer coordinate system represents that the area needs to be enlarged;
the transformation function building module: is used for calculating the feature map after the local amplification of the rectangular region and the original feature map x according to the size and the amplification factor of the rectangle in the set AmMapping relation f and inverse transformation f of feature point coordinates-1；
Local transformation and target regression module:according to the mapping relation f to xmThe feature layer is locally transformed to obtain a transformed feature map xm2Then x is addedm2Continuously sending the convolution layer to obtain the position and the class information of the detection frame;
a result inverse transformation module: for calculating the detection box at xm2Coordinate information on feature map, xm2The position information on the feature map is inversely transformed by f-1Is obtained at xmAnd finally, calculating the detection result on the characteristic diagram to obtain the detection result under the input scale.
CN201910963413.2A 2019-10-11 2019-10-11 Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation Active CN110796028B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910963413.2A CN110796028B (en) 2019-10-11 2019-10-11 Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910963413.2A CN110796028B (en) 2019-10-11 2019-10-11 Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation

Publications (2)

Publication Number Publication Date
CN110796028A true CN110796028A (en) 2020-02-14
CN110796028B CN110796028B (en) 2021-08-17

Family

ID=69440227

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910963413.2A Active CN110796028B (en) 2019-10-11 2019-10-11 Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation

Country Status (1)

Country Link
CN (1) CN110796028B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104966054A (en) * 2015-06-11 2015-10-07 西安电子科技大学 Weak and small object detection method in visible image of unmanned plane
CN105389797A (en) * 2015-10-16 2016-03-09 西安电子科技大学 Unmanned aerial vehicle video small-object detecting method based on super-resolution reconstruction
CN109461164A (en) * 2018-09-21 2019-03-12 武汉大学 A kind of infrared small target detection method based on direction nuclear reconstitution
CN109472226A (en) * 2018-10-29 2019-03-15 上海交通大学 A kind of sleep behavioral value method based on deep learning
CN109740445A (en) * 2018-12-13 2019-05-10 天津津航技术物理研究所 A kind of method for detecting infrared puniness target becoming size

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104966054A (en) * 2015-06-11 2015-10-07 西安电子科技大学 Weak and small object detection method in visible image of unmanned plane
CN105389797A (en) * 2015-10-16 2016-03-09 西安电子科技大学 Unmanned aerial vehicle video small-object detecting method based on super-resolution reconstruction
CN109461164A (en) * 2018-09-21 2019-03-12 武汉大学 A kind of infrared small target detection method based on direction nuclear reconstitution
CN109472226A (en) * 2018-10-29 2019-03-15 上海交通大学 A kind of sleep behavioral value method based on deep learning
CN109740445A (en) * 2018-12-13 2019-05-10 天津津航技术物理研究所 A kind of method for detecting infrared puniness target becoming size

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MOISES LODEIRO-SANTIAGO 等: "Secure UAV-Based System to Detect Small Boats Using Neural Networks", 《HINDAWI》 *
王树梅: "数字图像中小目标自适应检测算法", 《计算机工程与应用》 *

Also Published As

Publication number Publication date
CN110796028B (en) 2021-08-17

Similar Documents

Publication Publication Date Title
CN112270249B (en) Target pose estimation method integrating RGB-D visual characteristics
CN110163271B (en) Panoramic image target detection method based on spherical projection grid and spherical convolution
CN110414507B (en) License plate recognition method and device, computer equipment and storage medium
US11734918B2 (en) Object identification apparatus, moving body system, object identification method, object identification model learning method, and object identification model learning apparatus
CN109145872B (en) CFAR and Fast-RCNN fusion-based SAR image ship target detection method
CN113192646B (en) Target detection model construction method and device for monitoring distance between different targets
CN112801158A (en) Deep learning small target detection method and device based on cascade fusion and attention mechanism
CN110097047B (en) Vehicle detection method based on deep learning and adopting single line laser radar
CN110659545B (en) Training method of vehicle identification model, vehicle identification method, device and vehicle
CN110110618B (en) SAR target detection method based on PCA and global contrast
CN116188999B (en) Small target detection method based on visible light and infrared image data fusion
CN111738033B (en) Vehicle driving information determination method and device based on plane segmentation and vehicle-mounted terminal
CN112446356B (en) Method for detecting text with arbitrary shape in natural scene based on multiple polar coordinates
CN106327522A (en) Infrared small target detection method based on multi-direction morphological filtering complex cloud background
CN106803262A (en) The method that car speed is independently resolved using binocular vision
CN106887012A (en) A kind of quick self-adapted multiscale target tracking based on circular matrix
CN111611956B (en) Rail detection method and system for subway visual image
CN112396036A (en) Method for re-identifying blocked pedestrians by combining space transformation network and multi-scale feature extraction
CN116597270A (en) Road damage target detection method based on attention mechanism integrated learning network
CN108509826B (en) Road identification method and system for remote sensing image
CN112446353B (en) Video image trace line detection method based on depth convolution neural network
CN110796028B (en) Unmanned aerial vehicle image small target detection method and system based on local adaptive geometric transformation
CN102129687B (en) Self-adapting target tracking method based on local background subtraction under dynamic scene
CN115984592A (en) Point-line fusion feature matching method based on SuperPoint + SuperGlue
CN115187959B (en) Method and system for landing flying vehicle in mountainous region based on binocular vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant