CN112348105A - Unmanned aerial vehicle image matching optimization method - Google Patents
Unmanned aerial vehicle image matching optimization method Download PDFInfo
- Publication number
- CN112348105A CN112348105A CN202011289613.3A CN202011289613A CN112348105A CN 112348105 A CN112348105 A CN 112348105A CN 202011289613 A CN202011289613 A CN 202011289613A CN 112348105 A CN112348105 A CN 112348105A
- Authority
- CN
- China
- Prior art keywords
- matrix
- image
- unmanned aerial
- aerial vehicle
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000005457 optimization Methods 0.000 title claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims abstract description 83
- 238000012216 screening Methods 0.000 claims abstract description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an unmanned aerial vehicle image matching optimization method, which comprises the following steps: detecting characteristic points of an unmanned aerial vehicle aerial image A and an unmanned aerial vehicle aerial image B to be matched by adopting a SURF algorithm, setting a Hessian matrix threshold of the SURF algorithm, and obtaining N pairs of matching points; traversing the matching points in the image A and the image B, screening out M pairs of matching points uniformly distributed in the image A and the image B, and constructing to obtain an H matrix; respectively solving H by using RANSAC algorithm and LMEDS algorithm1Matrix sum H2A matrix; to H1Matrix sum H2Evaluating the matrix to obtain an optimal matrix; and checking the optimal matrix to obtain the most reasonable H matrix. Through the scheme, the method has the advantages of simple logic, less calculation workload, accurate matching and the like, and has high practical value and popularization value in the technical field of image processing.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an unmanned aerial vehicle image matching optimization method.
Background
With the rapid development and popularization of multi-axis unmanned aerial vehicles, the unmanned aerial vehicle aerial photography technology is applied to more and more fields, and the matching and the identification of aerial images of the unmanned aerial vehicles also become research hotspots. Image matching and identification are comprehensive operation of various technologies, and are widely applied to a plurality of fields such as artificial intelligence, safety protection, unmanned aerial vehicle aerial photography, auxiliary driving, image remote sensing, computer vision and the like.
At present, in the prior art, the unmanned aerial vehicle image mostly adopts feature points for matching, for example, in the chinese patent with the patent application number "201810735162.8" and the name "a fast unmanned aerial vehicle image matching method based on local feature fusion", the method adopts: respectively carrying out 3-to-3 grid blocking on the reference image and the image to be matched, dividing one image into 9 sub-regions, and extracting invariant features in the sub-regions; extracting feature vectors from the invariant feature regions by using feature descriptors; judging initial homonymous features by comparing the similarity between the feature vectors to obtain stable initial matching; counting the number of matching points in each grid, and performing region MSERs feature matching on the region of which the number of matching points in the grid is less than a threshold value; and deleting the mismatching point pairs by using affine invariance of the Mahalanobis distance. The technique performs direct image matching by matching of feature points. In addition, in wang zhenhua "a matching method applied to an aerial image of an unmanned aerial vehicle", firstly, image data and GPS information transmitted by the unmanned aerial vehicle through 4G are received; secondly, reading corresponding geographic information characteristic point data in a database according to the GPS information; and finally, completing image matching by using the feature points. In order to accelerate the matching speed, the SURF characteristics are adopted to replace SIFT characteristics to complete operation.
The above-mentioned technology adopts the characteristic point to carry on the direct matching, and delete the mismatching point; since the matched feature points are random, uniformly distributed feature points cannot be obtained, so that the matched images have the problems of repetition and no matching. If only local feature points are used to calculate the H matrix, it is very easy to solve a more abnormal value, resulting in too large distortion of the image. When the unmanned aerial vehicle acquires the image, the coverage rate needs to reach 70% -80%, and the feature points are distributed in all places of the image.
Therefore, an unmanned aerial vehicle image matching optimization method with simple logic, less calculation workload and accurate matching is urgently needed to be provided.
Disclosure of Invention
Aiming at the problems, the invention aims to provide an unmanned aerial vehicle image matching optimization method, and the technical scheme adopted by the invention is as follows:
an unmanned aerial vehicle image matching optimization method comprises the following steps:
detecting characteristic points of an unmanned aerial vehicle aerial image A and an unmanned aerial vehicle aerial image B to be matched by adopting a SURF algorithm, setting a Hessian matrix threshold of the SURF algorithm, and obtaining N pairs of matching points; n is a natural number more than or equal to 4;
traversing the matching points in the image A and the image B, screening out M pairs of matching points uniformly distributed in the image A and the image B, and constructing to obtain an H matrix; m is less than or equal to N;
the expression of the H matrix is as follows:
in homogeneous coordinates h221, and respectively obtaining H by using RANSAC algorithm and LMEDS algorithm1Matrix sum H2A matrix;
to H1Matrix sum H2Evaluating the matrix to obtain an optimal matrix;
and checking the optimal matrix to obtain the most reasonable H matrix.
Further, the pair H1Matrix sum H2Evaluating the matrix to obtain an optimal matrix, comprising the following steps:
if H is1Matrix sum H2One of the matrices satisfies | h20+h21If | is greater than 0.0005, then eliminate pairsA corresponding matrix;
if H is1Matrix sum H2If none of the matrices are eliminated, then the matrix is selected according to | | h00|+|h01|+|h10|+|h11And l-2, selecting the matrix with the minimum value to obtain the optimal matrix.
Further, the checking the optimal matrix to obtain the most reasonable H matrix includes the following steps:
and (3) obtaining the length and width deformation value of any image according to the optimal matrix, wherein the expression is as follows:
P'i＝H*Pi
wherein, the four points of the original image of the image are homogenized with the coordinate P1(0,0,1),P2(w,0,1),P3(w,h,1),P4(0, h, 1); w represents the original image width of the image, and h represents the original image height of the image; the four transformed points are P'i(i＝1..4)；
And (3) obtaining the lengths of the four sides after the image transformation, wherein the lengths are expressed as:
d1＝||P'1-P'2||,d2＝||P'2-P'3||,d3＝||P'3-P'4||d4＝||P'4-P'1||
wherein, | | represents the Euclidean distance between two points;
and if the length change values of the four edges after the image transformation are more than 2, eliminating the H matrix to obtain the most reasonable H matrix.
Compared with the prior art, the invention has the following beneficial effects:
(1) the method comprises the steps of setting a Hessian matrix threshold parameter to obtain a number of pairs of matching points; traversing and selecting uniformly distributed feature matching points from the matched feature point pairs to ensure reliable matching;
(2) according to the method, the H matrix is skillfully calculated by adopting multiple algorithms, and the H matrix is evaluated and checked, wherein firstly, the algorithms adopted for calculating the H matrix are all iterative fitting and are not necessarily optimal solutions, so that the H matrix is calculated by adopting the multi-algorithm by adopting the idea of integrated learning, the optimization can be realized, and the accuracy of final matching is improved. In addition, if the H matrix solved by a single algorithm is used as projection transformation, large deformation can be generated, which obviously does not accord with the scene of the image acquired by the unmanned aerial vehicle and brings large errors to the subsequent unmanned aerial vehicle splicing; according to different application scenes, the H matrix is evaluated by adopting different evaluation means, so that a better H matrix can be screened conveniently.
In conclusion, the method has the advantages of simple logic, less calculation workload, accurate matching and the like, and has high practical value and popularization value in the technical field of image processing.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope of protection, and it is obvious for those skilled in the art that other related drawings can be obtained according to these drawings without inventive efforts.
FIG. 1 is a schematic diagram of feature point screening according to the present invention.
FIG. 2 is a schematic diagram of feature point screening according to the present invention (II).
Detailed Description
To further clarify the objects, technical solutions and advantages of the present application, the present invention will be further described with reference to the accompanying drawings and examples, and embodiments of the present invention include, but are not limited to, the following examples. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
As shown in fig. 1 to fig. 2, the present embodiment provides an unmanned aerial vehicle image matching optimization method, including the following steps:
firstly, matching by adopting a pyramid SURF method:
in this embodiment, the matching is performed on the graph a and the graph B, and first, stricter SURF feature point detection is adopted, in which the Hessian matrix threshold parameter is set to 1200. If the number of feature points is less, more relaxed SURF feature point detection is adopted, and the Hessian matrix threshold parameter is gradually reduced, for example, set to 800, 400. In the present embodiment, there are a total of N pairs of matching points. Pts1 represents the set of feature points of diagram a, and Pts2 represents the set of feature points of diagram B. If N ≧ 4, continue. And if N is less than 4, the Heisson threshold is reduced for re-matching.
And secondly, screening M pairs of matching points to ensure that the characteristic points are uniformly distributed on the image: dividing the image into 16 by 16 regions; each time the regions are traversed, matching points are taken out, and if the regions have no points, skipping is carried out; until M points are filled. In this embodiment, M is 50, and if N < M, then all N are selected.
Thirdly, calculating an H matrix by multiple algorithms:
wherein, the expression of the H matrix is:
in homogeneous coordinates h221, and respectively obtaining H by using RANSAC algorithm and LMEDS algorithm1Matrix sum H2And (4) matrix.
And fourthly, evaluating the H matrix, and screening the optimal H:
(1) magnitude of shear deformation
If | H of a certain H matrix20+h21If the | is more than 0.0005, the method is eliminated;
(2) torsional deformation
If H is1Matrix sum H2If none of the matrices are eliminated, then the matrix is selected according to | | h00|+|h01|+|h10|+|h11And l-2, selecting the matrix with the minimum value to obtain the optimal matrix.
Fifthly, checking the H matrix:
(1) and (3) obtaining the length and width deformation value of any image according to the optimal matrix, wherein the expression is as follows:
P'i＝H*Pi
wherein, the original image of the image is leveled in four pointsCoordinate P1(0,0,1),P2(w,0,1),P3(w,h,1),P4(0, h, 1); w represents the original image width of the image, and h represents the original image height of the image; the four transformed points are P'i(i＝1..4)；
(2) And (3) obtaining the lengths of the four sides after the image transformation, wherein the lengths are expressed as:
d1＝||P'1-P'2||,d2＝||P'2-P'3||,d3＝||P'3-P'4||,d4＝||P'4-P'1||
wherein, | | represents the Euclidean distance between two points;
(3) and if the length change values of the four edges after the image transformation are more than 2, eliminating the H matrix to obtain the most reasonable H matrix. Namely, it is
d1W > 2 or d1If the/w is less than 0.5, the materials are eliminated;
d2h > 2 or d2/h<0.5 is eliminated;
d3w > 2 or d3If the/w is less than 0.5, the materials are eliminated;
d4h > 2 or d4The reaction is eliminated when the/h is less than 0.5;
in this embodiment, the most reasonable H matrix is finally obtained through the first step to the fifth step.
The above-mentioned embodiments are only preferred embodiments of the present invention, and do not limit the scope of the present invention, but all the modifications made by the principles of the present invention and the non-inventive efforts based on the above-mentioned embodiments shall fall within the scope of the present invention.
Claims (3)
1. An unmanned aerial vehicle image matching optimization method is characterized by comprising the following steps:
detecting characteristic points of an unmanned aerial vehicle aerial image A and an unmanned aerial vehicle aerial image B to be matched by adopting a SURF algorithm, setting a Hessian matrix threshold of the SURF algorithm, and obtaining N pairs of matching points; n is a natural number more than or equal to 4;
traversing the matching points in the image A and the image B, screening out M pairs of matching points uniformly distributed in the image A and the image B, and constructing to obtain an H matrix; m is less than or equal to N;
the expression of the H matrix is as follows:
in homogeneous coordinates h221, and respectively obtaining H by using RANSAC algorithm and LMEDS algorithm1Matrix sum H2A matrix;
to H1Matrix sum H2Evaluating the matrix to obtain an optimal matrix;
and checking the optimal matrix to obtain the most reasonable H matrix.
2. The unmanned aerial vehicle image matching optimization method of claim 1, wherein the pair H is1Matrix sum H2Evaluating the matrix to obtain an optimal matrix, comprising the following steps:
if H is1Matrix sum H2One of the matrices satisfies | h20+h21If the | is more than 0.0005, the corresponding matrix is eliminated;
if H is1Matrix sum H2If none of the matrices are eliminated, then the matrix is selected according to | | h00|+|h01|+|h10|+|h11And l-2, selecting the matrix with the minimum value to obtain the optimal matrix.
3. The unmanned aerial vehicle image matching optimization method of claim 1, wherein the checking the optimal matrix to obtain the most reasonable H matrix comprises the following steps:
and (3) obtaining the length and width deformation value of any image according to the optimal matrix, wherein the expression is as follows:
Pi'＝H*Pi
wherein, the four points of the original image of the image are homogenized with the coordinate P1(0,0,1),P2(w,0,1),P3(w,h,1),P4(0, h, 1); w denotes original imageWidth, h represents the original image height of the image; the four transformed points are Pi'(i＝1..4)；
And (3) obtaining the lengths of the four sides after the image transformation, wherein the lengths are expressed as:
d1＝||P′1-P′2||，d2＝||P′2-P′3||，d3＝||P′3-P′4||，d4＝||P′4-P′1||
wherein, | | represents the Euclidean distance between two points;
and if the length change values of the four edges after the image transformation are more than 2, eliminating the H matrix to obtain the most reasonable H matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011289613.3A CN112348105B (en) | 2020-11-17 | 2020-11-17 | Unmanned aerial vehicle image matching optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011289613.3A CN112348105B (en) | 2020-11-17 | 2020-11-17 | Unmanned aerial vehicle image matching optimization method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112348105A true CN112348105A (en) | 2021-02-09 |
| CN112348105B CN112348105B (en) | 2023-09-01 |
Family
ID=74364041
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011289613.3A Active CN112348105B (en) | 2020-11-17 | 2020-11-17 | Unmanned aerial vehicle image matching optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348105B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104574421A (en) * | 2015-01-29 | 2015-04-29 | 北方工业大学 | Large-breadth small-overlapping-area high-precision multispectral image registration method and device |
| CN106940876A (en) * | 2017-02-21 | 2017-07-11 | 华东师范大学 | A kind of quick unmanned plane merging algorithm for images based on SURF |
| CN108961162A (en) * | 2018-03-12 | 2018-12-07 | 北京林业大学 | A kind of unmanned plane forest zone Aerial Images joining method and system |
| CN109697692A (en) * | 2018-12-29 | 2019-04-30 | 安徽大学 | One kind being based on the similar feature matching method of partial structurtes |
| CN110458183A (en) * | 2019-06-25 | 2019-11-15 | 上海圭目机器人有限公司 | A kind of characteristic matching optimization algorithm of image adaptive |
| WO2020199424A1 (en) * | 2019-04-01 | 2020-10-08 | 苏州中晟宏芯信息科技有限公司 | Optimal h-matrix generation method and device |
-
2020
- 2020-11-17 CN CN202011289613.3A patent/CN112348105B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104574421A (en) * | 2015-01-29 | 2015-04-29 | 北方工业大学 | Large-breadth small-overlapping-area high-precision multispectral image registration method and device |
| CN106940876A (en) * | 2017-02-21 | 2017-07-11 | 华东师范大学 | A kind of quick unmanned plane merging algorithm for images based on SURF |
| CN108961162A (en) * | 2018-03-12 | 2018-12-07 | 北京林业大学 | A kind of unmanned plane forest zone Aerial Images joining method and system |
| CN109697692A (en) * | 2018-12-29 | 2019-04-30 | 安徽大学 | One kind being based on the similar feature matching method of partial structurtes |
| WO2020199424A1 (en) * | 2019-04-01 | 2020-10-08 | 苏州中晟宏芯信息科技有限公司 | Optimal h-matrix generation method and device |
| CN110458183A (en) * | 2019-06-25 | 2019-11-15 | 上海圭目机器人有限公司 | A kind of characteristic matching optimization algorithm of image adaptive |
Non-Patent Citations (2)
| Title |
|---|
| 张春雨等: "视频拼接中最优自适应单应性矩阵求解算法", 吉林大学学报(工学版), no. 04 * |
| 王晓红等: "结合SURF算法和单应性矩阵的无人机影像匹配", 测绘通报, no. 07 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112348105B (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | A real-time detection approach for bridge cracks based on YOLOv4-FPM | |
| Zhao et al. | Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications | |
| CN111028277B (en) | SAR and optical remote sensing image registration method based on pseudo-twin convolution neural network | |
| Xie et al. | Multilevel cloud detection in remote sensing images based on deep learning | |
| CN108960211B (en) | Multi-target human body posture detection method and system | |
| US8885943B2 (en) | Face detection method and apparatus | |
| CN111507901B (en) | Aerial image splicing and positioning method based on aerial GPS and scale invariant constraint | |
| CN110689043A (en) | Vehicle fine granularity identification method and device based on multiple attention mechanism | |
| CN113947766B (en) | Real-time license plate detection method based on convolutional neural network | |
| CN112733885A (en) | Point cloud identification model determining method and point cloud identification method and device | |
| Qu et al. | Image seamless stitching and straightening based on the image block | |
| CN116188999B (en) | Small target detection method based on visible light and infrared image data fusion | |
| Zhou et al. | Nonrigid feature matching for remote sensing images via probabilistic inference with global and local regularizations | |
| CN110084743B (en) | Image splicing and positioning method based on multi-flight-zone initial flight path constraint | |
| CN112883850A (en) | Multi-view aerospace remote sensing image matching method based on convolutional neural network | |
| CN110704652A (en) | Vehicle image fine-grained retrieval method and device based on multiple attention mechanism | |
| US20200005078A1 (en) | Content aware forensic detection of image manipulations | |
| CN116246119A (en) | 3D target detection method, electronic device and storage medium | |
| Kong et al. | Local Stereo Matching Using Adaptive Cross‐Region‐Based Guided Image Filtering with Orthogonal Weights | |
| CN115063454A (en) | Multi-target tracking matching method, device, terminal and storage medium | |
| Luo et al. | Dynamic multitarget detection algorithm of voxel point cloud fusion based on pointrcnn | |
| Sriram et al. | Analytical review and study on object detection techniques in the image | |
| CN114595352A (en) | Image identification method and device, electronic equipment and readable storage medium | |
| WO2023222643A1 (en) | Method for image segmentation matching | |
| CN112348105A (en) | Unmanned aerial vehicle image matching optimization method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |