CN112465025A - Image feature matching method and system based on neighborhood inference - Google Patents
Image feature matching method and system based on neighborhood inference Download PDFInfo
- Publication number
- CN112465025A CN112465025A CN202011353954.2A CN202011353954A CN112465025A CN 112465025 A CN112465025 A CN 112465025A CN 202011353954 A CN202011353954 A CN 202011353954A CN 112465025 A CN112465025 A CN 112465025A
- Authority
- CN
- China
- Prior art keywords
- matching
- feature
- feature point
- preliminary
- point set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000001514 detection method Methods 0.000 claims description 11
- 238000010276 construction Methods 0.000 claims description 8
- 238000010845 search algorithm Methods 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000000969 carrier Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000004643 material aging Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
- G06F18/295—Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image feature matching method and system based on neighborhood inference, which can be used for: constructing a redundant feature point set R of a reference image; constructing a feature point set T of the target image; acquiring a candidate matching point set of each feature point in the feature point set T in a redundant feature point set R of the reference image; performing feature matching on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs; and eliminating the error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image and the target image. The method is used for increasing the number of the characteristic points for characteristic matching, improving the matching precision of the image characteristic points and reducing the mismatching rate.
Description
Technical Field
The invention relates to the field of image processing, in particular to an image feature matching method and system based on neighborhood inference.
Background
The cultural relics are the witnesses and carriers of human material culture and spiritual culture, but the cultural relics gradually disappear due to natural disasters, artificial damage, material aging and the like, and the related technologies of protection and repair are urgently needed.
The multi-view three-dimensional reconstruction technology based on feature point matching relies on a motion inference scheme to carry out three-dimensional reconstruction on cultural relics, and the implementation of the technology is to carry out feature tracking (feature detection and feature matching) on an acquired multi-view image sequence, recover the motion track of a camera and the geometric structure of a real cultural relic and describe the appearance of the cultural relic. The technology can provide support for the preservation of the global appearance of the cultural relics and the local patterns of the cultural relics, and has important engineering application value in the aspect of cultural relic protection.
At present, a feature matching method for a multi-view three-dimensional reconstruction technology often uses traditional SIFT, GMS and other methods for detection and matching, but because of a lot of factors interfering with the reconstruction effect in reality, an input image is affected by light shadow, lack of texture, occlusion and the like, resulting in the following disadvantages in traditional feature matching (causing the accuracy and integrity of a subsequently reconstructed three-dimensional model to be not guaranteed):
1. the number of detected feature points is relatively small;
2. the matching precision of the characteristic points is relatively low;
3. the mismatch rate is also relatively high.
Therefore, the invention provides an image feature matching method and system based on neighborhood inference, which are used for solving the problems.
Disclosure of Invention
In view of the above disadvantages of the prior art, the present invention provides a neighborhood inference based image feature matching method and system, which are used to increase the number of feature points for feature matching, improve the matching accuracy of image feature points, and reduce the mismatching rate.
In a first aspect, the present invention provides a neighborhood inference based image feature matching method, including the steps of:
constructing a redundant feature point set R of a reference image;
constructing a feature point set T of the target image; the target image and the reference image are two read-in images to be subjected to feature matching;
acquiring a candidate matching point set of each feature point in the feature point set T in a redundant feature point set R of the reference image;
performing feature matching on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs;
and eliminating the error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image and the target image.
Further, the method for constructing the redundant feature point set R of the reference image comprises:
acquiring preset deformation images of different scale spaces of a reference image;
detecting the reference image and the feature points on each acquired deformation image by adopting a first feature point detector;
collecting all the characteristic points detected by the first characteristic point detector to form the redundant characteristic point set R;
the method for constructing the feature point set T of the target image comprises the following steps: and detecting the characteristic points on the target image by adopting a second characteristic point detector, and collecting the detected characteristic points on the target image to form the characteristic point set T.
Further, a candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image is obtained, and the implementation method is as follows: and acquiring a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by adopting an approximate nearest neighbor search algorithm.
Further, the feature matching is performed on each candidate matching point in all the obtained candidate matching point sets based on the spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs, and the implementation method is as follows:
constructing a triangular subdivision graph based on all feature points in the feature point set T to obtain a spatial neighborhood graph corresponding to the feature point set T;
and in the obtained spatial neighborhood map, performing feature matching on each candidate matching point in all the obtained candidate matching point sets to obtain a group of matching feature point pairs, wherein the group of matching feature point pairs are the group of preliminary matching feature point pairs.
Further, rejecting the error matching in the preliminary matching feature point pair to obtain a feature matching point pair of the reference image and the target image, wherein the implementation method comprises the following steps:
judging each obtained preliminary matching characteristic point pair (V)s，Ks) Whether the matching is correct or not is judged, and all the preliminary matching feature point pairs judged to be correct are obtained, so that feature matching point pairs of the reference image and the target image are obtained; said (V)s，Ks) For the s-th pair of feature points of all the obtained preliminary matching pairs of feature points, wherein VsAs feature points on the reference image, KsThe feature points of the target image are s 1,2, …, and L is the total number of all the obtained preliminary matching feature point pairs;
wherein each of the obtained preliminary matching feature point pairs (V) is judgeds，Ks) The method for realizing whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
In a second aspect, the present invention provides a neighborhood inference based image feature matching system, comprising:
the target image reading unit is used for reading two images to be subjected to feature matching: a target image and a reference image;
a redundant feature point set construction unit for constructing a redundant feature point set R of the reference image;
a feature point set construction unit for constructing a feature point set T of the target image;
the candidate matching point set acquisition unit is used for acquiring a candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image;
a preliminary matching feature point pair obtaining unit, configured to perform feature matching on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs;
and the error matching rejection unit is used for rejecting error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image and the target image.
Further, the redundant feature point set constructing unit includes:
the deformation image acquisition module is used for acquiring deformation images of preset different scale spaces of the reference image;
the first feature detection module is used for detecting the feature points on the reference image and each acquired deformation image by adopting a first feature point detector;
the redundant feature point set forming module is used for collecting all the feature points detected by the first feature detection module to form the redundant feature point set R;
the feature point set constructing unit includes:
the second feature detection module is used for detecting feature points on the target image by adopting a second feature point detector;
and the characteristic point set constructing module is used for collecting the characteristic points on the target image detected by the second characteristic detecting module to form the characteristic point set T.
Further, the candidate matching point set obtaining unit obtains a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by using an approximate nearest neighbor search algorithm.
Further, the preliminary matching feature point pair obtaining unit includes:
the spatial neighborhood map building module is used for building a triangular subdivision map based on all the feature points in the feature point set T to obtain a spatial neighborhood map corresponding to the feature point set T;
and the preliminary matching module is used for performing feature matching on each candidate matching point in all the obtained candidate matching point sets in the space neighborhood map obtained by the space neighborhood map construction module to obtain a group of matched feature point pairs, wherein the group of matched feature point pairs are the group of preliminary matched feature point pairs.
Further, the mismatch culling unit includes:
a correct matching judgment module for judging each preliminary matching characteristic point pair (V) obtained by the preliminary matching characteristic point pair obtaining units，Ks) If it is a correct match, s ═ 1,2, …, L; wherein: (V)s，Ks) For the s-th characteristic point pair, V, of all the preliminary matching characteristic point pairs obtained by the preliminary matching characteristic point pair obtaining unitsAs feature points on the reference image, KsL is the total number of all the preliminary matching characteristic point pairs obtained by the preliminary matching characteristic point pair obtaining unit;
the characteristic matching point pair obtaining module is used for obtaining all the preliminary matching characteristic point pairs which are judged to be correctly matched by the correct matching judging module to obtain characteristic matching point pairs of the reference image and the target image;
wherein the correct matching judgment module judges each preliminary matching characteristic point pair (V) obtained by the preliminary matching characteristic point pair obtaining units，Ks) The method for judging whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
The beneficial effect of the invention is that,
(1) according to the image feature matching method and system based on neighborhood inference provided by the invention, redundant feature point sets R of the reference image are constructed for feature matching of the reference image and the target image, which is beneficial to increasing the quantity/number of feature points for feature matching of the reference image and the target image to a certain extent.
(2) The image feature matching method and system based on neighborhood inference provided by the invention can acquire the candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image, perform feature matching on each candidate matching point in the acquired candidate matching point sets based on the spatial neighborhood map of the feature point set T to acquire a group of preliminary matching feature point pairs, and then acquire the feature matching point pairs of the reference image and the target image after eliminating the error matching in the preliminary matching feature point pairs, which is beneficial to improving the matching precision of the image feature points and reducing the error matching rate to a certain extent.
(3) Compared with the GMS method, the method does not need to divide grids for the reference image and the target image, and directly operates based on the peripheral circular area (defined by the radius threshold value) of the characteristic points, thereby being beneficial to eliminating the error matching, therefore, the matching precision of the image feature points is improved, the mismatching rate is reduced, and the method is more convenient to realize.
In addition, the invention has reliable design principle, simple structure and very wide application prospect.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a method of one embodiment of the invention.
Fig. 2 is a schematic block diagram of a system according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
FIG. 1 is a schematic flow chart diagram of an image feature matching method based on neighborhood inference according to one embodiment of the invention.
As shown in fig. 1, the method 100 includes:
and 150, eliminating error matching in the preliminary matching feature point pairs to obtain feature matching point pairs of the reference image and the target image.
Optionally, as an embodiment of the present invention, the method for constructing the redundant feature point set R of the reference image includes:
acquiring preset deformation images of different scale spaces of a reference image;
detecting the reference image and the feature points on each acquired deformation image by adopting a first feature point detector;
and collecting all the characteristic points detected by the first characteristic point detector to form the redundant characteristic point set R.
Optionally, as an embodiment of the present invention, the method for constructing the feature point set T of the target image includes: and detecting the characteristic points on the target image by adopting a second characteristic point detector, and collecting the detected characteristic points on the target image to form the characteristic point set T.
Preferably, both the first feature point detector and the second feature point detector use surf (speed Up Robust feature) feature point detectors. In a specific implementation, the SURF feature point detector may be replaced by another feature point detector.
Optionally, as an embodiment of the present invention, the implementation method of step 130 is: and acquiring a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by adopting an approximate nearest neighbor search algorithm.
Optionally, as an embodiment of the present invention, the implementation method of step 140 is: constructing a triangular subdivision graph based on all feature points in the feature point set T to obtain a spatial neighborhood graph corresponding to the feature point set T;
and in the obtained spatial neighborhood map, performing feature matching on each candidate matching point in all the obtained candidate matching point sets to obtain a group of matching feature point pairs, wherein the group of matching feature point pairs are the group of preliminary matching feature point pairs.
Optionally, as an embodiment of the present invention, the implementation method of step 150 is:
judging each obtained preliminary matching characteristic point pair (V)s，Ks) Whether the matching is correct or not is judged, and all the preliminary matching feature point pairs judged to be correct are obtained, so that feature matching point pairs of the reference image and the target image are obtained; said (V)s，Ks) For the s-th feature point pair of all the preliminary matching feature point pairs obtained in step 140, where VsAs feature points on the reference image, KsThe feature points of the target image are s ═ 1,2, …, and L is the total number of all pairs of preliminary matching feature points obtained in step 140;
wherein each of the obtained preliminary matching feature point pairs (V) is judgeds，Ks) The method for realizing whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching characteristic point pairs by using all preliminary matching characteristic point pairs in a region (circular region) with a preset radius threshold phi as a circle center and a radius;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
In order to facilitate understanding of the present invention, the image feature matching method based on neighborhood inference provided by the present invention is further described below by using the principle of the image feature matching method based on neighborhood inference of the present invention and combining the process of performing neighborhood inference based image feature matching on the reference image a and the target image B in the embodiment.
The reference image a and the target image B are two images to be subjected to feature matching read in advance.
Specifically, the image feature matching method based on neighborhood inference comprises the following steps:
step E1: a redundant feature point set R of the reference image a is constructed.
In this embodiment, a redundant feature point set of the reference image a is constructed by the simulated deformation of the image, and the number of feature points used for image feature matching is increased to a certain extent.
In this embodiment, in step E1, the method for constructing the redundant feature point set R of the reference image a includes:
step E11: and acquiring deformation images of the reference image A in preset different scale spaces.
Specifically, in the present embodiment, the deformed images obtained by rotating the reference image a by 45 degrees, 90 degrees, 135 degrees, 180 degrees, 125 degrees, and 270 degrees are acquired, and the deformed image 1, the deformed image 2, the deformed image 3, the deformed image 4, the deformed image 5, and the deformed image 6 of the reference image a are acquired in this order.
Step E12: and detecting the characteristic points on the reference image A and each acquired deformation image by adopting a first characteristic point detector.
Specifically, with the SURF feature point detector, feature points on the reference image a, the deformed image 1, the deformed image 2, the deformed image 3, the deformed image 4, the deformed image 5, and the deformed image 6 are detected, respectively.
Step E13: and collecting all the characteristic points detected by the first characteristic point detector to form the redundant characteristic point set R.
Specifically, all the feature points detected by the SURF feature point detector in step E12 are collected to form the redundant feature point set R.
Step E2: and constructing a feature point set T of the target image B.
Specifically, a second feature point detector is adopted to detect feature points on the target image B, and all the detected feature points on the target image B are collected to form the feature point set T.
In the present embodiment, the second feature point detector also employs a SURF feature point detector.
Step E3: and acquiring a candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image A.
Note that the feature point set T ═ T formed in step E2 is set to { T ═ T }1，t2，…，tn}＝{tiI is 1,2, …, n is the total number of characteristic points in the characteristic point set T, T isiIs the ith feature point in the feature point set T.
Note that the redundant feature point set R ═ R constructed in step E1 above is set to { R ═ R }1，r2，…，rm}＝{rjI j is 1,2, …, m is the total number of feature points in the redundant feature point set R, RjIs the jth feature point in the redundant feature point set R.
Specifically, the implementation step of the step E3 includes:
(1) each feature point T in the pre-defined feature point set TiIs C (t)i)，
Is tiI ═ 1,2, …, n;
(2) each feature point T in the feature point set T is obtained by operating an approximate nearest neighbor search algorithm (namely, a nearest neighbor search method) on the redundant feature point set RiConstructing a corresponding candidate matching point set C (t) from all candidate matching points in the redundant feature point set R of the reference image Ai)，i＝1，2，…，n。
It can be seen that through step E3, the feature point T in the feature point set T is obtainediCandidate matching point set C (t) in redundant feature point set R of reference image Ai)，i＝1，2，…，n。
Step E4: and performing feature matching on each candidate matching point in all the obtained candidate matching point sets based on the spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs.
In this embodiment, the step E4 adopts a second-order invariant graph constraint algorithm to match all the candidate matching point sets C (t) acquired in the step E3i) Each candidate matching point in (i ═ 1,2, …, n) is subjected to feature matching, so as to obtain a group of preliminary matching feature point pairs, and the specific implementation method may include:
based on all the feature points in the feature point set T (i.e. T)1，t2，…，tn) Constructing a triangular subdivision map to obtain a spatial neighborhood map (namely the constructed triangular subdivision map) corresponding to the characteristic point set T;
in the obtained space neighborhood graph, a second-order MRF (Markov random field model) matching constraint algorithm is adopted to carry out matching on all the obtained candidate matching point sets C (t)i) And (i-1, 2, …, n) performing feature matching on each candidate matching point to obtain a set of matching feature point pairs, wherein the set of matching feature point pairs is the set of preliminary matching feature point pairs.
Step E5: and eliminating the error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image A and the target image B.
L preliminary matching feature point pairs are obtained in step E4.
The step E5 is implemented by the following method:
step 1), judging each obtained preliminary matching characteristic point pair (V)s，Ks) (s-1, 2, …, L) is a correct match.
Said (V)s，Ks) For the s-th feature point pair of all the preliminary matching feature point pairs obtained in step E4, where VsCorresponding to the feature point, K, on the reference image AsIs the feature point of the target image B.
Wherein each of the obtained preliminary matching feature point pairs (V) is judgeds，Ks) The method for realizing whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
For ease of understanding, the following is a preliminary matching feature point pair (V)2，K2) For example, for each of the preliminary matching feature point pairs (V) obtained by the above determinations，Ks) Whether the matching is a correct matching implementation method is explained.
Specifically, in the present embodiment, the preliminary matching feature point pair (V) is determined2，K2) The method for realizing whether the matching is correct or not comprises the following steps:
collect characteristic points V2Forming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points K2Forming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
determining whether the statistical number reaches a preset number threshold Q (which may be set by a person skilled in the art based on experience, for example, the number threshold Q is set to 5. in this embodiment), and if so, determining a preliminary matching feature point pair (V)2，K2) If the matching is correct, otherwise, judging the characteristic point pair (V)2，K2) Is a false match.
In the present embodiment, the method for determining whether the other preliminary matching feature point pairs are correctly matched may refer to the feature point pair (V)2，K2)。
And step 2), acquiring all the preliminary matching feature point pairs which are judged to be correctly matched in the step 1), and obtaining the feature matching point pairs of the reference image A and the target image B.
All the preliminary matching feature point pairs determined to be correctly matched in the step 1) acquired in the step 2) are feature matching point pairs of the obtained reference image A and the target image B. The realization is convenient.
In this embodiment, the size of the radius threshold φ is 15 pixels.
The method is different from the GMS method in that the method does not need to divide grids for the reference image and the target image, and directly operates based on the peripheral circular area (defined by the radius threshold phi) of the characteristic points, so that not only can the mismatching be eliminated, but also the method is more convenient to realize.
FIG. 2 is a diagram of an embodiment of a neighborhood inference based image feature matching system according to the invention.
As shown in fig. 2, the system 200 includes:
a target image reading unit 206 for reading in two images to be subjected to feature matching: a target image and a reference image;
a redundant feature point set constructing unit 201 for constructing a redundant feature point set R of a reference image;
a feature point set constructing unit 202 configured to construct a feature point set T of the target image;
a candidate matching point set obtaining unit 203, configured to obtain a candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image;
a preliminary matching feature point pair obtaining unit 204, configured to perform feature matching on each candidate matching point in all the obtained candidate matching point sets based on the spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs;
an error matching rejecting unit 205, configured to reject error matching in the preliminary matching feature point pairs, to obtain feature matching point pairs of the reference image and the target image.
Optionally, as an embodiment of the present invention, the redundant feature point set constructing unit 201 includes:
the deformation image acquisition module is used for acquiring deformation images of preset different scale spaces of the reference image;
the first feature detection module is used for detecting the feature points on the reference image and each acquired deformation image by adopting a first feature point detector;
and the redundant feature point set forming module is used for collecting all the feature points detected by the first feature detection module to form the redundant feature point set R.
Optionally, as an embodiment of the present invention, the feature point set constructing unit 202 includes:
the second feature detection module is used for detecting feature points on the target image by adopting a second feature point detector;
and the characteristic point set constructing module is used for collecting the characteristic points on the target image detected by the second characteristic detecting module to form the characteristic point set T.
Optionally, as an embodiment of the present invention, the candidate matching point set obtaining unit 203 obtains a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by using an approximate nearest neighbor search algorithm.
Optionally, as an embodiment of the present invention, the preliminary matching feature point pair obtaining unit 204 includes:
the spatial neighborhood map building module is used for building a triangular subdivision map based on all the feature points in the feature point set T to obtain a spatial neighborhood map corresponding to the feature point set T;
and the preliminary matching module is used for performing feature matching on each candidate matching point in all the obtained candidate matching point sets in the space neighborhood map obtained by the space neighborhood map construction module to obtain a group of matched feature point pairs, wherein the group of matched feature point pairs are the group of preliminary matched feature point pairs.
Optionally, as an embodiment of the present invention, the mismatch culling unit 205 includes:
a correct matching judgment module, configured to judge each preliminary matching feature point pair (V) obtained by the preliminary matching feature point pair obtaining unit 204s，Ks) If it is a correct match, s ═ 1,2, …, L; wherein: (V)s，Ks) For the s-th characteristic point pair, V, of all the preliminary matching characteristic point pairs obtained by the preliminary matching characteristic point pair obtaining unit 204sAs feature points on the reference image, KsL is the total number of all preliminary matching feature point pairs obtained by the preliminary matching feature point pair obtaining unit 204;
the characteristic matching point pair obtaining module is used for obtaining all the preliminary matching characteristic point pairs which are judged to be correctly matched by the correct matching judging module to obtain characteristic matching point pairs of the reference image and the target image;
wherein the correct matching judgment module judges each preliminary matching feature point pair (V) obtained by the preliminary matching feature point pair obtaining unit 204s，Ks) The method for judging whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
The same and similar parts in the various embodiments in this specification may be referred to each other. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the description in the method embodiment.
It should be noted that, in this specification, the "reference image" and the "target image" are two input images to be subjected to feature matching, where the "reference" and the "target" are only used for distinguishing the two images to be subjected to feature matching, and in a specific implementation, a person skilled in the art may designate any one of the two images to be subjected to feature matching as the "reference image" and designate the other one as the "target image" according to actual needs, and may also replace the "reference image" and the "target image" with the "first image" and the "second image".
Although the present invention has been described in detail by referring to the drawings in connection with the preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made on the embodiments of the present invention by those skilled in the art without departing from the spirit and scope of the present invention, and these modifications or substitutions are within the scope of the present invention/any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An image feature matching method based on neighborhood inference, which is characterized by comprising the following steps:
constructing a redundant feature point set R of a reference image;
constructing a feature point set T of the target image; the target image and the reference image are two read-in images to be subjected to feature matching;
acquiring a candidate matching point set of each feature point in the feature point set T in a redundant feature point set R of the reference image;
performing feature matching on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs;
and eliminating the error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image and the target image.
2. The neighborhood inference based image feature matching method of claim 1,
the method for constructing the redundant feature point set R of the reference image comprises the following steps:
acquiring preset deformation images of different scale spaces of a reference image;
detecting the reference image and the feature points on each acquired deformation image by adopting a first feature point detector;
collecting all the characteristic points detected by the first characteristic point detector to form the redundant characteristic point set R;
the method for constructing the feature point set T of the target image comprises the following steps: and detecting the characteristic points on the target image by adopting a second characteristic point detector, and collecting the detected characteristic points on the target image to form the characteristic point set T.
3. The image feature matching method based on neighborhood inference as claimed in claim 1, wherein a candidate matching point set of each feature point in a feature point set T in a redundant feature point set R of a reference image is obtained by: and acquiring a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by adopting an approximate nearest neighbor search algorithm.
4. The neighborhood inference-based image feature matching method according to claim 1, wherein feature matching is performed on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of a feature point set T to obtain a set of preliminary matching feature point pairs, and the implementation method is as follows:
constructing a triangular subdivision graph based on all feature points in the feature point set T to obtain a spatial neighborhood graph corresponding to the feature point set T;
and in the obtained spatial neighborhood map, performing feature matching on each candidate matching point in all the obtained candidate matching point sets to obtain a group of matching feature point pairs, wherein the group of matching feature point pairs are the group of preliminary matching feature point pairs.
5. The neighborhood inference-based image feature matching method according to claim 1, wherein the false matching in the preliminary matching feature point pairs is eliminated to obtain the feature matching point pairs of the reference image and the target image, and the implementation method is as follows:
judging each obtained preliminary matching characteristic point pair (V)s，Ks) Whether the matching is correct or not is judged, and all the preliminary matching feature point pairs judged to be correct are obtained, so that feature matching point pairs of the reference image and the target image are obtained; said (V)s，Ks) For the s-th feature point pair of all the preliminary matching feature point pairs, where VsAs feature points on the reference image, KsThe method comprises the steps of obtaining characteristic points of a target image, wherein s is 1,2, and L is the total number of all obtained preliminary matching characteristic point pairs;
wherein each of the obtained preliminary matching feature point pairs (V) is judgeds，Ks) Is thatThe method for realizing the correct matching comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
6. An image feature matching system based on neighborhood inference, comprising:
the target image reading unit is used for reading two images to be subjected to feature matching: a target image and a reference image;
a redundant feature point set construction unit for constructing a redundant feature point set R of the reference image;
a feature point set construction unit for constructing a feature point set T of the target image;
the candidate matching point set acquisition unit is used for acquiring a candidate matching point set of each feature point in the feature point set T in the redundant feature point set R of the reference image;
a preliminary matching feature point pair obtaining unit, configured to perform feature matching on each candidate matching point in all the obtained candidate matching point sets based on a spatial neighborhood map of the feature point set T to obtain a group of preliminary matching feature point pairs;
and the error matching rejection unit is used for rejecting error matching in the preliminary matching feature point pair to obtain the feature matching point pair of the reference image and the target image.
7. The neighborhood inference based image feature matching system of claim 6, wherein the redundant feature point set construction unit comprises:
the deformation image acquisition module is used for acquiring deformation images of preset different scale spaces of the reference image;
the first feature detection module is used for detecting the feature points on the reference image and each acquired deformation image by adopting a first feature point detector;
the redundant feature point set forming module is used for collecting all the feature points detected by the first feature detection module to form the redundant feature point set R;
the feature point set constructing unit includes:
the second feature detection module is used for detecting feature points on the target image by adopting a second feature point detector;
and the characteristic point set constructing module is used for collecting the characteristic points on the target image detected by the second characteristic detecting module to form the characteristic point set T.
8. The neighborhood inference based image feature matching system according to claim 6, wherein said candidate matching point set obtaining unit obtains a candidate matching point set of each feature point in the feature point set T on the redundant feature point set R by using an approximate nearest neighbor search algorithm.
9. The neighborhood inference based image feature matching system of claim 6, wherein the preliminary matching feature point pair obtaining unit includes:
the spatial neighborhood map building module is used for building a triangular subdivision map based on all the feature points in the feature point set T to obtain a spatial neighborhood map corresponding to the feature point set T;
and the preliminary matching module is used for performing feature matching on each candidate matching point in all the obtained candidate matching point sets in the space neighborhood map obtained by the space neighborhood map construction module to obtain a group of matched feature point pairs, wherein the group of matched feature point pairs are the group of preliminary matched feature point pairs.
10. The neighborhood inference based image feature matching system of claim 6, wherein the mismatch culling unit comprises:
a correct matching judgment module for judging each preliminary matching characteristic point pair (V) obtained by the preliminary matching characteristic point pair obtaining units，Ks) If it is a correct match, s 1, 2.., L; wherein (V)s，Ks) For the s-th characteristic point pair, V, of all the preliminary matching characteristic point pairs obtained by the preliminary matching characteristic point pair obtaining unitsAs feature points on the reference image, KsL is the total number of all the preliminary matching characteristic point pairs obtained by the preliminary matching characteristic point pair obtaining unit;
the characteristic matching point pair obtaining module is used for obtaining all the preliminary matching characteristic point pairs which are judged to be correctly matched by the correct matching judging module to obtain characteristic matching point pairs of the reference image and the target image;
wherein the correct matching judgment module judges each preliminary matching characteristic point pair (V) obtained by the preliminary matching characteristic point pair obtaining units，Ks) The method for judging whether the matching is correct or not comprises the following steps:
collect characteristic points VsForming a first group of preliminary matching feature point pairs by using all preliminary matching feature point pairs in a region with a preset radius threshold phi as a radius as a circle center;
collect the feature points KsForming a second group of preliminary matching feature point pairs by taking all the preliminary matching feature point pairs in the area with the radius threshold phi as the circle center and the radius threshold phi as the radius;
counting the number of the same preliminary matching feature point pairs in the first group of preliminary matching feature point pairs and the second group of preliminary matching feature point pairs, and recording the number as a statistical number;
judging whether the statistical quantity reaches a preset quantity threshold value Q:
if yes, judging the characteristic point pair (V)s，Ks) Is a correct match;
if not, judging the characteristic point pair (V)s，Ks) Is a false match.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011353954.2A CN112465025A (en) | 2020-11-26 | 2020-11-26 | Image feature matching method and system based on neighborhood inference |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011353954.2A CN112465025A (en) | 2020-11-26 | 2020-11-26 | Image feature matching method and system based on neighborhood inference |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112465025A true CN112465025A (en) | 2021-03-09 |
Family
ID=74809644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011353954.2A Pending CN112465025A (en) | 2020-11-26 | 2020-11-26 | Image feature matching method and system based on neighborhood inference |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112465025A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114140365A (en) * | 2022-01-27 | 2022-03-04 | 荣耀终端有限公司 | Event frame-based feature point matching method and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120148164A1 (en) * | 2010-12-08 | 2012-06-14 | Electronics And Telecommunications Research Institute | Image matching devices and image matching methods thereof |
| CN109101985A (en) * | 2018-06-03 | 2018-12-28 | 东南大学 | It is a kind of based on adaptive neighborhood test image mismatch point to elimination method |
-
2020
- 2020-11-26 CN CN202011353954.2A patent/CN112465025A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120148164A1 (en) * | 2010-12-08 | 2012-06-14 | Electronics And Telecommunications Research Institute | Image matching devices and image matching methods thereof |
| CN109101985A (en) * | 2018-06-03 | 2018-12-28 | 东南大学 | It is a kind of based on adaptive neighborhood test image mismatch point to elimination method |
Non-Patent Citations (2)
| Title |
|---|
| JIA-WANG BIAN ET AL.: "GMS：Grid-Based Motion Statistics for Fast，Ultra-Robust Feature Correspondence", 《INTERNATIONAL JOURNAL OF COMPUTER VISION(2020)》 * |
| TOBY COLLINS ET AL.: "An Analysis of Errors in Graph-Based Keypoint Matching and Proposed Solutions", 《EUROPEAN CONFERENCE ON COMPUTER VISION》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114140365A (en) * | 2022-01-27 | 2022-03-04 | 荣耀终端有限公司 | Event frame-based feature point matching method and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106952288B (en) | Based on convolution feature and global search detect it is long when block robust tracking method | |
| KR101532864B1 (en) | Planar mapping and tracking for mobile devices | |
| CN106997605B (en) | A method of foot type video is acquired by smart phone and sensing data obtains three-dimensional foot type | |
| CN107123161A (en) | A kind of the whole network three-dimensional rebuilding method of contact net zero based on NARF and FPFH | |
| CN109598290A (en) | A kind of image small target detecting method combined based on hierarchical detection | |
| CN106651827A (en) | Fundus image registering method based on SIFT characteristics | |
| CN108550166B (en) | Spatial target image matching method | |
| CN102521816A (en) | Real-time wide-scene monitoring synthesis method for cloud data center room | |
| CN107230197B (en) | Tropical cyclone objective strength determination method based on satellite cloud image and RVM | |
| CN110263920A (en) | Convolutional neural networks model and its training method and device, method for inspecting and device | |
| CN112163588A (en) | Intelligent evolution-based heterogeneous image target detection method, storage medium and equipment | |
| CN110309808A (en) | A kind of adaptive smog root node detection method under a wide range of scale space | |
| CN110232387A (en) | A kind of heterologous image matching method based on KAZE-HOG algorithm | |
| CN111898419B (en) | Partitioned landslide detection system and method based on cascaded deep convolutional neural network | |
| JP2011002919A (en) | Method and device for detecting object | |
| CN104751185A (en) | SAR image change detection method based on mean shift genetic clustering | |
| JP6482130B2 (en) | Geometric verification apparatus, program and method | |
| Ando et al. | Robust surface reconstruction of plant leaves from 3D point clouds | |
| Wang et al. | Automatic fault surface detection by using 3D Hough transform | |
| CN109409418A (en) | A kind of winding detection method based on bag of words | |
| CN109165603B (en) | Ship detection method and device | |
| CN110147837A (en) | The intensive object detection method of any direction, system and the equipment focused based on feature | |
| CN112906795A (en) | Whistle vehicle judgment method based on convolutional neural network | |
| CN112465025A (en) | Image feature matching method and system based on neighborhood inference | |
| CN110334704A (en) | Threedimensional model interest point extraction method and system based on Layered Learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210309 |
|
| RJ01 | Rejection of invention patent application after publication |