CN114862902A - Illumination self-adaptive ORB feature extraction and matching method based on quadtree - Google Patents
Illumination self-adaptive ORB feature extraction and matching method based on quadtree Download PDFInfo
- Publication number
- CN114862902A CN114862902A CN202210528680.9A CN202210528680A CN114862902A CN 114862902 A CN114862902 A CN 114862902A CN 202210528680 A CN202210528680 A CN 202210528680A CN 114862902 A CN114862902 A CN 114862902A
- Authority
- CN
- China
- Prior art keywords
- quadtree
- image
- matching
- points
- key points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000005286 illumination Methods 0.000 title claims abstract description 31
- 238000000605 extraction Methods 0.000 title claims abstract description 28
- 230000001965 increasing effect Effects 0.000 claims abstract description 7
- 238000012216 screening Methods 0.000 claims abstract description 5
- 238000009826 distribution Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 16
- 230000003044 adaptive effect Effects 0.000 claims description 10
- 238000011156 evaluation Methods 0.000 claims description 10
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000002776 aggregation Effects 0.000 claims description 6
- 238000004220 aggregation Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 abstract description 6
- 230000009466 transformation Effects 0.000 description 8
- 230000008859 change Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000000265 homogenisation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/223—Analysis of motion using block-matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/40—Tree coding, e.g. quadtree, octree
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an illumination self-adaptive ORB (object relational database) feature extraction and matching method based on a quadtree, which comprises the following steps of: step 1: acquiring a camera motion state, and constructing a pyramid image according to the camera motion state; step 2: extracting FAST key points; and step 3: adding or removing the key points by using an improved quadtree algorithm; and 4, step 4: extracting BRIEF descriptors of the key points to obtain feature points; and 5: acquiring matched feature point pairs according to the Hamming distance of the feature points; step 6: and rejecting the mismatching characteristic point pairs by using an improved RANSAC algorithm. An improved pyramid image construction method, an improved quadtree algorithm and a screening strategy of matching feature point pairs are provided, so that the robustness of the algorithm to illumination is enhanced and the running speed of the algorithm is increased.
Description
Technical Field
The invention relates to the technical field of image processing and stereoscopic vision, in particular to an illumination self-adaptive ORB feature extraction and matching method based on a quadtree.
Background
Meanwhile, the positioning and mapping (SLAM) is rapidly developed in recent years, and plays an important role in the fields of unmanned systems and equipment, intelligent robots, augmented reality and the like. Among them, feature extraction is the most important step in the SLAM algorithm. Historically, the search for feature extraction methods has never been stopped, and a number of excellent feature extraction algorithms, including the SIFT algorithm, SURF algorithm, and ORB algorithm, have emerged. Finally, the ORB algorithm is widely used in SLAM systems. In addition, a quadtree algorithm is added in the SLAM system in the feature extraction stage to improve the uniformity of feature distribution, and an image pyramid is added to solve the scale problem in image feature matching. Subsequently, in order to improve the performance of the ORB algorithm in the SLAM system, a series of improved ORB algorithms are emerged. Yao et al propose an ORB uniform distribution algorithm for improving quadtrees, which considers the overall contrast of an image when extracting feature points, sets the maximum depth of the quadtree according to the number of layers of different pyramids, and eliminates redundant feature points. Yang et al propose a local area adaptive threshold's improvement ORB feature extraction algorithm, divide into blocks to original image and each layer pyramid image first, adopt the maximum between-class variance method to extract the adaptive threshold, then use the adaptive threshold to extract the ORB feature point.
The ORB feature extraction algorithm can be applied in the SLAM system all the time, and shows that the performance of the algorithm is relatively mature and can be competent for the task of part of SLAM. The algorithm has the characteristics of high operation efficiency, less required time and certain robustness on factors such as image scale, rotation, illumination and the like. However, practical applications find that the ORB algorithm still has partial problems. First, the applicability of the ORB algorithm to illumination changes needs to be improved. The application of the ORB algorithm to illumination change is mainly shown in that the characteristic point extraction effect is better indoors, but enough characteristic points cannot be detected in the region with stronger illumination change in the outdoor image. Second, the running time of the ORB algorithm is to be promoted. In the whole SLAM algorithm running process, the ORB algorithm still occupies most of time, and the increase of the running speed of the ORB algorithm is always the bottleneck of the performance increase of the SLAM system.
Disclosure of Invention
In order to solve the technical problems provided by the background art, the invention provides an illumination self-adaptive ORB feature extraction and matching method based on a quadtree, and provides an improved pyramid image construction method, an improved quadtree algorithm and a screening strategy of matching feature point pairs so as to enhance the robustness of the algorithm to illumination and improve the running speed of the algorithm.
In order to achieve the purpose, the invention adopts the following technical scheme:
an illumination self-adaptive ORB feature extraction and matching method based on a quadtree comprises the following steps:
step 1: acquiring a camera motion state, and constructing a pyramid image according to the camera motion state;
step 2: extracting FAST key points;
and step 3: adding or removing key points by using an improved quadtree algorithm;
and 4, step 4: extracting BRIEF descriptors of the key points to obtain feature points;
and 5: acquiring matched feature point pairs according to the Hamming distance of the feature points;
step 6: and rejecting the mismatching characteristic point pairs by using an improved RANSAC algorithm.
Further, in the step 1, the motion state of the camera is acquired and a pyramid image is constructed, and the steps are as follows:
step 1-1: acquiring a motion state of a camera, wherein the motion state of the camera comprises a motion direction of the camera and motion speeds in the motion directions;
step 1-2: quantizing the motion state of the camera to obtain quantized camera motion parameters;
step 1-3: constructing scale parameters of the pyramid images according to the quantized camera motion parameters;
step 1-4: and constructing a pyramid image according to the scale parameters, and removing redundant pyramid image layers.
Further, in the step 3, adding or removing the key points by using an improved quadtree algorithm, the steps are as follows:
step 3-1: counting the number of key points extracted from image blocks in the nodes of the quadtree;
step 3-2: if the number of the extracted key points is less than the set threshold value, executing the step 3-3, and if the number of the extracted key points is more than the set threshold value, executing the step 3-9;
step 3-3: increasing brightness evaluation, and performing over-bright and over-dark estimation on the brightness of the image block;
step 3-4: if the evaluation result is that the brightness is too bright, executing step 3-5, and if the evaluation result is that the brightness is too dark, executing step 3-6;
step 3-5: introducing Gamma conversion, enhancing the over-bright image block, and executing the step 3-7 after the enhancement is finished;
step 3-6: introducing histogram equalization to enhance the over-dark image blocks;
step 3-7: extracting FAST key points again from the enhanced image blocks;
step 3-8: counting the number of the key points in the image block, executing the step 3-9 if the increment of the key points exceeds a specified threshold, and executing the step 3-11 if the increment of the key points does not exceed the specified threshold;
step 3-9: the quadtree nodes continue to split;
step 3-10: the splitting depth of the quadtree nodes reaches a set threshold;
step 3-11: the quadtree nodes stop splitting continuously;
step 3-12: and (4) retaining the key point with the best quality in the image block in the node of the quadtree, and deleting other key points.
Further, in step 3-3, the brightness evaluation process is as follows:
setting a quadtree node splitting depth threshold eta;
determining the ID of the quad-tree node which does not reach the splitting depth threshold value;
calculating the relative position coordinates of the central point of the image block in the quad-tree node ID in the whole image;
calculating the size data of the image block in the ID of the quad-tree node;
uniformly and dispersedly collecting n pixel points in an image block in a quad-tree node ID according to the relative position coordinates and the size data, wherein the collection modes are various;
for the ith acquisition mode, calculating the brightness value u of the image block i The formula is as follows:
u i ＝[I 0 (x 0 ，y 0 )+I 1 (x 1 ，y 1 )+……+I n (x n ，y n )]/n
I n (x n ，y n ) Representing the pixel value of the collected nth pixel point;
calculating a plurality of brightness values u for a plurality of acquisition modes i And selecting one of the final brightness values u of the image block according to actual requirements or calculating the final brightness value u according to a mean value, a median value, an extreme value and the like.
Further, in the step 6, the improved RANSAC algorithm is used to eliminate the pairs of mismatching feature points, and the steps are as follows:
step 6-1: counting all characteristic points with matching relation;
step 6-2: judging the feature points with the matching relation, executing a step 6-3 if the feature points belong to the lowest level quadtree node, and executing a step 6-4 if the feature points do not belong to the lowest level quadtree node;
step 6-3: if the quadtree node where the feature point is located is not adjacent, executing step 6-4, and if the quadtree node where the feature point is located is adjacent, executing step 6-5;
step 6-4: b type characteristic point pairs in sparse distribution are determined, and step 6-7 is executed;
step 6-5: determining a class A characteristic point pair of aggregation distribution;
step 6-6: rejecting mismatching characteristic point pairs in the matching direction aiming at the A-type characteristic point pairs;
step 6-7: constructing a simplified data set, carrying out RANSAC iteration on the simplified data set, and removing mismatching characteristic point pairs;
and 6-8: and obtaining the best matching point pair.
Further, in step 6-6, the calculation process of the matching direction is as follows:
and (3) calculating the matching direction of the characteristic point pairs in the two images:
(x 1 ，y 1 ) Representing feature points in the first image, (x) 2 ，y 2 ) And representing the matched characteristic points in the second image, and theta represents the matching direction between the calculated matched point pairs, and if the matching direction is too large, the characteristic point pairs are regarded as mismatching.
Further, in step 6-7, the reduced dataset includes:
the data in the simplified data set comprises two parts, wherein one part is an A-type characteristic point pair which is removed by the matching direction and is in mismatching, and the other part is a B-type characteristic point pair;
the RANSAC iteration time for the reduced data set is reduced, and meanwhile, the A-type characteristic point pairs which are mismatched are eliminated from the reduced data set.
Further, in step 6-7, the process of rejecting the pairs of mismatching feature points is as follows:
RANSAC iteration is carried out on the A-type characteristic point pairs, mismatching characteristic point pairs are removed in the iteration process, and an optimal homography matrix is determined;
and screening the B-type characteristic point pairs by using the optimal homography matrix, and rejecting mismatching characteristic point pairs.
Compared with the prior art, the invention has the beneficial effects that:
1) according to the illumination self-adaptive ORB feature extraction and matching method based on the quad-tree, brightness evaluation is added in the quad-tree algorithm to judge image blocks with high exposure and low illumination, Gamma transformation is introduced to enhance the image blocks with high exposure, and a histogram equalization algorithm is introduced to enhance the image blocks with low illumination, so that the improved quad-tree algorithm has robustness and adaptability to illumination, and the accuracy of feature extraction and matching is improved;
2) according to the illumination self-adaptive ORB feature extraction and matching method based on the quadtree, the method for improving the pyramid image construction by utilizing the motion state of the camera eliminates redundant pyramid image layers and improves the operation speed of the algorithm;
3) according to the illumination self-adaptive ORB feature extraction and matching method based on the quadtree, the simplified data set is constructed by using the method of eliminating the mismatching feature point pairs in the matching direction, and RANSAC iterative computation is performed on the simplified data set, so that the improved RANSAC algorithm eliminates the mismatching feature point pairs, meanwhile, the iterative time is reduced, and the operation speed of the algorithm and the matching accuracy are improved.
Drawings
Fig. 1 is a flowchart of an illumination adaptive ORB feature extraction and matching method based on a quadtree according to an embodiment of the present invention;
FIG. 2 is a sub-flowchart for constructing a pyramid image according to an embodiment of the present invention;
FIG. 3 is a sub-flowchart of an improved quadtree algorithm provided by an embodiment of the present invention;
fig. 4 is a sub-flowchart of an improved RANSAC algorithm provided by an embodiment of the present invention;
fig. 5 is a schematic diagram of a motion mode of a camera according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of directed splitting of a quadtree node according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a pixel point collection method for luminance evaluation according to an embodiment of the present invention.
Detailed Description
The following detailed description of the present invention will be made with reference to the accompanying drawings.
In the flowcharts of fig. 1 to 4, the 1 st letter S of the reference numeral indicates a step, the value 1 of the 2 nd digit indicates a flowchart, the 3 rd digit indicates a step of a main flowchart, and the 4 th digit indicates a step of a sub-flowchart.
The invention relates to an illumination self-adaptive ORB (object relational database) feature extraction and matching method based on a quadtree, wherein the flow is shown in figure 1 and comprises the following steps:
s1.1.0: camera motion patterns are acquired and pyramid images are constructed.
According to the invention, the camera motion state is introduced into the construction of the pyramid image to obtain the quantized camera motion parameters, the scale parameters of the pyramid image are constructed by using the camera motion parameters, and redundant pyramid image layers are removed, so that the operation speed of the algorithm is increased.
S1.1.0, acquiring the camera motion pattern and constructing a pyramid image, the sub-process is shown in fig. 2, and the steps are as follows:
s1.1.1: the camera motion state is acquired.
The motion state of the camera is determined by the external device on which the camera is mounted, and the motion information shared by the external device is read to determine the motion state of the camera.
The embodiment of the invention summarizes the motion states of the camera into four types: the camera motion parallel to the target feature point to the left, the camera motion parallel to the target feature point to the right, the camera motion perpendicular to the target feature point to the front and the camera motion perpendicular to the target feature point to the back can be divided into the combination of the four motion states; the motion state of the camera also includes the speed of motion of the camera in each direction of motion.
As shown in fig. 5, (a) shows the camera motion parallel to the target feature point, and (b) shows the camera motion perpendicular to the target feature point; in the camera motion parallel to the target feature points, the image target feature points have no scale change, and a pyramid image is not constructed; the scale of the image target feature point is changed in the camera motion perpendicular to the target feature point, the image feature point is increased in the camera motion perpendicular to the target feature point forwards, the image feature point is decreased in the camera motion perpendicular to the target feature point backwards, and a pyramid image is constructed.
S1.1.2: camera state parameters are quantized.
Quantifying camera state parameters includes quantifying a direction of motion and a speed of motion of the camera.
Quantizing the camera state parameters includes:
1) the parameter + v represents the camera motion forward perpendicular to the target feature point, with a motion velocity v, and a norm range of v of [0, + ∞);
2) parameter-v represents the backward camera motion perpendicular to the target feature point, with a motion velocity v;
3) the parameter v-0 represents the camera motion parallel to the target feature point.
S1.1.3: and constructing a pyramid image scale parameter.
And constructing pyramid image scale parameters by combining the quantized camera state parameters.
The construction of the scale parameters of the pyramid images is divided into two cases:
1) the camera state parameter is-v, the camera moves backwards perpendicular to the target feature point, at this time, the image feature point becomes small, the original image needs to be amplified, and the pyramid image scale parameter s is constructed as follows:
s＝α b *v+ 1
α b a coefficient is constructed for the pyramid image, the specific numerical value can be determined according to the camera state parameter measured by the external equipment, and the value of the embodiment of the invention is 0.4;
2) the camera state parameter is + v, the camera moves forward perpendicular to the target feature point, the image feature point becomes large at this time, the original image needs to be reduced, and the pyramid image scale parameter s is constructed as follows:
α f the specific numerical value of the coefficient is constructed for the pyramid image and can be determined according to the camera state parameter measured by the external equipment, and the value of the embodiment of the invention is 0.4.
S1.1.4: a pyramid image is constructed.
And constructing the pyramid image of the original image according to the scale parameters of the pyramid image.
The process of constructing the pyramid image is as follows:
1) setting the size of the zeroth layer original image: the width is W, the height is H, and the scale parameter s is used for calculating the scale of the original image as follows:
W s ＝W*s
H s ＝H*s
W s and H s Representing the constructed pyramid image scale;
2) and carrying out scale transformation on the original image to obtain a pyramid image after the scale transformation.
Compared with the traditional eight-layer pyramid image construction strategy, the embodiment of the invention only constructs one layer of new pyramid image on the basis of the original image, eliminates redundant pyramid image layers and improves the operation speed of the algorithm.
In the embodiment of the invention, the original image and the constructed new layer of pyramid image are collectively called the pyramid image, and the number m of layers of the pyramid image is 2.
S1.2.0: extracting FAST key points.
And extracting FAST key points from the pyramid images.
Distributing the number of the key points to each layer of image according to the image area of the pyramid image; the smaller the area of the pyramid image is, the lower the corresponding image resolution is, and the fewer key points of the extracted image are; the larger the area of the pyramid image is, the higher the corresponding image resolution is, and the more image key points are extracted.
Firstly, calculating the number of extracted key points of FAST of pyramid images of each layer, and the process is as follows:
1) setting the number of extracted key points as X, and setting the area of the zeroth layer original image as C-H-W;
2) the total area S of the pyramid images is calculated as follows:
3) calculating the distribution quantity X of key points in unit area avg Such asThe following:
4) calculating the number X of key points to be distributed to the ith (i is 0, 1) layer pyramid image i The following are:
then, extracting the key points, wherein the process is as follows:
1) performing edge expansion on the image, wherein the expanded width is g;
2) selecting a pixel point P (x, y), and assuming that the pixel value of the point is P;
3) setting a threshold T ═ p × 40%;
4) selecting all pixel points on a circle with P (x, y) as the center of the circle and g as the radius;
5) if the pixel value of continuous B points of the pixel point on the circle is larger than P + T or smaller than P-T, the pixel point P (x, y) is determined as a key point;
6) and (5) circularly executing the steps 2) and 5) until all pixel points of the complete image are traversed.
Through the distribution of the number of key points and the extraction of the key points, the key points are extracted from the regions of the pyramid image, wherein the gray level change threshold of the pixels exceeds T.
S1.3.0: and adding or removing the key points by using an improved quadtree algorithm.
FAST key points can be distributed unevenly, and the embodiment of the invention adds or eliminates the key points by improving the quadtree algorithm; when adding the key points, the influence of illumination on the extraction result of the key points is mainly considered, the image enhancement under high exposure and low illumination is carried out on the sparse distribution area of the key points, the key points are extracted from the enhanced image area, and the effect of adding the key points is achieved; and (4) removing the key points from the key point aggregation distribution area to realize key point homogenization.
The improved quadtree algorithm has robustness and adaptability to illumination, and the accuracy rate of feature extraction and matching is improved
S1.3.0, adding or removing key points by using an improved quadtree algorithm, wherein the sub-process is shown in fig. 3 and comprises the following steps:
s1.3.1: and counting the number of key points in the nodes of the quadtree.
And finishing the directional splitting of the nodes of the quadtree on the original image, and counting the number of key points in each quadtree node.
The process of directional splitting of the quadtree nodes is as follows:
1) setting an original image as an original node N, dividing the original image into four parts with equal areas, namely splitting the node N into four sub-nodes N1, N2, N3 and N4, wherein each sub-node corresponds to an image block of the original image, and at this time, completing the first splitting of a quadtree node, as shown in FIG. 6;
2) setting a key point number threshold f to be 2, counting the number of key points in the image block of the quadtree node, continuously splitting the nodes of which the number of the key points in the quadtree node exceeds the threshold f, and stopping continuously splitting the nodes of which the number of the key points in the quadtree node does not exceed the threshold f; as shown in fig. 6, the number of key points in the nodes N1, N2, and N3 is less than the set threshold f, and the nodes stop continuing splitting; when the number of key points in the node N4 exceeds a threshold value f, the node continues to split, and at the moment, the node N4 is split into four child nodes, namely N41, N42, N43 and N44; the number of key points in the nodes N42, N43 and N44 is less than a set threshold f, and the nodes stop continuously splitting; when the number of the key points in the node N41 exceeds a set threshold value f, the node continues to split, and at the moment, the node N41 is split into four child nodes N411, N412, N413 and N414;
3) detecting that the splitting depth of the quadtree node reaches a maximum splitting depth threshold eta, and stopping the quadtree node from continuing splitting; as shown in FIG. 6, the node N413 of the quadtree reaches the maximum splitting depth, the node N413 stops continuing splitting, and the key points in the image block of the node N413 are counted to obtain key points C0, C1 and C2 of the aggregation distribution.
S1.3.2: and judging whether the number of key points in the quad-tree node is less than a set threshold value.
If the number of the key points extracted from the image block in the quad-tree node is less than the set threshold value, S1.3.3 is executed; if the number of the key points extracted from the image block in the quad-tree node is more than the set threshold value, S1.3.9 is executed.
Regarding the node of the quadtree with the splitting depth reaching the maximum splitting, the node is regarded as a key point aggregation distribution area; and regarding the nodes of the quadtree of which the splitting depth does not reach the maximum splitting, considering the nodes as the sparse distribution area of the key points.
S1.3.3: and performing brightness evaluation on the image blocks in the nodes.
Evaluating the brightness of the image through the pixel value of the gray level image, and adding brightness evaluation behind an image block in a quad-tree node with sparsely distributed key points; and collecting pixel points of the image block, and then carrying out over-bright and over-dark estimation on the brightness of the pixel points.
The S1.3.3 brightness assessment process is as follows:
defining the splitting of the quadtree node to the maximum splitting depth as the quadtree node is completely split, and defining the non-splitting of the quadtree node to the maximum splitting depth as the quadtree node is not completely split; determining a quad-tree node ID of a depth threshold which is not completely split, calculating the relative position of an image block in the whole image and the size of the image block in the quad-tree node ID, determining the center of the image block according to the size of the image block, and determining the relative position of the center of the image block in the whole image according to the relative position of the image block; and according to the relative position of the center of the image block and the size of the image block, uniformly and dispersedly collecting n pixel points in the image block.
As shown in fig. 7, the embodiment of the present invention provides three ways of collecting pixel points; taking the relative position of the center of the image block in the quad-tree node in the whole image as the middle point, fig. 7(a) uniformly selects n pixel points on the rectangular frame with the image block size reduced by 10%, fig. 7(b) uniformly selects n pixel points on the diagonal line of the rectangular frame with the image block size reduced by 10%, and fig. 7(c) uniformly selects n pixel points on the central line of the rectangular frame with the image block size reduced by 10%.
For the ith acquisition mode, calculating the brightness value u of the image block i The formula is as follows:
u i ＝[I 0 (x 0 ，y 0 )+I 1 (x 1 ，y 1 )+……+I n (x n ，y n )]/n
I n (x n ，y n ) And expressing the pixel value of the collected nth pixel point.
Calculating three acquisition modes to obtain a plurality of brightness values u i And selecting one of the final brightness values u of the image block according to actual requirements or calculating the final brightness value u according to a mode of a mean value, a median value, an extreme value and the like.
S1.3.4: if the statistical result shows that the brightness of the image block in the node of the quadtree is too bright overall, S1.3.5 is executed; if the statistics show that the brightness of the image block within this quadtree node is too dark overall, S1.3.6 is performed.
In the real-time example of the invention, whether the image block is too bright is judged by comparing the brightness value u with the pixel value of the image block.
S1.3.5: and Gamma conversion is performed, and S1.3.7 is executed after the completion.
Calling a Gamma transformation algorithm, and processing the image blocks in the over-bright quadtree nodes to obtain an image with higher contrast; the Gamma transformation performs exponential transformation on the pixel values of the original image, and the contrast of the image is stretched.
The Gamma transformation process is as follows:
1) and calculating Gamma index xi by using the brightness value u, wherein the formula is as follows:
ξ＝ψ(u)
the transformation function ψ (u) relates ξ to the luminance value u and transforms u into the appropriate exponent;
2) substituting xi into a Gamma function to stretch the pixel value of the image, and the formula is as follows:
s G ＝Γ(ξ)＝cp G ξ
c is a constant, p G Is the pixel value, s, of the original image G And stretching the pixel values of the image by a Gamma function.
Experiments prove that the Gamma transform has a good effect of enhancing the image with high exposure, namely over-brightness, and the enhanced image block can extract more key points.
S1.3.6: and (6) histogram equalization.
Calling a traditional histogram equalization algorithm to process the image blocks in the excessively dark quadtree nodes to obtain image blocks with higher contrast; histogram equalization adjusts the distribution of the pixel values of the original image and stretches the contrast of the image.
Experiments prove that the histogram equalization has a good effect of enhancing images with low illumination, namely, too dark brightness, and the enhanced image blocks can extract more key points.
S1.3.7: the FAST key points were extracted again.
The FAST keypoints are extracted again for the enhanced image block as per S1.2.0.
S1.3.8: and judging whether the increase quantity of the key points exceeds a specified threshold value.
Counting the number of the key points in the image block in the quad-tree node; if the keypoint increment exceeds a specified threshold, S1.3.9 is executed; if the keypoint increment does not exceed the specified threshold, S1.3.11 is performed.
S1.3.9: the quadtree nodes continue to split.
The number of key points in the image block is large, and the nodes of the quadtree continue to be split.
S1.3.10: the quadtree nodes are split to a specified depth.
And directionally splitting the nodes of the quadtree to reach the maximum depth eta.
S1.3.11: the quadtree nodes stop continuing to split.
Carrying out key point statistics on the image blocks after the quadtree is split; if no key point exists in the image block or the number of the key points is less than a specified threshold, the quadtree node stops continuously splitting; if the maximum depth η is reached, the quadtree nodes stop splitting.
S1.3.12: and (4) retaining the key point with the best quality in the image block in the node of the quadtree, and deleting other key points.
The key points of the original image are the key points with the best reserved quality, namely the key points with the strongest texture features.
S1.4.0: the keypoint descriptor is computed.
And calling a BRIEF algorithm, calculating BRIEF descriptors of all key points of the original image, and determining the BRIEF descriptors as the feature points of the original image.
S1.5.0: and matching the characteristic point pairs.
Obtaining the characteristic points of the original image through the calculation of the steps; obtaining the characteristic points of the matched images through the calculation of the same step; and calculating the Hamming distance between the characteristic points of the two images, and determining the two characteristic points with small Hamming distance as the characteristic point pairs with matching relationship.
S1.6.0: and the RANSAC algorithm is improved to remove mismatching characteristic point pairs.
The improved RANSAC algorithm constructs a simplified data set by using a method of eliminating mismatching characteristic point pairs in a matching direction, RANSAC iterative computation is performed on the simplified data set, the mismatching characteristic point pairs are eliminated, the matching accuracy is improved, and the running speed of the algorithm is increased.
S1.6.0, using the improved RANSAC algorithm to eliminate the mismatching feature point pairs, the sub-process is shown in FIG. 4, and the steps are as follows:
s1.6.1: counting all characteristic points with matching relation in the original image;
s1.6.2: judging the feature points with matching relation in the original image, and if the feature points belong to the lowest level quadtree nodes, executing S1.6.3; if the feature point does not belong to the lowest level quadtree node, S1.6.4 is performed;
s1.6.3: if the quadtree nodes where the feature points are located are not adjacent, S1.6.4 is executed; if the quadtree nodes where the feature points are located are adjacent, S1.6.5 is executed;
s1.6.4: obtaining B-type matching point pairs between the original image and the matching image, and performing S1.6.7;
the B-type feature points have the characteristic of sparse distribution.
S1.6.5: obtaining an A-type matching point pair between the original image and the matching image;
the class a feature points have the characteristic of an aggregate distribution.
S1.6.6: rejecting mismatching characteristic point pairs through the matching direction;
in the characteristic point aggregation distribution area, the difference of the matching direction of the correct matching characteristic point pair is not large, and the difference of the matching direction of the mis-matching characteristic point pair and the matching direction of the correct matching characteristic point pair is large; and calculating the matching direction of the characteristic points, and eliminating mismatching characteristic point pairs according to the size of the matching direction.
The S1.6.6 matching direction is calculated as follows:
calculating the direction of the characteristic point pairs of the original image and the matched image;
(x 1 ，y 1 ) Representing feature points in the original image, (x) 2 ，y 2 ) Representing the feature points in the matching image that match it, and theta represents the direction between the calculated pairs of matching points.
And if the matching direction theta exceeds a set threshold value R, the matching point pair is considered as mismatching, and the matching characteristic point pair is removed.
S1.6.7: RANSAC iteration is carried out on the simplified data set, and mismatching feature point pairs are removed;
constructing a simplified data set, calling an original RANSAC algorithm to carry out iterative computation on the simplified data set to obtain an optimal homography matrix, and eliminating mismatching characteristic point pairs through RANSAC iteration and the optimal homography matrix.
The reduced dataset comprises:
the data in the reduced dataset comprises two parts, wherein one part is a class A characteristic point pair which is removed by the matching direction and is mismatched, and the other part is a class B characteristic point pair.
RANSAC iteration time for the simplified data set is reduced, and the operation speed of the algorithm is increased; a type-A characteristic point pairs which are mismatched are removed from the simplified data set, and the accuracy of the algorithm is improved.
S1.6.7 the process of rejecting the mismatched characteristic point pairs is as follows:
performing RANSAC iteration on the A-type characteristic point pairs which are subjected to mismatching elimination in the matching direction, eliminating the mismatching characteristic point pairs again in the iteration process, and determining an optimal homography matrix;
and screening the B-type characteristic point pairs by using the optimal homography matrix, and rejecting mismatching characteristic point pairs.
S1.6.8: and obtaining the best matching point pair.
The variables not described in the above example formulas are all common knowledge variables.
The above embodiments are implemented on the premise of the technical solution of the present invention, and detailed embodiments and specific operation procedures are given, but the scope of the present invention is not limited to the above embodiments. The methods used in the above examples are conventional methods unless otherwise specified.
Claims (8)
1. An illumination self-adaptive ORB feature extraction and matching method based on a quadtree is characterized by comprising the following steps:
step 1: acquiring a camera motion state, and constructing a pyramid image according to the camera motion state;
step 2: extracting FAST key points;
and step 3: adding or removing the key points by using an improved quadtree algorithm;
and 4, step 4: extracting BRIEF descriptors of all key points to obtain feature points;
and 5: acquiring matched feature point pairs according to the Hamming distance of the feature points;
step 6: and rejecting the mismatching characteristic point pairs by using an improved RANSAC algorithm.
2. The method for extracting and matching an illumination adaptive ORB feature based on a quadtree as claimed in claim 1, wherein in the step 1, the motion state of the camera is obtained and a pyramid image is constructed, and the steps are as follows:
step 1-1: acquiring a motion state of a camera, wherein the motion state of the camera comprises a motion direction of the camera and motion speeds in the motion directions;
step 1-2: quantizing the motion state of the camera to obtain quantized camera motion parameters;
step 1-3: constructing scale parameters of the pyramid images according to the quantized camera motion parameters;
step 1-4: and constructing a pyramid image according to the scale parameters, and removing redundant pyramid image layers.
3. The illumination adaptive ORB feature extraction and matching method based on the quadtree as claimed in claim 1, wherein in the step 3, adding or removing the key points by using the improved quadtree algorithm comprises the following steps:
step 3-1: counting the number of key points extracted from image blocks in the nodes of the quadtree;
step 3-2: if the number of the extracted key points is less than the set threshold value, executing the step 3-3, and if the number of the extracted key points is more than the set threshold value, executing the step 3-9;
step 3-3: increasing brightness evaluation, and performing over-bright and over-dark estimation on the brightness of the image block;
step 3-4: if the evaluation result is that the brightness is too bright, executing the step 3-5, and if the evaluation result is that the brightness is too dark, executing the step 3-6;
step 3-5: introducing Gamma conversion, enhancing the over-bright image block, and executing the step 3-7 after the enhancement is finished;
step 3-6: introducing histogram equalization to enhance the over-dark image blocks;
step 3-7: extracting FAST key points again from the enhanced image blocks;
step 3-8: counting the number of the key points in the image block, executing the step 3-9 if the increment of the key points exceeds a specified threshold, and executing the step 3-11 if the increment of the key points does not exceed the specified threshold;
step 3-9: the quadtree nodes continue to split;
step 3-10: the splitting depth of the quadtree nodes reaches a set threshold;
step 3-11: the quadtree nodes stop splitting continuously;
step 3-12: and (4) retaining the key point with the best quality in the image block in the node of the quadtree, and deleting other key points.
4. A quadtree-based illumination adaptive ORB feature extraction and matching method as claimed in claim 3, wherein in step 3-3, the brightness estimation process is as follows:
setting a quadtree node splitting depth threshold eta;
determining the ID of the quad-tree node which does not reach the splitting depth threshold value;
calculating the relative position coordinates of the central point of the image block in the quad-tree node ID in the whole image;
calculating the size data of the image block in the ID of the quad-tree node;
uniformly and dispersedly collecting n pixel points in an image block in a quad-tree node ID according to the relative position coordinates and the size data, wherein the collection modes are various;
for the ith acquisition mode, calculating the brightness value u of the image block i The formula is as follows:
u i ＝[I 0 (x 0 ，y 0 )+I 1 (x 1 ，y 1 )+……+I n (x n ，y n )]/n
I n (x n ,y n ) Representing the pixel value of the collected nth pixel point;
calculating a plurality of brightness values u for a plurality of acquisition modes i And selecting one of the final brightness values u of the image block according to actual requirements or calculating the final brightness value u according to a mean value, a median value, an extreme value and the like.
5. The illumination adaptive ORB feature extraction and matching method based on the quadtree as claimed in claim 1, wherein in the step 6, the improved RANSAC algorithm is used to eliminate the mismatching feature point pairs, and the steps are as follows:
step 6-1: counting all characteristic points with matching relation;
step 6-2: judging the feature points with the matching relation, executing a step 6-3 if the feature points belong to the lowest level quadtree node, and executing a step 6-4 if the feature points do not belong to the lowest level quadtree node;
step 6-3: if the quadtree node where the feature point is located is not adjacent, executing step 6-4, and if the quadtree node where the feature point is located is adjacent, executing step 6-5;
step 6-4: b-type characteristic point pairs in sparse distribution are determined, and step 6-7 is executed;
step 6-5: determining a class A characteristic point pair of aggregation distribution;
step 6-6: for the A-type characteristic point pairs, rejecting mismatching characteristic point pairs through the matching direction;
step 6-7: constructing a simplified data set, carrying out RANSAC iteration on the simplified data set, and removing mismatching characteristic point pairs;
and 6-8: and obtaining the best matching point pair.
6. An illumination adaptive ORB feature extraction and matching method based on quadtree as claimed in claim 5, wherein in the step 6-6, the matching direction is calculated as follows:
and (3) calculating the matching direction of the characteristic point pairs in the two images:
(x 1 ，y 1 ) Representing feature points in the first image, (x) 2 ，y 2 ) And representing the matched characteristic points in the second image, and theta represents the matching direction between the calculated matched point pairs, and if the matching direction is too large, the characteristic point pairs are regarded as mismatching.
7. The method of claim 5, wherein in the steps 6-7, the reduced data set comprises:
the data in the simplified data set comprises two parts, wherein one part is an A-type characteristic point pair which is removed by the matching direction and is in mismatching, and the other part is a B-type characteristic point pair;
the RANSAC iteration time for the reduced data set is reduced, and meanwhile, the A-type characteristic point pairs which are mismatched are eliminated from the reduced data set.
8. The illumination adaptive ORB feature extraction and matching method based on the quadtree as claimed in claim 5, wherein in the steps 6-7, the process of rejecting the mis-matching feature point pairs is as follows:
RANSAC iteration is carried out on the A-type feature point pairs, mismatching feature point pairs are removed in the iteration process, and an optimal homography matrix is determined;
and screening the B-type characteristic point pairs by using the optimal homography matrix, and rejecting mismatching characteristic point pairs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210528680.9A CN114862902A (en) | 2022-05-16 | 2022-05-16 | Illumination self-adaptive ORB feature extraction and matching method based on quadtree |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210528680.9A CN114862902A (en) | 2022-05-16 | 2022-05-16 | Illumination self-adaptive ORB feature extraction and matching method based on quadtree |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114862902A true CN114862902A (en) | 2022-08-05 |
Family
ID=82637761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210528680.9A Pending CN114862902A (en) | 2022-05-16 | 2022-05-16 | Illumination self-adaptive ORB feature extraction and matching method based on quadtree |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114862902A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665256A (en) * | 2023-06-05 | 2023-08-29 | 长春理工大学 | Fingerprint matching method based on fingerprint image local area quality |
| CN117315274A (en) * | 2023-11-28 | 2023-12-29 | 淄博纽氏达特机器人***技术有限公司 | Visual SLAM method based on self-adaptive feature extraction |
-
2022
- 2022-05-16 CN CN202210528680.9A patent/CN114862902A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116665256A (en) * | 2023-06-05 | 2023-08-29 | 长春理工大学 | Fingerprint matching method based on fingerprint image local area quality |
| CN116665256B (en) * | 2023-06-05 | 2024-03-15 | 长春理工大学 | Fingerprint matching method based on fingerprint image local area quality |
| CN117315274A (en) * | 2023-11-28 | 2023-12-29 | 淄博纽氏达特机器人***技术有限公司 | Visual SLAM method based on self-adaptive feature extraction |
| CN117315274B (en) * | 2023-11-28 | 2024-03-19 | 淄博纽氏达特机器人***技术有限公司 | Visual SLAM method based on self-adaptive feature extraction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114862902A (en) | Illumination self-adaptive ORB feature extraction and matching method based on quadtree | |
| CN115249246B (en) | Optical glass surface defect detection method | |
| CN107038416B (en) | Pedestrian detection method based on binary image improved HOG characteristics | |
| CN110782477A (en) | Moving target rapid detection method based on sequence image and computer vision system | |
| CN111339924B (en) | Polarized SAR image classification method based on superpixel and full convolution network | |
| CN109685045A (en) | A kind of Moving Targets Based on Video Streams tracking and system | |
| CN110706294A (en) | Method for detecting color difference degree of colored textile fabric | |
| CN110276764A (en) | K-Means underwater picture background segment innovatory algorithm based on the estimation of K value | |
| CN115393657B (en) | Metal pipe production abnormity identification method based on image processing | |
| CN109472770B (en) | Method for quickly matching image characteristic points in printed circuit board detection | |
| CN109035196A (en) | Saliency-Based Image Local Blur Detection Method | |
| CN113052859A (en) | Super-pixel segmentation method based on self-adaptive seed point density clustering | |
| CN111199245A (en) | Rape pest identification method | |
| CN116630971B (en) | Wheat scab spore segmentation method based on CRF_Resunate++ network | |
| CN115272319B (en) | Ore granularity detection method | |
| CN111709305B (en) | Face age identification method based on local image block | |
| CN116721121A (en) | Plant phenotype color image feature extraction method | |
| CN113033345B (en) | V2V video face recognition method based on public feature subspace | |
| CN112446417A (en) | Spindle-shaped fruit image segmentation method and system based on multilayer superpixel segmentation | |
| CN106611418A (en) | Image segmentation algorithm | |
| CN112241954B (en) | Full-view self-adaptive segmentation network configuration method based on lump differentiation classification | |
| CN110532892B (en) | Method for detecting road vanishing point of single image of unstructured road | |
| CN113496159B (en) | Multi-scale convolution and dynamic weight cost function smoke target segmentation method | |
| CN115063615A (en) | Repeated texture image matching method based on Delaunay triangulation | |
| CN111723737B (en) | Target detection method based on multi-scale matching strategy deep feature learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |