CN117333687A - Feature point matching method and device based on binocular vision - Google Patents

Feature point matching method and device based on binocular vision Download PDF

Info

Publication number
CN117333687A
CN117333687A CN202311326180.8A CN202311326180A CN117333687A CN 117333687 A CN117333687 A CN 117333687A CN 202311326180 A CN202311326180 A CN 202311326180A CN 117333687 A CN117333687 A CN 117333687A
Authority
CN
China
Prior art keywords
feature point
characteristic
points
matching
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311326180.8A
Other languages
Chinese (zh)
Inventor
李景
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Desai Xiwei Intelligent Transportation Technology Co ltd
Original Assignee
Guangzhou Desai Xiwei Intelligent Transportation Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Desai Xiwei Intelligent Transportation Technology Co ltd filed Critical Guangzhou Desai Xiwei Intelligent Transportation Technology Co ltd
Priority to CN202311326180.8A priority Critical patent/CN117333687A/en
Publication of CN117333687A publication Critical patent/CN117333687A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/757—Matching configurations of points or features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the technical field of driving control, and discloses a feature point matching method and device based on binocular vision, wherein the method comprises the following steps: collecting an image set of a target scene; extracting feature points of the image set according to a feature point extraction algorithm to obtain feature point extraction results corresponding to the image set; calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points; determining a plurality of matched feature point groups to be screened from all the first feature points and all the second feature points according to all the similarities; and screening all the matched characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matched characteristic point group. Therefore, the implementation of the invention can improve the screening accuracy of the matched characteristic point group.

Description

Feature point matching method and device based on binocular vision
Technical Field
The invention relates to the technical field of driving control, in particular to a feature point matching method and device based on binocular vision.
Background
Binocular vision (Binocular Stereo Vision) is an important form of machine vision, and is a method for acquiring three-dimensional geometric information of an object by calculating position deviation between corresponding points of images based on parallax principles and by acquiring two images of the object to be measured from different positions by using imaging equipment.
Currently, in order to facilitate the use of epipolar constraint and calculation of coordinates of a target point, an epipolar line correction is usually performed on images of two imaging devices, and the images are adjusted to be in the same plane. However, in the process of adjusting the image to the same plane by adopting the epipolar correction method, once the image edge is distorted or the image is deformed, the characteristic points in the image are easily subjected to incorrect matching, so that the situation of obtaining incorrect three-dimensional geometric information of objects (such as vehicles, pedestrians, mountain bodies and the like) in the image is caused. It is important to provide a technical scheme for improving the matching accuracy of the image feature points.
Disclosure of Invention
The invention provides a feature point matching method and device based on binocular vision, which can improve the matching accuracy of a matching feature point group to be screened in an image and is beneficial to improving the acquisition accuracy of three-dimensional information of an object in the image.
In order to solve the technical problem, the first aspect of the present invention discloses a feature point matching method based on binocular vision, which comprises the following steps:
acquiring an image set of a target scene, wherein the image set comprises a first type image of an object to be molded in the target scene, which is acquired by a first image acquisition device in a binocular image acquisition device in the target scene, and a second type image of the object to be molded, which is acquired by a second image acquisition device in the binocular image acquisition device;
extracting feature points from a plurality of image positions in the image set according to a predetermined feature point extraction algorithm to obtain feature point extraction results corresponding to the image set, wherein the feature point extraction results comprise first type feature points corresponding to the first type image and second type feature points corresponding to the second type image, and the first type feature points and the second type feature points are feature points of the object to be molded;
calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points;
Determining a plurality of matched characteristic point groups to be screened from all the first characteristic points and all the second characteristic points according to the similarity between all the first characteristic points and all the second characteristic points, wherein each matched characteristic point group comprises a first characteristic point and a second characteristic point;
and screening all the matched characteristic point groups according to the epipolar positions corresponding to the calculated image sets to obtain at least one target matched characteristic point group, wherein all the target matched characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for building a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model.
The second aspect of the invention discloses a feature point matching device based on binocular vision, which comprises:
the system comprises an acquisition module, a first image acquisition device and a second image acquisition device, wherein the acquisition module is used for acquiring an image set of a target scene, and the image set comprises a first type image aiming at an object to be molded in the target scene, which is acquired by the first image acquisition device in a binocular image acquisition device in the target scene, and a second type image aiming at the object to be molded, which is acquired by the second image acquisition device in the binocular image acquisition device;
The extraction module is used for extracting characteristic points from a plurality of image positions in the image set according to a predetermined characteristic point extraction algorithm to obtain a characteristic point extraction result corresponding to the image set, wherein the characteristic point extraction result comprises a first type of characteristic points corresponding to the first type of images and a second type of characteristic points corresponding to the second type of images, and the first type of characteristic points and the second type of characteristic points are characteristic points of the object to be molded;
the computing module is used for computing the similarity between each first characteristic point and each second characteristic point according to the computed Harr wavelet characteristic value of each first characteristic point of the first type of characteristic points and the computed Harr wavelet characteristic value of each second characteristic point of the second type of characteristic points;
the determining module is used for determining a plurality of matched characteristic point groups to be screened from all the first characteristic points and all the second characteristic points according to the similarity between all the first characteristic points and all the second characteristic points, wherein each matched characteristic point group comprises a first characteristic point and a second characteristic point;
the screening module is used for screening all the matching characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matching characteristic point group, wherein all the target matching characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for building a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model.
The third aspect of the invention discloses another feature point matching device based on binocular vision, which comprises:
a memory storing executable program code;
a processor coupled to the memory;
the processor invokes the executable program code stored in the memory to execute the feature point matching method based on binocular vision disclosed in the first aspect of the present invention.
A fourth aspect of the present invention discloses a computer storage medium storing computer instructions for performing the binocular vision-based feature point matching method disclosed in the first aspect of the present invention when the computer instructions are called.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, an image set of a target scene is acquired, wherein the image set comprises a first type image of an object to be molded in the target scene, which is acquired by a first image acquisition device in a binocular image acquisition device in the target scene, and a second type image of the object to be molded, which is acquired by a second image acquisition device in the binocular image acquisition device; extracting feature points from a plurality of image positions in an image set according to a predetermined feature point extraction algorithm to obtain feature point extraction results corresponding to the image set, wherein the feature point extraction results comprise first type feature points corresponding to a first type image and second type feature points corresponding to a second type image, and the first type feature points and the second type feature points are feature points of an object to be modeled; calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points; according to the similarity between all the first characteristic points and all the second characteristic points, a plurality of matched characteristic point groups to be screened are determined from all the first characteristic points and all the second characteristic points, and each matched characteristic point group comprises a first characteristic point and a second characteristic point; screening all the matched characteristic point groups according to the epipolar positions corresponding to the calculated image sets to obtain at least one target matched characteristic point group, wherein all the target matched characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for building a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model. Therefore, the method and the device can acquire the image set of the target scene, extract the characteristic points from a plurality of image positions in the image set according to the characteristic point extraction algorithm which is determined in advance, obtain the characteristic point extraction result corresponding to the image set, screen the matched characteristic point set according to the Harr wavelet characteristic value of each first characteristic point in the first characteristic points in the calculated characteristic point extraction result and the Harr wavelet characteristic value of each second characteristic point in the second characteristic points in the characteristic point extraction result, accurately calculate the similarity of each first characteristic point and each second characteristic point, determine a plurality of matched characteristic point sets to be screened from all the first characteristic points and all the second characteristic points according to the similarity of all the first characteristic points and all the second characteristic points which are calculated accurately, improve the matching accuracy of the matched characteristic point sets, screen all the matched characteristic point sets according to the line positions corresponding to the calculated image set, obtain at least one target matched characteristic point set, improve the screening accuracy of the target matched characteristic point sets, further improve the three-dimensional modeling accuracy of the three-dimensional modeling object to be monitored and be better than the three-dimensional modeling object to be accurately modeled, and further improve the three-dimensional modeling accuracy of the three-dimensional modeling object to be accurately monitored and accurately.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a feature point matching method based on binocular vision according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a comparison of the image distortion correction before and after the image distortion correction according to an embodiment of the present invention;
FIG. 3 is a flow chart of another feature point matching method based on binocular vision according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a box filter according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of extremum screening according to an embodiment of the present invention;
FIG. 6 is a schematic plan view of an epipolar plane and imaging plane according to one embodiment of the present disclosure;
FIG. 7 is a schematic diagram of Harr wavelet features for determining a feature point according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a principal direction of determining a feature point according to an embodiment of the present invention;
Fig. 9 is a schematic structural diagram of a feature point matching device based on binocular vision according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of another feature point matching device based on binocular vision according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of a feature point matching device based on binocular vision according to an embodiment of the present invention.
Detailed Description
In order to make the technical solution of the present invention better understood by those skilled in the art, the technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which are obtained by persons of ordinary skill in the art without creative efforts, are within the protection scope of the present invention based on the embodiments in the present invention.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or article that comprises a list of steps or elements is not limited to only those listed but may optionally include other steps or elements not listed or inherent to such process, method, article, or article.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.
The invention discloses a feature point matching method and device based on binocular vision, which can collect an image set of a target scene, extract feature points from a plurality of image positions in the image set according to a feature point extraction algorithm determined in advance to obtain feature point extraction results corresponding to the image set, screen all matched feature point groups according to Harr wavelet feature values of each first feature point in the calculated feature point extraction results and Harr wavelet feature values of each second feature point in the second feature points in the feature point extraction results, accurately calculate similarity of each first feature point and each second feature point, determine a plurality of matched feature point groups to be screened from all the first feature points and all the second feature points according to similarity of all the first feature points and all the second feature points calculated accurately, improve matching accuracy of the matched feature point groups, screen all the matched feature point groups according to polar line positions corresponding to the calculated image set, improve the accuracy of the matched feature point groups, improve three-dimensional modeling operation of the three-dimensional object to be accurately modeled, and be better than three-dimensional object to be modeled, and the three-dimensional object to be modeled accurately and be better than three-dimensional object to be modeled. The following will describe in detail.
Example 1
Referring to fig. 1, fig. 1 is a flow chart of a feature point matching method based on binocular vision according to an embodiment of the present invention. The feature point matching method based on binocular vision described in fig. 1 may be applied to a feature point matching device based on binocular vision, where the device may include a matching device or a matching server, and the matching server may include a cloud server or a local server. As shown in fig. 1, the binocular vision-based feature point matching method may include the following operations:
101. a set of images of a target scene is acquired.
In the embodiment of the invention, the image set can comprise a first type image (such as a left-eye camera) of a binocular image acquisition device (such as a binocular camera arranged on a vehicle, an unmanned aerial vehicle and other equipment) in a target scene and a second type image (such as a right-eye camera) of the binocular image acquisition device, wherein the first type image is acquired by the first type image aiming at an object to be molded in the target scene, and the second type image is acquired by the second type image aiming at the object to be molded. Wherein the first type of image comprises one or more first images and the second type of image comprises one or more second images. The object to be molded can be a vehicle on a road, a pedestrian on the road, a mountain, a tree, a building and other objects beside the road, and the embodiment of the invention is not limited.
102. And extracting the characteristic points from a plurality of image positions in the image set according to a predetermined characteristic point extraction algorithm to obtain a characteristic point extraction result corresponding to the image set.
In the embodiment of the present invention, optionally, the feature point extraction algorithm may include one or more combinations of Harris algorithm, SIFT algorithm, SURF algorithm, ORB algorithm, and the like, which is not limited in the embodiment of the present invention.
For example, assuming that the SURF algorithm is adopted, feature points of the first-type image and the second-type image may be extracted through a Hessian matrix. Wherein the definition of the Hessian matrix is as follows:
（1）
wherein, the discriminant of the Hessian matrix can be obtained according to the above formula (1), as follows:
（2）
alternatively, since the feature points need to have scale independence, gaussian filtering is required before constructing the Hessian matrix, as follows:
（3）
wherein L is xx ，L xy L and L yy Is the second order derivative of the gaussian filtered image in all directions. By passing throughThe value of the Hessian matrix determinant can be calculated for each pixel in the image and used to discriminate the feature points. Specifically, convolution generated by the template and the image is converted into box filtering operation, and the Gaussian second-order differential template needs to be simplified, so that the simplified template is composed of only a plurality of rectangular areas, and the rectangular areas are filled with the same value. In the simplified template, the white area has a positive value, the black area has a negative value, and the gray area has a value of 0, as shown in fig. 4. To locate feature points in different sizes of the image, a 3 x 3 neighborhood non-maximum suppression is used, i.e., each pixel point is compared in size to 26 points in its three-dimensional neighborhood, as shown in fig. 5. At this time, the local extremum is preserved, and then the characteristic point of the sub-pixel level is obtained by adopting a 3-dimensional linear interpolation method.
In the embodiment of the invention, the feature point extraction result comprises a first type feature point corresponding to a first type image and a second type feature point corresponding to a second type image, wherein the first type feature point and the second type feature point are feature points of an object to be molded. Optionally, the first type of feature points include one or more first feature points, and the second type of feature points include one or more second feature points. The first feature point and the second feature point may include road signs, may include obstacles, and may include other specific protruding points of the road surface, which is not limited in the embodiment of the present invention.
For example, assuming that the first type of image includes a first image, performing feature extraction on the first image may obtain one or more first feature points of the first image.
103. And calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points.
104. And determining a plurality of matched feature point groups to be screened from all the first feature points and all the second feature points according to the similarity between all the first feature points and all the second feature points.
In the embodiment of the invention, each matching characteristic point group comprises a first characteristic point and a second characteristic point. For example, for each first feature point, a second feature point corresponding to the maximum similarity is selected from the similarities between the first feature point and all the second feature points, and the first feature point and the second feature point with the maximum similarity to the first feature point are determined as a matched feature point group to be screened.
105. And screening all the matched characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matched characteristic point group.
It should be noted that, in the embodiment of the present invention, only the epipolar position corresponding to the image set is calculated, and epipolar correction is not required for the image set.
In the embodiment of the invention, all the target matching characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for establishing a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model.
It can be seen that, implementing the feature point matching method based on binocular vision described in the embodiments of the present invention can collect an image set of a target scene, and perform feature point extraction on a plurality of image positions in the image set according to a feature point extraction algorithm determined in advance, to obtain a feature point extraction result corresponding to the image set, and screen all matched feature point groups according to a Harr wavelet feature value of each first feature point in a first type feature point in the calculated feature point extraction result and a Harr wavelet feature value of each second feature point in a second type feature point in the feature point extraction result, accurately calculate similarity of each first feature point and each second feature point, determine a plurality of matched feature point groups to be screened from all first feature points and all second feature points according to similarity of all first feature points and all second feature points calculated accurately, so as to obtain at least one target matched feature point group according to a polar line position corresponding to the calculated image set, screen all matched feature point groups, and accurately perform modeling operation to improve three-dimensional modeling accuracy, thereby being capable of improving three-dimensional modeling accuracy, and being beneficial to improving three-dimensional modeling accuracy.
In an alternative embodiment, before the capturing the image set of the target scene in step 101, the method may further include:
installing a binocular image acquisition device in a target scene, and calibrating the binocular image acquisition device to obtain calibration parameters of the binocular image acquisition device;
and carrying out distortion correction on the image acquired by the binocular image acquisition device according to the calibration parameters of the binocular image acquisition device.
In the embodiment of the present invention, optionally, the calibration parameters of the binocular image capturing apparatus may include one or more of a distortion parameter of the binocular image capturing apparatus, an internal reference matrix of the binocular image capturing apparatus, and an external reference matrix of the binocular image capturing apparatus. The distortion parameter is an error in the installation and assembly of the binocular image capturing apparatus (the lens of the binocular image capturing apparatus often cannot satisfy an ideal linear relationship between an object and an image, which may cause distortion of the image). The distortion parameters can be obtained by calibrating the binocular image acquisition device, and then the binocular image acquisition device is subjected to distortion correction according to the distortion parameters, so that the error can be eliminated, wherein images which can be acquired by the binocular image acquisition device before and after the distortion correction can be shown in fig. 2. The image shown in (a) in fig. 2 is an image that can be acquired by the binocular image acquisition device before the distortion correction is performed, and the image shown in (b) in fig. 2 is an image that can be acquired by the binocular image acquisition device after the distortion correction is performed, which is not limited in the embodiment of the present invention.
Therefore, the optional embodiment can calibrate the binocular image acquisition device installed in the target scene to obtain the calibration parameters of the binocular image acquisition device, and correct the distortion of the image acquired by the binocular image acquisition device according to the calibration parameters of the binocular image acquisition device, so that the accuracy of correcting the distortion of the image acquired by the binocular image acquisition device can be improved, the accuracy and the reliability of extracting the feature points of the corrected image can be improved, and the occurrence of the situation of extracting the error of the feature points of the image caused by the distortion correction error can be reduced.
In another optional embodiment, in the step 102, feature point extraction is performed on a plurality of image positions in the image set according to a predetermined feature point extraction algorithm, so as to obtain a feature point extraction result corresponding to the image set, which may include:
according to the SURF algorithm, extracting feature points from a plurality of image positions in an image set to obtain an initial feature point extraction result corresponding to the image set;
according to the predetermined rest feature point extraction algorithm, verifying the initial feature point extraction result corresponding to the image set to obtain a verification result;
When the verification result indicates that verification is passed, determining an initial feature point extraction result corresponding to the image set as a target feature point extraction result, and triggering and executing the operation of calculating the similarity between each first feature point and each second feature point according to the calculated Harr wavelet feature value of each first feature point of the first type feature point and the Harr wavelet feature value of each second feature point of the second type feature point in the step 103;
and when the verification result indicates that verification fails, updating the initial feature point extraction result according to the verification result to obtain an updated initial feature point extraction result, triggering and executing the rest feature point extraction algorithm which is determined in advance, and verifying the initial feature point extraction result corresponding to the image set to obtain a verification result.
In the embodiment of the present invention, optionally, the remaining feature point extraction algorithm may include one or more combinations of Harris algorithm, SIFT algorithm and ORB algorithm, which is not limited in the embodiment of the present invention.
As can be seen, in this optional embodiment, feature point extraction can be performed on a plurality of image positions in an image set according to the SURF algorithm, so as to obtain an initial feature point extraction result corresponding to the image set, and according to the rest of feature point extraction algorithms, the initial feature point extraction result corresponding to the image set is checked, so as to obtain a check result, and when the check is passed, step 103 is executed; when the verification fails, updating the initial feature point extraction result according to the verification result, continuously executing the rest feature point extraction algorithm which is determined in advance, verifying the initial feature point extraction result corresponding to the image set, and obtaining the verification result, so that the accuracy of verifying the initial feature point extraction result can be improved, and the occurrence of feature point extraction errors caused by verification errors can be reduced.
Example two
Referring to fig. 3, fig. 3 is a flow chart of a feature point matching method based on binocular vision according to an embodiment of the present invention. The feature point matching method based on binocular vision described in fig. 3 may be applied to a feature point matching device based on binocular vision, where the device may include a feature point matching device based on binocular vision or a feature point matching server based on binocular vision, where the feature point matching server based on binocular vision may include a cloud server or a local server, and the embodiment of the present invention is not limited. As shown in fig. 3, the binocular vision-based feature point matching method may include the following operations:
201. a set of images of a target scene is acquired.
202. And extracting the characteristic points from a plurality of image positions in the image set according to a predetermined characteristic point extraction algorithm to obtain a characteristic point extraction result corresponding to the image set.
203. And calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points.
204. And determining a plurality of matched feature point groups to be screened from all the first feature points and all the second feature points according to the similarity between all the first feature points and all the second feature points.
205. And screening all the matched characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matched characteristic point group.
In the embodiment of the present invention, for other descriptions of step 201 to step 205, please refer to the detailed descriptions of step 101 to step 105 in the first embodiment, and the description of the embodiment of the present invention is omitted.
206. And for each target matching characteristic point group, acquiring a first initial coordinate corresponding to a first characteristic point of the target matching characteristic point group and a second initial coordinate corresponding to a second characteristic point of the target matching characteristic point group.
In the embodiment of the present invention, optionally, the first initial coordinate corresponding to the first feature point of each target matching feature point group and the second initial coordinate corresponding to the second feature point of the target matching feature point group may be pixel coordinates, which is not limited in the embodiment of the present invention.
207. And carrying out coordinate conversion on the first initial coordinates corresponding to the first feature points of the target matching feature point group according to a preset coordinate conversion algorithm to obtain first target coordinates corresponding to the target matching feature point group.
In the embodiment of the present invention, optionally, the coordinate transformation algorithm may be a transformation algorithm corresponding to the projection matrix. For example, assume that the first projection matrix of the first image acquisition device is P 1 The first image acquisition deviceThe two projection matrixes are P 2 The specific coordinate transformation formula may be as follows:
（4）
wherein, the formula (4) can be converted into two formulas as follows:
（5）
（6）
according to formula (4), for Z in formula (5) c1 And Z in formula (6) c2 The elimination can be performed to obtain the following formula:
（7）
the above formula (4) -formula (7) can be solved by a least square method to obtain second target coordinates (for example, three-dimensional world coordinates) of the first feature points in each target matching feature point group. The second target coordinates of the second feature points in the target matching feature point group may also be calculated by the above method.
208. And carrying out coordinate conversion on second initial coordinates corresponding to the second feature points of the target matching feature point group according to a preset coordinate conversion algorithm to obtain second target coordinates corresponding to the target matching feature point group.
It should be noted that, the step 207 and the step 208 are not sequentially executed, that is, the step 207 may be executed first and then the step 208 may be executed first, the step 208 may be executed then the step 207 may be executed, and the step 207 and the step 208 may be executed simultaneously, which is not limited in the embodiment of the present invention.
209. And carrying out three-dimensional modeling on all the target matching characteristic point groups according to the first target coordinates corresponding to all the target matching characteristic point groups and the second target coordinates corresponding to all the target matching characteristic point groups to obtain a three-dimensional model of the object to be modeled.
For example, if the binocular image capturing apparatus is applied to a landslide detection binocular camera, three-dimensional modeling can be performed on a mountain to obtain a three-dimensional model of the mountain. The three-dimensional model is used for executing corresponding supervision operation on the object to be modeled according to the determined three-dimensional information of the object to be modeled, and the embodiment of the invention is not limited.
For example, when the three-dimensional model of the object to be modeled is a three-dimensional model of a mountain, three-dimensional information of the mountain can be obtained from the three-dimensional model of the mountain, and whether the mountain is at risk of landslide is analyzed according to the three-dimensional information of the mountain. Further, if it is analyzed that the mountain is at risk of landslide, vehicles on the road surface near the mountain may be pre-warned to prompt the vehicles to perform obstacle avoidance operations (such as bypassing the mountain, emergency stopping, etc.).
It can be seen that, implementing the feature point matching method based on binocular vision described in the embodiments of the present invention can collect an image set of a target scene, and perform feature point extraction on a plurality of image positions in the image set according to a feature point extraction algorithm determined in advance, to obtain a feature point extraction result corresponding to the image set, and screen all matched feature point groups according to a Harr wavelet feature value of each first feature point in a first type feature point in the calculated feature point extraction result and a Harr wavelet feature value of each second feature point in a second type feature point in the feature point extraction result, accurately calculate similarity of each first feature point and each second feature point, determine a plurality of matched feature point groups to be screened from all first feature points and all second feature points according to similarity of all first feature points and all second feature points calculated accurately, so as to obtain at least one target matched feature point group according to a polar line position corresponding to the calculated image set, screen all matched feature point groups, and accurately perform modeling operation to improve three-dimensional modeling accuracy, thereby being capable of improving three-dimensional modeling accuracy, and being beneficial to improving three-dimensional modeling accuracy. In addition, the first initial coordinates corresponding to the first feature points of each target matching feature point group and the second initial coordinates corresponding to the second feature points of the target matching feature point group can be obtained, and the first initial coordinates corresponding to the first feature points of the target matching feature point group are subjected to coordinate conversion according to a preset coordinate conversion algorithm to obtain the first target coordinates corresponding to the target matching feature point group, and the second initial coordinates corresponding to the second feature points of the target matching feature point group are subjected to coordinate conversion to obtain the second target coordinates corresponding to the target matching feature point group.
In an alternative embodiment, each first feature point has a corresponding first image position and each second feature point has a corresponding second image position. And in the step 205, screening all the matching feature point groups according to the epipolar positions corresponding to the calculated image set to obtain at least one target matching feature point group, which may include:
for each matching feature point group, calculating the distance between the matching feature point group and the epipolar position according to the first image position corresponding to the first feature point of the matching feature point group, the second image position corresponding to the second feature point of the matching feature point group and the epipolar position corresponding to the calculated image set, and obtaining the distance corresponding to the matching feature point group;
judging whether the distance corresponding to the matching characteristic point group is larger than or equal to a preset distance;
if the distance corresponding to the matched characteristic point group is larger than or equal to the preset distance, deleting the matched characteristic point group;
and if the distance corresponding to the matched characteristic point group is smaller than the preset distance, determining the matched characteristic point group as a target matched characteristic point group.
It should be noted that, in this alternative embodiment, the selected target matching feature point set needs to satisfy the condition of the limit constraint. Optionally, the preset distance may be 0.05 meters, or 0.01 meters, or other preset distance values, which are not limited in the embodiment of the present invention.
It can be seen that, in this alternative embodiment, the distance between each matching feature point group and the epipolar position corresponding to the image set can be calculated according to the first image position corresponding to the first feature point of each matching feature point group, the second image position corresponding to the second feature point of the matching feature point group, and the epipolar position corresponding to the image set, so as to obtain the distance corresponding to the matching feature point group, so that the calculation accuracy of the distance corresponding to the matching feature point group can be improved, whether the distance corresponding to the matching feature point group is greater than or equal to the preset distance can be judged, so as to obtain a judgment result, and the matching feature point group can be intelligently selected and deleted, or the matching feature point group can be intelligently selected and determined as the target matching feature point group, so that the screening accuracy and reliability of the target matching feature point group can be improved, and the intelligent degree of the screening target matching feature point group can also be improved.
In this alternative embodiment, as an alternative implementation manner, the method may further include:
acquiring position parameters corresponding to a binocular image acquisition device, a first imaging plane corresponding to a first image and a second imaging plane corresponding to a second image;
Determining an epipolar plane corresponding to the image set according to the position parameters corresponding to the binocular image acquisition device;
and calculating the polar line position corresponding to the image set according to the first imaging plane corresponding to the first image, the second imaging plane corresponding to the second image and the epipolar plane corresponding to the image set.
In the embodiment of the present invention, optionally, the position parameters corresponding to the binocular image capturing device may include a target position where the object to be modeled is located, a first position where the first image capturing device is located, and a second position where the second image capturing device is located.
In the embodiment of the present invention, as shown in fig. 6, P in fig. 6 is a three-dimensional target point corresponding to a real object to be modeled, O is a focal point of the first image capturing device, O ' is a focal point of the second image capturing device, pi is an imaging plane of the first image capturing device, pi ' is an imaging plane of the second image capturing device, and P ' on the plane are imaging corresponding points of the solid point P. In order to know the position information of the polar line l 'corresponding to the P point or obtain the position information of the polar line l from the P', it is necessary to know the coordinates of the three planes (two imaging planes and one epipolar plane).
For example, assuming that the pixel coordinates of the P-point are known, the pixel coordinates of the line l' are required.
The relationship of P point relative to the O point camera coordinate system can be found by the following formula:
（8）
furthermore, the position of the imaging planes pi and pi 'with respect to the O-point and O' -point camera coordinate systems, respectively, is also known, i.e. a plane parallel to the X, Y plane, with a Z-axis intercept f (f being the effective focal length, i.e. the distance of the optical center from the image plane). At this time, we need to know the relationship between the O-point camera coordinate system and the O' -point camera coordinate system, which can be represented by an eigenmatrix E (the physical meaning of the eigenmatrix E is a matrix of mutual conversion of the left and right image coordinate systems, and describes the relationship between corresponding points of the same spatial point projection on the left and right camera image planes, with the unit of mm).
Let the coordinates of the camera coordinate system of the P point in the left camera (i.e. O point) be P cl The coordinates of the camera coordinate system of the P point in the right camera (namely the O' point) are P cr Then P point is at left cameraThe relational expression with the right camera is as follows:
（9）
the E can be obtained by calibrating the camera, namely, the E can be converted by an external parameter matrix. The positions of the image planes pi 'and O' in the left camera coordinate system can be found by the eigenvmatrix. And knowing the coordinates of P and O' in the left camera coordinate system, determining a plane according to the two intersecting straight lines, and obtaining the position of the polar plane in the left camera coordinate system. At this time, the linear equation of l 'in the left camera coordinate system can be constrained by the plane equations of the polar plane and the image plane pi'. And finally, substituting the pixel coordinate into the formula (9) again, and performing axis shifting and unit pixel drawing to obtain the pixel coordinate of l 'on the image plane pi'.
Therefore, according to the alternative embodiment, the corresponding position parameters of the binocular image acquisition device, the first imaging plane corresponding to the first image and the second imaging plane corresponding to the second image can be obtained, the epipolar plane corresponding to the image set is accurately determined according to the corresponding position parameters of the binocular image acquisition device, and the epipolar plane corresponding to the image set is accurately determined according to the first imaging plane corresponding to the first image, the second imaging plane corresponding to the second image and the epipolar plane corresponding to the accurately determined image set, the polar line position corresponding to the image set is calculated, so that the calculation accuracy of the polar line position can be improved, and the accuracy and the reliability of screening the matched characteristic point group to be screened can be improved through the accurately calculated polar line position.
In another optional embodiment, before screening all the matching feature point groups according to the epipolar positions corresponding to the computed image sets in step 205 to obtain at least one target matching feature point group, the method may further include:
for each matching feature point group, judging whether the first feature point of the matching feature point group is matched with the second feature point of the matching feature point group;
If the first characteristic point of the matched characteristic point group is not matched with the second characteristic point of the matched characteristic point group, deleting the matched characteristic point group;
if the first characteristic point of the matching characteristic point group is matched with the second characteristic point of the matching characteristic point group, the matching characteristic point group is determined to be an alternative matching characteristic point group, and the above-mentioned operations of screening all the matching characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matching characteristic point group are triggered and executed.
In the embodiment of the invention, specifically, the above-mentioned trigger is executed according to the polar line position corresponding to the calculated image set, and all the matching characteristic point groups are screened to obtain at least one operation of the target matching characteristic point group, wherein all the matching characteristic point groups in the operation are candidate matching characteristic point groups.
It can be seen that, this optional embodiment can determine whether the first feature point of each matching feature point group is matched with the second feature point of the matching feature point group, so as to obtain a determination result, and intelligently select to delete the matching feature point group according to the determination result, or intelligently select to determine the matching feature point group as an alternative matching feature point group, and trigger to execute a screening operation for the matching feature point group, so that the speed and accuracy of screening the alternative matching feature point group can be improved, thereby being beneficial to improving the speed and accuracy of further screening the correctly screened alternative matching feature point group.
In this optional embodiment, as an optional implementation manner, for each matching feature point group, determining whether the first feature point of the matching feature point group matches the second feature point of the matching feature point group may include:
for each matching feature point group, selecting one of the first feature point and the second feature point of the matching feature point group as a reference feature point of the matching feature point group;
determining the images of the rest characteristic points except the reference characteristic point in all the characteristic points of the matched characteristic point group as reference images;
judging whether the reference characteristic points of the matched characteristic point group exist in the reference image or not;
if the reference characteristic points of the matching characteristic point group exist in the reference image, judging whether the residual characteristic points of the matching characteristic point group exist in the residual images except the reference image in the images of all the characteristic points of the matching characteristic point group;
if the residual feature points of the matched feature point group exist in the residual image, determining that the first feature points of the matched feature point group are matched with the second feature points of the matched feature point group;
and if the residual characteristic points of the matched characteristic point group are not found in the residual image or the reference characteristic points of the matched characteristic point group are not found in the reference image, determining that the first characteristic points of the matched characteristic point group are not matched with the second characteristic points of the matched characteristic point group.
For example, if a certain first feature point a in the first type of image finds a matching second feature point B in the second type of image, and the second feature point B in the second type of image can also find a matching first feature point a in the first type of image, it may be determined that the first feature point a and the second feature point B match.
As can be seen, the optional implementation manner can select one of the first feature point and the second feature point of each matching feature point group as a reference feature point of the matching feature point group, determine an image in which the remaining feature points except the reference feature point are located in all the feature points of the matching feature point group as a reference image, and determine whether the reference feature point of the matching feature point group is present in the reference image, if the reference feature point of the matching feature point group is present in the reference image, further determine whether the remaining feature point of the matching feature point group is present in the remaining images except the reference image in all the images, if the remaining feature point of the matching feature point group is present in the remaining images, determine that the first feature point of the matching feature point group is matched with the second feature point of the matching feature point group, and if the remaining feature point of the matching feature point group is not present in the remaining images, or if the reference feature point of the matching feature point group is not present in the reference image, determine whether the first feature point of the matching feature point group is not present in the remaining images is wrong or not present in the remaining images, and accordingly, if the first feature point of the first feature point group is not matched with the second feature point of the matching feature point group is not present in the remaining images is not present, and the second feature point is not matching with the second feature point is not present in the matching feature point group is found, and the matching feature point is not matching is found, and is good.
In yet another alternative embodiment, the method may further comprise:
for each feature point in all first feature points of the first type feature points and all second feature points of the second type feature points, acquiring a Harr wavelet response value set of the feature point, wherein the Harr wavelet response value set of each feature point comprises Harr wavelet response values of each neighborhood point in all neighborhood points of the feature point, and each neighborhood point has a corresponding neighborhood position;
for each feature point, calculating a plurality of vector directions of the feature point according to Harr wavelet response values of all neighborhood points of the feature point and neighborhood positions corresponding to all neighborhood points of the feature point;
screening out the vector direction with the maximum corresponding value from all the vector directions of the feature points, wherein the vector direction is used as the main direction of the feature points, and the main direction has the corresponding horizontal direction and vertical direction;
and calculating the Harr wavelet characteristic value of the characteristic point according to all Harr wavelet response values in the horizontal direction corresponding to the main direction of the characteristic point and all Harr wavelet response values in the vertical direction corresponding to the main direction of the characteristic point.
For example, for each feature point, a square frame is taken around the feature point, the side length of the frame is 20s (s is the scale value where the feature point is located), and the direction of the frame band of the square frame of the feature point is the main direction of the feature point. The square box is then divided into 16 sub-regions, each sub-region counting the horizontal and vertical harr wavelet characteristics of 25 pixels (both horizontal and vertical are relative to the main direction). The harr wavelet characteristics include the sum of horizontal direction values Sum of absolute values in horizontal direction->Sum of vertical directions->And the sum of absolute values in the vertical direction +.>A schematic of this process is shown in fig. 7.
Thus, there are 4 values per sub-region, so each feature point is a vector of 16×4=64 dimensions. By comparing feature descriptors, feature points in the two images that are identical or similar to the descriptors can be considered as matching feature point sets.
It can be seen that, in this optional embodiment, the Harr wavelet response value set of each feature point can be obtained, and according to the Harr wavelet response values of all the neighborhood points of the feature point and the neighborhood positions corresponding to all the neighborhood points of the feature point, a plurality of vector directions of the feature point are calculated, so that the calculation accuracy of the vector directions of the feature point can be improved, and the vector direction with the largest corresponding value is screened out from all the vector directions of the feature point, and used as the main direction of the feature point, the screening accuracy of the vector directions of each feature point can be improved, and according to all the Harr wavelet response values in the horizontal direction corresponding to the main direction of the feature point, the Harr wavelet characteristic value of the feature point is calculated, so that the calculation accuracy and speed of the Harr wavelet characteristic value of each feature point can be improved. Therefore, the method is beneficial to improving the calculation accuracy and reliability of the similarity between all the first characteristic points and all the second characteristic points through the Harr wavelet characteristic values of all the characteristic points which are calculated accurately.
In this optional embodiment, as an optional implementation manner, for each feature point, calculating, according to Harr wavelet response values of all neighboring points of the feature point and neighboring positions corresponding to all neighboring points of the feature point, a plurality of vector directions of the feature point may include:
for each feature point, acquiring a weight coefficient of each neighborhood point of the feature point;
for each feature point, grouping all the neighborhood points of the feature point according to the neighborhood positions corresponding to all the neighborhood points of the feature point and a plurality of preset neighborhood position ranges to obtain a plurality of neighborhood groups of the feature point, wherein all the neighborhood points in the same neighborhood group correspond to the same neighborhood position range;
for each neighborhood group of each feature point, calculating the product of the weight coefficient of each neighborhood point in the neighborhood group and the weight coefficient of the neighborhood point, and calculating the sum of the products of all the neighborhood points in the neighborhood group and the weight coefficients of all the neighborhood points to obtain the vector direction of the neighborhood group, wherein the vector directions of each feature point comprise the vector direction of each neighborhood group in the neighborhood groups of each feature point.
For example, for each feature point, taking the feature point as the center, calculating a Harr wavelet response value set (the Harr wavelet side length is taken as 4 s) of all points in a 60-degree sector in a neighborhood with a radius of 6s (s is the scale value of the feature point), giving a gaussian weight coefficient to the response values, so that the response contribution close to the feature point is large, the response contribution far from the feature point is small, and then adding all the response values in the 60-degree sector to form a new vector, so as to obtain the vector direction of the feature point in the 60-degree sector. The multiple vector directions of the feature point can be obtained by traversing the entire circular area, as shown in fig. 8.
Therefore, the optional implementation manner can obtain the weight coefficient of each neighborhood point of each feature point, group all the neighborhood points of the feature point according to the neighborhood positions corresponding to all the neighborhood points of the feature point and the preset multiple neighborhood position ranges, obtain multiple neighborhood groups of the feature point, improve the grouping accuracy of all the neighborhood groups of each feature point, calculate the product of each neighborhood point in each neighborhood group of the feature point and the weight coefficient of each neighborhood point, calculate the sum of the products of all the neighborhood points in the neighborhood group and the weight coefficient of all the neighborhood points, and obtain the vector direction of the neighborhood group, thereby improving the calculation accuracy and reliability of all the vector directions of each feature point, and being beneficial to reducing the occurrence of the screening error of the main direction of the feature point caused by the calculation error of the vector direction of each feature point.
Example III
Referring to fig. 9, fig. 9 is a schematic structural diagram of a feature point matching device based on binocular vision according to an embodiment of the present invention. The feature point matching device based on binocular vision described in fig. 9 may include a matching device or a matching server, where the matching server may include a cloud server or a local server, and the embodiment of the present invention is not limited. As shown in fig. 9, the binocular vision-based feature point matching apparatus may include:
The acquisition module 301 is configured to acquire an image set of a target scene, where the image set includes a first type of image for an object to be molded in the target scene acquired by a first image acquisition device in a binocular image acquisition device in the target scene and a second type of image for the object to be molded acquired by a second image acquisition device in the binocular image acquisition device.
The extracting module 302 is configured to extract feature points from a plurality of image positions in the image set according to a predetermined feature point extracting algorithm, so as to obtain a feature point extracting result corresponding to the image set, where the feature point extracting result includes a first type feature point corresponding to the first type image and a second type feature point corresponding to the second type image, and the first type feature point and the second type feature point are feature points of the object to be molded.
The calculating module 303 is configured to calculate a similarity between each first feature point and each second feature point according to the calculated Harr wavelet feature value of each first feature point of the first class feature points and the calculated Harr wavelet feature value of each second feature point of the second class feature points.
The determining module 304 is configured to determine a plurality of matching feature point groups to be screened from all the first feature points and all the second feature points according to the similarity between all the first feature points and all the second feature points, where each matching feature point group includes a first feature point and a second feature point.
The screening module 305 is configured to screen all the matching feature point groups according to the epipolar positions corresponding to the computed image set, to obtain at least one target matching feature point group, where all the target matching feature point groups are used to perform a three-dimensional modeling operation on an object to be modeled, and the three-dimensional modeling operation is used to build a three-dimensional model of the object to be modeled and determine three-dimensional information of the object to be modeled according to the three-dimensional model.
It can be seen that, implementing the feature point matching device based on binocular vision described in the embodiment of the present invention can collect an image set of a target scene, and perform feature point extraction on a plurality of image positions in the image set according to a feature point extraction algorithm determined in advance, to obtain a feature point extraction result corresponding to the image set, and screen all matched feature point groups according to a Harr wavelet feature value of each first feature point in a first type feature point in the calculated feature point extraction result and a Harr wavelet feature value of each second feature point in a second type feature point in the feature point extraction result, accurately calculate similarity of each first feature point and each second feature point, determine a plurality of matched feature point groups to be screened from all first feature points and all second feature points according to similarity of all first feature points and all second feature points calculated accurately, so as to obtain at least one target matched feature point group according to a polar line position corresponding to the calculated image set, screen all matched feature point groups, and accurately perform modeling operation to improve three-dimensional modeling accuracy, thereby being capable of improving three-dimensional modeling accuracy, and further being beneficial to three-dimensional, and further improving three-dimensional modeling accuracy.
In an alternative embodiment, each first feature point has a corresponding first image position and each second feature point has a corresponding second image position. And, the screening module 305 screens all the matching feature point groups according to the epipolar position corresponding to the computed image set, and the method for obtaining at least one target matching feature point group may specifically include:
for each matching feature point group, calculating the distance between the matching feature point group and the epipolar position according to the first image position corresponding to the first feature point of the matching feature point group, the second image position corresponding to the second feature point of the matching feature point group and the epipolar position corresponding to the calculated image set, and obtaining the distance corresponding to the matching feature point group;
judging whether the distance corresponding to the matching characteristic point group is larger than or equal to a preset distance;
if the distance corresponding to the matched characteristic point group is larger than or equal to the preset distance, deleting the matched characteristic point group;
and if the distance corresponding to the matched characteristic point group is smaller than the preset distance, determining the matched characteristic point group as a target matched characteristic point group.
Therefore, the optional embodiment can improve the calculation accuracy of the distance corresponding to the matching feature point group, judge whether the distance corresponding to the matching feature point group is greater than or equal to the preset distance, obtain a judging result, and intelligently select and delete the matching feature point group according to the judging result, or intelligently select and determine the matching feature point group as the target matching feature point group, so that the screening accuracy and reliability of the target matching feature point group can be improved, and the intelligent degree of screening the target matching feature point group can also be improved.
In this alternative embodiment, as an alternative implementation, as shown in fig. 10, the apparatus may further include:
the first obtaining module 306 is configured to obtain a position parameter corresponding to the binocular image capturing device, a first imaging plane corresponding to the first image, and a second imaging plane corresponding to the second image, where the position parameter corresponding to the binocular image capturing device includes a target position where the object to be modeled is located, a first position where the first image capturing device is located, and a second position where the second image capturing device is located;
the determining module 304 is further configured to determine an epipolar plane corresponding to the image set according to a position parameter corresponding to the binocular image capturing device;
the calculating module 303 is further configured to calculate a epipolar position corresponding to the image set according to the first imaging plane corresponding to the first image, the second imaging plane corresponding to the second image, and the epipolar plane corresponding to the image set.
Therefore, the optional implementation manner can accurately calculate the polar line position corresponding to the image set, so that the accuracy and the reliability of screening the matched characteristic point group to be screened are improved through the accurately calculated polar line position.
In another alternative embodiment, as shown in fig. 10, the apparatus may further include:
A judging module 307, configured to judge, for each matching feature point group, whether a first feature point of the matching feature point group is matched with a second feature point of the matching feature point group before the screening module screens all the matching feature point groups according to the epipolar positions corresponding to the computed image set to obtain at least one target matching feature point group;
a deleting module 308, configured to delete the matched feature point set if the judging module 307 judges that the first feature point of the matched feature point set is not matched with the second feature point of the matched feature point set;
the determining module 304 is further configured to determine the matching feature point set as an alternative matching feature point set if the judging module 307 judges that the first feature point of the matching feature point set matches the second feature point of the matching feature point set, and trigger the performing operation of screening all the matching feature point sets according to the epipolar position corresponding to the calculated image set to obtain at least one target matching feature point set.
It can be seen that, this optional embodiment can determine whether the first feature point of each matching feature point group is matched with the second feature point of the matching feature point group, so as to obtain a determination result, and intelligently select to delete the matching feature point group according to the determination result, or intelligently select to determine the matching feature point group as an alternative matching feature point group, and trigger to execute a screening operation for the matching feature point group, so that the speed and accuracy of screening the alternative matching feature point group can be improved, thereby being beneficial to improving the speed and accuracy of further screening the correctly screened alternative matching feature point group.
In this optional embodiment, as an optional implementation manner, the determining module 307 may specifically determine, for each matching feature point group, whether the first feature point of the matching feature point group matches the second feature point of the matching feature point group, by including:
for each matching feature point group, selecting one of the first feature point and the second feature point of the matching feature point group as a reference feature point of the matching feature point group;
determining the images of the rest characteristic points except the reference characteristic point in all the characteristic points of the matched characteristic point group as reference images;
judging whether the reference characteristic points of the matched characteristic point group exist in the reference image or not;
if the reference characteristic points of the matching characteristic point group exist in the reference image, judging whether the residual characteristic points of the matching characteristic point group exist in the residual images except the reference image in the images of all the characteristic points of the matching characteristic point group;
if the residual feature points of the matched feature point group exist in the residual image, determining that the first feature points of the matched feature point group are matched with the second feature points of the matched feature point group;
And if the residual characteristic points of the matched characteristic point group are not found in the residual image or the reference characteristic points of the matched characteristic point group are not found in the reference image, determining that the first characteristic points of the matched characteristic point group are not matched with the second characteristic points of the matched characteristic point group.
As can be seen, this alternative embodiment can determine whether the reference feature point of the matching feature point group exists in the reference image, if it is determined that the reference feature point of the matching feature point group exists in the reference image, it is further determined whether the residual feature point of the matching feature point group exists in the residual images except for the reference image in all the images, if it is determined that the residual feature point of the matching feature point group exists in the residual images, it is possible to determine that the first feature point of the matching feature point group is matched with the second feature point of the matching feature point group, and if it is determined that the residual feature point of the matching feature point group does not exist in the residual images, or if it is determined that the reference feature point of the matching feature point group does not exist in the reference image, it is possible to determine that the first feature point of the matching feature point group is not matched with the second feature point of the matching feature point group, and it is possible to improve the accuracy and reliability of determining whether the first feature point of each matching feature point group is matched with the second feature point of the matching feature point group, thereby being beneficial to reduce the occurrence of the candidate feature point group that has been selected by the error.
In yet another alternative embodiment, as shown in fig. 10, the apparatus may further include:
a second obtaining module 309, configured to obtain, for each feature point in all first feature points of the first type feature points and all second feature points of the second type feature points, a Harr wavelet response value set of the feature point, where the Harr wavelet response value set of each feature point includes a Harr wavelet response value of each neighborhood point in all neighborhood points in a neighborhood of the feature point, and each neighborhood point has a corresponding neighborhood position;
the calculating module 303 is further configured to calculate, for each feature point, a plurality of vector directions of the feature point according to Harr wavelet response values of all neighboring points of the feature point and neighboring positions corresponding to all neighboring points of the feature point;
the screening module 305 is further configured to screen a vector direction with the largest corresponding value from all vector directions of the feature point, where the vector direction is used as a main direction of the feature point, and the main direction has a corresponding horizontal direction and a corresponding vertical direction;
the calculating module 303 is further configured to calculate the Harr wavelet characteristic value of the feature point according to all the Harr wavelet response values in the horizontal direction corresponding to the main direction of the feature point and all the Harr wavelet response values in the vertical direction corresponding to the main direction of the feature point.
It can be seen that the alternative embodiment can accurately calculate a plurality of vector directions of the feature point, accurately screen out the vector direction with the largest corresponding value from all vector directions of the feature point, use the vector direction as the main direction of the feature point, and accurately calculate the Harr wavelet characteristic value of the feature point according to all Harr wavelet response values in the horizontal direction corresponding to the main direction of the feature point. Therefore, the method is beneficial to improving the calculation accuracy and reliability of the similarity between all the first characteristic points and all the second characteristic points through the Harr wavelet characteristic values of all the characteristic points which are calculated accurately.
In this optional embodiment, as an optional implementation manner, for each feature point, the calculating module 303 may specifically calculate, according to Harr wavelet response values of all neighboring points of the feature point and neighboring positions corresponding to all neighboring points of the feature point, a plurality of vector directions of the feature point by:
for each feature point, acquiring a weight coefficient of each neighborhood point of the feature point;
for each feature point, grouping all the neighborhood points of the feature point according to the neighborhood positions corresponding to all the neighborhood points of the feature point and a plurality of preset neighborhood position ranges to obtain a plurality of neighborhood groups of the feature point, wherein all the neighborhood points in the same neighborhood group correspond to the same neighborhood position range;
For each neighborhood group of each feature point, calculating the product of the weight coefficient of each neighborhood point in the neighborhood group and the weight coefficient of the neighborhood point, and calculating the sum of the products of all the neighborhood points in the neighborhood group and the weight coefficients of all the neighborhood points to obtain the vector direction of the neighborhood group, wherein the vector directions of each feature point comprise the vector direction of each neighborhood group in the neighborhood groups of each feature point.
Therefore, the optional implementation manner can calculate the product of each neighborhood point in each neighborhood group of each feature point and the weight coefficient of the neighborhood point, and calculate the sum of the products of all the neighborhood points in the neighborhood group and the weight coefficients of all the neighborhood points to obtain the vector direction of the neighborhood group, so that the calculation accuracy and reliability of all the vector directions of each feature point can be improved, and the occurrence of the situation of screening errors of the main direction of each feature point caused by the calculation errors of the vector direction of each feature point is reduced.
In yet another alternative embodiment, as shown in fig. 10, the apparatus may further include:
the third obtaining module 310 is further configured to, after the screening module 305 screens all the matching feature point groups according to the epipolar positions corresponding to the computed image sets to obtain at least one target matching feature point group, obtain, for each target matching feature point group, a first initial coordinate corresponding to a first feature point of the target matching feature point group and a second initial coordinate corresponding to a second feature point of the target matching feature point group;
The conversion module 311 is configured to perform coordinate conversion on a first initial coordinate corresponding to a first feature point of the target matching feature point set according to a preset coordinate conversion algorithm to obtain a first target coordinate corresponding to the target matching feature point set, and perform coordinate conversion on a second initial coordinate corresponding to a second feature point of the target matching feature point set to obtain a second target coordinate corresponding to the target matching feature point set;
the modeling module 312 is configured to perform three-dimensional modeling on all the target matching feature point groups according to the first target coordinates corresponding to all the target matching feature point groups and the second target coordinates corresponding to all the target matching feature point groups, so as to obtain a three-dimensional model of the object to be modeled, where the three-dimensional model is configured to perform a corresponding supervision operation on the object to be modeled according to the determined three-dimensional information of the object to be modeled.
Therefore, the accuracy and reliability of converting the initial coordinate corresponding to each feature point into the target coordinate can be improved, so that the modeling accuracy of the three-dimensional model of the object to be modeled can be improved through all target coordinates of all target matching feature point groups obtained through accurate conversion, the monitoring accuracy of the object to be modeled can be improved through the three-dimensional model of the object to be modeled, and the occurrence of safety accidents of a target scene can be reduced.
Example IV
Referring to fig. 11, fig. 11 is a schematic structural diagram of a feature point matching device based on binocular vision according to an embodiment of the present invention. As shown in fig. 11, the binocular vision-based feature point matching apparatus may include:
a memory 401 storing executable program codes;
a processor 402 coupled with the memory 401;
the processor 402 invokes the executable program code stored in the memory 401 to perform the steps in the binocular vision based feature point matching method described in the first or second embodiment of the present invention.
Example five
The embodiment of the invention discloses a computer storage medium which stores computer instructions for executing the steps in the feature point matching method based on binocular vision described in the first or second embodiment of the invention when the computer instructions are called.
Example six
An embodiment of the present invention discloses a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the binocular vision based feature point matching method described in the first or second embodiment.
The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above detailed description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product that may be stored in a computer-readable storage medium including Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium that can be used for computer-readable carrying or storing data.
Finally, it should be noted that: the embodiment of the invention discloses a feature point matching method and device based on binocular vision, which are disclosed by the embodiment of the invention only and are only used for illustrating the technical scheme of the invention, but not limiting the technical scheme; while the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that; the technical scheme recorded in the various embodiments can be modified or part of technical features in the technical scheme can be replaced equivalently; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.

Claims (10)

1. A feature point matching method based on binocular vision, the method comprising:
acquiring an image set of a target scene, wherein the image set comprises a first type image of an object to be molded in the target scene, which is acquired by a first image acquisition device in a binocular image acquisition device in the target scene, and a second type image of the object to be molded, which is acquired by a second image acquisition device in the binocular image acquisition device;
Extracting feature points from a plurality of image positions in the image set according to a predetermined feature point extraction algorithm to obtain feature point extraction results corresponding to the image set, wherein the feature point extraction results comprise first type feature points corresponding to the first type image and second type feature points corresponding to the second type image, and the first type feature points and the second type feature points are feature points of the object to be molded;
calculating the similarity between each first characteristic point and each second characteristic point according to the calculated Harr wavelet characteristic value of each first characteristic point of the first type characteristic points and the Harr wavelet characteristic value of each second characteristic point of the second type characteristic points;
determining a plurality of matched characteristic point groups to be screened from all the first characteristic points and all the second characteristic points according to the similarity between all the first characteristic points and all the second characteristic points, wherein each matched characteristic point group comprises a first characteristic point and a second characteristic point;
and screening all the matched characteristic point groups according to the epipolar positions corresponding to the calculated image sets to obtain at least one target matched characteristic point group, wherein all the target matched characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for building a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model.
2. The binocular vision based feature point matching method of claim 1, wherein each of the first feature points has a corresponding first image location and each of the second feature points has a corresponding second image location;
and screening all the matching feature point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matching feature point group, including:
for each matching feature point group, calculating the distance between the matching feature point group and the polar line position according to the first image position corresponding to the first feature point of the matching feature point group, the second image position corresponding to the second feature point of the matching feature point group and the polar line position corresponding to the calculated image set, and obtaining the distance corresponding to the matching feature point group;
judging whether the distance corresponding to the matching characteristic point group is larger than or equal to a preset distance;
if the distance corresponding to the matched characteristic point group is larger than or equal to the preset distance, deleting the matched characteristic point group;
if the distance corresponding to the matched characteristic point group is smaller than the preset distance, determining the matched characteristic point group as a target matched characteristic point group;
And, the method further comprises:
acquiring position parameters corresponding to the binocular image acquisition device, a first imaging plane corresponding to the first image and a second imaging plane corresponding to the second image, wherein the position parameters corresponding to the binocular image acquisition device comprise a target position of the object to be molded, a first position of the first image acquisition device and a second position of the second image acquisition device;
determining an epipolar plane corresponding to the image set according to the position parameters corresponding to the binocular image acquisition device;
and calculating the polar line position corresponding to the image set according to the first imaging plane corresponding to the first image, the second imaging plane corresponding to the second image and the epipolar plane corresponding to the image set.
3. The binocular vision-based feature point matching method of claim 1 or 2, wherein before the filtering all the matching feature point groups according to the epipolar positions corresponding to the calculated image sets to obtain at least one target matching feature point group, the method further comprises:
judging whether the first characteristic points of the matched characteristic point groups are matched with the second characteristic points of the matched characteristic point groups or not for each matched characteristic point group;
If the first characteristic point of the matched characteristic point group is not matched with the second characteristic point of the matched characteristic point group, deleting the matched characteristic point group;
if the first characteristic point of the matching characteristic point group is judged to be matched with the second characteristic point of the matching characteristic point group, the matching characteristic point group is determined to be an alternative matching characteristic point group, and the operation of screening all the matching characteristic point groups according to the calculated polar line position corresponding to the image set to obtain at least one target matching characteristic point group is triggered and executed.
4. The binocular vision based feature point matching method of claim 3, wherein the determining whether the first feature point of the matched feature point group is matched with the second feature point of the matched feature point group for each of the matched feature point groups comprises:
for each matching feature point group, selecting one of the first feature point and the second feature point of the matching feature point group as a reference feature point of the matching feature point group;
determining the images of the rest characteristic points except the reference characteristic point in all the characteristic points of the matched characteristic point group as reference images;
Judging whether the reference characteristic points of the matching characteristic point group exist in the reference image or not;
if the reference characteristic points of the matching characteristic point group exist in the reference image, judging whether the residual characteristic points of the matching characteristic point group exist in the residual images except the reference image in the images of all the characteristic points of the matching characteristic point group;
if the residual characteristic points of the matched characteristic point group exist in the residual image, determining that the first characteristic points of the matched characteristic point group are matched with the second characteristic points of the matched characteristic point group;
and if the residual characteristic points of the matched characteristic point group are not found in the residual image or the reference characteristic points of the matched characteristic point group are not found in the reference image, determining that the first characteristic points of the matched characteristic point group are not matched with the second characteristic points of the matched characteristic point group.
5. The binocular vision based feature point matching method of claim 1, 2 or 4, further comprising:
for each feature point in all first feature points of the first type of feature points and all second feature points of the second type of feature points, acquiring a Harr wavelet response value set of the feature point, wherein the Harr wavelet response value set of each feature point comprises Harr wavelet response values of each neighborhood point in all neighborhood points of the feature point, and each neighborhood point has a corresponding neighborhood position;
For each feature point, calculating a plurality of vector directions of the feature point according to Harr wavelet response values of all the neighborhood points of the feature point and neighborhood positions corresponding to all the neighborhood points of the feature point;
screening out the vector direction with the maximum corresponding value from all the vector directions of the characteristic points, and taking the vector direction as the main direction of the characteristic points, wherein the main direction has the corresponding horizontal direction and vertical direction;
and calculating the Harr wavelet characteristic value of the characteristic point according to all Harr wavelet response values in the horizontal direction corresponding to the main direction of the characteristic point and all Harr wavelet response values in the vertical direction corresponding to the main direction of the characteristic point.
6. The binocular vision-based feature point matching method of claim 5, wherein for each of the feature points, calculating a plurality of vector directions of the feature point according to Harr wavelet response values of all the neighborhood points of the feature point and neighborhood positions corresponding to all the neighborhood points of the feature point comprises:
for each feature point, acquiring a weight coefficient of each neighborhood point of the feature point;
for each feature point, grouping all the neighborhood points of the feature point according to the neighborhood positions corresponding to all the neighborhood points of the feature point and a plurality of preset neighborhood position ranges to obtain a plurality of neighborhood groups of the feature point, wherein all the neighborhood points in the same neighborhood group correspond to the same neighborhood position range;
And for each neighborhood group of each characteristic point, calculating the product of the weight coefficient of each neighborhood point in the neighborhood group and the weight coefficient of each neighborhood point, and calculating the sum of the products of all neighborhood points in the neighborhood group and the weight coefficients of all neighborhood points to obtain the vector direction of the neighborhood group, wherein the vector directions of each characteristic point comprise the vector direction of each neighborhood group in the neighborhood groups of each characteristic point.
7. The binocular vision based feature point matching method of claim 1, 2, 4 or 6, wherein after screening all the matching feature point groups according to the epipolar positions corresponding to the calculated image sets to obtain at least one target matching feature point group, the method further comprises:
for each target matching feature point group, acquiring a first initial coordinate corresponding to a first feature point of the target matching feature point group and a second initial coordinate corresponding to a second feature point of the target matching feature point group;
according to a preset coordinate conversion algorithm, performing coordinate conversion on first initial coordinates corresponding to first feature points of the target matching feature point group to obtain first target coordinates corresponding to the target matching feature point group, and performing coordinate conversion on second initial coordinates corresponding to second feature points of the target matching feature point group to obtain second target coordinates corresponding to the target matching feature point group;
And carrying out three-dimensional modeling on all the target matching characteristic point groups according to the first target coordinates corresponding to all the target matching characteristic point groups and the second target coordinates corresponding to all the target matching characteristic point groups to obtain a three-dimensional model of the object to be modeled, wherein the three-dimensional model is used for executing corresponding supervision operation on the object to be modeled according to the determined three-dimensional information of the object to be modeled.
8. A binocular vision-based feature point matching apparatus, the apparatus comprising:
the system comprises an acquisition module, a first image acquisition device and a second image acquisition device, wherein the acquisition module is used for acquiring an image set of a target scene, and the image set comprises a first type image aiming at an object to be molded in the target scene, which is acquired by the first image acquisition device in a binocular image acquisition device in the target scene, and a second type image aiming at the object to be molded, which is acquired by the second image acquisition device in the binocular image acquisition device;
the extraction module is used for extracting characteristic points from a plurality of image positions in the image set according to a predetermined characteristic point extraction algorithm to obtain a characteristic point extraction result corresponding to the image set, wherein the characteristic point extraction result comprises a first type of characteristic points corresponding to the first type of images and a second type of characteristic points corresponding to the second type of images, and the first type of characteristic points and the second type of characteristic points are characteristic points of the object to be molded;
The computing module is used for computing the similarity between each first characteristic point and each second characteristic point according to the computed Harr wavelet characteristic value of each first characteristic point of the first type of characteristic points and the computed Harr wavelet characteristic value of each second characteristic point of the second type of characteristic points;
the determining module is used for determining a plurality of matched characteristic point groups to be screened from all the first characteristic points and all the second characteristic points according to the similarity between all the first characteristic points and all the second characteristic points, wherein each matched characteristic point group comprises a first characteristic point and a second characteristic point;
the screening module is used for screening all the matching characteristic point groups according to the polar line positions corresponding to the calculated image set to obtain at least one target matching characteristic point group, wherein all the target matching characteristic point groups are used for executing three-dimensional modeling operation on the object to be modeled, and the three-dimensional modeling operation is used for building a three-dimensional model of the object to be modeled and determining three-dimensional information of the object to be modeled according to the three-dimensional model.
9. A binocular vision-based feature point matching apparatus, the apparatus comprising:
A memory storing program code;
a processor coupled to the memory;
the processor invokes the program code stored in the memory to perform the binocular vision based feature point matching method of any one of claims 1-7.
10. A computer storage medium storing computer instructions which, when invoked, are operable to perform the binocular vision based feature point matching method of any one of claims 1-7.
CN202311326180.8A 2023-10-13 2023-10-13 Feature point matching method and device based on binocular vision Pending CN117333687A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311326180.8A CN117333687A (en) 2023-10-13 2023-10-13 Feature point matching method and device based on binocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311326180.8A CN117333687A (en) 2023-10-13 2023-10-13 Feature point matching method and device based on binocular vision

Publications (1)

Publication Number Publication Date
CN117333687A true CN117333687A (en) 2024-01-02

Family

ID=89278945

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311326180.8A Pending CN117333687A (en) 2023-10-13 2023-10-13 Feature point matching method and device based on binocular vision

Country Status (1)

Country Link
CN (1) CN117333687A (en)

Similar Documents

Publication Publication Date Title
CN104299244B (en) Obstacle detection method and device based on monocular camera
CN106960454B (en) Depth of field obstacle avoidance method and equipment and unmanned aerial vehicle
CN112102409B (en) Target detection method, device, equipment and storage medium
CN107980138A (en) A kind of false-alarm obstacle detection method and device
CN109426277B (en) Method and device for planning movement track
CN110619660A (en) Object positioning method and device, computer readable storage medium and robot
CN109214254B (en) Method and device for determining displacement of robot
CN111383264B (en) Positioning method, positioning device, terminal and computer storage medium
KR20200094075A (en) Method and device for merging object detection information detected by each of object detectors corresponding to each camera nearby for the purpose of collaborative driving by using v2x-enabled applications, sensor fusion via multiple vehicles
Silva et al. Camera calibration using a color-depth camera: Points and lines based DLT including radial distortion
CN112907675A (en) Calibration method, device, system, equipment and storage medium of image acquisition equipment
KR20200142391A (en) Method for Estimating 3D Marker Cordinetes of Optical Position Tracking System
CN113408454A (en) Traffic target detection method and device, electronic equipment and detection system
CN110673607B (en) Feature point extraction method and device under dynamic scene and terminal equipment
CN109658451B (en) Depth sensing method and device and depth sensing equipment
CN114821497A (en) Method, device and equipment for determining position of target object and storage medium
CN113673288B (en) Idle parking space detection method and device, computer equipment and storage medium
CN117333687A (en) Feature point matching method and device based on binocular vision
US20220335732A1 (en) Method and system for recognizing surrounding driving environment based on svm original image
CN112802112B (en) Visual positioning method, device, server and storage medium
CN113326793B (en) Remote pedestrian position identification method, system and storage medium
CN112991372B (en) 2D-3D camera external parameter calibration method based on polygon matching
CN111754713B (en) Video monitoring method, device and system
CN113723432A (en) Intelligent identification and positioning tracking method and system based on deep learning
CN113591640A (en) Road guardrail detection method and device and vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination