WO2020107267A1 - 一种图像特征点匹配方法及装置 - Google Patents

一种图像特征点匹配方法及装置 Download PDF

Info

Publication number
WO2020107267A1
WO2020107267A1 PCT/CN2018/117965 CN2018117965W WO2020107267A1 WO 2020107267 A1 WO2020107267 A1 WO 2020107267A1 CN 2018117965 W CN2018117965 W CN 2018117965W WO 2020107267 A1 WO2020107267 A1 WO 2020107267A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature point
matching
image
block
blocks
Prior art date
Application number
PCT/CN2018/117965
Other languages
English (en)
French (fr)
Inventor
占云龙
赵强
左韶军
林天鹏
王林召
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2018/117965 priority Critical patent/WO2020107267A1/zh
Priority to CN201880093374.9A priority patent/CN112119426A/zh
Publication of WO2020107267A1 publication Critical patent/WO2020107267A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction

Definitions

  • the present application relates to the field of image processing, and in particular, to an image feature point matching method and device.
  • Image feature point matching is widely used in many application scenarios such as image detection, image recognition, and image tracking.
  • the image information is described and characterized by extracting image feature points, and then the similarity of the two images is calculated by feature point matching, which provides a basis for judgment in subsequent applications such as image tracking, image detection, and image recognition.
  • feature point matching provides a basis for judgment in subsequent applications such as image tracking, image detection, and image recognition.
  • it is a common method to describe images by feature points, and then perform image matching based on feature points.
  • the traditional image feature point matching method usually uses brute force search to perform brute-force exhaustive matching on the entire feature point set to cover all matching feasibility. That is, for a feature point to be matched in the reference image, the current image is used. All feature points are matched one by one, and then the optimal feature point is selected as the matching feature point matching the feature point to be matched.
  • the above method requires a long matching time and a large storage space, so the matching efficiency is low.
  • the embodiments of the present application provide an image feature point matching method and device, which are used to improve the efficiency of feature point matching. To achieve the above purpose, the embodiments of the present application adopt the following technical solutions:
  • an image feature point matching method includes: storing a set of N feature points in a first image in a first buffer according to a preset order, the first image includes N blocks, each The block corresponds to a set of feature points, N is a positive integer; for the feature points to be matched of the second image, M blocks are determined from the N blocks of the first image according to the position of the feature points to be matched in the second image, the second image Including N blocks, M is an integer greater than or equal to 1; get the feature point set of each block in the M blocks from the first buffer; match the features of each block in the M blocks block by block according to the feature points to be matched Point set to match at least one matching feature point from the feature point set of M blocks.
  • storing the N feature point sets in the first image in the first buffer according to a preset order can improve the efficiency of subsequently acquiring the feature point set of each block in the M blocks, and at the same time according to the to-be-matched
  • the position of the feature point in the second image determines M blocks from the N blocks of the first image, which can effectively reduce the matching range, and then subsequently match the feature point set of each block in the M blocks according to the feature points to be matched.
  • the feature point set of one block can be used as a unit for matching
  • the feature point set of M blocks is matched block by block by pipeline operation, thereby reducing the memory consumption and matching time during the feature point matching process, thereby improving the image The efficiency of feature point matching.
  • the N blocks of the first image correspond to the N blocks of the second image in one-to-one correspondence, and the corresponding two blocks have the same shape and size.
  • the accuracy of determining the M blocks in the first image can be improved, that is, the accuracy of determining the effective matching range can be improved, thereby reducing unnecessary feature point matching operations.
  • the method before storing the set of N feature points in the first image in the first buffer in a specified order, the method further includes: according to the size and preset of the first image A partition rule, which divides the first image into N blocks, and determines the feature point set of each block in the N blocks of the first image; wherein, the preset partition rule includes at least one of the following rules: fit image edge partition , Partitioning of non-fitting image edges, partitioning by area, partitioning by regular shapes, partitioning by irregular shapes.
  • the flexibility and diversity of dividing the first image into N blocks can be improved.
  • the method further includes: according to the number of feature points included in each feature point set of the N feature point sets and a preset order, determine the corresponding feature point set of each block.
  • a position index where the position index includes a start position index and/or an end position index, and the position index corresponding to the feature point set of a block is used to index the position of the feature point set of the block in the first buffer.
  • determining M blocks from the N blocks of the first image according to the position of the feature point to be matched in the second image includes: according to the feature point to be matched in the second image Position and matching radius, or according to the position of the feature point to be matched in the second image, matching radius and preset offset, determine the matching area; according to the matching block selection strategy, select the matching area from the N blocks of the first image M blocks within.
  • the accuracy of determining the M blocks in the first image can be improved, that is, the accuracy of determining the effective matching range can be improved, thereby reducing unnecessary feature point matching operations.
  • the feature point set of each block in the M blocks is matched block by block according to the feature points to be matched to match at least one matching feature point from the feature point set of the M blocks , Including: for M blocks, determine the matching information between each feature point in the feature point set of each block and the feature point to be matched for each block; according to the matching information, the feature point set in each block meets the pre- The conditional feature points are determined as matching feature points.
  • an effective way of matching at least one matching feature point from the feature point set of M blocks is provided.
  • the matching information includes: a descriptor distance and/or a matching angle difference.
  • a simple and effective matching information is provided.
  • the feature point satisfying the preset condition means that the matching information of the feature point satisfies at least one of the following: the best descriptor distance, the next best descriptor distance, and the best matching angle difference , Or the next best matching angle.
  • the feature point set of each block in the M blocks is matched block by block according to the feature points to be matched to match at least one matching feature point from the feature point set of the M blocks , which also includes: filtering the matching feature points matched by the feature point set of each block according to the matching information and the preset filtering strategy to obtain at least one matching feature point; wherein, the preset filtering strategy includes at least one of the following strategies: repetition Matching filtering, angle difference histogram filtering, descriptor distance filtering, or block matching statistical filtering.
  • the flexibility and diversity in screening at least one matching feature point can be improved.
  • the method when one matching feature point in the first image corresponds to the first feature point to be matched and the second feature point to be matched in the second image, the method further includes: It is determined that the matching feature points in the first image correspond to the feature points to be matched with the best matching information among the first feature points to be matched and the second feature points to be matched.
  • the problem that the same matching feature point corresponds to at least two feature points to be matched can be avoided, and thus the problem of mismatching feature points can be solved.
  • an image feature point matching device in a second aspect, includes: a storage unit configured to store a set of N feature points in a first image in a first buffer in a preset order, the first image including N Blocks, each block corresponds to a set of feature points, and N is a positive integer; the determining unit is used for the feature points to be matched of the second image, according to the position of the feature points to be matched in the second image from the N of the first image M blocks are determined in the block, and the second image includes N blocks, M is an integer greater than or equal to 1; an obtaining unit is used to obtain a set of feature points of each of the M blocks from the first buffer; a matching unit , Used to match the feature point set of each block in the M blocks block by block according to the feature points to be matched, to obtain at least one matching feature point.
  • the N blocks of the first image correspond to the N blocks of the second image in one-to-one correspondence, and the corresponding two blocks have the same shape and size.
  • the determining unit is further configured to: divide the first image into N blocks according to the size of the first image and a preset dividing rule, and determine N of the first image A set of feature points for each block in the block; wherein, the preset partitioning rule includes at least one of the following rules: fitting image edge division, non-fitting image edge division, area division, regular shape division, and irregular shape division.
  • the determining unit is further configured to determine the feature point set of each block according to the number of feature points and the preset order included in each feature point set of the N feature point sets Corresponding position index.
  • the position index includes a start position index and/or an end position index.
  • the position index corresponding to the feature point set of a block is used to index the position of the feature point set of the block in the first buffer.
  • the determining unit is further configured to: according to the position and matching radius of the feature point to be matched in the second image, or according to the position and matching radius of the feature point to be matched in the second image And the preset offset to determine the matching area; according to the matching block selection strategy, select M blocks located in the matching area from the N blocks in the first image.
  • the matching unit is further used to: for M blocks, determine the matching between each feature point in the feature point set of each block and the feature point to be matched for each block Information; according to the matching information, the feature points in the feature point set of each block that meet the preset condition are determined as matching feature points.
  • the matching information includes: a descriptor distance and/or a matching angle difference.
  • the feature point satisfying the preset condition means that the matching information of the feature point satisfies at least one of the following: the best descriptor distance, the next best descriptor distance, and the best matching angle difference , Or the next best matching angle.
  • the matching unit is further configured to: according to the matching information and the preset screening strategy, filter the matching feature points matched by the feature point set of each block to obtain at least one matching feature point
  • the preset filtering strategy includes at least one of the following strategies: repeated matching filtering, angle difference histogram filtering, descriptor distance filtering, or block matching statistical filtering.
  • the determining unit when a matching feature point in the first image corresponds to the first feature point to be matched and the second feature point to be matched in the second image, the determining unit further uses Yu: It is determined that the matching feature point in the first image corresponds to the feature point to be matched with the best matching information among the first feature point to be matched and the second feature point to be matched.
  • an image feature point matching device includes: an input interface, an arithmetic unit, and a first buffer; wherein, the input interface is used to receive a set of N feature points in the first image, the first The image includes N blocks, each block corresponds to a set of feature points, and N is a positive integer; an operator is used to store the set of N feature points in the first image in the first buffer in a preset order; , Which is also used to determine M blocks from the N blocks of the first image according to the position of the feature points to be matched in the second image for the feature points of the second image, the second image includes N blocks, and M is greater than Or an integer equal to 1; get the feature point set of each block in the M blocks from the first buffer; the operator is also used to match the feature point set of each block in the M blocks block by block according to the feature points to be matched To match at least one matching feature point from the feature point set of M blocks.
  • the device further includes an output interface for outputting at least
  • the N blocks of the first image correspond to the N blocks of the second image one-to-one, and the corresponding two blocks have the same shape and size.
  • the arithmetic unit is further configured to: divide the first image into N blocks according to the size of the first image and a preset dividing rule, and determine N of the first image A set of feature points for each block in the block; wherein, the preset partitioning rule includes at least one of the following rules: fitting image edge division, non-fitting image edge division, area division, regular shape division, and irregular shape division.
  • the arithmetic unit is further configured to determine the feature point set of each block according to the number of feature points and the preset order included in each feature point set of the N feature point sets Corresponding position index, the position index includes a start position index and/or an end position index, and the position index corresponding to the feature point set of a block is used to index the position of the feature point set of the block in the first buffer.
  • the arithmetic unit is further configured to: according to the position and matching radius of the feature point to be matched in the second image, or according to the position and matching radius of the feature point to be matched in the second image And the preset offset to determine the matching area; according to the matching block selection strategy, select M blocks located in the matching area from the N blocks in the first image.
  • the arithmetic unit is further used to: for M blocks, determine the matching between each feature point in the feature point set of each block and the feature point to be matched for each block Information; according to the matching information, the feature points in the feature point set of each block that meet the preset condition are determined as matching feature points.
  • the matching information includes: a descriptor distance and/or a matching angle difference.
  • the feature point satisfying the preset condition means that the matching information of the feature point satisfies at least one of the following: the best descriptor distance, the next best descriptor distance, and the best matching angle difference , Or the next best matching angle.
  • the arithmetic unit is further configured to: according to the matching information and the preset filtering strategy, filter the matching feature points matched by the feature point set of each block to obtain at least one matching feature point
  • the preset filtering strategy includes at least one of the following strategies: repeated matching filtering, angle difference histogram filtering, descriptor distance filtering, or block matching statistical filtering.
  • the operator when a matching feature point in the first image corresponds to the first feature point to be matched and the second feature point to be matched in the second image, the operator also uses Yu: It is determined that the matching feature point in the first image corresponds to the feature point to be matched with the best matching information among the first feature point to be matched and the second feature point to be matched.
  • a readable storage medium in which instructions are stored, and when the readable storage medium runs on a device, the device is allowed to perform the first aspect or any one of the first aspects
  • the image feature point matching method provided by the implementation.
  • a computer program product which, when the computer program product runs on a computer, causes the computer to execute the image feature point matching method provided by the first aspect or any possible implementation manner of the first aspect .
  • any of the device, computer storage medium or computer program product of the image feature point matching method provided above is used to perform the corresponding method provided above, therefore, the beneficial effects that can be achieved can be referred to The beneficial effects of the corresponding methods provided in this article will not be repeated here.
  • FIG. 1 is a schematic structural diagram of an image processing device according to an embodiment of the present application.
  • FIG. 2 is a first schematic flowchart of an image feature point matching method provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram 1 of storing multiple feature point sets provided by an embodiment of the present application.
  • FIG. 4 is a second schematic diagram of storing multiple feature point sets provided by an embodiment of the present application.
  • FIG. 5 is a second schematic flowchart of an image feature point matching method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a regular shape and an irregular shape provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a distribution of N blocks provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a distribution of M blocks provided by an embodiment of the present application.
  • FIG. 9 is a first schematic structural diagram of an image feature point matching device according to an embodiment of the present application.
  • FIG. 10 is a second schematic structural diagram of an image feature point matching device provided by an embodiment of the present application.
  • At least one refers to one or more, and “multiple” refers to two or more.
  • And/or describes the relationship of the related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist at the same time, B exists alone, where A, B can be singular or plural.
  • At least one of the following or a similar expression refers to any combination of these items, including any combination of a single item or a plurality of items.
  • At least one (a) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c may be single or multiple.
  • the character "/" generally indicates that the related object is a "or” relationship.
  • the words “first” and “second” do not limit the number and the execution order.
  • FIG. 1 is a schematic structural diagram of an image processing device according to an embodiment of the present application.
  • the image processing device may be a mobile phone, a tablet computer, a computer, a notebook computer, a video camera, a camera, a wearable device, an in-vehicle device, or a terminal device, etc. .
  • the above-mentioned devices are collectively referred to as image processing devices in this application.
  • the embodiment of the present application takes the image processing device as a mobile phone as an example for description.
  • the mobile phone includes: a memory 101, a processor 102, a sensor component 103, a multimedia component 104, an audio component 105, a power supply component 106, and the like.
  • the memory 101 can be used to store data, software programs, and modules; mainly including a storage program area and a storage data area, wherein the storage program area can store an operating system and application programs required for at least one function, such as a sound playback function, an image playback function, etc. ;
  • the storage data area can store data created according to the use of the mobile phone, such as audio data, image data, phone book, etc.
  • the mobile phone may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the processor 102 is the control center of the mobile phone, and uses various interfaces and lines to connect the various parts of the entire device, by running or executing the software programs and/or modules stored in the memory 101, and calling the data stored in the memory 101 to execute Various functions and processing data of the mobile phone, so as to monitor the mobile phone as a whole.
  • the processor 102 may integrate an application processor (Application Processor, AP) and an application specific integrated circuit (Application Specific Integrated Circuit, ASIC); wherein, the AP mainly processes an operating system, a user interface, an application program, etc.
  • Application Processor Application Processor
  • ASIC Application Specific Integrated Circuit
  • an ASIC can be used to process related processes of image feature point matching in the embodiments of the present application. It can be understood that the above-mentioned ASIC may not be integrated into the processor 102.
  • the processor 102 may further include other hardware circuits or accelerators, such as field programmable gate arrays or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in conjunction with the disclosure of the present application.
  • the processor 102 may also be a combination that realizes a computing function, for example, including one or more microprocessor combinations, a combination of a digital signal processor and a microprocessor, and so on.
  • the sensor assembly 103 includes one or more sensors, which are used to provide various aspects of status assessment for the mobile phone.
  • the sensor component 103 may include a light sensor, such as a CMOS or CCD image sensor, for detecting the distance between an external object and the mobile phone, or used in an imaging application, that is, it becomes an integral part of a camera or a camera.
  • the sensor assembly 103 can also include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor, and the sensor assembly 103 can detect the acceleration/deceleration, orientation, on/off status of the mobile phone, and the relative positioning of the components, or Changes in cell phone temperature, etc.
  • the multimedia component 104 provides a screen of an output interface between the mobile phone and the user.
  • the screen may be a touch panel, and when the screen is a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding action, but also detect the duration and pressure related to the touch or sliding operation.
  • the multimedia component 104 further includes at least one camera.
  • the multimedia component 104 includes a front camera and/or a rear camera. When the mobile phone is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data.
  • Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.
  • the audio component 105 may provide an audio interface between the user and the mobile phone.
  • the audio component 105 may include an audio circuit, a speaker, and a microphone.
  • the audio circuit can transmit the converted electrical signal of the received audio data to the speaker, and the speaker converts it into a sound signal output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is converted into audio by the audio circuit. Data, and then output the audio data to send to, for example, another mobile phone, or output the audio data to the processor 102 for further processing.
  • the power component 106 is used to provide power for various components of the mobile phone.
  • the power component 106 may include a power management system, one or more power sources, and other components associated with the generation, management, and distribution of power by the mobile phone.
  • the mobile phone may further include a wireless fidelity (WiFi) module, a Bluetooth module, etc., and the embodiments of the present application will not be repeated here.
  • WiFi wireless fidelity
  • Bluetooth Bluetooth module
  • FIG. 1 does not constitute a limitation on the mobile phone, and may include more or less components than those shown in the figure, or a combination of certain components, or a different arrangement of components.
  • FIG. 2 is a schematic flowchart of an image feature point matching method provided by an embodiment of the present application.
  • the execution subject of the method may be an image processing device, and may specifically be a unit with a processing function and a storage function in the image processing device, such as the above diagram 1 shows the ASIC in the processor 102.
  • the method includes the following steps.
  • S201 Store the N feature point sets in the first image in the first buffer in a preset order.
  • the first image includes N blocks, and each block corresponds to a feature point set.
  • the first image may include N blocks, and N is a positive integer, and the size and shape of the N blocks in the first image may be the same or different.
  • Each block corresponds to a set of feature points, which may refer to a set of feature points in the block.
  • the first image includes N blocks, and each block corresponds to a feature point set, so that the first image has N feature point sets, that is, the N feature point sets correspond to the N blocks of the first image one-to-one.
  • the preset order may be set in advance by those skilled in the art according to actual needs, for example, the preset order includes a row order, a column order, a Z-type order, an X-type order, or other orders.
  • the N feature point sets in the first image are stored in the first buffer in the row order, the column order, and the Z-type order, respectively.
  • the feature point set of each block can pass through the block To indicate.
  • the N feature point sets in the first image when the N feature point sets in the first image are stored in the first buffer according to the preset order, the N feature point sets may be continuously stored in the first buffer according to the preset order, or according to The N feature point sets are discontinuously stored in the first buffer in a preset order; wherein, feature points in the same feature point set are continuously stored between.
  • the feature point sets of 16 blocks in the first image may be shown in FIG. 4 in the first buffer, and (a) in FIG. 4 Taking the set of 16 feature points as a continuous storage method for example, (b) in FIG. 4 takes the set of 16 feature points as a non-continuous storage method for example.
  • the position index corresponding to each feature point set may also be determined. Specifically, according to the number of feature points included in each feature point set of the N feature point sets and the preset order, a position index corresponding to the feature point set of each block is determined, and the position index may include a starting position index ( It is used to indicate the start position in the first buffer) and/or the end position index (used to indicate the end position in the first buffer).
  • the start position of the block feature point set in the first buffer can be determined by the start position index, and the end position can be passed The start position index and the number of feature points included in the feature point set are calculated; if the position index corresponding to the feature point set of a block includes the end position index, the start of the feature point set of the block in the first buffer The position can be calculated by the end position index and the number of feature points included in the feature point set, and the end position can be determined by the end position index; if the position index corresponding to the feature point set of a block includes the start position index and the end position index, Then, the start position of the feature point set in the first buffer may be determined by the start position index, and the end position may be determined by the end position index.
  • the number of feature points included in a feature point set of a block is NUM. If the starting position index corresponding to the feature point set is Add_k, the end position index of the feature point set in the first buffer is (Add_k +NUM-1); if the end position index corresponding to the feature point set is Add_w, the start position index of the feature point set in the first buffer is (Add_w-NUM+1).
  • the start address of m blocks can be ADDR_start ⁇ 0, n_1, n_1+n_2, ..., n_1+n_2+...+n_m-1 ⁇
  • the end address of m blocks can be ADDR_end ⁇ -1+n_1 , -1+n_1+n_2, ..., -1+sum(NUM) ⁇ .
  • the image processing device may also store the position index corresponding to the feature point set of each block in the first image, for example, in the first image stored by the image processing device
  • the position index corresponding to the feature point set of each block includes: a block identifier and a position index, and the block identifier is used to indicate the block corresponding to the position index.
  • the feature points are usually extracted from the first image.
  • the extracted feature points are stored out of order.
  • the feature points of different blocks may be stored adjacent to each other, and multiple feature points of the same block are scattered and stored. As a result, it takes a long search time and addressing calculation time to subsequently read multiple feature points of the same block.
  • N feature point sets that is, storing multiple feature points belonging to the same block continuously
  • the feature point sets of different blocks are stored in a certain order, which can improve subsequent feature points Reading efficiency.
  • the method may further include: S200.
  • S200 Divide the first image into N blocks according to the size of the first image and a preset division rule, and determine the feature point set of each of the N blocks in the first image.
  • the preset division rule may be set by a person skilled in the art according to actual conditions in advance, for example, the preset division rule may include at least one of the following rules: fit image edge division, non-fit image edge division, uniform division, no Evenly divided, divided by area, divided into regular shapes, divided into irregular shapes.
  • Fit image edge division can refer to dividing the complete image, in this case, the size of the image itself is used as the size of the divided area; non-fit image edge division refers to dividing the image area except the edge area ,
  • the edge area may include an area corresponding to one or more edges of the image, in this case, the size of the divided area is smaller than the size of the image itself; uniform division may mean that the size and shape of the divided blocks are the same; Uneven division can mean that the size and shape of the divided blocks are different; division by area can mean different divisions of different areas in the image, such as different shapes or sizes of blocks corresponding to different areas.
  • Regular shape division means that the shape of the N blocks obtained by dividing the image is a regular shape, such as a rectangle, square, circle, hexagon, diamond, or triangle as shown in (a) in FIG. 6; irregular shape The division means that the shape of the N blocks obtained by dividing the image is an irregular shape, such as an ellipse, a cross, a trapezoid, an L, an X, or a ring as shown in (b) in FIG. 6.
  • the preset division rule corresponding to (a) in FIG. 7 may be: fit image edge division+even division; the preset division rule corresponding to (b) in FIG. 7 may be: non-fit image edge division+even division. +Regular shape division; the preset division rule corresponding to (c) in FIG. 7 may be: fit image edge division+division by region+regular shape division; the preset division rule corresponding to (d) in FIG. 7 may be : Partition of the fit image edge + partition of regular shape.
  • determining the feature point set of each block in the N blocks of the first image may include: the image processing device extracts the feature points in the first image, and according to the position of each feature point and the N blocks of the first image The location range of each block in the block determines multiple feature points belonging to the same block, that is, the feature point set of each block is obtained.
  • feature point extraction can use any of the following algorithms: Scale-Invariant Feature Transformation (SIFT), Fast Robust Feature (Speeded UpRobust Feature, SURF), BRIEF calculation with direction and rotation Oriented (FAST) and Rotated (BRIEF, ORB), Gaussian function difference (Difference of Gaussian, DOG), learning invariant feature transform (Learned Invariant Feature Transform, LIFT) and other algorithms.
  • SIFT Scale-Invariant Feature Transformation
  • FAST Fast Robust Feature
  • FAST direction and rotation Oriented
  • BRIEF Rotated
  • Gaussian function difference Difference of Gaussian
  • DOG learning invariant feature transform
  • LIFT Learning Invariant Feature Transform
  • Common information recorded in the extracted feature points may include but is not limited to: coordinates (x, y), the zoom layer (octave) where the current feature point is located, response score (response), zoom ratio or zoom factor (scale), Information such as angle, descriptor, or index.
  • the extracted feature points can be used directly, or can be filtered by different screening methods, such as octree screening, etc., which are not specifically limited in the embodiments of the present application.
  • the image processing device may also count and store the block statistical information of each block.
  • the block statistics of a block may include one or more of the following information: block identification (used to uniquely identify a block, such as block number, etc.), number of feature points, and response score information (such as the maximum response score value, Average response score value, or minimum response score value, etc.) feature point distribution information (for example, octave-based distribution, block center point coordinate-based distribution, response score value distribution map, etc.).
  • the above process of dividing the first image into N blocks and the process of extracting feature points of the first image may not be in any order.
  • the block statistics of each block can be counted in the process of extracting feature points, so that the image processing device does not need to wait for all the features of the first image Statistics are collected after the point extraction is completed, so that the waiting time and the time for re-reading the feature points of the first image can be reduced.
  • S202 For the feature points to be matched in the second image, select M blocks from the N blocks in the first image according to the positions of the feature points to be matched in the second image, where M is an integer greater than or equal to 1.
  • the size of the second image is the same as the size of the first image, and the second image and the first image may have the same or similar features.
  • the first image and the second image may be two adjacent ones in the same video stream. Frame image, or two frames of the same size and similar.
  • the second image includes N blocks and corresponds one-to-one with the N blocks of the first image, and the shapes, sizes, and positions of the corresponding two blocks may be the same.
  • Each block in the second image may have multiple feature points to be matched, and the multiple feature points to be matched in each block may be multiple feature points extracted in the block.
  • the position of the feature point to be matched in the second image corresponds to its corresponding position in the first image, that is, the position of the feature point to be matched in the second image is the same as its corresponding position in the first image .
  • the first image and the second image are represented by the same coordinate system, and the positions of the feature points to be matched in the two coordinate systems are the same.
  • the matching area may be determined according to the position and matching radius of the feature point to be matched in the second image, for example, in the first In the image, the position corresponding to the feature point to be matched in the first image is taken as the center of the circle (or referred to as the selected center point), the matching radius is taken as the radius, and the determined circular area is taken as the matching area.
  • the initial matching area may also be determined according to the position of the feature point to be matched in the second image, the matching radius, and the preset offset, For example, in the first image, the position corresponding to the position of the feature point to be matched in the first image after being shifted by a preset offset is taken as the center of the circle, the matching radius is used as the radius, and the determined circular area is used as the matching area. Then, according to the matching block selection strategy, M blocks located in the matching area are selected from the N blocks in the first image, for example, a block identifier corresponding to each of the M blocks is determined.
  • the matching block selection strategy may be a rectangular block set, an X-shaped block set, a cross-shaped block set, or a diamond-shaped block set.
  • the above-mentioned matching radius, preset offset, and matching block selection strategy can be set in advance by those skilled in the art according to actual conditions or dynamically set during the matching process, for example, according to the block where the feature point to be matched is located Set the shape, size, and the above-mentioned preset division rules, etc.; or according to relevant experience, or use adaptive algorithms such as adaptive configuration.
  • the same matching radius, preset offset or matching block selection strategy can be set, or different matching radius, preset offset or matching block selection strategy can be set, This embodiment of the present application does not specifically limit this.
  • the position of the feature point to be matched in the first image is the center of the circle and the matching radius is the radius.
  • the matching block selection strategy is a rectangular block set
  • the M blocks are selected as shown in (a) in FIG. 8
  • the matching block selection strategy is a cross block set
  • the M blocks are selected as As shown in (b) of FIG. 8
  • the above matching block selection strategy is an X-type block set
  • the M blocks selected are as shown in (c) of FIG. 8.
  • the block where the circle center is located at the reference number 10 in the first image in FIG. 4 is used as an example for description, and the circle area in FIG. 8 represents the matching area.
  • S203 Acquire the feature point set of each block of the M blocks from the first buffer.
  • the image processing device acquires the corresponding feature point set of each block in the M blocks from the storage space storing the position index corresponding to the feature point set of each block according to the block identifier of each block in the M blocks Location index. Furthermore, according to the position index corresponding to the feature point set of each block in the M blocks, the feature point set of each block is obtained from the first buffer.
  • the feature point set of one block can be read at a time, or the feature point set of a block can be read one by one continuously; or, multiple feature points can be read in parallel from the feature point set of a block, It is also possible to read the feature points of different blocks in parallel from the feature point sets of multiple blocks in the M blocks (when reading in parallel, the subsequent steps can also be executed in parallel).
  • S204 Match the feature point set of each block in the M blocks block by block according to the feature points to be matched, so as to match at least one matching feature point from the feature point set of the M blocks.
  • each matching feature point in at least one matching feature point obtained by matching and the feature point to be matched may form a matching pair.
  • the best matcher algorithm, KNN matching algorithm, or left and right bidirectional matching algorithm can be used to match the feature point set of the block with the feature point to be matched.
  • the best matching algorithm may refer to selecting the most similar feature point from the feature point set of the block as the matching feature point of the feature point to be matched for the feature point to be matched.
  • the left and right bidirectional matching algorithm may refer to selecting the most similar feature point (denoted as feature point Q(x, x) from the feature point set of the block for the feature point to be matched (denoted as feature point P(x, y)).
  • the feature point Q(x, y) at the same time, for the feature point Q(x, y), use the same method to select a most similar feature point (represented as the feature point P'(x, y)) from the first image, if P'(x ,y) is consistent with P(x,y), then determine the feature point Q(x,y) as the matching feature point of the feature point P(x,y) to be matched, otherwise the matching is invalid; using the left and right bidirectional matching algorithm requires The matching process is performed twice, so the matching cost is relatively high.
  • the image processing device can also use other different matching algorithms during matching.
  • the above description is based on the best matching algorithm, the KNN matching algorithm, and the left and right bidirectional matching algorithms as examples, and does not limit the embodiments of the present application.
  • the image processing device may also determine the matching information between each feature point in the feature point set and the feature point to be matched, the The matching information may include: descriptor distance and/or matching angle difference. After that, the image processing device determines the feature points in the feature point set that satisfy the preset condition as matching feature points according to the matching information, where the feature points that satisfy the preset condition may refer to the matching information of the feature point satisfying at least one of the following Item: best descriptor distance, next best descriptor distance, best matching angle difference, or second best matching angle.
  • the descriptor distance can be Manhattan distance, Euclidean distance, Hamming distance, etc.
  • the Manhattan distance (D1) can be calculated as follows
  • the Euclidean distance (D2) can be calculated as shown in formula (2) below
  • the Hamming distance (D3) can be calculated as shown in formula (3) below.
  • XOR in the above formula (3) is the exclusive OR operator of logical operation.
  • the optimal descriptor distance can be used to represent the most similar distance between the descriptors of the two feature points.
  • the optimal descriptor distance can refer to the calculation The smallest non-negative value in all descriptor distances obtained, and the corresponding matching feature point at this time may be called the best matching feature point.
  • the second best descriptor distance may refer to the descriptor distance that is second only to the best descriptor distance among all the descriptor distances corresponding to the feature point set, and the corresponding matching feature point may be called a second best matching feature point.
  • the matching angle difference may refer to the difference between the angles of two matched feature points, and the angle of each feature point may be obtained when the feature point is extracted.
  • the best matching angle difference may refer to the minimum value of all the angle differences corresponding to the feature point set.
  • the corresponding matching feature point may be referred to as the best matching feature point;
  • the next best matching angle may refer to the feature point set corresponding to Among all the angle differences, the matching angle difference is second only to the best matching angle difference, and the corresponding matching feature point may be referred to as the second best matching feature point.
  • the image processing device may also store the matching result of the feature points (that is, the above-mentioned matching pair information), the matching state of the feature points, and related statistical information, and so on.
  • the image processing device may store the feature point matching result in the second buffer, and the second buffer may include: the index of the feature point to be matched One or more of the number, the index number of the best matching feature point, the best descriptor distance, the best matching angle difference, and so on.
  • the second buffer may further include: the index number of the second best matching feature point, the second best descriptor distance, the second best matching angle difference, and the like.
  • the image processing device may also store the matching state of the feature point in a status register, which may include: the index number of the feature point, and the index of the block where the feature point is located Number, the index number of the feature point to be matched with the feature point, the index number of the block where the feature point to be matched with the feature point is located, one or more of the best descriptor distance, the best matching angle difference, etc. Pcs.
  • the image processing device may also store the statistical information (eg, angle difference histogram) of the matching angle difference corresponding to the feature point set in the angle difference statistics register.
  • the matching angle difference between a matching feature point and the feature point to be matched is 45 degrees
  • 45 degrees belongs to the interval [30,60)
  • the number of feature points corresponding to [30,60) increases by 1
  • the interval is 2.
  • the image processing device may also store the block matching statistical information of the block in a block statistical register, which is mainly used to store block information and block matching results.
  • the block statistics register may include: the index number of the block where the feature point to be matched, the index number of the block in the first image, the number of valid matching points, and the average/maximum/minimum value of the best descriptor distance corresponding to the matching feature point in the block Wait.
  • the image processing device can also add or delete various items of information in the block statistical register according to the requirements of subsequent applications, which is not specifically limited in the embodiments of the present application.
  • the matching feature points corresponding to each feature point to be matched can be processed according to the above steps, so that the feature points of the first image and the second image are matched.
  • a matching feature point of a block in the first image corresponds to both the first feature point to be matched and the second feature point to be matched in the second image
  • the matching feature points correspond to the feature points to be matched with the best matching information among the first feature points to be matched and the second feature points to be matched, that is, the matching feature points are used as the matching feature points with the best matching information to be matched.
  • the image processing device may perform processing in a pipeline operation mode.
  • the second buffer also stores corresponding information in a pipeline operation mode.
  • the matching status of each feature point is also recorded in the status register.
  • the embodiment of the present application may adopt the following method to deal with the above-mentioned problem of repeated matching, so as to avoid modifying the written memory again.
  • Q(x, y) is marked in the above status register. If there is no match, the matching feature points P(x, y) and Q(x, y) are valid, and the relevant information about the matching of P(x, y) and Q(x, y) is written into the above status register. If the matching feature point P(x, y) is matched and the best matching feature point Q(x, y) and the best descriptor distance (denoted as dist0) are obtained, Q(x, y) is marked in the above status register.
  • the best matching point is P(x,y), and the best descriptor is updated to dist0 and other statistical information, and the relevant information of P(x,y) and Q(x,y) matching is written into the above status register .
  • the image processing device may also update the above status register and angle difference statistical register to avoid register information and The inconsistency of the actual situation.
  • the image processing device may also filter the matching feature points in each block of the first image according to the matching information and the preset filtering strategy to obtain at least one matching feature point.
  • the preset filtering strategy includes at least one of the following strategies: repeated matching filtering, angle difference histogram filtering, block descriptor distance filtering, or block matching statistical filtering.
  • each matching pair can be screened using the same method until all matching pairs are screened.
  • the process of the image processing device using the repeated matching and screening strategy may be: separately reading from the second buffer (for example, starting from the starting address) information of each matching pair (that is, the matching result of the feature points), for example
  • the read matching pair information is the matching result of the feature point P(x,y) in the second image and the feature point Q(x,y) in the first image, that is, the image processing device queries the above information according to each matching pair information Status register. If the matching result obtained from the query is consistent with the information of the matching pair, the current matching pair is determined to be valid, otherwise the current matching pair is determined to be invalid.
  • the image processing device may mark it as invalid in the second buffer, or directly delete related information from the second buffer. Since the status register is updated after processing the problem of repeated matching, the information of the status register is up-to-date and there are no duplicate matching pairs.
  • angle difference histogram screening The principle of the above-mentioned angle difference histogram screening is that: the objects in the two images undergo similar spatial transformation, then the angle space distribution of the correct matching pair should be similar, so the distribution should be similar, and the abnormal angle difference distribution can be considered Is a mismatch.
  • the principle of the block descriptor distance screening is that the objects in the two images undergo similar spatial transformations, then the blocks where the correct matching pair is located should also be similar, so the block descriptor distance should be similar, and the abnormal block descriptor distance It can be considered as a mismatch.
  • the process of the image processing device using the block descriptor distance screening strategy may be: reading the statistical information of the best descriptor distance corresponding to the feature point set of each block in the first image from the block statistics register, according to the The average/maximum/minimum value of the best descriptor distance in the statistical information, or the average/maximum/minimum value of the best descriptor distance and the number of valid matching feature points of each block determine the decision threshold of F blocks BDIST ⁇ dist_1, dist_2, ..., dist_f ⁇ , or obtained based on prior information, big data analysis, machine learning, etc., or obtained from external dynamic input; read each matching pair information from the second buffer (ie Feature point matching result), the feature point matching result includes the best descriptor distance bdist of the current matching pair and the block identifier blk_i corresponding to the second image; the query blk_i is in the above BDIST ⁇ dist_1,dist_2,...,dist_f ⁇ The corresponding
  • the image processing device may mark it as invalid in the second buffer, or directly delete the relevant information from the second buffer.
  • the principle of the above block matching statistical screening is that the objects in the two images undergo similar spatial transformation, and the feature points of the adjacent area in the first image are originally, after the similar transformation, they should also be in the adjacent area in the second image.
  • the effectiveness of matching pairs can be analyzed by analyzing the aggregation of the feature points of each block in the first image to the feature points of the second image, that is, the feature points in the same block in the first image, in the second image It should also be clustered in similar areas, and matching pairs with abnormal distributions with large spatial spans can be regarded as false matches.
  • the process of the image processing device using the block descriptor sub-distance filtering strategy may be: reading from the above status register each matching block identifier in the first image and block identifier in the second image; from the above block The number of effective matching feature points of each block of the first image is read in the statistical register. Assume that there are A blocks.
  • the effective matching feature points of A blocks are NUM ⁇ b_1, b_2, ..., b_a ⁇ , and the total number of points is B; Obtain the effective matching feature points in each block of the first image corresponding to the block identification set in the second image to obtain the distribution block identification array BLKNUM[B], and determine the value of each block in the first image according to the distribution block identification array BLKNUM[B]
  • Each block corresponds to the best matching block and the second best matching block in the second image (optionally, stored in the above-mentioned block statistics register); each matching pair information (that is, feature point matching) is determined from the above-mentioned second buffer Result), the matching result of the feature point includes the best descriptor distance bdist of the current matching pair, the block identifier blk_i corresponding to the first image and the block identifier blk_j corresponding to the second image; query based on the read matching pair information
  • the above block statistics register is used to obtain blk_bestindex and
  • blk_bestindex and blk_best2index are used to form an effective block strategy: for example, the block composition set BLK_SET covered by the connection of blk_bestindex and blk_best2index is the effective set. If blk_j falls within the block set BLK_SET, the current matching pair is valid, otherwise the current matching pair is invalid ;
  • Multi-block strategy centered on the blk_bestindex block a set BLK_SET consisting of a rectangular block centered on the blk_bestindex block (the size of the rectangular block can be set), a cross-shaped block, an X-shaped block, etc. Then the current matching pair is valid, otherwise the current matching pair is invalid.
  • the image processing device may also use two or more of the above filtering strategies to perform simultaneous filtering, or use other filtering strategies. Screening strategies, etc. Through the screening based on the above different information, the correct matching rate can be improved, and at the same time a higher screening efficiency can be ensured.
  • the image processing device can also output the information of the above-mentioned second buffer and each register to output the final matching result. If the best matching is performed, the relevant result of the best matching is output, and the output of the next best matching can be selected not to be output. Relevant results; if the best match and the next best match are performed, the best match and the second best match can be output.
  • the image processing device may also output the matching result during the above screening process. For example, when a matching pair is determined to be a valid matching pair, the matching pair is directly output, and when a matching pair is determined to be an invalid matching pair, the matching is not output. Correct.
  • the ASIC can match the feature point set to be matched with the feature point set of each block in the M blocks of the first image in a pipelined manner, thereby effectively saving the ASIC Storage resources, while improving the matching efficiency of feature points.
  • the image processing device includes a hardware structure and/or a software module corresponding to each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed by hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
  • the embodiments of the present application may divide the functional modules of the device (eg, image processing device) for screening local feature points according to the above method examples.
  • each functional module may be divided corresponding to each function, or two or more than two
  • the functions are integrated in one processing module.
  • the above integrated modules may be implemented in the form of hardware or software function modules. It should be noted that the division of the modules in the embodiments of the present application is schematic, and is only a division of logical functions. In actual implementation, there may be another division manner.
  • FIG. 9 shows a schematic structural diagram of an image feature point matching device involved in the above embodiment
  • the device includes: a storage unit 901, a determination unit 902, and an acquisition unit 903 ⁇ matchunit 904.
  • the storage unit 901 is used to support the device to execute S201 in the above method embodiment
  • the determination unit 902 is used to support the device to execute S202 and S200 in the above method embodiment
  • the acquisition unit 903 is used to support the device to perform the above method implementation S203 in the example
  • the matching unit 904 is used to support the device to execute S204 in the above method embodiment. All relevant content of each step involved in the above method embodiments can be referred to the function description of the corresponding function module, which will not be repeated here.
  • the apparatus may be a processor in an image processing device (for example, the ASIC in FIG. 1 ).
  • the apparatus includes: an input interface 1001 , An arithmetic unit 1002 and a buffer 1003, the buffer 1003 includes a first buffer.
  • the device may further include an output interface 1004.
  • the input interface 1001 may be used to support the device to receive the N feature point sets in the first image, the size of the first image, the matching radius, the preset offset, the first image, the second image, and so on.
  • the information received by the input interface 1001 may be stored in an external memory, such as the memory 101 of the image processing device shown in FIG. 1; in some feasible embodiments, there may be only one input interface , Can also have multiple input interfaces.
  • the arithmetic unit 1002 may be used to support the device to execute S200-S204 in the above method embodiments, and/or other technical processes described herein. For specific descriptions, please refer to the relevant descriptions in the above method embodiments. Repeat again.
  • the buffer 1003 can be used to cache the relevant information that needs to be cached during the execution of the above method embodiment by the arithmetic unit 1002, for example, the position index after the arrangement of the N feature points in the first image, the feature points Matching information between matching feature points, and feature points that satisfy preset conditions, etc.
  • the buffer 1003 may further include the second buffer, the status register, and the angle difference statistical register in the foregoing method embodiment.
  • the output interface 1004 can be used to output at least one matching feature point from the feature point set of M blocks. In some feasible embodiments, there may be only one output interface, or there may be multiple output interfaces. In some feasible embodiments, the processing result output by the output interface 1004 may be sent to an external memory for storage, or may be sent to another processing flow to continue processing.
  • the components of the image feature point matching device provided by the embodiments of the present application are respectively used to implement the functions of the corresponding steps of the foregoing image feature point matching method. Since the foregoing image feature point matching method embodiments have already Each step has been described in detail, and will not be repeated here.
  • storing the N feature point sets in the first image in the first buffer according to a preset order can improve the efficiency of subsequently acquiring the feature point set of each block in the M blocks, and according to The feature points to be matched determine the M blocks from the N blocks of the first image at the position of the second image, which can effectively reduce the matching range, and then subsequently match the features of each block of the M blocks according to the feature points to be matched
  • the feature point set of one block can be used as a unit for matching.
  • the feature point set of M blocks is matched block by block by pipeline operation, thereby reducing the memory consumption and matching time in the matching process and improving the feature points. Matching efficiency.
  • the disclosed device and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the modules or units is only a division of logical functions.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed in multiple different places . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a readable storage medium.
  • the technical solutions of the embodiments of the present application may essentially be part of or contribute to the existing technology, or all or part of the technical solutions may be embodied in the form of software products, which are stored in a storage medium
  • several instructions are included to enable the terminal to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the foregoing storage media include various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种图像特征点匹配方法及装置，涉及图像处理领域，用于提高特征点的匹配效率。所述方法包括：按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，所述第一图像包括N个块，每个块对应一个特征点集合；对于第二图像的待匹配特征点，根据所述待匹配特征点在所述第二图像的位置从所述第一图像的N个块中确定M个块，所述第二图像包括N个块；从所述第一缓存器中获取所述M个块中每个块的特征点集合；根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，以从所述M个块的特征点集合中匹配出至少一个匹配特征点。

Description

一种图像特征点匹配方法及装置 技术领域
本申请涉及图像处理领域，尤其涉及一种图像特征点匹配方法及装置。
背景技术
图像特征点匹配被广泛应用于图像检测、图像识别、图像跟踪等多个应用场景中。通过提取图像特征点来对图像信息进行描述和表征，然后通过特征点匹配来计算两幅图像的相似性，为后续的图像跟踪、图像检测和图像识别等应用提供判断依据。在特征信息描述的场景中，通过特征点来进行描述图像，然后基于特征点进行图像匹配是一种常用的方法。
传统的图像特征点匹配方法通常是采用暴力搜索的方法对整个特征点集合进行暴力穷举匹配，以覆盖所有的匹配可行性，即对于参考图像中的一个待匹配特征点，使用当前图像中的全部特征点进行逐一匹配，然后选择出最优的特征点作为与该待匹配特征点匹配的匹配特征点。但是，上述方法需要较长的匹配时间和较大的存储空间，因此匹配效率较低。
发明内容
本申请的实施例提供一种图像特征点匹配方法及装置，用于提高特征点匹配的效率。为达到上述目的，本申请的实施例采用如下技术方案：
第一方面，提供一种图像特征点匹配方法，该方法包括：按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，第一图像包括N个块，每个块对应一个特征点集合，N为正整数；对于第二图像的待匹配特征点，根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，第二图像包括N个块，M为大于或等于1的整数；从第一缓存器中获取M个块中每个块的特征点集合；根据待匹配特征点逐块匹配M个块中每个块的特征点集合，以从M个块的特征点集合中匹配出至少一个匹配特征点。上述技术方案中，按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，能够提高后续获取M个块中每个块的特征点集合的效率，同时根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，能够有效减小匹配范围，进而后续根据待匹配特征点逐块匹配M个块中每个块的特征点集合时，能够以一个块的特征点集合为单位进行匹配，M个块的特征点集合通过流水式操作逐块进行匹配，从而降低了特征点匹配过程中的内存消耗和匹配时间，进而提高了图像特征点匹配的效率。
在第一方面的一种可能的实现方式中，第一图像的N个块与第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。上述可能的实现方式中，能够提高确定第一图像中的所述M个块的准确性，即提高确定有效匹配范围的准确性，进而减少不必要的特征点匹配操作。
在第一方面的一种可能的实现方式中，按照指定顺序将第一图像中的N个特征点 集合存储在第一缓存器中之前，该方法还包括：根据第一图像的尺寸和预设划分规则，将第一图像划分为N个块，并确定第一图像的N个块中每个块的特征点集合；其中，预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。上述可能的实现方式中，能够提高将第一图像划分为N个块的灵活性和多样性。
在第一方面的一种可能的实现方式中，该方法还包括：根据N个特征点集合中每个特征点集合包括的特征点数量和预设顺序，确定每个块的特征点集合对应的位置索引，位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引块的特征点集合在第一缓存器中的位置。上述可能的实现方式中，提供了一种简单有效的指示每个特征点集合的位置的方法，进而能够提高后续获取M个块的特征点集合的效率。
在第一方面的一种可能的实现方式中，根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，包括：根据待匹配特征点在第二图像的位置和匹配半径，或者根据待匹配特征点在第二图像的位置、匹配半径和预设偏移量，确定匹配区域；根据匹配块选择策略，从第一图像的N个块中选择位于匹配区域内的M个块。上述可能的实现方式中，能够提高确定第一图像中的所述M个块的准确性，即提高确定有效匹配范围的准确性，进而减少不必要的特征点匹配操作。
在第一方面的一种可能的实现方式中，根据待匹配特征点逐块匹配M个块中每个块的特征点集合，以从M个块的特征点集合中匹配出至少一个匹配特征点，包括：对于M个块，逐块确定每个块的特征点集合中的每个特征点与待匹配特征点之间的匹配信息；根据匹配信息，将每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。上述可能的实现方式中，提供了一种有效的M个块的特征点集合中匹配出至少一个匹配特征点的方式。
在第一方面的一种可能的实现方式中，该匹配信息包括：描述子距离和/或匹配角度差。上述可能的实现方式中，提供了一种简单有效的匹配信息。
在第一方面的一种可能的实现方式中，满足预设条件的特征点是指特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。上述可能的实现方式中，提供了一种有效的确定匹配特征点的方式。
在第一方面的一种可能的实现方式中，根据待匹配特征点逐块匹配M个块中每个块的特征点集合，以从M个块的特征点集合中匹配出至少一个匹配特征点，还包括：根据匹配信息和预设筛选策略，筛选每个块的特征点集合匹配出的匹配特征点，得到至少一个匹配特征点；其中，预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。上述可能的实现方式中，能够提高筛选至少一个匹配特征点的灵活性和多样性。
在第一方面的一种可能的实现方式中，当第一图像中的一个匹配特征点与第二图像中的第一待匹配特征点和第二待匹配特征点对应时，该方法还包括：确定第一图像中的匹配特征点与第一待匹配特征点和第二待匹配特征点中匹配信息最优的待匹配特征点对应。上述可能的实现方式中，能够避免同一匹配特征点与至少两个待匹配特征点对应的问题，进而解决特征点误匹配的问题。
第二方面，提供一种图像特征点匹配装置，该装置包括：存储单元，用于按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，第一图像包括N个块，每个块对应一个特征点集合，N为正整数；确定单元，用于对于第二图像的待匹配特征点，根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，第二图像包括N个块，M为大于或等于1的整数；获取单元，用于从第一缓存器中获取M个块中每个块的特征点集合；匹配单元，用于根据待匹配特征点逐块匹配M个块中每个块的特征点集合，得到至少一个匹配特征点。
在第二方面的一种可能的实现方式中，第一图像的N个块与第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。
在第二方面的一种可能的实现方式中，确定单元，还用于：根据第一图像的尺寸和预设划分规则，将第一图像划分为N个块，并确定第一图像的N个块中每个块的特征点集合；其中，预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。
在第二方面的一种可能的实现方式中，确定单元，还用于：根据N个特征点集合中每个特征点集合包括的特征点数量和预设顺序，确定每个块的特征点集合对应的位置索引，位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引块的特征点集合在第一缓存器中的位置。
在第二方面的一种可能的实现方式中，确定单元，还用于：根据待匹配特征点在第二图像的位置和匹配半径，或者根据待匹配特征点在第二图像的位置、匹配半径和预设偏移量，确定匹配区域；根据匹配块选择策略，从第一图像的N个块中选择位于匹配区域内的M个块。
在第二方面的一种可能的实现方式中，匹配单元，还用于：对于M个块，逐块确定每个块的特征点集合中的每个特征点与待匹配特征点之间的匹配信息；根据匹配信息，将每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。
在第二方面的一种可能的实现方式中，该匹配信息包括：描述子距离和/或匹配角度差。
在第二方面的一种可能的实现方式中，满足预设条件的特征点是指特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
在第二方面的一种可能的实现方式中，匹配单元，还用于：根据匹配信息和预设筛选策略，筛选每个块的特征点集合匹配出的匹配特征点，得到至少一个匹配特征点；其中，预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。
在第二方面的一种可能的实现方式中，当第一图像中的一个匹配特征点与第二图像中的第一待匹配特征点和第二待匹配特征点对应时，确定单元，还用于：确定第一图像中的匹配特征点与第一待匹配特征点和第二待匹配特征点中匹配信息最优的待匹配特征点对应。
第三方面，提供一种图像特征点匹配该装置，该装置包括：输入接口、运算器、第一缓存器；其中，输入接口，用于接收第一图像中的N个特征点集合，第一图像包 括N个块，每个块对应一个特征点集合，N为正整数；运算器，用于按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中；运算器，还用于对于第二图像的待匹配特征点，根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，第二图像包括N个块，M为大于或等于1的整数；从第一缓存器中获取M个块中每个块的特征点集合；运算器，还用于根据待匹配特征点逐块匹配M个块中每个块的特征点集合，以从M个块的特征点集合中匹配出至少一个匹配特征点。可选的，该装置还包括：输出接口，用于输出至少一个匹配特征点。
在第三方面的一种可能的实现方式中，第一图像的N个块与第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。
在第三方面的一种可能的实现方式中，运算器，还用于：根据第一图像的尺寸和预设划分规则，将第一图像划分为N个块，并确定第一图像的N个块中每个块的特征点集合；其中，预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。
在第三方面的一种可能的实现方式中，运算器，还用于：根据N个特征点集合中每个特征点集合包括的特征点数量和预设顺序，确定每个块的特征点集合对应的位置索引，该位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引块的特征点集合在第一缓存器中的位置。
在第三方面的一种可能的实现方式中，运算器，还用于：根据待匹配特征点在第二图像的位置和匹配半径，或者根据待匹配特征点在第二图像的位置、匹配半径和预设偏移量，确定匹配区域；根据匹配块选择策略，从第一图像的N个块中选择位于匹配区域内的M个块。
在第三方面的一种可能的实现方式中，运算器，还用于：对于M个块，逐块确定每个块的特征点集合中的每个特征点与待匹配特征点之间的匹配信息；根据匹配信息，将每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。
在第三方面的一种可能的实现方式中，该匹配信息包括：描述子距离和/或匹配角度差。
在第三方面的一种可能的实现方式中，满足预设条件的特征点是指特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
在第三方面的一种可能的实现方式中，运算器，还用于：根据匹配信息和预设筛选策略，筛选每个块的特征点集合匹配出的匹配特征点，得到至少一个匹配特征点；其中，预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。
在第三方面的一种可能的实现方式中，当第一图像中的一个匹配特征点与第二图像中的第一待匹配特征点和第二待匹配特征点对应时，运算器，还用于：确定第一图像中的匹配特征点与第一待匹配特征点和第二待匹配特征点中匹配信息最优的待匹配特征点对应。
本申请的又一方面，提供一种可读存储介质，可读存储介质中存储有指令，当可读存储介质在设备上运行时，使得设备执行第一方面或者第一方面的任一种可能的实 现方式所提供的图像特征点匹配方法。
本申请的又一方面，提供一种计算机程序产品，当计算机程序产品在计算机上运行时，使得计算机执行第一方面或者第一方面的任一种可能的实现方式所提供的图像特征点匹配方法。
可以理解地，上述提供的任一种图像特征点匹配方法的装置、计算机存储介质或者计算机程序产品均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
附图说明
图1为本申请实施例提供的一种图像处理设备的结构示意图；
图2为本申请实施例提供的一种图像特征点匹配方法的流程示意图一；
图3为本申请实施例提供的一种多个特征点集合的存储示意图一；
图4为本申请实施例提供的一种多个特征点集合的存储示意图二；
图5为本申请实施例提供的一种图像特征点匹配方法的流程示意图二；
图6为本申请实施例提供的一种规则形状和非规则形状的示意图；
图7为本申请实施例提供的一种N个块的分布示意图；
图8为本申请实施例提供的一种M个块的分布示意图；
图9为本申请实施例提供的一种图像特征点匹配装置的结构示意图一；
图10为本申请实施例提供的一种图像特征点匹配装置的结构示意图二。
具体实施方式
本申请中，“至少一个”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的情况，其中A，B可以是单数或者复数。“以下至少一项(个)”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，a-b，a-c，b-c或a-b-c，其中a、b和c可以是单个，也可以是多个。字符“/”一般表示前后关联对象是一种“或”的关系。另外，在本申请的实施例中，“第一”、“第二”等字样并不对数量和执行次序进行限定。
需要说明的是，本申请中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
图1为本申请实施例提供的一种图像处理设备的结构示意图，该图像处理设备可以为手机、平板电脑、计算机、笔记本电脑、摄像机、照相机、可穿戴设备、车载设备、或终端设备等等。为方便描述，本申请中将上面提到的设备统称为图像处理设备。本申请实施例以该图像处理设备为手机为例进行说明，该手机包括：存储器101、处理器102、传感器组件103、多媒体组件104、音频组件105和电源组件106等。
下面结合图1对手机的各个构成部件进行具体的介绍：
存储器101可用于存储数据、软件程序以及模块；主要包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需的应用程序，比如声音 播放功能、图像播放功能等；存储数据区可存储根据手机的使用所创建的数据，比如音频数据、图像数据、电话本等。此外，手机可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
处理器102是手机的控制中心，利用各种接口和线路连接整个设备的各个部分，通过运行或执行存储在存储器101内的软件程序和/或模块，以及调用存储在存储器101内的数据，执行手机的各种功能和处理数据，从而对手机进行整体监控。在本申请实施例中，处理器102可集成应用处理器(Application Processor，AP)和专用集成电路(Application Specific integrated Circuit，ASIC)；其中，AP主要处理操作***、用户界面和应用程序等，ASIC是专门为某一项功能开发的专用集成芯片，比如ASIC可用于处理本申请实施例中图像特征点匹配的相关过程等。可以理解的是，上述ASIC也可以不集成到处理器102中。
除此以外，处理器102还可进一步包括其他硬件电路或加速器，如现场可编程门阵列或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器102也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，数字信号处理器和微处理器的组合等。
传感器组件103包括一个或多个传感器，用于为手机提供各个方面的状态评估。其中，传感器组件103可以包括光传感器，如CMOS或CCD图像传感器，用于检测外部物体与手机的距离，或者在成像应用中使用，即成为相机或摄像头的组成部分。此外，传感器组件103还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器，通过传感器组件103可以检测到手机的加速/减速、方位、打开/关闭状态，组件的相对定位，或手机的温度变化等。
多媒体组件104在手机和用户之间的提供一个输出接口的屏幕，该屏幕可以为触摸面板，且当该屏幕为触摸面板时，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。此外，多媒体组件104还包括至少一个摄像头，比如，多媒体组件104包括一个前置摄像头和/或后置摄像头。当手机处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜***或具有焦距和光学变焦能力。
音频组件105可提供用户与手机之间的音频接口，比如，音频组件105可以包括音频电路、扬声器和麦克风。音频电路可将接收到的音频数据转换后的电信号，传输到扬声器，由扬声器转换为声音信号输出；另一方面，麦克风将收集的声音信号转换为电信号，由音频电路接收后转换为音频数据，再输出音频数据以发送给比如另一手机，或者将音频数据输出至处理器102以便进一步处理。
电源组件106用于为手机的各个组件提供电源，电源组件106可以包括电源管理***，一个或多个电源，及其他与手机生成、管理和分配电力相关联的组件。
尽管未示出，手机还可以包括无线保真(Wireless Fidelity，WiFi)模块、蓝牙模 块等，本申请实施例在此不再赘述。本领域技术人员可以理解，图1中示出的手机结构并不构成对手机的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
图2为本申请实施例提供的一种图像特征点匹配方法的流程示意图，该方法的执行主体可以是图像处理设备，具体可以是图像处理设备中具有处理功能和存储功能的单元，比如上述图1所示的处理器102中的ASIC。参见图2，该方法包括以下几个步骤。
S201：按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，第一图像包括N个块，每个块对应一个特征点集合。
其中，第一图像可以包括N个块，N为正整数，且第一图像中N个块的大小和形状可以相同，也可以不同。每个块对应一个特征点集合，可以是指该块内的特征点组成的集合。第一图像包括N个块，每个块对应一个特征点集合，从而第一图像有N个特征点集合，即N个特征点集合与第一图像的N个块一一对应。
另外，预设顺序可以由本领域技术人员根据实际需要事先进行设置，比如预设顺序包括行顺序、列顺序、Z型顺序、X型顺序或者其他顺序等。示例性的，如图3所示，分别按照行顺序、列顺序和Z型顺序将第一图像中的N个特征点集合存储在第一缓存器。需要说明的是，图3中以第一图像包括16个块(即N＝16)、且16个块的大小和形状均相同为例进行说明，每个块的特征点集合可通过该块内的标号进行表示。
具体的，按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中时，可以按照预设顺序将N个特征点集合连续地存储在第一缓存器中，或者按照预设顺序将N个特征点集合非连续地存储在第一缓存器中；其中，同一特征点集合内的特征点之间是连续存储的。示例性的，以上述图3所示的按行顺序存储为例，则第一图像中16个块的特征点集合在第一缓存器中可以如图4所示，图4中的(a)以16个特征点集合为连续存储方式为例，图4中的(b)以16个特征点集合为非连续存储方式为例进行说明。
可选的，在将第一图像中的N个特征点集合存储在第一缓存器中时，还可以确定每个特征点集合对应的位置索引。具体的，根据N个特征点集合中每个特征点集合包括的特征点数量和所述预设顺序，确定每个块的特征点集合对应的位置索引，该位置索引可以包括起始位置索引(用于指示在第一缓存器中的起始位置)和/或结束位置索引(用于指示在第一缓存器中的结束位置)。可选的，若一个块的特征点集合对应的位置索引包括起始位置索引，则该块的特征点集合在第一缓存器中的起始位置可由该起始位置索引确定，结束位置可通过该起始位置索引与该特征点集合包括的特征点数量计算得到；若一个块的特征点集合对应的位置索引包括结束位置索引，则该块的特征点集合在第一缓存器中的起始位置可通过该结束位置索引与该特征点集合包括的特征点数量计算得到，结束位置可由该结束位置索引确定；若一个块的特征点集合对应的位置索引包括起始位置索引和结束位置索引，则该特征点集合在第一缓存器中的起始位置可由该起始位置索引确定，结束位置可由该结束位置索引确定。
示例性的，某一块的特征点集合包括的特征点数量为NUM，若该特征点集合对应的起始位置索引为Add_k，则该特征点集合在第一缓存器中的结束位置索引为 (Add_k+NUM-1)；若该特征点集合对应的结束位置索引为Add_w，则该特征点集合在第一缓存器中的起始位置索引为(Add_w-NUM+1)。示例性的，若第一图像包括m个块、且m个块内的特征点数量为NUM{n_1，n_2，…，n_m)为例，则按行顺序存储，则相对于第一缓存器的起始地址而言，m个块的开始地址可以为ADDR_start{0，n_1，n_1+n_2，…，n_1+n_2+…+n_m-1}，m个块的结束地址可以为ADDR_end{-1+n_1，-1+n_1+n_2，…，-1+sum(NUM)}。
可选的，在确定每个特征点集合对应的位置索引之后，图像处理设备还可以存储第一图像中每个块的特征点集合对应的位置索引，比如，图像处理设备存储的第一图像中每个块的特征点集合对应的位置索引中包括：块标识和位置索引，块标识用于指示该位置索引对应的块。
通常对第一图像进行特征点提取，提取得到的特征点是乱序存放的，比如不同块的特征点之间可能是相邻存储的，同一块的多个特征点是分散存储的，这样会导致后续读取同一块的多个特征点时需要花费较长的搜索时间和寻址计算时间。本申请实施例中通过对N个特征点集合进行有序化存放，即将属于同一块的多个特征点进行连续性存储，将不同块的特征点集合按照一定顺序存储，从而能够提高后续特征点的读取效率。
进一步的，在S201之前，参见图5，该方法还可以包括：S200。
S200：根据第一图像的尺寸和预设划分规则，将第一图像划分为N个块，并确定第一图像的N个块中每个块的特征点集合。
其中，预设划分规则可以由本领域技术人员根据实际情况事先进行设置，比如，预设划分规则可以包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、均匀划分、不均匀划分、按区域划分、规则形状划分、不规则形状划分。贴合图像边缘划分可以是指对完整的图像进行划分，此时是以图像本身的尺寸作为被划分区域的尺寸；不贴合图像边缘划分是指对图像中除边缘区域之外的区域进行划分，该边缘区域可以包括图像的一个或者多个边缘对应的区域，此时是以小于图像本身的尺寸作为被划分区域的尺寸；均匀划分可以是指划分得到的块的大小和形状是相同的；不均匀划分可以是指划分得到的块的大小和形状是不同的；按区域划分可以是指对图像中的不同区域进行不同的划分，比如不同区域对应划分得到的块的形状不同或者大小不同等；规则形状划分是指对图像划分得到的N个块的形状是规则形状，比如图6中的(a)所示的长方形、正方形、圆形、六边形、菱形或者三角形等；不规则形状划分是指对图像划分得到的N个块的形状是不规则形状，比如图6中的(b)所示的椭圆、十字形、梯形、L形、X形或者环形等。
示例性的，如图7所示对上述预设划分规则进行举例说明。图7中的(a)对应的预设划分规则可以是：贴合图像边缘划分+均匀划分；图7中的(b)对应的预设划分规则可以是：不贴合图像边缘划分+均匀划分+规则形状划分；图7中的(c)对应的预设划分规则可以是：贴合图像边缘划分+按区域划分+规则形状划分；图7中的(d)对应的预设划分规则可以是：贴合图像边缘划分+规则形状划分。
需要说明的是，上述所列举的划分规则仅为示例性的，在实际应用时还可以包括其他的划分规则，同时上述图6所示的规则形状和不规则形状也是示例性的，并不对 本申请实施例构成限定。
此外，确定第一图像的N个块中每个块的特征点集合，可以包括：图像处理设备提取第一图像中的特征点，并根据每个特征点的位置和第一图像的N个块中每个块的位置范围，确定属于同一个块的多个特征点，即得到每个块的特征点集合。其中，特征点提取可以采用以下算法中的任一种：尺度不变特征转换(Scale-Invariant Feature Transform，SIFT)、快速鲁棒特征(Speeded Up Robust Feature，SURF)、带方向和旋转的BRIEF算子(Oriented FAST and Rotated BRIEF，ORB)、高斯函数差(Difference of Gaussian，DOG)、学习不变特征变换(Learned Invariant Feature Transform，LIFT)等算法。对第一帧图进行特征点提取，可以是通过不同缩放比例进行多层缩放提取的特征点，相应的坐标可以以映射回第一图像为准。在提取的特征点中记录的常用信息可以包括但不限于：坐标(x，y)、当前特征点所在的缩放层(octave)、响应分数值(response)、缩放比例或缩放系数(scale)、角度值(angle)、描述子(descriptor)、或者索引(index)等信息。此外，对提取得到的特征点可以直接使用，也可以采用不同的筛选方法进行筛选，如octree筛选等，本申请实施例对此不作具体限定。
进一步地，在确定第一图像中每个块的特征点集合时，图像处理设备还可以统计每个块的块统计信息并进行存储。比如，一个块的块统计信息可以包括以下信息中的一个或者多个：块标识(用于唯一标识一个块，比如块序号等)、特征点数量、响应分数信息(比如，最大响应分数值、平均响应分数值、或者最小响应分数值等)、特征点分布信息(比如，基于octave的分布、基于块中心点坐标的分布、响应分数值分布的分布图等)。
需要说明的是，上述对将第一图像划分为N个块的过程与提取第一图像的特征点的过程可以不分先后顺序。当先将第一图像划分为N个块，后提取第一图像的特征点时，可以在提取特征点的过程中统计每个块的块统计信息，这样图像处理设备无需等第一图像的所有特征点提取完成之后再进行统计，从而可以减小等待时间和重新读取第一图像的特征点的时间。
S202：对于第二图像的待匹配特征点，根据该待匹配特征点在第二图像的位置从第一图像的N个块中选择M个块，M为大于或等于1的整数。
其中，第二图像的尺寸与第一图像的尺寸相同，第二图像与第一图像中可以存在相同或者相似的特征，比如，第一图像和第二图像可以是同一视频流中相邻的两帧图像、或者是尺寸相同且相似的两帧图像。可选的，第二图像包括N个块且与第一图像的N个块一一对应，对应的两个块的形状、大小和位置可以是相同的。第二图像中的每个块可以有多个待匹配特征点，每个块中的多个待匹配特征点可以是该块内提取到的多个特征点。
另外，该待匹配特征点在第二图像中的位置与其对应在第一图像中的位置对应，即可以认为该待匹配特征点在第二图像中的位置与其对应在第一图像中的位置相同。比如，第一图像和第二图像通过相同的坐标系进行表示，该待匹配特征点在两个坐标系中的位置相同。
具体的，当第一图像和第二图像中相同或者相似特征的位置相差较小或者无变化时，可以根据该待匹配特征点在第二图像的位置和匹配半径确定匹配区域，比如在第 一图像中，以该待匹配特征点对应在第一图像中的位置为圆心(或者称为选取中心点)，以匹配半径为半径，确定的圆形区域作为匹配区域。或者，当第一图像和第二图像中相同或者相似特征的位置相差较大时，还可以根据该待匹配特征点在第二图像的位置、匹配半径和预设偏移量确定初始匹配区域，比如在第一图像中，将该待匹配特征点对应在第一图像中的位置按照预设偏移量偏移之后的位置作为圆心，以匹配半径作为半径，确定的圆形区域作为匹配区域。之后，根据匹配块选择策略，从第一图像的N个块中选择位于该匹配区域内的M个块，比如，确定所述M个块中每个块对应的块标识。可选的，上述匹配块选择策略可以是矩形块集合、X型块集合、十字型块集合或者菱形块集合等。
需要说明的是，上述匹配半径、预设偏移量、以及匹配块选择策略均可以由本领域技术人员根据实际情况事先进行设置或者在匹配过程中动态进行设置，比如根据待匹配特征点所在块的形状、大小、以及上述预设划分规则等进行设置；或者根据相关经验进行设置，或者使用配套的算法等自适应性的配置等。对于不同的待匹配特征点所在的块或者区域，可以设置相同的匹配半径、预设偏移量或者匹配块选择策略，也可以设置不同的匹配半径、预设偏移量或者匹配块选择策略，本申请实施例对此不作具体限定。
示例性的，结合图3所示的第一图像的N个块，假设以该待匹配特征点对应在第一图像中的位置为圆心、以匹配半径为半径，确定的匹配区域如图8所示，则当上述匹配块选择策略为矩形块集合时，选择的M个块如图8中的(a)所示；当上述匹配块选择策略为十字型块集合时，选择的M个块如图8中的(b)所示；当上述匹配块选择策略为X型块集合时，选择的M个块如图8中的(c)所示。图4中上述圆心位于第一图像的标号10所在的块为例进行说明，图8中的圆所在区域表示上述匹配区域。
S203：从第一缓存器中获取M个块中每个块的特征点集合。
具体的，图像处理设备根据M个块中每个块的块标识，从存储每个块的特征点集合对应的位置索引的存储空间中，获取M个块中每个块的特征点集合对应的位置索引。进而，根据M个块中每个块的特征点集合对应的位置索引，从第一缓存器中获取每个块的特征点集合。可选的，可以一次性读取一个块的特征点集合，也可以连续逐个地读取一个块的特征点集合；或者，可以从一个块的特征点集合中一次并行读取多个特征点，也可以从M个块中多个块的特征点集合中并行读取不同块的特征点等(当以并行的方式读取时，后续步骤也可以并行执行)。
S204：根据该待匹配特征点逐块匹配M个块中每个块的特征点集合，以从M个块的特征点集合中匹配出至少一个匹配特征点。
对于M个块，可以使用该待匹配特征点逐块匹配M个块中每个块的特征点集合(即以块为单位进行匹配)，进而从M个块的特征点集合中匹配出至少一个匹配特征点，匹配得到的至少一个匹配特征点中的每个匹配特征点与该待匹配特征点可以形成一个匹配对。在匹配过程中，可以使用最佳匹配(best matcher)算法、KNN匹配算法、或者左右双向匹配算法等对该块的特征点集合与该待匹配特征点进行匹配。其中，最佳匹配算法可以是指对于该待匹配特征点，从该块的特征点集合中选择一个最相似的特征点作为该待匹配特征点的匹配特征点。KNN(n＝2)匹配算法可以是指对于该待 匹配特征点，从该块的特征点集合中选择n(比如，n＝2)个最相似的匹配点作为该待匹配特征点的匹配特征点。左右双向匹配算法可以是指对于该待匹配特征点(表示为特征点P(x,y))，从该块的特征点集合中选择一个最相似的特征点(表示为特征点Q(x,y))，同时对于特征点Q(x,y)，使用同样的方式从第一图像中选择一个最相似的特征点(表示为特征点P’(x,y))，若P’(x,y)与P(x,y)一致，则确定特征点Q(x,y)为该待匹配特征点P(x,y)的匹配特征点，否则匹配无效；使用左右双向匹配算法需要对匹配过程执行2次，所以匹配代价较大。上述三种匹配算法中，使用最佳匹配算法得到的至少一个匹配特征点的数量与M个块的数量相同，使用KNN匹配算法得到的至少一个匹配特征点的数量大于(此时，n≥2)或等于(此时，n＝1)M个块的数量，使用左右双向匹配算法得到的至少一个匹配特征点的数量小于或等于M个块的数量。
需要说明的是，图像处理设备在匹配时还可以使用其他不同的匹配算法，上述仅以最佳匹配算法、KNN匹配算法和左右双向匹配算法为例进行说明，并不对本申请实施例构成限定。
进一步地，对于M个块中每个块的特征点集合，在匹配过程中，图像处理设备还可以确定该特征点集合中的每个特征点与该待匹配特征点之间的匹配信息，该匹配信息可以包括：描述子距离和/或匹配角度差。之后，图像处理设备根据该匹配信息，将该特征点集合中满足预设条件的特征点确定为匹配特征点，这里满足预设条件的特征点可以是指该特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
其中，该描述子距离可以是曼哈顿距离、欧几里德距离、汉明距离等。以匹配的两个特征点的描述子des1(x1，x2，x3，…，xn)与描述子des2(y1，y2，y3，…，yn)为例，其曼哈顿距离(D1)的计算可以如下公式(1)所示，欧几里德距离(D2)的计算可以如下公式(2)所示，汉明距离(D3)的计算可以如下公式(3)所示。上述公式(3)中的XOR为逻辑运算的异或运算符。
D1＝|x1–y1|+|x2-y2|+…+|xn-yn|    (1)
D2＝((x1–y1)^2+(x2-y2)^2+…+(xn–yn)^2)^0.5  (2)
D3＝XOR(x1，y1)+XOR(x2，y2)+…+XOR(xn，yn)  (3)
另外，最佳描述子距离可以用于表示两个特征点的描述子最相似的距离，比如对于上述曼哈顿距离、欧几里德距离或者汉明距离而言，最佳描述子距离可以是指计算得到的所有描述子距离中最小非负值，此时对应的匹配特征点可以称为最佳匹配特征点。相应的，次佳描述子距离可以是指该特征点集合对应的所有描述子距离中仅次于最佳描述子距离的描述子距离，对应的匹配特征点可以称为次佳匹配特征点。
匹配角度差可以是指匹配的两个特征点的角度之差，每个特征点的角度在提取该特征点时即可得到。最佳匹配角度差可以是指该特征点集合对应的所有角度差中的最小值，此时对应的匹配特征点可以称为最佳匹配特征点；次佳匹配角度可以是指该特征点集合对应的所有角度差中仅次于最佳匹配角度差的匹配角度差，对应的匹配特征点可以称为次佳匹配特征点。
可选的，在匹配过程中，图像处理设备还可以将特征点匹配结果(即上述匹配对信息)、特征点的匹配状态和相关统计信息等进行存储。
具体的，在匹配M个块中每个块的特征点集合的过程中，图像处理设备可以将特征点匹配结果存储在第二缓存器中，第二缓存器可以包括：待匹配特征点的索引号、最佳匹配特征点的索引号、最佳描述子距离、最佳匹配角度差等中的一个或者多个。可选的，第二缓存器中还可以包括：次佳匹配特征点的索引号、次佳描述子距离、次佳匹配角度差等。
对于该特征点集合中的每个特征点，图像处理设备还可以将该特征点的匹配状态存储在状态寄存器中，该状态寄存器可以包括：该特征点的索引号、该特征点所在块的索引号、与该特征点匹配的待匹配特征点的索引号、与该特征点匹配的待匹配特征点所在块的索引号，最佳描述子距离、最佳匹配角度差等信息中的一个或者多个。
对于该特征点集合，图像处理设备还可以将该特征点集合对应的匹配角度差的统计信息(比如，角度差直方图)存储在角度差统计寄存器中。比如，该角度差统计寄存器对[0，360]度角度均匀划分成m(m为正整数)个区间，每个区间为360/m度，统计m个区间(比如m＝12，即均匀分成12个区间，每个区间为30度，m个区间范围可以为[0,30)、[30,60)、……、[330，360))中属于每个区间范围的匹配角度差的个数，如果某一匹配角度差为负值，则可以将匹配角度差加上360变成正数。比如，某一匹配特征点与待匹配特征点的匹配角度差为45度，则45度属于区间[30,60)，[30,60)对应的特征点统计个数加1，该匹配特征点属于[30,60)区间，可通过[30,60)对应的角度区间索引(比如，序号)来表示，比如45/(360/12)向上取整＝2，即该匹配特征点属于角度序号为2的区间。
对于M个块中的每个块，图像处理设备还可以将该块的块匹配统计信息存储在块统计寄存器中，该块统计寄存器主要用于存储块信息和块的匹配结果等。该块统计寄存器可以包括：待匹配特征点所在块的索引号、第一图像中块的索引号、有效匹配点数、块内匹配特征点对应的最佳描述子距离的均值/最大值/最小值等。在实际应用中，图像处理设备还可以根据后续应用的需求对该块统计寄存器中的各项信息进行增加或者删除等，本申请实施例对此不作具体限定。
进一步地，对于第二图像中的所有待匹配特征点，均可以按照上述步骤处理得到每个待匹配特征点对应的匹配特征点，从而实现第一图像与第二图像的特征点匹配。
可选的，当第一图像中某个块的某一匹配特征点与第二图像中的第一待匹配特征点和第二待匹配特征点均对应时，可以确定第一图像中的所述匹配特征点与第一待匹配特征点和第二待匹配特征点中匹配信息最优的待匹配特征点对应，即将所述匹配特征点作为匹配信息最优的待匹配特征点的匹配特征点。
在实际应用时，图像处理设备可以是以流水作业的方式进行处理的，上述第二缓存器也是按照流水作业的方式存储相应的信息，上述状态寄存器中也会记录每个特征点的匹配状态，本申请实施例可以采用如下方法处理上述重复匹配的问题，从而避免对已经写入的内存进行再次修改。
具体的，如果待匹配特征点(表示为P(x,y))匹配后得到最佳匹配特征点(表示为Q(x,y))，Q(x,y)在上述状态寄存器中标记还未被匹配，则待匹配特征点P(x,y)与Q(x,y)匹配有效，将P(x,y)与Q(x,y)匹配的相关信息写入上述状态寄存器中。如果待匹配特征点P(x,y)匹配后得到最佳匹配特征点Q(x,y)、最佳描述子距离(表示为dist0)， Q(x,y)在上述状态寄存器中标记已被匹配，被匹配点为特征点K(x,y)\最佳描述子距离为dist1，则比较最佳匹配距离dist1与dist0，如果dist0>＝dist1，则P(x,y)与Q(x,y)不是最优的，丢弃掉；如果dist0<dist1，则P(x,y)与Q(x,y)更优需要保留，更新上述状态寄存器中特征点Q(x,y)对应的最佳匹配点为P(x,y)，并更新最佳描述子为dist0和其他统计信息，将P(x,y)与Q(x,y)匹配的相关信息写入上述状态寄存器中。
此外，在对第二图像中的所有待匹配特征点均进行匹配之后，图像处理设备还可以对上述状态寄存器和角度差统计寄存器进行更新，以避免因为处理上述重复匹配的问题而导致寄存器信息与实际情况不一致的问题。
进一步地，在S204中，图像处理设备还可以根据该匹配信息和预设筛选策略，筛选第一图像的每个块内的匹配特征点，得到至少一个匹配特征点。预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、块描述子距离筛选、或者块匹配统计筛选。
对于第一图像和第二图像匹配之后得到的所有匹配对(即第二图像中的每个待匹配特征点与其匹配得到的至少一个匹配特征点形成的匹配对)，因为图像纹理存在相似性等因素而使得匹配结果中存在误匹配的问题，因此图像处理设备可以对得到的所有匹配对进行筛选。下面以一个匹配对为例进行说明，每个匹配对都可以采用相同的方法进行筛选，直到筛选完所有匹配对为止。
具体的，图像处理设备使用重复匹配筛选策略的过程可以为：从第二缓存器中分别读取(比如，从起始地址开始读取)每个匹配对信息(即特征点匹配结果)，比如读取的匹配对信息为第二图像中的特征点P(x,y)与第一图像中的特征点Q(x,y)的匹配结果，即图像处理设备根据每个匹配对信息查询上述状态寄存器，若查询得到的匹配结果与该匹配对信息一致，则确定当前匹配对有效，否则确定当前匹配对无效，比如根据特征点Q(x,y)查询得到的特征点为P’(x,y)，若P’(x,y)与P(x,y)一致，则确定P(x,y)与Q(x,y)有效，否则确定P(x,y)与Q(x,y)无效。当确定P(x,y)与Q(x,y)无效时，图像处理设备可以在第二缓存器中将其标记为无效，或者直接从第二缓存器中删除相关信息。由于上述状态寄存器是在处理完上述重复匹配的问题之后更新的，所以上述状态寄存器的信息是最新的，无重复匹配对。
上述角度差直方图筛选的原理在于：两张图像中的物体经过相似的空间变换，则正确匹配对的角度空间分配应该是相似的，所以分布应该是类似的，异常的角度差分布则可以认为是错误匹配。具体的，图像处理设备使用角度差直方图筛选策略的过程可以为：从上述角度差统计寄存器中读取第一图像中每个块的特征点集合对应的匹配角度差的统计信息，分析该统计信息的波形分布，并获取最大k(k≤m，k为正整数)个波峰所对应的角度序号集合{angle1，angle2，…，anglek}，即表示最可能的角度差分布空间；根据先验信息、大数据分析、机器学习等方法从{angle1，angle2，…，anglek}中获得有效的w(1<＝w<＝k，w为正整数)个波峰集合ANGLE{bangle_1，bangle_2，…，bangle_m}，或者由外部动态输入，该波峰集合ANGLE用来筛选匹配对；从第二缓存器中分别读取每个匹配对信息(即特征点匹配结果)，该特征点匹配结果中包括所属的角度区间索引bangle；如果角度区间索引bangle属于上述ANGLE{bangle_1，bangle_2，…，bangle_m}集合中，则确定当前匹配对有效，否则确定当前匹配对无效。 当确定当前匹配对无效时，图像处理设备可以在第二缓存器中将其标记为无效，或者直接从第二缓存器中删除相关信息。
块描述子距离筛选的原理在于：两张图像中的物体经过相似的空间变换，则正确匹配对所在的块也应该是相似的，所以块描述子距离应该是相似的，异常的块描述子距离可以认为是错误匹配。具体的，图像处理设备使用块描述子距离筛选策略的过程可以为：从上述块统计寄存器中读取第一图像中每个块的特征点集合对应的最佳描述子距离的统计信息，根据该统计信息中的最佳描述子距离的均值/最大值/最小值、或者最佳描述子距离的均值/最大值/最小值和每个块的有效匹配特征点数确定F个分块的判决阈值BDIST{dist_1，dist_2，…，dist_f}，或者基于先验信息、大数据分析、机器学习等方法获取，或者由外部动态输入来获取；从第二缓存器中分别读取每个匹配对信息(即特征点匹配结果)，该特征点匹配结果中包括当前匹配对的最佳描述子距离bdist和对应在第二图像中的块标识blk_i；查询blk_i在上述BDIST{dist_1，dist_2，…，dist_f}中对应的判决阈值为dist_i，若bdist<＝dist_i，则当前匹配对有效，否则当前匹配对无效。当确定当前匹配对无效时，图像处理设备可以在第二缓存器中将其标记为无效，或者直接从第二缓存器中删除相关信息。
上述块匹配统计筛选的原理在于：两张图像中的物体经过相似的空间变换，原先在第一图像中临近区域的特征点，经过相似的变换之后，在第二图像中也应该是在临近区域，所以可以通过分析第一图像中各个块的特征点映射到第二图像的特征点的聚合性来分析匹配对的有效性，即在第一图像中相同块中的特征点，在第二图像也应该聚集在相似的区域，空间跨度很大的异常分布的匹配对可以认为是错误匹配。具体的，图像处理设备使用块描述子距离筛选策略的过程可以为：从上述状态寄存器中读取每个匹配对应在第一图像中的块标识和在第二图像中的块标识；从上述块统计寄存器中读取第一图像的每个块的有效匹配特征点数，假设有A个块，A个块的有效匹配特征点数分别为NUM{b_1，b_2，…，b_a}，总点数为B；获取第一图像的每个块内的有效匹配特征点对应在第二图像中的块标识集合，得到分布块标识数组BLKNUM[B]，根据分布块标识数组BLKNUM[B]确定第一图像的每个块对应在第二图像中的最佳匹配块和次佳匹配块(可选的，存储在上述块统计寄存器中)；从上述第二缓存器中确定每个匹配对信息(即特征点匹配结果)，该特征点匹配结果包括当前匹配对的最佳描述子距离bdist、对应在第一图像中的块标识blk_i和对应在第二图像中的块标识blk_j；根据读取的匹配对信息查询上述块统计寄存器以获取blk_i所对应的blk_bestindex和blk_best2index，blk_bestinde表示blk_i对应的最佳匹配块，blk_best2index表示blk_i对应的次佳匹配块。选择有效块范围策略：
1)以blk_bestindex为有效匹配块策略：如果blk_j等于blk_bestindex，则当前匹配对有效，否则当前匹配对无效；
2)以blk_bestindex和blk_best2index构成有效区块策略：比如以blk_bestindex和blk_best2index连线所覆盖的块构成集合BLK_SET为有效集合，如果blk_j落在块集合BLK_SET内，则当前匹配对有效，否则当前匹配对无效；
3)以blk_bestindex块为中心的多块策略：以blk_bestindex块为中心的矩形块(矩形块大小可设置)、十字型块、X型块等构成的集合BLK_SET，如果blk_j落在块集 合BLK_SET内，则当前匹配对有效，否则当前匹配对无效。
需要说明的是，上述仅以不同的筛选策略为例进行说明，在实际应用中，图像处理设备还可以使用上述筛选策略中的两个或者两个以上的筛选策略同时进行筛选，或者采用其他的筛选策略等进行筛选。通过基于上述不同信息的筛选，可以提高正确匹配率，同时保证较高的筛选效率。
进一步的，图像处理设备还可以输出上述第二缓存器和各个寄存器的信息，以输出最终的匹配结果，如果执行最佳匹配，则输出最佳匹配的相关结果，可以选择不输出次佳匹配的相关结果；如果执行最佳匹配和次佳匹配，则可以输出最佳匹配的相关结果和次佳匹配的相关结果。可选的，图像处理设备还可以在上述筛选过程中输出匹配结果，比如，在确定一个匹配对为有效匹配对时直接输出该匹配对，在确定一个匹配对为无效匹配对时不输出该匹配对。
需要说明的是，当由ASIC执行本申请的方法时，ASIC可以通过流水式方式匹配待匹配特征点与第一图像的M个块中每个块的特征点集合，从而可以有效地节省ASIC中的存储资源，同时提高特征点的匹配效率。
上述主要从图像处理设备的角度对本申请实施例提供的筛选局部特征点的方法进行了介绍。可以理解的是，该图像处理设备为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的网元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对筛选局部特征点的装置(例如，图像处理设备)进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图9示出了上述实施例中所涉及的一种图像特征点匹配装置的结构示意图，该装置包括：存储单元901、确定单元902、获取单元903和匹配单元904。其中，存储单元901用于支持该装置执行上述方法实施例中的S201；确定单元902用于支持该装置执行上述方法实施例中的S202和S200；获取单元903用于支持该装置执行上述方法实施例中的S203；匹配单元904用于支持该装置执行上述方法实施例中的S204。上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
图10为本申请实施例所涉及的另一种图像特征点匹配装置的结构示意图，该装置可以为图像处理设备中的处理器(比如，图1中的ASIC)，该装置包括：输入接口1001、运算器1002和缓存器1003，缓存器1003包括第一缓存器。可选的，该装置还可以包括输出接口1004。
其中，输入接口1001可用于支持该装置接收第一图像中的N个特征点集合、第一图像的尺寸、匹配半径、预设偏移量、第一图像、以及第二图像等。在一些可行的 实施例中，该输入接口1001接收的信息可以存储在外部的存储器中，比如图1所示的图像处理设备的存储器101中；在一些可行的实施例中，可以只有一个输入接口，也可以有多个输入接口。
运算器1002，可用于支持该装置执行上述方法实施例中的S200-S204，和/或本文所描述的其他技术过程，具体描述参见上述方法实施例中的相关描述，本申请实施例在此不再赘述。
缓存器1003，可用于缓存运算器1002执行上述方法实施例过程中需要缓存的相关信息，比如，第一图像中的N个特征点集合排列后的位置索引、第一图像中的特征点与待匹配特征点之间的匹配信息、以及满足预设条件的特征点等。缓存器1003还可以包括上述方法实施例中的第二缓存器、状态寄存器、角度差统计寄存器等。
输出接口1004，可用于输出从M个块的特征点集合中匹配出至少一个匹配特征点等。在一些可行的实施例中，可以只有一个输出接口，也可以有多个输出接口。在一些可行的实施例中，该输出接口1004输出的处理结果可以送到外部的存储器中存储，也可以送到另外的处理流程中继续进行处理等。
本申请实施例提供的上述图像特征点匹配装置的各组成部分分别用于实现相对应的前述图像特征点匹配方法的各步骤的功能，由于在前述的图像特征点匹配方法实施例中，已经对各步骤进行了详细说明，在此不再赘述。
在本申请实施例中，按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，能够提高后续获取M个块中每个块的特征点集合的效率，同时根据待匹配特征点在第二图像的位置从第一图像的N个块中确定M个块，能够有效减小匹配范围，进而后续根据待匹配特征点逐块匹配M个块中每个块的特征点集合时，能够以一个块的特征点集合为单位进行匹配，M个块的特征点集合通过流水式操作逐块进行匹配，从而降低了匹配过程中的内存消耗和匹配时间，提高了特征点匹配的效率。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质 上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得终端执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (30)

  1. 一种图像特征点匹配方法，其特征在于，所述方法包括：
    按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，所述第一图像包括N个块，每个块对应一个特征点集合，所述N为正整数；
    对于第二图像的待匹配特征点，根据所述待匹配特征点在所述第二图像的位置从所述第一图像的N个块中确定M个块，所述第二图像包括N个块，所述M为大于或等于1的整数；
    从所述第一缓存器中获取所述M个块中每个块的特征点集合；
    根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，以从所述M个块的特征点集合中匹配出至少一个匹配特征点。
  2. 根据权利要求1所述的方法，其特征在于，所述第一图像的N个块与所述第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。
  3. 根据权利要求1或2所述的方法，其特征在于，所述按照指定顺序将第一图像中的N个特征点集合存储在第一缓存器中之前，所述方法还包括：
    根据所述第一图像的尺寸和预设划分规则，将所述第一图像划分为N个块，并确定所述第一图像的N个块中每个块的特征点集合；其中，所述预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述方法还包括：
    根据N个特征点集合中每个特征点集合包括的特征点数量和所述预设顺序，确定每个块的特征点集合对应的位置索引，所述位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引所述块的特征点集合在所述第一缓存器中的位置。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述根据所述待匹配特征点在所述第二图像的位置从所述第一图像的N个块中确定M个块，包括：
    根据所述待匹配特征点在所述第二图像的位置和匹配半径，或者根据所述待匹配特征点在所述第二图像的位置、匹配半径和预设偏移量，确定匹配区域；
    根据匹配块选择策略，从所述第一图像的N个块中选择位于所述匹配区域内的M个块。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，以从所述M个块的特征点集合中匹配出至少一个匹配特征点，包括：
    对于所述M个块，逐块确定每个块的特征点集合中的每个特征点与所述待匹配特征点之间的匹配信息；
    根据所述匹配信息，将所述每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。
  7. 根据权利要求6所述的方法，其特征在于，所述匹配信息包括：描述子距离和/或匹配角度差。
  8. 根据权利要求7所述的方法，其特征在于，所述满足预设条件的特征点是指所述特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
  9. 根据权利要求6-8任一项所述的方法，其特征在于，所述根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，以从所述M个块的特征点集合中匹配出至少一个匹配特征点，还包括：
    根据所述匹配信息和预设筛选策略，筛选所述每个块的特征点集合匹配出的匹配特征点，得到所述至少一个匹配特征点；其中，所述预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。
  10. 根据权利要求6-9任一项所述的方法，其特征在于，当所述第一图像中的一个匹配特征点与所述第二图像中的第一待匹配特征点和第二待匹配特征点对应时，所述方法还包括：
    确定所述第一图像中的所述匹配特征点与所述第一待匹配特征点和所述第二待匹配特征点中匹配信息最优的待匹配特征点对应。
  11. 一种图像特征点匹配装置，其特征在于，所述装置包括：
    存储单元，用于按照预设顺序将第一图像中的N个特征点集合存储在第一缓存器中，所述第一图像包括N个块，每个块对应一个特征点集合，所述N为正整数；
    确定单元，用于对于第二图像的待匹配特征点，根据所述待匹配特征点在所述第二图像的位置从所述第一图像的N个块中确定M个块，所述第二图像包括N个块，所述M为大于或等于1的整数；
    获取单元，用于从所述第一缓存器中获取所述M个块中每个块的特征点集合；
    匹配单元，用于根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，得到至少一个匹配特征点。
  12. 根据权利要求11所述的装置，其特征在于，所述第一图像的N个块与所述第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。
  13. 根据权利要求11或12所述的装置，其特征在于，所述确定单元，还用于：
    根据所述第一图像的尺寸和预设划分规则，将所述第一图像划分为N个块，并确定所述第一图像的N个块中每个块的特征点集合；其中，所述预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。
  14. 根据权利要求11-13任一项所述的装置，其特征在于，所述确定单元，还用于：
    根据N个特征点集合中每个特征点集合包括的特征点数量和所述预设顺序，确定每个块的特征点集合对应的位置索引，所述位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引所述块的特征点集合在所述第一缓存器中的位置。
  15. 根据权利要求11-14任一项所述的装置，其特征在于，所述确定单元，还用于：
    根据所述待匹配特征点在所述第二图像的位置和匹配半径，或者根据所述待匹配特征点在所述第二图像的位置、匹配半径和预设偏移量，确定匹配区域；
    根据匹配块选择策略，从所述第一图像的N个块中选择位于所述匹配区域内的M个块。
  16. 根据权利要求11-15任一项所述的装置，其特征在于，所述匹配单元，还用于：
    对于所述M个块，逐块确定每个块的特征点集合中的每个特征点与所述待匹配特征点之间的匹配信息；
    根据所述匹配信息，将所述每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。
  17. 根据权利要求16所述的装置，其特征在于，所述匹配信息包括：描述子距离和/或匹配角度差。
  18. 根据权利要求17所述的装置，其特征在于，所述满足预设条件的特征点是指所述特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
  19. 根据权利要求16-18任一项所述的装置，其特征在于，所述匹配单元，还用于：
    根据所述匹配信息和预设筛选策略，筛选所述每个块的特征点集合匹配出的匹配特征点，得到所述至少一个匹配特征点；其中，所述预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。
  20. 根据权利要求16-19任一项所述的装置，其特征在于，当所述第一图像中的一个匹配特征点与所述第二图像中的第一待匹配特征点和第二待匹配特征点对应时，所述确定单元，还用于：
    确定所述第一图像中的所述匹配特征点与所述第一待匹配特征点和所述第二待匹配特征点中匹配信息最优的待匹配特征点对应。
  21. 一种图像特征点匹配装置，其特征在于，所述装置包括：输入接口、运算器和第一缓存器；其中，
    所述输入接口，用于接收第一图像中的N个特征点集合，所述第一图像包括N个块，每个块对应一个特征点集合，所述N为正整数；
    所述运算器，用于按照预设顺序将所述第一图像中的N个特征点集合存储在所述第一缓存器中；
    所述运算器，还用于对于第二图像的待匹配特征点，根据所述待匹配特征点在所述第二图像的位置从所述第一图像的N个块中确定M个块，所述第二图像包括N个块，所述M为大于或等于1的整数；从所述第一缓存器中获取所述M个块中每个块的特征点集合；
    所述运算器，还用于根据所述待匹配特征点逐块匹配所述M个块中每个块的特征点集合，以从所述M个块的特征点集合中匹配出至少一个匹配特征点。
  22. 根据权利要求21所述的装置，其特征在于，所述第一图像的N个块与所述 第二图像的N个块一一对应，且对应的两个块的形状和尺寸相同。
  23. 根据权利要求21或22所述的装置，其特征在于，所述运算器，还用于：
    根据所述第一图像的尺寸和预设划分规则，将所述第一图像划分为N个块，并确定所述第一图像的N个块中每个块的特征点集合；其中，所述预设划分规则包括以下规则中的至少一个：贴合图像边缘划分、不贴合图像边缘划分、按区域划分、规则形状划分、不规则形状划分。
  24. 根据权利要求21-23任一项所述的装置，其特征在于，所述运算器，还用于：
    根据N个特征点集合中每个特征点集合包括的特征点数量和所述预设顺序，确定每个块的特征点集合对应的位置索引，所述位置索引包括起始位置索引和/或结束位置索引，一个块的特征点集合对应的位置索引用于索引所述块的特征点集合在所述第一缓存器中的位置。
  25. 根据权利要求21-24任一项所述的装置，其特征在于，所述运算器，还用于：
    根据所述待匹配特征点在所述第二图像的位置和匹配半径，或者根据所述待匹配特征点在所述第二图像的位置、匹配半径和预设偏移量，确定匹配区域；
    根据匹配块选择策略，从所述第一图像的N个块中选择位于所述匹配区域内的M个块。
  26. 根据权利要求21-25任一项所述的装置，其特征在于，所述运算器，还用于：
    对于所述M个块，逐块确定每个块的特征点集合中的每个特征点与所述待匹配特征点之间的匹配信息；
    根据所述匹配信息，将所述每个块的特征点集合中满足预设条件的特征点确定为匹配特征点。
  27. 根据权利要求26所述的装置，其特征在于，所述匹配信息包括：描述子距离和/或匹配角度差。
  28. 根据权利要求27所述的装置，其特征在于，所述满足预设条件的特征点是指所述特征点的匹配信息满足以下至少一项：最佳描述子距离、次佳描述子距离、最佳匹配角度差、或者次佳匹配角度。
  29. 根据权利要求26-28任一项所述的装置，其特征在于，所述运算器，还用于：
    根据所述匹配信息和预设筛选策略，筛选所述每个块的特征点集合匹配出的匹配特征点，得到所述至少一个匹配特征点；其中，所述预设筛选策略包括以下策略中的至少一个：重复匹配筛选、角度差直方图筛选、描述子距离筛选、或者块匹配统计筛选。
  30. 根据权利要求26-29任一项所述的装置，其特征在于，当所述第一图像中的一个匹配特征点与所述第二图像中的第一待匹配特征点和第二待匹配特征点对应时，所述运算器，还用于：
    确定所述第一图像中的所述匹配特征点与所述第一待匹配特征点和所述第二待匹配特征点中匹配信息最优的待匹配特征点对应。
PCT/CN2018/117965 2018-11-28 2018-11-28 一种图像特征点匹配方法及装置 WO2020107267A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/117965 WO2020107267A1 (zh) 2018-11-28 2018-11-28 一种图像特征点匹配方法及装置
CN201880093374.9A CN112119426A (zh) 2018-11-28 2018-11-28 一种图像特征点匹配方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/117965 WO2020107267A1 (zh) 2018-11-28 2018-11-28 一种图像特征点匹配方法及装置

Publications (1)

Publication Number Publication Date
WO2020107267A1 true WO2020107267A1 (zh) 2020-06-04

Family

ID=70854166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/117965 WO2020107267A1 (zh) 2018-11-28 2018-11-28 一种图像特征点匹配方法及装置

Country Status (2)

Country Link
CN (1) CN112119426A (zh)
WO (1) WO2020107267A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022262386A1 (zh) * 2021-06-18 2022-12-22 哲库科技(上海)有限公司 图像处理装置与方法、处理芯片和电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113052945B (zh) * 2021-03-09 2024-05-10 中国人民解放军陆军防化学院 一种要图计算机自动评判方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103426186A (zh) * 2013-09-05 2013-12-04 山东大学 一种改进的surf快速匹配方法
CN103955888A (zh) * 2014-05-12 2014-07-30 中国人民解放军空军预警学院监控***工程研究所 一种基于sift的高清视频图像拼接方法及装置
CN104778682A (zh) * 2014-01-14 2015-07-15 三星泰科威株式会社 对特征点采样的方法、图像匹配方法和图像匹配设备

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6168355B2 (ja) * 2011-12-16 2017-07-26 日本電気株式会社 情報処理システム、情報処理方法、通信端末およびその制御方法と制御プログラム
CN108875451B (zh) * 2017-05-10 2023-04-07 腾讯科技（深圳）有限公司 一种定位图像的方法、装置、存储介质和程序产品
CN107980140B (zh) * 2017-10-16 2021-09-14 厦门熵基科技有限公司 一种掌静脉的识别方法及装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103426186A (zh) * 2013-09-05 2013-12-04 山东大学 一种改进的surf快速匹配方法
CN104778682A (zh) * 2014-01-14 2015-07-15 三星泰科威株式会社 对特征点采样的方法、图像匹配方法和图像匹配设备
CN103955888A (zh) * 2014-05-12 2014-07-30 中国人民解放军空军预警学院监控***工程研究所 一种基于sift的高清视频图像拼接方法及装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022262386A1 (zh) * 2021-06-18 2022-12-22 哲库科技(上海)有限公司 图像处理装置与方法、处理芯片和电子设备

Also Published As

Publication number Publication date
CN112119426A (zh) 2020-12-22

Similar Documents

Publication Publication Date Title
US11610362B2 (en) Data volume sculptor for deep learning acceleration
US10096091B2 (en) Image generating method and apparatus
US9665542B2 (en) Determining median value of an array on vector SIMD architectures
WO2019222889A1 (zh) 一种图像特征提取方法及装置
US9275275B2 (en) Object tracking in a video stream
CN109598250B (zh) 特征提取方法、装置、电子设备和计算机可读介质
US11763470B2 (en) Method and apparatus for aligning image frames, electronic device and storage medium
WO2019218388A1 (zh) 一种事件数据流的处理方法及计算设备
US10929965B2 (en) Histogram statistics circuit and multimedia processing system
WO2022017006A1 (zh) 视频处理方法、装置、终端设备及计算机可读存储介质
WO2020107267A1 (zh) 一种图像特征点匹配方法及装置
KR102421604B1 (ko) 이미지 처리 방법, 장치 및 전자 기기
CN111210506A (zh) 一种三维还原方法、***、终端设备和存储介质
JP7204786B2 (ja) 視覚的検索方法、装置、コンピュータ機器及び記憶媒体
TWI783572B (zh) 物件追蹤方法及物件追蹤裝置
CN111401206A (zh) 一种全景图共享方法、***、设备和介质
CN112580435B (zh) 人脸定位方法、人脸模型训练与检测方法及装置
WO2022165675A1 (zh) 一种手势识别方法、装置、终端设备及可读存储介质
WO2020244076A1 (zh) 人脸识别方法、装置、电子设备及存储介质
WO2019237286A1 (zh) 一种筛选局部特征点的方法及装置
WO2021139178A1 (zh) 图像合成方法及相关设备
WO2021087773A1 (zh) 识别方法、装置、电子设备及存储介质
JP2022534314A (ja) ピクチャに基づいた多次元情報の統合方法及び関連機器
US20140072244A1 (en) Image processing and recording system and method thereof
CN112733565A (zh) 二维码粗定位方法、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18941939

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18941939

Country of ref document: EP

Kind code of ref document: A1