CN114429192B - Image matching method and device and electronic equipment - Google Patents

Image matching method and device and electronic equipment Download PDF

Info

Publication number
CN114429192B
CN114429192B CN202210353580.7A CN202210353580A CN114429192B CN 114429192 B CN114429192 B CN 114429192B CN 202210353580 A CN202210353580 A CN 202210353580A CN 114429192 B CN114429192 B CN 114429192B
Authority
CN
China
Prior art keywords
key point
dynamic
activation
keypoint
detector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210353580.7A
Other languages
Chinese (zh)
Other versions
CN114429192A (en
Inventor
张天柱
张哲�
高源�
何建峰
张勇东
吴枫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202210353580.7A priority Critical patent/CN114429192B/en
Publication of CN114429192A publication Critical patent/CN114429192A/en
Application granted granted Critical
Publication of CN114429192B publication Critical patent/CN114429192B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The application discloses an image matching method, an image matching device and electronic equipment, wherein the method and the device are applied to the electronic equipment, and particularly a dynamic key point detector is constructed; and constructing a dynamic key point activation graph based on the dynamic key point detector. The scheme can dynamically generate the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image, thereby effectively realizing the detection of the dynamic key points of various challenging factors and further realizing the robust image matching.

Description

Image matching method and device and electronic equipment
Technical Field
The present application relates to the field of computer vision technologies, and in particular, to an image matching method, an image matching apparatus, and an electronic device.
Background
Finding the pixel-level correspondence between a pair of images is one of basic tasks of computer vision, and has wide application in the fields of visual positioning, attitude estimation, synchronous positioning, map construction and the like. The conventional image matching methods can be divided into two types, which are an irrelevant key point detection method and a relevant key point detection method. The objective of the extraneous key point detection method is to establish the correspondence between dense points in an image, and consider all possible matches as candidate matches. Because of the lack of a keypoint detection process, the computation is costly.
Compared with a method for detecting irrelevant key points, the image matching algorithm based on key point detection has the advantage of low cost, and is widely researched, and the aim is to perform sparse matching on the extracted key points by using a designed key point detector. The detector-based approach first designs keypoint detectors to detect locally repeatable salient points, then extracts descriptors from local regions around each keypoint, and finally selects a set of high confidence matches from all possible candidate matches between pairs of keypoints.
The traditional image matching method based on the detector usually uses a fixed detector to extract key points, and simultaneously, the similarity between the key point detector and the feature vector is directly calculated through dot product operation to obtain a key point activation graph. These two points limit the adaptability of the conventional detector-based method, so that it cannot cope with different types of challenges existing in the real application scenario, such as changes in illumination and viewpoint, resulting in poor robustness of the matching process.
Disclosure of Invention
In view of this, the present application provides an image matching method, an image matching device, and an electronic device, which are used for implementing matching between images based on dynamic keypoint detection, so as to improve robustness of a matching process.
In order to achieve the above object, the proposed solution is as follows:
an image matching method is applied to electronic equipment and comprises the following steps:
constructing a dynamic key point detector of an input image;
and constructing a dynamic key point activation graph based on the dynamic key point detector.
Optionally, the constructing a dynamic key point detector includes the steps of:
extracting a plurality of features of the input image by using an improved L2-Net network, wherein the plurality of features form a feature set;
constructing the dynamic keypoint detector based on the feature set, the dynamic keypoint detector comprising a plurality of keypoint detectors interacting with each other.
Optionally, the constructing a dynamic key point activation map based on the dynamic key point detector includes:
generating an activation map for each of said keypoint detectors using a group correlation layer;
aggregating the multiple activation graphs by using the learned weight to obtain multiple activation graphs corresponding to different characteristic channels;
and processing a plurality of activation graphs corresponding to different characteristic channels to obtain the dynamic key point activation graph.
Optionally, the loss function of the dynamic key point detector includes a cosine similarity loss function and an activation map peak loss function.
An image matching apparatus applied to an electronic device, the image matching apparatus comprising:
a first construction module configured to construct a dynamic keypoint detector;
a second construction module configured to construct a dynamic keypoint activation map based on the dynamic keypoint detector.
Optionally, the first building module includes:
a feature extraction unit configured to extract a plurality of features of the input image using a modified L2-Net network, the plurality of features constituting a feature set;
a build execution unit configured to build the dynamic keypoint detector based on the feature set, the dynamic keypoint detector comprising a plurality of keypoint detectors interacting with each other.
Optionally, the second building module includes:
an activation map generation unit configured to generate an activation map for each of the keypoint detectors using a group correlation layer;
the activation map aggregation unit is configured to aggregate the activation maps by using the learned weights to obtain a plurality of activation maps corresponding to different feature channels;
and the activation map processing unit is configured to process a plurality of activation maps corresponding to different feature channels to obtain the dynamic key point activation map.
Optionally, the loss function of the dynamic key point detector includes a cosine similarity loss function and an activation map peak loss function.
An electronic device comprising an image matching apparatus as described above.
An electronic device comprising at least one processor and a memory coupled to the processor, wherein:
the memory is for storing a computer program or instructions;
the processor is configured to execute the computer program or the instructions to enable the electronic device to implement the electronic device as described above.
From the technical scheme, the application discloses an image matching method, an image matching device and electronic equipment, wherein the method and the device are applied to the electronic equipment, and particularly a dynamic key point detector is constructed; and constructing a dynamic key point activation graph based on the dynamic key point detector. The scheme can dynamically generate the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image, thereby effectively realizing the detection of the dynamic key points of various challenging factors and further realizing the robust image matching.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of an image matching method according to an embodiment of the present application;
fig. 2 is a block diagram of an image matching apparatus according to an embodiment of the present application;
FIG. 3 is a block diagram of another image matching apparatus according to an embodiment of the present application;
FIG. 4 is a block diagram of another image matching apparatus according to an embodiment of the present application;
fig. 5 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Example one
Fig. 1 is a flowchart of an image matching method according to an embodiment of the present application.
As shown in fig. 1, the image matching method provided by the present embodiment is applied to an electronic device, which can be understood as a device required by image matching, such as a control device on an unmanned aerial vehicle, a conductive vehicle, a satellite vehicle, or an unmanned vehicle, and includes the following steps:
and S1, constructing a dynamic key point detector of the input image.
That is, after the electronic apparatus obtains an image and outputs the image to the electronic apparatus as an input image, a dynamic keypoint detector is constructed based on the input image. The specific construction process of the dynamic key point detector is as follows.
First, a plurality of features of an input image are extracted using a modified L2-Net network.
For an input image
Figure DEST_PATH_IMAGE001
Firstly, the improved L2-Net network is used for extracting features
Figure 437405DEST_PATH_IMAGE002
. The application designs a group of prototype key point detectors
Figure DEST_PATH_IMAGE003
The adaptive detector is learned from the current input image. To is coming toModeling the interactions between different prototype keypoint detectors, we used the self-attention mechanism:
Figure 26649DEST_PATH_IMAGE004
to model the interaction between the prototype keypoint detector and the input image feature map, we used a cross-attention mechanism:
Figure DEST_PATH_IMAGE005
then, a dynamic keypoint detector is constructed based on the feature set.
Set d = dk=dlThereby obtaining a set of keypoint detectors adapted to the current input image
Figure 744069DEST_PATH_IMAGE006
。
Where interaction refers to data transmission between prototype detectors.
i, j ∈ 1, 2, . . . , N；
Figure DEST_PATH_IMAGE007
Is the characteristic channel dimension;
h, w are feature map height and width;
Figure 749066DEST_PATH_IMAGE008
is a linear projection;
Figure DEST_PATH_IMAGE009
is the ith query vector query,
Figure 501121DEST_PATH_IMAGE010
is the j-th key value vector key,
Figure DEST_PATH_IMAGE011
is the jth value vector value;
S i,j is Q i And KjSimilarity between them;
Figure 258337DEST_PATH_IMAGE012
is the updated ith prototype keypoint detector;
Figure DEST_PATH_IMAGE013
is the jth feature map;
Figure 197474DEST_PATH_IMAGE014
is that
Figure 927533DEST_PATH_IMAGE009
And
Figure DEST_PATH_IMAGE015
similarity between them;
Figure 206198DEST_PATH_IMAGE016
is the generated ith keypoint detector.
And S2, constructing a dynamic key point activation graph based on the dynamic key point detector.
After the dynamic keypoint detectors are obtained, a dynamic keypoint activation graph can be constructed. The specific scheme is as follows.
First, an activation map is generated for each keypoint detector using a set of correlation layers.
A set of correlation layers is designed and an activation map is generated for each keypoint detector based on the set of correlation layers. It is considered here that under different scenarios, the importance of different feature channels of the feature map obtained by using the feature extraction network is different.
The feature map is aggregated by adopting a direct aggregation mode, namely points on the feature map
Figure DEST_PATH_IMAGE017
And multiplying the signal with the channel directly corresponding to the detector D to calculate similarity as a response value of the point, wherein the equivalent position of each dimension of the characteristic channel is equivalent.
The method of the present application divides the signature channels into g groups of d/g, the matrix operations described herein
Figure 137245DEST_PATH_IMAGE018
. To each point F ij In fact, for g packets, each group is multiplied correspondingly, resulting in g similarities.
Thus, is at
Figure DEST_PATH_IMAGE019
And weighting and summing the g similarities by using the dynamically generated weights to obtain the difference of the importance of different characteristic channels, namely, the difference of the importance of the different characteristic channels is considered.
Considering the different importance of different feature channels, the present application divides the feature channels into
Figure 829258DEST_PATH_IMAGE020
Groups of detectors each obtaining a grouping of characteristic channels
Figure DEST_PATH_IMAGE021
And feature map of feature channel grouping
Figure 238374DEST_PATH_IMAGE022
. And realizing the grouped activation graph by using matrix operation.
Figure DEST_PATH_IMAGE023
Then, the plurality of activation maps are aggregated using the learned weights.
Aggregating the activation maps of the different groups generated by each detector with learned weights:
Figure 496180DEST_PATH_IMAGE024
，
get corresponding to the characteristic channel
Figure 329619DEST_PATH_IMAGE020
Of a group
Figure 102403DEST_PATH_IMAGE020
And (5) activating the graph. To adaptively consider the importance of different feature channels, we generate an aggregate weight mask using the feature map of the current input image
Figure DEST_PATH_IMAGE025
It is mixed with
Figure 49630DEST_PATH_IMAGE026
Element by element multiplication, and designing a convolution kernel with learning ability
Figure DEST_PATH_IMAGE027
Is
Figure 302888DEST_PATH_IMAGE028
Convolutional layer
Figure DEST_PATH_IMAGE029
It was polymerized:
Figure 575738DEST_PATH_IMAGE030
finally, through processing a plurality of activation graphs, a dynamic key point activation graph capable of considering the importance of different feature channel weights in a self-adaptive mode according to the current input image is obtained.
It can be seen from the foregoing technical solutions that, the present embodiment provides an image matching method, which is applied to an electronic device, and specifically, a dynamic key point detector is constructed; and constructing a dynamic key point activation graph based on the dynamic key point detector. The scheme can dynamically generate the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image, thereby effectively realizing the detection of the dynamic key points of various challenging factors and further realizing the robust image matching.
In the design of the loss function, two factors are considered in the application. In order to make the detected key points repeatable, the cosine similarity loss function is adopted in the application. In order to focus the different detectors on salient regions, the present application employs an activation map peak loss function.
The pre-similarity loss function is as follows:
Figure DEST_PATH_IMAGE031
the activation map peak loss function is as follows:
Figure 976763DEST_PATH_IMAGE032
example two
Fig. 2 is a block diagram of an image matching apparatus according to an embodiment of the present application.
As shown in fig. 2, the image matching apparatus provided in the present embodiment is applied to an electronic device, which can be understood as a device required by image matching, such as a control device on an unmanned aerial vehicle, a conductive vehicle, a satellite vehicle, or an unmanned vehicle, and includes a first building module 10 and a second building module 20.
The first construction module is used for constructing a dynamic key point detector of an input image.
That is, after the electronic apparatus obtains an image and outputs the image to the electronic apparatus as an input image, a dynamic key point detector is constructed based on the input image. The first building block specifically includes a feature extraction unit 11 and a building execution unit 12, as shown in fig. 3.
The feature extraction unit is used for extracting a plurality of features of the input image by utilizing the improved L2-Net network.
For an input image
Figure 586736DEST_PATH_IMAGE034
First, the features are extracted by using the improved L2-Net network
Figure DEST_PATH_IMAGE035
. The application designs a group of prototype key point detectors
Figure 694500DEST_PATH_IMAGE036
The adaptive detector is learned from the current input image. To model the interaction between different prototype keypoint detectors, we used the self-attention mechanism:
Figure DEST_PATH_IMAGE037
to model the interaction between the prototype keypoint detector and the input image feature map, we used a cross-attention mechanism:
Figure 400901DEST_PATH_IMAGE038
the construction execution unit is used for constructing a dynamic key point detector based on the feature set.
Setting d = dk=dlThereby obtaining a set of keypoint detectors adapted to the current input image
Figure 617118DEST_PATH_IMAGE006
。
Where interaction refers to data transmission between prototype detectors.
i, j ∈ 1, 2, . . . , N；
Figure 906148DEST_PATH_IMAGE007
Is the characteristic channel dimension;
h, w are feature map height and width;
Figure 461895DEST_PATH_IMAGE008
is a linear projection;
Figure 935601DEST_PATH_IMAGE009
is the ith query vector query,
Figure 780061DEST_PATH_IMAGE015
is the j-th key value vector key,
Figure DEST_PATH_IMAGE039
is the jth value vector value;
Figure 872782DEST_PATH_IMAGE040
and
Figure DEST_PATH_IMAGE041
；
Figure 283034DEST_PATH_IMAGE012
is the updated ith prototype keypoint detector;
Figure 599746DEST_PATH_IMAGE042
is the jth feature map;
Figure DEST_PATH_IMAGE043
is that
Figure 462660DEST_PATH_IMAGE009
And
Figure 841295DEST_PATH_IMAGE015
similarity between them;
Figure 43738DEST_PATH_IMAGE044
is the ith keypoint detector generated.
The second construction module is used for constructing a dynamic key point activation map based on the dynamic key point detector.
After the dynamic keypoint detectors are obtained, a dynamic keypoint activation graph can be constructed. The module includes an activation map generating unit 21, an activation map aggregating unit 22, and an activation map processing unit 23, as shown in fig. 4.
The activation map generation unit is used for generating an activation map for each keypoint detector by using a group correlation layer.
A set of correlation layers is designed and an activation map is generated for each keypoint detector based on the set of correlation layers. It is considered here that under different scenarios, the importance of different feature channels of the feature map obtained by using the feature extraction network is different.
The feature map is subjected to aggregation processing by adopting a direct aggregation mode, namely points on the feature map
Figure 859247DEST_PATH_IMAGE017
And multiplying the corresponding channels directly corresponding to the detector D to calculate similarity as a response value of the point, wherein the equivalent position of each dimension of the characteristic channel is equivalent.
The method of the present application divides the signature channels into g groups of d/g, the matrix operations described herein
Figure 475036DEST_PATH_IMAGE018
. To each point F ij In fact, for g packets, each group is multiplied correspondingly, resulting in g similarities.
Thus, is at
Figure 906630DEST_PATH_IMAGE019
In the method, g similarity is weighted and summed by using the dynamically generated weight, so that different importance of different characteristic channels is obtained and considered.
Considering the different importance of different feature channels, the present application divides the feature channels into
Figure 619371DEST_PATH_IMAGE020
Groups of detectors each obtaining a grouping of characteristic channels
Figure DEST_PATH_IMAGE045
And feature map of feature channel grouping
Figure 481148DEST_PATH_IMAGE022
. And realizing the grouped activation graph by using matrix operation.
Figure 787495DEST_PATH_IMAGE046
And the activation graph aggregation unit is used for aggregating the plurality of activation graphs by using the learned weight.
Aggregating the activation maps of the different groups generated by each detector with learned weights:
Figure DEST_PATH_IMAGE047
，
get corresponding to the characteristic channel
Figure 88027DEST_PATH_IMAGE020
Is grouped into
Figure 796220DEST_PATH_IMAGE020
And (5) opening an activation graph. To adaptively consider the importance of different feature channels, we generate an aggregate weight mask using the feature map of the current input image
Figure 422373DEST_PATH_IMAGE048
It is then mixed with
Figure 12754DEST_PATH_IMAGE026
Element by element multiplication, and designing a convolution kernel with learning ability
Figure 789080DEST_PATH_IMAGE027
Is/are as follows
Figure 945255DEST_PATH_IMAGE028
Convolutional layer
Figure 148835DEST_PATH_IMAGE029
It was polymerized:
Figure 288829DEST_PATH_IMAGE050
the activation graph processing unit is used for processing a plurality of activation graphs corresponding to different feature channels.
Through the processing of a plurality of activation graphs, a dynamic key point activation graph capable of considering the importance of different feature channel weights in a self-adaptive mode according to a current input image is obtained.
As can be seen from the foregoing technical solutions, the present embodiment provides an image matching apparatus, which is applied to an electronic device, and is specifically used for constructing a dynamic key point detector; and constructing a dynamic key point activation graph based on the dynamic key point detector. According to the scheme, the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image can be dynamically generated, so that the detection of the dynamic key points of various challenging factors can be effectively realized, and the robust image matching is further realized.
The pre-similarity loss function is as follows:
Figure DEST_PATH_IMAGE051
the activation map peak loss function is as follows:
Figure 69179DEST_PATH_IMAGE052
EXAMPLE III
The present embodiment provides an electronic device, which can be understood as a device required by image matching, such as a control device on an unmanned aerial vehicle, a conductive vehicle, a satellite vehicle, or an unmanned vehicle, and the electronic device is provided with the image matching apparatus provided in the previous embodiment. The device is used for constructing a dynamic key point detector; and constructing a dynamic key point activation graph based on the dynamic key point detector. The scheme can dynamically generate the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image, thereby effectively realizing the detection of the dynamic key points of various challenging factors and further realizing the robust image matching.
Example four
Fig. 5 is a block diagram of an electronic device according to an embodiment of the present application. A (c)
As shown in fig. 5, the electronic device provided in this embodiment can be understood as a device required by image matching, such as a control device on a drone, a conductive vehicle, a satellite vehicle, or a drone vehicle, and the electronic device at least includes a processor 101 and a memory 102, which are connected through a data bus 103, the memory is used for storing a computer program or instructions, and the processor is used for executing the computer program or instructions to enable the electronic device to implement the image matching method disclosed in the first embodiment.
The image matching method specifically comprises the steps of constructing a dynamic key point detector; and constructing a dynamic key point activation graph based on the dynamic key point detector. The scheme can dynamically generate the dynamic key point detector and the dynamic key point activation graph which are adaptive to the current input image, thereby effectively realizing the detection of the dynamic key points of various challenging factors and further realizing the robust image matching.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including the preferred embodiment and all changes and modifications that fall within the true scope of the embodiments of the present invention.
Finally, it should also be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or terminal equipment comprising the element.
The technical solutions provided by the present invention are described in detail above, and the principle and the implementation of the present invention are explained in this document by applying specific examples, and the description of the above examples is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (6)

1. An image matching method applied to electronic equipment is characterized by comprising the following steps:
a dynamic keypoint detector for constructing an input image, comprising: extracting a plurality of features of an input image by using an improved L2-Net network, wherein the plurality of features form a feature set, and constructing the dynamic key point detector based on the feature set, wherein the dynamic key point detector comprises a plurality of key point detectors which are interactive with each other;
constructing a dynamic key point activation graph based on the dynamic key point detectors, wherein the dynamic key point activation graph is generated for each key point detector by using a group of related layers, a plurality of activation graphs are aggregated by using learned weights to obtain a plurality of activation graphs corresponding to different feature channels, and the plurality of activation graphs corresponding to the different feature channels are processed to obtain the dynamic key point activation graph; the group correlation layer is used for dividing the dynamic key point detector and the characteristic channels of the characteristic diagram into g groups, and performing inner product operation on the vectors of the corresponding groups to obtain g inner product results.
2. The image matching method of claim 1, wherein the loss function of the dynamic keypoint detector comprises a cosine similarity loss function and an activation map peak loss function.
3. An image matching apparatus applied to an electronic device, the image matching apparatus comprising:
a first building module configured to build a dynamic keypoint detector, said first building module comprising a feature extraction unit configured to extract a plurality of features of an input image using a modified L2-Net network, said plurality of features constituting a feature set, and a building execution unit configured to build said dynamic keypoint detector based on said feature set, said dynamic keypoint detector comprising a plurality of keypoint detectors interacting with each other;
a second construction module configured to construct a dynamic keypoint activation graph based on the dynamic keypoint detectors, the second construction module including an activation graph generation unit configured to generate an activation graph for each of the keypoint detectors using one group-related layer, an activation graph aggregation unit configured to aggregate a plurality of the activation graphs using learned weights to obtain a plurality of activation graphs corresponding to different feature channels, and an activation graph processing unit configured to process the plurality of activation graphs corresponding to the different feature channels to obtain the dynamic keypoint activation graph; the group correlation layer is used for dividing the dynamic key point detector and the characteristic channels of the characteristic diagram into g groups, and performing inner product operation on the vectors of the corresponding groups to obtain g inner product results.
4. The image matching apparatus of claim 3, wherein the loss function of the dynamic keypoint detector comprises a cosine similarity loss function and an activation map peak loss function.
5. An electronic device characterized by comprising the image matching apparatus of claim 3 or 4.
6. An electronic device comprising at least one processor and a memory coupled to the processor, wherein:
the memory is used for storing computer programs or instructions;
the processor is configured to execute the computer program or instructions to cause the electronic device to implement the image matching method of claim 1 or 2.
CN202210353580.7A 2022-04-02 2022-04-02 Image matching method and device and electronic equipment Active CN114429192B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210353580.7A CN114429192B (en) 2022-04-02 2022-04-02 Image matching method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210353580.7A CN114429192B (en) 2022-04-02 2022-04-02 Image matching method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN114429192A CN114429192A (en) 2022-05-03
CN114429192B true CN114429192B (en) 2022-07-15

Family

ID=81314377

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210353580.7A Active CN114429192B (en) 2022-04-02 2022-04-02 Image matching method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN114429192B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116129228B (en) * 2023-04-19 2023-07-18 中国科学技术大学 Training method of image matching model, image matching method and device thereof

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111368673A (en) * 2020-02-26 2020-07-03 华南理工大学 Method for quickly extracting human body key points based on neural network
CN111767905A (en) * 2020-09-01 2020-10-13 南京晓庄学院 Improved image method based on landmark-convolution characteristics
EP3731154A1 (en) * 2019-04-26 2020-10-28 Naver Corporation Training a convolutional neural network for image retrieval with a listwise ranking loss function
CN111882532A (en) * 2020-07-15 2020-11-03 中国科学技术大学 Method for extracting key points in lower limb X-ray image
CN113689459A (en) * 2021-07-30 2021-11-23 南京信息工程大学 GMM (Gaussian mixture model) combined with YOLO (YOLO) based real-time tracking and graph building method in dynamic environment
CN113688928A (en) * 2021-08-31 2021-11-23 禾多科技（北京）有限公司 Image matching method and device, electronic equipment and computer readable medium
CN114022522A (en) * 2021-08-30 2022-02-08 北京邮电大学 Multi-time-phase remote sensing image registration method and system based on multi-scale receptive field

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110287348A (en) * 2019-04-15 2019-09-27 南京邮电大学 A kind of GIF format picture searching method based on machine learning
CN112347964B (en) * 2020-11-16 2023-03-24 复旦大学 Behavior detection method and device based on graph network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3731154A1 (en) * 2019-04-26 2020-10-28 Naver Corporation Training a convolutional neural network for image retrieval with a listwise ranking loss function
CN111368673A (en) * 2020-02-26 2020-07-03 华南理工大学 Method for quickly extracting human body key points based on neural network
CN111882532A (en) * 2020-07-15 2020-11-03 中国科学技术大学 Method for extracting key points in lower limb X-ray image
CN111767905A (en) * 2020-09-01 2020-10-13 南京晓庄学院 Improved image method based on landmark-convolution characteristics
CN113689459A (en) * 2021-07-30 2021-11-23 南京信息工程大学 GMM (Gaussian mixture model) combined with YOLO (YOLO) based real-time tracking and graph building method in dynamic environment
CN114022522A (en) * 2021-08-30 2022-02-08 北京邮电大学 Multi-time-phase remote sensing image registration method and system based on multi-scale receptive field
CN113688928A (en) * 2021-08-31 2021-11-23 禾多科技（北京）有限公司 Image matching method and device, electronic equipment and computer readable medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Focus Your Attention: A Bidirectional Focal Attention Network for Image-Text Matching;Chunxiao Liu 等;《https://arxiv.org/abs/1909.11416》;20190925;全文 *
Paying Attention to Multiscale Feature Maps in Multimodal Image Matching;Aviad Moreshet 等;《https://arxiv.org/abs/2103.11247》;20210320;全文 *
图像匹配方法研究综述;贾迪 等;《中国图象图形学报》;20190604;第24卷(第5期);全文 *

Also Published As

Publication number Publication date
CN114429192A (en) 2022-05-03

Similar Documents

Publication Publication Date Title
CN107704838B (en) Target object attribute identification method and device
KR102486699B1 (en) Method and apparatus for recognizing and verifying image, and method and apparatus for learning image recognizing and verifying
CN111325318B (en) Neural network training method, neural network training device and electronic equipment
CN109063776B (en) Image re-recognition network training method and device and image re-recognition method and device
CN113449700B (en) Training of video classification model, video classification method, device, equipment and medium
CN111914908B (en) Image recognition model training method, image recognition method and related equipment
WO2021163103A1 (en) Light-weight pose estimation network with multi-scale heatmap fusion
EP4040378A1 (en) Burst image-based image restoration method and apparatus
CN111241924A (en) Face detection and alignment method and device based on scale estimation and storage medium
CN114429192B (en) Image matching method and device and electronic equipment
CN111310821A (en) Multi-view feature fusion method, system, computer device and storage medium
CN112348116A (en) Target detection method and device using spatial context and computer equipment
CN108875506B (en) Face shape point tracking method, device and system and storage medium
Kuang et al. DenseGAP: graph-structured dense correspondence learning with anchor points
CN116092183A (en) Gesture recognition method and device, electronic equipment and storage medium
CN114240770A (en) Image processing method, device, server and storage medium
Liu et al. A SVD-based visual attention detection algorithm of SAR image
Wen et al. Weakly paired multimodal fusion using multilayer extreme learning machine
Zhou et al. Geometric rectification‐based neural network architecture for image manipulation detection
CN113569860B (en) Instance segmentation method, training method of instance segmentation network and device thereof
CN114581796B (en) Target tracking system, method and computer device thereof
Rajendran et al. RelMobNet: End-to-end relative camera pose estimation using a robust two-stage training
CN113177546A (en) Target detection method based on sparse attention module
CN108960014B (en) Image processing method, device and system and storage medium
JPWO2021038840A5 (en)

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant