CN116030106A - Infrared and visible light image registration method based on phase characteristics and edge characteristics - Google Patents
Infrared and visible light image registration method based on phase characteristics and edge characteristics Download PDFInfo
- Publication number
- CN116030106A CN116030106A CN202310111506.9A CN202310111506A CN116030106A CN 116030106 A CN116030106 A CN 116030106A CN 202310111506 A CN202310111506 A CN 202310111506A CN 116030106 A CN116030106 A CN 116030106A
- Authority
- CN
- China
- Prior art keywords
- visible light
- infrared
- scale
- image
- phase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an infrared and visible light image registration method based on phase characteristics and edge characteristics. Acquiring infrared and visible light images in the same scene, preprocessing the image scale, and acquiring phase consistency information; respectively obtaining a maximum moment diagram and a minimum moment diagram of an infrared image and a visible image, obtaining a corresponding overlapped moment diagram, constructing a scale space for the overlapped moment diagram, and finally screening feature points by a non-maximum suppression method; calculating a maximum direction index at each scale according to a multi-angle multi-scale convolution sequence in the phase consistency information of the infrared and visible light images, constructing a multi-scale maximum index map, and constructing feature descriptors of feature points by combining phase consistency edges; and matching the feature points through the Euclidean distance between the feature descriptors to obtain an optimal transformation matrix between the infrared and visible light images. The problem of registration failure of the traditional method caused by overlarge difference between infrared and visible light images is solved, and infrared and visible light image registration with high precision and good robustness is realized.
Description
Technical Field
The invention belongs to the field of image registration and the field of computer vision, and particularly relates to an infrared and visible light image registration method based on phase characteristics and edge characteristics.
Background
In recent years, the technology of multi-source sensor imaging is mature, and massive multi-mode image data are emerging. The infrared and visible light image processing technology is used as a hot spot branch in the multi-mode image processing technology, and has wide application in the fields of military, medical treatment, survey and the like. The infrared and visible light images have larger difference, and are mainly expressed in that: in principle, nonlinear radiation distortion exists in infrared and visible light images; visible light images are rich in texture from the visual effect, and infrared images lack details. The registration difficulty is greater due to the apparent difference between the infrared and visible images and the lack of consistent information.
Current registration methods mainly include region-based methods and feature-based methods. The region-based methods mainly include mutual information, normalized cross correlation, and the like. The method has two main disadvantages, namely, the method is only suitable for image pairs containing translational changes, and registration fails when the images have rotation, scale and other changes; another disadvantage is that its registration effect depends on the richness of the texture in the window of the region, and if there is no texture or weak texture in the window, an incorrect match is likely to be obtained. The feature-based method obtains the transformation relation between images by establishing a reliable matching relation between feature points, and the method has obvious advantages in accuracy and practicability. However, as the spectrum distance between the long-wave infrared light and the visible light is far, the problems of gradient inversion and intensity distortion exist in the infrared light image and the visible light image, so that the traditional method has low registration success rate and is difficult to obtain a stable registration result.
Disclosure of Invention
In order to solve the technical problems mentioned in the background art, the invention provides an infrared and visible light image registration method based on phase characteristics and edge characteristics.
In order to achieve the above purpose, the present invention provides the following technical solutions:
an infrared and visible light image registration method based on phase characteristics and edge characteristics comprises the following steps:
(1) Acquiring infrared and visible light images in the same scene, preprocessing the image scale, and then respectively carrying out convolution calculation on the infrared and visible light images by adopting a Log-Gabor filter to acquire phase consistency information;
(2) According to the phase consistency information of the infrared and visible light images, respectively obtaining a maximum moment diagram and a minimum moment diagram of the infrared and visible light images, then weighting and superposing the maximum moment diagram and the minimum moment diagram to obtain corresponding superposition moment diagrams, constructing a scale space for the superposition moment diagrams, uniformly extracting characteristic points on different scale images in a blocking way, and finally screening the characteristic points through a non-maximum suppression method;
(3) Calculating a maximum direction index at each scale according to a multi-angle multi-scale convolution sequence in the phase consistency information of the infrared and visible light images, constructing a multi-scale maximum index map, and constructing a feature descriptor of a feature point according to the multi-scale maximum index map and the phase consistency edge map of the infrared and visible light images;
(4) And matching the feature points through the Euclidean distance between the feature descriptors, screening the mismatching points, and finally obtaining an optimal transformation matrix between the infrared and visible light images.
Further, in the step (1), the specific process is as follows:
(101) For the input infrared and visible light images, the image with higher resolution is downsampled, so that the resolution of the infrared and visible light is the same.
(102) For an input image I (x, y), a two-dimensional Log-Gabor filter (LGF) is adopted to convolve the image, and response values and phase consistency information of scale filters in different directions of each pixel are obtained.
Further, in step (102), the process of obtaining the convolution information is:
[e s,o (x,y),o s,o (x,y)]＝[I(x,y)*LGF s,o even ,I(x,y)*LGF s,o odd ]
wherein the LGF s,o even And LGF s,o odd Representing even and odd symmetric Log-Gabor wavelets in the scale s and direction o, respectively, represent convolution operations. Then calculate the amplitude A at pixel point I (x, y) s,o (x, y) and phase angle
The response value, the calculation formula is:
the definition of phase consistency is:
wherein PC (x, y) represents the phase consistency response value, W, at pixel point I (x, y) o (x, y) is a weight factor. T is the preset noise threshold value and,
represents a downward rounding, ε is a minimum constant, +.>
Is the average phase angle.
Further, in step (2), the feature point extraction specifically includes the steps of:
(201) And calculating the maximum moment and the minimum moment of the response value of each pixel by utilizing the response values of the scale filter in different directions to form a maximum moment diagram and a minimum moment diagram which are the same as the original diagram in size, and carrying out weighted superposition on the maximum moment diagram and the minimum moment diagram to obtain a superposed moment diagram through normalization.
(202) And constructing a scale space for the superimposed moment diagram, dividing each scale image into image blocks which are not overlapped and have the same size, extracting Harris corner points in each image block, if the number of the extracted corner points is lower than n, reducing the threshold value of the extracted corner points of Harris, re-extracting the corner points, wherein n is a preset constant, and finally screening the characteristic points by using a non-maximum value inhibition method.
Further, in step (201), the process of building the superimposed moment diagram includes the steps of:
calculating a maximum moment diagram and a minimum moment diagram by using the phase consistency information, wherein the formula is as follows:
wherein:
the superposition mode of the maximum moment diagram and the minimum moment diagram is as follows:
M'＝αM+(1-α)m
wherein alpha is a weighting factor, and then M' is normalized to obtain a superimposed moment diagram.
Further, in step (3), the method of constructing the feature descriptor is as follows:
(301) The convolution sequence setting the Log-Gabor filter has N s Individual dimensions and N o In each direction, constructing a maximum index map in each scale s
Wherein A is s,o (x, y) is the response value, k, of the filter with scale s and direction o at the image pixel point (x, y) Ns (x, y) is the maximum direction index under the scale s, and the value is 1-N o ，
A multi-scale maximum index map is formed.
(302) Constructing phase characteristics specifically comprises the following steps: in the multi-scale maximum index map, a selected rectangular region centered on a feature point is divided into 6×6 sub-regions, each consisting of 16×16 pixels. In each subarea, carrying out histogram statistics on the multi-scale maximum index map, and finally carrying out N s The result weighted accumulation of the individual scales comprises the following specific calculation processes:
wherein V is s For the phase characteristic at scale s, V is the phase characteristic of the sub-region, α s Is a scale factor, and
the features of the individual sub-regions are then connected. Because the index map has values of 1 to N o Each histogram has a size of N o The phase feature part length of each feature descriptor is 6×6×n o 。
(303) Constructing edge features, specifically including: the edge map is obtained from the maximum moment map of phase consistency, a rectangular area is selected on the edge map by taking a characteristic point as a center, and the rectangular area is divided into 4×4 sub-areas, and each sub-area consists of 16×16 pixels. Statistics of N in each sub-region edge Gradient direction histogram of each direction, then connecting the features of the sub-regions, each feature descriptor dimension being 4×4×n edge ，N edge Is a preset constant.
(304) The two groups of descriptors are respectively normalized and then connected to form a new descriptor, and the dimension of the new descriptor is (6 multiplied by N) o +4×4×N edge )。
Further, in the step (4), the specific process of feature matching is as follows:
(401) And assuming that m characteristic points of the infrared image exist, n characteristic points of the visible light image exist, sequentially calculating Euclidean distances between the m characteristic points of the infrared image and the n characteristic points of the visible light image, and taking a point with the minimum Euclidean distance as a matching point.
(402) And (3) performing outlier rejection by using a fast matching algorithm (FSC) and obtaining an optimal transformation matrix.
The beneficial effects are that: the invention solves the problem of registration failure of the traditional method caused by overlarge difference between infrared and visible light images, and realizes the registration of the infrared and visible light images with high precision and good robustness.
Drawings
Fig. 1 is a basic flow chart of the present invention.
Detailed Description
The technical scheme of the present invention will be described in detail below with reference to the accompanying drawings.
The invention provides a registration method of infrared and visible light images based on phase characteristics and edge characteristics, wherein a registration flow chart is shown in fig. 1, and the method comprises the following steps:
step one: acquiring infrared and visible light images in the same scene, performing scale pretreatment on the images, and then respectively performing convolution calculation on the infrared and visible light images by adopting a Log-Gabor filter to obtain phase consistency information;
step two: according to the phase consistency information of the infrared and visible light images, respectively obtaining a maximum moment diagram and a minimum moment diagram of the infrared and visible light images, then weighting and superposing the maximum moment diagram and the minimum moment diagram to obtain corresponding superposition moment diagrams, constructing a scale space for the superposition moment diagrams, uniformly extracting characteristic points on different scale images in a blocking way, and finally screening the characteristic points through a non-maximum suppression method;
step three: calculating a maximum direction index at each scale according to a multi-angle multi-scale convolution sequence in the phase consistency information of the infrared and visible light images, constructing a multi-scale maximum index map, and constructing a feature descriptor of a feature point according to the multi-scale maximum index map and the phase consistency edge map of the infrared and visible light images;
step four: and matching the feature points through the Euclidean distance between the feature descriptors, screening the mismatching points, and finally obtaining an optimal transformation matrix between the infrared and visible light images.
In this embodiment, the first step may be implemented by the following preferred scheme:
(1) For the input infrared and visible light images, the image with higher resolution is downsampled, so that the resolution of the infrared and visible light images is the same, and the scale difference of the infrared and visible light images can be reduced.
(2) For the input image I (x, y), the image is convolved with a two-dimensional Log-Gabor filter (LGF):
[e s,o (x,y),o s,o (x,y)]＝[I(x,y)*LGF s,o even ,I(x,y)*LGF s,o odd ]
wherein the LGF s,o even And LGF s,o odd Representing even and odd symmetric Log-Gabor wavelets in the scale s and direction o, respectively, represent convolution operations. Then calculate the amplitude A at each pixel point I (x, y) s,o (x, y) and phase angle
The response value, the calculation formula is:
the definition of phase consistency is
Wherein PC (x, y) represents the phase consistency response value, W, at pixel point I (x, y) o (x, y) is a weight factor. T is the preset noise threshold value and,
represents a downward rounding, ε is a minimum constant, +.>
Is the average phase angle.
In this embodiment, the specific process of uniformly extracting the corner points by using the superimposed moment diagram in the second step is as follows:
(1) The maximum moment and the minimum moment of each pixel are calculated to form a maximum moment diagram and a minimum moment diagram by using response values of filters in different directions and scales, and the calculation method comprises the following steps:
wherein:
the superposition method of the maximum moment diagram and the minimum moment diagram comprises the following steps:
M'＝αM+(1-α)m
wherein alpha is a weighting factor, and then M' is normalized to obtain a superimposed moment diagram.
(2) And constructing a scale space for the superimposed moment diagram, dividing each scale image into image blocks which are not overlapped and have the same size, extracting Harris corner points in each image block, if the number of the extracted corner points is lower than n, reducing the threshold value of the extracted corner points of Harris, re-extracting the corner points, wherein n is a preset constant, and finally screening the characteristic points by using a non-maximum value inhibition method.
In this embodiment, the method for constructing the feature descriptor in the third step is as follows:
(1) The convolution sequence setting the Log-Gabor filter has N s Individual dimensions and N o In each direction, a maximum index map is built in each scale s:
wherein A is s,o (x, y) is the response value, k, of the filter with scale s and direction o at the image pixel point (x, y) Ns (x, y) is the maximum direction index under the scale s, and the value is 1-N o ，
A multi-scale maximum index map is formed.
(2) Constructing phase characteristics specifically comprises the following steps: in the multi-scale maximum index map, a selected rectangular region centered on a feature point is divided into 6×6 sub-regions, each consisting of 16×16 pixels. In each subarea, counting the multi-scale maximum index map, and finally adding N s The result weighted accumulation of the individual scales comprises the following specific calculation processes:
wherein V is s For the phase characteristic at scale s, V is the phase characteristic of the sub-region, α s Is a scale factor, and
the features of the individual sub-regions are then connected. Because the index map has values of 1 to N o Each histogram has a size of N o The phase feature part length of each feature point descriptor is 6×6×n o 。
(3) Constructing edge features, specifically including: the edge map is obtained from the maximum moment map of phase consistency, a rectangular area is selected on the edge map by taking a characteristic point as a center, and the rectangular area is divided into 4×4 sub-areas, and each sub-area consists of 16×16 pixels. Statistics of N in each sub-region edge Gradient direction histogram of individual directions, feature dimension 4×4×n edge ，N edge Is a preset constant.
(4) The two groups of descriptors are respectively normalized and then connected to form a new descriptor, and the dimension of the new descriptor is (6 multiplied by N) o +4×4×N edge )。
In this embodiment, the specific procedure of the fourth step is as follows:
(1) And assuming that m characteristic points of the infrared image exist, n characteristic points of the visible light image exist, sequentially calculating Euclidean distances between the m characteristic points of the infrared image and the n characteristic points of the visible light image, and taking a point with the minimum Euclidean distance as a matching point.
(2) And (3) performing outlier rejection by using a fast matching algorithm (FSC) and obtaining an optimal affine transformation matrix.
The foregoing merely illustrates the technical idea of the present invention in connection with a specific preferred embodiment, and the scope of the invention is not limited thereto, but any modification made on the basis of the technical scheme according to the technical idea of the present invention falls within the scope of the invention.
Claims (5)
1. The infrared and visible light image registration method based on the phase characteristics and the edge characteristics is characterized by comprising the following steps of:
step one: acquiring an infrared image and a visible light image under the same scene, performing scale pretreatment on the infrared image and the visible light image, and then performing convolution calculation on the infrared image and the visible light image respectively by adopting a filter to obtain phase consistency information of the infrared image and the visible light image;
step two: according to the phase consistency information of the infrared image and the visible light image, respectively obtaining a maximum moment diagram and a minimum moment diagram of the infrared image and the visible light image, then weighting and superposing the maximum moment diagram and the minimum moment diagram to obtain a superposition moment diagram of the infrared image and the visible light image, constructing a scale space for the superposition moment diagram, uniformly extracting feature points on different scale images in a blocking way, and finally screening the feature points through a non-maximum value suppression method;
step three: constructing a multi-scale maximum index map according to a multi-angle multi-scale convolution sequence in the phase consistency information of the infrared and visible light images, and constructing feature descriptors of feature points according to the multi-scale maximum index map and the phase consistency edge map of the infrared and visible light images;
step four: and matching the feature points through the Euclidean distance between the feature descriptors, screening the mismatching points, and finally obtaining an optimal transformation matrix between the infrared and visible light images.
2. The method for registering an infrared and visible light image based on phase features and edge features as claimed in claim 1, wherein the implementation process of the step one is as follows:
(1.1) downsampling the infrared image and the visible light image such that the resolution of the infrared image and the visible light image are the same;
and (1.2) carrying out convolution calculation on the infrared image and the visible light image after the downsampling process by adopting a two-dimensional Log-Gabor filter, and respectively acquiring response values and phase consistency information of the two-dimensional Log-Gabor filter with different directions and dimensions of each pixel in the infrared image and the visible light image after the downsampling process.
3. The method for registering an infrared and visible light image based on phase features and edge features as claimed in claim 2, wherein the process of extracting feature points in the second step comprises the steps of:
(2.1) calculating the maximum moment and the minimum moment of the response value, forming a maximum moment diagram and a minimum moment diagram which have the same resolution as the infrared image and the visible light image after the downsampling process, weighting and superposing the maximum moment diagram and the minimum moment diagram, and respectively obtaining a superposition moment diagram of the infrared image and the visible light image through normalization;
and (2.2) respectively constructing a scale space for the superposition moment diagram of the infrared image and the visible light image to obtain scale images, dividing each scale image into image blocks which are not overlapped and have the same size, extracting angular points in each image block through a Harris angular point detection algorithm, reducing the threshold value of the extracted angular points of the Harris detection algorithm if the number of the extracted angular points is lower than n, re-extracting the angular points, wherein n is a preset constant, and finally screening characteristic points by a non-maximum suppression method to obtain the characteristic points of the infrared image and the visible light image.
4. The method for registering an infrared and visible light image based on phase and edge features of claim 3, wherein the constructing of the feature descriptors in step three comprises:
(3.1) the Multi-angle Multi-scale convolution sequence in the set phase consistency information has N s Individual dimensions and N o The maximum index map is built in each scale in each direction, and the calculation method comprises the following steps:
wherein A is s,o (x, y) is the response value, k, of the two-dimensional Log-Gabor filter at the pixel point (x, y) of the infrared and visible light images when the scale is s and the direction is o Ns (x, y) is the maximum index map of pixel point (x, y) at scale s,
a multi-scale maximum index diagram is formed;
(3.2) constructing a phase feature, wherein in the multi-scale maximum index map, a selected rectangular area centered on a feature point is divided into 6×6 sub-areas, each sub-area consisting of 16×16 pixels; in each subarea, carrying out histogram statistics on the multi-scale maximum index map, and finally carrying out N s The result weighted accumulation of the individual scales comprises the following specific calculation processes:
wherein V is s For the phase characteristic at scale s, V is the phase characteristic of the sub-region, α s Is a scale weight and
then the phase characteristics of the sub-regions are connected, because the maximum index map has the value of 1-N o The obtained histogram size of each sub-region is N o The phase feature length of each feature point descriptor is 6×6×n o ；
(3.3) constructing edge features, obtaining an edge map, namely a maximum moment map obtained in the step (2.1), selecting a rectangular area on the edge map by taking feature points as the center, dividing the rectangular area into 4 multiplied by 4 sub-areas, wherein each sub-area consists of 16 multiplied by 16 pixels, and counting N in each sub-area edge Gradient direction histogram of each direction, each feature point descriptorIs 4 x N in phase characteristic length edge ，N edge Is a preset constant;
(4) The two groups of descriptors are respectively normalized and then connected to form a new descriptor, and the phase characteristic length of the new descriptor is (6 multiplied by N) o +4×4×N edge )。
5. The method for registering an infrared and visible light image based on phase and edge features as claimed in claim 4, wherein the feature matching in step four comprises the steps of:
(4.1) assuming that m characteristic points of the infrared image exist, n characteristic points of the visible light image exist, sequentially calculating Euclidean distances between the m characteristic points of the infrared image and the n characteristic points of the visible light image, and taking a point with the minimum Euclidean distance as a matching point;
and (4.2) carrying out mismatching point elimination by using a fast matching algorithm FSC and obtaining an optimal transformation matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310111506.9A CN116030106A (en) | 2023-02-14 | 2023-02-14 | Infrared and visible light image registration method based on phase characteristics and edge characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310111506.9A CN116030106A (en) | 2023-02-14 | 2023-02-14 | Infrared and visible light image registration method based on phase characteristics and edge characteristics |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116030106A true CN116030106A (en) | 2023-04-28 |
Family
ID=86073905
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310111506.9A Pending CN116030106A (en) | 2023-02-14 | 2023-02-14 | Infrared and visible light image registration method based on phase characteristics and edge characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116030106A (en) |
-
2023
- 2023-02-14 CN CN202310111506.9A patent/CN116030106A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111784576B (en) | Image stitching method based on improved ORB feature algorithm | |
| CN105809640B (en) | Low illumination level video image enhancement based on Multi-sensor Fusion | |
| US11398053B2 (en) | Multispectral camera external parameter self-calibration algorithm based on edge features | |
| WO2021098083A1 (en) | Multispectral camera dynamic stereo calibration algorithm based on salient feature | |
| CN110956661B (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| CN111080529A (en) | Unmanned aerial vehicle aerial image splicing method for enhancing robustness | |
| CN112184604B (en) | Color image enhancement method based on image fusion | |
| CN104599258B (en) | A kind of image split-joint method based on anisotropic character descriptor | |
| CN111080709B (en) | Multispectral stereo camera self-calibration algorithm based on track feature registration | |
| CN110969669B (en) | Visible light and infrared camera combined calibration method based on mutual information registration | |
| WO2021017588A1 (en) | Fourier spectrum extraction-based image fusion method | |
| CN108765476B (en) | Polarized image registration method | |
| CN106023187B (en) | A kind of method for registering images based on SIFT feature and angle relative distance | |
| CN109118544B (en) | Synthetic aperture imaging method based on perspective transformation | |
| CN110910456B (en) | Three-dimensional camera dynamic calibration method based on Harris angular point mutual information matching | |
| CN112254656B (en) | Stereoscopic vision three-dimensional displacement measurement method based on structural surface point characteristics | |
| CN110880191B (en) | Infrared stereo camera dynamic external parameter calculation method based on histogram equalization | |
| CN109919832A (en) | One kind being used for unpiloted traffic image joining method | |
| CN105678720A (en) | Image matching judging method and image matching judging device for panoramic stitching | |
| CN115601407A (en) | Infrared and visible light image registration method | |
| CN110910457B (en) | Multispectral three-dimensional camera external parameter calculation method based on angular point characteristics | |
| CN110070626B (en) | Three-dimensional object retrieval method based on multi-view classification | |
| CN115424249B (en) | Self-adaptive detection method for small and weak targets in air under complex background | |
| CN116030106A (en) | Infrared and visible light image registration method based on phase characteristics and edge characteristics | |
| CN113409334B (en) | Centroid-based structured light angle point detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |