Disclosure of Invention
The invention mainly aims to provide an image feature extraction method and an image feature extraction device, aiming at avoiding the main direction estimation and the subsequent direction normalization operation of a local image block by constructing a local coordinate system of the local image block of an image, so as to accurately obtain the local invariant feature representing the image.
In order to achieve the above object, the present invention provides an image feature extraction method, including:
acquiring a first local image block of an image, and determining a central point and a sampling point in the first local image block;
establishing a local coordinate system according to the central point and the sampling point;
extracting local invariant features at the sampling points according to the local coordinate system;
and fusing the local invariant features at the sampling points to obtain image features.
Preferably, the step of extracting the local invariant features at the sampling points according to the local coordinate system comprises:
setting a preset frequency and a preset direction at the sampling point according to the local coordinate system to obtain wavelets with a preset quantity;
acquiring a second local image block which takes the sampling point as the center in the first local image block;
and performing inner product calculation on the wavelets and the second local image blocks of the preset number to obtain responses of the preset number, wherein the responses represent local invariant features at the sampling points.
Preferably, the step of fusing the local invariant features at the sampling points to obtain image features includes:
obtaining a phase of each of the responses;
and encoding each frequency according to the phase by using a preset bit, and calculating to obtain a histogram with a preset length so as to obtain the image characteristics.
Preferably, the step of encoding each frequency according to the phase by a predetermined bit and calculating a histogram with a predetermined length further comprises:
acquiring a histogram of another image, and calculating a distance value between the histograms of the two images;
and matching the local invariant features of the two images according to the distance value.
Preferably, the step of matching the local invariant feature according to the distance value includes:
comparing the distance value to a predetermined value;
and if the distance value is close to the preset value, judging that the two images are similar.
In addition, to achieve the above object, the present invention also proposes an extraction device including:
the system comprises an acquisition module, a sampling module and a processing module, wherein the acquisition module is used for acquiring a first local image block of an image and determining a central point and a sampling point in the first local image block;
the establishing module is used for establishing a local coordinate system according to the central point and the sampling point;
the extraction module is used for extracting local invariant features at the sampling points according to the local coordinate system;
and the fusion processing module is used for fusing the local invariant features at the sampling points to obtain image features.
Preferably, the extraction module comprises:
the setting unit is used for setting wavelets with preset frequency and preset direction at the sampling points according to the local coordinate system to obtain wavelets with preset quantity;
a first acquisition unit configured to acquire a second partial image block centered on the sample point in the first partial image block;
and the first calculation unit is used for carrying out inner product calculation on the wavelets and the second local image blocks of the preset number to obtain the response of the preset number, and the response represents the local invariant feature at the sampling point.
Preferably, the fusion processing module includes:
a second acquisition unit configured to acquire a phase of each of the responses;
and the second calculation unit is used for encoding each frequency by a preset bit according to the phase and calculating to obtain a histogram with a preset length so as to obtain the image characteristics.
Preferably, the extraction device further comprises:
the calculation module is used for acquiring the histogram of the other image and calculating the distance value between the histograms of the two images;
and the matching module is used for matching the local invariant features of the two images according to the distance value.
Preferably, the matching module comprises:
a comparison unit for comparing the distance value with a predetermined value;
and the judging unit is used for judging that the two images are similar if the magnitude of the distance value is close to the preset value.
According to the extraction method and the extraction device of the image features, the first local image block of the image is obtained, the central point and the sampling point in the first local image block are determined, then the local coordinate system is established according to the central point and the sampling point, and the local invariant features at the sampling point are extracted according to the local coordinate system, so that the local invariant features at the sampling point are fused to obtain the image features. Therefore, by constructing a rotatable and invariant local coordinate system for the local image block of the image, the estimation of the main direction and the subsequent direction normalization operation of the local image block can be avoided, and the local invariant feature representing the image can be accurately obtained.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides an image feature extraction method, and referring to fig. 1, in an embodiment, the image feature extraction method includes the following steps:
step S10, acquiring a first local image block of an image, and determining a central point and a sampling point in the first local image block;
step S20, establishing a local coordinate system according to the central point and the sampling point;
in this embodiment, due to the influences of factors such as rotation, shooting angle, distance and the like, the same object may look completely different in different images, and therefore, the local invariant feature of the image is an important basis for understanding the image and classifying and identifying the object in the image, and is an important basis for understanding subsequent images. Therefore, the invention provides an image feature extraction method based on a local coordinate system, so that when the rotation and the shooting angle are changed, the local invariant feature representing the image can be accurately extracted.
In this embodiment, a small local area of the image, that is, the first local image block, is extracted from the image, and assuming that the central point of the first local image block is P and the sampling points of the extracted features around the first local image block are Pi, a local coordinate system as shown in fig. 2 can be established, where the coordinate system takes a connection line between the central point P and the sampling points Pi as a y-axis, a vertical direction as an x-axis, and an included angle between the x-axis and a horizontal direction as θ. Due to the arrangement of the horizontal direction theta, the angle correction function can be realized, so that the local coordinate system can still keep rotating unchanged when being influenced by factors such as rotation, shooting angle change and the like.
In the preferred embodiment, a Gabor filter is used to set parameters and extract local invariant features of the image. In image processing, the Gabor function is a linear filter for edge extraction, and the frequency and direction expression of the Gabor filter is similar to the human visual system, and thus is suitable for texture expression and separation. In the spatial domain, a two-dimensional Gabor filter is a gaussian kernel function modulated by a sinusoidal plane wave, and specifically, filters in different forms such as Log-Gabor can be adopted, and the filters can be specifically selected according to actual needs.
Step S30, extracting local invariant features at the sampling points according to the local coordinate system;
and step S40, fusing the local invariant features at the sampling points to obtain image features.
In this embodiment, a predetermined number of wavelets can be obtained by predetermining wavelets in frequency and predetermined direction according to the local coordinate system, for example, 40 Gabor wavelets in 8 directions and 5 frequencies can be designed, and of course, in other embodiments, the frequency and the direction in the local coordinate system can also be set reasonably according to the specific situation of the image, and the present invention is not limited to this embodiment. And then performing convolution or inner product calculation on the wavelets of the preset number and the second local image block taking the sampling point as the center to obtain the responses of the preset number such as 40. The impulse response of a Gabor filter can be defined as a sine wave (a sinusoidal plane wave for a two-dimensional Gabor filter) multiplied by a gaussian function. Due to the multiplicative convolution property, the fourier transform of the impulse response of a Gabor filter is the convolution of the fourier transform of its harmonic function and the fourier transform of a gaussian function. The filter consists of a real part and an imaginary part, which are mutually orthogonal. And then, according to the response obtained at the sampling point Pi, sequentially carrying out two-bit or four-bit coding and histogram calculation, obtaining a histogram through the two-bit or four-bit coding calculation, and finally, taking the histogram as the representation of the local invariant feature, thereby extracting the complete local invariant feature of the image.
The method for extracting the image features comprises the steps of obtaining a first local image block of an image, determining a central point and a sampling point in the first local image block, establishing a local coordinate system according to the central point and the sampling point, and extracting local invariant features at the sampling point according to the local coordinate system so as to fuse the local invariant features at the sampling point to obtain the image features. Therefore, by constructing a rotatable and invariant local coordinate system for the local image block of the image, the estimation of the main direction and the subsequent direction normalization operation of the local image block can be avoided, and the local invariant feature representing the image can be accurately obtained.
Further, as shown in fig. 3, on the basis of the embodiment of fig. 1, in this embodiment, the step S30 includes:
step S301, according to the local coordinate system, setting a preset frequency and a preset direction at the sampling point to obtain wavelets with preset quantity;
in this embodiment, the local Gabor features at the sampling points Pi are extracted from the local coordinate system, and 40 Gabor wavelets with 5 frequencies and 8 directions are designedu=0~4,v=0~7;
Wherein,(xi,yi) Is the coordinate of sampling point Pi, f is frequencyThe ratio, v is the direction, σ is the standard deviation of the gaussian function, and j is the complex number. It will of course be appreciated that these parameter values may be based onThe actual needs are reasonably set.
Step S302, acquiring a second local image block which takes the sampling point as the center in the first local image block;
step S303, performing inner product calculation on the predetermined number of wavelets and the second local image block to obtain the predetermined number of responses, where the responses represent local invariant features at the sampling points.
In this example, 40 small waves were usedAnd the second local image block F (x, y) taking the sampling point Pi as the center is subjected to inner product operation to obtain 40 responses
In this embodiment, each response represents a local invariant feature of each wavelet, and the responses representing each wavelet need to be fused to calculate a histogram representing image features.
In an embodiment, as shown in fig. 4, on the basis of the embodiment of fig. 1, in this embodiment, the step S40 includes:
step S401, obtaining the phase of each response;
step S402, encoding each frequency according to the phase by a predetermined bit, and calculating to obtain a histogram with a predetermined length, so as to obtain the image feature.
In the present embodiment, the response obtained at the sampling point PiIs a complex number, calculated for each soundPhase of responseAnd generating a binary code according to the following formula
The above two-bit code is 8-bit coded according to each frequency u, and 10 integers of 0 to 255 can be obtained:
and after extracting the feature points with unchanged rotation of the n sampling points Pi with the central point P according to the steps, fusing the feature points to form a histogram representing the unchanged features of the whole image. For each frequency, can be based onAndtwo length 255 histograms corresponding to a binary code obtained by calculation
It is understood that the above-mentioned encoding method can also be modified, such as using 4-bit encoding or encoding according to real and imaginary values.
In the prior art, the subsequent gradient direction histogram only records the direction information of the image gray level change and does not contain important information such as the intensity and frequency of the gray level change, so that the characteristic discrimination is poor, and the local texture representation with stronger discrimination and description capacity can be obtained by adopting Gabor characteristics, so that different image information can be better distinguished.
In an embodiment, as shown in fig. 5, on the basis of the embodiment of fig. 1, in this embodiment, after the step S402, the method further includes:
step S50, acquiring the histogram of another image, and calculating the distance value between the histograms of the two images;
in this embodiment, a histogram of another image is obtained, where the another image may be the same as, similar to, or completely unrelated to the selected image, and the similarity between the two images can be determined by calculating a distance value between the histograms of the two images. Thus, the applicability is wider.
And step S60, matching the local invariant features of the two images according to the distance value.
In this embodiment, the matching of the local invariant features is performed according to the distance value, and a specific calculation formula is as follows:
wherein,andthe 255-dimensional feature histograms extracted from the two images are respectively.
In an embodiment, as shown in fig. 6, on the basis of the embodiment of fig. 5, in this embodiment, the step S60 includes:
step S601, comparing the distance value with a preset value;
in this embodiment, the distance value may be compared with a predetermined value, where the predetermined value may be reasonably set according to actual needs, and in this preferred embodiment, the predetermined value may be preferably 0.
Step S602, if the distance value is close to the predetermined value, it is determined that the two images are similar.
In this embodiment, if it is determined that the magnitude of the distance value is close to the predetermined value, for example, 0, it may be determined that the two images are similar. Further, when the distance value between the histograms of the two images is smaller, if the value range is within 0-1, the similarity of the two images is higher; conversely, when the distance value between the histograms of the two images is larger, if the value range is larger than 1, the similarity between the two images is lower.
The invention also provides an extraction device 1, with reference to fig. 7, in an embodiment, the extraction device 1 comprises:
the system comprises an acquisition module 10, a processing module and a processing module, wherein the acquisition module is used for acquiring a first local image block of an image and determining a central point and a sampling point in the first local image block;
the establishing module 20 is used for establishing a local coordinate system according to the central point and the sampling point;
in this embodiment, due to the influences of factors such as rotation, shooting angle, distance and the like, the same object may look completely different in different images, and therefore, the local invariant feature of the image is an important basis for understanding the image and classifying and identifying the object in the image, and is an important basis for understanding subsequent images. Therefore, the invention provides an image feature extraction method based on a local coordinate system, so that when the rotation and the shooting angle are changed, the local invariant feature representing the image can be accurately extracted.
In this embodiment, a small local area of the image, that is, the first local image block, is extracted from the image, and assuming that the central point of the first local image block is P and the sampling points of the extracted features around the first local image block are Pi, a local coordinate system as shown in fig. 2 can be established, where the coordinate system takes a connection line between the central point P and the sampling points Pi as a y-axis, a vertical direction as an x-axis, and an included angle between the x-axis and a horizontal direction as θ. Due to the arrangement of the horizontal direction theta, the angle correction function can be realized, so that the local coordinate system can still keep rotating unchanged when being influenced by factors such as rotation, shooting angle change and the like.
In the preferred embodiment, a Gabor filter is used to set parameters and extract local invariant features of the image. In image processing, the Gabor function is a linear filter for edge extraction, and the frequency and direction expression of the Gabor filter is similar to the human visual system, and thus is suitable for texture expression and separation. In the spatial domain, a two-dimensional Gabor filter is a gaussian kernel function modulated by a sinusoidal plane wave, and specifically, filters in different forms such as Log-Gabor can be adopted, and the filters can be specifically selected according to actual needs.
An extraction module 30, configured to extract a local invariant feature at the sampling point according to the local coordinate system;
and the fusion processing module 40 is configured to fuse the local invariant features at the sampling points to obtain image features.
In this embodiment, a predetermined number of wavelets can be obtained by predetermining wavelets in frequency and predetermined direction according to the local coordinate system, for example, 40 Gabor wavelets in 8 directions and 5 frequencies can be designed, and of course, in other embodiments, the frequency and the direction in the local coordinate system can also be set reasonably according to the specific situation of the image, and the present invention is not limited to this embodiment. And then performing convolution or inner product calculation on the wavelets of the preset number and the second local image block taking the sampling point as the center to obtain the responses of the preset number such as 40. The impulse response of a Gabor filter can be defined as a sine wave (a sinusoidal plane wave for a two-dimensional Gabor filter) multiplied by a gaussian function. Due to the multiplicative convolution property, the fourier transform of the impulse response of a Gabor filter is the convolution of the fourier transform of its harmonic function and the fourier transform of a gaussian function. The filter consists of a real part and an imaginary part, which are mutually orthogonal. And then, according to the response obtained at the sampling point Pi, sequentially carrying out two-bit or four-bit coding and histogram calculation, obtaining a histogram through the two-bit or four-bit coding calculation, and finally, taking the histogram as the representation of the local invariant feature, thereby extracting the complete local invariant feature of the image.
The extraction device provided by the invention obtains the first local image block of the image, determines the central point and the sampling point in the first local image block, then establishes a local coordinate system according to the central point and the sampling point, and extracts the local invariant features at the sampling point according to the local coordinate system so as to fuse the local invariant features at the sampling point to obtain the image features. Therefore, by constructing a rotatable and invariant local coordinate system for the local image block of the image, the estimation of the main direction and the subsequent direction normalization operation of the local image block can be avoided, and the local invariant feature representing the image can be accurately obtained.
Further, as shown in fig. 8, on the basis of the embodiment of fig. 7, in this embodiment, the extracting module 30 includes:
a setting unit 301, configured to set a predetermined frequency and a predetermined direction at the sampling point according to the local coordinate system, so as to obtain a predetermined number of wavelets;
in this embodiment, the local Gabor features at the sampling points Pi are extracted from the local coordinate system, and 40 Gabor wavelets with 5 frequencies and 8 directions are designedu=0~4,v=0~7;
Wherein,(xi,yi) Is the coordinate of sampling point Pi, f is frequencyThe ratio, v is the direction, σ is the standard deviation of the gaussian function, and j is the complex number. It will of course be appreciated that these parameter values may be based onThe actual needs are reasonably set.
A first acquisition unit 302 configured to acquire a second partial image block centered on the sample point in the first partial image block;
a first calculating unit 303, configured to perform inner product calculation on the predetermined number of wavelets and the second local image block to obtain a response of the predetermined number, where the response represents a local invariant feature at the sampling point.
In this example, 40 small waves were usedAnd the second local image block F (x, y) taking the sampling point Pi as the center is subjected to inner product operation to obtain 40 responses
In this embodiment, each response represents a local invariant feature of each wavelet, and the responses representing each wavelet need to be fused to calculate a histogram representing image features.
In an embodiment, as shown in fig. 9, on the basis of the above embodiment of fig. 7, in this embodiment, the fusion processing module 40 includes:
a second acquisition unit 401 for acquiring a phase of each of the responses;
a second calculating unit 402, configured to perform encoding of a predetermined bit for each frequency according to the phase, and calculate a histogram with a predetermined length to obtain the image feature.
In the present embodiment, at the sampling pointPi derived responseIs a complex number, calculated by calculating the phase of each responseAnd generating a binary code according to the following formula
The above two-bit code is 8-bit coded according to each frequency u, and 10 integers of 0 to 255 can be obtained:
and after extracting the feature points with unchanged rotation of the n sampling points Pi with the central point P according to the steps, fusing the feature points to form a histogram representing the unchanged features of the whole image. For each frequency, can be based onAndtwo length 255 histograms corresponding to a binary code obtained by calculation
It is understood that the above-mentioned encoding method can also be modified, such as using 4-bit encoding or encoding according to real and imaginary values.
In the prior art, the subsequent gradient direction histogram only records the direction information of the image gray level change and does not contain important information such as the intensity and frequency of the gray level change, so that the characteristic discrimination is poor, and the local texture representation with stronger discrimination and description capacity can be obtained by adopting Gabor characteristics, so that different image information can be better distinguished.
In an embodiment, as shown in fig. 10, on the basis of the above embodiment of fig. 7, in this embodiment, the extraction device 1 further includes:
a calculation module 50, configured to obtain a histogram of another image and calculate a distance value between the histograms of the two images;
in this embodiment, a histogram of another image is obtained, where the another image may be the same as, similar to, or completely unrelated to the selected image, and the similarity between the two images can be determined by calculating a distance value between the histograms of the two images. Thus, the applicability is wider.
And a matching module 60, configured to match the local invariant features of the two images according to the distance value.
In this embodiment, the matching of the local invariant features is performed according to the distance value, and a specific calculation formula is as follows:
wherein,andthe 255-dimensional feature histograms extracted from the two images are respectively.
In an embodiment, as shown in fig. 11, on the basis of the above embodiment of fig. 1, in this embodiment, the matching module 60 includes:
a comparing unit 601 for comparing the distance value with a predetermined value;
in this embodiment, the distance value may be compared with a predetermined value, where the predetermined value may be reasonably set according to actual needs, and in this preferred embodiment, the predetermined value may be preferably 0.
A determining unit 602, configured to determine that the two images are similar if the magnitude of the distance value is close to the predetermined value.
In this embodiment, if it is determined that the magnitude of the distance value is close to the predetermined value, for example, 0, it may be determined that the two images are similar. Further, when the distance value between the histograms of the two images is smaller, if the value range is within 0-1, the similarity of the two images is higher; conversely, when the distance value between the histograms of the two images is larger, if the value range is larger than 1, the similarity between the two images is lower.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.