CN108256519B - Infrared image significance detection method based on global and local interaction - Google Patents
Infrared image significance detection method based on global and local interaction Download PDFInfo
- Publication number
- CN108256519B CN108256519B CN201711322406.1A CN201711322406A CN108256519B CN 108256519 B CN108256519 B CN 108256519B CN 201711322406 A CN201711322406 A CN 201711322406A CN 108256519 B CN108256519 B CN 108256519B
- Authority
- CN
- China
- Prior art keywords
- structural
- local
- global
- adaptive recursive
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V10/507—Summing image-intensity values; Histogram projection analysis
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an infrared image saliency detection method based on global and local interaction, which comprises the steps of calculating a structural local self-adaptive recursive kernel, constructing an affine matrix to improve saliency map performance, establishing global constraint based on a Gaussian mixture model, filtering noise interference by using a structural filtering method, and integrating local and global models to calculate a final saliency map. The invention can effectively estimate the visual salient target information, improve the effective follow-up area for the target tracking and target identification in the later period, reduce the search consumption of the machine vision algorithm, improve the operation efficiency of the algorithm, reduce the operation power consumption of hardware, improve the resource utilization rate of image signals and provide effective image preprocessing support for the visual task in the later period.
Description
Technical Field
The invention relates to an infrared image significance detection method, in particular to an infrared image significance detection method based on global and local interaction, and belongs to the technical field of image understanding processing.
Background
Saliency detection plays an important role in understanding analysis methods for night vision images (including low-light, infrared images), and it also plays an important role in machine vision applications.
One of documents (x.hou, l.zhang, Dynamic visual attribute: search for coding length entries, in: d.koller, d.schuurmans, y.bengio, l.bottou (Eds.), advance in Neural Information Processing Systems 21,2009.) proposes a Dynamic visual model based on the rarity of features, and applies (ICL) to estimate the gain in entropy of each feature. Document two (t.liu, z.yuan, j.sun, j.wang, n.zheng, x.tang, h. -y.shum, Learning to detect a present object, IEEE TPAMI 33(2),2011.) proposes a method of binary significance detection by training a conditional random field that incorporates a novel set of features such as multi-level contrast, center-around histogram, and color space distribution. However, the above method is proposed based on natural images, and the effect of application to infrared images is not good. Document three (c.n.xinwang, l.xu, scientific detection using spatial consistent-spatial aspects combination, innovative Physics & Technology 72,2015.) uses the luminance contrast and contour features of Infrared images to estimate the Saliency of Infrared images. However, this approach may lead to erroneous estimation results, with the salient regions containing background noise.
Disclosure of Invention
The invention aims to solve the defects of the prior art and provides an infrared image saliency detection method based on global and local interaction.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
an infrared image significance detection method based on global and local interaction comprises the following steps:
the method comprises the following steps: a structural locally adaptive recursive kernel is computed,
let F denote the feature image extracted from the input image I, and equation (1) is:
F(x,y)=Γ(I,x,y) (1)
where Γ () represents a multi-dimensional function that extracts image features,
using the position, gradient, brightness, LBP and HOG information of the infrared image as the features of the image, in the feature image F, each pixel can be described as a 7-dimensional vector, and formula (2) is:
wherein (x, y) represents the position of the pixel,representing edge information of the pixel, HOG () representing HOG features, Lu () representing luminance features, LBP () representing LBP features,
a certain region R of F in the feature image can be described as a multi-dimensional covariance matrix CRThe formula (3) is:
wherein z isi(i ═ 1., k) denotes all the feature points in the region R, and μ denotes ziIs determined by the average value of (a) of (b),
a calculation method of a structural local self-adaptive recursive kernel is disclosed as the following formula (4):
wherein, l belongs to [12],P2Denotes the total number of pixels in the local window, Δ x denotes the coordinate relationship between the center of the window and the surrounding pixels, and s ═ x1,x2,z(x1,x2)},z(x1,x2) Is a pixel (x)1,x2) The gray value of (a);
step two: an affine matrix is constructed, the performance of the saliency map is improved,
the similarity between two regions is represented by the distance between the structural local adaptive recursive kernels of the two regions, and the relevance w of the structural local adaptive recursive kernel in a region m and the structural local adaptive recursive kernel in another region nmnEquation (5) is:
wherein, sl ism,slnMeans representing the structural locally adaptive recursive kernels of the regions m, n, respectively, Q (n) representing a set of neighborhoods of the region n, σ1Is a parameter that controls the degree of similarity, MCS () represents a cosine similarity matrix,
then, a row-normalized affine matrix is constructed, with equation (6) as:
A=D-1·W (6)
wherein the affine matrix W ═ Wmn]N×NUsed to represent the similarity between any pair of nodes, the angle matrix D ═ diag { D }1,d2,...,dNIn which d isn=∑nwmnRepresents the sum of the relevance of the region n to all other regions,
based on a given affine matrix, the saliency of a local region is defined by a descriptor of a structural locally adaptive recursive kernel, and formula (7) is:
wherein, amnRepresents the degree of association, S, calculated in equation (6)SLARKRepresenting a low-quality saliency map based on structural locally adaptive recursive kernels, N representing the number of structural locally adaptive recursive kernels;
step three: based on the Gaussian mixture model, global constraints are established,
first, a global condition is defined and its cost is minimized, and equation (8) is:
wherein, b1,b2A gaussian mixture model representing the foreground and background in the infrared image respectively,
the descriptor of each structural locally adaptive recursive kernel can be regarded as a weight combined with a gaussian mixture model, and belongs to the probability of a neighborhood, and formula (9) is:
wherein, PnRepresents a linear operation for extracting the nth structural local recursive kernel region from the infrared image, wmnIs calculated by the formula (5) to obtainmIs a covariance matrix, phi denotes a gaussian distribution;
step four: by using a structural filtering method, noise interference is filtered,
based on a structural local adaptive recursive kernel, a filtering method is designed to further filter background noise contained in a gaussian model, and formula (10) is as follows:
wherein S isG(x2) Denotes SGMiddle pixel x2The value of the significance of (a) is,is a normalization factor, R (x)2) Is represented by x1As pixels in the neighborhood of the center of the circle,
step five: integrating the local and global models, computing the final saliency map,
the formula (11) is:
S*=αS1+(1-α)S2 (11)
wherein α is a balance factor, S1Representing a local saliency map, S2Representing a global saliency map.
The invention has the following beneficial effects:
the target structure information can be effectively marked by adopting the area covariance, and the difference between the background and the target can be effectively distinguished; and the global and local information is considered, the salient target information can be effectively mined, the visual saliency can be recalculated through an effective integration frame, and the accuracy of saliency detection is improved. And when the global saliency map is calculated, a Gaussian mixture model is utilized to establish global clue constraint, and the interference of noise is reduced through structural filtering. The method can effectively estimate the visual salient target information, improve effective follow-up areas for target tracking and target identification in the later period, reduce the search consumption of a machine vision algorithm, improve the operation efficiency of the algorithm, reduce the operation power consumption of hardware, improve the resource utilization rate of image signals and provide effective image preprocessing support for visual tasks in the later period.
Drawings
FIG. 1 is a schematic diagram of the infrared image saliency detection method of the present invention.
Fig. 2 is a partial saliency map obtained in step two of the present invention.
FIG. 3 shows the second step of the present inventionLocal saliency maps at different values are taken.
FIG. 4 is a saliency map of the global model of the present invention.
FIG. 5 shows the fourth step of the present inventionLocal saliency maps at different values are taken.
FIG. 6 is a graph of the significance detection results of the present invention.
Detailed Description
The invention provides an infrared image significance detection method based on global and local interaction. The technical solution of the present invention is described in detail below with reference to the accompanying drawings so that it can be more easily understood and appreciated.
An infrared image saliency detection method based on global and local interaction, as shown in fig. 1, includes the following steps:
the method comprises the following steps: a structural locally adaptive recursive kernel is computed,
let F denote the feature image extracted from the input image I, and equation (1) is:
F(x,y)=Γ(I,x,y) (1)
where Γ () represents a multi-dimensional function that extracts image features,
using the position, gradient, brightness, LBP and HOG information of the infrared image as the features of the image, in the feature image F, each pixel can be described as a 7-dimensional vector, and formula (2) is:
wherein (x, y) is as followsShowing the location of the pixel(s),representing edge information of the pixel, HOG () representing HOG features, Lu () representing luminance features, LBP () representing LBP features,
a certain region R of F in the feature image can be described as a multi-dimensional covariance matrix CRThe formula (3) is:
wherein z isi(i ═ 1., k) denotes all the feature points in the region R, and μ denotes ziIs determined by the average value of (a) of (b),
the calculation method of the structural local self-adaptive recursive kernel has the following formula (4):
wherein l belongs to [12],P2Denotes the total number of pixels in the local window, Δ x denotes the coordinate relationship between the center of the window and the surrounding pixels, and s ═ x1,x2,z(x1,x2)},z(x1,x2) Is a pixel (x)1,x2) The gray value of (a);
step two: an affine matrix is constructed, the performance of the saliency map is improved,
the similarity between two regions is represented by the distance between the structural local adaptive recursive kernels of the two regions, and the relevance w of the structural local adaptive recursive kernel in a region m and the structural local adaptive recursive kernel in another region nmnEquation (5) is:
wherein, sl ism,slnRespectively representing the junctions of the regions m, nMean of a constitutive locally adaptive recursive kernel, Q (n), representing a set of neighborhoods of the region n, σ1Is a parameter for controlling the degree of similarity, MCS () represents a cosine similarity matrix,
then, a row-normalized affine matrix is constructed, with equation (6) as:
A=D-1·W (6)
wherein the affine matrix W ═ Wmn]N×NUsed to represent the similarity between any pair of nodes, the angle matrix D ═ diag { D }1,d2,...,dNIn which d isn=∑nwmnRepresents the sum of the relevance of the region n to all other regions,
based on a given affine matrix, the saliency of a local region is defined by a descriptor of a structural local adaptive recursive kernel, and the formula (7) is as follows:
wherein, amnRepresents the degree of association, S, calculated in equation (6)SLARKRepresenting a low-quality saliency map based on structural locally adaptive recursive kernels, N representing the number of structural locally adaptive recursive kernels;
step three: based on the Gaussian mixture model, global constraints are established,
first, a global condition is defined and its cost is minimized, and equation (8) is:
wherein, b1,b2A gaussian mixture model representing the foreground and background in the infrared image respectively,
the descriptor of each structural locally adaptive recursive kernel can be regarded as a weight combined with a gaussian mixture model, and belongs to the probability of a neighborhood, and formula (9) is:
wherein, PnRepresents a linear operation for extracting the nth structural local recursive kernel region from the infrared image, wmnIs calculated by the formula (5) to obtainmIs a covariance matrix, Φ represents a gaussian distribution;
step four: the structural filtering method is used for filtering noise interference,
based on a structural local adaptive recursive kernel, a filtering method is designed to further filter background noise contained in a gaussian model, and formula (10) is as follows:
wherein S isG(x2) Denotes SGMiddle pixel x2The value of the significance of (a) is,is a normalization factor, R (x)2) Is represented by x1Are the pixels in the neighborhood of the center of the circle,
step five: integrating the local and global models, computing the final saliency map,
the formula (11) is:
S*=αS1+(1-α)S2 (11)
wherein α is a balance factor, S1Representing a local saliency map, S2Representing a global saliency map.
As shown in fig. 2, the local saliency map obtained in step two in the present invention is shown, where (a) in fig. 2 is an input image, (b) in fig. 2 is a local saliency map, and (c) in fig. 2 is a local saliency map to which affine matrix calculation is added. The influence of the affine matrix is increased, so that the saliency map is more accurate, and the background noise is well suppressed.
As shown in FIG. 3, it is the formula (5) in step twoLocal saliency maps at different values are taken. Wherein, in (a) of FIG. 3In (b) of FIG. 3In (c) of FIG. 3In (d) in FIG. 3
As shown in fig. 4, a saliency map of the global model. Fig. 4 (a) is an input image, fig. 4 (b) is a global saliency map, and fig. 4 (c) is a global saliency map to which a structural filtering calculation is added. Through the processing of structural filtering, the target is highlighted, and the background noise is also suppressed.
As shown in FIG. 5, it is a formula (10) of the four middle of the stepsLocal saliency maps at different values are taken. In (a) of FIG. 5In (b) of FIG. 5In (c) of FIG. 5In (d) of FIG. 5
As shown in fig. 6, it is a significance detection result diagram of the invention.
Through the above description, it can be found that the infrared image saliency detection method based on global and local interaction can effectively estimate the visual saliency target information, improve the effective follow-up region for target tracking and target identification in the later period, reduce the search consumption of the machine vision algorithm, improve the operation efficiency of the algorithm, reduce the operation power consumption of hardware, improve the resource utilization rate of image signals, and provide effective image preprocessing support for the visual task in the later period.
The technical solutions of the present invention are fully described above, it should be noted that the specific embodiments of the present invention are not limited by the above description, and all technical solutions formed by equivalent or equivalent changes in structure, method, or function according to the spirit of the present invention by those skilled in the art are within the scope of the present invention.
Claims (1)
1. An infrared image significance detection method based on global and local interaction comprises the following steps:
the method comprises the following steps: a structural locally adaptive recursive kernel is computed,
let F denote the feature image extracted from the input image I, and equation (1) is:
F(x,y)=Γ(I,x,y) (1)
where Γ () represents a multi-dimensional function that extracts image features,
using the position, gradient, brightness, LBP and HOG information of the infrared image as the features of the image, in the feature image F, each pixel is described as a 7-dimensional vector, and formula (2) is:
wherein (x, y) represents the position of the pixel,representation imageEdge information of pixels, HOG () representing HOG features, Lu () representing luminance features, LBP () representing LBP features,
a certain region R of F in the characteristic image is described as a multi-dimensional covariance matrix CRThe formula (3) is:
wherein z isiDenotes all the feature points in the region R, where i ═ 1., k, μ denotes ziIs determined by the average value of (a) of (b),
the calculation method of the structural local self-adaptive recursive kernel has the following formula (4):
wherein l belongs to [12],P2Denotes the total number of pixels in the local window, Δ x denotes the coordinate relationship between the center of the window and the surrounding pixels, and s ═ x1,x2,z(x1,x2)},z(x1,x2) Is a pixel (x)1,x2) The gray value of (a);
step two: an affine matrix is constructed, the performance of the saliency map is improved,
the similarity between two regions is represented by the distance between the structural local adaptive recursive kernels of the two regions, and the relevance w of the structural local adaptive recursive kernel in a region m and the structural local adaptive recursive kernel in another region nmnEquation (5) is:
wherein, sl ism,slnDenotes the mean of the structural locally adaptive recursive kernels of the regions m, n, respectively, Ω (n) denotes a set of neighborhoods of the region n, σ1Is a parameter controlling the degree of similarity, MCS () represents a cosine similarity matrix,
then, a row-normalized affine matrix is constructed, with equation (6) as:
A=D-1·W (6)
wherein the affine matrix W ═ Wmn]N×NUsed to represent the similarity between any pair of nodes, the angle matrix D ═ diag { D }1,d2,...,dNIn which d isn=∑nwmnRepresents the sum of the relevance of the region n to all other regions,
based on a given affine matrix, the saliency of a local region is defined by a descriptor of a structural locally adaptive recursive kernel, and formula (7) is:
wherein, amnRepresents the degree of association, S, calculated in equation (6)SLARKRepresenting a low-quality saliency map based on structural locally adaptive recursive kernels, N representing the number of structural locally adaptive recursive kernels;
step three: based on the Gaussian mixture model, global constraints are established,
first, a global condition is defined and its cost is minimized, and equation (8) is:
wherein, b1,b2A gaussian mixture model representing the foreground and background in the infrared image respectively,
the descriptor of each structural local adaptive recursive kernel is regarded as the weight combined with the Gaussian mixture model, and belongs to the probability of the neighborhood, and the formula (9) is:
wherein, PnRepresents a linear operation for extracting the nth structural local recursive kernel region from the infrared image, wmnThe method is calculated by a formula (5), Cm is a covariance matrix, and phi represents Gaussian distribution;
step four: the structural filtering method is used for filtering noise interference,
based on a structural local adaptive recursive kernel, a filtering method is designed to further filter background noise contained in a gaussian model, and a formula (10) is as follows:
wherein S isG(x2) Denotes SGMiddle pixel x2The value of the significance of (a) is,is a normalization factor, R (x)2) Is represented by x1As pixels in the neighborhood of the center of the circle,
step five: integrating the local and global models, calculating the final saliency map,
the formula (11) is:
S*=αS1+(1-α)S2 (11)
wherein α is a balance factor, S1Represents a local saliency map, S2Representing a global saliency map.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711322406.1A CN108256519B (en) | 2017-12-13 | 2017-12-13 | Infrared image significance detection method based on global and local interaction |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201711322406.1A CN108256519B (en) | 2017-12-13 | 2017-12-13 | Infrared image significance detection method based on global and local interaction |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108256519A CN108256519A (en) | 2018-07-06 |
CN108256519B true CN108256519B (en) | 2022-06-17 |
Family
ID=62722584
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201711322406.1A Active CN108256519B (en) | 2017-12-13 | 2017-12-13 | Infrared image significance detection method based on global and local interaction |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108256519B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113822352B (en) * | 2021-09-15 | 2024-05-17 | 中北大学 | Infrared dim target detection method based on multi-feature fusion |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104599291A (en) * | 2015-01-21 | 2015-05-06 | 内蒙古科技大学 | Structural similarity and significance analysis based infrared motion target detection method |
CN106295542A (en) * | 2016-08-03 | 2017-01-04 | 江苏大学 | A kind of road target extracting method of based on significance in night vision infrared image |
CN106846331A (en) * | 2016-12-22 | 2017-06-13 | 中国科学院文献情报中心 | Joint vision significance and figure cut the image automatic segmentation method of optimization |
CN107240096A (en) * | 2017-06-01 | 2017-10-10 | 陕西学前师范学院 | A kind of infrared and visual image fusion quality evaluating method |
-
2017
- 2017-12-13 CN CN201711322406.1A patent/CN108256519B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104599291A (en) * | 2015-01-21 | 2015-05-06 | 内蒙古科技大学 | Structural similarity and significance analysis based infrared motion target detection method |
CN106295542A (en) * | 2016-08-03 | 2017-01-04 | 江苏大学 | A kind of road target extracting method of based on significance in night vision infrared image |
CN106846331A (en) * | 2016-12-22 | 2017-06-13 | 中国科学院文献情报中心 | Joint vision significance and figure cut the image automatic segmentation method of optimization |
CN107240096A (en) * | 2017-06-01 | 2017-10-10 | 陕西学前师范学院 | A kind of infrared and visual image fusion quality evaluating method |
Non-Patent Citations (1)
Title |
---|
《Saliency detection based on global and local short-term》;Qiang Fan 等;《Neurocomputing》;20151024;第81-89页 * |
Also Published As
Publication number | Publication date |
---|---|
CN108256519A (en) | 2018-07-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108256562B (en) | Salient target detection method and system based on weak supervision time-space cascade neural network | |
CN110782477A (en) | Moving target rapid detection method based on sequence image and computer vision system | |
Xin et al. | A self-adaptive optical flow method for the moving object detection in the video sequences | |
Kim et al. | Background subtraction using illumination-invariant structural complexity | |
CN109993744B (en) | Infrared target detection method under offshore backlight environment | |
CN115116132B (en) | Human behavior analysis method for depth perception in Internet of things edge service environment | |
CN112633294A (en) | Significance region detection method and device based on perceptual hash and storage device | |
CN110135435B (en) | Saliency detection method and device based on breadth learning system | |
Alsanad et al. | Real-time fuel truck detection algorithm based on deep convolutional neural network | |
CN104715476A (en) | Salient object detection method based on histogram power function fitting | |
Liu et al. | A shadow imaging bilinear model and three-branch residual network for shadow removal | |
Xu et al. | Extended non-local feature for visual saliency detection in low contrast images | |
CN108256519B (en) | Infrared image significance detection method based on global and local interaction | |
CN108647605B (en) | Human eye gaze point extraction method combining global color and local structural features | |
Xie et al. | 3D surface segmentation from point clouds via quadric fits based on DBSCAN clustering | |
CN109410134A (en) | A kind of self-adaptive solution method based on image block classification | |
Hassan et al. | A hue preserving uniform illumination image enhancement via triangle similarity criterion in HSI color space | |
CN102129687B (en) | Self-adapting target tracking method based on local background subtraction under dynamic scene | |
CN109117852B (en) | Unmanned aerial vehicle image adaptation area automatic extraction method and system based on sparse representation | |
CN114022520B (en) | Robot target tracking method based on Kalman filtering and twin network | |
Wang et al. | Image haze removal using a hybrid of fuzzy inference system and weighted estimation | |
CN116109682A (en) | Image registration method based on image diffusion characteristics | |
CN112329796B (en) | Infrared imaging cloud detection method and device based on visual saliency | |
CN115035397A (en) | Underwater moving target identification method and device | |
Lin et al. | Infrared small target detection based on YOLO v4 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |