CN102314687B - Method for detecting small targets in infrared sequence images - Google Patents

Abstract

The invention discloses a method for detecting small targets in infrared sequence images, which belongs to the field of image data processing. The method includes the following steps: (1) image preprocessing step: obtaining the differential image of each frame of image of an obtained infrared sequence image; (2) background modeling step: obtaining the characteristic image of each frame of image; (3) smallest non-uniform image segmentation step: working out an optimal segmentation threshold value, and utilizing the optimal segmentation threshold value to segment the obtained characteristic image, so that a small infrared target can be detected. For an image with a uniformly distributed background, the method can effectively rebuild the background; the smallest non-uniform image segmentation method converts the segmentation problem into the segmentation problem of images, the optimal segmentation threshold value is sought from the perspective of segmentation energy, and targets can be more accurately segmented out.

Description

Method for detecting small target in infrared sequence image

Technical Field

The invention belongs to an image data processing method, and particularly relates to a method for detecting an infrared small target under a complex background.

Background

Infrared small targets can be said to be a relative concept, by "small" is meant small in their geometric dimensions in the image, and small targets are generally considered to be targets between 1 x 1 pixel to 10 x 10 pixels in size on the image. The small target has no shape information for identification, low signal-to-noise ratio and unknown position and motion rate, so that the detection of the infrared small target is difficult. Currently existing infrared small target detection algorithms mainly include filter-based detection algorithms, wavelet transform-based detection algorithms, mathematical morphology-based detection algorithms, and some learning-based or motion information-based detection algorithms. The method is simple and effective, but has poor effect under the conditions of small target and low signal-to-noise ratio, and can not effectively filter interference; the detection algorithm based on wavelet transform detects by extracting different characteristics between a target area and a background, but in the case of a small target, the wavelet generally cannot effectively extract the characteristics of the target; the method based on mathematical morphology mainly applies top-hat transformation to inhibit background, but is very sensitive to the selection of structural elements and is also very sensitive to the extraction of small targets under low signal-to-noise ratio; the learning-based method can only achieve a good detection effect if a good training sample is obtained, but generally, the obtaining of the good training sample is difficult. In short, a series of detection algorithms exist at present, and have certain limitations.

The invention content is as follows:

the invention provides a method for detecting a small target in an infrared sequence image. .

The invention discloses a method for detecting a small target of an infrared sequence image, which comprises the following steps:

(1) an image preprocessing step:

in order to have a better detection effect, firstly, preprocessing an image to achieve the aim of enhancing a target by a single frame; firstly, removing the line mean value of each line of the original image; and then, performing mean filtering on the image by using two windows with different sizes, and calculating the difference value of the images to obtain a difference image.

(2) Background modeling step:

starting from a pixel neighborhood, firstly, a difference image f of any two adjacent frames₁And f₂Respectively processed in blocks, and f is₁Is regarded as a vector from f₂Searching k pixel blocks in the nearest neighbor, removing the first pixel block with the nearest neighbor measure by Euclidean distance, reconstructing the pixel block by using the remaining k-1 pixel blocks under the condition of satisfying the minimum reconstruction error, and comparing f with f₁Each pixel block in the image is processed identically, and finally a reconstructed image is obtained; from difference images f₁Subtracting the reconstructed image to obtain a characteristic image;

the basis for this is that the backgrounds are relatively uniformly distributed, each background block has more pixel blocks in the image, the number of targets is small, the background can be well reconstructed, the error of the target reconstruction is large, and when the reconstructed error image is taken as a feature image, the target can be highlighted.

(3) And (3) minimal non-equilibrium graph cutting:

firstly, establishing a grid graph model G (V, E) among pixels, wherein nodes are pixels, edges are constraints w (u, V) among adjacent pixels, and the constraints w (u, V) are used for defining the smoothness of the adjacent pixels;

and (2) regarding the grid model as a three-dimensional graph, converting the segmentation problem into the cutting problem of the graph, wherein the height is pixel gray scale, the plane coordinate is the position of a pixel, the cutting surface can be regarded as a threshold value t of image segmentation, the threshold value t divides the image into two parts A and B, the cutting energy between the A and B and the size of an optimization function are calculated, the circular traversal is performed from the maximum value of the t to the bottom, the t of the minimum value of the optimization function is taken as the optimal segmentation threshold value, and then the obtained characteristic image is segmented by using the segmentation threshold value, so that the infrared small target is detected.

Wherein, the background modeling step process is as follows:

(2.1) for any two adjacent frame images f₁And f₂Respectively carrying out block processing on the raw materials;

(2.2) for f₁Is expressed in vector form, in frame f₂Searching k nearest pixel blocks;

(2.3) for f₁Except for f₂Of the first pixel block closest thereto, from which it is at f₂The corresponding other k-1 pixel blocks are reconstructed according to a reconstruction formula, and the minimum reconstruction error is met, wherein the reconstruction formula is as follows:

${\left|\right|B_1^i-\mathrm{\Σ}{w}_j{B}_2^j\left|\right|}_2,j=1,...,k-1$

wherein,

denotes f₁The ith pixel block in (a),

denotes f₂Of the j-th nearest pixel block, w_jThe weight coefficient of the jth nearest neighbor pixel block;

(2.4)f₁the reconstructed block value of the ith pixel block in (1) is taken as

F is obtained by averaging the reconstructed values of two blocks in the overlapped region₁After the block value of each pixel block is reconstructed, a reconstructed background image f can be obtained_r；

(2.5) finally, image f₁And the reconstructed background image f_rDifference is made to obtain a characteristic image f₁-f_rMost of the background in the feature image at this time is removed, and the target can be highlighted.

Wherein, the minimal non-equilibrium graph cutting step process is as follows:

(3.1) firstly establishing a grid graph model G (V, E) among pixels, wherein nodes are pixels, edges are constraints w (u, V) among adjacent pixels, and the constraints w (u, V) are used for defining the smoothness of the adjacent pixels; if the grid graph model is regarded as a three-dimensional graph, the segmentation problem is converted into the cutting problem of the graph, wherein the height is the pixel gray level, the plane coordinate is the position of the pixel, and the cutting surface can be regarded as a threshold value t of image segmentation; the image segmentation is to segment the node in the graph into two parts, namely A and B, wherein A is higher than the cutting surface and B is lower than the cutting surface.

(3.2) calculating the cleavage energy between A and B, defined as follows:

$\mathrm{Cut}(A,B)=\underset{u\∈A,v\∈B}{\mathrm{\Σ}}w(u,v)$

to accurately segment the small target boundary to achieve the minimum segmentation, the cutting energy Cut (A, B) is used for measurement when A and B are required to be segmented at the position with the minimum constraint; since the division threshold is mainly selected here, the energy calculation is only related to the threshold t, that is, the energy calculation formula can be rewritten as:

$\mathrm{Cut}(A,B)=\underset{I\left(u\right)\&GreaterEqual;t,I\left(v\right)<t}{\mathrm{\&Sigma;}}w(u,v)$

wherein, i (u) is the gray value of the pixel u, i (v) is the gray value of the pixel v, t is the current threshold, w (u, v) is the constraint between the current pixel u and the neighboring pixel v, and the formula is as follows:

$w(u,v)=\left\{\begin{array}{cc}\exp -\left[\right||I\left(u\right)-I\left(v\right)||+||u-v|\left|\right],& \left|\right|u-v\left|\right|<D\\ 0,& \mathrm{otherwise}\end{array}\right.$

the | u-v | < D in the formula represents that the distance between the pixel point u and the pixel point v is less than D, and v is the pixel of eight neighborhoods of u under the general condition, and certainly can also take a larger range, and the size of D depends on the size of the selected neighborhood, and the neighborhood is of different size, and the distance D is also of different size.

(3.3) based on the least-squares graph cut principle, the following optimization function can be constructed:

$L(A,B)=\frac{\mathrm{Cut}(A,B)}{\left|A\right|}+\frac{\mathrm{Cut}(A,B)}{\left|B\right|}$

wherein | A | and | B | are the total number of pixels of A, B respectively;

starting from the maximum t, searching downwards all the time, stopping when L (A, B) reaches the minimum value, and the value of t is the optimal segmentation threshold value.

The background modeling method provided by the invention starts from the angle of a pixel neighborhood, and for a certain pixel block, the pixel block is reconstructed from a plurality of pixel blocks of another frame, and the minimum reconstruction error is met, for an image with relatively uniform background distribution, each background block has more pixel blocks in the image, and targets are fewer, so that the background can be well reconstructed, and the error of target reconstruction is larger, thereby well achieving the purpose of removing the background; the minimum non-equilibrium graph cutting method converts the cutting problem into the cutting problem of the graph, and from the viewpoint of cutting energy, the optimal cutting threshold is searched, so that the target can be more accurately cut.

Drawings

FIG. 1 is a flow chart of the present invention;

FIG. 2(a) is an artwork containing small terrestrial infrared objects;

FIG. 2(b) is the result after pretreatment;

FIG. 2(c) is the result after background removal;

FIG. 2(d) is the result of a minimum imbalance map cut;

FIG. 3(a) is an artwork containing small sea-sky infrared targets;

FIG. 3(b) is the result after pretreatment;

FIG. 3(c) is the result after background removal;

fig. 3(d) is the result of the least squares graph cut.

Detailed Description

The invention is described in further detail below with reference to the figures and specific examples.

The invention relates to a method for detecting a small target of an infrared sequence image, which comprises the following specific steps:

(1) image pre-processing

In order to achieve better detection effect, the image map 2(a) needs to be preprocessed first to achieve the purpose of single frame enhancement. Firstly, removing the line mean value of each line of the original image, then respectively carrying out mean value filtering on the image after the mean value is removed by adopting a 15 × 15 window and a 3 × 3 window, and finally calculating the image difference value after two times of filtering to obtain a difference image figure 2(b), namely the image after preprocessing.

2. Background modeling, background removal

(2.1) preprocessing the image f of two adjacent frames₁And f₂The partitioning is performed, where the selected block size is 11 x 11 and the overlap size between blocks is 2.

(2.2) for f₁Is expressed in vector form, in frame f₂Searching k pixel blocks in nearest neighbor, measuring nearest neighborThe degrees use the ordinary euclidean distance.

(2.3) to avoid reconstructing a block containing an object directly from the corresponding object block, for f₁Except for f₂Of the first pixel block closest thereto, from which it is at f₂The corresponding other k-1 pixel blocks are reconstructed according to a reconstruction formula, and the minimum reconstruction error is met, wherein the reconstruction formula is as follows:

${\left|\right|B_1^i-\mathrm{\&Sigma;}{w}_j{B}_2^j\left|\right|}_2,j=1,...,k-1$

wherein,

denotes f₁The ith pixel block in (a),

denotes f₂Of the j-th nearest pixel block, w_jThe formula represents the calculation of the two norms for the weight coefficient of the jth nearest pixel block and meets the condition of minimum error.

(2.4)f₁The reconstructed block value of the ith pixel block in (1) is taken as

F is obtained by averaging the reconstructed values of two blocks in the overlapped region₁After the block value of each pixel block is reconstructed, a reconstructed background image f can be obtained_r；

(2.5) finally, image f₁And the reconstructed background image f_rDifference is made to obtain a characteristic image f₁-f_rAt this time, most of the background in the feature image is removed, and the object can be highlighted, as shown in fig. 2 (c).

3. Minimal disparity map segmentation

(3.1) firstly establishing a grid graph model G (V, E) among pixels, wherein nodes are pixels, edges are constraints w (u, V) among adjacent pixels, and the constraints w (u, V) are used for defining the smoothness of the adjacent pixels; if the grid graph model is regarded as a three-dimensional graph, the segmentation problem is converted into the cutting problem of the graph, wherein the height is the pixel gray level, the plane coordinate is the position of the pixel, and the cutting surface can be regarded as a threshold value t of image segmentation; the image segmentation is to segment the node in the graph into two parts, namely A and B, wherein A is higher than the cutting surface and B is lower than the cutting surface.

(3.2) aiming at the current threshold t, calculating the current cutting energy Cut (A, B) according to a formula, wherein the pixel values of pixel points u and v meet the condition that I (u) is more than or equal to t, I (v) is less than t, u and v take eight neighborhoods,

$\mathrm{Cut}(A,B)=\underset{I\left(u\right)\&GreaterEqual;t,I\left(v\right)<t}{\mathrm{\&Sigma;}}w(u,v)$

w (u, v) is the constraint between the current pixel u and the neighborhood pixel v, and the formula is as follows:

$w(u,v)=\left\{\begin{array}{cc}\exp -\left[\right||I\left(u\right)-I\left(v\right)||+||u-v|\left|\right],& \left|\right|u-v\left|\right|<D\\ 0,& \mathrm{otherwise}\end{array}\right.$

where | u-v | < D in the equation means that the distance between pixel u and pixel v is less than D, where v is the pixel of the eight neighbourhood of u, the size of D depends on the size of the neighbourhood selected, the neighbourhoods are of different sizes and the distance D is also different.

(3.3) counting the sum of the pixels of the part A which is larger than the threshold value t to be used as a value of | A |; and counting the total number of the pixels of the part B smaller than the threshold value t to be used as a value of | B |, and taking 8 neighborhoods from u and v.

(3.4) substituting the obtained values of Cut (A, B), | A |, and | B | into an optimization function to calculate the size of L (A, B), wherein the optimization function formula is as follows:

$L(A,B)=\frac{\mathrm{Cut}(A,B)}{\left|A\right|}+\frac{\mathrm{Cut}(A,B)}{\left|B\right|}$

and (3.5) circularly performing the steps, starting from the maximum value, searching downwards until the threshold t is reached, so that L (A, B) takes the threshold t of the minimum value, namely the optimal segmentation threshold, segmenting the feature image obtained in the step 2 by using the threshold, namely segmenting a small target, and finally segmenting the result as shown in the step (d) of fig. 2.

The method of the invention can be applied to the detection of the infrared small target under various image backgrounds, such as the detection of the infrared small target under the sea-sky background, and the processing steps are the same as the above embodiment. FIG. 3(a) is an artwork including small sea-sky infrared targets; FIG. 3(b) results after pretreatment; FIG. 3(c) results after background removal; FIG. 3(d) the result of the least squares graph cut.

Claims (2)

1. A method for detecting small targets in infrared sequence images comprises the following steps:

(1) an image preprocessing step:

for each frame of image in the infrared sequence image, firstly, removing the mean value of each line in the image, then respectively carrying out mean value filtering on the image by using two windows with different sizes, and finally calculating the difference value of the image after the two times of mean value filtering to obtain the difference value image of each frame of image;

(2) a background modeling step, namely establishing a reconstructed image of the difference image of each frame of image and obtaining a characteristic image of each frame of image, wherein the background modeling step specifically comprises the following steps:

firstly, respectively carrying out blocking processing on a difference image of each frame of image and a difference image of an adjacent frame of image of each frame of image;

secondly, taking any pixel block of the difference image of each frame of image as a vector, searching k pixel blocks of the nearest neighbor from another difference image, removing one pixel block with the nearest distance to the pixel block, reconstructing the pixel block according to the remaining k-1 neighbor pixel blocks, and reconstructing all the pixel blocks of the difference image of each frame to obtain a reconstructed image of the difference image, wherein k is a positive integer greater than 1;

then, subtracting the difference image of each frame of image from the reconstructed image thereof to obtain a characteristic image of each frame of image;

(3) and (3) minimal non-equilibrium graph cutting:

for the characteristic image of each frame image, firstly establishing a grid graph model G (V, E) among pixels, wherein nodes are pixels and form a set V, edges are constraints w (u, V) among adjacent pixels and form a set E, and the constraints w (u, V) are used for defining the smoothness of the adjacent pixels;

secondly, regarding a grid graph model G (V, E) as a three-dimensional graph, wherein the height is pixel gray, a plane coordinate is the position of a pixel, a threshold value t is used as a cutting surface, a node in the three-dimensional graph is divided into an A part and a B part, the cutting energy and the size of an optimization function between the A part and the B part are calculated, and the optimization function is circularly traversed downwards from the maximum value of the threshold value t, so that the threshold value of the minimum value is taken as an optimal segmentation threshold value;

finally, the optimal segmentation threshold is used for segmenting the obtained characteristic image of each frame of image, so that the infrared small target can be segmented, and the detection is completed;

in the background modeling in step (2), the reconstruction formula for reconstructing the pixel block is as follows:

j=1,...,k-1

wherein,

for the difference image f in which the pixel blocks to be reconstructed are located₁The ith pixel block in (a),

difference image f representing adjacent frames₂Of the j-th nearest pixel block, w_jIs the weight coefficient of the jth pixel block, i, j is a positive integer, | | | | | luminance₂Representing the condition that a two-norm is calculated and the minimum error is met;

in the step (3), the cutting energy between the image a and the image B is defined as follows:

$\mathrm{Cut}(A,B)=\underset{u\&Element;A,v\&Element;B}{\mathrm{\&Sigma;}}w(u,v)$

the above energy calculation formula can be rewritten as:

$\mathrm{Cut}(A,B)=\underset{I\left(u\right)\&GreaterEqual;t,I\left(v\right)<t}{\mathrm{\&Sigma;}}w(u,v)$

wherein, i (u) is the gray value of the pixel u, i (v) is the gray value of the pixel v, t is the current threshold, and w (u, v) is the constraint between the current pixel u and the neighboring pixel v, and the calculation formula is as follows:

$w(u,v)=\left\{\begin{array}{cc}\exp (-[\left|\right|I\left(u\right)-I\left(v\right)\left|\right|+\left|\right|u-v\left|\right|),& \left|\right|u-v\left|\right|<D\\ 0,& \mathrm{otherwisse}\end{array}\right.$

in the formula, | | u-v | < D represents that the distance between the pixel point u and the pixel point v is less than D, and D is a positive integer;

further, in the step (3), the optimization function is:

$L(A,B)=\frac{\mathrm{Cut}(A,B)}{\left|A\right|}+\frac{\mathrm{Cut}(A,B)}{\left|B\right|}$

wherein | A | and | B | are the total number of pixels of image A and image B, respectively.

2. The method of claim 1, wherein the measure of nearest neighbor is Euclidean distance.

Images