CN112215146A - Weak and small target joint detection and tracking system and method based on random finite set - Google Patents
Weak and small target joint detection and tracking system and method based on random finite set Download PDFInfo
- Publication number
- CN112215146A CN112215146A CN202011086220.2A CN202011086220A CN112215146A CN 112215146 A CN112215146 A CN 112215146A CN 202011086220 A CN202011086220 A CN 202011086220A CN 112215146 A CN112215146 A CN 112215146A
- Authority
- CN
- China
- Prior art keywords
- target
- measurement
- track
- infrared
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The invention provides a weak and small target joint detection and tracking system and method based on a random finite set, which comprises an infrared image measuring module, a target measuring domain acquiring module and a target tracking module, wherein the infrared image measuring module is used for acquiring infrared measuring images of a ground environment and a target so as to acquire the target measuring domain; the target prediction module is used for calculating a target measurement domain to obtain a prediction state of a target and a target prediction track label, and then performing Gaussian sampling; the target updating module is used for calculating a likelihood function and calculating an updating state of a target and a target updating track label according to the likelihood function; and establishing an assumed cost function associated with the target and the measurement, and selecting an assumption corresponding to the maximum weight according to the assumed weight to obtain the target state and the track estimation value at the current moment. According to the invention, on the basis of obtaining target infrared measurement in a complex environment, effective detection and tracking of infrared dim targets in a complex noise environment are realized through measurement difference, adaptive extraction of new targets and optimal track distribution.
Description
Technical Field
The invention belongs to the technical field of target detection and tracking, and particularly relates to a weak and small target joint detection and tracking system and method based on a random finite set.
Background
In modern war, along with the development of stealth technology and the increasing complexity of battlefield environment, the target with low signal-to-noise ratio presents weak target characteristics in sensor measurement, and the target is easily buried in noise and cannot be effectively associated with the measurement, so that the traditional tracking method is difficult to effectively detect and track the target. Moreover, most of the traditional tracking methods are based on a radar system, but because the radar system also radiates the self characteristic outwards while detecting the target, the radar measurement system is easy to expose the position of the radar measurement system, so that the radar measurement system is attacked by an enemy and is difficult to survive in the current war. The infrared imaging technology has the advantages of strong anti-interference capability on target detection, high tracking precision and the like in a complex battlefield, so that the research on the infrared image-based multi-target detection and tracking system and method has important significance. Meanwhile, as the traditional tracking algorithms JPDAF and MHT need to divide the targets through data association, association errors are easily caused in a complex scene to cause the reduction of the tracking performance of the JPDAF and the MHT, a tracking technology based on a Random Finite Set (RFS) is introduced to solve the problem, the number of the targets which are unknown and time-varying can be jointly detected and tracked without complex data association processing, and the tracking precision can be effectively improved. However, in the field of tracking multiple weak and small targets in a complex scene, research combining infrared image measurement and an RFS tracking technology is still in a starting stage, and the problems of high calculation complexity, high target quantity false alarm probability under a complex background, low tracking precision and the like exist.
Disclosure of Invention
In order to overcome the defects of the prior art and overcome the defects of the existing infrared image-based weak and small target detection and tracking algorithm, the invention aims to provide a weak and small target combined detection and tracking system and method based on a random finite set.
In order to achieve the purpose, the invention adopts the technical scheme that:
a weak and small target joint detection and tracking system based on a random finite set comprises:
the infrared image measurement module is used for scanning in real time by using an infrared sensor to obtain infrared measurement images of the ground environment and the target, then inhibiting background noise of the infrared measurement images of the ground environment and the target by using the gray level characteristics of the infrared images, and filtering ineffective measurement to obtain a target measurement domain;
the target prediction module is used for calculating a target measurement domain according to a Bayesian rule and by utilizing a Markov transfer matrix to obtain a prediction state and a target prediction track label of a target, and carrying out Gaussian sampling on the prediction target according to the prediction state and the target prediction track label of the target to obtain a particle set corresponding to each prediction target;
the target updating module is used for calculating the likelihood function of the predicted target particles in the particle set corresponding to each predicted target by using the amplitude information corresponding to different positions in the target measurement domain, and calculating the updating state of the target and the target updating track label according to the predicted target particle likelihood function and the normalized particle weight; and establishing an assumed cost function associated with the target and the measurement, calculating m optimal distribution hypotheses according to the assumed cost function associated with the target and the measurement, and finally selecting a hypothesis corresponding to the maximum weight according to the assumed weight to obtain a target state and a track estimation value at the current moment so as to realize the tracking of multiple targets.
The system is further improved in that the system also comprises a data display module connected with the target updating module, wherein the data display module is used for displaying the measured data, the real running track of the target, the estimated values of the state and the number of the target, and the estimated error curve of the number and the track of the target.
The invention has the further improvement that the infrared image measuring module comprises a constant false alarm rate image segmentation module and an infrared target gray characteristic measuring and extracting module;
the constant false alarm rate image segmentation module is used for carrying out background and clutter suppression on the infrared measurement images of the ground environment and the target to obtain a candidate measurement domain;
and the infrared target gray characteristic measurement and extraction module is used for extracting a target measurement domain from the candidate measurement domain.
In a further improvement of the invention, the target prediction module comprises a fresh target prediction module and a survival target prediction module;
the new target prediction module is used for generating a new target track table according to the target measurement domain; when the new target information is known, a target measurement domain is used as a measurement set to participate in updating the target track table, when the new target information is unknown, a self-adaptive new target algorithm is adopted to measure an area which is not associated with a target in the target measurement domain, new track particles are generated, a likelihood function corresponding to the particles is calculated, the new target measurement area is determined through resampling of the new particles, and the new target track table is initialized;
and the survival target prediction module is used for corresponding the updated target at the last moment to the sampled particles, calculating the survival state and the weight of the particles according to a Bayesian rule and a state transition equation, forming a target prediction set with the new target, and generating a predicted track table, wherein the predicted track table comprises a target state and a target track label.
The invention is further improved in that the target updating module comprises a target state updating module and a target track updating module;
the target state updating module is used for calculating the updating state and the normalization weight of each target corresponding particle in the predicted track table by using the predicted state of the target according to the ground environment of the current moment and the infrared measurement diagram of the target by adopting a particle filtering method, and taking the updating state and the normalization weight of each target corresponding particle in the predicted track table as the target state estimation value in the updated track table;
and the target track updating module is used for pairing the target prediction state and different positions in the target measurement domain at the current moment to obtain corresponding track hypotheses, then calculating the cost value of each track hypothesis, calculating m optimal distribution track hypotheses aiming at the track hypothesis cost by using a mutty algorithm, and finally extracting the maximum weight track hypothesis to obtain the track estimation value of the target.
The weak and small target joint detection and tracking method based on the random finite set comprises the following steps:
1) the method comprises the steps of utilizing an infrared sensor to conduct real-time scanning to obtain infrared measurement images of a ground environment and a target, then utilizing the gray level characteristics of the infrared images to restrain background noise of the infrared measurement images of the ground environment and the target, and then filtering ineffective measurement to obtain a target measurement domain;
2) calculating a target measurement domain according to Bayesian criteria and by using a Markov transfer matrix to obtain a predicted state and a target predicted track label of a target, and performing Gaussian sampling on the predicted target according to the predicted state and the target predicted track label of the target to obtain a particle set corresponding to each predicted target;
3) calculating a likelihood function of a predicted target particle in a particle set corresponding to each predicted target by using amplitude information corresponding to different positions in a target measurement domain, and calculating an update state of the target and a target update track label according to the predicted target particle likelihood function and the normalized particle weight; and establishing an assumed cost function associated with the target and the measurement, calculating m optimal distribution hypotheses according to the assumed cost function associated with the target and the measurement, and finally selecting a hypothesis corresponding to the maximum weight according to the assumed weight to obtain a target state and a track estimation value at the current moment so as to realize the tracking of multiple targets.
The further improvement of the invention is that the background noise of the ground environment and the infrared measurement image of the target is then inhibited by utilizing the gray level characteristics of the infrared image, the ineffective measurement is filtered, and the specific process of obtaining the target measurement domain is as follows:
calculating a measurement likelihood function by calculating the distance gradient between the resolution unit of the candidate measurement domain and the surrounding neighborhood resolution units, and distinguishing the infrared weak target from the background according to the distance gradient and the measurement likelihood function to determine the target measurement domain at the current moment.
The invention is further improved in that the specific process of the step 2) is as follows:
generating a new target track table according to the target measurement domain; when the new target information is known, a target measurement domain is used as a measurement set to participate in updating the target track table, when the new target information is unknown, a self-adaptive new target algorithm is adopted to measure an area which is not associated with a target in the target measurement domain, new track particles are generated, a likelihood function corresponding to the particles is calculated, the new target measurement area is determined through resampling of the new particles, and the new target track table is initialized;
and calculating the prediction state of the corresponding particle of each track in the target track table at the last moment by using a shortest path algorithm and a state transition equation, normalizing the weight to generate a survival target track table, and finally forming a target prediction track table together with the new target so as to obtain a particle set corresponding to each prediction target.
The invention is further improved in that the specific process of the step 3) is as follows: according to the infrared measurement data at the current moment, calculating a likelihood function and a weight value of each target prediction particle in a particle set corresponding to each prediction target to obtain an updated weight of the particles when each target is associated with different measurements in a track list, calculating the sum of non-normalized particle weights and an associated cost function of the target and the measurements, finally calculating an optimal distribution hypothesis through a mutty algorithm, and taking a target track value and a potential distribution value corresponding to the track hypothesis with the maximum weight as target states and number estimates to realize the tracking of multiple targets.
A further improvement of the present invention is that, in step 3), the cost function is assumed to be:
wherein:represents the measurement at the resolution cell (i, j) at time k,representing target statesRight of (1)The weight of the steel is heavy,
the likelihood function is:
wherein:represents the scattering intensity of the target at the resolution cell (i, j) at time k, σ is the standard deviation of the measured noise,representing the infrared measurement amplitude at the resolution unit (i, j) at the moment k;
removing the associated measurement in the current measurement set to obtain a measurement set Z which is not associated with the targetk,φ;
In metrology sets Z not associated with the targetk,φNewly generating target particles in the region, calculating a likelihood function associated with the particles and measurement, and resampling to obtain a newly generated target measurement region at the moment k;
for theGenerating new target particles in the measurement region, wherein the weight of each particle is as follows:
wherein:respectively represent the target statesThe set of resolution elements in the x-direction and y-direction,is in a target stateRelative to the measurementA likelihood function of (a);
normalization of particle weights:obtaining a weighted set of particles corresponding to each unassociated measurement:the particle weight reflects the possibility that the particle state is a new target state, and finally resampling is adopted to retain the particles with large weight,and obtaining a region where the resampled particles are located, wherein the region is a new target measurement region at the k moment.
Compared with the prior art, the invention has the following beneficial effects: the method is characterized in that the original infrared measurement image data in the complex environment is segmented, and the target measurement domain is extracted by utilizing the gray level characteristic of target infrared measurement. Meanwhile, the method combines a random finite set theory and a weak target tracking algorithm, utilizes the designed target measurement extraction method to estimate the target state, and can not output the target track compared with the traditional tracking method. The invention provides a weak and small target combined detection and tracking system based on a random finite set method for detecting and tracking multiple weak and small targets in a complex battlefield environment, and compared with the existing weak and small target tracking system, the weak and small target combined detection and tracking system is more targeted, and realizes effective detection and tracking of infrared weak and small targets in a complex noise environment through measurement difference, adaptive extraction of newly-generated targets, optimal distribution of flight paths and other algorithms on the basis of obtaining target infrared measurement in the complex environment.
Furthermore, after the multi-target state and the track are subjected to filtering estimation, pruning and merging operation are carried out on the updated multi-target track list according to the assumed track and the weight, and the estimation precision can be further improved and the calculation complexity of the algorithm can be reduced by optimizing the updated track list.
Drawings
Fig. 1 is an image of the intensity contribution of the target to the observation, where (a) is a target self-amplitude image of the 1 st frame, (b) is a target self-amplitude image of the 15 th frame, (c) is a target self-amplitude image of the 21 st frame, (d) is a target self-amplitude image of the 25 th frame, and (e) is a target self-amplitude image of the 36 th frame.
Fig. 2 shows the original ir intensity observation image, (a) is the ir measurement raw image of the 1 st frame, (b) is the ir measurement raw image of the 15 th frame, (c) is the ir measurement raw image of the 21 st frame, (d) is the ir measurement raw image of the 25 th frame, and (e) is the ir measurement raw image of the 36 th frame.
Fig. 3 is an image of a target measurement processed only by using a conventional image segmentation algorithm, wherein (a) is an infrared measurement raw image of a 1 st frame, (b) is an infrared measurement raw image of a 15 th frame, (c) is an infrared measurement raw image of a 21 st frame, (d) is an infrared measurement raw image of a 25 th frame, and (e) is an infrared measurement raw image of a 36 th frame.
Fig. 4 shows the infrared measurement pre-processed image obtained by the improved algorithm, wherein (a) is the pre-processed measurement image of the 1 st frame, (b) is the pre-processed measurement image of the 15 th frame, (c) is the pre-processed measurement image of the 21 st frame, (d) is the pre-processed measurement image of the 25 th frame, and (e) is the pre-processed measurement image of the 36 th frame.
Fig. 5 is an image of the extracted measurement area intensity observation, where (a) an amplitude image of the measurement area is extracted for the 1 st frame, (b) an amplitude image of the measurement area is extracted for the 15 th frame, (c) an amplitude image of the measurement area is extracted for the 21 st frame, (d) an amplitude image of the measurement area is extracted for the 25 th frame, and (e) an amplitude image of the measurement area is extracted for the 36 th frame.
FIG. 6 is a diagram of an estimated value and a true value of a target trajectory.
Figure 7 is a target OSPA distance and potential distribution image of the improved algorithm.
Figure 8 is a graph of the target OSPA distance and potential distribution of the original algorithm.
Fig. 9 is a schematic diagram of the system structure of the present invention.
FIG. 10 is a flow chart of the system use of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the present invention is a weak and small target joint detection and tracking system based on a random finite set, which suppresses background noise and optimizes the generation of a new target based on target measurement extraction and a self-adaptive new target method, and realizes effective detection and tracking of multiple weak and small targets in a complex environment, and the system includes:
the infrared image measuring module: the method is used for scanning in real time by using an infrared sensor installed by an unmanned aerial vehicle based on the current environment, acquiring infrared measurement images of the ground environment and the target, exchanging sensing information with a remote center through a data link, then suppressing background noise of the infrared measurement images of the ground environment and the target by using the gray level characteristics of the infrared images, and filtering ineffective measurement to obtain a target measurement domain.
The infrared image measuring module specifically comprises a constant false alarm rate image segmentation module and an infrared target gray characteristic measuring and extracting module; the constant false alarm rate image segmentation module is used for performing background and clutter suppression on the infrared measurement images of the ground environment and the target by adopting a proper threshold value to obtain a candidate measurement domain; on the premise of ensuring certain precision, the suspected target is segmented in the full measurement domain, the number of candidate measurement domains is reduced, and the possible suspected target measurement regions are represented in the graph;
the infrared target gray characteristic measurement and extraction module is used for extracting a target measurement domain from the obtained candidate measurement domain; specifically, because the intensity is relatively gentle relative to the infrared background region, the intensity of the resolution unit where the infrared target is located is relatively high, the scattering characteristic meets a certain exponential characteristic, and the intensity is exponentially attenuated along with the distance in the target neighborhood direction, the measurement likelihood function can be calculated by calculating the distance gradient between the resolution unit of the candidate measurement region and the surrounding neighborhood resolution units, the infrared weak target and the background are distinguished according to the distance gradient and the measurement likelihood function, and the target measurement region at the current moment is determined.
The target prediction module is used for calculating a target measurement domain according to a Bayesian rule and by utilizing a Markov transfer matrix to obtain a prediction state and a target prediction track label of a target, and carrying out Gaussian sampling on the prediction target according to the prediction state and the target prediction track label of the target to obtain a particle set corresponding to each prediction target;
the target prediction module is divided into a new target prediction module and a survival target prediction module according to the new target and the survival target;
the new target prediction module is used for generating a new target track table according to the target measurement domain; when the new target information is known, the target measurement domain is used as a measurement set to participate in updating the target track table, when the new target information is unknown, a self-adaptive new target algorithm is adopted, measurement regions which are not associated with the target are selected in the target measurement domain, track particles are newly generated in the regions, likelihood functions corresponding to the particles are calculated, the new target measurement region is determined through resampling of the new particles, and the new target track table is initialized.
And the survival target prediction module is used for calculating the survival state and the weight of the particles according to the Bayesian criterion and the state transition equation and the sampling particles corresponding to the updated target at the last moment, and forming a target prediction set together with the new target to generate a predicted track table, wherein the predicted track table comprises a target state and a target track label.
A target update module: the method is used for calculating the likelihood function of the predicted target by using the target measurement domain, and further obtaining the estimated values of the target state and the track, thereby realizing the detection and tracking of the target. Specifically, the method is used for calculating the likelihood function of the predicted target particles in the particle set corresponding to each predicted target by using the amplitude information corresponding to different positions in the target measurement domain, and calculating the update state of the target and the target update track label according to the predicted target particle likelihood function and the normalized particle weight; and establishing an assumed cost function associated with the target and the measurement, calculating m optimal distribution hypotheses according to the assumed cost function associated with the target and the measurement, and finally selecting a hypothesis corresponding to the maximum weight according to the assumed weight to obtain a target state and a track estimation value at the current moment so as to realize the tracking of multiple targets.
The target updating module specifically comprises a target state updating module and a target track updating module; and the target state updating module is used for calculating the updating state and the normalization weight of each particle corresponding to each target in the predicted track table by using the predicted state of the target and the target track label obtained by the target predicting module according to the ground environment and the infrared measurement diagram of the target at the current moment and by adopting a particle filtering method, and taking the updating state and the normalization weight of each particle corresponding to each target in the predicted track table as the estimated value of the target state in the updated track table.
The target track updating module is used for obtaining corresponding track hypotheses by pairing different positions in a target measurement domain of a target prediction state and the current moment, then calculating the cost value of each track hypothesis, calculating m optimal distribution track hypotheses aiming at track hypothesis cost by using a mutty algorithm, and finally extracting the maximum weight track hypothesis to obtain a track estimation value of the target.
A data display module: the data display module is connected with the target updating module and is used for displaying the measured data including the infrared measured image, the real running track of the target and the estimated value of the target state and the number, and generating an estimated error curve of the target number and the track expressed by OSPA criterion.
The data display module specifically comprises an infrared measurement image display module, a target number display module, a target track display module and an OSPA error estimation display module, wherein the infrared measurement image display module is used for displaying a currently scanned infrared target amplitude image on an interface, the image comprises target intensity and noise intensity, 40 frames of observation data are generated in total according to an infrared observation model, no target exists in a scene monitored at an initial moment, and target motion information is set as shown in a table 1 in consideration of new and lost states of the target which can appear in a tracking scene under the condition that target derivation is eliminated.
TABLE 1 information about objects in a scene
The monitoring area is divided into 100 × 100 resolution units, that is, nx my 100, and the width Δ x of the resolution unit is made to be Δ y 1, so that a scene with a signal-to-noise ratio smaller than 10 can be selected in the system in consideration of a more complex target tracking environment, and a tracked target meets the characteristics of a weak target.
The observed intensity contribution images of the target at the 1 st frame, the 15 th frame, the 21 st frame, the 25 th frame and the 36 th frame, the original infrared intensity observation image and the extracted measured area intensity observation image are shown in fig. 1, fig. 2 and fig. 3, and the signal-to-noise ratio is set to 9.3.
As shown in fig. 2, the conventional image segmentation algorithm segments the measurement image through a threshold, and although the measurement noise in the original infrared observation value can be suppressed and the background noise can be removed, a large amount of background noise still exists in the infrared image to cover the target intensity, so that the target cannot be effectively observed, and the subsequent target tracking noise is greatly affected.
On the basis of the traditional image segmentation technology, the method utilizes the measurement difference and the gray characteristic of the target to suppress the background noise in the image, and screens candidate measurement through the measurement likelihood function to determine the target measurement area, and has the following effects.
Preprocessing the infrared measurement image through the measurement difference and the infrared target gray level characteristic;
further determining a target measurement area through a measurement likelihood function;
as can be seen from fig. 2, the target is completely submerged in the background noise and clutter, and effective measurement information about the target cannot be obtained in the original image, but the measurement difference and the infrared intensity diffusion characteristic of the target are used to process the original image by the method provided by the present invention, so that the measurement area corresponding to the live target and the new target can be obtained.
The target number display module and the target track display module can display the target number and the target track in one scanning period, and the OSPA error estimation display module displays the deviation degree between the target estimation value and the real value for evaluating the performance of the algorithm, which is shown in FIG. 4 and FIG. 5.
Namely, the data display module displays the measurement information, the target state and the number estimation information at each moment, including the self amplitude and the global measurement amplitude of each target, the target tracking situation and the estimation deviation situation, and provides a basis for the base station to evaluate and track the weak and small targets.
The invention also provides a weak and small target joint detection and tracking method based on the random finite set, which comprises the following steps: firstly, an infrared measurement image is generated according to a selected scene, an optimal measurement domain is extracted to serve as a measurement set of a target, and then state estimation and tracking of small and weak targets are achieved through the delta-GLMB-TBD algorithm based on the infrared image. The method specifically comprises the following steps:
1) measuring and preprocessing an infrared image: the method is used for scanning in real time by using an infrared sensor installed by an unmanned aerial vehicle based on the current environment, acquiring infrared measurement images of the ground environment and the target, exchanging sensing information with a remote center through a data link, then suppressing background noise of the infrared measurement images of the ground environment and the target by using the gray level characteristics of the infrared images, and filtering ineffective measurement to obtain a target measurement domain. The specific process is as follows:
the intensity of the infrared image preprocessing based on the gray level characteristics of the infrared weak and small target is relatively gentle relative to the infrared background area, the intensity of the resolution unit where the infrared target is located is relatively high, the scattering characteristic meets certain exponential characteristics, the intensity of the scattering characteristic decays along with the distance index in the target neighborhood direction, therefore, the measurement likelihood function can be calculated by calculating the distance gradient between the resolution unit of the candidate measurement area and the surrounding neighborhood resolution units, the infrared weak target and the background are distinguished according to the distance gradient and the measurement likelihood function, and the target measurement area at the current moment is determined. Specifically, an infrared measurement image at the current moment is acquired from an infrared sensor, the image is represented as a series of gray values in a time-space domain, the measurement image is preprocessed by using a constant false alarm rate of a low threshold to inhibit background noise, then difference calculation is carried out on the measurement gray level images at the current moment and the previous moment, since the clutter change degree is possibly very small in a very short time and the change degree possibility caused by target motion is relatively high, the target signal intensity can be enhanced through measurement difference, then according to the exponential characteristic of the infrared target scattering intensity, namely, in the central area of the target, the target and the background have high contrast and are represented as regional singularity, the distance attenuation characteristic is presented around the target, the measurement domain in a certain range around the target is taken as a candidate measurement domain, then the measurement at different distances is extracted by using a median filtering method and corresponding distance gradient values are calculated, finally, a target measurement domain is obtained by setting a proper threshold value;
the background suppression of the infrared image and the extraction of the target measurement unit are completed through the steps.
2) And (3) predicting the multi-target state: the method is used for calculating a target measurement domain according to a Bayesian rule and by utilizing a Markov transfer matrix to obtain a prediction state and a target prediction track label of a target, and performing Gaussian sampling on the prediction target to obtain a particle set corresponding to each prediction target. The method specifically comprises the following steps:
the perception information source is an infrared image obtained by an infrared detector; on the target state and track representation level, realizing the unified representation of the target by using a random finite set theory;
calculating an associated cost function of current moment measurement and a target by utilizing the preprocessed infrared measurement image according to a target measurement domain obtained in the infrared image measurement and preprocessing steps, filtering the measurement with a larger cost function value in a measurement set and obtaining the measurement which is not associated with the target, then adopting a self-adaptive new-generation target algorithm to generate new particles in the unassociated measurement regions, resampling according to the weight occupied by the likelihood function of the particles, taking the region where the resampled particles are located as the current moment new-generation target measurement region and generating a new-generation target track table;
and calculating the prediction state of each corresponding particle of each track in the updated track table at the last moment by using a shortest path algorithm and a state transition equation, normalizing the weight to generate a survival target track table, and finally forming a target prediction track table together with the new target.
The specific process of the self-adaptive new target algorithm when the new target state is unknown is as follows:
measuring the new target possibly only if no measurement associated with any existing target is generated in the current moment measurement set, and determining the new target measurement domain by calculating an associated cost function of the candidate measurement domain and the target;
the associated cost function is defined as:
wherein:represents the measurement at the resolution cell (i, j) at time k,representing target statesThe weight of (a) is determined,representing target statesRelative to the measurementThe likelihood function of (2).
The measurement likelihood function is defined as:
wherein:representing a k time resolution cell(i,j)The scattering intensity of the target, sigma, is the standard deviation of the measured noise, and the rest parameters are defined as the above formula parameters.
The measurement with the larger correlation cost function value is considered to be related to the current target set, so that the correlation measurements are removed from the current concentrated measurement set to obtain a measurement set Z which is not related to the targetk,φ;
In unassociated measurement set Zk,φNewly generating target particles in the region, then calculating a likelihood function associated with the particles and measurement, and resampling to obtain a region where the particles with a larger weight are, namely, the region is considered as a k-moment newly generated target measurement region;
for theGenerating target particles in the measurement region, wherein the weight of each particle is defined as:
wherein:respectively represent the target statesThe set of resolution elements in the x-direction and y-direction,is in a target stateRelative to the measurementThe likelihood function of (2).
Normalization of particle weights:a set of weighted particles corresponding to each unassociated measurement can be obtained:the particle weight reflects the possibility that the particle state is a new target state, and finally resampling is adopted to retain the particles with large weight, and the region where the resampled particles are located is regarded as a new target measurement region at the k moment.
After the multi-target state and track are subjected to filtering estimation, the method further comprises the following steps:
pruning and merging the updated multi-target track list according to the assumed track and the weight, and optimizing the updated track list to further improve the estimation precision and reduce the calculation complexity of the algorithm
3) Updating the multi-target state: on the basis of obtaining the set of predicted target particles,
calculating a likelihood function of the predicted target particles in the particle set corresponding to each predicted target by using the target amplitude information, updating the particle weight to obtain a state updating value of the corresponding predicted target, then establishing a correlation hypothesis cost matrix of the target and the measurement, calculating m optimal distribution hypotheses of the target and the measurement correlation cost matrix through a mutty algorithm, and finally extracting a maximum weight track hypothesis to obtain a track estimation value of the target to realize the tracking of multiple targets. The specific process is as follows:
according to the infrared measurement data at the current moment, the likelihood function and the weight value of each target prediction particle in the track list are calculated, the updating weight of the particles when each target in the track list is associated with different measurements is obtained, and the sum of the non-normalized particle weights and the associated cost function of the target and the measurements are calculated. And finally, calculating an optimal distribution hypothesis through a mutty algorithm, and taking a target track value and a potential distribution value corresponding to the track hypothesis with the maximum weight as the target state and number estimation.
The weak and small target joint detection and tracking method based on the random finite set comprises the following specific steps of:
assume that the target measurement at time k-1 is Zk-1Target measurement at time k is ZkFirst, a first order measurement differential is calculated, expressed as:
wherein:respectively representing a measured value at the time k, a measured value at the time k-1 and a measured difference at the time k, (i, j) represents a resolution unit at the ith row and the jth column, and nx and my respectively represent the maximum row number and the maximum column number of the resolution unit.
The variation degree of the amplitude of the resolution unit in the images at two adjacent moments can be obtained through the measured first-order difference, and the signal intensity of the target can be enhanced by using the measured difference;
the method for restraining the background noise by the constant false alarm rate directly utilizes the image gray mean value and variance and the false alarm probability to determine the segmentation threshold of the infrared image so as to reduce the number of candidate target measurement areas and improve the operation speed of the algorithm, wherein the threshold is expressed as follows:
wherein: μ denotes a mean value of the image, σ denotes a standard deviation of the image, and k denotes a constant.
After background noise is suppressed by a constant false alarm rate method, a possible measurement area at the current moment is extracted by a distance gradient method based on gray level characteristics, and the scattering intensity of an infrared target is defined as follows:
wherein:representing the target metrology value at the resolution cell (i, j) at time k,indicating the target state at time kThe infrared scattering intensity at the resolving element (i, j),representing the measured noise value at the resolution cell (I, j) at time k, IkRepresenting target intensity, Σ being a blur parameter, (x)k,yk) Indicating the target position, (i Δx,jΔy) Indicating the position of the measurement unit.
From the above equation, when l targets appear in the monitoring area at the time k, the intensity contribution of the monitoring area to the measured value is limited to the cell where the monitoring area is located and the resolution cell whose neighborhood distance is less than p; generally, the intensity of an infrared image background area is relatively gentle, the gray level difference values of surrounding pixel points are smaller in performance at different distances compared with that of an infrared target, the distance gradient between the infrared target and the surrounding pixels at a corresponding central resolution unit pixel meets the exponential attenuation characteristic, namely, the infrared scattering amplitude of the infrared target is exponentially weakened along with the increase of the distance, and the infrared target presents a symmetrical characteristic; the distance gradient between the central pixel of the background clutter and the surrounding background is usually higher only in a certain direction and has stronger spatial coherence, so that the infrared weak target and the background noise can be distinguished by utilizing the distance gradient characteristic of the target and the surrounding neighborhood background, and a possible target measurement area at the current moment is extracted;
finally, newly generating target particles in the candidate target measurement area, and extracting a resolution unit where the particles with a larger weight are located as a target measurement domain combination at the moment k by calculating a likelihood function of the particles and performing resampling;
as shown in fig. 7, the specific process of the present invention includes the following steps:
step 1: and opening an application program of the demonstration system, and entering an infrared small and weak target joint detection and tracking system starting interface.
Step 2: clicking the upper left corner of the page to select a scene, and selecting any scene.
And step 3: and clicking the next step, and generating a target real motion track and background noise distribution for the selected scene by combining the table 1.
And 4, step 4: and clicking a display button in a menu bar to select the target motion track, so that the real motion track of the target can be displayed.
And 5, clicking an operation button, starting the system to detect and track the weak and small targets, and synchronously displaying the infrared measurement image at each moment and extracting the measurement area measurement.
Step 6: and clicking a display button in a menu bar, selecting a target tracking track, and displaying the tracking condition on a two-dimensional plane relative to the real motion track of the target. A tracking effect evaluation is selected and a target tracking effect graph with OSPA evaluation criteria may be displayed.
Claims (10)
1. Weak and small target joint detection and tracking system based on random finite set is characterized by comprising:
the infrared image measurement module is used for scanning in real time by using an infrared sensor to obtain infrared measurement images of the ground environment and the target, then inhibiting background noise of the infrared measurement images of the ground environment and the target by using the gray level characteristics of the infrared images, and filtering ineffective measurement to obtain a target measurement domain;
the target prediction module is used for calculating a target measurement domain according to a Bayesian rule and by utilizing a Markov transfer matrix to obtain a prediction state and a target prediction track label of a target, and carrying out Gaussian sampling on the prediction target according to the prediction state and the target prediction track label of the target to obtain a particle set corresponding to each prediction target;
the target updating module is used for calculating the likelihood function of the predicted target particles in the particle set corresponding to each predicted target by using the amplitude information corresponding to different positions in the target measurement domain, and calculating the updating state of the target and the target updating track label according to the predicted target particle likelihood function and the normalized particle weight; and establishing an assumed cost function associated with the target and the measurement, calculating m optimal distribution hypotheses according to the assumed cost function associated with the target and the measurement, and finally selecting a hypothesis corresponding to the maximum weight according to the assumed weight to obtain a target state and a track estimation value at the current moment so as to realize the tracking of multiple targets.
2. The system for jointly detecting and tracking weak and small targets based on random finite sets as claimed in claim 1, further comprising a data display module connected to the target update module, wherein the data display module is used for displaying the estimated values including the measured data, the real moving trajectory of the target, the target state and number, and the estimated error curves of the number and trajectory of the target.
3. The system for jointly detecting and tracking weak and small targets based on random finite sets according to claim 1, wherein the infrared image measurement module comprises a constant false alarm rate image segmentation module and an infrared target gray characteristic measurement extraction module;
the constant false alarm rate image segmentation module is used for carrying out background and clutter suppression on the infrared measurement images of the ground environment and the target to obtain a candidate measurement domain;
and the infrared target gray characteristic measurement and extraction module is used for extracting a target measurement domain from the candidate measurement domain.
4. The system for jointly detecting and tracking weak and small random finite set-based targets of claim 1, wherein the target prediction module comprises a fresh target prediction module and a survival target prediction module;
the new target prediction module is used for generating a new target track table according to the target measurement domain; when the new target information is known, a target measurement domain is used as a measurement set to participate in updating the target track table, when the new target information is unknown, a self-adaptive new target algorithm is adopted to measure an area which is not associated with a target in the target measurement domain, new track particles are generated, a likelihood function corresponding to the particles is calculated, the new target measurement area is determined through resampling of the new particles, and the new target track table is initialized;
and the survival target prediction module is used for corresponding the updated target at the last moment to the sampled particles, calculating the survival state and the weight of the particles according to a Bayesian rule and a state transition equation, forming a target prediction set with the new target, and generating a predicted track table, wherein the predicted track table comprises a target state and a target track label.
5. The system for jointly detecting and tracking weak and small targets based on random finite sets according to claim 1, wherein the target updating module comprises a target state updating module and a target track updating module;
the target state updating module is used for calculating the updating state and the normalization weight of each target corresponding particle in the predicted track table by using the predicted state of the target according to the ground environment of the current moment and the infrared measurement diagram of the target by adopting a particle filtering method, and taking the updating state and the normalization weight of each target corresponding particle in the predicted track table as the target state estimation value in the updated track table;
and the target track updating module is used for pairing the target prediction state and different positions in the target measurement domain at the current moment to obtain corresponding track hypotheses, then calculating the cost value of each track hypothesis, calculating m optimal distribution track hypotheses aiming at the track hypothesis cost by using a mutty algorithm, and finally extracting the maximum weight track hypothesis to obtain the track estimation value of the target.
6. The weak and small target joint detection and tracking method based on the random finite set is characterized by comprising the following steps of:
1) the method comprises the steps of utilizing an infrared sensor to conduct real-time scanning to obtain infrared measurement images of a ground environment and a target, then utilizing the gray level characteristics of the infrared images to restrain background noise of the infrared measurement images of the ground environment and the target, and then filtering ineffective measurement to obtain a target measurement domain;
2) calculating a target measurement domain according to Bayesian criteria and by using a Markov transfer matrix to obtain a predicted state and a target predicted track label of a target, and performing Gaussian sampling on the predicted target according to the predicted state and the target predicted track label of the target to obtain a particle set corresponding to each predicted target;
3) calculating a likelihood function of a predicted target particle in a particle set corresponding to each predicted target by using amplitude information corresponding to different positions in a target measurement domain, and calculating an update state of the target and a target update track label according to the predicted target particle likelihood function and the normalized particle weight; and establishing an assumed cost function associated with the target and the measurement, calculating m optimal distribution hypotheses according to the assumed cost function associated with the target and the measurement, and finally selecting a hypothesis corresponding to the maximum weight according to the assumed weight to obtain a target state and a track estimation value at the current moment so as to realize the tracking of multiple targets.
7. The weak and small target joint detection and tracking method based on the random finite set as claimed in claim 6, wherein the specific process of using the infrared image gray scale feature to suppress the background noise of the infrared measurement image of the ground environment and the target and filtering the invalid measurement to obtain the target measurement domain is as follows:
calculating a measurement likelihood function by calculating the distance gradient between the resolution unit of the candidate measurement domain and the surrounding neighborhood resolution units, and distinguishing the infrared weak target from the background according to the distance gradient and the measurement likelihood function to determine the target measurement domain at the current moment.
8. The weak and small target joint detection and tracking method based on the random finite set as claimed in claim 6, wherein the specific process of step 2) is as follows:
generating a new target track table according to the target measurement domain; when the new target information is known, a target measurement domain is used as a measurement set to participate in updating the target track table, when the new target information is unknown, a self-adaptive new target algorithm is adopted to measure an area which is not associated with a target in the target measurement domain, new track particles are generated, a likelihood function corresponding to the particles is calculated, the new target measurement area is determined through resampling of the new particles, and the new target track table is initialized;
and calculating the prediction state of the corresponding particle of each track in the target track table at the last moment by using a shortest path algorithm and a state transition equation, normalizing the weight to generate a survival target track table, and finally forming a target prediction track table together with the new target so as to obtain a particle set corresponding to each prediction target.
9. The weak and small target joint detection and tracking method based on the random finite set as claimed in claim 6, wherein the specific process of step 3) is as follows: according to the infrared measurement data at the current moment, calculating a likelihood function and a weight value of each target prediction particle in a particle set corresponding to each prediction target to obtain an updated weight of the particles when each target is associated with different measurements in a track list, calculating the sum of non-normalized particle weights and an associated cost function of the target and the measurements, finally calculating an optimal distribution hypothesis through a mutty algorithm, and taking a target track value and a potential distribution value corresponding to the track hypothesis with the maximum weight as target states and number estimates to realize the tracking of multiple targets.
10. The random finite set-based weak and small target joint detection and tracking method according to claim 6, wherein in step 3), the cost function is assumed to be:
wherein:represents the measurement at the resolution cell (i, j) at time k,representing target statesThe weight of (a) is determined,representing target statesRelative to the measurementA likelihood function of (a);
the likelihood function is:
wherein:represents the scattering intensity of the target at the resolution cell (i, j) at time k, σ is the standard deviation of the measured noise,representing the infrared measurement amplitude at the resolution unit (i, j) at the moment k;
removing the associated measurement in the current measurement set to obtain a measurement set Z which is not associated with the targetk,φ;
In metrology sets Z not associated with the targetk,φNewly generating target particles in the region, calculating a likelihood function associated with the particles and measurement, and resampling to obtain a newly generated target measurement region at the moment k;
for theGenerating new target particles in the measurement region, wherein the weight of each particle is as follows:
wherein:respectively represent the target statesThe set of resolution elements in the x-direction and y-direction,is in a target stateRelative to the measurementA likelihood function of (a);
normalization of particle weights:obtaining a weighted set of particles corresponding to each unassociated measurement:the particle weight reflects the possibility that the particle state is a new target state, and finally resampling is adopted to retain the particles with large weight,and obtaining a region where the resampled particles are located, wherein the region is a new target measurement region at the k moment.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011086220.2A CN112215146B (en) | 2020-10-12 | 2020-10-12 | Weak and small target joint detection and tracking system and method based on random finite set |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011086220.2A CN112215146B (en) | 2020-10-12 | 2020-10-12 | Weak and small target joint detection and tracking system and method based on random finite set |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112215146A true CN112215146A (en) | 2021-01-12 |
CN112215146B CN112215146B (en) | 2022-04-22 |
Family
ID=74053622
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202011086220.2A Active CN112215146B (en) | 2020-10-12 | 2020-10-12 | Weak and small target joint detection and tracking system and method based on random finite set |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112215146B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114155425A (en) * | 2021-12-13 | 2022-03-08 | 中国科学院光电技术研究所 | Weak and small target detection method based on Gaussian Markov random field motion direction estimation |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103902819A (en) * | 2014-03-21 | 2014-07-02 | 哈尔滨工程大学 | Particle optimizing probability hypothesis density multi-target tracking method based on variation filtering |
CN104297748A (en) * | 2014-10-20 | 2015-01-21 | 西安电子科技大学 | Before-radar-target-detecting tracking method based on track enhancing |
CN106707272A (en) * | 2016-07-01 | 2017-05-24 | 电子科技大学 | Multi-target tracking method based on theory of random sets |
CN107680120A (en) * | 2017-09-05 | 2018-02-09 | 南京理工大学 | Tracking Method of IR Small Target based on rarefaction representation and transfer confined-particle filtering |
WO2018119912A1 (en) * | 2016-12-29 | 2018-07-05 | 深圳大学 | Target tracking method and device based on parallel fuzzy gaussian and particle filter |
CN111539510A (en) * | 2020-04-16 | 2020-08-14 | 中国人民解放军海军航空大学 | MIMO radar target tracking method based on parallel segmentation particle filtering |
CN111562571A (en) * | 2020-05-28 | 2020-08-21 | 江南大学 | Maneuvering multi-target tracking and track maintaining method for unknown new-born strength |
CN111736145A (en) * | 2020-06-28 | 2020-10-02 | 电子科技大学 | Multi-maneuvering-target Doppler radar tracking method based on Gaussian mixed probability hypothesis density filtering |
-
2020
- 2020-10-12 CN CN202011086220.2A patent/CN112215146B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103902819A (en) * | 2014-03-21 | 2014-07-02 | 哈尔滨工程大学 | Particle optimizing probability hypothesis density multi-target tracking method based on variation filtering |
CN104297748A (en) * | 2014-10-20 | 2015-01-21 | 西安电子科技大学 | Before-radar-target-detecting tracking method based on track enhancing |
CN106707272A (en) * | 2016-07-01 | 2017-05-24 | 电子科技大学 | Multi-target tracking method based on theory of random sets |
WO2018119912A1 (en) * | 2016-12-29 | 2018-07-05 | 深圳大学 | Target tracking method and device based on parallel fuzzy gaussian and particle filter |
CN107680120A (en) * | 2017-09-05 | 2018-02-09 | 南京理工大学 | Tracking Method of IR Small Target based on rarefaction representation and transfer confined-particle filtering |
CN111539510A (en) * | 2020-04-16 | 2020-08-14 | 中国人民解放军海军航空大学 | MIMO radar target tracking method based on parallel segmentation particle filtering |
CN111562571A (en) * | 2020-05-28 | 2020-08-21 | 江南大学 | Maneuvering multi-target tracking and track maintaining method for unknown new-born strength |
CN111736145A (en) * | 2020-06-28 | 2020-10-02 | 电子科技大学 | Multi-maneuvering-target Doppler radar tracking method based on Gaussian mixed probability hypothesis density filtering |
Non-Patent Citations (6)
Title |
---|
CHENMING LI等: "Detection and Tracking of Moving Targets for Thermal Infrared Video Sequences", 《SENSORS》 * |
LIMING HOU 等: "Extension of Nonlinear δ-generalized labeled multi-Bernoulli Filter in Multi-Target Tracking", 《CAC》 * |
RONALD MAHLER: "A brief survey of advances in random-set fusion", 《ICCAIS》 * |
李云湘: "随机有限集多目标跟踪技术研究", 《中国博士学位论文全文数据库》 * |
程慧等: "基于箱粒子PHD滤波的多目标视频跟踪方法", 《计算机辅助设计与图形学学报》 * |
连峰: "基于随机有限集的多目标跟踪方法研究", 《中国博士学位论文全文数据库》 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114155425A (en) * | 2021-12-13 | 2022-03-08 | 中国科学院光电技术研究所 | Weak and small target detection method based on Gaussian Markov random field motion direction estimation |
CN114155425B (en) * | 2021-12-13 | 2023-04-07 | 中国科学院光电技术研究所 | Weak and small target detection method based on Gaussian Markov random field motion direction estimation |
Also Published As
Publication number | Publication date |
---|---|
CN112215146B (en) | 2022-04-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109816641B (en) | Multi-scale morphological fusion-based weighted local entropy infrared small target detection method | |
CN101930072B (en) | Multi-feature fusion based infrared small dim moving target track starting method | |
CN104766320B (en) | Many Bernoulli Jacob under thresholding is measured filter Faint target detection and tracking | |
CN104899866B (en) | A kind of intelligentized infrared small target detection method | |
CN107703496B (en) | Interactive multimode Bernoulli filtering maneuvering weak target tracking-before-detection method | |
CN104834915B (en) | A kind of small infrared target detection method under complicated skies background | |
CN106548488B (en) | A kind of foreground detection method based on background model and inter-frame difference | |
CN109633633B (en) | Life signal enhancement method based on segmented classification enhancement processing | |
CN111311644B (en) | Moving target detection method based on video SAR | |
CN104766079B (en) | A kind of remote method for detecting infrared puniness target | |
CN111027496A (en) | Infrared dim target detection method based on space-time joint local contrast | |
Hu et al. | Face Detection based on SSD and CamShift | |
CN112184749B (en) | Moving target tracking method based on video SAR cross-domain combination | |
CN111323756B (en) | Marine radar target detection method and device based on deep learning | |
CN108334885A (en) | A kind of video satellite image space object detection method | |
CN103745216A (en) | Radar image clutter suppression method based on airspace feature | |
CN110400294A (en) | A kind of Infrared Target Detection system and detection method | |
CN112215146B (en) | Weak and small target joint detection and tracking system and method based on random finite set | |
CN114373130A (en) | Space-based infrared dark and weak moving target detection method | |
CN114549642B (en) | Low-contrast infrared dim target detection method | |
Liu et al. | Space target extraction and detection for wide-field surveillance | |
CN116385866B (en) | SAR image-based railway line color steel house change detection method and device | |
CN110969605A (en) | Method and system for detecting moving small target based on space-time saliency map | |
CN116862832A (en) | Three-dimensional live-action model-based operator positioning method | |
CN115797769A (en) | Infrared weak and small target detection method based on infrared characteristics and space-time significance |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |