CN114882068B - Multi-target tracking method, device, computer equipment and storage medium - Google Patents
Multi-target tracking method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114882068B CN114882068B CN202210434733.0A CN202210434733A CN114882068B CN 114882068 B CN114882068 B CN 114882068B CN 202210434733 A CN202210434733 A CN 202210434733A CN 114882068 B CN114882068 B CN 114882068B
- Authority
- CN
- China
- Prior art keywords
- target
- tracking
- track
- track set
- lost
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 74
- 238000001514 detection method Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 8
- 230000002596 correlated effect Effects 0.000 abstract 2
- 238000012549 training Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000000712 assembly Effects 0.000 description 1
- 238000000429 assembly Methods 0.000 description 1
- 230000000386 athletic effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of target tracking, and discloses a multi-target tracking method, a multi-target tracking device, computer equipment and a storage medium, wherein the multi-target tracking method comprises the following steps: acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises a non-lost track set and a lost track set; acquiring a first target set in a first tracking image based on a target detection algorithm; correlating the first target set with the non-lost track set through a first correlation algorithm, and obtaining a second target set and a second track set which are not correlated; correlating the second target set with the lost track set through a second correlation algorithm, and obtaining a third target set and a third track set which are not correlated; and associating the third target set with the second track set and the third track set through a third association algorithm. According to the application, the tracking target is associated with the target tracking track through the hierarchical association algorithm, so that the performance of multi-target tracking is effectively improved.
Description
Technical Field
The application relates to the technical field of target tracking, in particular to a multi-target tracking method, a multi-target tracking device, computer equipment and a storage medium.
Background
The multi-target tracking technology is an important basic technology in the field of computer vision, and aims to simultaneously track all tracking targets in a video and obtain a complete target tracking track. Multi-target tracking has very wide applications in real life, such as demographics, video analysis, motion recognition, abnormal behavior detection, athletic event analysis, biological research, human-machine interaction, robotic navigation, unmanned, etc.
Therefore, for the multi-target tracking technology, the improvement of the performance of the multi-target tracking technology has important significance in the application scene.
Disclosure of Invention
Based on the technical problems, the application provides a multi-target tracking method, a multi-target tracking device, a multi-target tracking computer device and a multi-target tracking storage medium, wherein tracking targets are associated with target tracking tracks through a hierarchical association algorithm, so that the performance of multi-target tracking is effectively improved.
In order to solve the technical problems, the application adopts the following technical scheme:
a multi-target tracking method, comprising:
acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises a non-lost track set and a lost track set;
inputting the first tracking image into a target detection algorithm, and acquiring a first target set in the first tracking image;
associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and correlating the tracking targets in the third target set with the target tracking tracks in the second track set and the third track set through a third correlation algorithm.
A multi-target tracking device, comprising:
the data acquisition module is used for acquiring a first tracking image and a first track set of the current frame, wherein the first track set comprises a non-lost track set and a lost track set;
the target acquisition module is used for inputting the first tracking image into a target detection algorithm and acquiring a first target set in the first tracking image;
the first association module is used for associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
the second association module is used for associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and the third association module is used for associating the tracking target in the third target set with the target tracking track in the second track set and the third track set through a third association algorithm.
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the multi-target tracking method described above.
A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the multi-target tracking method described above.
Compared with the prior art, the application has the beneficial effects that:
according to the method, the device, the computer equipment and the storage medium, different re-identification networks are selected for feature extraction and similarity calculation according to the condition that adjacent frame image pairs are associated and non-adjacent frame image pairs are associated when the tracking target is associated with the target tracking track, the tracking target is associated with the target tracking track in a layering association mode, the advantages of different association algorithms are fully exerted according to different tracking states, so that a better multi-target tracking effect is obtained, and multi-target tracking performance can be effectively improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. Wherein:
FIG. 1 is a flow chart of a multi-objective tracking method.
Fig. 2 is a flow chart of a first association algorithm.
Fig. 3 is a flow chart of a second association algorithm.
FIG. 4 is a flow chart of a method for classifying an missed set of tracks and a missing set of tracks.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. It will be apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without the need for inventive faculty, are within the scope of the present disclosure, based on the described embodiments of the present disclosure.
It should be appreciated that "system," "apparatus," "unit," and/or "module" as used in this specification is a method for distinguishing between different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
As used in this specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
A flowchart is used in this specification to describe the operations performed by the system according to embodiments of the present specification. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.
Referring to fig. 1, in some embodiments, a multi-target tracking method includes:
s101, acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an undelivered track set and a lost track set;
specifically, according to the prior art of target tracking, for the first track set, the target tracking tracks included in the first track set are formed by associating a plurality of tracking targets, the tracking targets in the target tracking tracks are arranged in time sequence, and the tracking target at the tail end of the track is the latest associated tracking target in the track.
Specifically, the target tracking track in the non-lost track set means that the target tracking track end tracking target is the tracking target in the last frame of tracking image, which indicates that the target tracking track is not lost in the last frame of tracking target; correspondingly, the loss of the target tracking track in the track set means that the target tracking track end tracking target is not the tracking target in the previous frame tracking image, and the loss of the target tracking track in the previous frame tracking target is indicated.
S102, inputting a first tracking image into a target detection algorithm, and acquiring a first target set in the first tracking image;
specifically, the first tracking image is input into a target detection algorithm, a detection response set is obtained through the target detection algorithm, the first tracking target is cut based on the coordinate position of each detection response in the detection response set, and image blocks obtained through summarizing and cutting form the first target set.
Specifically, the target detection algorithm may employ YOLO, fast R-CNN, R-FCN, or SSD.
S103, associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
s104, associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
s105, associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
In this embodiment, the tracking target and the target tracking track are hierarchically associated by three association algorithms, so that the advantages of different association algorithms are fully exerted according to different tracking states, a better multi-target tracking effect is obtained, and the multi-target tracking performance can be effectively improved.
Preferably, after associating the tracking target in the third target set with the target tracking track in the second track set and the third track set by using a third association algorithm, the method further includes: obtaining a fourth target set which is not successfully associated by the third association algorithm; initializing the tracking target in the fourth target set as a new target tracking track and storing the new target tracking track in the first track set.
And if the fourth target set is a tracking target which is not successfully associated by the hierarchical association algorithm, indicating that the tracking target in the fourth target set is a new target which does not appear before. Thus, the tracked objects in the fourth set of objects should be initialized to a new object tracking trajectory for the multi-object tracking operation of the next frame of tracked images.
Preferably, for the association operation, the association is to add the successfully paired tracking target to the corresponding target tracking track end so as to update the target tracking track.
Preferably, the third association algorithm comprises: and associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set based on a IoU association algorithm.
Referring to fig. 2, in some embodiments, the first association algorithm includes:
s201, inputting tracking targets in a first target set into a first network model, and acquiring first target characteristics of unmasked background information of the tracking targets;
specifically, the first network model is a Strong-Baseline network, and the network is a Baseline network with a BN layer and is mainly used for acquiring target features of unshielded background information.
S202, calculating a feature distance between a first target feature of a tracking target in a first target set and a first target feature of a tracking target at the tail end of a target tracking track in a non-lost track set to obtain a first feature distance;
specifically, the feature distance calculation adopts a Euclidean distance algorithm or a Manhattan distance algorithm, and the obtained first feature distance is a two-dimensional distance matrix.
And S203, correlating the first characteristic distances through a Hungary algorithm to obtain a successfully paired tracking target and a target tracking track.
In this embodiment, since the target tracking track end tracking target in the non-lost track set is the tracking target of the previous frame, the first association algorithm is specifically that the current frame tracking target is associated with the adjacent frame of the previous frame tracking target.
In the state of adjacent frame association, as the interference of background information is not large and even the adjacent frame association can be assisted, the target features which do not mask the background information are adopted for association at the moment, so that the performance of the multi-target tracking method is improved.
Referring to fig. 3, in some embodiments, the second association algorithm includes:
s301, inputting tracking targets in a second target set into a second network model to acquire second target features of shielding background information of the tracking targets;
specifically, the second network model is an FPR network, which is a re-recognition network based on a foreground information attention mechanism and pyramid reconstruction, and is mainly used for acquiring target features shielded from background information.
S302, calculating a characteristic distance between a second target characteristic of a tracking target in a second target set and a second target characteristic of a tracking target at the tail end of a target tracking track in a lost track set to obtain a second characteristic distance;
specifically, the feature distance calculation adopts a Euclidean distance algorithm or a Manhattan distance algorithm, and the obtained second feature distance is a two-dimensional distance matrix.
And S303, correlating the second characteristic distance through a Hungary algorithm to obtain a successfully paired tracking target and a target tracking track.
In this embodiment, since the target tracking track end tracking target in the lost track set is not the tracking target of the previous frame, the second association algorithm is specifically that the current frame tracking target is associated with a non-adjacent frame of the non-previous frame tracking target.
In the state of non-adjacent frame association, the interference of background information is larger, so that the target features shielding the background information are adopted for association, and the performance of the multi-target tracking method is improved by eliminating the background interference.
In some embodiments, referring to fig. 4, the classification method of the missing track set and the missing track set includes:
s401, obtaining a fifth target set on a second tracking image of the previous frame;
specifically, the second tracking image of which the fifth target set is the last frame stores the result when the multi-target tracking operation is performed. Correspondingly, the first target set obtained by the first tracking image of the current frame is also stored so as to be used for classifying the track set when the next frame tracking image performs multi-target operation.
S402, acquiring a target image of a target tracked at the tail end of a target tracking track in a first track set;
s403, comparing the target image with the tracking target in the fifth target set, and judging whether the target image exists in the fifth target set;
specifically, for comparison of the target image and the tracking target, an image similarity algorithm is adopted to calculate the similarity of the target image and the tracking target, if the similarity meets a preset threshold value, the target image is similar to the tracking target, the target image exists in the fifth target set, and otherwise, the target image does not exist in the fifth target set.
S404, if the fifth target set has the target image, classifying the target tracking track into a non-lost track set;
and S405, if the fifth target set does not have the target image, classifying the target tracking track into a lost track set.
In this embodiment, the non-lost track set and the lost track set are obtained by screening and classifying the target tracking tracks in the first track set and used for the association operation of the subsequent target and the track.
In the above embodiment, the basic process of the multi-target tracking method of the present application is as follows:
firstly, the existing re-recognition network aims at enabling a network model to automatically ignore factors such as background and directly extract sign information of a target pedestrian body through a series of training skills, but due to the limitation of the existing training technology, a part of background information is often extracted from the network model, and particularly when a shelter shields the pedestrian body, the shelter interference is strong, and recognition errors are easy to cause.
In view of this, some researchers propose to mask background information on a re-recognition network structure, such a network generally distinguishes the background and the human body by means of human body semantic segmentation, is not easily interfered by a shielding object, but may mask some information (such as a knapsack carried by a pedestrian) beneficial to recognition, and is interfered by inaccurate semantic segmentation.
The applicant finds that when the re-identification network is applied to the multi-target tracking method in the actual multi-target tracking operation process, the background information of the tracking target is not greatly interfered when adjacent frames are associated, and even the adjacent frames can be assisted to be associated, so that the method is more suitable for the re-identification network without shielding the background information. The background information is greatly influenced when the non-adjacent frames are associated, and the method is more suitable for the re-identification network for shielding the background information. According to the research result, the application provides a multi-target tracking method for switching re-identification networks along with tracking states, which aims at characteristic extraction and similarity calculation of re-identification networks with different structures selected by adjacent frame image pairs and non-adjacent frame image pairs, and then uses a special hierarchical association algorithm for association, so that the multi-target tracking performance can be improved effectively
Firstly, training two different re-identification network models offline, wherein the first network model adopts a Strong-base network, and the second network model adopts an FPR network. The training methods of the two network models are the same, and specifically comprise:
acquiring a training set, preprocessing the training set and calibrating a real label;
inputting the training set into a network model to be trained to obtain an output result;
constructing a loss function based on the output result and the real label;
and carrying out iterative training on the network model based on the loss function to obtain the trained network model.
Secondly, the input video stream to be subjected to multi-target tracking is set as follows:
wherein,,representing the total frame number of the video stream, < > x->Is->The frame tracks the image.
Then, selecting a tracking image which is required to be tracked currently and inputting the tracking image into a target detection algorithm, and obtaining a detection response set through the target detection algorithm after inputting a first tracking image of a current frame:
wherein,,indicate->First>Individual detection response->Indicate->The number of detected responses in the frame tracking image.
And (3) taking each image block which is cut out from the current frame tracking image and used for detecting the response coordinate position as a first target set, sending the first target set into a re-recognition network model trained offline, and respectively obtaining corresponding re-recognition target characteristics:
wherein the first target feature of the unshielded background information is:
wherein,,indicate->First>First target feature of the individual tracking targets, +.>Indicate->The frame tracks the number of tracked objects in the image.
Wherein the second target feature of shielding the background information is:
wherein,,indicate->First>Second target feature of the tracking target +.>Indicate->The frame tracks the number of tracked objects in the image.
Thirdly, carrying out hierarchical association on the tracking target and the target tracking track:
for the first level of association, specifically include:
firstly, calculating a feature distance between a first target feature of a tracking target in a first target set and a first target feature of a tracking target at the tail end of a target tracking track in a non-lost track set to obtain a first feature distance;
wherein the first target feature of the target tracking tail end tracking target in the target tracking track set without losing is specifically that；
Wherein the first characteristic distance，/>Indicate->Track of strip target and +.>A first feature distance of each tracking target;
and then, correlating the first characteristic distance through a Hungary algorithm to obtain a successfully paired tracking target and a target tracking track. Once the tracking target is successfully matched with the target tracking track, the target tracking track is updated, and specifically, the tracking target is added to the tail end of the target tracking track;
and finally, storing the tracking targets which are not successfully paired in the first target set into the second target set, and storing the target tracking tracks which are not successfully paired in the missed track set into the second track set.
For the second level association, specifically including:
firstly, calculating a characteristic distance between a second target characteristic of a tracking target in a second target set and a second target characteristic of a tracking target at the tail end of a target tracking track in a lost track set to obtain a second characteristic distance;
wherein the first target feature of the target tracking tail end tracking target in the target tracking track set without losing is specifically that；
Wherein the first characteristic distance，/>Indicate->Track of strip target and +.>A second feature distance of the individual tracking targets;
and then, correlating the second characteristic distance through a Hungary algorithm to obtain the successfully paired tracking target and the target tracking track. Once the tracking target is successfully matched with the target tracking track, the target tracking track is updated, and specifically, the tracking target is added to the tail end of the target tracking track;
and finally, storing the tracking targets which are not successfully paired in the second target set into a third target set, and storing the target tracking tracks which are not successfully paired in the lost track set into the third track set.
For the third level association, specifically include:
and associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set based on a IoU association algorithm.
And finally, storing the tracking targets which are not matched in the third target set in the fourth target set.
Fourth, for the tracking target in the fourth target set, it is initialized to a new target tracking trajectory.
Fifthly, after the target tracking track is updated through the hierarchical association, the tracking task of the frame of tracking image is completed, the next frame of tracking image is input, the steps are executed again until all the tracking images in the video stream complete the multi-target tracking operation, and a final target tracking track is formed.
In some embodiments, a multi-target tracking apparatus is also disclosed, comprising:
the data acquisition module is used for acquiring a first tracking image and a first track set of the current frame, wherein the first track set comprises a non-lost track set and a lost track set;
the target acquisition module is used for inputting the first tracking image into a target detection algorithm and acquiring a first target set in the first tracking image;
the first association module is used for associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
the second association module is used for associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and the third association module is used for associating the tracking target in the third target set with the target tracking track in the second track set and the third track set through a third association algorithm.
In order to solve the technical problem, the application also discloses a computer device which is characterized by comprising a memory and a processor, wherein the memory stores a computer program, and the computer program when executed by the processor causes the processor to execute the steps of the multi-target tracking method.
The computer equipment can be computing equipment such as a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or D interface display memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like. Of course, the memory may also include both internal storage units of the computer device and external storage devices. In this embodiment, the memory is often used to store an operating system and various application software installed on the computer device, such as program codes of the multi-target tracking method. In addition, the memory may be used to temporarily store various types of data that have been output or are to be output.
The processor may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process data, for example, the program code for executing the multi-object tracking method.
In order to solve the technical problem, the application also discloses a computer readable storage medium which stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the multi-target tracking method.
Wherein the computer-readable storage medium stores an interface display program executable by at least one processor to cause the at least one processor to perform the steps of the multi-target tracking method as described above.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server or a network device, etc.) to perform the method according to the embodiments of the present application.
The above is an embodiment of the present application. The foregoing embodiments and the specific parameters in the embodiments are only for clarity of the verification process of the application, and are not intended to limit the scope of the application, which is defined by the claims, and all equivalent structural changes made by the application of the specification and drawings of the application are included in the scope of the application.
Claims (10)
1. A multi-target tracking method, comprising:
acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises a non-lost track set and a lost track set;
inputting the first tracking image into a target detection algorithm, and acquiring a first target set in the first tracking image;
associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
associating tracking targets in the third target set with target tracking tracks in the second track set and the third track set through a third association algorithm;
the target tracking track in the non-lost track set means that the target tracking track tracks that the target is not lost in the last frame; the loss of the target tracking track in the track set means that the target tracking track is lost in the last frame of tracking target;
the first association algorithm associates the current frame tracking target with the adjacent frame of the previous frame tracking target, and associates the current frame tracking target with the adjacent frame of the previous frame tracking target by adopting target features without shielding background information;
the second association algorithm associates the current frame tracking target with a non-adjacent frame of the non-previous frame tracking target, and associates the non-adjacent frames by adopting target features for shielding background information;
the third level of association is to associate the tracking target in the third target set with the target tracking tracks in the second track set and the third track set based on IoU association algorithm.
2. The multi-target tracking method according to claim 1, wherein after associating the tracking target in the third target set with the target tracking trajectories in the second track set and the third track set by a third association algorithm, further comprising:
obtaining a fourth target set which is not successfully associated by the third association algorithm;
initializing tracking targets in the fourth target set into new target tracking tracks, and storing the new target tracking tracks into the first track set.
3. The multi-target tracking method of claim 1, wherein:
the association is to add the successfully paired tracking target to the corresponding target tracking track tail end so as to update the target tracking track.
4. The multi-target tracking method of claim 1 wherein the first correlation algorithm comprises:
inputting the tracking targets in the first target set into a first network model, and acquiring first target characteristics of the unmasked background information of the tracking targets;
calculating a feature distance between a first target feature of a tracking target in the first target set and a first target feature of a target tracking track tail end tracking target in the non-lost track set to obtain a first feature distance;
and correlating the first characteristic distance through a Hungary algorithm to obtain a successfully matched tracking target and a target tracking track.
5. The multi-target tracking method of claim 1 wherein the second correlation algorithm comprises:
inputting the tracking targets in the second target set into a second network model to acquire second target features of the tracking targets for shielding background information;
calculating a characteristic distance between a second target characteristic of the tracking target in the second target set and a second target characteristic of the tracking target at the tail end of the target tracking track in the lost track set to obtain a second characteristic distance;
and correlating the second characteristic distance through a Hungary algorithm to obtain a successfully matched tracking target and a target tracking track.
6. The multi-target tracking method of claim 1 wherein the third correlation algorithm comprises:
and associating the tracking targets in the third target set with the target tracking tracks in the second track set and the third track set based on a IoU association algorithm.
7. The multi-target tracking method of claim 1, wherein the classification method of the non-missing track set and the missing track set comprises:
acquiring a fifth target set on a second tracking image of the previous frame;
acquiring a target image of a target tracked at the tail end of a target tracking track in the first track set;
comparing the target image with tracking targets in the fifth target set, and judging whether the target image exists in the fifth target set;
if the target image exists in the fifth target set, classifying the target tracking track into the non-lost track set;
and if the target image does not exist in the fifth target set, classifying the target tracking track into the lost track set.
8. A multi-target tracking apparatus, comprising:
the system comprises a data acquisition module, a first tracking module and a second tracking module, wherein the data acquisition module is used for acquiring a first tracking image of a current frame and a first track set, and the first track set comprises an undelivered track set and a lost track set;
the target acquisition module is used for inputting the first tracking image into a target detection algorithm and acquiring a first target set in the first tracking image;
the first association module is used for associating the tracking target in the first target set with the target tracking track in the non-lost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
the second association module is used for associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
the third association module is used for associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm;
the target tracking track in the non-lost track set means that the target tracking track tracks that the target is not lost in the last frame; the loss of the target tracking track in the track set means that the target tracking track is lost in the last frame of tracking target;
the first association algorithm associates the current frame tracking target with the adjacent frame of the previous frame tracking target, and associates the current frame tracking target with the adjacent frame of the previous frame tracking target by adopting target features without shielding background information;
the second association algorithm associates the current frame tracking target with a non-adjacent frame of the non-previous frame tracking target, and associates the non-adjacent frames by adopting target features for shielding background information;
the third level of association is to associate the tracking target in the third target set with the target tracking tracks in the second track set and the third track set based on IoU association algorithm.
9. A computer device, characterized by: comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the multi-target tracking method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized by: a computer program is stored which, when executed by a processor, causes the processor to perform the steps of the multi-target tracking method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210434733.0A CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210434733.0A CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114882068A CN114882068A (en) | 2022-08-09 |
| CN114882068B true CN114882068B (en) | 2023-09-01 |
Family
ID=82671845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210434733.0A Active CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114882068B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012112807A (en) * | 2010-11-25 | 2012-06-14 | Mitsubishi Electric Corp | Multi-target tracking device |
| CN103927763A (en) * | 2014-03-24 | 2014-07-16 | 河海大学 | Identification processing method for multi-target tracking tracks of image sequences |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN106600631A (en) * | 2016-11-30 | 2017-04-26 | 郑州金惠计算机***工程有限公司 | Multiple target tracking-based passenger flow statistics method |
| CN107193032A (en) * | 2017-03-31 | 2017-09-22 | 长光卫星技术有限公司 | Multiple mobile object based on satellite video quickly tracks speed-measuring method |
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN110472496A (en) * | 2019-07-08 | 2019-11-19 | 长安大学 | A kind of traffic video intelligent analysis method based on object detecting and tracking |
| CN110728697A (en) * | 2019-09-30 | 2020-01-24 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Infrared dim target detection tracking method based on convolutional neural network |
| CN111652910A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Target tracking algorithm based on object space relationship |
| CN111739053A (en) * | 2019-03-21 | 2020-10-02 | 四川大学 | Online multi-pedestrian detection tracking method under complex scene |
| CN112487934A (en) * | 2020-11-26 | 2021-03-12 | 电子科技大学 | Strong data association integrated real-time multi-target tracking method based on ReID (ReID) characteristics |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN113674328A (en) * | 2021-07-14 | 2021-11-19 | 南京邮电大学 | Multi-target vehicle tracking method |
| CN113706574A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Movement track determining method and device, electronic equipment and storage medium |
-
2022
- 2022-04-24 CN CN202210434733.0A patent/CN114882068B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012112807A (en) * | 2010-11-25 | 2012-06-14 | Mitsubishi Electric Corp | Multi-target tracking device |
| CN103927763A (en) * | 2014-03-24 | 2014-07-16 | 河海大学 | Identification processing method for multi-target tracking tracks of image sequences |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN106600631A (en) * | 2016-11-30 | 2017-04-26 | 郑州金惠计算机***工程有限公司 | Multiple target tracking-based passenger flow statistics method |
| CN107193032A (en) * | 2017-03-31 | 2017-09-22 | 长光卫星技术有限公司 | Multiple mobile object based on satellite video quickly tracks speed-measuring method |
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN111739053A (en) * | 2019-03-21 | 2020-10-02 | 四川大学 | Online multi-pedestrian detection tracking method under complex scene |
| CN110472496A (en) * | 2019-07-08 | 2019-11-19 | 长安大学 | A kind of traffic video intelligent analysis method based on object detecting and tracking |
| CN110728697A (en) * | 2019-09-30 | 2020-01-24 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Infrared dim target detection tracking method based on convolutional neural network |
| CN113706574A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Movement track determining method and device, electronic equipment and storage medium |
| CN111652910A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Target tracking algorithm based on object space relationship |
| CN112487934A (en) * | 2020-11-26 | 2021-03-12 | 电子科技大学 | Strong data association integrated real-time multi-target tracking method based on ReID (ReID) characteristics |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN113674328A (en) * | 2021-07-14 | 2021-11-19 | 南京邮电大学 | Multi-target vehicle tracking method |
Non-Patent Citations (1)
| Title |
|---|
| 牛通等.基于深度学习的分层关联多行人跟踪.《计算机工程与应用》.2021,第57卷(第8期),96-102. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114882068A (en) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108470332B (en) | Multi-target tracking method and device | |
| CN110197146B (en) | Face image analysis method based on deep learning, electronic device and storage medium | |
| CN107145862B (en) | Multi-feature matching multi-target tracking method based on Hough forest | |
| CN111640140A (en) | Target tracking method and device, electronic equipment and computer readable storage medium | |
| US11526698B2 (en) | Unified referring video object segmentation network | |
| Lin et al. | Integrating graph partitioning and matching for trajectory analysis in video surveillance | |
| CN109492576B (en) | Image recognition method and device and electronic equipment | |
| CN110765860A (en) | Tumble determination method, tumble determination device, computer apparatus, and storage medium | |
| CN110009662B (en) | Face tracking method and device, electronic equipment and computer readable storage medium | |
| CN112989962B (en) | Track generation method, track generation device, electronic equipment and storage medium | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN111696133B (en) | Real-time target tracking method and system | |
| CN112001932A (en) | Face recognition method and device, computer equipment and storage medium | |
| Dai et al. | Instance segmentation enabled hybrid data association and discriminative hashing for online multi-object tracking | |
| Liu et al. | Hand Gesture Recognition Based on Single‐Shot Multibox Detector Deep Learning | |
| CN110363193B (en) | Vehicle weight recognition method, device, equipment and computer storage medium | |
| CN115984320A (en) | Multi-target tracking method based on long-short-term motion prediction and ordered data association | |
| Liang et al. | Human action segmentation and classification based on the Isomap algorithm | |
| CN114676756A (en) | Image recognition method, image recognition device and computer storage medium | |
| CN111814653B (en) | Method, device, equipment and storage medium for detecting abnormal behavior in video | |
| CN114882068B (en) | Multi-target tracking method, device, computer equipment and storage medium | |
| CN116227573B (en) | Segmentation model training method, image segmentation device and related media | |
| Veeraraghavan et al. | Learning dynamic event descriptions in image sequences | |
| Shi et al. | Saliency-based abnormal event detection in crowded scenes | |
| CN113240638B (en) | Target detection method, device and medium based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |