CN114882068A - Multi-target tracking method, device, computer equipment and storage medium - Google Patents
Multi-target tracking method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114882068A CN114882068A CN202210434733.0A CN202210434733A CN114882068A CN 114882068 A CN114882068 A CN 114882068A CN 202210434733 A CN202210434733 A CN 202210434733A CN 114882068 A CN114882068 A CN 114882068A
- Authority
- CN
- China
- Prior art keywords
- target
- tracking
- track
- algorithm
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of target tracking, and discloses a multi-target tracking method, a multi-target tracking device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an unreleased track set and a lost track set; acquiring a first target set in a first tracking image based on a target detection algorithm; associating the first target set with the undislost track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated; associating the second target set with the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated; and associating the third target set with the second track set and the third track set through a third association algorithm. The method and the device correlate the tracking target with the target tracking track through a hierarchical correlation algorithm, and effectively improve the performance of multi-target tracking.
Description
Technical Field
The application relates to the technical field of target tracking, in particular to a multi-target tracking method, a multi-target tracking device, computer equipment and a storage medium.
Background
The multi-target tracking technology is an important basic technology in the field of computer vision, and aims to simultaneously track all tracked targets in a video and obtain a complete target tracking track. Multi-target tracking has very wide application in real life, such as people counting, video analysis, motion recognition, abnormal behavior detection, sports event analysis, biological research, man-machine interaction, robot navigation, unmanned driving, etc.
Therefore, for the multi-target tracking technology, the performance improvement of the multi-target tracking technology has important significance in the application scene.
Content of application
Based on the technical problems, the application provides a multi-target tracking method, a multi-target tracking device, a computer device and a storage medium, wherein a tracking target and a target tracking track are associated through a hierarchical association algorithm, and the performance of multi-target tracking is effectively improved.
In order to solve the technical problems, the technical scheme adopted by the application is as follows:
a multi-target tracking method comprises the following steps:
acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an unreleased track set and a lost track set;
inputting the first tracking image into a target detection algorithm to obtain a first target set in the first tracking image;
associating the tracking target in the first target set with the target tracking track in the undiseased track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
A multi-target tracking apparatus, comprising:
the data acquisition module is used for acquiring a first tracking image and a first track set of the current frame, wherein the first track set comprises an unreleased track set and a lost track set;
the target acquisition module is used for inputting the first tracking image into a target detection algorithm and acquiring a first target set in the first tracking image;
the first association module is used for associating the tracking target in the first target set with the target tracking track in the missed track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
the second association module is used for associating the tracking targets in the second target set with the target tracking tracks in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and the third association module is used for associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the multi-target tracking method described above.
A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the multi-target tracking method described above.
Compared with the prior art, the beneficial effects of this application are:
the method, the device, the computer equipment and the storage medium select different re-recognition networks to perform feature extraction and similarity calculation aiming at the condition that when a tracking target is associated with a target tracking track, the adjacent frame image pair is associated with the non-adjacent frame image pair, associate the tracking target with the target tracking track in a hierarchical association mode, fully play the advantages of different association algorithms according to different tracking states to obtain a better multi-target tracking effect, and effectively improve the multi-target tracking performance.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. Wherein:
fig. 1 is a schematic flow chart of a multi-target tracking method.
Fig. 2 is a flowchart of a first correlation algorithm.
Fig. 3 is a flow chart of a second correlation algorithm.
Fig. 4 is a flowchart illustrating a classification method for an unreleased track set and a lost track set.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings of the embodiments of the present disclosure. It is to be understood that the described embodiments are only a few embodiments of the present disclosure, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the described embodiments of the disclosure without any inventive step, are within the scope of protection of the disclosure.
It should be understood that "system", "device", "unit" and/or "module" as used in this specification is a method for distinguishing different components, elements, parts or assemblies at different levels. However, other words may be substituted by other expressions if they accomplish the same purpose.
As used in this specification and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.
Flow charts are used in this description to illustrate operations performed by a system according to embodiments of the present description. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes.
Referring to fig. 1, in some embodiments, a multi-target tracking method includes:
s101, acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an unreleased track set and a lost track set;
specifically, as known in the art of target tracking, for a first track set, a target tracking track included in the first track set is formed by associating a plurality of tracking targets, the tracking targets in the target tracking track are arranged in time sequence, and a track end tracking target is a latest associated tracking target in the track.
Specifically, the target tracking track in the track set which is not lost means that the tracking target at the end of the target tracking track is the tracking target in the previous frame of tracking image, which indicates that the target tracking track is not lost in the previous frame of tracking target; correspondingly, when the target tracking track in the track set is lost, the fact that the target tracking track at the tail end of the target tracking track is not the tracking target in the previous frame of tracking image indicates that the target tracking track is lost in the previous frame of tracking target.
S102, inputting the first tracking image into a target detection algorithm, and acquiring a first target set in the first tracking image;
specifically, a first tracking image is input into a target detection algorithm, a detection response set is obtained through the target detection algorithm, the first tracking target is cut based on the coordinate position of each detection response in the detection response set, and image blocks obtained through cutting are collected to form a first target set.
Specifically, the target detection algorithm may employ YOLO, Faster R-CNN, R-FCN, or SSD.
S103, associating the tracking target in the first target set with the target tracking track in the undiseased track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
s104, associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and S105, associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
In the embodiment, the tracking target and the target tracking track are hierarchically associated through three association algorithms, the advantages of different association algorithms are fully exerted according to different tracking states to obtain a better multi-target tracking effect, and the multi-target tracking performance can be effectively improved.
Preferably, after the tracking targets in the third target set are associated with the target tracking tracks in the second track set and the third track set by the third association algorithm, the method further includes: obtaining a fourth target set which is not successfully associated by a third association algorithm; and initializing the tracking target in the fourth target set into a new target tracking track, and storing the new target tracking track into the first track set.
And if the fourth target set is a tracking target which is not successfully associated by the hierarchical association algorithm, the tracking target in the fourth target set is a new target which does not appear before. Therefore, the tracking targets in the fourth target set should be initialized to new target tracking tracks for the multi-target tracking operation of the next frame of tracking image.
Preferably, for the association operation, the association is to add the successfully paired tracking target to the corresponding target tracking track end to update the target tracking track.
Preferably, the third correlation algorithm comprises: and associating the tracking target in the third target set with the target tracking track in the second track set and the third track set based on IoU association algorithm.
Referring to fig. 2, in some embodiments, the first correlation algorithm includes:
s201, inputting a tracking target in a first target set into a first network model, and acquiring first target characteristics of the tracking target without shielding background information;
specifically, the first network model is a Strong-Baseline network, and the network is a Baseline network with a BN layer and is mainly used for obtaining target features of unmasked background information.
S202, calculating the characteristic distance between the first target characteristic of the tracking target in the first target set and the first target characteristic of the tracking target at the tail end of the target tracking track in the unremoved track set to obtain a first characteristic distance;
specifically, the characteristic distance calculation adopts an Euclidean distance algorithm or a Manhattan distance algorithm, and the obtained first characteristic distance is a two-dimensional distance matrix.
And S203, associating the first characteristic distance through the Hungarian algorithm to obtain a tracking target and a target tracking track which are successfully paired.
In this embodiment, since the target tracking track end tracking target in the missed track set is the tracking target of the previous frame, the first association algorithm specifically associates the current frame tracking target with the adjacent frame of the previous frame tracking target.
In the state of the correlation of the adjacent frames, because the interference of the background information is not large, even the correlation of the adjacent frames can be helped, the target characteristics of the unmasked background information are adopted for correlation at the moment, so that the performance of the multi-target tracking method is improved.
Referring to fig. 3, in some embodiments, the second correlation algorithm includes:
s301, inputting the tracked target in the second target set into a second network model, and acquiring a second target characteristic of the tracked target shielding background information;
specifically, the second network model is an FPR network, which is a re-recognition network based on a foreground information attention mechanism and pyramid reconstruction, and is mainly used for acquiring target features shielding background information.
S302, calculating the characteristic distance between the second target characteristic of the tracking target in the second target set and the second target characteristic of the tracking target at the tail end of the target tracking track in the lost track set to obtain a second characteristic distance;
specifically, the characteristic distance calculation adopts an Euclidean distance algorithm or a Manhattan distance algorithm, and the obtained second characteristic distance is a two-dimensional distance matrix.
And S303, associating the second characteristic distances through Hungarian algorithm to obtain successfully paired tracking targets and target tracking tracks.
In this embodiment, since the target tracking track end tracking target in the missing track set is not the tracking target of the previous frame, the second association algorithm specifically associates the current frame tracking target with the non-adjacent frame of the non-previous frame tracking target.
In the non-adjacent frame association state, because the interference of the background information is large, the target characteristics for shielding the background information are adopted for association, and the performance of the multi-target tracking method is improved by eliminating the background interference.
In some embodiments, referring to fig. 4, the method for classifying the set of non-lost tracks and the set of lost tracks includes:
s401, acquiring a fifth target set on the second tracking image of the previous frame;
specifically, the fifth target set is obtained by storing the second tracking image of the previous frame during the multi-target tracking operation. Correspondingly, the first target set obtained by the first tracking image of the current frame is also stored, so that the first target set is used for carrying out classification operation on the track set when the multi-target operation is carried out on the tracking image of the next frame.
S402, acquiring a target image of a target tracking target at the tail end of a target tracking track in a first track set;
s403, comparing the target image with the tracking target in the fifth target set, and judging whether the target image exists in the fifth target set;
specifically, for comparison between the target image and the tracking target, an image similarity algorithm is used for calculating the similarity between the target image and the tracking target, if the similarity meets a preset threshold, the target image is similar to the tracking target, a target image exists in the fifth target set, and otherwise, the target image does not exist in the fifth target set.
S404, if the target image exists in the fifth target set, classifying the target tracking track into a non-lost track set;
s405, if the target image does not exist in the fifth target set, classifying the target tracking track into a lost track set.
In this embodiment, by screening and classifying the target tracking tracks in the first track set, an unreleased track set and a lost track set are obtained for subsequent association operations between the target and the tracks.
By integrating the above embodiments, the basic process of the multi-target tracking method of the present application is as follows:
firstly, the existing re-recognition network aims to enable a network model to automatically ignore factors such as background and the like through a series of training skills and directly extract mark information belonging to a target pedestrian human body, but due to the limitation of the existing training technology, part of background information is often extracted from the network model, and particularly when a sheltering object shelters the body of a pedestrian, the sheltering object has strong interference, so that the recognition error is easily caused.
In view of this, some researchers propose to shield background information on a re-recognition network structure, and such a network generally distinguishes the background and the human body by human semantic segmentation, is not easily interfered by a shielding object, but may shield some information (such as a backpack carried by a pedestrian) beneficial to recognition, and may be interfered by inaccurate semantic segmentation.
The applicant finds that when the re-identification network is applied to the multi-target tracking method in the actual multi-target tracking operation process, the interference of the background information of the tracked target is not large when adjacent frames are associated, even the association of the adjacent frames can be helped, and the re-identification network is more suitable for the re-identification network without shielding the background information. And when non-adjacent frames are associated, the influence of the background information is particularly large, so that the method is more suitable for a re-identification network for shielding the background information. According to the research results, the multi-target tracking method for switching the re-identification network along with the tracking state is provided, the re-identification networks with different structures are selected for adjacent frame image pairs and non-adjacent frame image pairs to carry out feature extraction and similarity calculation, then, the association is carried out by utilizing a special hierarchical association algorithm, the advantages and the disadvantages can be brought forward, and the multi-target tracking performance can be effectively improved
Firstly, two different re-recognition network models are trained offline, the first network model adopts a Strong-Baseline network, and the second network model adopts an FPR network. The training methods of the two network models are the same, and specifically comprise the following steps:
acquiring a training set, and carrying out pretreatment and real label calibration on the training set;
inputting the training set into a network model to be trained to obtain an output result;
constructing a loss function based on the output result and the real label;
and carrying out iterative training on the network model based on the loss function to obtain the trained network model.
Secondly, the input video stream to be subjected to multi-target tracking is set as follows:
wherein F represents the total frame number of the video stream, I i An image is tracked for the ith frame.
Then, selecting a tracking image which needs to be tracked currently and inputting the tracking image into a target detection algorithm, and after inputting a first tracking image of a current frame, obtaining a detection response set through the target detection algorithm:
wherein the content of the first and second substances,
representing the M-th detection response, M, in the tracking image of the i-th frame i Indicating the number of detected responses in the tracking image of the ith frame.
And (3) taking an image block of each detection response coordinate position cut from the current frame tracking image as a first target set, sending the image block into a re-recognition network model trained offline, and respectively obtaining corresponding re-recognition target characteristics:
wherein, the first target feature of the unmasked background information is:
wherein f is i m First object feature, M, representing the mth tracked object in the ith frame of tracked image i Indicating the number of tracked targets in the tracking image of the ith frame.
Wherein, the second target feature for shielding background information is:
wherein the content of the first and second substances,
second object feature, M, representing the mth tracking object in the ith frame of tracking image i Indicating the number of tracked targets in the tracking image of the ith frame.
Thirdly, carrying out hierarchical association on the tracking target and the target tracking track:
for the first level of association, the method specifically includes:
firstly, performing characteristic distance calculation on first target characteristics of tracking targets in a first target set and first target characteristics of tracking targets at tail ends of target tracking tracks in an unreleased track set to obtain first characteristic distances;
wherein, the first target feature of the target tracking track end tracking target in the unremoved track set is specifically F t ；
Wherein the first characteristic distance cost 1 ，cost 1 (a, b) representing a first characteristic distance between the item a target tracking track and the item b target tracking track;
and then, correlating the first characteristic distance through Hungarian algorithm to obtain a successfully-paired tracking target and target tracking track. Once the tracking target and the target tracking track are successfully paired, updating the item target tracking track, specifically adding the tracking target to the tail end of the item target tracking track;
and finally, storing the tracking targets which are not successfully paired in the first target set into a second target set, and storing the tracking tracks of the targets which are not successfully paired in the missed track set into the second track set.
For the second level association, the method specifically comprises the following steps:
firstly, calculating the characteristic distance between the second target characteristic of the tracking target in the second target set and the second target characteristic of the tracking target at the tail end of the target tracking track in the lost track set to obtain a second characteristic distance;
the first target feature of the target tracking track end tracking target in the unremoved track set is specifically G t ；
Wherein the first characteristic distance cost 2 ，cost 2 (a, b) a second characteristic distance between the item a mark tracking track and the item b tracking target is represented;
and then, correlating the second characteristic distance through a Hungarian algorithm to obtain a tracking target and a target tracking track which are successfully paired. Once the tracking target and the target tracking track are successfully paired, updating the item target tracking track, specifically adding the tracking target to the tail end of the item target tracking track;
and finally, storing the tracking targets which are not successfully paired in the second target set into a third target set, and storing the tracking tracks of the targets which are not successfully paired in the lost track set into the third track set.
For the third-level association, the method specifically comprises the following steps:
and associating the tracking target in the third target set with the target tracking track in the second track set and the third track set based on IoU association algorithm.
And finally, storing the tracking targets which are not matched in the third target set into the fourth target set.
And fourthly, initializing the tracking targets in the fourth target set into new target tracking tracks.
And fifthly, after the target tracking track is updated through the hierarchical association, the tracking task of the frame of tracking image is completed, then the next frame of tracking image is input, and the steps are executed again until all the tracking images in the video stream are subjected to multi-target tracking operation, and a final target tracking track is formed.
In some embodiments, there is also disclosed a multi-target tracking apparatus, comprising:
the data acquisition module is used for acquiring a first tracking image and a first track set of the current frame, wherein the first track set comprises an unreleased track set and a lost track set;
the target acquisition module is used for inputting the first tracking image into a target detection algorithm and acquiring a first target set in the first tracking image;
the first association module is used for associating the tracking targets in the first target set with the target tracking tracks in the missed track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
the second association module is used for associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and the third association module is used for associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
In order to solve the technical problem, the present application further discloses a computer device, which is characterized by comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-target tracking method.
The computer device may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The computer equipment can carry out man-machine interaction with a user in a keyboard mode, a mouse mode, a remote controller mode, a touch panel mode or a voice control equipment mode.
The memory includes at least one type of readable storage medium including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or D interface display memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the computer device. Of course, the memory may also include both internal and external storage devices of the computer device. In this embodiment, the memory is used to store an operating system and various application software installed in the computer device, such as program codes of the multi-target tracking method. In addition, the memory may also be used to temporarily store various types of data that have been output or are to be output.
The processor may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to execute the program code stored in the memory or process data, for example, execute the program code of the multi-target tracking method.
In order to solve the above technical problem, the present application further discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the multi-target tracking method.
Wherein the computer readable storage medium stores an interface display program executable by at least one processor to cause the at least one processor to perform the steps of the multi-target tracking method as described above.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application or portions contributing to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (such as a ROM/RAM, a magnetic disk, and an optical disk), and includes several instructions for enabling a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in the embodiments of the present application.
The above is an embodiment of the present application. The embodiments and specific parameters in the embodiments are only used for clearly illustrating the verification process of the application and are not used for limiting the patent protection scope of the application, which is defined by the claims, and all the equivalent structural changes made by using the contents of the specification and the drawings of the application should be included in the protection scope of the application.
Claims (10)
1. The multi-target tracking method is characterized by comprising the following steps:
acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an unreleased track set and a lost track set;
inputting the first tracking image into a target detection algorithm to obtain a first target set in the first tracking image;
associating the tracking target in the first target set with the target tracking track in the undiseased track set through a first association algorithm, and obtaining a second target set and a second track set which are not associated;
associating the tracking target in the second target set with the target tracking track in the lost track set through a second association algorithm, and obtaining a third target set and a third track set which are not associated;
and associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set through a third association algorithm.
2. The multi-target tracking method according to claim 1, wherein after associating the tracking targets in the third target set with the target tracking tracks in the second track set and the third track set by a third association algorithm, the method further comprises:
obtaining a fourth target set which is not successfully associated by a third association algorithm;
initializing the tracking target in the fourth target set to be a new target tracking track, and storing the new target tracking track in the first track set.
3. The multi-target tracking method according to claim 1, characterized in that:
and the association is to add the successfully matched tracking target to the corresponding target tracking track tail end so as to update the target tracking track.
4. The multi-target tracking method of claim 1, wherein the first correlation algorithm comprises:
inputting the tracked target in the first target set into a first network model to obtain a first target characteristic of the tracked target without shielding background information;
calculating the characteristic distance between the first target characteristic of the tracking target in the first target set and the first target characteristic of the tracking target at the tail end of the target tracking track in the unremoved track set to obtain a first characteristic distance;
and associating the first characteristic distance through Hungarian algorithm to obtain a tracking target and a target tracking track which are successfully paired.
5. The multi-target tracking method of claim 1, wherein the second correlation algorithm comprises:
inputting the tracked targets in the second target set into a second network model to obtain second target characteristics of the tracked targets for shielding background information;
calculating the characteristic distance between the second target characteristic of the tracking target in the second target set and the second target characteristic of the tracking target at the tail end of the target tracking track in the lost track set to obtain a second characteristic distance;
and associating the second characteristic distance through a Hungarian algorithm to obtain a tracking target and a target tracking track which are successfully paired.
6. The multi-target tracking method of claim 1, wherein the third correlation algorithm comprises:
and associating the tracking target in the third target set with the target tracking tracks in the second track set and the third track set based on IoU association algorithm.
7. The multi-target tracking method according to claim 1, wherein the classification method of the set of missed tracks and the set of missed tracks comprises:
acquiring a fifth target set on the second tracking image of the previous frame;
acquiring a target image of a target tracking target at the tail end of a target tracking track in the first track set;
comparing the target image with the tracking target in the fifth target set, and judging whether the target image exists in the fifth target set;
if the target image exists in the fifth target set, classifying the target tracking track into the undiseased track set;
if the target image does not exist in the fifth target set, the target tracking track is classified into a lost track set.
8. A multi-target tracking apparatus, comprising:
the data acquisition module is used for acquiring a first tracking image and a first track set of a current frame, wherein the first track set comprises an unreleased track set and a lost track set;
a target obtaining module, configured to input the first tracking image into a target detection algorithm, and obtain a first target set in the first tracking image;
a first association module, configured to associate a tracking target in the first target set with a target tracking track in the missed track set through a first association algorithm, and obtain a second target set and a second track set that are not associated;
a second association module, configured to associate a tracking target in the second target set with a target tracking track in the lost track set through a second association algorithm, and obtain a third target set and a third track set that are not associated;
a third association module, configured to associate, through a third association algorithm, a tracked target in the third target set with a target tracked trajectory in the second trajectory set and the third trajectory set.
9. A computer device, characterized by: comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to carry out the steps of the multi-target tracking method according to any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: a computer program is stored which, when executed by a processor, causes the processor to carry out the steps of the multi-target tracking method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210434733.0A CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210434733.0A CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114882068A true CN114882068A (en) | 2022-08-09 |
| CN114882068B CN114882068B (en) | 2023-09-01 |
Family
ID=82671845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210434733.0A Active CN114882068B (en) | 2022-04-24 | 2022-04-24 | Multi-target tracking method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114882068B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012112807A (en) * | 2010-11-25 | 2012-06-14 | Mitsubishi Electric Corp | Multi-target tracking device |
| CN103927763A (en) * | 2014-03-24 | 2014-07-16 | 河海大学 | Identification processing method for multi-target tracking tracks of image sequences |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN106600631A (en) * | 2016-11-30 | 2017-04-26 | 郑州金惠计算机***工程有限公司 | Multiple target tracking-based passenger flow statistics method |
| CN107193032A (en) * | 2017-03-31 | 2017-09-22 | 长光卫星技术有限公司 | Multiple mobile object based on satellite video quickly tracks speed-measuring method |
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN110472496A (en) * | 2019-07-08 | 2019-11-19 | 长安大学 | A kind of traffic video intelligent analysis method based on object detecting and tracking |
| CN110728697A (en) * | 2019-09-30 | 2020-01-24 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Infrared dim target detection tracking method based on convolutional neural network |
| CN111652910A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Target tracking algorithm based on object space relationship |
| CN111739053A (en) * | 2019-03-21 | 2020-10-02 | 四川大学 | Online multi-pedestrian detection tracking method under complex scene |
| CN112487934A (en) * | 2020-11-26 | 2021-03-12 | 电子科技大学 | Strong data association integrated real-time multi-target tracking method based on ReID (ReID) characteristics |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN113674328A (en) * | 2021-07-14 | 2021-11-19 | 南京邮电大学 | Multi-target vehicle tracking method |
| CN113706574A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Movement track determining method and device, electronic equipment and storage medium |
-
2022
- 2022-04-24 CN CN202210434733.0A patent/CN114882068B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012112807A (en) * | 2010-11-25 | 2012-06-14 | Mitsubishi Electric Corp | Multi-target tracking device |
| CN103927763A (en) * | 2014-03-24 | 2014-07-16 | 河海大学 | Identification processing method for multi-target tracking tracks of image sequences |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN106600631A (en) * | 2016-11-30 | 2017-04-26 | 郑州金惠计算机***工程有限公司 | Multiple target tracking-based passenger flow statistics method |
| CN107193032A (en) * | 2017-03-31 | 2017-09-22 | 长光卫星技术有限公司 | Multiple mobile object based on satellite video quickly tracks speed-measuring method |
| CN108921873A (en) * | 2018-05-29 | 2018-11-30 | 福州大学 | The online multi-object tracking method of Markovian decision of filtering optimization is closed based on nuclear phase |
| CN111739053A (en) * | 2019-03-21 | 2020-10-02 | 四川大学 | Online multi-pedestrian detection tracking method under complex scene |
| CN110472496A (en) * | 2019-07-08 | 2019-11-19 | 长安大学 | A kind of traffic video intelligent analysis method based on object detecting and tracking |
| CN110728697A (en) * | 2019-09-30 | 2020-01-24 | 华中光电技术研究所（中国船舶重工集团有限公司第七一七研究所） | Infrared dim target detection tracking method based on convolutional neural network |
| CN113706574A (en) * | 2020-05-20 | 2021-11-26 | 杭州海康威视数字技术股份有限公司 | Movement track determining method and device, electronic equipment and storage medium |
| CN111652910A (en) * | 2020-05-22 | 2020-09-11 | 重庆理工大学 | Target tracking algorithm based on object space relationship |
| CN112487934A (en) * | 2020-11-26 | 2021-03-12 | 电子科技大学 | Strong data association integrated real-time multi-target tracking method based on ReID (ReID) characteristics |
| CN112561963A (en) * | 2020-12-18 | 2021-03-26 | 北京百度网讯科技有限公司 | Target tracking method and device, road side equipment and storage medium |
| CN113674328A (en) * | 2021-07-14 | 2021-11-19 | 南京邮电大学 | Multi-target vehicle tracking method |
Non-Patent Citations (6)
| Title |
|---|
| JAEYONG JU等: "Online Multi-Object Tracking based on Hierarchical Association Framework", pages 34 - 42 * |
| YIFU ZHANG等: "ByteTrack: Multi-Object Tracking by Associating Every Detection Box", pages 1 - 14 * |
| 张家豪: "基于多任务学习与代理模型的多目标跟踪方法", no. 04, pages 138 - 956 * |
| 牛通等: "基于深度学习的分层关联多行人跟踪", vol. 57, no. 8, pages 96 - 102 * |
| 胡瑶: "基于阴影的SAR多目标跟踪方法研究", no. 01, pages 136 - 2222 * |
| 金标等: "基于多级跟踪队列的运动目标跟踪遮挡处理", vol. 31, no. 8, pages 0815001 - 1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114882068B (en) | 2023-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108470332B (en) | Multi-target tracking method and device | |
| US11526698B2 (en) | Unified referring video object segmentation network | |
| CN114119676B (en) | Target detection tracking identification method and system based on multi-feature information fusion | |
| CN112001932B (en) | Face recognition method, device, computer equipment and storage medium | |
| CN109492576B (en) | Image recognition method and device and electronic equipment | |
| CN110765860A (en) | Tumble determination method, tumble determination device, computer apparatus, and storage medium | |
| CN113344206A (en) | Knowledge distillation method, device and equipment integrating channel and relation feature learning | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN113255557B (en) | Deep learning-based video crowd emotion analysis method and system | |
| CN114170516B (en) | Vehicle weight recognition method and device based on roadside perception and electronic equipment | |
| CN111696133B (en) | Real-time target tracking method and system | |
| CN110009662B (en) | Face tracking method and device, electronic equipment and computer readable storage medium | |
| KR20230171966A (en) | Image processing method and device and computer-readable storage medium | |
| Yang et al. | Parsing 3D motion trajectory for gesture recognition | |
| CN110363193B (en) | Vehicle weight recognition method, device, equipment and computer storage medium | |
| Liang et al. | Human action segmentation and classification based on the Isomap algorithm | |
| CN114676756A (en) | Image recognition method, image recognition device and computer storage medium | |
| CN113269038B (en) | Multi-scale-based pedestrian detection method | |
| CN111814653B (en) | Method, device, equipment and storage medium for detecting abnormal behavior in video | |
| CN113689475A (en) | Cross-border head trajectory tracking method, equipment and storage medium | |
| Veeraraghavan et al. | Learning dynamic event descriptions in image sequences | |
| CN115527083B (en) | Image annotation method and device and electronic equipment | |
| CN114882068B (en) | Multi-target tracking method, device, computer equipment and storage medium | |
| CN113762027B (en) | Abnormal behavior identification method, device, equipment and storage medium | |
| CN114445787A (en) | Non-motor vehicle weight recognition method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |