WO2020215492A1 - 基于YOLOv3多伯努利视频多目标检测跟踪方法 - Google Patents
基于YOLOv3多伯努利视频多目标检测跟踪方法 Download PDFInfo
- Publication number
- WO2020215492A1 WO2020215492A1 PCT/CN2019/094662 CN2019094662W WO2020215492A1 WO 2020215492 A1 WO2020215492 A1 WO 2020215492A1 CN 2019094662 W CN2019094662 W CN 2019094662W WO 2020215492 A1 WO2020215492 A1 WO 2020215492A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- frame
- detection
- detection frame
- tracking
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Definitions
- the invention relates to a multi-target detection and tracking method based on YOLOv3 multi-Bernoulli video, which belongs to the field of machine vision and intelligent information processing.
- the target detection and tracking method based on data association was mainly used in the early stage.
- the target detector was first used to detect multiple targets in the video sequence, and then the video multi-target tracking was completed with the help of technologies such as data association.
- Typical data associations include: multi-hypothesis tracking, joint probabilistic data association, graph decomposition, dynamic programming, etc.
- Random Finite Set (RFS) theory has achieved certain advantages in tracking an unknown and varying number of multiple targets.
- the random set modeling of target states and observations can avoid complex data association operations.
- Professor Mahler proposed Probability Hypothesis Density (PHD) and Multi-Bernoulli (MeMBer) filters random finite set theory has been widely used in the field of target tracking.
- multi-target tracking algorithms based on random finite set theory mainly include two categories: multi-target tracking algorithms based on probability hypothesis density (PHD)/potential probability hypothesis density (CPHD) and multi-Bernoulli (MeMBer)/ Multi-target tracking algorithm based on potential equilibrium multi-Bernoulli (CBMeMBer).
- Typical closed solutions include: particle filter PHD/CPHD, Gaussian mixture PHD/CPHD, particle filter CBMeMBer and Gaussian mixture CBMeMBer, etc.
- Particle Filter Multiple Bernoulli (PFMB) technology recursively approximates the posterior probability density of the multi-target state set with the help of multi-target Bayesian estimation theory, which can improve the tracking accuracy of multi-targets with varying numbers.
- the PFMB method is difficult to detect new targets, and when multiple targets are occluded and interfered with each other, the tracking accuracy decreases, and even the target is underestimated.
- the present invention provides a method based on YOLOv3. Nuuli video multi-target detection and tracking method. In the detection and tracking process of the method, the YOLOv3 technology is used to detect the kth and k+1 frames of the video sequence.
- the number of the kth frame detection frame is recorded as n, and the detection frame state set is The number of detection frames in the k+1 frame is m, and the detection frame state set is among them, Represents the state vector of the i-th detection frame, parameter Respectively represent the abscissa and ordinate of the upper left corner of the i-th detection frame in the kth frame, as well as the width, height and label of the detection frame;
- the middle frame of the video k>0, uses the detection frame, target trajectory information and target template information to achieve new target determination, missed target re-identification, and survival target optimization tracking; among them, the survival target optimization tracking is in the Do Bernoulli filter framework
- the anti-interference convolution feature is used to represent the detection frame of the kth frame and the k+1th frame, which are respectively denoted as with Represents the convolution feature vector of the i-th detection frame in the kth frame; calculate the similarity matrix ⁇ corresponding to the convolution feature of the detection frame, namely:
- the detection frame uses the detection frame, target trajectory information and target template information to achieve new target determination, missing target re-identification, and survival target optimization tracking, including:
- the matching of adjacent detection frames includes:
- performing target recognition according to the detection frame of the adjacent frame, the target template set, and the surviving target trajectory set includes:
- the target of the detection frame is the surviving target, and the target label is If Is empty, but matches the target with label l b in the surviving target trajectory set, it can also be judged that the target in the detection frame is a surviving target, and a label is assigned to it, namely
- the detection frame is determined to be interference clutter.
- the method further includes: constructing a target motion model according to the confidence of the detection frame;
- the target motion model is a random walk model
- the YOLOv3 detector detects the target, and the confidence of the detection frame When greater than the confidence threshold T B , the detection frame is used to adjust the target state, that is
- Is the target state vector labeled l i in the k-1 frame, Is the detection frame with the label l i in the kth frame, and the confidence of the detection frame Indicates the probability that the detection frame contains the target and the score of the degree of matching between the detection frame and the target, e(k-1) represents zero-mean Gaussian white noise, ⁇ is the learning rate between the target state and the corresponding detection frame information, and the confidence of the detection frame The higher the value, the larger the value of ⁇ , which means the more confidence in the detection result.
- Dobernulli parameter set Represents the posterior probability distribution of multiple targets, where M k-1 is the number of targets existing at k-1, Represents the existence probability of target l i at time k-1, Represents the probability distribution of the target l i at time k-1, represented by a set of weighted particles:
- the multi-objective probability distribution is still a multi-Bernoulli parameter set, expressed as:
- M ⁇ ,k is the number of newborn targets in the kth frame.
- the multi-target prediction probability distribution of the k-1th frame is:
- the surviving target trajectory, target template, and target detection result are used to merge and update the target template, namely:
- the detector can detect these two targets, it means that the target is slightly occluded, and the target template is adaptively updated;
- the other can be determined The target is an occluded target.
- the template update is stopped, and the displacement difference of the first two frames of the target is used to estimate the target real-time velocity v and the movement direction ⁇ to predict the target, and the size of the target frame remains unchanged;
- the occluded target is judged based on the similarity between the target frame and the template, and the occluded target is processed in the same way as in the case (2); if the target is in the tracking process When the target is separated or reappears, the target is re-identified according to the match between the detection result and the target template.
- the anti-jamming convolution feature is used to target the target.
- the details are described in depth, and the target state is accurately estimated with the help of an adaptive PFMB method.
- the detection result and the tracking result are especially interactively fused to improve the estimation accuracy of the multi-target state.
- the present invention is also aimed at YOLOv3 in the complex environment video multi-target detection, there will also be problems of detection errors and inaccurate detection results.
- the present invention integrates the YOLOv3 detection method under the PFMB filtering framework, which not only solves the problem of unknown new targets in the PFMB method
- Estimation problems, target identification problems, and fusion modeling of detection confidence and filtering likelihood can also effectively improve the tracking accuracy of multiple video targets in complex environments.
- Figure 1 is an algorithm framework diagram of the method of the present invention.
- Figure 2 is a schematic diagram of the working principle of YOLOv3.
- Figure 3 is a diagram of the YOLOv3 network structure.
- Figure 4 is the experimental results of EnterExitCrossingPaths1cor sequence.
- Figure 5 is a comparison chart of the estimated number of experimental targets in the EnterExitCrossingPaths1cor sequence.
- Figure 6 is a comparison diagram of the OSPA distance estimation in the EnterExitCrossingPaths1cor sequence experiment.
- FIG. 7 shows the results of Jogging sequence experiments.
- Figure 8 is a comparison diagram of the estimated number of experimental targets in the Jogging sequence.
- Figure 9 is a comparison diagram of Jogging column experimental OSPA distance estimation.
- FIG. 10 shows the results of the Subway sequence experiment.
- Figure 11 is a comparison chart of the estimated number of targets in the Subway sequence experiment.
- Figure 12 is a comparison diagram of OSPA distance estimation in Subway sequence experiment.
- FIG. 13 shows the results of the Human4 sequence experiment.
- Figure 14 is a comparison diagram of the estimated number of human 4 sequence experiments.
- Figure 15 is a comparison diagram of the human4 sequence experiment OSPA distance estimation.
- FIG. 16 shows the results of the Suv sequence experiment.
- Figure 17 is a comparison diagram of the estimated number of targets in the Suv sequence experiment.
- Figure 18 is a comparison diagram of OSPA distance estimation in Suv sequence experiment.
- the target component through Bernoulli parameters (r (i) , p (i) ) to parameterize the posterior probability distribution of the target, where r (i) and p (i) represent the existence probability and probability distribution of the i-th target, respectively .
- MBF uses Bayesian theory to iteratively update the posterior probability distribution of multiple targets to achieve the state estimation of multiple targets.
- M k-1 represents the number of surviving targets in the k-1 frame.
- the predicted probability distribution of multiple targets can be expressed as:
- ⁇ f 1 ( ⁇ ), f 2 ( ⁇ )> represents the standard inner product ⁇ f 1 (x)f 2 (x)dx
- P S, k is the target survival probability
- ⁇ ) Is the target state transition function
- M ⁇ ,k represents the number of newly born targets in the k-th frame.
- the updated multi-target posterior probability density can be calculated from the multi-Bernoulli parameter of the missed target And measuring the updated Do Bernoulli parameters Approximately, namely:
- p D, k (x) is the target detection probability
- x) is the measurement likelihood function
- YOLOv3 uses Darknet-53 as the feature extraction network.
- the network structure is shown in Figure 3. It consists of continuous 3 ⁇ 3 and 1 ⁇ 1 convolutional layers, and integrates the residual block of the Residual Neural Network (ResNet) (Residual block), the entire network is divided into multiple sub-segments to train stage by stage, and the residual of each sub-segment is trained using a shortcut connection method to achieve the smallest overall residual.
- Residual Neural Network Residual Neural Network
- YOLOv3 predicts bounding boxes on three different scales, each scale predicts 3 bounding boxes, and local feature interaction is performed within the scale.
- a series of convolutional layers are added to obtain a feature map.
- Perform position regression and classification this process is the smallest scale prediction; connect the convolutional layer upsampling in the previous scale to the last 16 ⁇ 16 feature map, and output the prediction information after multiple convolutions; the same is true , Connect the mid-scale convolutional layer upsampling with the last 32 ⁇ 32 feature map, and obtain the bounding box prediction on the largest scale after a series of convolutions.
- YOLOv3 adopts the idea of anchor box in Faster R-CNN, and generates a priori of 9 bounding boxes through the k-means clustering method, and each size predicts 3 bounding boxes.
- YOLOv3 divides the image into G ⁇ G grids, then the predicted tensor for each scale is G ⁇ G ⁇ [3*(4+1+80)], where the number 3 represents 3 prediction bounding boxes, and the number 4 represents each bounding box contains four predicted offset value (t x, t y, t w, t h), (t x, t y, t w, t h) the center coordinates of the bounding box is the predicted offset value and width, A high offset value, the number 1 represents the confidence score of the bounding box, and the number 80 represents the conditional category probability P r (Class i /Object) that the predicted bounding box belongs to 80 classes of objects.
- the bounding box uses the sum of squared error losses during training.
- the confidence of the bounding box is determined by the probability that the bounding box contains the target P r (object) and the accuracy of the bounding box constitute:
- the conditional class probability P r (Class i /Object) represents the probability that it belongs to a certain class under the premise that the bounding box contains the target.
- YOLOv3 combines conditional category probabilities and bounding box confidence to obtain bounding box category confidence scores (class specific confidence scores, C) to indicate the probability that the bounding box contains the target classified into each category and how well the bounding box matches the target.
- the confidence of the border category can be expressed as:
- This embodiment provides a multi-target detection and tracking method based on YOLOv3 multi-Bernoulli video.
- the method includes:
- YOLOv3 algorithm to detect the kth and k+1 frames of the video sequence, record the number of the kth frame detection frame as n, and the detection frame state set as The number of detection frames in the k+1 frame is m, and the detection frame state set is among them, Represents the state vector of the i-th detection frame, parameter Respectively represent the abscissa and ordinate of the upper left corner of the i-th detection frame in the kth frame, as well as the width, height and label of the detection frame.
- the anti-interference convolution feature is used to represent the detection frame of the kth frame and the k+1th frame, which are respectively denoted as with Represents the convolution feature vector of the i-th detection frame in the kth frame; calculate the similarity matrix ⁇ corresponding to the convolution feature of the detection frame, namely:
- the intersection of the target box and the IOU are added as a restriction: if If the intersection ratio IOU of the two detection frames is greater than the intersection ratio threshold T u , it can be determined that the two detection frames match.
- the target of the detection frame is the surviving target, and the target label is If Is empty, but it matches the target with label l b in the trajectory of the surviving target, it can also be determined that the target in the detection frame is a surviving target, and a label is assigned to it, namely
- the detection frame is interference clutter.
- Step 3 Target prediction
- the target motion model is a random walk model
- the YOLOv3 detector detects the target, and the confidence of the detection frame When greater than the confidence threshold T B , the detection frame is used to adjust the target state, that is
- Is the target state vector labeled l i in the k-1 frame, Is the detection frame with the label l i in the kth frame, and the confidence of the detection frame Indicates the probability that the detection frame contains the target and the score of the degree of matching between the detection frame and the target, e(k-1) represents zero-mean Gaussian white noise, ⁇ is the learning rate between the target state and the corresponding detection frame information, and the confidence of the detection frame The higher and the larger ⁇ , the more confidence in the detection result.
- the use of a good detection frame to adjust the target state can eliminate the accumulated error caused by long-term tracking and better optimize the prediction result.
- the Dobernulli parameter set is used Represents the posterior probability distribution of multiple targets, where M k-1 is the number of targets existing at k-1, Represents the existence probability of target l i at time k-1, Represents the probability distribution of the target l i at time k-1, represented by a set of weighted particles:
- the multi-objective probability distribution is still a multi-Bernoulli parameter set, expressed as:
- M ⁇ ,k is the number of newborn targets in the kth frame.
- the target frame randomly sample m background sample frames with a rectangular frame the size of the target frame.
- the distance between the background sample frame and the center of the target frame must be greater than 1/4 of the target frame width in the horizontal direction or greater than the vertical distance 1/4 the height of the target frame, then normalize the size of the target frame and the background sample frame to n ⁇ n, and gray-scale the image to obtain the sample set ⁇ I, B 1 , B 2 ,..., B m ⁇ , where I represents the target Frame image, B i is the i-th background sample image.
- Extract feature maps that weaken background information The feature maps are expanded in rows and sequentially spliced to obtain the final one-dimensional convolution feature f, where,
- vec(C) is the column vector by concatenating all the elements in C.
- the only solution of sparse representation is obtained by soft-shrinking method:
- ⁇ is the median of the tensor C.
- f 1 and f 2 are the corresponding target frame convolution features.
- the target feature atlas and background feature atlas corresponding to the target are used to calculate the convolution features corresponding to the particles:
- Il is the normalized image of the candidate frame represented by the particle. Expand S i in rows to obtain d one-dimensional vectors, which are sequentially connected to the feature f l of the candidate frame corresponding to the final particle.
- Step 5 Target status update and extraction
- the present invention adopts a random resampling method to resample the sampled particle set to avoid particle degradation and eliminate Bernoulli components with too low probability.
- Step 6 Target occlusion processing mechanism
- the detector may have three situations: 1) When the target is not completely occluded When the detector may detect two targets, the method of the present invention proposes to use an update mechanism to track the target and update the template; 2) When only one target can be detected by the detector, the other target can be determined to be an occluded target.
- the template update is stopped for the occluded target, and the displacement difference of the first two frames of the target is used to estimate the real-time speed v and the moving direction ⁇ of the target, and the target is predicted and estimated, and the size of the target frame remains unchanged; 3) Both targets cannot When detected by the detector, the occluded target is judged based on the similarity between the target and the template, and the occluded target is processed in the same way as in the second case. If the target disappears or is lost during the tracking process, when the target separates or reappears, the target can be re-identified according to the matching of the detection result and the target template.
- Step 7 Target template update
- the video sequence data used in the present invention is the sequence Huamn4 in Visual Tracker Benchmark TB50, the sequence Jogging, Subway, and Suv in Visual Tracker Benchmark TB100, and the sequence EnterExitCrossingPaths1cor in the CAVUAR data set. These 5 groups of typical video sequences are from different scenarios. It also includes dynamic background, target close proximity, target deformation, image blur, target size change, target occlusion and other interference situations.
- the evaluation indicators used in the experiment are Multiple Object Tracking Accuracy (MOTA), Multiple Object Tracking Precision (MOTP), the number of complete trajectory targets (Mostly Tracked, MT), and the number of tag jumps (Identity). Switch, IDs) are defined as follows:
- m t is the number of lost targets in the t-th frame
- fp t is the number of misdetected targets in the t-th frame
- mme t is the number of jumps in the target label in the tracking trajectory in the t-th frame
- g t is The actual number of targets in the t frame.
- MT The number of complete targets in the trajectory (MT), which represents the number of target trajectories whose trajectories account for more than 80% of the true trajectory length, and describes the integrity of the trajectory.
- Tag jump number which indicates the number of times the target tag changes during the tracking process.
- the method of this application is implemented by Matlab2016a, which runs on a workstation with Intel Core i7-8700, 3.2GHz, 12 cores, 16GB memory, and NVIDIA Geforce GTX 1080 Ti graphics card, and is combined with the traditional particle multi-Bernoulli filter (PFMB) method and the IOU-Tracker method proposed in the paper "High-Speed Tracking-by-Detection Without Using Image Information” published by Erik Bochinski et al. in 2017 for performance comparison and analysis. Because the traditional PFMB method lacks a detection mechanism for new-born targets, YOLOv3 is also used in this experiment for detection, and the detection result determined as a new-born target is taken as the new-born target of PFMB and then tracked.
- PFMB particle multi-Bernoulli filter
- the video sequence used in this experiment is the EnterExitCrossingPaths1cor sequence in the CAVUAR data set. This sequence contains 383 frames of images.
- the fusion of the detector result, the existence of the target trajectory and the target template of the present invention can realize the re-identification of the target and re-add the target to the tracking trajectory.
- Figure 4 shows the experimental results of the EnterExitCrossingPaths1cor sequence.
- Figure 4(a) is the detection result of the YOLOv3 algorithm
- Figure 4(b) is the tracking result of the traditional PFMB method
- Figure 4(c) is the tracking result of the IOU-Tracker method
- Figure 4 (d) is the tracking result of the method of the present invention. It can be seen that when the target is occluded, the detector may not be able to detect the occluded target, which greatly reduces the detection effect.
- the traditional PFMB method is used. When there is occlusion, the target is followed.
- Figure 5 is a comparison chart of the estimated number of targets in the EnterExitCrossingPaths1cor sequence experiment
- Figure 6 is a comparison chart of the OSPA distance estimation of the EnterExitCrossingPaths1cor sequence experiment. Missing tracking results in a large deviation of the tracking frame and an increase in the OSPA value; the IOU-Tracker method is affected by the performance of the detector. During the target occlusion process, the number of targets decreases and the OSPA value rises sharply. The method of the present invention deals with long-time occluded targets , There may also be target missed tracking. When the target is out of occlusion, it can effectively re-identify the missed tracking target. The overall tracking performance is better than the traditional PFMB method and IOU-Tracker method. In particular, it can be seen from the four evaluation indicators of MOTA, MOTP, MT, and IDs shown in Table 1.
- Table 1 shows the results of 50 Monte Carlo simulations. It can be seen that the traditional PFMB method has the lowest values of indicators MOTA, MOTP, and MT, and IDs is 1. This is because the method lacks the target re-recognition process, which leads to the target being lost. When it reappears, it will not be able to be associated with the previous trajectory, and the label will jump. In the IOU-Tracker method, when the target is close to or hidden, the target label will frequently jump, resulting in the highest IDs. The method of the present invention can effectively re-identify the target, reduce the problem of target label jump, and effectively reduce track fragments.
- the video sequence used is the Jogging sequence and Subway sequence in the Visual Tracker Benchmark TB100 data set.
- the Jogging sequence is an intersection scene where the camera moves, which contains 3 moving targets, and the target is blocked.
- the Subway sequence contains 8 targets, and there are problems such as close proximity and frequent occlusion of multiple targets.
- FIG. 7(a) is the detection result of the YOLOv3 algorithm
- Figure 7(b) is the tracking result of the traditional PFMB method
- Figure 7(c) is the tracking result of the IOU-Tracker method
- 7(d) is the tracking result of the method of the present invention.
- the YOLOv3 detector is difficult to correctly detect the occluded target.
- the IOU-Tracker method loses the missed target.
- the IOU-Tracker method defines the re-detected target as a new target, the traditional PFMB Even if the method does not lose the target, the deviation degree of the tracking frame increases.
- the method of the present invention can well integrate the detection result and tracking result of the target, realize the adjustment of the tracking frame, and obtain more accurate tracking results.
- Figure 8 shows a comparison chart of the number of targets in the Jogging sequence experiment.
- the traditional PFMB method is consistent with the number of targets of the present invention.
- Figure 9 shows the comparison chart of the OSPA distance estimation in the Jogging sequence experiment. It can be seen that the YOLOv3 detector The existence of missing detection results in inaccurate estimation of the number of targets in the IOU-Tracker method, and a decrease in missing estimates.
- the OSPA value rises sharply and the accuracy is not high; the traditional PFMB method increases the deviation of the tracking frame after the target is occluded, resulting in the OSPA value The rise increases, and the method of the present invention has better anti-blocking ability, and the tracking performance is obviously better than the traditional PFMB method and the IOU-Tracker method.
- Figure 10(a) is the detection result of the YOLO v3 algorithm
- Figure 10(b) is the tracking result of the traditional PFMB method
- Figure 10(c) is the tracking result of the IOU-Tracker method
- Figure 10(d) shows the tracking result of the method of the present invention. It can be seen that when multiple targets are close to or occluded each other, the detector is difficult to detect the occluded target or the detection result has a large deviation, resulting in frequent loss of targets in the IOU-Tracker method.
- the traditional PFMB method deals with occlusion and close proximity problems, Large deviations may also occur, or even tracking loss, as shown in the 17th and 19th frames in Figure 10(b), and the method of the present invention can better track the occluded target.
- Figure 11 shows the comparison diagram of the target number estimation of the Subway sequence
- Figure 12 shows the comparison diagram of the OSPA distance estimation of the Subway sequence.
- the quantitative analysis results of the Subway sequence are shown in Table 2.2.
- multiple targets are also in a state of obscuring each other for a long time and are in close proximity.
- the method of the present invention is significantly better than the traditional PFMB method and IOU in terms of MOTA, MOTP, MT, and IDs. -Tracker method.
- the Human4 sequence in the video sequence Visual Tracker Benchmark TB50 data set is used.
- the sequence contains a total of 667 frames of images, which is the scene of the camera moving traffic lights.
- the video is caused by camera movement or rapid target movement.
- the target is ambiguous, and many targets frequently deform.
- Figure 13 shows the results of the Human4 sequence experiment.
- Figure 13 (a) is the detection result of the YOLO v3 algorithm
- (b) is the tracking result of the traditional PFMB method
- (c) is the tracking result of the IOU-Tracker method
- (d) is the method of the present invention
- the tracking results It can be seen that the image blur may cause the performance of the detector to decrease, and some targets are missed, as shown in the 50th and 74th frames in Figure 13(a).
- the target deformation has little impact on the performance of the detector.
- the traditional PFMB method can keep up with the target, the tracking frame of some targets will deviate.
- the method of the present invention can better deal with these two situations and has higher tracking. Accuracy.
- Figure 13 shows the comparison chart of human4 sequence target number estimation
- Figure 14 shows the comparison chart of Human4 sequence OSPA distance estimation. It can be seen that the method of the present invention can better deal with the problems of image blur and target deformation, and the tracking performance is better than the traditional PFMB method and the IOU-Tracker method.
- the traditional PFMB method causes the target to miss or only track a part of the target area, resulting in lower values of MOTA, MOTP, and MT; the IOU-Tracker method is better
- the detection result of the detector is used, so the MOTP value is equivalent to the method of the present invention, and the target size change and image blurring cause the performance of the detector to decrease, and some targets are missed, which makes the MOTA value too small and the IDs value to be the largest.
- a large number of target tags jump and the track estimation is not accurate.
- the method of the present invention can better correlate the detection frame, effectively reduce the jump of the target label, and is obviously better than the IOU-Tracker in IDs.
- the Suv sequence in the video sequence Visual Tracker Benchmark TB100 data set is used.
- the sequence contains 400 frames of images. It is a road scene with a dynamic background of the camera moving and contains 6 targets. There is a large displacement caused by fast motion.
- Figure 16 shows the experimental results of the Suv sequence.
- Figure 16(a) is the detection result of the YOLO v3 algorithm
- Figure 16(b) is the tracking result of the traditional PFMB method
- Figure 16(c) is the tracking result of the IOU-Tracker method
- 16(d) is the tracking result of the method of the present invention. It can be seen that the large displacement of the target has no effect on the detector. The IOU-Tracker method performs well, while the traditional PFMB method will cause the target to be lost. The method of the present invention uses the result of the detector to adjust the tracking process. The displacement target tracking results are also significantly better than the other two methods.
- Figure 17 is a comparison diagram of the target number estimation of Suv sequence
- Figure 18 is a comparison diagram of OSPA distance estimation of Suv sequence. It can be seen that the traditional PFMB method lacks a processing mechanism for large displacement of the target. Therefore, when the target displacement is large, the target may be lost or the tracking frame deviates greatly, resulting in a large OSPA value, and IOU-Tracker
- the method and the method of the present invention can use the result of the detector to improve the ability to process large displacements of the target, and have better tracking accuracy.
- the method of the present invention can effectively process the large displacement of the target by fusing the detection results, and is significantly better than the traditional PFMB method in various indicators, and is also significantly better than the IOU-Tracker method in MOTA, MT, and IDs.
- the multi-target detection and tracking method based on YOLOv3 multi-Bernoulli video provided by this application is interactively fused with the detection result and the tracking result.
- the method proposed in this application can fuse the detection results and tracking results of the targets, realize the adjustment of the tracking frame, and obtain comparison Accurate tracking results; when there are image blurring and target deformation, the method proposed in this application can better correlate the detection frame and effectively reduce the jump of the target label; and when there is a large displacement of the target, this application
- the proposed method can use the results of the detector to adjust the tracking process, so as to prevent the occurrence of missing targets.
- Part of the steps in the embodiments of the present invention can be implemented by software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于YOLOv3多伯努利视频多目标检测跟踪方法,属于机器视觉、智能信息处理领域。本发明在多伯努利滤波框架下引入YOLOv3检测技术,采用抗干扰的卷积特征描述目标,并交互融合检测结果和跟踪结果,实现对数目未知且时变的视频多目标状态进行精确估计;跟踪过程中,将匹配的检测框与目标轨迹及目标模板相结合,实时进行目标新生判断与遮挡目标重识别,同时考虑检测目标和估计目标的身份标记信息,实现对目标身份识别及航迹跟踪,可以有效提高对被遮挡目标的跟踪精度,减少轨迹碎片。实验表明,本发明具有良好的跟踪效果和鲁棒性,能广泛满足智能视频监控、人机交互、智能交通管制等***的实际设计需求。
Description
本发明涉及基于YOLOv3多伯努利视频多目标检测跟踪方法,属于机器视觉、智能信息处理领域。
复杂环境下视频多目标跟踪应用领域中,除了存在光照变化、目标形变、目标被遮挡等问题,还存在目标数目未知、新生目标不确定、目标交叉或紧邻运动、目标消失及杂波干扰等复杂情况,一直是多目标跟踪领域中研究的难点和具有挑战性的问题。
针对视频多目标跟踪问题,早期主要采用基于数据关联的目标检测跟踪方法,先采用目标检测器对视频序列进行多目标检测,然后借助数据关联等技术完成对视频多目标跟踪。典型的数据关联如:多假设跟踪、联合概率数据关联、图分解、动态规划等,虽然这些方法在视频多目标跟踪中取得了一定的效果,但由于复杂的数据关联运算,一定程度上降低了算法的运算效率,此外,对数目未知且变化的多目标跟踪,存在目标数目及状态估计不准确的问题。
近年来,随机有限集(Random Finite Set,RFS)理论在对数目未知且变化的多目标跟踪中取得了一定优势,分别对目标状态和观测进行随机集建模,可避免复杂的数据关联运算。自Mahler教授提出概率假设密度(Probability hypothesis density,PHD)和多伯努利(MeMBer)滤波器之后,随机有限集理论在目标跟踪领域得到了广泛地应用。概括来说,基于随机有限集理论的多目标跟踪算法主要包含两大类:基于概率假设密度(PHD)/势概率假设密度(CPHD)的多目标跟踪算法和基于多伯努利(MeMBer)/势均衡多伯努利(CBMeMBer)的多目标跟踪算法。典型的闭合解有:粒子滤波PHD/CPHD、高斯混合PHD/CPHD、粒子滤波CBMeMBer和高斯混合CBMeMBer等。尤其是粒子多伯努利滤波(Particle Filter Multiple Bernoulli,PFMB)技术,借助多目标贝叶斯估计理论递推近似多目标状态集的后验概率密度,可提高对数目变化多目标的跟踪精度。但PFMB方法难以检测新生目标,且当多目标之间出现相互遮挡和干扰时,跟踪精度下降,甚至出现目标被漏估计的问题。
发明内容
为了解决目前存在的现有目标跟踪方法无法检测新生目标以及当多目标之间出现相互遮挡和干扰时,跟踪精度下降,甚至出现目标被漏估计的问题,本发明提供了一种基于YOLOv3多伯努利视频多目标检测跟踪方法,所述方法检测跟踪过程中,采用YOLOv3技术检测第k和k+1帧视频序列,记第k帧检测框个数为n,检测框状态集为
第k+1帧检测框个数为m,其检测框状态集为
其中,
表示第i个检测框状态向量,参数
分别表示第k帧第i个检测框左上角的横坐标、纵坐标,以及检测框的宽、高和标签;
将第k和k+1帧视频序列的检测框进行匹配;对于视频的初始帧,k=0,将初始帧中已匹配的检测框作为初始的新生目标加入目标模板集和目标轨迹集中;对于视频的中间帧,k>0,利用检测框、目标轨迹信息和目标模板信息实现新生目标判定、漏跟目标重识别和存活目标优化跟踪;其中,存活目标优化跟踪是在多伯努利滤波框架下,利用当前帧置信度大于给定置信度阈值T
b的检测框信息,优化对应目标的跟踪过程。
可选的,所述对于视频的初始帧,k=0,将已匹配的检测框作为新生目标加入目标模板集和目标轨迹集中,包括:
其中,
表示第k帧中第i个检测框与第k+1帧中第j个检测框的相似度;从相似度矩阵Λ的每一行选择值最大、且大于相似度阈值T
l的两个检测框作为匹配结果,将最终匹配的检测框对作为初始的新生目标加入目标模板集和目标轨迹集中。
可选的,将初始两帧的检测框进行匹配,并判定新生目标,将最终匹配的检测框对作为初始的新生目标加入目标模板集中,包括:
假定同一个目标在相邻两帧不会出现特别大的位移变化,在相似度匹配的基础上,加入目标框的交 并比IOU作为限制:若两个检测框的交并比IOU大于交并比阈值T
u,则可判定两个检测框匹配;
从相似度矩阵Λ的每一行选择值最大、且大于相似度阈值T
l,同时交并比IOU大于交并比阈值T
u的两个检测框作为匹配结果,将最终匹配的检测框对作为初始的新生目标,分配标签
并加入目标轨迹集
中,其中,
为目标状态向量,各个分量分别表示第i个目标框左上角横坐标、纵坐标以及目标框的宽、高和标签,M
k为第k帧目标的个数,给已匹配的检测框对添加与初始的新生目标对应的目标标签,即
同时建立新生目标模板。
可选的,所述对于视频的中间帧,k>0,利用检测框、目标轨迹信息和目标模板信息来实现新生目标判定、漏跟目标重识别和存活目标优化跟踪,包括:
对相邻的检测框进行匹配,确定新生目标与重识别目标的判定条件,再根据相邻帧的检测框、目标模板集和存活目标轨迹集来判定目标是否为新生目标、重识别目标或存活目标。
可选的,所述对相邻的检测框进行匹配,包括:
计算第k和k+1帧视频序列中检测框之间的相似度矩阵Λ,从相似度矩阵Λ的每一行选择值最大且大于相似度阈值T
l的两个检测框,且这两个检测框的交并比IOU大于交并比阈值T
u,则可判定两个检测框匹配,假设第k帧中的第i个检测框与第k+1帧中的第j个检测框匹配,则为第k+1帧中匹配目标的标签赋值,即:
可选的,所述对于当前帧检测框目标,根据相邻帧的检测框、目标模板集和存活目标轨迹集来进行目标识别,包括:
(1)新生目标识别
其中
表示标签为l
i的新生目标状态向量,M
Γ表示新生目标个数,n表示目标模板的个数;给已匹配的检测框对添加与新生目标对应相同的目标标签,即
同时将新生目标加入目标模板集,并根据新生参数初始化新生目标的采样粒子;
(2)漏跟目标重识别
若
为空,与第k+1帧第j个检测框匹配,在存活目标轨迹集中没有与之匹配的目标,但与目标模板中标签为l
a的目标匹配,表示该检测框包含的目标在之前帧漏检,且出现漏跟情况,在第k帧,对该目标进行了重识别,将其重新加入到存活目标轨迹集中:
(3)存活目标识别
(4)干扰杂波识别
可选的,所述方法还包括:根据检测框置信度构造目标运动模型;
其中,
为第k-1帧中标签为l
i目标状态向量,
为第k帧中标签为l
i目标的检测框,检测框置信度
表示检测框包含目标的概率和检测框与目标匹配程度的得分,e(k-1)表示零均值高斯白噪声,η为目标状态与对应检测框信息之间的学习率,检测框的置信度越高,η越大,表示越信任检测结果。
可选的,在k-1时刻,采用多伯努利参数集
表示多目标的后验概率分布,其中,M
k-1为k-1时刻存在目标的数目,
表示在k-1时刻目标l
i的存在概率,
表示k-1时刻目标l
i的概率分布,由一组加权粒子表示:
多目标概率分布预仍为多伯努利参数集,表示为:
其中,
其中,
其中,M
Γ,k为第k帧新生目标数目。
可选的,假设第k-1帧多目标预测概率分布为:
通过粒子的量测似然更新多目标后验概率分布:
根据下式获得更新后的多伯努利参数集:
可选的,目标运动过程中,采用存活目标轨迹、目标模板和目标检测结果进行融合更新目标模板,即:
其中,
分别表示第k和k-1帧时目标l
i对应的卷积特征模板,
为第k-1帧目标l
i跟踪结果卷积特征的稀疏表示,
为第k帧目标l
i检测框卷积特征的稀疏表示,ρ为模板的学习速率,
为第k帧目标l
i对应检测框的置信度,
为检测框的学习率。
可选的,检测跟踪过程中,当相邻两个目标框的交并比IOU大于阈值T
u时,判定这两个目标出现遮挡情况;
此时,(1)当检测器能检测出这两个目标,表示目标轻度遮挡,对目标模板进行自适应更新;(2)当只有一个目标能被检测器检测出来时,可判定另一个目标为被遮挡目标,对被遮挡目标,停止模板更新,采用该目标前两帧的位移差,估计目标实时速度v与运动方向θ,对目标进行预测,目标框的大小保持不变;(3)当两个目标都无法通过检测器检测出来时,依据目标框与模板的相似度判断被遮挡目标,与第(2)种情况中采用同样的方式处理被遮挡的目标;若目标在跟踪过程中消失或者跟丢,在目标分离或者重新出现时,根据检测结果与目标模板的匹配对目标进行重识别。
本发明有益效果是:
通过引入YOLOv3技术对视频序列中的多目标进行检测,并将其作为新生目标,考虑到检测中存在目标过检测或漏检测、以及检测结果不精确的问题,采用抗干扰的卷积特征对目标细节进行深度描述,并借助自适应PFMB方法进行目标状态精确估计,本发明中尤其是将检测结果与跟踪结果进行交互融合,以提高多目标状态的估计精度。本发明还针对YOLOv3在对复杂环境的视频多目标检测,同样会存在检测错误和检测结果不准确的问题,本发明在PFMB滤波框架下融入YOLOv3检测方法,既解决了PFMB方法中未知新生目标的估计问题、目标身份的识别问题,同时将检测置信度和滤波似然进行融合建模,也可以有效提高对复杂环境下数目变化的视频多目标跟踪精度。
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1是本发明方法的算法框架图。
图2是YOLOv3工作原理示意图。
图3是YOLOv3网络结构图。
图4是EnterExitCrossingPaths1cor序列实验结果。
图5是EnterExitCrossingPaths1cor序列实验目标数目估计对比图。
图6是EnterExitCrossingPaths1cor序列实验OSPA距离估计对比图。
图7是Jogging序列实验结果。
图8是Jogging序列实验目标数目估计对比图。
图9是Jogging列实验OSPA距离估计对比图。
图10是Subway序列实验结果。
图11是Subway序列实验目标数目估计对比图。
图12是Subway序列实验OSPA距离估计对比图。
图13是Human4序列实验结果。
图14是Human4序列实验目标数目估计对比图。
图15是Human4序列实验OSPA距离估计对比图。
图16是Suv序列实验结果。
图17是Suv序列实验目标数目估计对比图。
图18是Suv序列实验OSPA距离估计对比图。
为使本发明的目的、技术方案和优点更加清楚,下面将结合附图对本发明实施方式作进一步地详细描述。
首先对本申请涉及的相关内容进行介绍如下:
1、多伯努利滤波原理
在空间χ上,将多目标状态RFS表示为X={X
(1),…,X
(M)},X
(i)={(r
(i),p
(i))}为第i个目标分量,通过伯努利参数(r
(i),p
(i))来参数化目标后验概率分布,其中r
(i)和p
(i)分别表示第i个目标的存在概率和概率分布。MBF采用贝叶斯理论对多目标的后验概率分布进行迭代更新,以达到对多目标的状态估计。
假设第k-1帧多目标的后验概率分布为:
其中,M
k-1表示第k-1帧存活目标的个数。则预测的多目标概率分布可表示为:
<f
1(·),f
2(·)>表示标准内积∫f
1(x)f
2(x)dx,P
S,k为目标存活概率,f
k|k-1(x|·)为目标状态转移函数,M
Γ,k表示第k帧新生目标的数目。
若第k帧,预测的多目标概率分布为:
其中,
ψ
k,z=g
k(z|x)p
D,k(x)
p
D,k(x)为目标检测概率,g
k(z|x)表示量测似然函数,Z
k、κ
k(z)分别表示量测集和杂波密度函数,第k帧目标数目估计为M
k=M
k|k-1+|Z
k|。
2、YOLOv3目标检测
YOLOv3的工作原理如图2所示。
YOLOv3采用Darknet-53作为特征提取的网络,该网络结构如图3所示,由连续的3×3和1×1卷积层组成,融合残差网络(Residual Neural Network,ResNet)的残差块(Residual block),将整个网络分为多个子段逐阶段训练,采用shortcut的连接方式对每个子段的残差进行训练,从而达到总体残差最小。
YOLOv3在三个不同尺度上预测边界框,每个尺度预测3个边界框,尺度内进行局部特征交互,在基础网络之后,添加一系列卷积层得到特征图(feature map),在此基础上,进行位置回归与分类,此过程为最小尺度预测;将上一尺度中的卷积层上采样与最后一个16×16大小的特征图连接,再次通过多个卷积后输出预测信息;同理,将中间尺度的卷积层上采样与最后一个32×32大小的特征图连接,经过一系列卷积得到最大尺度上的边界框预测。
YOLOv3采用Faster R-CNN中锚框(anchor box)的思想,通过k-means聚类的方法生成9个边界框的先验,每个尺寸各预测3个边界框。YOLOv3将图像划分为G×G个网格,则每个尺度预测的张量为G×G×[3*(4+1+80)],其中数字3表示3个预测边界框,数字4表示每个预测边界框包含4个偏移值(t
x,t
y,t
w,t
h),(t
x,t
y,t
w,t
h)为预测边界框中心坐标偏移值和宽、高的偏移值,数字1表示边界框的置信度(confidence score),数字80表示预测边界框属于80类物体条件类别概率P
r(Class
i/Object)。
若边界框中不包含物体,P
r(object)的值为0,置信度为0,否则P
r(object)的值为1,置信度为预测框与真实框之间的交并比。条件类别概率P
r(Class
i/Object)表示在边界框包含目标的前提下,它属于某个类别的概率。YOLOv3联合条件类别概率和边框置信度,得出边界框类别置信度(class specific confidence scores,C)来表示边界框包含目标归类于每个类别的可能性和边界框与目标匹配的好坏,边框类别置信度可表示为:
实施例一:
本实施例提供一种基于YOLOv3多伯努利视频多目标检测跟踪方法,参见图1,所述方法包括:
步骤一:初始化
1.1参数初始化,初始时刻k=0,视频总帧数为N,初始化采样粒子最大数目为L
max,粒子最小数目为L
min,初始目标存在概率P
s=0.99。
1.2目标检测,
采用YOLOv3算法检测第k和k+1帧视频序列,记第k帧检测框个数为n,检测框状态集为
第k+1帧检测框个数为m,其检测框状态集为
其中,
表示第i个检测框状态向量,参数
分别表示第k帧第i个检测框左上角的横坐标、纵坐标,以及检测框的宽、高和标签。
1.3初始帧目标框匹配
其中,
表示第k帧中第i个检测框与第k+1帧中第j个检测框的相似度;从相似度矩阵Λ的每一行选择值最大、且大于相似度阈值T
l的两个检测框作为匹配结果,将最终匹配的检测框对作为初始的新生目标加入目标模板集和目标轨迹集中。
由于视频各帧中目标连续变化,假定同一个目标在相邻两帧不会出现特别大的位移变化,因此,在相似似然匹配的基础上,加入目标框的交并比IOU作为限制:若两个检测框的交并比IOU大于交并比阈值T
u,则可判定两个检测框匹配。从相似度矩阵Λ的每一行选择值最大、且大于相似度阈值T
l,同时交并比IOU大于交并比阈值T
u的两个检测框作为匹配结果,将最终匹配的检测框对作为初始的新生目标,并分配标签
并加入目标轨迹集
中,其中,
为目标状态向量,各个分量分别表示第i个目标框左上角横坐标、纵坐标以及目标框的宽、高和标签,M
k为第k帧目标的个数,给已匹配的检测框对添加与初始的新生目标对应的目标标签,即
同时建立新生目标模板。
步骤二:目标识别
当k>0时,根据相邻帧的检测框、目标模板集和存在目标轨迹集来判定目标是否为新生目标、重识别目标或存在目标。
2.1检测框匹配
计算第k和k+1帧视频序列中检测框之间的相似度矩阵Λ,从相似度矩阵Λ的每一行选择值最大且大于相似度阈值T
l的两个检测框,且这两个检测框的交并比IOU大于交并比阈值T
u,则可判定两个检测框匹配,假设第k帧中的第i个检测框与第k+1帧中的第j个检测框匹配,则为第k+1帧中匹配目标的标签赋值,即:
2.2目标识别
(1)新生目标识别
其中
表示标签为l
i的新生目标状态向量,M
Γ表示新生目标个数,n表示目标模板的个数。给已匹配的检测框对添加与新生目标对应相同的目标标签,即
同时将新生目标加入目标模板集,并根据新生参数初始化新生目标的采样粒子。
(2)漏跟目标重识别
若
为空,与第k+1帧第j个检测框匹配,在存活目标轨迹集中没有与之匹配的目标,但与目标模板集中标签为l
a的目标匹配,表示该检测框包含的目标在之前帧漏检,且出现漏跟情况,在第k帧,对该目标进行了重识别,将其重新加入到存活目标轨迹中:
(3)存活目标识别
(4)干扰杂波识别
步骤三:目标预测
3.1目标运动模型
其中,
为第k-1帧中标签为l
i目标状态向量,
为第k帧中标签为l
i目标的检测框,检测框置信度
表示检测框包含目标的概率和检测框与目标匹配程度的得分,e(k-1)表示零均值高斯白噪声,η为目标状态与对应检测框信息之间的学习率,检测框的置信度越高,η越大,表示越信任检测结果,利用好的检测框去调整目标状态,能消除长时间跟踪而导致的累加误差,可以较好地优化预测结果。
3.2目标预测
在k-1时刻,采用多伯努利参数集
表示多目标的后验概率分布,其中,M
k-1为k-1时刻存在目标的数目,
表示在k-1时刻目标l
i的存在概率,
表示k-1时刻目标l
i的概率分布,由一组加权粒子表示:
多目标概率分布预仍为多伯努利参数集,表示为:
其中,
其中,
其中,M
Γ,k为第k帧新生目标数目。
步骤四:相似度计算
4.1目标卷积特征提取
(1)构造卷积核
在目标框周围,以目标框大小的矩形框随机采样m个背景样本框,背景样本框与目标框中心位置的距离,要求在水平方向距离大于1/4目标框宽度或者在竖直方向距离大于1/4目标框高度,然后将目标框与背景样本框尺寸规范化为n×n,并灰度化图像,得到样本集{I,B
1,B
2,…,B
m},其中I表示目标框图像,B
i为第i个背景样本图像。使用大小为w×w的滑动窗口,以步长Δ分别对样本集图像进行卷积操作,得到目标图像块集合y={Y
1,Y
2,…,Y
l}和背景图像块集合
其中,Y
i∈R
w×w,Z
ij∈R
w×w,l=(n-w+Δ)
2,为保留梯度信息,弱化亮度影响,将所有图像块减去自身均值并二范数归一化处理,最终,使用k-means算法,从目标图像块集合中选出d个图像块作为目标卷积核:
(2)提取卷积特征
在目标特征图上进行背景信息减除:
4.2特征的稀疏表示
将特征图集看作三维的张量C∈R
(n-w+Δ)×(n-w+Δ),对张量进行稀疏化表示,凸显目标的特征,利用稀疏向量f去逼近vec(C),使以下目标函数最小化:
其中,λ是张量C的中位数。
4.3计算相似度
两个目标框的相似度计算公式为:
其中f
1,f
2为对应的目标框卷积特征,在计算粒子与目标的量测似然时,利用目标对应的目标特征图集和背景特征图集来计算粒子对应的卷积特征:
其中I
l为粒子表示的候选框的规范化之后的图像。将S
i按行展开,得到d个一维向量,顺序连接为最终粒子对应候选框的特征f
l。
步骤五:目标状态更新及提取
5.1目标状态更新
假设第k-1帧多目标预测概率分布为:
通过粒子的量测似然更新多目标后验概率分布:
根据下式获得更新后的多伯努利参数集:
5.2目标状态提取
为了防止粒子出现退化,本发明采用随机重采样方式对采样粒子集进行重采样,来避免粒子退化,剔除存在概率过小的伯努利分量。根据更新后的多目标后验概率分布,提取存在概率大于0.5的伯努利分量对应的目标状态.
步骤六:目标遮挡处理机制
当相邻两个目标框的交并比IOU大于阈值T
I时,可判定这两个目标出现紧邻,且部分遮挡,此时,检测器可能出现三种情况:1)当目标没有被完全遮挡时,检测器可能检测出两个目标,本发明方法中提出采用更新机制对目标进行跟踪和模板更新;2)只有一个目标能被检测器检测出来时,可判定另一个目标为被遮挡目标,提出对被遮挡目标,停止模板更新,采用该目标前两帧的位移差估计目标实时速度v与运动方向θ,对目标进行预测估计,目标框的大小保持不变;3)两个目标都无法通过检测器检测出来时,依据目标与模板的相似度判断被遮挡目标,与第二种情况中采用同样的方式处理被遮挡的目标。若目标在跟踪过程中消失或者跟丢,在目标分离或者重新出现时,可根据检测结果与目标模板的匹配对目标进行重识别。
步骤七:目标模板更新
目标运动过程中,周围环境及自身状态会不断发生变化,如背景变化、自身扭曲、旋转及尺度变化等,因此,需要对目标模板进行实时更新,综合考虑采用存活目标轨迹、目标模板和目标检测结果进行融合更新目标模板,即:
其中,
分别表示第k和k-1帧时目标l
i对应的卷积特征模板,
为第k-1帧目标l
i跟踪结果卷积特征的稀疏表示,
为第k帧目标l
i检测框卷积特征的稀疏表示,ρ为模板的学习速率,
为第k帧目标l
i对应检测框的置信度,
为检测框的学习率,采用较好的检测框去更新目标模板,可以有效去除跟踪过程中的累积误差。同时,在运动过程中,目标状态会发生不断变化,采用高准确率的检测框去更新模板,可有效地将目标最新状态加入到模板中,更好地适应目标后续跟踪。
为验证本申请提出的上述基于YOLOv3多伯努利视频多目标检测跟踪方法的效果,特实验如下:
1、实验条件及参数
本发明采用的视频序列数据为Visual Tracker Benchmark TB50中的序列Huamn4,Visual Tracker Benchmark TB100中的序列Jogging、Subway、Suv,以及CAVUAR数据集中的序列EnterExitCrossingPaths1cor,这5组典型视频序列分别来源于不同场景,且包含动态背景、目标紧邻、目标形变、图片模糊、目标尺寸变化、目标遮挡等干扰情况。实验中采用的评价指标为多目标跟踪正确度(Multiple Object Tracking Accuracy,MOTA)、多目标跟踪精度(Multiple Object Tracking Precision,MOTP)、轨迹完整目标数目(Mostly Tracked,MT)、标签跳变数(Identity Switch,IDs),分别定义如下:
1)多目标跟踪正确度(MOTA)
其中,m
t为第t帧中被跟丢的目标数量,fp
t为第t帧误检的目标数,mme
t为第t帧中跟踪轨迹中目标标签发生跳变的数目,g
t表示第t帧中目标的实际个数。
2)多目标跟踪精度(MOTP)
3)轨迹完整目标数目(MT),表示目标跟踪轨迹占真实轨迹长度80%以上的目标轨迹数目,刻画了轨迹的完整程度。
4)标签跳变数(IDs),表示跟踪过程中目标标签发生变化的次数。
2、实验及结果分析
本申请方法采用Matlab2016a实现,在处理器为Intel Core i7-8700、3.2GHz,12核,内存为16GB,显卡为NVIDIA Geforce GTX 1080 Ti的工作站上运行,并与传统的粒子多伯努利滤波(PFMB)方法和2017年Erik Bochinski等在发表论文《High-Speed Tracking-by-Detection Without Using Image Informations》中提出的IOU-Tracker方法进行性能比较与分析。由于传统PFMB方法中,缺少对新生目标检测机制,本实验中也采用YOLOv3进行检测,并将判定为新生目标的检测结果作为PFMB的新生目标,然后进行跟踪。
具体实验从四个方面对发明方法进行性能评估,即:目标重识别、目标紧邻与遮挡、图像模糊与目标形变、目标大位移等,实验结果如下。
实验一:目标重识别
本实验采用的视频序列为CAVUAR数据集中EnterExitCrossingPaths1cor序列,该序列包含383帧图像,存在目标紧邻和较长时间的目标遮挡问题,同时伴随目标逐渐出现导致形变较大的情况。由于较长时间遮挡,很容易导致目标跟丢,本发明融合检测器结果、存在目标轨迹和目标模板能够实现对目标的重识别,将目标重新加入到跟踪轨迹中。
图4给出了EnterExitCrossingPaths1cor序列实验结果,其中,图4(a)为YOLOv3算法检测结果,图4(b)为传统PFMB方法跟踪结果,图4(c)为IOU-Tracker方法跟踪结果,图4(d)为本发明方法的跟踪结果。可以看出,当目标被遮挡时,检测器可能无法检测出被遮挡目标,大幅度降低检测效果,如图4(a)中第93帧,采用传统的PFMB方法,存在遮挡时,目标被跟丢;由于IOU-Tracker完全抛弃使用图像信息,只利用目标检测结果进行跟踪处理,所以该方法也无法继续跟踪漏检目标,当目标再次被检测到时,将会被定义为新目标,难以与历史目标关联;而本发明方法,对于较长时间被遮挡的目标,由于存在概率逐渐降低导致目标消失,但当目标再次出现时,能有效地进行重识别,重新加入到目标跟踪轨迹中。
图5为EnterExitCrossingPaths1cor序列实验目标数目估计对比图,图6为EnterExitCrossingPaths1cor序列实验OSPA距离估计对比图,可以看出,传统的PFMB方法缺少目标遮挡处理机制,在目标被被遮挡后,容易出现误跟和漏跟情况,导致跟踪框偏离较大,OSPA值上升;IOU-Tracker方法受检测器性能影响,目标被遮挡过程中,目标数目减少,OSPA值急剧上升,本发明方法处理长时间被遮挡目标时,也可能出现目标漏跟的情况,当目标脱离遮挡后,能有效地对漏跟目标进行重识别,整体跟踪性能优于传统PFMB方法和IOU-Tracker方法。尤其从表1所示的MOTA、MOTP、MT、IDs四个评价指标可以看出。
表1为50次蒙特卡洛仿真结果,可以看出,传统PFMB方法的指标MOTA、MOTP、MT的值最低,IDs为1,是因为该方法缺少目标重识别过程,导致目标被跟丢后,再重新出现时将无法与之前的轨迹关联上,从而会出现标签跳变情况。IOU-Tracker方法在目标紧邻或遮挡时,目标标签将会出现频繁跳变,导致IDs的值最高。而本发明方法能有效地对目标进行重识别,减少目标标签跳变的问题,有效减少轨迹碎片。
表1目标重识别跟踪性能评价(表中↑表示值越大越好,↓表示值越小越好)
实验二:目标紧邻与遮挡
采用视频序列为Visual Tracker Benchmark TB100数据集中的Jogging序列、Subway序列。Jogging序列为相机移动的路口场景,包含3个运动目标,存在目标被遮挡的情况。Subway序列包含8个目标,存在多个目标紧邻和频繁被遮挡等问题。
Jogging序列的实验结果如图7所示,其中,图7(a)为YOLOv3算法检测结果,图7(b)为传统PFMB方法跟踪结果,图7(c)为IOU-Tracker方法跟踪结果,图7(d)为本发明方法的跟踪结果。可以看出,YOLOv3检测器难以正确检测出被遮挡目标,IOU-Tracker方法丢失漏检目标,当目标脱离遮挡状态时,IOU-Tracker方法将重新检测到的目标定义为新的目标,传统的PFMB方法即使没有丢失目标,但是跟踪框的偏离程度却增大,而本发明的方法能很好地融合目标的检测结果和跟踪结果,实现对跟踪框进行调整,可以获得比较准确的跟踪结果。
图8给出了Jogging序列实验目标数目变化估计对比图,其中传统PFMB方法与本发明方法的目标数目变化一致,图9给出了Jogging序列实验OSPA距离估计对比图,可以看出,YOLOv3检测器存在漏检,导致IOU-Tracker方法目标数目估计不准确,出现漏估计减少,OSPA值急剧上升较大,精度不高;传统PFMB方法在目标被遮挡后,跟踪框的偏离程度增加,导致OSPA值上升增大,而本发明方法具有较好抗遮挡能力,跟踪性能明显优于传统PFMB方法与和IOU-Tracker方法。
Subway序列的实验结果如图10所示,其中,图10(a)为YOLO v3算法检测结果,图10(b)为传统PFMB方法跟踪结果,图10(c)为IOU-Tracker方法跟踪结果,图10(d)为本发明方法的跟踪结果。可以看出,当多个目标邻近或相互遮挡的时候,检测器难以检测到被遮挡目标或者检测结果偏差较大,导致IOU-Tracker方法频繁丢失目标,传统的PFMB方法处理遮挡、紧邻问题时,也会出现较大的偏差,甚至跟丢,如图10(b)中第17、19帧,而本发明方法能够较好地对被遮挡目标进行跟踪。
图11给出Subway序列目标数目估计对比图,图12给出了Subway序列OSPA距离估计对比图。可以看出,由于YOLOv3检测器难以检测到部分出现和被遮挡目标,所以IOU-Tracker方法目标数目变化大,OSPA值较高,传统的PFMB方法在跟踪过程中丢失目标,OSPA值突增,而本发明方法能够较好地处理目标遮挡问题,具有较高的跟踪精度。
进行50次蒙特卡洛仿真,Jogging序列跟踪的量化统计结果如表2.1所示。可以看出,虽然传统PFMB方法与本发明方法的多目标跟踪正确度MOTA相当,但遮挡恢复后,传统PFMB方法的跟踪框会出现偏差,导致跟踪误差偏大,所以MOTP值比本发明方法要小。此外,由于检测器对被遮挡目标检测效果差,且该序列中两个目标一直处于紧邻状态,所以IOU-Tracker方法在对存在紧邻的序列Jogging上跟踪精度较低,且目标标签出现频繁跳变。
Subway序列的定量分析结果如表2.2所示,该序列中,多个目标也长期处于互相遮挡、紧邻状态,本发明方法在MOTA、MOTP、MT、IDs上都明显要优于传统PFMB方法与IOU-Tracker方法。
表2.1 Jogging序列目标紧邻与遮挡
表2.2 Subway序列目标紧邻与遮挡
实验三:图像模糊与目标形变
采用视频序列Visual Tracker Benchmark TB50数据集中的Human4序列,该序列共包含667帧图像,为相机移动红绿灯路口场景,其中,包括三种类型17个目标,视频中存在由于相机运动或目标快速运动等造成的目标模糊情况,且存在许多目标频繁发生形变问题。
图13为Human4序列实验结果,其中,图13(a)为YOLO v3算法检测结果,(b)为传统PFMB方法跟踪结果,(c)为IOU-Tracker方法跟踪结果,(d)为本发明方法的跟踪结果。可以看出,图片模糊可能导致检测器性能下降,部分目标被漏检,如图13(a)中第50帧、第74帧。目标形变对检测器的性能影响较小,虽然,传统PFMB方法能跟上目标,但部分目标的跟踪框会出现偏离,而本发明方法能够较好地处理这两种情况,具有较高的跟踪精度。
图13给出Human4序列目标数目估计对比图,图14给出了Human4序列OSPA距离估计对比图。可以看出,本发明方法能较好的处理图像模糊与目标形变问题,跟踪性能优于传统的PFMB方法和IOU-Tracker方法。
进行50次蒙特卡洛仿真,定量分析结果如表3所示。可以看出,由于频繁的目标形变、图像模糊而导致传统PFMB方法中出现目标漏跟或仅跟踪到目标的部分区域,导致MOTA、MOTP、MT的值都比较低;IOU-Tracker方法能较好地利用检测器的检测结果,所以,MOTP值与本发明方法相当,而目标尺寸变化和图像模糊导致检测器性能下降,存在部分目标被漏检,使得MOTA值偏小,IDs值最大,即存在大量目标标签跳变,航迹估计不准确。本发明方法能较好地对检测框进行关联,有效减少目标标签的跳变,在IDs上明显优于IOU-Tracker。
表3目标模糊与形变跟踪性能评价
实验四:目标大位移
实验中采用视频序列Visual Tracker Benchmark TB100数据集中的Suv序列,该序列共包含400帧图像,为相机移动动态背景的公路场景,包含6个目标,存在快速运动导致的大位移情况。
图16给出了Suv序列实验结果,其中,图16(a)为YOLO v3算法检测结果,图16(b)为传统PFMB方法跟踪结果,图16(c)为IOU-Tracker方法跟踪结果,图16(d)为本发明方法的跟踪结果。可以看出,目标的大位移对检测器没有影响,IOU-Tracker方法表现良好,而传统PFMB方法会出现目标跟丢的情况,本发明方法由于利用检测器的结果去调整跟踪过程,对于较大位移的目标跟踪结果也明显要优于另外两种方法。
图17为Suv序列目标数目估计对比图,图18为Suv序列OSPA距离估计对比图。可以看出,传统的PFMB方法由于缺少对目标大位移的处理机制,所以,当目标位移较大时,可能出现目标丢失或跟踪框偏离较大的情况,导致OSPA值较大,而IOU-Tracker方法与本发明方法能利用检测器结果提高对目标大位移的处理能力,具有较好的跟踪精度。
进行50次蒙特卡洛仿真,定量分析结果如表4所示。对大位移目标进行跟踪,传统的PFMB方法容易出现漏跟、位置偏移过大的情况因此,四个评价指标结果都比较差;由于视频序列也还存在目标遮挡和紧邻等情况,导致IOU-Tracker方法跟踪结果中,目标标签也出现频繁的跳变情况,所以MOTA值偏低,IDs值偏高。而本发明方法能融合检测结果有效地处理目标大位移情况,在各项指标上都明显优于传统PFMB方法,在MOTA,MT,IDs上也明显优于IOU-Tracker方法。
表4目标大位移跟踪性能评价
通过上述实验可知,本申请提供的基于YOLOv3多伯努利视频多目标检测跟踪方法,通过将检测结果与跟踪结果进行交互融合,在目标重识别过程中,对于较长时间被遮挡的目标,当其再次出现时,能有效地进行重识别;而对于存在多个目标紧邻和频繁被遮挡情况时,本申请提出的方法能够融合目标的检测结果和跟踪结果,实现对跟踪框进行调整,获得比较准确的跟踪结果;而对于存在图像模糊与目标形变情况时,本申请提出的方法能够较好地对检测框进行关联,有效减少目标标签的跳变;而对于存在目标大位移情况时,本申请提出的方法能够利用检测器的结果去调整跟踪过程,从而防止出现目标跟丢的情况发生。
本发明实施例中的部分步骤,可以利用软件实现,相应的软件程序可以存储在可读取的存储介质中,如光盘或硬盘等。
以上所述仅为本发明的较佳实施例,并不用以限制本发明,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。
Claims (11)
- 一种多目标检测跟踪方法,其特征在于,所述方法检测跟踪过程中,采用YOLOv3技术检测第k和k+1帧视频序列,记第k帧检测框个数为n,检测框状态集为 第k+1帧检测框个数为m,其检测框状态集为 其中, 表示第i个检测框状态向量,参数 分别表示第k帧第i个检测框左上角的横坐标、纵坐标,以及检测框的宽、高和标签;将第k和k+1帧视频序列的检测框进行匹配;对于视频的初始帧,k=0,将初始帧中已匹配的检测框作为初始的新生目标加入目标模板集和目标轨迹集中;对于视频的中间帧,k>0,利用检测框、目标轨迹信息和目标模板信息实现新生目标判定、漏跟目标重识别和存活目标优化跟踪;其中,存活目标优化跟踪是在多伯努利滤波框架下,利用当前帧置信度大于给定置信度阈值T b的检测框信息,优化对应目标的跟踪过程。
- 根据权利要求2所述的方法,其特征在于,将初始两帧的检测框进行匹配,并判定新生目标,将最终匹配的检测框对作为初始的新生目标加入目标模板集中,包括:假定同一个目标在相邻两帧不会出现特别大的位移变化,在相似度匹配的基础上,加入目标框的交并比IOU作为限制:若两个检测框的交并比IOU大于交并比阈值T u,则可判定两个检测框匹配;
- 根据权利要求3所述的方法,其特征在于,所述对于视频的中间帧,k>0,利用检测框、目标轨迹信息和目标模板信息来实现新生目标判定、漏跟目标重识别和存活目标优化跟踪,包括:对相邻的检测框进行匹配,确定新生目标与重识别目标的判定条件,再根据相邻帧的检测框、目标模板集和存活目标轨迹集来判定目标是否为新生目标、重识别目标或存活目标。
- 根据权利要求5所述的方法,其特征在于,所述对于当前帧检测框目标,根据相邻帧的检测框、目标模板集和存活目标轨迹集来进行目标识别,包括:(1)新生目标识别其中 表示标签为l i的新生目标状态向量,M Γ表示新生目标个数,n表示目标模板的个数;给已匹配的检测框对添加与新生目标对应相同的目标标签,即 同时将新生目标加入目标模板集,并根据新生参数初始化新生目标的采样粒子;(2)漏跟目标重识别若 为空,与第k+1帧第j个检测框匹配,在存活目标轨迹集中没有与之匹配的目标,但与目标模板集中标签为l a的目标匹配,表示该检测框包含的目标在之前帧漏检,且出现漏跟情况,在第k帧,对该目标进行了重识别,将其重新加入到存活目标轨迹集中:(3)存活目标识别(4)干扰杂波识别
- 根据权利要求7所述的方法,其特征在于,在k-1时刻,采用多伯努利参数集 表示多目标的后验概率分布,其中,M k-1为k-1时刻存在目标的数目, 表示在k-1时刻目标l i的存在概率, 表示k-1时刻目标l i的概率分布,由一组加权粒子表示:多目标概率分布预仍为多伯努利参数集,表示为:其中,其中,其中,M Γ,k为第k帧新生目标数目。
- 根据权利要求10所述的方法,其特征在于,检测跟踪过程中,当相邻两个目标框的交并比IOU大于阈值T u时,判定这两个目标出现遮挡情况;此时,(1)当检测器能检测出这两个目标,表示目标轻度遮挡,对目标模板进行自适应更新;(2)当只有一个目标能被检测器检测出来时,可判定另一个目标为被遮挡目标,对被遮挡目标,停止模板更新,采用该目标前两帧的位移差,估计目标实时速度v与运动方向θ,对目标进行预测,目标框的大小保持不变;(3)当两个目标都无法通过检测器检测出来时,依据目标框与模板的相似度判断被遮挡目标,与第(2)种情况中采用同样的方式处理被遮挡的目标;若目标在跟踪过程中消失或者跟丢,在目标分离或者重新出现时,根据检测结果与目标模板的匹配对目标进行重识别。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/869,645 US11094070B2 (en) | 2019-04-23 | 2020-05-08 | Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910328735.X | 2019-04-23 | ||
CN201910328735.XA CN110084831B (zh) | 2019-04-23 | 2019-04-23 | 基于YOLOv3多伯努利视频多目标检测跟踪方法 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/869,645 Continuation US11094070B2 (en) | 2019-04-23 | 2020-05-08 | Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020215492A1 true WO2020215492A1 (zh) | 2020-10-29 |
Family
ID=67416267
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/094662 WO2020215492A1 (zh) | 2019-04-23 | 2019-07-04 | 基于YOLOv3多伯努利视频多目标检测跟踪方法 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110084831B (zh) |
WO (1) | WO2020215492A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113205544A (zh) * | 2021-04-27 | 2021-08-03 | 武汉大学 | 基于交并比估计的空间注意力强化学习跟踪方法 |
Families Citing this family (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110472588B (zh) * | 2019-08-19 | 2020-11-24 | 上海眼控科技股份有限公司 | 锚点框确定方法、装置、计算机设备和存储介质 |
CN110647818A (zh) * | 2019-08-27 | 2020-01-03 | 北京易华录信息技术股份有限公司 | 一种遮挡目标物体的识别方法及装置 |
CN112465859A (zh) * | 2019-09-06 | 2021-03-09 | 顺丰科技有限公司 | 快速运动目标的检测方法、装置、设备和储存介质 |
CN110765363B (zh) * | 2019-09-27 | 2023-05-05 | 复旦大学 | 一种基于高斯分布表示的深度推荐*** |
CN110780164B (zh) * | 2019-11-04 | 2022-03-25 | 华北电力大学(保定) | 基于yolo的绝缘子红外故障定位诊断方法及装置 |
CN111107427B (zh) * | 2019-11-20 | 2022-01-28 | Oppo广东移动通信有限公司 | 图像处理的方法及相关产品 |
CN112850436A (zh) * | 2019-11-28 | 2021-05-28 | 宁波微科光电股份有限公司 | 一种电梯智能光幕的行人趋势检测方法及*** |
CN111126235B (zh) * | 2019-12-18 | 2023-06-16 | 浙江大华技术股份有限公司 | 一种船只违章停泊检测处理方法及装置 |
CN111079694A (zh) * | 2019-12-28 | 2020-04-28 | 神思电子技术股份有限公司 | 一种柜面助手履职监控装置和方法 |
CN111274917B (zh) * | 2020-01-17 | 2023-07-18 | 江南大学 | 一种基于深度检测的长时目标跟踪方法 |
CN111292355B (zh) * | 2020-02-12 | 2023-06-16 | 江南大学 | 一种融合运动信息的核相关滤波多目标跟踪方法 |
CN111340855A (zh) * | 2020-03-06 | 2020-06-26 | 电子科技大学 | 一种基于轨迹预测的道路移动目标检测方法 |
CN111402295B (zh) * | 2020-03-11 | 2023-08-08 | 桂林理工大学 | 基于对象检测和跟踪的移动物体识别方法 |
CN111524095A (zh) * | 2020-03-24 | 2020-08-11 | 西安交通大学 | 一种用于旋转物体的目标检测方法 |
CN111754545B (zh) * | 2020-06-16 | 2024-05-03 | 江南大学 | 一种基于iou匹配的双滤波器视频多目标跟踪方法 |
CN111797802B (zh) * | 2020-07-14 | 2023-06-02 | 华侨大学 | 一种基于ai视觉的扶梯不安全行为实时预警方法 |
CN111680689B (zh) * | 2020-08-11 | 2021-03-23 | 武汉精立电子技术有限公司 | 一种基于深度学习的目标检测方法、***及存储介质 |
US11889227B2 (en) | 2020-10-05 | 2024-01-30 | Samsung Electronics Co., Ltd. | Occlusion processing for frame rate conversion using deep learning |
CN112507874B (zh) * | 2020-12-10 | 2023-02-21 | 上海芯翌智能科技有限公司 | 一种用于检测机动车加塞行为的方法与设备 |
CN112529942B (zh) * | 2020-12-22 | 2024-04-02 | 深圳云天励飞技术股份有限公司 | 多目标跟踪方法、装置、计算机设备及存储介质 |
CN112329893A (zh) * | 2021-01-04 | 2021-02-05 | 中国工程物理研究院流体物理研究所 | 基于数据驱动的异源多目标智能检测方法及*** |
CN112967320B (zh) * | 2021-04-02 | 2023-05-30 | 浙江华是科技股份有限公司 | 一种基于桥梁防撞的船舶目标检测跟踪方法 |
CN113139468B (zh) * | 2021-04-24 | 2023-04-11 | 西安交通大学 | 融合局部目标特征与全局特征的视频摘要生成方法 |
CN113537077B (zh) * | 2021-07-19 | 2023-05-26 | 江苏省特种设备安全监督检验研究院 | 基于特征池优化的标签多伯努利视频多目标跟踪方法 |
CN113674306A (zh) * | 2021-07-29 | 2021-11-19 | 杭州宇泛智能科技有限公司 | 基于鱼眼镜头的行人轨迹获取方法、***、装置和介质 |
CN113421288B (zh) * | 2021-08-23 | 2021-12-17 | 杭州云栖智慧视通科技有限公司 | 一种多目标实时轨迹跟踪中的静止轨迹碎片改进方法 |
CN113920164B (zh) * | 2021-10-27 | 2024-05-24 | 浙江工商大学 | 一种剧场环境下基于近红外防伪油墨的演员身份重识别方法 |
CN113724298B (zh) * | 2021-11-01 | 2022-03-18 | 深圳市城市交通规划设计研究中心股份有限公司 | 多点位感知融合方法及装置、计算机可读存储介质 |
CN114092515B (zh) * | 2021-11-08 | 2024-03-05 | 国汽智控(北京)科技有限公司 | 用于障碍遮挡的目标跟踪检测方法、装置、设备及介质 |
CN113989332B (zh) * | 2021-11-16 | 2022-08-23 | 苏州魔视智能科技有限公司 | 一种目标跟踪方法、装置、存储介质及电子设备 |
CN114972418B (zh) * | 2022-03-30 | 2023-11-21 | 北京航空航天大学 | 基于核自适应滤波与yolox检测结合的机动多目标跟踪方法 |
CN115471773B (zh) * | 2022-09-16 | 2023-09-15 | 北京联合大学 | 一种面向智慧教室的学生跟踪方法及*** |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103176164A (zh) * | 2013-04-11 | 2013-06-26 | 北京空间飞行器总体设计部 | 基于无线传感器网络的多目标无源跟踪方法 |
CN105354860A (zh) * | 2015-08-26 | 2016-02-24 | 西安电子科技大学 | 基于箱粒子滤波的扩展目标CBMeMBer跟踪方法 |
CN106408594A (zh) * | 2016-09-28 | 2017-02-15 | 江南大学 | 基于多伯努利特征协方差的视频多目标跟踪方法 |
CN106910205A (zh) * | 2017-03-03 | 2017-06-30 | 深圳市唯特视科技有限公司 | 一种基于随机有限集滤波器耦合的多目标跟踪方法 |
CN109002783A (zh) * | 2018-07-02 | 2018-12-14 | 北京工业大学 | 救援环境中的人体检测以及姿态识别方法 |
CN109508444A (zh) * | 2018-12-18 | 2019-03-22 | 桂林电子科技大学 | 区间量测下交互式多模广义标签多伯努利的快速跟踪方法 |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104766320B (zh) * | 2015-04-02 | 2017-06-13 | 西安电子科技大学 | 阈值化量测下的多伯努利滤波弱目标检测与跟踪方法 |
CN105182291B (zh) * | 2015-08-26 | 2017-08-25 | 西安电子科技大学 | 自适应目标新生强度的phd平滑器的多目标跟踪方法 |
US10242266B2 (en) * | 2016-03-02 | 2019-03-26 | Mitsubishi Electric Research Laboratories, Inc. | Method and system for detecting actions in videos |
CN108876819A (zh) * | 2018-06-11 | 2018-11-23 | 深圳市唯特视科技有限公司 | 一种基于泊松多伯努利滤波的三维多目标追踪算法 |
CN109325407B (zh) * | 2018-08-14 | 2020-10-09 | 西安电子科技大学 | 基于f-ssd网络滤波的光学遥感视频目标检测方法 |
CN109118523B (zh) * | 2018-09-20 | 2022-04-22 | 电子科技大学 | 一种基于yolo的图像目标跟踪方法 |
CN109598742A (zh) * | 2018-11-27 | 2019-04-09 | 湖北经济学院 | 一种基于ssd算法的目标跟踪方法及*** |
-
2019
- 2019-04-23 CN CN201910328735.XA patent/CN110084831B/zh active Active
- 2019-07-04 WO PCT/CN2019/094662 patent/WO2020215492A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103176164A (zh) * | 2013-04-11 | 2013-06-26 | 北京空间飞行器总体设计部 | 基于无线传感器网络的多目标无源跟踪方法 |
CN105354860A (zh) * | 2015-08-26 | 2016-02-24 | 西安电子科技大学 | 基于箱粒子滤波的扩展目标CBMeMBer跟踪方法 |
CN106408594A (zh) * | 2016-09-28 | 2017-02-15 | 江南大学 | 基于多伯努利特征协方差的视频多目标跟踪方法 |
CN106910205A (zh) * | 2017-03-03 | 2017-06-30 | 深圳市唯特视科技有限公司 | 一种基于随机有限集滤波器耦合的多目标跟踪方法 |
CN109002783A (zh) * | 2018-07-02 | 2018-12-14 | 北京工业大学 | 救援环境中的人体检测以及姿态识别方法 |
CN109508444A (zh) * | 2018-12-18 | 2019-03-22 | 桂林电子科技大学 | 区间量测下交互式多模广义标签多伯努利的快速跟踪方法 |
Non-Patent Citations (1)
Title |
---|
WANG, DIANWEI ET AL.: "An improved infrared video image pedestrian detection algorithm", JOURNAL OFXI'AN UNIVERSITY OF POSTS AND TELECOMMUNICATIONS, vol. 23, no. 4, 31 July 2018 (2018-07-31), pages 48 - 52 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113205544A (zh) * | 2021-04-27 | 2021-08-03 | 武汉大学 | 基于交并比估计的空间注意力强化学习跟踪方法 |
CN113205544B (zh) * | 2021-04-27 | 2022-04-29 | 武汉大学 | 基于交并比估计的空间注意力强化学习跟踪方法 |
Also Published As
Publication number | Publication date |
---|---|
CN110084831A (zh) | 2019-08-02 |
CN110084831B (zh) | 2021-08-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020215492A1 (zh) | 基于YOLOv3多伯努利视频多目标检测跟踪方法 | |
US11094070B2 (en) | Visual multi-object tracking based on multi-Bernoulli filter with YOLOv3 detection | |
Maksai et al. | Non-markovian globally consistent multi-object tracking | |
CN107563313B (zh) | 基于深度学习的多目标行人检测与跟踪方法 | |
Jana et al. | YOLO based Detection and Classification of Objects in video records | |
WO2023065395A1 (zh) | 作业车辆检测与跟踪方法和*** | |
Wojek et al. | Monocular visual scene understanding: Understanding multi-object traffic scenes | |
Wang et al. | Robust video-based surveillance by integrating target detection with tracking | |
CN111127513A (zh) | 一种多目标跟踪方法 | |
CN110288627B (zh) | 一种基于深度学习和数据关联的在线多目标跟踪方法 | |
CN106778712B (zh) | 一种多目标检测与跟踪方法 | |
EP2131328A2 (en) | Method for automatic detection and tracking of multiple objects | |
CN111932583A (zh) | 一种基于复杂背景下的时空信息一体化智能跟踪方法 | |
CN103971386A (zh) | 一种动态背景场景下的前景检测方法 | |
CN110532921B (zh) | 基于ssd检测广义标签多伯努利视频多目标跟踪方法 | |
KR20190023389A (ko) | 변화점 검출을 활용한 다중클래스 다중물체 추적 방법 | |
Dai et al. | Instance segmentation enabled hybrid data association and discriminative hashing for online multi-object tracking | |
CN106780567B (zh) | 一种融合颜色和梯度直方图的免疫粒子滤波扩展目标跟踪方法 | |
Yang et al. | A probabilistic framework for multitarget tracking with mutual occlusions | |
Li et al. | Robust object tracking via multi-feature adaptive fusion based on stability: contrast analysis | |
He et al. | Fast online multi-pedestrian tracking via integrating motion model and deep appearance model | |
CN114926859A (zh) | 一种结合头部跟踪的密集场景下行人多目标跟踪方法 | |
CN114627339B (zh) | 茂密丛林区域对越境人员的智能识别跟踪方法及存储介质 | |
Vo et al. | Object tracking: an experimental and comprehensive study on vehicle object in video | |
Lu et al. | Hybrid deep learning based moving object detection via motion prediction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19925790 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19925790 Country of ref document: EP Kind code of ref document: A1 |