CN109919043B - Pedestrian tracking method, device and equipment - Google Patents
Pedestrian tracking method, device and equipment Download PDFInfo
- Publication number
- CN109919043B CN109919043B CN201910119437.XA CN201910119437A CN109919043B CN 109919043 B CN109919043 B CN 109919043B CN 201910119437 A CN201910119437 A CN 201910119437A CN 109919043 B CN109919043 B CN 109919043B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- candidate
- queue
- feature
- period
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a pedestrian tracking method, a pedestrian tracking device and pedestrian tracking equipment, wherein the method comprises the following steps: acquiring a video frame to be detected; detecting candidate pedestrians in a video frame to be detected; extracting pedestrian candidate characteristics of the pedestrian candidate; determining the difference between the features of the candidate pedestrians and the features stored in the feature queue, and determining the candidate pedestrians as target pedestrians when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian. The efficiency of the pedestrian tracking process can be improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a pedestrian tracking method, a pedestrian tracking device and pedestrian tracking equipment.
Background
Target tracking, such as pedestrian tracking, is an important aspect in the field of computer vision, and has wide application prospects in the fields of Artificial Intelligence (AI), video monitoring, human-computer interaction, robots, military guidance and the like.
In one existing approach, pedestrian tracking is accomplished through deep learning. Taking a Multi-Domain Convolutional Neural network (MDNet) as an example, using Multi-Domain training, using Convolutional Neural Network (CNN) features, using full connectivity for online fine tuning, and finally determining that a candidate frame with the maximum probability is a predicted target frame by using a process of selecting a candidate target frame set and combining a process of judging the probability that each target frame is a target.
However, the inventor finds that the prior art has at least the following problems in the process of implementing the invention:
in the deep learning mode, a large amount of convolution and full-connection operations exist in the neural network, and the computation amount consumed by the convolution and full-connection operations is large, so that the computation time is long. So, it is lower to realize pedestrian's tracking efficiency through the mode of deep learning.
Disclosure of Invention
The embodiment of the invention aims to provide a pedestrian tracking method, a pedestrian tracking device and pedestrian tracking equipment so as to improve the efficiency of a pedestrian tracking process. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a pedestrian tracking method, including:
acquiring a video frame to be detected;
detecting candidate pedestrians in the video frame to be detected;
extracting pedestrian candidate characteristics of the pedestrian candidate;
determining the difference between the features of the candidate pedestrians and the features stored in the feature queue, and determining the candidate pedestrians as target pedestrians when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian.
Optionally, the feature queue includes a short cycle queue and a long cycle queue;
the determining the difference between the candidate pedestrian feature and the feature stored in the feature queue, and determining the candidate pedestrian as the target pedestrian when the difference meets the preset condition includes:
when at least one of the queue length of the short-period queue reaching a first preset queue length and the queue length of the long-period queue reaching a second preset queue length is not met, determining a first difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue, and determining the candidate pedestrian as a target pedestrian when the first difference meets a first preset condition; wherein the length of the first preset queue is smaller than the length of the second preset queue;
and when the queue length of the short-period queue reaches the first preset queue length and the queue length of the long-period queue reaches the second preset queue length, determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue, and determining that the candidate pedestrian is a target pedestrian when the second difference meets a second preset condition.
Optionally, when the first difference satisfies a first preset condition, determining that the candidate pedestrian is the target pedestrian includes:
and when the first difference corresponding to the candidate pedestrian feature is smaller than a preset difference threshold value, and the number of the candidate pedestrians corresponding to the candidate pedestrian feature reaches a preset number threshold value, determining that the candidate pedestrian is the target pedestrian.
Optionally, the determining a first difference between the pedestrian candidate feature and a feature already stored in the short-period queue includes:
and calculating Euclidean distance between the pedestrian candidate feature and each saved feature in the short-period queue, and taking the Euclidean distance as a first difference between the pedestrian candidate feature and the saved feature.
Optionally, the method further includes:
and when the Euclidean distance corresponding to the candidate pedestrian features is smaller than a preset distance threshold value and reaches the preset number threshold value, adding the candidate pedestrian features to the short-period queue.
Optionally, the determining a second difference between the pedestrian feature candidate and the stored features in the short-period queue and the long-period queue includes:
determining a reconstruction error of the pedestrian candidate feature based on the constructed codebook matrix through a local constraint linear coding (LLC) algorithm, and taking the reconstruction error as a second difference between the pedestrian candidate feature and the features stored in the short-period queue and the long-period queue; the codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate feature and the codebook matrix.
Optionally, when the second difference satisfies a second preset condition, determining that the candidate pedestrian is the target pedestrian includes:
determining the minimum reconstruction error in the reconstruction errors respectively corresponding to the candidate pedestrians;
and when the minimum reconstruction error meets a maximum allowable reconstruction error threshold, determining that the candidate pedestrian corresponding to the minimum reconstruction error is the target pedestrian.
Optionally, after determining a reconstruction error between the pedestrian candidate feature and the constructed codebook matrix through a locally constrained linear coding LLC algorithm, the method further includes:
and when the reconstruction error meets a third preset condition, popping up a first saved feature in the short-period queue, and adding the pedestrian candidate feature to the short-period queue.
Optionally, the method further includes:
and when the number of the pedestrian feature candidates added to the short-period queue reaches the updated number of the long-period queue, selecting one pedestrian feature candidate meeting an adding condition from the pedestrian feature candidates added to the short-period queue and adding the selected pedestrian feature candidate to the long-period queue.
In a second aspect, an embodiment of the present invention provides a pedestrian tracking apparatus, including:
the acquisition module is used for acquiring a video frame to be detected;
the detection module is used for detecting candidate pedestrians in the video frame to be detected;
the extraction module is used for extracting pedestrian candidate characteristics of the pedestrian candidate;
the determining module is used for determining the difference between the features of the candidate pedestrian and the features stored in the feature queue, and determining the candidate pedestrian as a target pedestrian when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian.
Optionally, the feature queue includes a short cycle queue and a long cycle queue;
the determining module includes:
a first determining sub-module, configured to determine a first difference between the feature of the pedestrian candidate and a feature already stored in the short-period queue when at least one of a length of the short-period queue reaches a first preset queue length and a length of the long-period queue reaches a second preset queue length is not satisfied, and determine that the pedestrian candidate is a target pedestrian when the first difference satisfies a first preset condition; wherein the length of the first preset queue is smaller than the length of the second preset queue;
and the second determining submodule is used for determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue when the queue length of the short-period queue reaches the first preset queue length and the queue length of the long-period queue reaches the second preset queue length, and determining the candidate pedestrian as a target pedestrian when the second difference meets a second preset condition.
Optionally, the first determining sub-module is specifically configured to determine that the pedestrian candidate corresponding to the pedestrian candidate feature is the target pedestrian when the number of the first difference corresponding to the pedestrian candidate feature, which is smaller than a preset difference threshold, reaches a preset number threshold.
Optionally, the first determining sub-module is specifically configured to calculate, for each saved feature in the short-period queue, an euclidean distance between the pedestrian candidate feature and the saved feature, and use the euclidean distance as a first difference between the pedestrian candidate feature and the saved feature.
Optionally, the apparatus further comprises:
the first adding module is used for adding the pedestrian feature candidates to the short-period queue when the Euclidean distance corresponding to the pedestrian feature candidates is smaller than a preset distance threshold and the number of the pedestrian feature candidates reaches the preset number threshold.
Optionally, the second determining sub-module is specifically configured to determine, through a locally constrained linear coding LLC algorithm, a reconstruction error of the pedestrian candidate feature based on the constructed codebook matrix, and use the reconstruction error as a second difference between the pedestrian candidate feature and the features already stored in the short-period queue and the long-period queue; the codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate feature and the codebook matrix.
Optionally, the second determining sub-module is specifically configured to determine a minimum reconstruction error of reconstruction errors corresponding to each pedestrian candidate; and when the minimum reconstruction error meets a maximum allowable reconstruction error threshold, determining that the candidate pedestrian corresponding to the minimum reconstruction error is the target pedestrian.
Optionally, the apparatus further comprises:
the popping module is used for popping a first saved feature in the short-period queue when the reconstruction error meets a third preset condition;
a second adding module for adding the pedestrian candidate feature to the short-cycle queue.
Optionally, the apparatus further comprises:
and the third adding module is used for selecting one candidate pedestrian feature meeting an adding condition from the candidate pedestrian features added to the short-period queue to be added to the long-period queue when the number of the candidate pedestrian features added to the short-period queue reaches the updated number of the long-period queue.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the method steps of the first aspect when executing the program stored in the memory.
In yet another aspect of the present invention, there is also provided a computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to perform the method steps of the first aspect described above.
In yet another aspect of the present invention, the present invention further provides a computer program product containing instructions, which when executed on a computer, causes the computer to perform the method steps of the first aspect.
The pedestrian tracking method, the device and the equipment provided by the embodiment of the invention can acquire the video frame to be detected; detecting candidate pedestrians in a video frame to be detected; extracting pedestrian candidate characteristics of the pedestrian candidate; determining the difference between the features of the candidate pedestrians and the features stored in the feature queue, and determining the candidate pedestrians as target pedestrians when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian. In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1(a) is a flow chart of a pedestrian tracking method according to an embodiment of the present invention;
FIG. 1(b) is a schematic flow chart of the determination of a target pedestrian according to the embodiment of the invention;
FIG. 2 is a diagram illustrating two queues according to an embodiment of the present invention;
FIG. 3 is another flow chart of a pedestrian tracking method provided by an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a pedestrian tracking apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
Pedestrian tracking is the basis of various algorithms, and a stable and efficient pedestrian tracking method can effectively improve the efficiency of algorithms such as AI (artificial intelligence) and the like. At present, one of the mainstream pedestrian tracking methods is to perform pedestrian tracking in a deep learning manner. However, the deep learning method requires a neural network, and the neural network has a large number of operations such as convolution and full connection, and the operations such as convolution and full connection consume a large amount of computation, so that the computation time is also long. So, it is lower to realize pedestrian's tracking efficiency through the mode of deep learning.
The embodiment of the invention provides a pedestrian tracking method, which can store characteristics matched with a target pedestrian through a characteristic queue. Detecting candidate pedestrians in a video frame to be detected; extracting pedestrian candidate characteristics of the pedestrian candidate; determining the difference between the features of the candidate pedestrians and the features stored in the feature queue, and determining the candidate pedestrians as target pedestrians when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian. In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
The pedestrian tracking method provided by the embodiment of the invention is explained in detail below.
The pedestrian tracking method provided by the embodiment of the invention can be applied to electronic equipment. In particular, the electronic device may include a terminal, a server, a processor, and the like.
An embodiment of the present invention provides a pedestrian tracking method, as shown in fig. 1(a), including:
s101, acquiring a video frame to be detected.
Pedestrian tracking is to locate a pedestrian in each video frame, and when the video frame comprises a plurality of candidate pedestrians, determine which candidate pedestrian is a target pedestrian to be located and tracked.
The video of the pedestrian in the motion process can be shot, and the video frame included in the video can be the video frame to be detected. The electronic device can shoot the video through the image acquisition device, or the electronic device comprises an image acquisition module and the video is shot through image acquisition.
And S102, detecting the candidate pedestrian in the video frame to be detected.
Pedestrian detection frames, such as rectangular frames or the like, each including each of the pedestrian candidates may be determined.
And S103, extracting pedestrian candidate characteristics of the pedestrian candidate.
The pedestrian candidate feature may include a histogram feature, a color feature, an LBP (Local Binary Pattern) feature, and the like. The pedestrian candidate feature is not limited in the embodiment of the invention, and any feature that can describe the pedestrian candidate is within the protection scope of the embodiment of the invention. Specifically, the pedestrian feature candidate for extracting the pedestrian candidate may refer to the existing pedestrian feature extraction manner, and will not be described herein again.
In one implementation, the pedestrian candidate feature may be represented by a feature vector. Specifically, a pedestrian feature candidate vector representing the pedestrian feature candidate may be extracted.
In an optional embodiment of the present invention, a distance between each candidate pedestrian in a video frame to be detected and a position of a target pedestrian in a previous video frame may be calculated, specifically, a detection frame 1 of the target pedestrian in the previous video frame may be determined, and a center position coordinate of the detection frame 1 may be determined, for example, the center position coordinate may be determined as coordinate 1; determining a detection frame 2 of a candidate pedestrian in a video frame to be detected, and determining the coordinates of the center position of the detection frame 2, if the coordinates can be determined to be 2, the video frames can be understood to be located in the same coordinate system, so that the distance between the coordinates 1 and 2 is calculated, and the distance is the calculated distance between the candidate pedestrian in the video frame to be detected and the target pedestrian in the previous video frame. Specifically, the euclidean distance between the coordinates 1 and 2, and the like can be calculated.
Then, the pedestrian candidates are ranked according to the order of the distances from small to large, and the pedestrian candidate features of the pedestrian candidates are respectively extracted according to the ranked pedestrian candidates.
And S104, determining the difference between the features of the candidate pedestrian and the features stored in the feature queue, and determining the candidate pedestrian as the target pedestrian when the difference meets a preset condition.
Wherein the stored features in the feature queue are features matched with the target pedestrian. Specifically, it may be a feature vector of features matching the target pedestrian.
The number of feature queues may be one, two, or more.
The pedestrian feature candidate may be matched with the stored features in the feature queue, and a degree of matching of the pedestrian feature candidate with the stored features in the feature queue may be determined. Specifically, the degree of matching between the pedestrian feature candidate and the stored feature in the short-period queue can be reflected by the difference between the pedestrian feature candidate and the stored feature in the feature queue. Specifically, the difference between the pedestrian feature candidate and the stored feature may be represented by calculating the euclidean distance, manhattan distance, chebyshev distance, cosine of included angle, correlation coefficient, correlation distance, and the like of the pedestrian feature candidate and the stored feature.
In an implementation manner, the difference meeting the preset condition may be that the difference meets a preset threshold, and the preset threshold may be determined according to an actual situation.
In another implementation manner, the difference meeting the preset condition may be that the number of differences between the candidate pedestrian feature and each stored feature in the feature queue reaching the preset threshold reaches a preset number. Wherein, reaching the preset threshold value may include being less than or not less than the preset threshold value. Reaching the preset number may include being greater than or less than the preset number. If the predetermined number is 7, the feature queue includes 10 saved features, and 10 differences between the candidate pedestrian features and the 10 saved features are determined, and if 7 or more than 7 of the 10 differences are smaller than a predetermined threshold, it may be determined that the differences satisfy a predetermined condition. At this time, the pedestrian candidate corresponding to the pedestrian candidate feature may be determined as the target pedestrian.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
In an alternative embodiment of the present invention, the feature queue includes a short cycle queue and a long cycle queue.
Step S104: determining the difference between the pedestrian feature candidate and the stored feature in the feature queue, and determining the pedestrian candidate as the target pedestrian when the difference satisfies a preset condition, as shown in fig. 1(b), which may include:
s1041, when at least one of the length of the short-period queue reaches a first preset queue length and the length of the long-period queue reaches a second preset queue length is not satisfied, determining a first difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue, and determining the candidate pedestrian as a target pedestrian when the first difference satisfies a first preset condition.
The queue length of the short cycle queue is less than the queue length of the long cycle queue. It is to be understood that the first predetermined queue length is less than the second predetermined queue length.
Wherein the stored features in the short-period queue and the long-period queue are features matched with the target pedestrian. Specifically, it may be a feature vector matching the target pedestrian.
The pedestrian feature candidates may be respectively matched with the stored features of the short-period queue, and a degree of matching between the pedestrian feature candidates and the stored features in the short-period queue may be determined. Specifically, the degree of matching between the pedestrian feature candidate and the stored feature in the short-period queue may be reflected by the difference between the pedestrian feature candidate and the stored feature in the short-period queue.
In one implementation, a first difference between a candidate pedestrian feature and a stored feature in a short-period queue may be determined; and when the number of the first differences corresponding to the candidate pedestrian features smaller than the preset difference threshold reaches the preset number threshold, determining that the candidate pedestrian corresponding to the candidate pedestrian features is the target pedestrian.
The preset difference threshold and the preset number threshold can be determined according to actual requirements.
The first difference between the pedestrian feature candidate and the stored feature may be represented by calculating a euclidean distance, a manhattan distance, a chebyshev distance, an included angle cosine, a correlation coefficient, a correlation distance, and the like of the pedestrian feature candidate and the stored feature.
In an alternative embodiment of the present invention, for simplicity and convenience of calculation, the first difference between the pedestrian candidate feature and the saved feature may be represented by a euclidean distance between the pedestrian candidate feature and the saved feature.
Specifically, for each saved feature in the short-period queue, the euclidean distance between the pedestrian candidate feature and the saved feature is calculated, and the euclidean distance is taken as a first difference between the pedestrian candidate feature and the saved feature.
And when the Euclidean distance between the candidate pedestrian feature and the stored feature meets a first preset condition, determining that the candidate pedestrian is the target pedestrian.
In one implementation manner, when a number of first differences corresponding to the pedestrian candidate features, that is, euclidean distances between the pedestrian candidate features and the stored features, which are smaller than a preset difference threshold reaches a preset number threshold, it is determined that the pedestrian candidate corresponding to the pedestrian candidate features is the target pedestrian.
The preset distance threshold and the preset number threshold can be determined according to actual requirements.
In an alternative implementation manner, the preset number threshold may be half of the number of currently saved features included in the short-period queue, and the like. And if the Euclidean distance between the pedestrian candidate features and each stored feature in the short-period queue is less than a preset distance threshold value and the number of the stored features reaches half of the number of the current features, determining the pedestrian candidate corresponding to the pedestrian candidate features as the target pedestrian.
In another implementation manner, when the euclidean distance between the pedestrian candidate feature and each stored feature is greater than the preset distance threshold, the pedestrian candidate corresponding to the pedestrian candidate feature is determined as the target pedestrian. For example, when the number of the stored features is 10, and the euclidean distance between the pedestrian candidate feature and each stored feature in the short-period queue is only 1, if the number of the features is greater than the preset distance threshold, the pedestrian candidate corresponding to the pedestrian candidate feature may be determined to be the target pedestrian.
Compared with the deep learning mode, taking MDNet as an example, multi-domain training is used, CNN features are used, full connection is used for online fine tuning, a candidate target frame set is selected, the probability that each target frame is a target is judged, and the final candidate frame with the maximum probability is the predicted target frame. Because of a large amount of volume and full connection operations in the neural network, the computation amount of Processing one frame by the MDNet is about 1 × 10^9, the computation amount is parallelly computed on a Graphics Processing Unit (GPU) nvidia 1080 display card, and the time for Processing one frame is more than 1.2 seconds, but the method provided by the embodiment of the invention only needs to compute the Euclidean distance between each pedestrian feature and each element of the feature queue, takes the feature length 128 as an example, the computation amount is about 2 × 10^6, and in the experiment, the computation amount is serially computed on a Central Processing Unit (CPU) of an inter i5, and the time for Processing one frame is about 0.005 seconds. Therefore, the method provided by the embodiment of the invention can reduce the time consumed in the pedestrian tracking process and improve the efficiency of the pedestrian tracking process.
And S1042, when the queue length of the short-period queue reaches a first preset queue length and the queue length of the long-period queue reaches a second preset queue length, determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue, and when the second difference meets a second preset condition, determining that the candidate pedestrian is the target pedestrian.
Wherein the stored features in the short-period queue and the long-period queue are features matched with the target pedestrian.
The stored features in the short-period queue and the long-period queue may be combined, the pedestrian candidate feature may be matched with all the stored features in the short-period queue and the long-period queue, respectively, to determine the degree of matching between the pedestrian candidate feature and all the stored features in the short-period queue and the long-period queue, and specifically, the degree of matching between the pedestrian candidate feature and the stored features in the short-period queue and the long-period queue may be reflected by a difference between the pedestrian candidate feature and the stored features in the short-period queue and the long-period queue.
Specifically, the second difference between the pedestrian candidate feature and the stored feature may be reflected by calculating the euclidean distance, the manhattan distance, the chebyshev distance, the cosine of the included angle, the correlation coefficient, the correlation distance, and the like of the pedestrian candidate feature and the stored feature.
The second preset condition may include that the second difference is smaller than a preset value, and the like. Specifically, the preset value may be determined according to actual conditions.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
In an optional embodiment of the present invention, on the basis of the foregoing embodiment, in order to solve at least one of the case that the queue length of the short-cycle queue reaches the first preset queue length and the case that the queue length of the long-cycle queue reaches the second preset queue length, the embodiment of the present invention may further include a process of constructing a queue, and specifically, the process may include:
and when the first difference meets a first preset condition, adding the candidate pedestrian feature to the short-period queue.
Namely, when the Euclidean distance corresponding to the candidate pedestrian features is smaller than the preset distance threshold value and reaches the preset number threshold value, the candidate pedestrian features are added to the short-period queue.
Specifically, two first-in-first-out empty queues are created: short cycle queue Q1And long period queue Q2As shown in fig. 2. Wherein, the queue length can be determined according to actual requirements, and the long-period queue Q2Is longer than the short period queue Q1The queue length of (c). For example, the queue length of the short-period queue is 10, that is, the short-period queue simultaneously holds 10 features. The long period queue has a queue length of 20, i.e. the long period queue holds 20 features simultaneously.
When a target pedestrian is to be tracked, the target pedestrian can be detected and determined firstly, and then the target pedestrian is tracked in a subsequent video frame according to the characteristics and the like of the detected target pedestrian, namely the target pedestrian is detected and identified in the subsequent video frame. Such as determining a rectangle containing the target pedestrian in the first frame of the videoAnd detecting a target pedestrian from the rectangular frame, extracting the features of the target pedestrian to obtain a feature vector corresponding to the target pedestrian, and adding the feature vector to the short-period queue Q1And long period queue Q2It can also be understood as to the short-cycle queue Q1And long period queue Q2Initialization of (2).
All the pedestrian candidates in each video frame are detected in real time, for example, k pedestrian candidates (1,2, …, i, …, k) are detected for one video frame, and the process of calculating the distance between the pedestrian candidate in the video frame to be detected and the pedestrian target in the previous video frame may be specifically described in detail in the above embodiments according to the order from small to large of the distance between the pedestrian candidate in the video frame to be detected and the pedestrian target in the previous video frame, and will not be described herein again. And extracting the pedestrian candidate characteristic vector V corresponding to each pedestrian candidate according to the sorted pedestrian candidatesi(i ═ 1,2, …, k), e.g. short-period queue Q1There are m (m is 1,2, …,10) listed eigenvectors { C1 }1,C12,…,C1mV is calculated for the ith pedestrian candidateiAnd the jth feature vector C1 already listedjEuropean distance of
The preset distance T can be determined, and the pedestrian candidate feature vector V corresponding to the pedestrian candidate is calculatediAnd satisfies D in the saved feature vectorij<Number N of TiBut also the number of feature matches. Sequentially carrying out the above treatment according to the conditions that i is 0 to i is k, and if N is Ni>m × r, i.e. the pedestrian feature vector V if the pedestrian candidate corresponds toiAnd if the ratio of the number of the matched characteristic vectors to the number of the stored characteristic vectors to the total number of the columns in the queue is greater than r, determining the (m + 1) th target pedestrian characteristic vector, and simultaneously determining the candidate pedestrian corresponding to the candidate pedestrian characteristic vector as the target pedestrian.
And if Q1If not, then V is setiDirect enqueueing, i.e. ViAdding the short-period queue; otherwise, pop Q1In the first place hasEnqueued feature vector C11And is in Q1The tail end of the candidate pedestrian feature vector is pressed in.
In addition, when the number of the pedestrian feature candidates added to the short-period queue reaches the updated number of the long-period queue, one pedestrian feature candidate meeting the addition condition is selected from the pedestrian feature candidates added to the short-period queue and added to the long-period queue.
E.g. every 5 new candidate pedestrian feature vectors are added to Q1Sorting the 5 new pedestrian candidate feature vectors from small to large according to Euclidean distance, and adding the 3 rd pedestrian candidate feature vector to a long-period queue Q2. Thus, Q can be guaranteed to be2The characteristic vectors have little difference, and the characteristic of the difference of the characteristics of one circle of the pedestrian in the pedestrian surrounding tracking has good effect. Moreover, the feature vector of the candidate pedestrian in the middle is selected, so that the feature vector of the non-target pedestrian can be effectively prevented from being added to the long-period queue Q2Due to Q2The period is long, the feature vectors of the non-target pedestrians have a large interference effect in subsequent matching, so that the interference influence can be further reduced, and the stability of pedestrian tracking is improved.
On the basis of the above embodiment, in an optional embodiment of the present invention, sparse reconstruction is performed on the short-period queue and the long-period queue, and differences between the features of the candidate pedestrian and the features already stored in the short-period queue and the long-period queue are reflected by a reconstruction error.
Specifically, the step S1042 of determining the second difference between the pedestrian feature candidate and the saved features in the short-period queue and the long-period queue may include:
determining a reconstruction error of the pedestrian candidate feature based on the constructed codebook matrix through a local-constrained Linear Coding (LLC) algorithm, and taking the reconstruction error as a second difference between the pedestrian candidate feature and the features stored in the short-period queue and the long-period queue.
The codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate characteristics and the codebook matrix.
When the second difference satisfies the second preset condition in step S1042, determining that the candidate pedestrian is the target pedestrian may include:
determining the minimum reconstruction error in the reconstruction errors respectively corresponding to the candidate pedestrians; and when the minimum reconstruction error meets the maximum allowable threshold of the reconstruction error, determining the candidate pedestrian corresponding to the minimum reconstruction error as the target pedestrian.
In an alternative embodiment, step S1042: determining a second difference between the pedestrian feature candidate and a second saved feature in the short-period queue and the long-period queue, and determining that the pedestrian candidate is the target pedestrian when the second difference satisfies a second preset condition, as shown in fig. 3, may include:
s10421, determining a reconstruction error of the candidate pedestrian feature based on the constructed codebook matrix through an LLC algorithm.
The codebook matrix is constructed according to the short-period queue and the long-period queue. The reconstruction error is used for reflecting the difference degree of the pedestrian candidate characteristics and the codebook matrix.
Short cycle queue Q1And long period queue Q2Both queues are filled and the LLC algorithm is used to update both queues. Assuming that k candidate pedestrians are detected in a frame, the sequence is performed from small to large according to the distance between each candidate pedestrian in the video frame to be detected and the position of the target pedestrian in the previous video frame, and specifically, the process of calculating the distance between the candidate pedestrian in the video frame to be detected and the position of the target pedestrian in the previous video frame is described in detail in the above embodiments, and will not be repeated here. And extracting the pedestrian candidate characteristic vector V corresponding to each pedestrian candidate according to the sorted pedestrian candidatesi(i ═ 1,2, …, k), this time short-period queue Q1Feature vector C11,C12,…,C110And long period queue Q2Feature vector C21,C22,…,C220Get Q out1And Q2Concatenated together to give a queue C of length 30 ═ C1,C2,…,C30I.e. the codebook matrix.
Determining V by LLC sparse coding modeiReconstruction error based on codebook matrix C.
By taking reference to the LLC algorithm optimization objective function, the response coefficient vector alpha of the pedestrian feature vector on the codebook matrix C is obtained, which is equivalent to solving the optimization problem shown in the formula (1-1):
s.t.1Tα＝1
wherein, λ is a regular term coefficient for controlling the degree of sparsity; σ is used to adjust the speed of decay; α (i) is ViA vector of response coefficients on codebook matrix C; m is the number of eigenvectors in the codebook matrix, and is specifically 30; c. CjIs the element in the codebook matrix, j-1,2, …, 30.
The solution result of the LLC is sparse, only a small part is effective values other than 0, which can be considered as formula (1-1) to perform feature selection on it, and can simplify the original problem to obtain formula (1-2):
s.t.1Tα＝1
wherein, CiIs ViK in the codebook matrix C is a matrix of neighboring vectors,
the weight coefficient is corresponding to the characteristic vector of the waiting person.
The computational complexity can be greatly reduced using equation (1-2).
The reconstruction error e can be calculated by the following formula according to the solved alphai：
And S10422, when the reconstruction error meets a second preset condition, determining that the candidate pedestrian is the target pedestrian.
The second preset condition may include that the reconstruction error reaches a preset value. Specifically, the preset value may be determined according to actual requirements.
The minimum reconstruction error in the reconstruction errors respectively corresponding to the candidate pedestrians can be determined; and when the minimum reconstruction error meets the maximum allowable threshold of the reconstruction error, determining the candidate pedestrian corresponding to the minimum reconstruction error as the target pedestrian.
Specifically, by the above-described formulas (1-2) and (1-3), the i-th pedestrian feature candidate vector V can be calculatediReconstruction error e ofiAnd selecting the minimum reconstruction error esThe corresponding s-th pedestrian candidate is taken as a candidate target pedestrian. If es<er, where er is a reconstruction error maximum allowable threshold, a typical value may be set to 0.65, and the s-th pedestrian candidate may be determined to be the target pedestrian.
The embodiment of the invention provides a multi-queue tracking mode, and combines an LLC algorithm to calculate the reconstruction error based on a codebook matrix, and then performs matching through the reconstruction error to realize pedestrian tracking.
In an alternative embodiment of the present invention, the process of updating the queue is further included, which can also be understood as a maintenance phase of the queue. Specifically, in step S10421: after determining the reconstruction error between the pedestrian candidate feature and the constructed codebook matrix through the local constraint linear coding LLC algorithm, the method may further include:
and when the reconstruction error meets a third preset condition, popping up a first stored feature in the short-period queue, and adding the pedestrian candidate feature to the short-period queue.
The reconstruction error satisfying the third preset condition may include the reconstruction error being within a preset error range.
And when the number of the pedestrian feature candidates added to the short-period queue reaches the updated number of the long-period queue, selecting one pedestrian feature candidate meeting the addition condition from the pedestrian feature candidates added to the short-period queue and adding the selected pedestrian feature candidate to the long-period queue.
In an optional embodiment, when the number of the plurality of pedestrian feature candidates added to the short-period queue reaches the updated number of the long-period queue, a first stored feature in the long-period queue is popped up, and one pedestrian feature candidate is selected from the plurality of pedestrian feature candidates added to the short-period queue and added to the long-period queue.
In particular, to take into account the consistency and diversity of features in the queue, the third preset condition may be that the reconstruction error satisfies 0.03<es<0.5, then pop up the short period queue Q1The first saved eigenvector in the medium-short cycle queue, i.e. the earliest eigenvector in the short cycle queue, C11And corresponding pedestrian candidate feature vector VsPressing the candidate pedestrian feature vector into the tail of the queue, namely adding the corresponding candidate pedestrian feature vector into the tail of the short-period queue to realize the short-period queue Q1And (4) updating.
Similarly, every 5 new pedestrian candidate feature vectors are inserted into the short-period queue Q1Then, the 5 new pedestrian feature vector candidates are sorted from small to large according to Euclidean distance, and the 3 rd pedestrian feature vector candidate is updated to the long-period queue Q by the same method2. I.e. pop the first saved eigenvector in the long period queue, it can also be understood that the earliest eigenvector in the long period queue, C21And pressing the 3 rd candidate pedestrian feature vector into the tail part of the long-period queue, namely adding the 3 rd candidate pedestrian feature vector into the tail part of the long-period queue to realize the long-period queue Q2And (4) updating.
If the target pedestrian is completely occluded for several seconds or even tens of seconds, the characteristics of the target pedestrian itself may be greatly changed when the target pedestrian reappears. For example, initially the target pedestrian faces the lens with the front side facing, and after some time after being blocked by an obstacle or other pedestrian, the target pedestrian again appears with the back side facing the lens, in which case it is difficult to match the target using a feature queue because if the single queue is set too long, although the previous front information can be saved, such a queue is for the case of rapid change of pedestrian status in a short period of timeIt cannot be handled efficiently and is set too short, which obviously cannot be accommodated for this situation. Two queues of different lengths and periods are used, in which case the short period queue Q1Possibly invalid, but long-period queue Q2Is very effective because Q2The updating is slow, the stored characteristic time is long, and the characteristic of the front of the target pedestrian exists. Short cycle queue Q1Contains fast-changing short-time characteristics, is very effective for the situation of target fast change, and long-period queue Q2The pedestrian shielding device comprises the characteristics of stability and long time, and has a good effect on shielding pedestrians for a long time.
And determining the short-period queue Q through empirical parameters selected by multiple tests1The lower limit of the threshold value 0.03 and the upper limit of the threshold value 0.5 in the maintenance process, wherein the lower limit of the threshold value 0.03 can ensure that the characteristics of the push queue have diversity, and because each frame is possible to update, the identical characteristics are not required to be updated; the upper limit of 0.5 is to ensure that no interfering pedestrian can be pressed into the space Q when no target pedestrian is actually present, such as a target pedestrian is blocked1And even a few erroneous features are pushed into Q1On the one hand Q1The update is fast, the wrong feature is washed away quickly, and on the other hand Q is used1Update Q2So that the wrong pedestrian feature is more difficult to be pushed into Q2. And use every 5 updates to Q1Median of the features of (1) to update Q2Can ensure Q2The features of the wrong pedestrian are not easy to be pressed in, and the stored features have good diversity.
The method provided by the embodiment of the invention can maintain a short-period queue Q adaptive to rapid change1And a long period queue Q for stabilizing and storing different angle characteristics of the target pedestrian2。
Therefore, the pedestrian tracking is realized according to the queues in different periods, the tracking speed and the tracking effect can be considered simultaneously, and the high-speed and stable tracking is realized. The pedestrian tracking method can provide a high-speed and stable pedestrian tracking mode for various AI technologies using the pedestrian tracking process and the like. Meanwhile, the embodiment of the invention can effectively aim at the conditions of rapid change and long-time shielding of the pedestrian in the pedestrian tracking process.
An embodiment of the present invention provides a pedestrian tracking apparatus, as shown in fig. 4, including:
an obtaining module 401, configured to obtain a video frame to be detected;
a detection module 402, configured to detect a pedestrian candidate in a video frame to be detected;
an extraction module 403, configured to extract pedestrian candidate features of a pedestrian candidate;
a determining module 404, configured to determine a difference between a feature of the candidate pedestrian and a feature already stored in the feature queue, and determine that the candidate pedestrian is a target pedestrian when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
Optionally, the feature queue includes a short cycle queue and a long cycle queue;
a determination module 404 comprising:
the first determining submodule is used for determining a first difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue when at least one of the length of the short-period queue reaches a first preset queue length and the length of the long-period queue reaches a second preset queue length is not met, and determining the candidate pedestrian as a target pedestrian when the first difference meets a first preset condition; the features stored in the short-period queue and the long-period queue are features matched with the target pedestrian; the length of the first preset queue is smaller than that of the second preset queue;
and the second determining submodule is used for determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue when the queue length of the short-period queue reaches a first preset queue length and the queue length of the long-period queue reaches a second preset queue length, and determining the candidate pedestrian as the target pedestrian when the second difference meets a second preset condition.
Optionally, the first determining submodule is specifically configured to determine that the pedestrian candidate corresponding to the pedestrian candidate feature is the target pedestrian when the number of the first difference corresponding to the pedestrian candidate feature, which is smaller than the preset difference threshold, reaches the preset number threshold.
Optionally, the first determining sub-module is specifically configured to calculate, for each saved feature in the short-period queue, a euclidean distance between the pedestrian candidate feature and the saved feature, and use the euclidean distance as a first difference between the pedestrian candidate feature and the saved feature.
Optionally, the apparatus further comprises:
the first adding module is used for adding the candidate pedestrian features to the short-period queue when the Euclidean distance corresponding to the candidate pedestrian features is smaller than the preset distance threshold and the number of the candidate pedestrian features reaches the preset number threshold.
Optionally, the second determining submodule is specifically configured to determine, through a local constrained linear coding LLC algorithm, a reconstruction error of the pedestrian feature candidate based on the constructed codebook matrix, and use the reconstruction error as a second difference between the pedestrian feature candidate and the features already stored in the short-period queue and the long-period queue; the codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate characteristics and the codebook matrix.
Optionally, the second determining sub-module is specifically configured to determine a minimum reconstruction error of reconstruction errors corresponding to each pedestrian candidate; and when the minimum reconstruction error meets the maximum allowable threshold of the reconstruction error, determining the candidate pedestrian corresponding to the minimum reconstruction error as the target pedestrian.
Optionally, the apparatus further comprises:
the popping module is used for popping a first saved feature in the short-period queue when the reconstruction error meets a third preset condition;
and the second adding module is used for adding the pedestrian candidate characteristics to the short-period queue.
Optionally, the apparatus further comprises:
and the third adding module is used for selecting one candidate pedestrian feature meeting the adding condition from the candidate pedestrian features added to the short-period queue to be added to the long-period queue when the number of the candidate pedestrian features added to the short-period queue reaches the updated number of the long-period queue.
It should be noted that, the pedestrian tracking device provided by the embodiment of the present invention is a device to which the above-mentioned pedestrian tracking method is applied, and all embodiments of the above-mentioned pedestrian tracking method are applicable to the device, and can achieve the same or similar beneficial effects.
An embodiment of the present invention further provides an electronic device, as shown in fig. 5, including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, where the processor 501, the communication interface 502 and the memory 503 complete mutual communication through the communication bus 504.
A memory 503 for storing a computer program;
the processor 501 is configured to implement the method steps of the pedestrian tracking method in the above embodiments when executing the program stored in the memory 503.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In a further embodiment provided by the present invention, there is also provided a computer-readable storage medium having stored therein instructions which, when run on a computer, cause the computer to perform the method steps of the pedestrian tracking method in the above-described embodiment.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
In a further embodiment provided by the present invention, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method steps of the pedestrian tracking method in the above-described embodiment.
In the embodiment of the invention, the difference between the pedestrian candidate characteristics corresponding to the pedestrian candidate and the stored characteristics in the characteristic queue is calculated, and when the difference meets the condition, the pedestrian candidate is determined to be the target pedestrian. Thus, the efficiency of the pedestrian tracking process can be improved.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus, device, computer-readable storage medium, and computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for related matters, reference may be made to the partial description of the method embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (15)
1. A pedestrian tracking method, comprising:
acquiring a video frame to be detected;
detecting candidate pedestrians in the video frame to be detected;
extracting pedestrian candidate characteristics of the pedestrian candidate; the extracting pedestrian candidate features of the pedestrian candidate comprises: calculating the distance between each candidate pedestrian in the video frame to be detected and the position of the target pedestrian in the previous video frame; sequencing all the candidate pedestrians according to the sequence from small to large of the distance, and respectively extracting the candidate pedestrian characteristics of all the candidate pedestrians according to the sequenced candidate pedestrians;
determining the difference between the features of the candidate pedestrians and the features stored in the feature queue, and determining the candidate pedestrians as target pedestrians when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian;
the characteristic queue comprises a short-period queue and a long-period queue;
the determining the difference between the candidate pedestrian feature and the feature stored in the feature queue, and determining the candidate pedestrian as the target pedestrian when the difference meets the preset condition includes:
when at least one of the queue length of the short-period queue reaching a first preset queue length and the queue length of the long-period queue reaching a second preset queue length is not met, determining a first difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue, and determining the candidate pedestrian as a target pedestrian when the first difference meets a first preset condition; wherein the length of the first preset queue is smaller than the length of the second preset queue; when the first difference meets a first preset condition, determining that the candidate pedestrian is a target pedestrian comprises: when the number of first differences corresponding to the candidate pedestrian features, which are smaller than a preset difference threshold value, reaches a preset number threshold value, determining that the candidate pedestrian corresponding to the candidate pedestrian features is the target pedestrian;
and when the queue length of the short-period queue reaches the first preset queue length and the queue length of the long-period queue reaches the second preset queue length, determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue, and determining that the candidate pedestrian is a target pedestrian when the second difference meets a second preset condition.
2. The method of claim 1, wherein said determining a first difference between said pedestrian candidate feature and a saved feature in said short-cycle queue comprises:
and calculating Euclidean distance between the pedestrian candidate feature and each saved feature in the short-period queue, and taking the Euclidean distance as a first difference between the pedestrian candidate feature and the saved feature.
3. The method of claim 2, further comprising:
and when the Euclidean distance corresponding to the candidate pedestrian features is smaller than a preset distance threshold value and reaches the preset number threshold value, adding the candidate pedestrian features to the short-period queue.
4. The method of claim 1, wherein said determining a second difference between said pedestrian candidate feature and a saved feature in said short-cycle queue and said long-cycle queue comprises:
determining a reconstruction error of the pedestrian candidate feature based on the constructed codebook matrix through a local constraint linear coding (LLC) algorithm, and taking the reconstruction error as a second difference between the pedestrian candidate feature and the features stored in the short-period queue and the long-period queue; the codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate feature and the codebook matrix.
5. The method according to claim 1 or 4, wherein the determining that the pedestrian candidate is the target pedestrian when the second difference satisfies a second preset condition comprises:
determining the minimum reconstruction error in the reconstruction errors respectively corresponding to the candidate pedestrians;
and when the minimum reconstruction error meets a maximum allowable reconstruction error threshold, determining that the candidate pedestrian corresponding to the minimum reconstruction error is the target pedestrian.
6. The method according to claim 4, wherein after determining the reconstruction error of the pedestrian candidate feature and the constructed codebook matrix through the local constrained linear coding (LLC) algorithm, the method further comprises:
and when the reconstruction error meets a third preset condition, popping up a first saved feature in the short-period queue, and adding the pedestrian candidate feature to the short-period queue.
7. The method of claim 3 or 6, further comprising:
and when the number of the pedestrian feature candidates added to the short-period queue reaches the updated number of the long-period queue, selecting one pedestrian feature candidate meeting an adding condition from the pedestrian feature candidates added to the short-period queue and adding the selected pedestrian feature candidate to the long-period queue.
8. A pedestrian tracking apparatus, comprising:
the acquisition module is used for acquiring a video frame to be detected;
the detection module is used for detecting candidate pedestrians in the video frame to be detected;
the extraction module is used for extracting pedestrian candidate characteristics of the pedestrian candidate; the extraction module is specifically used for calculating the distance between each candidate pedestrian in the video frame to be detected and the position of the target pedestrian in the previous video frame; sequencing all the candidate pedestrians according to the sequence from small to large of the distance, and respectively extracting the candidate pedestrian characteristics of all the candidate pedestrians according to the sequenced candidate pedestrians;
the determining module is used for determining the difference between the features of the candidate pedestrian and the features stored in the feature queue, and determining the candidate pedestrian as a target pedestrian when the difference meets a preset condition; wherein the stored features in the feature queue are features matched with the target pedestrian;
the characteristic queue comprises a short-period queue and a long-period queue;
the determining module includes:
a first determining sub-module, configured to determine a first difference between the feature of the pedestrian candidate and a feature already stored in the short-period queue when at least one of a length of the short-period queue reaches a first preset queue length and a length of the long-period queue reaches a second preset queue length is not satisfied, and determine that the pedestrian candidate is a target pedestrian when the first difference satisfies a first preset condition; wherein the length of the first preset queue is smaller than the length of the second preset queue; the first determining submodule is specifically configured to determine that the pedestrian candidate corresponding to the pedestrian candidate feature is the target pedestrian when the number of the first differences corresponding to the pedestrian candidate feature, which are smaller than a preset difference threshold, reaches a preset number threshold;
and the second determining submodule is used for determining a second difference between the characteristics of the candidate pedestrian and the characteristics stored in the short-period queue and the long-period queue when the queue length of the short-period queue reaches the first preset queue length and the queue length of the long-period queue reaches the second preset queue length, and determining the candidate pedestrian as a target pedestrian when the second difference meets a second preset condition.
9. The apparatus according to claim 8, wherein the first determining sub-module is configured to, for each saved feature in the short-term queue, calculate a euclidean distance between the pedestrian candidate feature and the saved feature, and use the euclidean distance as the first difference between the pedestrian candidate feature and the saved feature.
10. The apparatus of claim 9, further comprising:
the first adding module is used for adding the pedestrian feature candidates to the short-period queue when the Euclidean distance corresponding to the pedestrian feature candidates is smaller than a preset distance threshold and the number of the pedestrian feature candidates reaches the preset number threshold.
11. The apparatus according to claim 8, wherein the second determining sub-module is specifically configured to determine a reconstruction error of the pedestrian candidate feature based on the constructed codebook matrix through a locally constrained linear coding (LLC) algorithm, and to use the reconstruction error as a second difference between the pedestrian candidate feature and the features already stored in the short-period queue and the long-period queue; the codebook matrix is constructed according to the short-period queue and the long-period queue; the reconstruction error is used for reflecting the difference degree of the pedestrian candidate feature and the codebook matrix.
12. The apparatus according to claim 8 or 11, wherein the second determining sub-module is specifically configured to determine a minimum reconstruction error of reconstruction errors corresponding to the candidate pedestrians; and when the minimum reconstruction error meets a maximum allowable reconstruction error threshold, determining that the candidate pedestrian corresponding to the minimum reconstruction error is the target pedestrian.
13. The apparatus of claim 11, further comprising:
the popping module is used for popping a first saved feature in the short-period queue when the reconstruction error meets a third preset condition;
a second adding module for adding the pedestrian candidate feature to the short-cycle queue.
14. The apparatus of claim 10 or 13, further comprising:
and the third adding module is used for selecting one candidate pedestrian feature meeting an adding condition from the candidate pedestrian features added to the short-period queue to be added to the long-period queue when the number of the candidate pedestrian features added to the short-period queue reaches the updated number of the long-period queue.
15. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
the memory is used for storing a computer program;
the processor, when executing the program stored in the memory, implementing the method steps of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910119437.XA CN109919043B (en) | 2019-02-18 | 2019-02-18 | Pedestrian tracking method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910119437.XA CN109919043B (en) | 2019-02-18 | 2019-02-18 | Pedestrian tracking method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109919043A CN109919043A (en) | 2019-06-21 |
| CN109919043B true CN109919043B (en) | 2021-06-04 |
Family
ID=66961660
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910119437.XA Active CN109919043B (en) | 2019-02-18 | 2019-02-18 | Pedestrian tracking method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109919043B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110749351A (en) * | 2019-11-05 | 2020-02-04 | 苏州美能华智能科技有限公司 | Method, device and system for determining attention target of object |
| CN112578351A (en) * | 2020-02-28 | 2021-03-30 | 加特兰微电子科技(上海)有限公司 | Target matching method, integrated circuit, radio device and apparatus |
| CN111914690B (en) * | 2020-07-15 | 2023-11-10 | 西安米克斯智能技术有限公司 | Target object medium-long-term tracking method in video identification |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875655A (en) * | 2018-06-25 | 2018-11-23 | 鲁东大学 | A kind of real-time target video tracing method and system based on multiple features |
| CN108921140A (en) * | 2018-08-07 | 2018-11-30 | 安徽云森物联网科技有限公司 | Pedestrian's recognition methods again |
-
2019
- 2019-02-18 CN CN201910119437.XA patent/CN109919043B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875655A (en) * | 2018-06-25 | 2018-11-23 | 鲁东大学 | A kind of real-time target video tracing method and system based on multiple features |
| CN108921140A (en) * | 2018-08-07 | 2018-11-30 | 安徽云森物联网科技有限公司 | Pedestrian's recognition methods again |
Non-Patent Citations (2)
| Title |
|---|
| Person Tracking and Frontal Face Capture with UAV;Qifeng Shen et al.;《2018 18th IEEE International Conference on Communication Technology》;20190103;第1412-1416页 * |
| Real-time Accurate Pedestrian Circular Tracking with UAV;Tao Zhong et al.;《17th IEEE International Conference on Communication Technology》;20180517;第1519-1523页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109919043A (en) | 2019-06-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111523621B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN107808122B (en) | Target tracking method and device | |
| CN109919043B (en) | Pedestrian tracking method, device and equipment | |
| CN112639828A (en) | Data processing method, method and equipment for training neural network model | |
| CN108875487B (en) | Training of pedestrian re-recognition network and pedestrian re-recognition based on training | |
| CN112560996B (en) | User portrait identification model training method, device, readable storage medium and product | |
| CN109871821B (en) | Pedestrian re-identification method, device, equipment and storage medium of self-adaptive network | |
| CN112907552B (en) | Robustness detection method, device and program product for image processing model | |
| US9047566B2 (en) | Quadratic regularization for neural network with skip-layer connections | |
| CN113033458B (en) | Action recognition method and device | |
| CN111524112A (en) | Steel chasing identification method, system, equipment and medium | |
| CN111126249A (en) | Pedestrian re-identification method and device combining big data and Bayes | |
| CN111105017A (en) | Neural network quantization method and device and electronic equipment | |
| WO2014176056A2 (en) | Data classification | |
| CN114882321A (en) | Deep learning model training method, target object detection method and device | |
| CN112966744A (en) | Model training method, image processing method, device and electronic equipment | |
| CN114078195A (en) | Training method of classification model, search method and device of hyper-parameters | |
| Jie et al. | Anytime recognition with routing convolutional networks | |
| CN114881129A (en) | Model training method and device, electronic equipment and storage medium | |
| CN115147680B (en) | Pre-training method, device and equipment for target detection model | |
| CN113011532A (en) | Classification model training method and device, computing equipment and storage medium | |
| CN114386503A (en) | Method and apparatus for training a model | |
| CN113887615A (en) | Image processing method, apparatus, device and medium | |
| CN115631381A (en) | Classification model training method, image classification device and electronic equipment | |
| CN110490058B (en) | Training method, device and system of pedestrian detection model and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |