CN113989761A - Object tracking method and device, electronic equipment and storage medium - Google Patents
Object tracking method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113989761A CN113989761A CN202111275859.XA CN202111275859A CN113989761A CN 113989761 A CN113989761 A CN 113989761A CN 202111275859 A CN202111275859 A CN 202111275859A CN 113989761 A CN113989761 A CN 113989761A
- Authority
- CN
- China
- Prior art keywords
- information
- image
- determining
- historical
- combination
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an object tracking method, an object tracking device, electronic equipment and a storage medium, wherein the object tracking method comprises the following steps: acquiring a current image and a historical image; under the condition that a preset object exists in the historical image, acquiring historical state information of at least one first preset object in the historical image based on the historical image; according to the historical state information, determining position prediction information corresponding to at least one first preset object; determining an image acquisition type according to the position prediction information; determining at least one combination under the condition that the image acquisition type is a cross-camera acquisition type, and determining distance information corresponding to the at least one combination and corresponding first characteristic information; determining weight information corresponding to at least one combination according to the distance information and the first characteristic information; and updating the object track information according to the weight information. According to the technical scheme of the invention, the target tracking of the all-round automatic driving is realized, the possibility of the object losing across cameras is reduced, and the tracking efficiency is improved.
Description
Technical Field
The present invention relates to the field of automatic driving technologies, and in particular, to an object tracking method and apparatus, an electronic device, and a storage medium.
Background
Automatic driving at the present stage belongs to the popular field, and research needs related to multi-target tracking appear. At present, most of cross-camera pedestrian tracking methods are based on the research in the field of surveillance videos.
The great difference from the all-round automatic driving is that the scenes of a plurality of monitoring cameras are overlapped, and the relative positions of the same target in the same scene can be shot. However, the positions of the commonly used panoramic fisheye cameras for the panoramic automatic driving are respectively positioned below the head of the vehicle, the tail of the vehicle and the two rearview mirrors, and the shooting directions of the four cameras are different. The four images have partial spatially corresponding image disappearance after being subjected to distortion removal, and background images which can be acquired by the four cameras have almost no similarity, which also increases certain difficulty for target matching in tracking.
It is known that autopilot is an area with high requirements on timeliness. After the detection and classification of the targets for the images acquired by the four cameras, the targets need to be tracked. The analysis of the image takes a long time, and how to reduce the calculation amount as much as possible in the tracking part becomes a difficult point, and the characteristic is that the time consumption is doubled when the number of cameras is large. For example, since four cameras are currently used for looking around and the orientations of the four cameras are completely different, the four cameras need to be analyzed simultaneously each time tracking analysis is performed, which brings about a certain efficiency problem.
Disclosure of Invention
The invention aims to provide an object tracking method, an object tracking device, an electronic device and a storage medium, wherein position prediction information is determined through a historical image, an image acquisition type is determined based on the position prediction information, at least one combination and weight information corresponding to each combination are determined based on the image acquisition type, and object track information is updated through the weight information, so that the object tracking of looking around automatic driving is realized, the possibility of losing an object across cameras is reduced, the time consumption is reduced to a certain extent, and the object tracking efficiency is improved.
In order to achieve the purpose, the invention provides the following scheme:
a method of object tracking, the method comprising:
acquiring a current image and a historical image;
under the condition that a preset object exists in the historical image, acquiring historical state information of at least one first preset object in the historical image based on the historical image;
according to the historical state information, determining position prediction information corresponding to the at least one first preset object;
determining an image acquisition type according to the position prediction information;
under the condition that the image acquisition type is a cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining distance information and corresponding first characteristic information corresponding to the at least one combination;
determining weight information corresponding to each combination according to the distance information and the first characteristic information;
and updating the object track information according to the weight information.
Optionally, after determining the image acquisition type according to the position prediction information, the method further includes:
under the condition that the image acquisition type is a non-cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining intersection and parallel ratio information and corresponding second characteristic information corresponding to the at least one combination;
and determining the weight information corresponding to the at least one combination according to the intersection ratio information and the second characteristic information.
Optionally, the determining an image obtaining type according to the position prediction information includes:
determining the image acquisition type as the cross-camera acquisition type when the position point corresponding to the position prediction information does not belong to a preset range;
and determining that the image acquisition type is a non-cross-camera acquisition type when the position point corresponding to the position prediction information belongs to a preset range.
Optionally, the determining, according to the historical state information, position prediction information corresponding to each of the at least one first preset object includes:
acquiring the current moment;
obtaining time difference value information according to the historical time and the current time;
and determining the position prediction information corresponding to the at least one combination according to the time difference information, the position information and the speed information.
Optionally, in the case that the image acquisition type is a cross-camera acquisition type, determining at least one combination based on the current image and the historical image includes:
identifying an image corresponding to the position prediction information in the current image, and taking a preset object in the image as the second preset object;
and combining the first preset object and each second preset object in the historical image to obtain at least one combination, wherein the image acquisition device corresponding to the current image and the image acquisition device corresponding to the historical image are adjacently arranged.
Optionally, after the acquiring the current image and the historical image, the method further includes:
and under the condition that the preset object exists in the current image and the preset object does not exist in the historical image, updating object track information, initializing the historical state information, and returning to the step of acquiring the current image and the historical image.
Optionally, the updating the object track information according to the weight information includes:
determining a target combination according to the weight information;
and performing object association matching based on the target combination, and updating the object track information according to the matched object.
In another aspect, the present invention further provides an object tracking apparatus, including:
the first information acquisition module is used for acquiring a current image and a historical image;
the second information acquisition module is used for acquiring the respective historical state information of at least one first preset object in the historical image on the basis of the historical image under the condition that the historical image has the preset object;
the first information determining module is used for determining position prediction information corresponding to the at least one first preset object according to the historical state information;
the image acquisition type determining module is used for determining the image acquisition type according to the position prediction information;
the second information determining module is used for determining at least one combination based on the current image and the historical image and determining distance information corresponding to the at least one combination and corresponding first characteristic information under the condition that the image acquisition type is a cross-camera acquisition type;
a third information determining module, configured to determine, according to the distance information and the first feature information, weight information corresponding to each of the at least one combination;
and the association matching module is used for updating the object track information according to the weight information.
In another aspect, the present invention further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above object tracking method.
In another aspect, the present invention also provides a non-transitory computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above object tracking method.
According to the object tracking method, the device, the electronic equipment and the storage medium, the position prediction information is determined through the historical image, the image acquisition type is determined based on the position prediction information, at least one combination and the weight information corresponding to each combination are determined based on the image acquisition type, the object track information is updated through the weight information, the target tracking of looking around automatic driving is achieved, the possibility that the object is lost across cameras is reduced, time consumption is reduced to a certain extent, and the target tracking efficiency is improved.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings used in the description of the embodiment or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art it is also possible to derive other drawings from these drawings without inventive effort.
FIG. 1 is a flowchart of a method of object tracking according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method after determining an image capture type according to position prediction information according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for determining an image capture type according to location prediction information according to an embodiment of the present invention;
fig. 4 is a flowchart of a method for determining location prediction information corresponding to at least one first preset object according to historical status information according to an embodiment of the present invention;
fig. 5 is a flowchart of a method for determining at least one combination based on a current image and a historical image when an image acquisition type is a cross-camera acquisition type according to an embodiment of the present invention;
FIG. 6 is a flowchart of a method after acquiring a current image and a historical image according to an embodiment of the present invention;
FIG. 7 is a flowchart of a method for updating object trajectory information according to weight information according to an embodiment of the present invention;
fig. 8 is a block diagram illustrating an object tracking apparatus according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a preset range in an object tracking method according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
An embodiment of an object tracking method according to the present invention is described below, and fig. 1 is a flowchart of a method of an object tracking method according to an embodiment of the present invention. It is noted that the present specification provides the method steps as described in the examples or flowcharts, but may include more or less steps based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system products may be executed sequentially or in parallel (e.g., in the context of parallel processors or multi-threaded processing) in accordance with the methods described in the embodiments or figures. As shown in fig. 1, the present embodiment provides an object tracking method, including:
s101, acquiring a current image and a historical image.
The current image may refer to an image acquired by a plurality of image acquisition devices at the current time. The history image may refer to a previous frame image with respect to the current image among the images acquired by the plurality of image acquisition devices. The historical image may be the latest frame image of an existing target trajectory. The plurality of image capturing devices may be disposed about the autonomous vehicle.
In practical application, the automatic driving vehicle can control a plurality of image acquisition devices around the vehicle to acquire images in real time in the running process, and the acquired images and the image acquisition time are correspondingly stored in a memory of the image acquisition devices. The controller of the object tracking apparatus may acquire an image corresponding to the current time and a previous frame image of the image from a memory of the image capturing device according to the current time, and may respectively use the image corresponding to the current time and the previous frame image of the image as the current image and the historical image.
S102, under the condition that the preset objects exist in the historical image, acquiring the respective historical state information of at least one first preset object in the historical image based on the historical image.
The preset object may refer to an object in which a ratio of a width to a height of a detection frame for object recognition in an image is smaller than a preset value; for example, the preset object may be a pedestrian. The first preset object may refer to a preset object in the history image. It can be understood that there may be no preset object or preset objects in the history image; further, one preset object may exist in the history image, or a plurality of preset objects may exist in the history image. The history state information may be state information of at least one first preset object in the history image. The historical state information may represent a motion state of the first preset object at a time corresponding to the historical image. The history state information may include a shooting time corresponding to the history image, a position (x, y) of the first preset object in a coordinate system corresponding to the history image, a width w and a height h of a detection frame of the first preset object in the history image, and a change speed corresponding to the four variables (x, y, w, h).
In practical application, image recognition processing is carried out on the historical image, and after a preset object is recognized, the preset object can be used as a first preset object; if a plurality of preset objects are detected, the plurality of preset objects may be used as a plurality of first preset objects. Establishing a coordinate system in the historical image, determining the position information of the midpoint of the detection frame of the first preset object in the coordinate system of the historical image, and taking the position information of at least one first preset object as the current coordinate information. Based on the comparison between the historical image and the previous frame image of the historical image, the change speed corresponding to the four variables in the historical image can be determined, so that the historical state information of the first preset object can be obtained.
S103, according to the historical state information, determining position prediction information corresponding to at least one first preset object.
The position prediction information may refer to information predicted at a position of the first preset object corresponding to the current frame. The position prediction information may be a position of an image coordinate system acquired by the image acquisition device corresponding to a shooting range in which the first preset object is located.
In practical application, according to the historical state information, the position prediction information of each first preset object can be obtained through a Kalman filter.
And S104, determining the image acquisition type according to the position prediction information.
The image acquisition type can represent the relative position relationship between the image acquisition equipment corresponding to the first preset object in the shooting range where the current frame may appear and the image acquisition equipment corresponding to the previous frame. The image acquisition type may include a cross-camera acquisition type and a non-cross-camera acquisition type.
In practical application, if the position of the first preset object in the previous frame corresponds to the first image acquisition device, it can be predicted according to the position prediction information that the first preset object may appear in the second image acquisition device in the current frame, the second image acquisition device is in an adjacent position to the first image acquisition device, and the image acquisition type may be a cross-camera acquisition type; similarly, if it can be predicted that the first preset object may still appear in the first image capturing device at the current frame according to the position prediction information, the image capturing type may be a non-cross-camera capturing type.
And S105, under the condition that the image acquisition type is the cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining distance information corresponding to the at least one combination and corresponding first characteristic information.
Wherein, the combination may refer to a combination consisting of a first preset object in the history image and each second preset object in the current image. The distance information may refer to a euclidean distance between histogram features corresponding to the detection frame of the first preset object and the detection frame of the second preset object in each combination after the detection frames of the first preset object and the second preset object are unified in size. The histogram feature can characterize the distribution of the image values in the detection frame. The first feature information corresponding to each combination can represent the similarity between the first preset object and the second preset object in the combination.
In practical application, the detection frames of the first preset object and the second preset object in each combination can be processed in a unified size; after the sizes of the two detection frames are unified, the histogram features of the two detection frames are obtained, and then the Euclidean distance between the two histogram features is obtained through calculation. Performing feature extraction on the detection frame of the first preset object and the detection frame of the second prediction object in each combination by using a pedestrian Re-identification (pedestrian Re-identification) model; through the two extracted feature information, the feature similarity between the first preset object and the second preset object in each combination can be calculated and obtained, and the feature similarity can be used as the first feature information.
And S106, determining weight information corresponding to at least one combination according to the distance information and the first characteristic information.
Wherein, the weight information of each combination can be used to characterize the matching degree between the first preset object and the second preset object in the combination.
In practical application, the weight information of each combination can be calculated according to the distance information corresponding to the combination and the first feature information corresponding to the combination.
The specific weight information W of each combination can be calculated according to the following formula:
W＝w1dist(FH1,FH2)+w2cos(Fea1,Fea2)+w3(|w1/h1-w2/h2|)
wherein, w1、w2And w3Respectively preset parameters, preferably; w is a1、w2And w3Dist (F) can be characterizedH1,FH2)，cos(Fea1,Fea2) And (w)1/h1-w2/h2) The proportional relationship between them. w is a1、w2And w3Can be set by experiment. By dist (F)H1,FH2) The Euclidean distance between the histogram features of the two detection frames can be calculated; by cos (Fea)1,Fea2) The feature similarity may be calculated as the first feature information. Specifically, FH1And FH2The histogram characteristics of the detection frame with the latest track and the detection frame of the current frame after the unified size are respectively existed. Fea1And Fea2Respectively representing feature information extracted using a pedestrian ReID model, Fea1Detecting the characteristic information, Fea, in the frame for the latest frame (the frame last to the current frame) of the existing target track2Feature information of the frame is detected for the current frame. h is1And h2Respectively representing the width and height of the latest detection frame of the existing track and the current frameThe width and height of the detection frame of (1).
Specifically, the calculation formula of the euclidean distance is:
and S107, updating the object track information according to the weight information.
The object trajectory information may refer to trajectory information of a preset object captured by an image capturing device mounted on the vehicle.
In practical application, based on weight information corresponding to at least one combination, matching of objects can be performed according to a KM (Kuhn-Munkres) algorithm (Hungarian matching algorithm with weights), and trajectory information of the objects is updated according to matching results.
The method comprises the steps of determining position prediction information through a historical image, determining an image acquisition type based on the position prediction information, determining at least one combination and weight information corresponding to each combination based on the image acquisition type, and updating object track information through the weight information, so that the target tracking of all-round automatic driving is realized, the possibility of loss of an object across cameras is reduced, time consumption is reduced to a certain extent, and the target tracking efficiency is improved.
Fig. 2 is a flowchart of a method after determining an image capturing type according to position prediction information according to an embodiment of the present invention. In one possible embodiment, as shown in fig. 2, after determining the image capturing type according to the position prediction information, the method may further include:
s201, under the condition that the image acquisition type is a non-cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining intersection and parallel ratio information corresponding to the at least one combination and corresponding second characteristic information.
The merging ratio information may refer to a merging ratio of a detection frame of a first preset object and a detection frame of a second preset object in the combination; specifically, the intersection-to-union ratio may refer to an area ratio of an intersection to a union of two rectangular boxes. The second feature information may characterize a similarity between the first preset object and the second preset object in the combination.
In practical application, the intersection and combination ratio of the first preset object and the second preset object in the combination can be directly calculated according to the detection frame of the first preset object and the detection frame of the second preset object, or the intersection and combination ratio of the first preset object and the second preset object after the two detection frames are respectively subjected to external expansion can be calculated. In this embodiment, the intersection-to-parallel ratio W is calculated by adopting the method of calculating the intersection-to-parallel ratio after the external expansionIOUThe specific calculation process is as follows:
Re＝wvvhost+wyawvyaw_rate
wherein Rc is1And Rc2Respectively representing two detection boxes participating in the calculation, ReRepresenting the external expansion ratio of the detection frame; v. ofhostAnd vyaw_rateRespectively representing the vehicle speed and the corresponding yaw rate. w is avAnd wyawRespectively are preset parameters, and represent corresponding weight relations. w is avAnd wyawCan be determined by multiple tests; preferably, wvAnd wyawCan be respectively 0.6 and 0.4, has obvious tracking effect and less occurrence of mismatching.
S202, determining weight information corresponding to at least one combination according to the intersection ratio information and the second characteristic information.
In practical application, the feature similarity can be measured by using the cosine distance, so that the second feature information is obtained. The weight information W of each combination can be obtained by the cross-over ratio information and the second feature information. The specific calculation formula is as follows:
W＝w1WIOU+w2 cos(Fea1,Fea2)
wherein, w1And w2Are respectively preset parameters, characterize WIOUAnd cos (Fea)1,Fea2) The proportional relationship of (c). Preferably, w1And w20.6 and 0.4 respectively, the target association matchesThe best results are obtained. Fea1And Fea2Respectively representing feature information extracted using a pedestrian ReID model, Fea1Feature information, Fea, inside the latest frame detection frame of the existing target track2Feature information of the frame is detected for the current frame.
It should be noted that, in the case that the image acquisition type is a non-cross-camera acquisition type, the method of determining at least one combination may be: under the condition that the image acquisition type is a non-cross-camera acquisition type, it can be determined that image acquisition equipment corresponding to the current image is the same as image acquisition equipment corresponding to the first preset image; the current image of the device identifier which is the same as the device identifier corresponding to the history image in which the first preset object is located in the combination can be obtained for analysis. Performing identification processing on the current image; in the case that the preset object exists in the current image, it may be recognized that each of the at least one preset object is a second preset object. And combining the first preset object with each second preset object in the current image. It is understood that the number of combinations is the same as the number of second preset objects.
After the image acquisition type is determined, different weight determination methods can be corresponding to different image acquisition types; after the image acquisition type is determined to be the non-cross-camera acquisition type, the images needing to be processed are determined and then are subjected to targeted processing, so that the data volume of analysis processing can be reduced, and the analysis efficiency is improved.
Fig. 3 is a flowchart of a method for determining an image capturing type according to position prediction information according to an embodiment of the present invention. In one possible embodiment, as shown in fig. 3, the step S104 may include:
and S301, determining that the image acquisition type is a cross-camera acquisition type when the position point corresponding to the position prediction information does not belong to a preset range.
Wherein the preset range may be determined according to a photographing range of each image capturing apparatus. The preset range may be the same range as the shooting range of the image capturing apparatus, or may be a range slightly smaller than the shooting range of the image capturing apparatus, and the disclosure is not particularly limited. In the present embodiment, the preset range is a range smaller than the image capturing range of the image capturing apparatus. The position point corresponding to the position prediction information may represent the position of the predicted first preset object in the current image in the form of a point. Specifically, the position point corresponding to the position prediction information may be a bottom center point of the predicted detection frame.
In practical application, the relative position of the detection frame predicted by each first preset object at the current moment relative to the historical image can be obtained through the position prediction information; and further, the specific position and the preset range of the midpoint of the bottom edge of the detection frame can be determined, and the image acquisition type is determined according to whether the midpoint of the bottom edge is in the preset range. It can be understood that when the midpoint of the bottom edge is within the preset range, the image acquisition type is a non-cross-camera acquisition type; and when the middle point of the bottom edge is outside the preset range, the image acquisition type is a cross-camera acquisition type. For example, as shown in fig. 9, W and H in the figure respectively indicate the width and height of an image, a gray area indicates a preset range, and a and B respectively indicate detection frames of two preset objects; the middle point of the bottom edge of the A is located in a preset range, and the image acquisition type is determined to be a non-trans-camera acquisition type; and B, determining the image acquisition type to be a cross-camera acquisition type when the middle point of the bottom edge of the B is positioned outside the preset range.
S302, when the position point corresponding to the position prediction information belongs to a preset range, determining that the image acquisition type is a non-cross-camera acquisition type.
Through the position relation between the position point corresponding to the position prediction information and the preset range, the relative position relation between the image acquisition equipment corresponding to the first preset object at the current moment and the historical moment can be determined, and therefore the image acquisition type can be determined quickly and accurately.
Fig. 4 is a flowchart of a method for determining location prediction information corresponding to at least one first preset object according to historical status information according to an embodiment of the present invention. In a possible embodiment, the historical state information includes a historical time, and position information and speed information corresponding to at least one first preset object, as shown in fig. 4, the step S103 may include:
s401, obtaining the current time.
The current time may refer to a shooting time corresponding to the current image.
In practical application, the image acquisition device synchronously saves the shooting time when image acquisition is carried out. The current time can be obtained according to the identification information of the current image.
S402, obtaining time difference value information according to the historical time and the current time.
The historical time may be a shooting time corresponding to the historical image, that is, a shooting time corresponding to the previous frame of image. The time difference information may refer to difference information between the historical time and the current time.
In practical application, time difference information can be obtained by taking a difference between the current time and the historical time.
And S403, determining at least one piece of position prediction information corresponding to each combination according to the time difference information, the position information and the speed information.
The position information of each first preset object may refer to a position coordinate of the first preset object in the history image. The position information may specifically include a position (x, y) of the first preset object in a coordinate system corresponding to the history image, a width w and a height h of a detection frame of the first preset object in the history image. The velocity information of each object may include a variation velocity corresponding to the above-described four variables (x, y, w, h).
In practical application, the position of the first preset object in the historical image coordinate system at the current moment and the width and the height of the detection frame can be calculated according to the time difference information, the position information and the speed information.
Fig. 5 is a flowchart of a method for determining at least one combination based on a current image and a historical image when an image acquisition type is a cross-camera acquisition type according to an embodiment of the present invention. In one possible embodiment, as shown in fig. 5, in the case that the image acquisition type is a cross-camera acquisition type, determining at least one combination based on the current image and the historical image may include:
s501, identifying an image corresponding to the position prediction information in the current image, and taking a preset object in the image as a second preset object.
In practical applications, the current image may include images captured by a plurality of image capturing devices at the current time. According to the relative position relationship between the position prediction information and the image where the first preset object is located and the image acquisition device corresponding to the shooting range where the first preset object is located, the device identifier of the image acquisition device corresponding to the position prediction information can be determined. And the image corresponding to the equipment identifier in the current image is the image corresponding to the position prediction information in the current image. It is understood that the position prediction information may reflect an image acquisition device corresponding to a shooting area where the first preset object may appear at the current time, and an image corresponding to the position prediction information in the current image may be determined according to the position prediction information. And performing identification processing on the image, and taking the preset object in the image as a second preset object. For example, when 2 preset objects are recognized in the image, the 2 preset objects are respectively used as 2 second preset objects.
S502, combining the first preset object and each second preset object in the historical image to obtain at least one combination, wherein the image acquisition device corresponding to the current image and the image acquisition device corresponding to the historical image are arranged adjacently.
It is understood that the two first preset objects may respectively correspond to different images and different image acquisition types according to the position prediction information. When the image acquisition type of one of the first preset objects is a cross-camera acquisition type, the first preset object is combined with each of the second preset objects, so that the combinations with the same number as the second preset objects can be obtained. For example, assuming that there are 2 second preset objects (object 21 and object 22, respectively) and the first preset object is object 11, 2 combinations can be combined, including the first combination (object 11 and object 21) and the second combination (object 11 and object 22), respectively.
Fig. 6 is a flowchart of a method after a current image and a historical image are acquired according to an embodiment of the present invention. In one possible embodiment, as shown in fig. 6, after acquiring the current image and the historical image, the method may further include:
s601, under the condition that the preset object exists in the current image and the preset object does not exist in the historical image, updating the object track information, initializing the historical state information, and returning to the step of obtaining the current image and the historical image.
It can be understood that, in the case that the preset object exists in the current image and the preset object does not exist in the history image, it can be stated that the current image is the first frame image in the track corresponding to the preset object.
In practical applications, when the preset object exists in the current image and the preset object does not exist in the historical image, the track information of the preset object may be newly added to the object track information to update the object track information, and the historical state information of the preset object is initialized. Specifically, the position of the detection frame of the preset object in the coordinate system of the historical image is used as the position information in the historical state information of the preset object; the change speeds of four of the variables are initialized to 0. After returning to the step of acquiring the current image and the historical image, the preset object is taken as a first preset object, and the historical state information of the preset object acquired in the subsequent step is the initialized information.
Fig. 7 is a flowchart of a method for updating object trajectory information according to weight information according to an embodiment of the present invention. In one possible embodiment, as shown in fig. 7, the step S107 may include:
and S701, determining a target combination according to the weight information.
The target combination can represent that the first preset object and the second preset object in the combination have better matching degree.
In practical application, a complete matching corresponding combination with the maximum weight of the weighted bipartite graph can be used as a target combination based on a KM algorithm according to the weight information of at least one combination.
S702, based on the target combination, performing object association matching, and updating the object track information according to the matched object.
In practical application, a first preset object in a target combination corresponds to a track, a second preset object in the target combination is added to the track, and object track information of the track is updated according to relevant information of the second preset object.
Fig. 8 is a block diagram of an object tracking apparatus according to an embodiment of the present invention. On the other hand, as shown in fig. 8, the present embodiment also provides an object tracking apparatus, including:
a first information obtaining module 10, configured to obtain a current image and a historical image;
the second information acquisition module 20 is configured to acquire, based on the history image, respective history state information of at least one first preset object in the history image when the preset object exists in the history image;
a first information determining module 30, configured to determine, according to the historical state information, respective corresponding position prediction information of at least one first preset object;
an image acquisition type determining module 40, configured to determine an image acquisition type according to the position prediction information;
the second information determining module 50 is configured to determine at least one combination based on the current image and the historical image and determine distance information and corresponding first feature information corresponding to the at least one combination when the image acquisition type is a cross-camera acquisition type;
a third information determining module 60, configured to determine, according to the distance information and the first feature information, weight information corresponding to each of the at least one combination;
and the association matching module 70 is configured to update the object track information according to the weight information.
On the other hand, an embodiment of the present invention further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the object tracking method described above.
In another aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which computer program instructions are stored, where the computer program instructions, when executed by a processor, implement the object tracking method.
It is noted that while for simplicity of explanation, the foregoing method embodiments have been presented as a series of interrelated states or acts, it should be appreciated by those skilled in the art that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Similarly, the modules of the object tracking apparatus refer to a computer program or a program segment for executing one or more specific functions, and the distinction between the modules does not mean that actual program codes are necessarily separated. Further, the above embodiments may be arbitrarily combined to obtain other embodiments.
In the foregoing embodiments, the descriptions of the embodiments have respective emphasis, and reference may be made to related descriptions of other embodiments for parts that are not described in detail in a certain embodiment. Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.
The foregoing description has disclosed fully preferred embodiments of the present invention. It should be noted that those skilled in the art can make modifications to the embodiments of the present invention without departing from the scope of the appended claims. Accordingly, the scope of the appended claims is not to be limited to the specific embodiments described above.
Claims (10)
1. An object tracking method, the method comprising:
acquiring a current image and a historical image;
under the condition that a preset object exists in the historical image, acquiring historical state information of at least one first preset object in the historical image based on the historical image;
according to the historical state information, determining position prediction information corresponding to the at least one first preset object;
determining an image acquisition type according to the position prediction information;
under the condition that the image acquisition type is a cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining distance information and corresponding first characteristic information corresponding to the at least one combination;
determining weight information corresponding to each combination according to the distance information and the first characteristic information;
and updating the object track information according to the weight information.
2. The method according to claim 1, wherein after determining the image capturing type according to the position prediction information, further comprising:
under the condition that the image acquisition type is a non-cross-camera acquisition type, determining at least one combination based on the current image and the historical image, and determining intersection and parallel ratio information and corresponding second characteristic information corresponding to the at least one combination;
and determining the weight information corresponding to the at least one combination according to the intersection ratio information and the second characteristic information.
3. The method of claim 1, wherein determining an image acquisition type based on the location prediction information comprises:
determining the image acquisition type as the cross-camera acquisition type when the position point corresponding to the position prediction information does not belong to a preset range;
and determining that the image acquisition type is a non-cross-camera acquisition type when the position point corresponding to the position prediction information belongs to a preset range.
4. The method according to claim 1, wherein the historical state information includes historical time, position information and speed information corresponding to at least one first preset object, and the determining, according to the historical state information, position prediction information corresponding to each of the at least one first preset object includes:
acquiring the current moment;
obtaining time difference value information according to the historical time and the current time;
and determining the position prediction information corresponding to the at least one combination according to the time difference information, the position information and the speed information.
5. The method of claim 1, wherein determining at least one combination based on the current image and the historical image in the case that the image acquisition type is a cross-camera acquisition type comprises:
identifying an image corresponding to the position prediction information in the current image, and taking a preset object in the image as the second preset object;
and combining the first preset object and each second preset object in the historical image to obtain at least one combination, wherein the image acquisition device corresponding to the current image and the image acquisition device corresponding to the historical image are adjacently arranged.
6. The method of claim 1, wherein after acquiring the current image and the historical image, further comprising:
and under the condition that the preset object exists in the current image and the preset object does not exist in the historical image, updating object track information, initializing the historical state information, and returning to the step of acquiring the current image and the historical image.
7. The method of claim 1, wherein updating the object trajectory information according to the weight information comprises:
determining a target combination according to the weight information;
and performing object association matching based on the target combination, and updating the object track information according to the matched object.
8. An object tracking apparatus, characterized in that the apparatus comprises:
the first information acquisition module is used for acquiring a current image and a historical image;
the second information acquisition module is used for acquiring the respective historical state information of at least one first preset object in the historical image on the basis of the historical image under the condition that the historical image has the preset object;
the first information determining module is used for determining position prediction information corresponding to the at least one first preset object according to the historical state information;
the image acquisition type determining module is used for determining the image acquisition type according to the position prediction information;
the second information determining module is used for determining at least one combination based on the current image and the historical image and determining distance information corresponding to the at least one combination and corresponding first characteristic information under the condition that the image acquisition type is a cross-camera acquisition type;
a third information determining module, configured to determine, according to the distance information and the first feature information, weight information corresponding to each of the at least one combination;
and the association matching module is used for updating the object track information according to the weight information.
9. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute the executable instructions to implement the object tracking method of any one of claims 1 to 7.
10. A non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the object tracking method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111275859.XA CN113989761A (en) | 2021-10-29 | 2021-10-29 | Object tracking method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111275859.XA CN113989761A (en) | 2021-10-29 | 2021-10-29 | Object tracking method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113989761A true CN113989761A (en) | 2022-01-28 |
Family
ID=79744777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111275859.XA Pending CN113989761A (en) | 2021-10-29 | 2021-10-29 | Object tracking method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113989761A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114596337A (en) * | 2022-03-03 | 2022-06-07 | 捻果科技(深圳)有限公司 | Self-recognition target tracking method and system based on linkage of multiple camera positions |
| CN114862946A (en) * | 2022-06-06 | 2022-08-05 | 重庆紫光华山智安科技有限公司 | Location prediction method, system, device, and medium |
-
2021
- 2021-10-29 CN CN202111275859.XA patent/CN113989761A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114596337A (en) * | 2022-03-03 | 2022-06-07 | 捻果科技(深圳)有限公司 | Self-recognition target tracking method and system based on linkage of multiple camera positions |
| CN114596337B (en) * | 2022-03-03 | 2022-11-25 | 捻果科技(深圳)有限公司 | Self-recognition target tracking method and system based on linkage of multiple camera positions |
| CN114862946A (en) * | 2022-06-06 | 2022-08-05 | 重庆紫光华山智安科技有限公司 | Location prediction method, system, device, and medium |
| CN114862946B (en) * | 2022-06-06 | 2023-04-18 | 重庆紫光华山智安科技有限公司 | Location prediction method, system, device, and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112883819B (en) | Multi-target tracking method, device, system and computer readable storage medium | |
| CN107358623B (en) | Relevant filtering tracking method based on significance detection and robustness scale estimation | |
| US9323991B2 (en) | Method and system for video-based vehicle tracking adaptable to traffic conditions | |
| US9390328B2 (en) | Static occlusion handling using directional pixel replication in regularized motion environments | |
| CN110610150B (en) | Tracking method, device, computing equipment and medium of target moving object | |
| CN106682619B (en) | Object tracking method and device | |
| CN108647587B (en) | People counting method, device, terminal and storage medium | |
| Klippenstein et al. | Quantitative evaluation of feature extractors for visual slam | |
| CN113989761A (en) | Object tracking method and device, electronic equipment and storage medium | |
| KR20160144149A (en) | A video surveillance apparatus for removing overlap and tracking multiple moving objects and method thereof | |
| CN115049700A (en) | Target detection method and device | |
| CN115063454B (en) | Multi-target tracking matching method, device, terminal and storage medium | |
| CN104933398A (en) | vehicle identification system and method | |
| CN105374049B (en) | Multi-corner point tracking method and device based on sparse optical flow method | |
| CN113469201A (en) | Image acquisition equipment offset detection method, image matching method, system and equipment | |
| CN115144828B (en) | Automatic online calibration method for intelligent automobile multi-sensor space-time fusion | |
| CN108898057B (en) | Method, device, computer equipment and storage medium for tracking target detection | |
| EP3044734B1 (en) | Isotropic feature matching | |
| Heimbach et al. | Improving object tracking accuracy in video sequences subject to noise and occlusion impediments by combining feature tracking with Kalman filtering | |
| CN115019241B (en) | Pedestrian identification and tracking method and device, readable storage medium and equipment | |
| Revaud et al. | Robust automatic monocular vehicle speed estimation for traffic surveillance | |
| KR101976952B1 (en) | System and method for detecting object using motion vector | |
| CN113869163B (en) | Target tracking method and device, electronic equipment and storage medium | |
| CN112802112B (en) | Visual positioning method, device, server and storage medium | |
| CN115797405A (en) | Multi-lens self-adaptive tracking method based on vehicle wheel base |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |