WO2021227519A1 - Target tracking method and apparatus, and computer-readable storage medium and robot - Google Patents

Target tracking method and apparatus, and computer-readable storage medium and robot Download PDF

Info

Publication number
WO2021227519A1
WO2021227519A1 PCT/CN2020/140413 CN2020140413W WO2021227519A1 WO 2021227519 A1 WO2021227519 A1 WO 2021227519A1 CN 2020140413 W CN2020140413 W CN 2020140413W WO 2021227519 A1 WO2021227519 A1 WO 2021227519A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
tracking
threshold
confidence
search area
Prior art date
Application number
PCT/CN2020/140413
Other languages
French (fr)
Chinese (zh)
Inventor
胡淑萍
程骏
张惊涛
郭渺辰
王东
顾在旺
庞建新
熊友军
Original Assignee
深圳市优必选科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市优必选科技股份有限公司 filed Critical 深圳市优必选科技股份有限公司
Publication of WO2021227519A1 publication Critical patent/WO2021227519A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • This application belongs to the field of robotics, and in particular relates to a target tracking method, device, computer-readable storage medium, and a robot.
  • the target tracking algorithm should be activated after the target is detected.
  • the tracking algorithm of the target has the advantages of high real-time performance/a certain anti-occlusion ability, but due to the limitation of the tracking algorithm itself, the current tracking algorithm cannot judge whether the target is lost, so the current tracking continues even after the tracking is lost. The situation of the process.
  • the current tracking algorithm trains related filters in real time, as time goes by, the target undergoes more and more deformations and occlusions, and the features learned by the algorithm contain more and more background information and more and more chaotic. Under circumstances, the wrong background information may be learned, and the tracking of the target may be gradually lost.
  • the embodiments of the present application provide a target tracking method, device, computer-readable storage medium, and robot to solve the problem that the current tracking algorithm cannot determine whether the target is lost, so that the current tracking process continues after the tracking is lost. Circumstances, and the problem of losing track of the target over time.
  • the first aspect of the embodiments of the present application provides a target tracking method, which may include:
  • the parameters of the correlation filter are updated, and the second threshold is greater than the first threshold.
  • the method further includes:
  • An initialization process is performed, and the initialization process includes: initializing the relevant filter.
  • the initialization of the correlation filter includes:
  • x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center
  • m is a preset multiple parameter
  • y is the m of the initial detection frame with the initial coordinates as the center
  • ⁇ 2 is the variance of the Gaussian function
  • Is the Fourier transform
  • ⁇ is the penalty constant for training
  • ⁇ represents the matrix item-by-item multiplication operation
  • ⁇ is the parameter of the correlation filter.
  • the tracking the target in the search area to obtain the position coordinates and the confidence of the target includes:
  • the confidence level is calculated according to the response matrix.
  • the calculating the confidence degree according to the response matrix includes:
  • p max is the largest response value in the response matrix, and r is the confidence level
  • CF w,h is the value in the wth row and h column of the response matrix, 1 ⁇ w ⁇ W, W is the number of rows of the response matrix, 1 ⁇ h ⁇ H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
  • the target tracking method may further include:
  • the parameters of the correlation filter are no longer updated.
  • said updating the parameters of the relevant filter includes:
  • x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center
  • y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation
  • the label of the filter ⁇ is the weight constant for parameter update.
  • the second aspect of the embodiments of the present application provides a target tracking device, which may include:
  • the search area determination module is used to determine the search area according to the position coordinates of the target and the detection frame;
  • a tracking module configured to track the target in the search area to obtain the position coordinates and confidence of the target
  • a tracking loss determination module configured to determine that the target has been tracked if the confidence level is less than a preset first threshold, and end the current tracking process
  • the model update module is configured to update the parameters of the correlation filter with the confidence level greater than a preset second threshold, where the second threshold is greater than the first threshold.
  • the target tracking device may further include:
  • the initial detection module is used to perform target detection in the designated area and determine the initial coordinates and initial detection frame of the detected target;
  • the initialization module is used to perform the initialization process, and the initialization process includes: initializing the relevant filter.
  • the initialization module may include:
  • the correlation filter initialization unit is used to initialize the parameters of the correlation filter according to the following formula:
  • x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center
  • m is a preset multiple parameter
  • y is the m of the initial detection frame with the initial coordinates as the center
  • ⁇ 2 is the variance of the Gaussian function
  • Is the Fourier transform
  • ⁇ is the penalty constant for training
  • ⁇ represents the matrix item-by-item multiplication operation
  • ⁇ is the parameter of the correlation filter.
  • the tracking module may include:
  • the response matrix calculation unit is configured to use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
  • a position coordinate determination unit configured to determine the maximum response value in the response matrix as the maximum response value, and determine the position coordinate corresponding to the maximum response value as the position coordinate;
  • the confidence calculation unit is configured to calculate the confidence according to the response matrix.
  • the confidence calculation unit is specifically configured to:
  • p max is the largest response value in the response matrix, and r is the confidence level
  • CF w,h is the value in the wth row and h column of the response matrix, 1 ⁇ w ⁇ W, W is the number of rows of the response matrix, 1 ⁇ h ⁇ H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
  • model update module is specifically configured to update the parameters of the relevant filter according to the following formula:
  • x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center
  • y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation
  • the label of the filter ⁇ is the weight constant for parameter update.
  • a third aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the steps of any of the foregoing target tracking methods.
  • the fourth aspect of the embodiments of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and running on the processor.
  • the processor executes the computer program when the computer program is executed. Steps of any of the above-mentioned target tracking methods.
  • the fifth aspect of the embodiments of the present application provides a computer program product.
  • the computer program product runs on a robot, the robot executes the steps of any of the above-mentioned target tracking methods.
  • the embodiment of the application determines the search area according to the position coordinates and the detection frame of the target; the target is tracked in the search area to obtain the target The position coordinates and the confidence level of; the confidence level is less than the preset first threshold, and the current tracking process is ended.
  • the confidence is introduced to evaluate the tracking effect.
  • the confidence is low, that is, less than the preset first threshold, it means that the target has been lost.
  • the current tracking process can be ended to avoid the situation that the current tracking process continues after the tracking is lost.
  • the confidence level is high, that is, greater than the preset second threshold, the parameters of the relevant filter will be updated, so as to avoid learning wrong background information and keep track of the target.
  • Figure 1 is a schematic flow chart of the initialization process
  • FIG. 2 is a flowchart of an embodiment of a target tracking method in an embodiment of the application
  • Figure 3 is a schematic diagram of the correlation between the detection frame and the search area of the target
  • FIG. 4 is a structural diagram of an embodiment of a target tracking device in an embodiment of this application.
  • Fig. 5 is a schematic block diagram of a robot in an embodiment of the application.
  • the term “if” can be interpreted as “when” or “once” or “in response to determination” or “in response to detection” depending on the context .
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • the target to be tracked may include, but is not limited to: people, vehicles, animals, and other moving objects.
  • the existing target tracking algorithms are divided into short-term tracking algorithms and long-term tracking algorithms: short-term tracking algorithms, that is, the target can be tracked well in a short time (less than 2 minutes), and as time goes by With the passage of time, the target undergoes more and more deformation and occlusion, the features learned by the algorithm become more and more chaotic, and the tracking of the target will gradually be lost; in the existing long-term tracking algorithm (2-20 minutes), in addition to the need for real-time tracking And the training tracker, it also comes with a detector. It also conducts real-time training during the tracking process.
  • the most classic long-term tracking algorithm is the Tracking-Learning-Detection (TLD) algorithm.
  • the embodiment of the application introduces the confidence level on the basis of the short-term tracker to realize the efficient tracking and tracking of the target: the correlation filter learns the characteristics of the target, and outputs the coordinates of the pixel point with the largest response to the target characteristic in the current frame To track the target, the response value of the current frame to the target feature can be used as the confidence level.
  • the correlation filter learns the characteristics of the target, and outputs the coordinates of the pixel point with the largest response to the target characteristic in the current frame
  • the response value of the current frame to the target feature can be used as the confidence level.
  • the features learned by the algorithm contain more and more background information and more and more chaotic conditions.
  • the confidence level can be judged during the tracking process, and the tracker is updated only when the tracking confidence level is high. , In case you learn the wrong background information. In the same way, the confidence is judged during the tracking process. When the tracking confidence is low, it means that the target is lost and the target needs to be re-detected.
  • target detection is the prerequisite for target tracking.
  • the target detector needs to be used for target detection in the designated area.
  • the detection method used can be any detection method in the prior art.
  • the embodiments of the present application do not specifically limit this.
  • the position coordinates and the bounding box (bb) of the target in the frame of image at this time are determined.
  • this position is marked as the initial coordinates
  • this One detection frame is marked as the initial detection frame.
  • the initialization process of target tracking can be performed.
  • the initialization process may include:
  • Step S101 Initialize the value of k.
  • k is the number of the image frame for tracking the target, and k is a positive integer.
  • the frame of image where the target is detected is regarded as the 0th frame.
  • Step S102 Initialize the correlation filter (CorrelationFilter, CF).
  • the parameters of the relevant filter are initialized according to the following formula:
  • x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center
  • m is a preset multiple parameter, and its specific value can be set according to the actual situation.
  • it is set here Set to 2.5, that is, the search area is 2.5 times the size of the detection frame
  • y is the Gaussian function of the area m times the size of the initial detection frame with the initial coordinates as the center, as the label for training the relevant filter
  • ⁇ 2 Is the variance of the Gaussian function, Is the Fourier transform, Inverse Fourier transform
  • ⁇ is the penalty constant for training
  • ⁇ represents the matrix item-by-item multiplication operation
  • ⁇ is the parameter of the correlation filter.
  • the process of tracking the target in the k-th frame of image may include:
  • Step S201 Determine a search area in the k-th frame of image according to the position coordinates of the target in the k-1th frame of image and the detection frame.
  • the dashed rectangular frame represents the detection frame of the target tracked in the k-1th frame
  • the search area in the kth frame is m times the size of the detection frame, as shown by the solid line frame in the figure.
  • the search area is centered on the position coordinates of the target in the k-1th frame of image.
  • Step S202 Track the target in the search area to obtain the position coordinates and confidence of the target in the k-th frame of image.
  • a correlation filter is used to track the target.
  • the target tracking algorithm may be, but is not limited to, such as Kalman filter, target detection and tracking, deep learning-based tracking algorithm, multi-target tracking algorithm, and so on.
  • a correlation filter can be used to track the target in the search area to obtain a response matrix as shown below:
  • z is the image of the search area
  • CF is the response matrix
  • the maximum response value in the response matrix may be determined as the maximum response value, and the position coordinates corresponding to the maximum response value may be determined as the position coordinates of the target in the k-th frame of image .
  • the confidence level can be calculated according to the following formula:
  • p max is the largest response value in the response matrix
  • r is the confidence level
  • the confidence level can be calculated according to the following formula:
  • CF w,h is the value in the wth row and h column of the response matrix, 1 ⁇ w ⁇ W, W is the number of rows of the response matrix, 1 ⁇ h ⁇ H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
  • Step S203 Perform an operation corresponding to the confidence level.
  • the confidence level is less than the preset first threshold, it is determined that the target has been lost, and the current tracking process is ended. In this process, you can also feedback the lost information to the preset control center.
  • the parameters of the correlation filter are updated.
  • the second threshold value is greater than the first threshold value, and specific values of the first threshold value and the second threshold value may be set according to actual conditions, which are not specifically limited in the embodiment of the present application.
  • the parameters of the relevant filter can be updated according to the following formula:
  • x is the image of the area m times the size of the detection frame in the k-th frame with the position coordinates of the target in the k-th frame image as the center
  • y is the position coordinates of the target in the k-th frame image As the center
  • ⁇ is the weight constant for parameter update.
  • the parameters of the relevant filter are no longer updated. If the current tracking process has not ended, directly continue to the next frame of image
  • the embodiment of the present application determines the search area according to the position coordinates and detection frame of the target; tracks the target in the search area to obtain the position coordinates and confidence of the target; if the confidence is If the degree is less than the preset first threshold, it is determined that the target has been lost, and the current tracking process is ended.
  • the confidence is introduced to evaluate the tracking effect. When the confidence is low, it means that the target has been lost. At this time, the current tracking process can be ended to avoid There is a situation where the current tracking process continues after the track is lost.
  • the confidence level is high, that is, greater than the preset second threshold, the parameters of the relevant filter will be updated, so as to avoid learning wrong background information and keep track of the target.
  • FIG. 4 shows a structural diagram of an embodiment of a target tracking device provided in an embodiment of the present application.
  • a target tracking device may include:
  • the search area determination module 401 is configured to determine the search area according to the position coordinates and the detection frame of the target;
  • the tracking module 402 is configured to track the target in the search area to obtain the position coordinates and confidence of the target;
  • the tracking loss determination module 403 is configured to terminate the current tracking process when the confidence level is less than a preset first threshold
  • the model update module 404 is configured to update the parameters of the correlation filter with the confidence level greater than a preset second threshold, where the second threshold is greater than the first threshold.
  • the target tracking device may further include:
  • the initial detection module is used to perform target detection in the designated area and determine the initial coordinates and initial detection frame of the detected target;
  • the initialization module is used to perform the initialization process, and the initialization process includes: initializing the relevant filter.
  • the initialization module may include:
  • the correlation filter initialization unit is used to initialize the parameters of the correlation filter according to the following formula:
  • x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center
  • m is a preset multiple parameter
  • y is the m of the initial detection frame with the initial coordinates as the center
  • ⁇ 2 is the variance of the Gaussian function
  • Is the Fourier transform
  • ⁇ is the penalty constant for training
  • ⁇ represents the matrix item-by-item multiplication operation
  • ⁇ is the parameter of the correlation filter.
  • the tracking module may include:
  • the response matrix calculation unit is configured to use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
  • a position coordinate determination unit configured to determine the maximum response value in the response matrix as the maximum response value, and determine the position coordinate corresponding to the maximum response value as the position coordinate;
  • the confidence calculation unit is configured to calculate the confidence according to the response matrix.
  • the confidence calculation unit is specifically configured to:
  • p max is the largest response value in the response matrix, and r is the confidence level
  • CF w,h is the value in the wth row and h column of the response matrix, 1 ⁇ w ⁇ W, W is the number of rows of the response matrix, 1 ⁇ h ⁇ H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
  • model update module is specifically configured to update the parameters of the relevant filter according to the following formula:
  • x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center
  • y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation
  • the label of the filter ⁇ is the weight constant for parameter update.
  • Fig. 5 shows a schematic block diagram of a robot provided by an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.
  • the robot 5 of this embodiment includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and running on the processor 50.
  • the processor 50 executes the computer program 52, the steps in the foregoing target tracking method embodiments are implemented.
  • the processor 50 executes the computer program 52, the functions of the modules/units in the foregoing device embodiments are realized.
  • the computer program 52 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 51 and executed by the processor 50 to complete This application.
  • the one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the robot 5.
  • FIG. 5 is only an example of the robot 5, and does not constitute a limitation on the robot 5. It may include more or less parts than shown, or a combination of some parts, or different parts, such as
  • the robot 5 may also include input and output devices, network access devices, buses, and the like.
  • the processor 50 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 51 may be an internal storage unit of the robot 5, such as a hard disk or memory of the robot 5.
  • the memory 51 may also be an external storage device of the robot 5, such as a plug-in hard disk equipped on the robot 5, a smart memory card (Smart Media Card, SMC), or a Secure Digital (SD) card, Flash Card, etc. Further, the memory 51 may also include both an internal storage unit of the robot 5 and an external storage device.
  • the memory 51 is used to store the computer program and other programs and data required by the robot 5.
  • the memory 51 can also be used to temporarily store data that has been output or will be output.
  • the disclosed device/robot and method can be implemented in other ways.
  • the device/robot embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program.
  • the computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electrical carrier signal telecommunications signal
  • software distribution media etc.
  • the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction.
  • the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

A target tracking method and apparatus, and a computer-readable storage medium and a robot, which belong to the technical field of robots. The method comprises: determining a search area according to the position coordinates of a target and a detection box thereof; tracking the target within the search area, so as to obtain the position coordinates and confidence of the target; and when the confidence is less than a preset first threshold, ending the current tracking process. By means of the method, during the process of tracking a target, confidence is introduced to evaluate a tracking effect. When the confidence is relatively low, that is, less than the preset first threshold, it is indicated that the target has been lost, and at this time the current tracking process can be ended, thereby preventing the situation of continuing the current tracking process after the target has been lost; and when the confidence is relatively high, that is, greater than a preset second threshold, parameters of a relevant filter are updated, thereby preventing the learning of incorrect background information and maintaining the tracking of the target.

Description

目标跟踪方法、装置、计算机可读存储介质及机器人Target tracking method, device, computer readable storage medium and robot
本申请要求于2020年5月15日在中国专利局提交的、申请号为202010413963.X的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with application number 202010413963.X filed at the Chinese Patent Office on May 15, 2020, the entire content of which is incorporated into this application by reference.
技术领域Technical field
本申请属于机器人技术领域，尤其涉及一种目标跟踪方法、装置、计算机可读存储介质及机器人。This application belongs to the field of robotics, and in particular relates to a target tracking method, device, computer-readable storage medium, and a robot.
背景技术Background technique
安防机器人作为用于协助人类完成安全防护工作的机器人，需要具备对安防场所内可能出现的目标进行检测和跟踪的能力。然而检测算法的高复杂度/低实时性及容易受到遮挡影响的特性限制了安防机器人对安防场所内出现的目标的实时跟踪能力，因此，在检测到目标之后应当启用目标跟踪算法。目前，目标的跟踪算法具有高实时性/具备一定的抗遮挡能力等优点，但是由于跟踪算法本身的局限性，目前的跟踪算法无法判断目标是否跟丢，从而出现在跟丢后仍然持续当前跟踪进程的情况。而且，目前的跟踪算法在实时训练相关滤波器时，随着时间的推移，目标所经历的形变和遮挡越来越多，算法学习的特征包含越来越多的背景信息、越来越混乱的情况，可能学习到错误的背景信息，会逐渐丢失对目标的跟踪。As a robot used to assist humans in completing safety protection work, security robots need to have the ability to detect and track targets that may appear in the security area. However, the high complexity/low real-time nature of the detection algorithm and the characteristics of being easily affected by occlusion limit the real-time tracking ability of the security robot to the target appearing in the security place. Therefore, the target tracking algorithm should be activated after the target is detected. At present, the tracking algorithm of the target has the advantages of high real-time performance/a certain anti-occlusion ability, but due to the limitation of the tracking algorithm itself, the current tracking algorithm cannot judge whether the target is lost, so the current tracking continues even after the tracking is lost. The situation of the process. Moreover, when the current tracking algorithm trains related filters in real time, as time goes by, the target undergoes more and more deformations and occlusions, and the features learned by the algorithm contain more and more background information and more and more chaotic. Under circumstances, the wrong background information may be learned, and the tracking of the target may be gradually lost.
技术问题technical problem
有鉴于此，本申请实施例提供了一种目标跟踪方法、装置、计算机可读存储介质及机器人，以解决目前的跟踪算法无法判断目标是否跟丢，从而出现在跟丢后仍然持续当前跟踪进程的情况，以及随着时间推移会逐渐丢失对目标的跟踪的问题。In view of this, the embodiments of the present application provide a target tracking method, device, computer-readable storage medium, and robot to solve the problem that the current tracking algorithm cannot determine whether the target is lost, so that the current tracking process continues after the tracking is lost. Circumstances, and the problem of losing track of the target over time.
技术解决方案Technical solutions
本申请实施例的第一方面提供了一种目标跟踪方法，可以包括：The first aspect of the embodiments of the present application provides a target tracking method, which may include:
根据目标的位置坐标和检测框，确定搜索区域；Determine the search area according to the location coordinates of the target and the detection frame;
在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；Tracking the target in the search area to obtain the position coordinates and confidence of the target;
所述置信度小于预设的第一阈值，结束当前跟踪进程；If the confidence level is less than the preset first threshold, the current tracking process is ended;
所述置信度大于预设的第二阈值，对相关滤波器的参数进行更新，所述第二阈值大于所述第一阈值。If the confidence level is greater than a preset second threshold, the parameters of the correlation filter are updated, and the second threshold is greater than the first threshold.
进一步地，在所述搜索区域中对所述目标进行跟踪之前，还包括：Further, before tracking the target in the search area, the method further includes:
在指定区域中进行目标检测，并确定检测到的目标的初始坐标和初始检测框；Perform target detection in the designated area, and determine the initial coordinates and initial detection frame of the detected target;
执行初始化过程，所述初始化过程包括：对相关滤波器进行初始化。An initialization process is performed, and the initialization process includes: initializing the relevant filter.
进一步地，所述对相关滤波器进行初始化包括：Further, the initialization of the correlation filter includes:
根据下式初始化相关滤波器的参数：Initialize the parameters of the relevant filter according to the following formula:
Figure PCTCN2020140413-appb-000001
Figure PCTCN2020140413-appb-000001
其中，
Figure PCTCN2020140413-appb-000002
x为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的图像，m为预设的倍数参数，y为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，σ 2为高斯函数的方差，
Figure PCTCN2020140413-appb-000003
为傅里叶变换，
Figure PCTCN2020140413-appb-000004
傅里叶反变换，λ为训练的罚项常数，□表示矩阵逐项相乘操作，α为相关滤波器的参数。 in,
Figure PCTCN2020140413-appb-000002
x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center, m is a preset multiple parameter, and y is the m of the initial detection frame with the initial coordinates as the center The Gaussian function of the double size area is used as the label of the training correlation filter, σ 2 is the variance of the Gaussian function,
Figure PCTCN2020140413-appb-000003
Is the Fourier transform,
Figure PCTCN2020140413-appb-000004
Inverse Fourier transform, λ is the penalty constant for training, □ represents the matrix item-by-item multiplication operation, and α is the parameter of the correlation filter.
进一步地，所述在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度，包括：Further, the tracking the target in the search area to obtain the position coordinates and the confidence of the target includes:
使用相关滤波器在所述搜索区域中对所述目标进行跟踪，得到如下所示的响应矩阵：Use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
Figure PCTCN2020140413-appb-000005
Figure PCTCN2020140413-appb-000005
其中，z为所述搜索区域的图像，CF为所述响应矩阵；Where z is the image of the search area, and CF is the response matrix;
将所述响应矩阵中最大的响应值确定为所述最大响应值，并将与所述最大响应值对应的位置坐标确定为所述位置坐标；Determining the maximum response value in the response matrix as the maximum response value, and determining the position coordinate corresponding to the maximum response value as the position coordinate;
根据所述响应矩阵计算所述置信度。The confidence level is calculated according to the response matrix.
进一步地，所述根据所述响应矩阵计算所述置信度，包括：Further, the calculating the confidence degree according to the response matrix includes:
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
r＝p max r=p max
其中，p max为所述响应矩阵中最大的响应值，r为所述置信度； Wherein, p max is the largest response value in the response matrix, and r is the confidence level;
或者or
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
Figure PCTCN2020140413-appb-000006
Figure PCTCN2020140413-appb-000006
其中，CF w,h为所述响应矩阵中第w行h列的数值，1≤w≤W，W为所述响应矩阵的行数，1≤h≤H，H为所述响应矩阵的列数，p min为所述响应矩阵中最小的响应值。 Where CF w,h is the value in the wth row and h column of the response matrix, 1≤w≤W, W is the number of rows of the response matrix, 1≤h≤H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
进一步地，所述目标跟踪方法还可以包括：Further, the target tracking method may further include:
若所述置信度小于等于所述第二阈值，且大于等于所述第一阈值，则不再对相关滤波器的参数进行更新。If the confidence level is less than or equal to the second threshold and greater than or equal to the first threshold, the parameters of the correlation filter are no longer updated.
进一步地，所述对相关滤波器的参数进行更新，包括：Further, said updating the parameters of the relevant filter includes:
根据下式更新相关滤波器的参数：Update the parameters of the relevant filter according to the following formula:
α＝(1-γ)α+γα′α=(1-γ)α+γα′
其中，
Figure PCTCN2020140413-appb-000007
x为以所述目标的位置坐标为中心，取检测框的m倍大小区域的图像，y为以所述目标的位置坐标为中心，取检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，γ为参数更新的权重常数。 in,
Figure PCTCN2020140413-appb-000007
x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center, y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation The label of the filter, γ is the weight constant for parameter update.
本申请实施例的第二方面提供了一种目标跟踪装置，可以包括：The second aspect of the embodiments of the present application provides a target tracking device, which may include:
搜索区域确定模块，用于根据目标的位置坐标和检测框，确定搜索区域；The search area determination module is used to determine the search area according to the position coordinates of the target and the detection frame;
跟踪模块，用于在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；A tracking module, configured to track the target in the search area to obtain the position coordinates and confidence of the target;
跟丢判定模块，用于若所述置信度小于预设的第一阈值，则判定所述目标已跟丢，结束当前跟踪进程；A tracking loss determination module, configured to determine that the target has been tracked if the confidence level is less than a preset first threshold, and end the current tracking process;
模型更新模块，用于所述置信度大于预设的第二阈值，对相关滤波器的参数进行更新，所述第二阈值大于所述第一阈值。The model update module is configured to update the parameters of the correlation filter with the confidence level greater than a preset second threshold, where the second threshold is greater than the first threshold.
进一步地，所述目标跟踪装置还可以包括：Further, the target tracking device may further include:
初始检测模块，用于在指定区域中进行目标检测，并确定检测到的目标的初始坐标和初始检测框；The initial detection module is used to perform target detection in the designated area and determine the initial coordinates and initial detection frame of the detected target;
初始化模块，用于执行初始化过程，所述初始化过程包括：对相关滤波器进行初始化。The initialization module is used to perform the initialization process, and the initialization process includes: initializing the relevant filter.
进一步地，所述初始化模块可以包括：Further, the initialization module may include:
相关滤波器初始化单元，用于根据下式初始化相关滤波器的参数：The correlation filter initialization unit is used to initialize the parameters of the correlation filter according to the following formula:
Figure PCTCN2020140413-appb-000008
Figure PCTCN2020140413-appb-000008
其中，
Figure PCTCN2020140413-appb-000009
x为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的图像，m为预设的倍数参数，y为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，σ 2为高斯函数的方差，
Figure PCTCN2020140413-appb-000010
为傅里叶变换，
Figure PCTCN2020140413-appb-000011
傅里叶反变换，λ为训练的罚项常数，□表示矩阵逐项相乘操作，α为相关滤波器的参数。 in,
Figure PCTCN2020140413-appb-000009
x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center, m is a preset multiple parameter, and y is the m of the initial detection frame with the initial coordinates as the center The Gaussian function of the double size area is used as the label of the training correlation filter, σ 2 is the variance of the Gaussian function,
Figure PCTCN2020140413-appb-000010
Is the Fourier transform,
Figure PCTCN2020140413-appb-000011
Inverse Fourier transform, λ is the penalty constant for training, □ represents the matrix item-by-item multiplication operation, and α is the parameter of the correlation filter.
进一步地，所述跟踪模块可以包括：Further, the tracking module may include:
响应矩阵计算单元，用于使用相关滤波器在所述搜索区域中对所述目标进行跟踪，得到如下所示的响应矩阵：The response matrix calculation unit is configured to use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
Figure PCTCN2020140413-appb-000012
Figure PCTCN2020140413-appb-000012
其中，z为所述搜索区域的图像，CF为所述响应矩阵；Where z is the image of the search area, and CF is the response matrix;
位置坐标确定单元，用于将所述响应矩阵中最大的响应值确定为所述最大响应值，并将与所述最大响应值对应的位置坐标确定为所述位置坐标；A position coordinate determination unit, configured to determine the maximum response value in the response matrix as the maximum response value, and determine the position coordinate corresponding to the maximum response value as the position coordinate;
置信度计算单元，用于根据所述响应矩阵计算所述置信度。The confidence calculation unit is configured to calculate the confidence according to the response matrix.
进一步地，所述置信度计算单元具体用于：Further, the confidence calculation unit is specifically configured to:
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
r＝p max r=p max
其中，p max为所述响应矩阵中最大的响应值，r为所述置信度； Wherein, p max is the largest response value in the response matrix, and r is the confidence level;
或者or
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
Figure PCTCN2020140413-appb-000013
Figure PCTCN2020140413-appb-000013
其中，CF w,h为所述响应矩阵中第w行h列的数值，1≤w≤W，W为所述响应矩阵的行数，1≤h≤H，H为所述响应矩阵的列数，p min为所述响应矩阵中最小的响应值。 Where CF w,h is the value in the wth row and h column of the response matrix, 1≤w≤W, W is the number of rows of the response matrix, 1≤h≤H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
进一步地，所述模型更新模块具体用于根据下式更新相关滤波器的参数：Further, the model update module is specifically configured to update the parameters of the relevant filter according to the following formula:
α＝(1-γ)α+γα′α=(1-γ)α+γα′
其中，
Figure PCTCN2020140413-appb-000014
x为以所述目标的位置坐标为中心，取检测框的m倍大小区域的图像，y为以所述目标的位置坐标为中心，取检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，γ为参数更新的权重常数。 in,
Figure PCTCN2020140413-appb-000014
x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center, y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation The label of the filter, γ is the weight constant for parameter update.
本申请实施例的第三方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述任一种目标跟踪方法的步骤。A third aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the steps of any of the foregoing target tracking methods.
本申请实施例的第四方面提供了一种机器人，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述任一种目标跟踪方法的步骤。The fourth aspect of the embodiments of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and running on the processor. The processor executes the computer program when the computer program is executed. Steps of any of the above-mentioned target tracking methods.
本申请实施例的第五方面提供了一种计算机程序产品，当计算机程序产品在机器人上运行时，使得机器人执行上述任一种目标跟踪方法的步骤。The fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on a robot, the robot executes the steps of any of the above-mentioned target tracking methods.
有益效果Beneficial effect
本申请实施例与现有技术相比存在的有益效果是：本申请实施例根据目标的位置坐标和检测框，确定搜索区域；在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；所述置信度小于预设的第一阈值，结束当前跟踪进程。通过本申请实施例，在对目标的跟踪过程中，引入了置信度来对跟踪的效果进行评估，当置信度较低，即小于预设的第一阈值时，则说明目标已跟丢，此时则可以结束当前跟踪进程，避免出现在跟丢后仍然持续当前跟踪进程的情况。当置信度较高，即大于预设的第二阈值时，才会对相关滤波器的参数进行更新，从而避免学习到错误的背景信息，保持对于目标的跟踪。Compared with the prior art, the embodiment of the application has the following beneficial effects: the embodiment of the application determines the search area according to the position coordinates and the detection frame of the target; the target is tracked in the search area to obtain the target The position coordinates and the confidence level of; the confidence level is less than the preset first threshold, and the current tracking process is ended. Through the embodiment of this application, in the process of tracking the target, the confidence is introduced to evaluate the tracking effect. When the confidence is low, that is, less than the preset first threshold, it means that the target has been lost. At this time, the current tracking process can be ended to avoid the situation that the current tracking process continues after the tracking is lost. When the confidence level is high, that is, greater than the preset second threshold, the parameters of the relevant filter will be updated, so as to avoid learning wrong background information and keep track of the target.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.
图1为初始化过程的示意流程图；Figure 1 is a schematic flow chart of the initialization process;
图2为本申请实施例中一种目标跟踪方法的一个实施例流程图；2 is a flowchart of an embodiment of a target tracking method in an embodiment of the application;
图3为目标的检测框及搜索区域的相关关系示意图；Figure 3 is a schematic diagram of the correlation between the detection frame and the search area of the target;
图4为本申请实施例中一种目标跟踪装置的一个实施例结构图；FIG. 4 is a structural diagram of an embodiment of a target tracking device in an embodiment of this application;
图5为本申请实施例中一种机器人的示意框图。Fig. 5 is a schematic block diagram of a robot in an embodiment of the application.
本发明的实施方式Embodiments of the present invention
为使得本申请的发明目的、特征、优点能够更加的明显和易懂，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，下面所描述的实施例仅仅是本申请一部分实施例，而非全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。In order to make the purposes, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the following The described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the existence of the described features, wholes, steps, operations, elements and/or components, but does not exclude one or more other features , The existence or addition of a whole, a step, an operation, an element, a component, and/or a collection thereof.
还应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。It should be further understood that the term "and/or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释 为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "once" or "in response to determination" or "in response to detection" depending on the context . Similarly, the phrase "if determined" or "if detected [described condition or event]" can be interpreted as meaning "once determined" or "in response to determination" or "once detected [described condition or event]" depending on the context ]" or "in response to detection of [condition or event described]".
另外，在本申请的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。In addition, in the description of this application, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.
在本申请实施例中，所需跟踪的目标可以包括但不限于：人员、车辆、动物以及其它运动物体。In the embodiment of the present application, the target to be tracked may include, but is not limited to: people, vehicles, animals, and other moving objects.
目前已有的目标跟踪算法分为短时跟踪算法和长时跟踪算法两种：短时跟踪算法，就是可以在短时间内(小于2分钟)对目标进行较好地跟踪，而随着时间的推移，目标所经历的形变和遮挡越来越多，算法学习的特征越来越混乱，会逐渐丢失对目标的跟踪；在已有长时跟踪算法(2-20分钟)中，除了需要实时跟踪和训练***，还自带一个检测器，同样也在跟踪过程中进行实时训练，最经典的长时跟踪算法为跟踪学习检测(Tracking-Learning-Detection，TLD)算法，这种跟踪算法由于同时训练检测器，在长时间跟踪的情况下不容易跟丢，但是检测器本身复杂度就较高，多余的训练也同时导致了算法的复杂度大幅度提升。能够实时训练的检测算法往往检测效果也较差，短时的跟踪效果也没有短时跟踪算法好，因此总的来看跟踪效果并不符合要求。无论是已有的短时跟踪算法还是长时跟踪算法，目标都有可能跟丢，但是都存在算法无法判断目标是否跟丢的问题。The existing target tracking algorithms are divided into short-term tracking algorithms and long-term tracking algorithms: short-term tracking algorithms, that is, the target can be tracked well in a short time (less than 2 minutes), and as time goes by With the passage of time, the target undergoes more and more deformation and occlusion, the features learned by the algorithm become more and more chaotic, and the tracking of the target will gradually be lost; in the existing long-term tracking algorithm (2-20 minutes), in addition to the need for real-time tracking And the training tracker, it also comes with a detector. It also conducts real-time training during the tracking process. The most classic long-term tracking algorithm is the Tracking-Learning-Detection (TLD) algorithm. Training the detector is not easy to follow in the case of long-term tracking, but the complexity of the detector itself is relatively high, and the redundant training also leads to a substantial increase in the complexity of the algorithm. Detection algorithms that can be trained in real time often have poor detection effects, and short-term tracking effects are not as good as short-term tracking algorithms. Therefore, the tracking effect does not meet the requirements in general. Whether it is an existing short-term tracking algorithm or a long-term tracking algorithm, the target may be lost, but there is a problem that the algorithm cannot determine whether the target is lost.
本申请实施例在短时***的基础上引入置信度，来实现对目标的高效跟踪及跟丢判断：相关滤波器学习目标的特征，并且输出当前帧中对目标特征响应最大的像素点坐标来对目标进行跟踪，此处可以以当前帧对目标特征的响应值为置信度，为了应对短时跟踪算法在实时训练相关滤波器时，随着时间的推移，目标所经历的形变和遮挡越来越多，算法学习的特征包含越来越多的背景信息、越来越混乱的情况，可以在跟踪过程中对置信度进行判断，只有当跟踪的置信度较高时才对***进行更新，以防学习到错误的背景信息。同理，在跟踪过程中对置信度进行判断，当跟踪的置信度较低时，表示目标被跟丢，需要对目标重新检测。The embodiment of the application introduces the confidence level on the basis of the short-term tracker to realize the efficient tracking and tracking of the target: the correlation filter learns the characteristics of the target, and outputs the coordinates of the pixel point with the largest response to the target characteristic in the current frame To track the target, the response value of the current frame to the target feature can be used as the confidence level. In order to deal with the short-term tracking algorithm in real-time training of the relevant filter, as time goes by, the target undergoes more deformation and occlusion. More and more, the features learned by the algorithm contain more and more background information and more and more chaotic conditions. The confidence level can be judged during the tracking process, and the tracker is updated only when the tracking confidence level is high. , In case you learn the wrong background information. In the same way, the confidence is judged during the tracking process. When the tracking confidence is low, it means that the target is lost and the target needs to be re-detected.
容易理解地，目标检测是进行目标跟踪的前提，在初始状态下，首先需要通过目标检测器在指定区域中进行目标检测，其中使用的检测方法可以为现有技术中的任意一种检测方法，本申请实施例对此不作具体限定。It is easy to understand that target detection is the prerequisite for target tracking. In the initial state, the target detector needs to be used for target detection in the designated area. The detection method used can be any detection method in the prior art. The embodiments of the present application do not specifically limit this.
当在某一帧图像中检测到目标后，则确定此时目标在该帧图像中的位置坐标和检测框(bounding box，bb)，此处，将这一位置坐标记为初始坐标，将这一检测框记为初始检测框。When a target is detected in a certain frame of image, the position coordinates and the bounding box (bb) of the target in the frame of image at this time are determined. Here, this position is marked as the initial coordinates, and this One detection frame is marked as the initial detection frame.
在目标检测完成后，则可以执行目标跟踪的初始化过程，如图1所示，所述初始化过程可以包括：After the target detection is completed, the initialization process of target tracking can be performed. As shown in FIG. 1, the initialization process may include:
步骤S101、对k的取值进行初始化。Step S101: Initialize the value of k.
在本申请实施例中，k为对目标进行跟踪的图像帧序号，k为正整数。此处将检测到目标的那一帧图像视作第0帧，在目标检测完成后，可以将k的取值初始化为1，即执行：k＝1。In the embodiment of the present application, k is the number of the image frame for tracking the target, and k is a positive integer. Here, the frame of image where the target is detected is regarded as the 0th frame. After the target detection is completed, the value of k can be initialized to 1, that is, execute: k=1.
步骤S102、对相关滤波器(CorrelationFilter，CF)进行初始化。Step S102: Initialize the correlation filter (CorrelationFilter, CF).
具体地，根据下式初始化相关滤波器的参数：Specifically, the parameters of the relevant filter are initialized according to the following formula:
Figure PCTCN2020140413-appb-000015
Figure PCTCN2020140413-appb-000015
其中，
Figure PCTCN2020140413-appb-000016
x为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的图像，m为预设的倍数参数，其具体取值可以 根据实际情况进行设置，优选地，此处将其设置为2.5，即搜索区域为检测框大小的2.5倍，y为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，σ 2为高斯函数的方差，
Figure PCTCN2020140413-appb-000017
为傅里叶变换，
Figure PCTCN2020140413-appb-000018
傅里叶反变换，λ为训练的罚项常数，□表示矩阵逐项相乘操作，α为相关滤波器的参数。 in,
Figure PCTCN2020140413-appb-000016
x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center, m is a preset multiple parameter, and its specific value can be set according to the actual situation. Preferably, it is set here Set to 2.5, that is, the search area is 2.5 times the size of the detection frame, and y is the Gaussian function of the area m times the size of the initial detection frame with the initial coordinates as the center, as the label for training the relevant filter, σ 2 Is the variance of the Gaussian function,
Figure PCTCN2020140413-appb-000017
Is the Fourier transform,
Figure PCTCN2020140413-appb-000018
Inverse Fourier transform, λ is the penalty constant for training, □ represents the matrix item-by-item multiplication operation, and α is the parameter of the correlation filter.
在对目标进行跟踪时，各帧图像的具体跟踪过程均类似，以下以其中任意一帧(第k帧)为例进行详细说明。如图2所示，在第k帧图像中对所述目标进行跟踪的过程可以包括：When tracking the target, the specific tracking process of each frame of image is similar, and any one of the frames (the k-th frame) is taken as an example for detailed description. As shown in FIG. 2, the process of tracking the target in the k-th frame of image may include:
步骤S201、根据目标在第k-1帧图像中的位置坐标和检测框，确定第k帧图像中的搜索区域。Step S201: Determine a search area in the k-th frame of image according to the position coordinates of the target in the k-1th frame of image and the detection frame.
如图3所示，虚线的矩形框表示在第k-1帧中跟踪得到的目标的检测框，则第k帧中的搜索区域为该检测框的m倍大小，如图中实线框所示，该搜索区域以所述目标在第k-1帧图像中的位置坐标为中心。As shown in Figure 3, the dashed rectangular frame represents the detection frame of the target tracked in the k-1th frame, and the search area in the kth frame is m times the size of the detection frame, as shown by the solid line frame in the figure. As shown, the search area is centered on the position coordinates of the target in the k-1th frame of image.
步骤S202、在所述搜索区域中对所述目标进行跟踪，得到所述目标在第k帧图像中的位置坐标和置信度。Step S202: Track the target in the search area to obtain the position coordinates and confidence of the target in the k-th frame of image.
在本实施例中，使用相关滤波器对所述目标进行跟踪。在其他实施例中，目标进行跟踪算法可以是，但不限于，如卡尔曼滤波、目标检测追踪、基于深度学习跟踪算法、多目标追踪算法等。In this embodiment, a correlation filter is used to track the target. In other embodiments, the target tracking algorithm may be, but is not limited to, such as Kalman filter, target detection and tracking, deep learning-based tracking algorithm, multi-target tracking algorithm, and so on.
具体地，可以使用相关滤波器在所述搜索区域中对所述目标进行跟踪，得到如下所示的响应矩阵：Specifically, a correlation filter can be used to track the target in the search area to obtain a response matrix as shown below:
Figure PCTCN2020140413-appb-000019
Figure PCTCN2020140413-appb-000019
其中，z为所述搜索区域的图像，CF为所述响应矩阵。Where, z is the image of the search area, and CF is the response matrix.
在本申请实施例中，可以将所述响应矩阵中最大的响应值确定为最大响应值，并将与所述最大响应值对应的位置坐标确定为所述目标在第k帧图像中的位置坐标。In the embodiment of the present application, the maximum response value in the response matrix may be determined as the maximum response value, and the position coordinates corresponding to the maximum response value may be determined as the position coordinates of the target in the k-th frame of image .
在本申请实施例的一种具体实现中，可以根据下式计算所述置信度：In a specific implementation of the embodiment of the present application, the confidence level can be calculated according to the following formula:
r＝p max r=p max
其中，p max为所述响应矩阵中最大的响应值，r为所述置信度。 Wherein, p max is the largest response value in the response matrix, and r is the confidence level.
在本申请实施例的另一种具体实现中，可以根据下式计算所述置信度：In another specific implementation of the embodiment of the present application, the confidence level can be calculated according to the following formula:
Figure PCTCN2020140413-appb-000020
Figure PCTCN2020140413-appb-000020
其中，CF w,h为所述响应矩阵中第w行h列的数值，1≤w≤W，W为所述响应矩阵的行数，1≤h≤H，H为所述响应矩阵的列数，p min为所述响应矩阵中最小的响应值。 Where CF w,h is the value in the wth row and h column of the response matrix, 1≤w≤W, W is the number of rows of the response matrix, 1≤h≤H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
步骤S203、执行与所述置信度对应的操作。Step S203: Perform an operation corresponding to the confidence level.
具体地，若所述置信度小于预设的第一阈值，则判定所述目标已跟丢，结束当前跟踪进程。在此过程中，还可以将跟丢的信息向预设的控制中心进行反馈。Specifically, if the confidence level is less than the preset first threshold, it is determined that the target has been lost, and the current tracking process is ended. In this process, you can also feedback the lost information to the preset control center.
若所述置信度大于预设的第二阈值，则对相关滤波器的参数进行更新。If the confidence level is greater than the preset second threshold, the parameters of the correlation filter are updated.
所述第二阈值大于所述第一阈值，所述第一阈值和所述第二阈值的具体取值可以根据实际情况进行设置，本申请实施例对此不作具体限定。The second threshold value is greater than the first threshold value, and specific values of the first threshold value and the second threshold value may be set according to actual conditions, which are not specifically limited in the embodiment of the present application.
在本申请实施例中，可以根据下式更新相关滤波器的参数：In the embodiment of the present application, the parameters of the relevant filter can be updated according to the following formula:
α＝(1-γ)α+γα′α=(1-γ)α+γα′
其中，
Figure PCTCN2020140413-appb-000021
x为以所述目标在第k帧图像中的位置坐标为中心，取第k帧图像中的检测框的m倍大小区域的图像，y为以所述目标在第k帧图像中的位置坐标为中心，取第k帧图像中的检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，γ为参数更新的权重常数。 in,
Figure PCTCN2020140413-appb-000021
x is the image of the area m times the size of the detection frame in the k-th frame with the position coordinates of the target in the k-th frame image as the center, and y is the position coordinates of the target in the k-th frame image As the center, take the Gaussian function of the area m times the size of the detection frame in the k-th frame image as the label for training the relevant filter, and γ is the weight constant for parameter update.
在更新完相关滤波器的参数之后，若当前跟踪进程尚未结束，则继续对下一帧图像中对所述目标进行跟踪的步骤，即执行：k＝k+1，并重新执行图2所示的跟踪步骤，直至当前跟踪进程结束为止。After updating the parameters of the relevant filter, if the current tracking process has not ended, continue to track the target in the next frame of image, that is, execute: k=k+1, and re-execute the steps shown in Figure 2 Until the end of the current tracking process.
若所述置信度小于等于所述第二阈值，且大于等于所述第一阈值，则不再对相关滤波器的参数进行更新，若当前跟踪进程尚未结束，则直接继续对下一帧图像中对所述目标进行跟踪的步骤，即执行：k＝k+1，并重新执行图2所示的跟踪步骤，直至当前跟踪进程结束为止。If the confidence level is less than or equal to the second threshold value and greater than or equal to the first threshold value, then the parameters of the relevant filter are no longer updated. If the current tracking process has not ended, directly continue to the next frame of image The step of tracking the target is to execute: k=k+1, and re-execute the tracking step shown in FIG. 2 until the current tracking process ends.
综上所述，本申请实施例根据目标的位置坐标和检测框，确定搜索区域；在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；若所述置信度小于预设的第一阈值，则判定所述目标已跟丢，结束当前跟踪进程。通过本申请实施例，在对目标的跟踪过程中，引入了置信度来对跟踪的效果进行评估，当置信度较低时，则说明目标已跟丢，此时则可以结束当前跟踪进程，避免出现在跟丢后仍然持续当前跟踪进程的情况。当置信度较高，即大于预设的第二阈值时，才会对相关滤波器的参数进行更新，从而避免学习到错误的背景信息，保持对于目标的跟踪。In summary, the embodiment of the present application determines the search area according to the position coordinates and detection frame of the target; tracks the target in the search area to obtain the position coordinates and confidence of the target; if the confidence is If the degree is less than the preset first threshold, it is determined that the target has been lost, and the current tracking process is ended. Through the embodiment of this application, in the process of tracking the target, the confidence is introduced to evaluate the tracking effect. When the confidence is low, it means that the target has been lost. At this time, the current tracking process can be ended to avoid There is a situation where the current tracking process continues after the track is lost. When the confidence level is high, that is, greater than the preset second threshold, the parameters of the relevant filter will be updated, so as to avoid learning wrong background information and keep track of the target.
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
对应于上文实施例所述的一种目标跟踪方法，图4示出了本申请实施例提供的一种目标跟踪装置的一个实施例结构图。Corresponding to the target tracking method described in the foregoing embodiment, FIG. 4 shows a structural diagram of an embodiment of a target tracking device provided in an embodiment of the present application.
本实施例中，一种目标跟踪装置可以包括：In this embodiment, a target tracking device may include:
搜索区域确定模块401，用于根据目标的位置坐标和检测框，确定搜索区域；The search area determination module 401 is configured to determine the search area according to the position coordinates and the detection frame of the target;
跟踪模块402，用于在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；The tracking module 402 is configured to track the target in the search area to obtain the position coordinates and confidence of the target;
跟丢判定模块403，用于所述置信度小于预设的第一阈值，结束当前跟踪进程；The tracking loss determination module 403 is configured to terminate the current tracking process when the confidence level is less than a preset first threshold;
模型更新模块404，用于所述置信度大于预设的第二阈值，对相关滤波器的参数进行更新，所述第二阈值大于所述第一阈值。The model update module 404 is configured to update the parameters of the correlation filter with the confidence level greater than a preset second threshold, where the second threshold is greater than the first threshold.
进一步地，所述目标跟踪装置还可以包括：Further, the target tracking device may further include:
初始检测模块，用于在指定区域中进行目标检测，并确定检测到的目标的初始坐标和初始检测框；The initial detection module is used to perform target detection in the designated area and determine the initial coordinates and initial detection frame of the detected target;
初始化模块，用于执行初始化过程，所述初始化过程包括：对相关滤波器进行初始化。The initialization module is used to perform the initialization process, and the initialization process includes: initializing the relevant filter.
进一步地，所述初始化模块可以包括：Further, the initialization module may include:
相关滤波器初始化单元，用于根据下式初始化相关滤波器的参数：The correlation filter initialization unit is used to initialize the parameters of the correlation filter according to the following formula:
Figure PCTCN2020140413-appb-000022
Figure PCTCN2020140413-appb-000022
其中，
Figure PCTCN2020140413-appb-000023
x为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的图像，m为预设的倍数参数，y为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，σ 2为高斯函数的方差，
Figure PCTCN2020140413-appb-000024
为傅里叶变换，
Figure PCTCN2020140413-appb-000025
傅里叶反变换，λ为训练的罚项常数，□表示矩阵逐项相乘操作，α为相关滤波器的参数。 in,
Figure PCTCN2020140413-appb-000023
x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center, m is a preset multiple parameter, and y is the m of the initial detection frame with the initial coordinates as the center The Gaussian function of the double size area is used as the label of the training correlation filter, σ 2 is the variance of the Gaussian function,
Figure PCTCN2020140413-appb-000024
Is the Fourier transform,
Figure PCTCN2020140413-appb-000025
Inverse Fourier transform, λ is the penalty constant for training, □ represents the matrix item-by-item multiplication operation, and α is the parameter of the correlation filter.
进一步地，所述跟踪模块可以包括：Further, the tracking module may include:
响应矩阵计算单元，用于使用相关滤波器在所述搜索区域中对所述目标进行跟踪，得到如下所示的响应矩阵：The response matrix calculation unit is configured to use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
Figure PCTCN2020140413-appb-000026
Figure PCTCN2020140413-appb-000026
其中，z为所述搜索区域的图像，CF为所述响应矩阵；Where z is the image of the search area, and CF is the response matrix;
位置坐标确定单元，用于将所述响应矩阵中最大的响应值确定为所述最大响应值，并将与所述最大响应值对应的位置坐标确定为所述位置坐标；A position coordinate determination unit, configured to determine the maximum response value in the response matrix as the maximum response value, and determine the position coordinate corresponding to the maximum response value as the position coordinate;
置信度计算单元，用于根据所述响应矩阵计算所述置信度。The confidence calculation unit is configured to calculate the confidence according to the response matrix.
进一步地，所述置信度计算单元具体用于：Further, the confidence calculation unit is specifically configured to:
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
r＝p max r=p max
其中，p max为所述响应矩阵中最大的响应值，r为所述置信度； Wherein, p max is the largest response value in the response matrix, and r is the confidence level;
或者or
根据下式计算所述置信度：Calculate the confidence level according to the following formula:
Figure PCTCN2020140413-appb-000027
Figure PCTCN2020140413-appb-000027
其中，CF w,h为所述响应矩阵中第w行h列的数值，1≤w≤W，W为所述响应矩阵的行数，1≤h≤H，H为所述响应矩阵的列数，p min为所述响应矩阵中最小的响应值。 Where CF w,h is the value in the wth row and h column of the response matrix, 1≤w≤W, W is the number of rows of the response matrix, 1≤h≤H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
进一步地，所述模型更新模块具体用于根据下式更新相关滤波器的参数：Further, the model update module is specifically configured to update the parameters of the relevant filter according to the following formula:
α＝(1-γ)α+γα′α=(1-γ)α+γα′
其中，
Figure PCTCN2020140413-appb-000028
x为以所述目标的位置坐标为中心，取检测框的m倍大小区域的图像，y为以所述目标的位置坐标为中心，取检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，γ为参数更新的权重常数。 in,
Figure PCTCN2020140413-appb-000028
x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center, y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation The label of the filter, γ is the weight constant for parameter update.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的装置，模块和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working processes of the above described devices, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的 部分，可以参见其它实施例的相关描述。In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.
图5示出了本申请实施例提供的一种机器人的示意框图，为了便于说明，仅示出了与本申请实施例相关的部分。Fig. 5 shows a schematic block diagram of a robot provided by an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.
如图5所示，该实施例的机器人5包括：处理器50、存储器51以及存储在所述存储器51中并可在所述处理器50上运行的计算机程序52。所述处理器50执行所述计算机程序52时实现上述各个目标跟踪方法实施例中的步骤。或者，所述处理器50执行所述计算机程序52时实现上述各装置实施例中各模块/单元的功能。As shown in FIG. 5, the robot 5 of this embodiment includes a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and running on the processor 50. When the processor 50 executes the computer program 52, the steps in the foregoing target tracking method embodiments are implemented. Alternatively, when the processor 50 executes the computer program 52, the functions of the modules/units in the foregoing device embodiments are realized.
示例性的，所述计算机程序52可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器51中，并由所述处理器50执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序52在所述机器人5中的执行过程。Exemplarily, the computer program 52 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 51 and executed by the processor 50 to complete This application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the robot 5.
本领域技术人员可以理解，图5仅仅是机器人5的示例，并不构成对机器人5的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述机器人5还可以包括输入输出设备、网络接入设备、总线等。Those skilled in the art can understand that FIG. 5 is only an example of the robot 5, and does not constitute a limitation on the robot 5. It may include more or less parts than shown, or a combination of some parts, or different parts, such as The robot 5 may also include input and output devices, network access devices, buses, and the like.
所述处理器50可以是中央处理单元(Central Processing Unit，CPU)，还可以是其它通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 50 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
所述存储器51可以是所述机器人5的内部存储单元，例如机器人5的硬盘或内存。所述存储器51也可以是所述机器人5的外部存储设备，例如所述机器人5上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器51还可以既包括所述机器人5的内部存储单元也包括外部存储设备。所述存储器51用于存储所述计算机程序以及所述机器人5所需的其它程序和数据。所述存储器51还可以用于暂时地存储已经输出或者将要输出的数据。The memory 51 may be an internal storage unit of the robot 5, such as a hard disk or memory of the robot 5. The memory 51 may also be an external storage device of the robot 5, such as a plug-in hard disk equipped on the robot 5, a smart memory card (Smart Media Card, SMC), or a Secure Digital (SD) card, Flash Card, etc. Further, the memory 51 may also include both an internal storage unit of the robot 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the robot 5. The memory 51 can also be used to temporarily store data that has been output or will be output.
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述***中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.
在本申请所提供的实施例中，应该理解到，所揭露的装置/机器人和方法，可以通过其它的方式实现。例如，以上所描述的装置/机器人实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一 点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。In the embodiments provided in this application, it should be understood that the disclosed device/robot and method can be implemented in other ways. For example, the device/robot embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or Components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims (10)

  1. 一种目标跟踪方法，其特征在于，包括：A target tracking method, characterized in that it comprises:
    根据目标的位置坐标和检测框，确定搜索区域；Determine the search area according to the location coordinates of the target and the detection frame;
    在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；Tracking the target in the search area to obtain the position coordinates and confidence of the target;
    所述置信度小于预设的第一阈值，结束当前跟踪进程；If the confidence level is less than the preset first threshold, the current tracking process is ended;
    所述置信度大于预设的第二阈值，对相关滤波器的参数进行更新，所述第二阈值大于所述第一阈值。If the confidence level is greater than a preset second threshold, the parameters of the correlation filter are updated, and the second threshold is greater than the first threshold.
  2. 根据权利要求1所述的目标跟踪方法，其特征在于，在所述搜索区域中对所述目标进行跟踪之前，还包括：The target tracking method according to claim 1, wherein before tracking the target in the search area, the method further comprises:
    在指定区域中进行目标检测，并确定检测到的目标的初始坐标和初始检测框；Perform target detection in the designated area, and determine the initial coordinates and initial detection frame of the detected target;
    执行初始化过程，所述初始化过程包括：对相关滤波器进行初始化。An initialization process is performed, and the initialization process includes: initializing the relevant filter.
  3. 根据权利要求2所述的目标跟踪方法，其特征在于，所述对相关滤波器进行初始化包括：The target tracking method according to claim 2, wherein the initializing the correlation filter comprises:
    根据下式初始化相关滤波器的参数：Initialize the parameters of the relevant filter according to the following formula:
    Figure PCTCN2020140413-appb-100001
    Figure PCTCN2020140413-appb-100001
    其中，
    Figure PCTCN2020140413-appb-100002
    x为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的图像，m为预设的倍数参数，y为以所述初始坐标为中心，取所述初始检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，σ 2为高斯函数的方差，
    Figure PCTCN2020140413-appb-100003
    为傅里叶变换，
    Figure PCTCN2020140413-appb-100004
    傅里叶反变换，λ为训练的罚项常数，□表示矩阵逐项相乘操作，α为相关滤波器的参数。     in,
    Figure PCTCN2020140413-appb-100002
    x is an image of an area m times the size of the initial detection frame with the initial coordinates as the center, m is a preset multiple parameter, and y is the m of the initial detection frame with the initial coordinates as the center The Gaussian function of the double size area is used as the label of the training correlation filter, σ 2 is the variance of the Gaussian function,
    Figure PCTCN2020140413-appb-100003
    Is the Fourier transform,
    Figure PCTCN2020140413-appb-100004
    Inverse Fourier transform, λ is the penalty constant for training, □ represents the matrix item-by-item multiplication operation, and α is the parameter of the correlation filter.
  4. 根据权利要求1所述的目标跟踪方法，其特征在于，所述在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度，包括：The target tracking method according to claim 1, wherein the tracking the target in the search area to obtain the position coordinates and the confidence of the target comprises:
    使用相关滤波器在所述搜索区域中对所述目标进行跟踪，得到如下所示的响应矩阵：Use a correlation filter to track the target in the search area to obtain a response matrix as shown below:
    Figure PCTCN2020140413-appb-100005
    Figure PCTCN2020140413-appb-100005
    其中，z为所述搜索区域的图像，CF为所述响应矩阵；Where z is the image of the search area, and CF is the response matrix;
    将所述响应矩阵中最大的响应值确定为所述最大响应值，并将与所述最大响应值对应的位置坐标确定为所述位置坐标；Determining the maximum response value in the response matrix as the maximum response value, and determining the position coordinate corresponding to the maximum response value as the position coordinate;
    根据所述响应矩阵计算所述置信度。The confidence level is calculated according to the response matrix.
  5. 根据权利要求4所述的目标跟踪方法，其特征在于，所述根据所述响应矩阵计算所述置信度，包括：The target tracking method according to claim 4, wherein the calculating the confidence degree according to the response matrix comprises:
    根据下式计算所述置信度：Calculate the confidence level according to the following formula:
    r＝p max r=p max
    其中，p max为所述响应矩阵中最大的响应值，r为所述置信度； Wherein, p max is the largest response value in the response matrix, and r is the confidence level;
    或者or
    根据下式计算所述置信度：Calculate the confidence level according to the following formula:
    Figure PCTCN2020140413-appb-100006
    Figure PCTCN2020140413-appb-100006
    其中，CF w,h为所述响应矩阵中第w行h列的数值，1≤w≤W，W为所述响应矩阵的行数，1≤h≤H，H为所述响应矩阵的列数，p min为所述响应矩阵中最小的响应值。 Where CF w,h is the value in the wth row and h column of the response matrix, 1≤w≤W, W is the number of rows of the response matrix, 1≤h≤H, and H is the column of the response matrix P min is the smallest response value in the response matrix.
  6. 根据权利要求1至5中任一项所述的目标跟踪方法，其特征在于，还包括：The target tracking method according to any one of claims 1 to 5, further comprising:
    所述置信度小于等于所述第二阈值，且大于等于所述第一阈值，不再对相关滤波器的参数进行更新。If the confidence is less than or equal to the second threshold, and greater than or equal to the first threshold, the parameters of the relevant filter are no longer updated.
  7. 根据权利要求6所述的目标跟踪方法，其特征在于，所述对相关滤波器的参数进行更新，包括：The target tracking method according to claim 6, wherein said updating the parameters of the correlation filter comprises:
    根据下式更新相关滤波器的参数：Update the parameters of the relevant filter according to the following formula:
    α＝(1-γ)α+γα′α=(1-γ)α+γα′
    其中，
    Figure PCTCN2020140413-appb-100007
    x为以所述目标的位置坐标为中心，取检测框的m倍大小区域的图像，y为以所述目标的位置坐标为中心，取检测框的m倍大小区域的高斯函数，作为训练相关滤波器的标签，γ为参数更新的权重常数。     in,
    Figure PCTCN2020140413-appb-100007
    x is the image of an area m times the size of the detection frame with the position coordinates of the target as the center, y is the Gaussian function of the area m times the size of the detection frame with the position coordinates of the target as the center, as the training correlation The label of the filter, γ is the weight constant for parameter update.
  8. 一种目标跟踪装置，其特征在于，包括：A target tracking device is characterized in that it comprises:
    搜索区域确定模块，用于根据目标的位置坐标和检测框，确定搜索区域；The search area determination module is used to determine the search area according to the position coordinates of the target and the detection frame;
    跟踪模块，用于在所述搜索区域中对所述目标进行跟踪，得到所述目标的位置坐标和置信度；A tracking module, configured to track the target in the search area to obtain the position coordinates and confidence of the target;
    跟丢判定模块，用于所述置信度小于预设的第一阈值，结束当前跟踪进程；The tracking loss determination module is used to end the current tracking process when the confidence level is less than a preset first threshold;
    模型更新模块，用于所述置信度大于预设的第二阈值，对相关滤波器的参数进行更新，所述第二阈值大于所述第一阈值。The model update module is configured to update the parameters of the correlation filter with the confidence level greater than a preset second threshold, where the second threshold is greater than the first threshold.
  9. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述的目标跟踪方法的步骤。A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the target tracking method according to any one of claims 1 to 7 A step of.
  10. 一种机器人，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7中任一项所述的目标跟踪方法的步骤。A robot, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claims 1 to 7 The steps of any one of the target tracking method.
PCT/CN2020/140413 2020-05-15 2020-12-28 Target tracking method and apparatus, and computer-readable storage medium and robot WO2021227519A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010413963.XA CN111696132B (en) 2020-05-15 2020-05-15 Target tracking method, device, computer readable storage medium and robot
CN202010413963.X 2020-05-15

Publications (1)

Publication Number Publication Date
WO2021227519A1 true WO2021227519A1 (en) 2021-11-18

Family

ID=72477853

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/140413 WO2021227519A1 (en) 2020-05-15 2020-12-28 Target tracking method and apparatus, and computer-readable storage medium and robot

Country Status (2)

Country Link
CN (1) CN111696132B (en)
WO (1) WO2021227519A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114332157A (en) * 2021-12-14 2022-04-12 北京理工大学 Long-term tracking method controlled by double thresholds
CN114549591A (en) * 2022-04-27 2022-05-27 南京甄视智能科技有限公司 Method and device for detecting and tracking time-space domain behaviors, storage medium and equipment
CN114708300A (en) * 2022-03-02 2022-07-05 北京理工大学 Anti-blocking self-adaptive target tracking method and system
CN115147458A (en) * 2022-07-21 2022-10-04 北京远度互联科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN115222771A (en) * 2022-07-05 2022-10-21 北京建筑大学 Target tracking method and device
CN115221981A (en) * 2022-09-20 2022-10-21 毫末智行科技有限公司 Target tracking method and device, terminal equipment and storage medium
CN115657654A (en) * 2022-07-26 2023-01-31 东莞康视达自动化科技有限公司 Visual identification method for food delivery robot
CN115984333A (en) * 2023-02-14 2023-04-18 北京拙河科技有限公司 Smooth tracking method and device for airplane target
CN116088580A (en) * 2023-02-15 2023-05-09 北京拙河科技有限公司 Flying object tracking method and device

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111696132B (en) * 2020-05-15 2023-12-29 深圳市优必选科技股份有限公司 Target tracking method, device, computer readable storage medium and robot
CN112561963A (en) * 2020-12-18 2021-03-26 北京百度网讯科技有限公司 Target tracking method and device, road side equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150055829A1 (en) * 2013-08-23 2015-02-26 Ricoh Company, Ltd. Method and apparatus for tracking object
CN105469430A (en) * 2015-12-10 2016-04-06 中国石油大学(华东) Anti-shielding tracking method of small target in large-scale scene
CN107154024A (en) * 2017-05-19 2017-09-12 南京理工大学 Dimension self-adaption method for tracking target based on depth characteristic core correlation filter
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111696132A (en) * 2020-05-15 2020-09-22 深圳市优必选科技股份有限公司 Target tracking method and device, computer readable storage medium and robot

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9454827B2 (en) * 2013-08-27 2016-09-27 Qualcomm Incorporated Systems, devices and methods for tracking objects on a display
US20160025850A1 (en) * 2014-06-03 2016-01-28 Watchstander, LLC Autonomous Robotic Mobile Threat Security System
CN110084829A (en) * 2019-03-12 2019-08-02 上海阅面网络科技有限公司 Method for tracking target, device, electronic equipment and computer readable storage medium
CN110033472B (en) * 2019-03-15 2021-05-11 电子科技大学 Stable target tracking method in complex infrared ground environment
CN110796676A (en) * 2019-10-10 2020-02-14 太原理工大学 Target tracking method combining high-confidence updating strategy with SVM (support vector machine) re-detection technology

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150055829A1 (en) * 2013-08-23 2015-02-26 Ricoh Company, Ltd. Method and apparatus for tracking object
CN105469430A (en) * 2015-12-10 2016-04-06 中国石油大学(华东) Anti-shielding tracking method of small target in large-scale scene
CN107154024A (en) * 2017-05-19 2017-09-12 南京理工大学 Dimension self-adaption method for tracking target based on depth characteristic core correlation filter
CN110956653A (en) * 2019-11-29 2020-04-03 中国科学院空间应用工程与技术中心 Satellite video dynamic target tracking method with fusion of correlation filter and motion estimation
CN111696132A (en) * 2020-05-15 2020-09-22 深圳市优必选科技股份有限公司 Target tracking method and device, computer readable storage medium and robot

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114332157A (en) * 2021-12-14 2022-04-12 北京理工大学 Long-term tracking method controlled by double thresholds
CN114332157B (en) * 2021-12-14 2024-05-24 北京理工大学 Long-time tracking method for double-threshold control
CN114708300A (en) * 2022-03-02 2022-07-05 北京理工大学 Anti-blocking self-adaptive target tracking method and system
CN114549591A (en) * 2022-04-27 2022-05-27 南京甄视智能科技有限公司 Method and device for detecting and tracking time-space domain behaviors, storage medium and equipment
CN114549591B (en) * 2022-04-27 2022-07-08 南京甄视智能科技有限公司 Method and device for detecting and tracking time-space domain behaviors, storage medium and equipment
CN115222771B (en) * 2022-07-05 2023-07-21 北京建筑大学 Target tracking method and device
CN115222771A (en) * 2022-07-05 2022-10-21 北京建筑大学 Target tracking method and device
CN115147458A (en) * 2022-07-21 2022-10-04 北京远度互联科技有限公司 Target tracking method and device, electronic equipment and storage medium
CN115657654A (en) * 2022-07-26 2023-01-31 东莞康视达自动化科技有限公司 Visual identification method for food delivery robot
CN115657654B (en) * 2022-07-26 2023-12-08 东莞康视达自动化科技有限公司 Visual recognition method for meal delivery robot
CN115221981A (en) * 2022-09-20 2022-10-21 毫末智行科技有限公司 Target tracking method and device, terminal equipment and storage medium
CN115984333B (en) * 2023-02-14 2024-01-19 北京拙河科技有限公司 Smooth tracking method and device for airplane target
CN115984333A (en) * 2023-02-14 2023-04-18 北京拙河科技有限公司 Smooth tracking method and device for airplane target
CN116088580B (en) * 2023-02-15 2023-11-07 北京拙河科技有限公司 Flying object tracking method and device
CN116088580A (en) * 2023-02-15 2023-05-09 北京拙河科技有限公司 Flying object tracking method and device

Also Published As

Publication number Publication date
CN111696132B (en) 2023-12-29
CN111696132A (en) 2020-09-22

Similar Documents

Publication Publication Date Title
WO2021227519A1 (en) Target tracking method and apparatus, and computer-readable storage medium and robot
US11961227B2 (en) Method and device for detecting and locating lesion in medical image, equipment and storage medium
US10950271B1 (en) Method for triggering events in a video
US10762644B1 (en) Multiple object tracking in video by combining neural networks within a bayesian framework
WO2020207190A1 (en) Three-dimensional information determination method, three-dimensional information determination device, and terminal apparatus
US10410362B2 (en) Method, device, and non-transitory computer readable storage medium for image processing
Lee et al. Robust stereo matching using adaptive random walk with restart algorithm
US20220375106A1 (en) Multi-target tracking method, device and computer-readable storage medium
CN109902705A (en) A kind of object detection model to disturbance rejection generation method and device
CN110686676A (en) Robot repositioning method and device and robot
CN111553234B (en) Pedestrian tracking method and device integrating facial features and Re-ID feature ordering
CN110263628B (en) Obstacle detection method, obstacle detection device, electronic apparatus, and storage medium
CN111612841A (en) Target positioning method and device, mobile robot and readable storage medium
CN110738650B (en) Infectious disease infection identification method, terminal device and storage medium
WO2021093011A1 (en) Unmanned vehicle driving decision-making method, unmanned vehicle driving decision-making device, and unmanned vehicle
CN113516750B (en) Three-dimensional point cloud map construction method and system, electronic equipment and storage medium
CN112634246B (en) Oral cavity image recognition method and related equipment
EP4378781A1 (en) Driving behavior determination method and related device thereof
CN113112542A (en) Visual positioning method and device, electronic equipment and storage medium
US11354923B2 (en) Human body recognition method and apparatus, and storage medium
CN111738085A (en) System construction method and device for realizing automatic driving and simultaneously positioning and mapping
CN112528925B (en) Pedestrian tracking and image matching method and related equipment
CN113869186B (en) Model training method and device, electronic equipment and computer readable storage medium
CN115527083A (en) Image annotation method and device and electronic equipment
CN111199179B (en) Target object tracking method, terminal equipment and medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20935211

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20935211

Country of ref document: EP

Kind code of ref document: A1