WO2020062546A1 - Target tracking processing method and electronic device - Google Patents

Target tracking processing method and electronic device Download PDF

Info

Publication number
WO2020062546A1
WO2020062546A1 PCT/CN2018/118621 CN2018118621W WO2020062546A1 WO 2020062546 A1 WO2020062546 A1 WO 2020062546A1 CN 2018118621 W CN2018118621 W CN 2018118621W WO 2020062546 A1 WO2020062546 A1 WO 2020062546A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
range
tracking
corner points
Prior art date
Application number
PCT/CN2018/118621
Other languages
French (fr)
Chinese (zh)
Inventor
何茂林
李方
梁迪
唐侨
Original Assignee
惠州市德赛西威汽车电子股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 惠州市德赛西威汽车电子股份有限公司 filed Critical 惠州市德赛西威汽车电子股份有限公司
Publication of WO2020062546A1 publication Critical patent/WO2020062546A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/269—Analysis of motion using gradient-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present invention relates to the field of data processing, and in particular, to a target tracking processing method and an electronic device.
  • target recognition is responsible for detecting the types and positions of various targets in each frame of the scene, while tracking is to associate the detected targets in the previous and subsequent frames to give an identity, and the detection and tracking cooperation can also be more accurate. Estimate the target position.
  • the invention provides a target tracking processing method, which can improve the data processing efficiency of the device.
  • An embodiment of the present invention provides a target tracking processing method, which is applied to an electronic device including at least three processing units.
  • the method includes:
  • a third processing unit determines a target range of the target object based on the tracking data of the corner points and an image range of the target object.
  • the acquiring multiple corner points of the target image, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points includes:
  • the corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points.
  • the acquiring multiple corner points of the target image and tracking the corresponding positions of the corner points in the target image in multiple frames includes:
  • Cross-validation is performed on the preliminary tracking data to obtain tracking data of corner points that satisfy a preset condition among the multiple corner points.
  • determining the target range of the target object based on the tracking data of the corner points and the image range of the target object includes:
  • the tracking data of the corner points includes a tracking range corresponding to each of the corner points in the corner point set;
  • the determining the target range of the target object based on the tracking data of the target corner point and the image range of the target object includes:
  • a target range of the target object is determined according to a preset weighting value, respectively.
  • determining the target corner point corresponding to the target object based on the tracking data of the corner point and the image range of the target object includes:
  • a corner point located within the pixel range is used as a target corner point.
  • the method before acquiring the preset mapping relationship between the target image and the image range, the method further includes: constructing, by the first processing unit, pixels in the target image and the image range. Mapping relationship.
  • the first processing unit is a vector operation processor.
  • the invention also provides an electronic device, which includes at least three processing units, wherein:
  • the first processing unit is configured to obtain a plurality of corner points of the target image, track corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;
  • the second processing unit is configured to detect a target object in the target image, and determine an image range corresponding to the target object;
  • the third processing unit is configured to determine a target range of the target object based on the tracking data of the corner points and an image range of the target object.
  • the first processing unit is a vector operation processor.
  • the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better. Deal with complex scenarios.
  • FIG. 1 is an implementation flowchart of a target tracking processing method according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of obtaining corner tracking data according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of determining a target range according to an embodiment of the present invention.
  • FIG. 4 is a flowchart of determining a target corner point according to an embodiment of the present invention.
  • FIG. 5 is a structural framework diagram of an electronic device according to an embodiment of the present invention.
  • FIG. 1 illustrates an implementation flow of a target tracking processing method provided by an embodiment of the present invention.
  • a target tracking processing method is applied to an electronic device.
  • the electronic device includes at least three processing units.
  • the method includes:
  • a first processing unit Through a first processing unit, multiple corner points of a target image are acquired, and corresponding positions of the corner points in the multi-frame target image are tracked to obtain tracking data of the corner points.
  • the corner point may be a feature point, that is, an image part having certain characteristics in the image, such as a face, a hand, etc. of a person in the image, which may be set as required.
  • the target image may be obtained through an image sensor of an electronic device, such as a camera.
  • multiple corner points of the target image are acquired, and feature portions in the image can be extracted by a preset algorithm, and these feature portions are matched through a database to determine the corner point positions.
  • the corner point is identified by a fast9 corner point detection algorithm.
  • the corresponding positions of the corner points in the multi-frame target image are tracked to obtain the tracking data of the corner points.
  • the position of a corner point in a target frame (such as the first frame) can be determined, and then The preset target tracking algorithm tracks the relative position change of the corner point in other frames to obtain the tracking data of the corner point.
  • the corner point tracking data may be parameters such as the moving trajectory of the corner point, the relative displacement distance, and the like.
  • the target tracking algorithm may adopt a pyramid LK optical flow tracking algorithm, by acquiring an image pyramid of the target image, and then tracking the optical flow of the corner points in the image pyramid by the pyramid LK optical flow tracking algorithm to track the movement of the corner points.
  • This step may be implemented in a first processing unit of the electronic device, and the first processing unit may be a processing unit for vector operations, such as a full-time vector operation processor, such as a GPU (Graphics Processing Unit, visual processor). ).
  • a full-time vector operation processor such as a GPU (Graphics Processing Unit, visual processor).
  • the operation efficiency can be greatly improved.
  • this step may also use a CPU (Central Processing Unit) or a DSP (Digital Signal Processing) to perform calculations.
  • CPU Central Processing Unit
  • DSP Digital Signal Processing
  • the target object may be one or more, such as a human body, a vehicle, or other obstacles. Recognizing the target object can be realized by a preset recognition algorithm, such as a neural network algorithm, etc. The specific recognition algorithm can be selected according to the actual situation.
  • the target object can be marked in advance by a recognition algorithm, and the approximate outline range of the marked object or the relative position in the target image can be selected using a detection frame. To determine the image range corresponding to the target object.
  • This step may be implemented in a second processing unit of the electronic device.
  • the second processing unit and the first processing unit are relatively independent from each other, so that step 102 and step 101 are in different process states.
  • the second processing unit may be General-purpose computing processors, such as CPU (Central Processing Unit) or DSP (Digital Signal Processing)
  • a third processing unit determines a target range of the target object according to the tracking data of the corner points and the image range of the target object.
  • the target range may be a range occupied by the target object in the image.
  • the target range defines a maximum width and a maximum height of the human body.
  • the tracking data of the corner points can be used to obtain the displacement of each corner point in the target image.
  • the corner points within the image range can be selected through the image range box of the target object, and the corner points of the target object within the image range can be obtained The displacement situation.
  • the image range of the target object and the displacement of the corner points within the target object range are weighted to obtain the target range of the target object.
  • This step may be implemented in a third processing unit of the electronic device.
  • the third processing unit is relatively independent of the second processing unit and the first processing unit, so that step 103 and step 102 and step 101 are in different process states. In order to allocate tasks to different cores to improve data processing efficiency.
  • the third processing unit and the second processing unit may also be the same processing unit.
  • the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better. Deal with complex scenarios.
  • FIG. 2 illustrates an implementation process of obtaining corner tracking data provided by an embodiment of the present application.
  • the acquiring multiple corner points of the target image, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points includes:
  • the preset area may be a certain area or the entire area of the target image manually preset to reduce redundant data of the target image and improve the efficiency of the algorithm.
  • the preset area may be a two-thirds height / width image area located in the middle of the target image. Or by identifying a possible moving direction of the host vehicle in the target image, and determining the position of the preset area in the target image according to the moving direction.
  • the size and position of the region can also be set according to actual needs.
  • an image pyramid of 4 or more layers can be constructed for the target image, and the corners of the images in these pyramids are algorithmically extracted to obtain the positions of the corners. Then, the corresponding positions of the corner points in the multi-frame target image are tracked to obtain the tracking data of the corner points.
  • the tracking algorithm may be a fast9 corner tracking algorithm to provide more accurate tracking data.
  • other tracking algorithms can also be used.
  • tracking the corresponding positions of the corner points in the multi-frame target image to obtain the tracking data of the corner points includes:
  • the forward pyramid LK tracking and the reverse pyramid LK tracking are respectively performed on the corner points to obtain preliminary tracking data; and the preliminary tracking data is cross-validated to obtain the tracking data of the corner points that meet the preset conditions from multiple corner points.
  • the above method can greatly improve the accuracy of tracking data of corner points. Through cross-validation of preliminary tracking data, corner points with poor tracking results can be removed, further reducing waste of resources, and improving data processing efficiency.
  • FIG. 3 illustrates an implementation process of determining a target range provided by an embodiment of the present application.
  • determining the target range of the target object based on the tracking data of the corner points and the image range of the target object includes:
  • the corner points in the image range of the target object can be determined, and the corner points in the image range are defined as the target corner points.
  • the corner points B and C may be used as the target corner points. If there are image ranges of N target objects, the image ranges of the N target objects can be associated with the corner points in the corresponding range.
  • Target range After obtaining the target corner point, you can determine the more accurate movement and morphological change of the target object (such as vehicle turning, human body moving sideways, etc.) according to the tracking situation of the corner point and the position change of the image range, and then determine the target object.
  • Target range After obtaining the target corner point, you can determine the more accurate movement and morphological change of the target object (such as vehicle turning, human body moving sideways, etc.) according to the tracking situation of the corner point and the position change of the image range, and then determine the target object.
  • the target object such as vehicle turning, human body moving sideways, etc.
  • the target object can be frame-selected through a tracking frame representing the target range.
  • the displacement value of the tracking frame can be obtained by the average displacement of the corner coordinates of the next frame and the corner coordinates of the previous frame, and the size change of the tracking frame can be obtained from the average distance between the target corners of the next frame.
  • the position change of the image range A can be combined with the position changes of the corner points B and C to use the height of the tracking frame
  • the width represents the body posture and movement of the human body at this time.
  • the length / width of the tracking frame can be changed according to the body posture and movement of the human body.
  • the required corner point is selected as the target corner point and the target corner point is tracked, so that the device does not need to perform image range recognition multiple times and diagonal points within the image range.
  • the recognition action greatly reduces the time complexity of this embodiment and improves the data processing efficiency.
  • step 302 may include:
  • the confidence level of the tracking range of the corner point and the image range of the target object is determined; according to the confidence level of the tracking range and the image range, the target range of the target object is determined according to preset weighting values.
  • the tracking range of the corner points is the approximate area occupied by the corner points in the target image.
  • the approximate area may be framed by a tracking frame.
  • the target range of the target object may be determined based on the tracking range of the corner point in combination with a preset weight value; If the confidence of the tracking range of the corner point is low and the confidence of the image range of the target object is high, the target range of the target object may be determined based on the image range of the target object in combination with a preset weight value.
  • the specific weighting method can be determined according to the actual situation.
  • the time complexity is linear time complexity, which improves the operation efficiency.
  • FIG. 4 illustrates an implementation process of determining a target corner point provided by an embodiment of the present application.
  • determining the target corner point corresponding to the target object based on the tracking data of the corner points and the image range of the target object includes:
  • the image range may be a parameter representing the relative position, including the approximate contour range of the target object.
  • the contour or relative position of the target object may be determined in the target image. The mapped pixel position.
  • the pixel range enclosed by the image range in the target image can be obtained from it, so as to better determine the position of the target object in the target image, and then use the target object in the target image
  • the corresponding corner point is selected as the target corner point in the approximate contour range box in the method to achieve the combination of the image range and the tracking data of the corner point to obtain the tracking data of the corner point within the image range of the target object.
  • the embodiment in FIG. 4 quickly determines the corner points in the image range through the mapping relationship, so that only a small amount of resources are required to obtain a better tracking effect on the target object, and the calculation efficiency is improved.
  • the method further includes:
  • a mapping relationship between pixels in the target image and the image range is constructed by the first processing unit.
  • the first processing unit is a vector operation processor.
  • the vector operation processor is used to construct a mapping relationship between pixels in the target image and the image range, and the processing efficiency can be greatly improved by utilizing the advantages of parallel operations.
  • FIG. 5 illustrates a structural framework of an electronic device according to an embodiment of the present application.
  • the electronic device 50 includes at least three processing units, where:
  • the first processing unit 51 is configured to acquire a plurality of corner points of a target image, and track corresponding positions of the corner points in the multi-frame target image to obtain tracking data of the corner points;
  • the second processing unit 52 is configured to detect a target object in a target image and determine an image range corresponding to the target object;
  • the third processing unit 53 is configured to determine a target range of the target object based on the tracking data of the corner points and the image range of the target object.
  • the electronic device 50 may be an in-vehicle electronic device 50, such as an ADAS (Advanced Driver Assistance System).
  • the electronic device 50 may further include a memory.
  • the processing unit is electrically connected to the memory.
  • the processing unit is the control center of the electronic device 50. It connects various parts of the entire electronic device 50 using various interfaces and lines, and executes the electronic device 50 by running or loading a computer program stored in the memory and calling data stored in the memory. Various functions and processing data, so as to monitor the electronic device 50 as a whole.
  • the first processing unit 51 is a vector operation processor.
  • the processor unit in the electronic device 50 loads the instructions corresponding to the process of one or more computer programs into the memory according to the following steps, and the processing unit runs the computer stored in the memory.
  • Programs to implement various functions such as:
  • first processing unit 51 multiple corner points of the target image are acquired, and corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points; through the second processing, A unit 52 detects and identifies a target object in the target image to determine an image range corresponding to the target object; and a third processing unit 53 according to the tracking data of the corner point and the image range of the target object, A target range of the target object is determined.
  • a storage medium stores multiple instructions, and the instructions are adapted to be loaded by a processing unit to execute any one of the foregoing target tracking processing methods, for example:
  • first processing unit 51 multiple corner points of the target image are acquired, and corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points; through the second processing, A unit 52 detects and identifies a target object in the target image to determine an image range corresponding to the target object; and a third processing unit 53 according to the tracking data of the corner point and the image range of the target object, A target range of the target object is determined.
  • the program may be stored in a computer-readable storage medium.
  • the storage medium may include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.
  • the electronic device and the target tracking processing method in the foregoing embodiment belong to the same concept, and any method step provided in the target tracking processing method embodiment can be run on the electronic device.
  • any method step provided in the target tracking processing method embodiment can be run on the electronic device.
  • please refer to the embodiment of the target tracking processing method, and any combination may be used to form an optional embodiment of the present application, which will not be repeated here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

A target tracking processing method, applicable in an electronic device (50) comprising at least three processing units, comprising: by means of a first processing unit (51), acquiring multiple corner points of a target image, tracking corresponding positions of the corner points in multiple frames of the target image to produce tracking data of the corner points (101); by means of a second processing unit (52), performing detection and identification with respect to a target object in the target image, determining an image range corresponding to the target object (102); and by means of a third processing unit (53), determining a target range of the target object on the basis of the tracking data of the corner points and of the image range of the target object (103). By assigning the tracking data and the identification of the image range to different processing units for processing, and by merging them to acquire the target range of the target object, the performance of a multicore processor can be effectively utilized to increase data processing efficiency and to better cope with a complicated scenario.

Description

目标跟踪处理方法、电子设备Target tracking processing method and electronic equipment 技术领域Technical field
本发明涉及数据处理领域，特别涉及一种目标跟踪处理方法、电子设备。The present invention relates to the field of data processing, and in particular, to a target tracking processing method and an electronic device.
背景技术Background technique
目标(车辆、行人等)的识别和跟踪是当前各种ADAS应用的关键技术。其中，目标识别负责检测出每一帧场景中各种目标的类型及位置，跟踪则是将前后两帧中检测到的目标关联起来，赋予一个身份标识，同时检测与跟踪配合也可以更精确的估算目标位置。The identification and tracking of targets (vehicles, pedestrians, etc.) is a key technology for various ADAS applications. Among them, target recognition is responsible for detecting the types and positions of various targets in each frame of the scene, while tracking is to associate the detected targets in the previous and subsequent frames to give an identity, and the detection and tracking cooperation can also be more accurate. Estimate the target position.
但是，现有的相关算法在同时实现对目标的识别和跟踪的过程中，计算量较大，使得设备的数据处理效率较低，不能很好地应对复杂场景。However, the existing related algorithms have a large amount of calculation in the process of simultaneously identifying and tracking the target, which makes the data processing efficiency of the device low, and it cannot deal with complex scenarios well.
发明内容Summary of the Invention
本发明提供一种目标跟踪处理方法，可以提高设备的数据处理效率。The invention provides a target tracking processing method, which can improve the data processing efficiency of the device.
本发明实施例提供一种目标跟踪处理方法，应用于包括至少三个处理单元的电子设备，所述方法包括：An embodiment of the present invention provides a target tracking processing method, which is applied to an electronic device including at least three processing units. The method includes:
通过第一处理单元，获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；Obtaining a plurality of corner points of the target image through a first processing unit, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points;
通过第二处理单元，对所述目标图像中的目标物体进行检测，确定所述目标物体对应的图像范围；Detecting a target object in the target image by a second processing unit to determine an image range corresponding to the target object;
通过第三处理单元，根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。A third processing unit determines a target range of the target object based on the tracking data of the corner points and an image range of the target object.
可选的，所述获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据，包括：Optionally, the acquiring multiple corner points of the target image, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points includes:
获取所述目标图像的预设区域；Acquiring a preset area of the target image;
构建所述预设区域的图像金字塔；Constructing an image pyramid of the preset area;
根据所述图像金字塔确定多个角点；Determining a plurality of corner points according to the image pyramid;
对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据。The corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points.
可选的，所述获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，包括：Optionally, the acquiring multiple corner points of the target image and tracking the corresponding positions of the corner points in the target image in multiple frames includes:
分别对所述角点进行前向金字塔LK跟踪及反向金字塔LK跟踪，获得初步跟踪数据；Perform forward pyramid LK tracking and reverse pyramid LK tracking on the corner points, respectively, to obtain preliminary tracking data;
对所述初步跟踪数据进行交叉验证，得到所述多个角点中满足预设条件的角点的跟踪数据。Cross-validation is performed on the preliminary tracking data to obtain tracking data of corner points that satisfy a preset condition among the multiple corner points.
可选的，所述根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标 物体的目标范围，包括：Optionally, determining the target range of the target object based on the tracking data of the corner points and the image range of the target object includes:
根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体对应的目标角点；根据所述目标角点的跟踪数据以及所述目标物体的图像范围，确定所述目标物体的目标范围。Determine the target corner corresponding to the target object according to the tracking data of the corner point and the image range of the target object; determine the target according to the tracking data of the target corner point and the image range of the target object The target range of the object.
可选的，所述角点的跟踪数据包括所述角点集合中每一所述角点对应的跟踪范围；Optionally, the tracking data of the corner points includes a tracking range corresponding to each of the corner points in the corner point set;
所述根据所述目标角点的跟踪数据以及所述目标物体的图像范围，确定所述目标物体的目标范围，包括：The determining the target range of the target object based on the tracking data of the target corner point and the image range of the target object includes:
确定所述角点的跟踪范围与所述目标物体的图像范围的置信度；Determining the confidence between the tracking range of the corner point and the image range of the target object;
根据所述跟踪范围与所述图像范围的置信度，分别按预设的加权值确定所述目标物体的目标范围。According to the confidence of the tracking range and the image range, a target range of the target object is determined according to a preset weighting value, respectively.
可选的，所述根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体对应的目标角点，包括：Optionally, determining the target corner point corresponding to the target object based on the tracking data of the corner point and the image range of the target object includes:
获取所述目标图像与所述图像范围之间预设的映射关系；Acquiring a preset mapping relationship between the target image and the image range;
根据所述映射关系，确定所述图像范围在所述目标图像中的多个映射像素；Determining a plurality of mapped pixels of the image range in the target image according to the mapping relationship;
根据所述映射像素确定所述图像范围在所述目标图像中所围成的像素范围；Determining a pixel range enclosed by the image range in the target image according to the mapped pixels;
将位于所述像素范围内的角点作为目标角点。A corner point located within the pixel range is used as a target corner point.
可选的，在所述获取所述目标图像与所述图像范围之间预设的映射关系之前，还包括：通过所述第一处理单元，构建所述目标图像中的像素与所述图像范围之间的映射关系。Optionally, before acquiring the preset mapping relationship between the target image and the image range, the method further includes: constructing, by the first processing unit, pixels in the target image and the image range. Mapping relationship.
可选的，所述第一处理单元为矢量运算处理器。Optionally, the first processing unit is a vector operation processor.
本发明还提供一种电子设备，所述电子设备包括至少三个处理单元，其中：The invention also provides an electronic device, which includes at least three processing units, wherein:
所述第一处理单元，用于获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；The first processing unit is configured to obtain a plurality of corner points of the target image, track corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;
所述第二处理单元，用于对所述目标图像中的目标物体进行检测，确定所述目标物体对应的图像范围；The second processing unit is configured to detect a target object in the target image, and determine an image range corresponding to the target object;
所述第三处理单元，用于根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。The third processing unit is configured to determine a target range of the target object based on the tracking data of the corner points and an image range of the target object.
可选的，所述第一处理单元为矢量运算处理器。Optionally, the first processing unit is a vector operation processor.
由上可知，通过将跟踪数据以及图像范围识别分配给不同的处理单元进行处理，并将其进行融合获得目标物体的目标范围，可以有效地利用多核处理器的性能提高数据处理效率，更好地应对复杂场景。As can be seen from the above, by assigning tracking data and image range recognition to different processing units for processing and fusing them to obtain the target range of the target object, the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better. Deal with complex scenarios.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本发明实施例提供的目标跟踪处理方法的实现流程图。FIG. 1 is an implementation flowchart of a target tracking processing method according to an embodiment of the present invention.
图2为本发明实施例提供的获得角点跟踪数据的实现流程图。FIG. 2 is a flowchart of obtaining corner tracking data according to an embodiment of the present invention.
图3为本发明实施例提供的确定目标范围的实现流程图。FIG. 3 is a flowchart of determining a target range according to an embodiment of the present invention.
图4为本发明实施例提供的确定目标角点的实现流程图。FIG. 4 is a flowchart of determining a target corner point according to an embodiment of the present invention.
图5为本发明实施例提供的电子设备的结构框架图。FIG. 5 is a structural framework diagram of an electronic device according to an embodiment of the present invention.
具体实施方式detailed description
下面结合附图对本发明的较佳实施例进行详细阐述，以使本发明的优点和特征更易被本领域技术人员理解，从而对本发明的保护范围作出更为清楚的界定。The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and the protection scope of the present invention is more clearly defined.
请参阅图1，图中示出了本发明实施例提供的目标跟踪处理方法的实现流程。Please refer to FIG. 1, which illustrates an implementation flow of a target tracking processing method provided by an embodiment of the present invention.
如图1所示，一种目标跟踪处理方法，应用于电子设备，该电子设备包括至少三个处理单元，该方法包括：As shown in FIG. 1, a target tracking processing method is applied to an electronic device. The electronic device includes at least three processing units. The method includes:
101、通过第一处理单元，获取目标图像的多个角点，对角点在多帧目标图像中的相应位置进行跟踪，得到角点的跟踪数据。101. Through a first processing unit, multiple corner points of a target image are acquired, and corresponding positions of the corner points in the multi-frame target image are tracked to obtain tracking data of the corner points.
其中，角点可以是特征点，也即图像中具有某种特征的图像部分，例如图像中人物的人脸、手部等，可以根据需要进行设定。The corner point may be a feature point, that is, an image part having certain characteristics in the image, such as a face, a hand, etc. of a person in the image, which may be set as required.
其中，目标图像可以是通过电子设备的图像传感器获取，例如摄像头。The target image may be obtained through an image sensor of an electronic device, such as a camera.
在一些实施例中，获取目标图像的多个角点，可以通过预设的算法提取图像中的特征部分，并经过数据库对这些特征部分进行匹配以确定角点位置。例如，通过fast9角点检测算法对该角点进行识别。In some embodiments, multiple corner points of the target image are acquired, and feature portions in the image can be extracted by a preset algorithm, and these feature portions are matched through a database to determine the corner point positions. For example, the corner point is identified by a fast9 corner point detection algorithm.
在一些实施例中，对角点在多帧目标图像中的相应位置进行跟踪，得到角点的跟踪数据，首先可以确定某帧目标图像中(如首帧)的某个角点位置，然后通过预设的目标跟踪算法跟踪该角点在其他帧中的相对位置变化来获得角点的跟踪数据。其中，角点的跟踪数据可以是该角点的移动轨迹、相对位移距离等参数。In some embodiments, the corresponding positions of the corner points in the multi-frame target image are tracked to obtain the tracking data of the corner points. First, the position of a corner point in a target frame (such as the first frame) can be determined, and then The preset target tracking algorithm tracks the relative position change of the corner point in other frames to obtain the tracking data of the corner point. The corner point tracking data may be parameters such as the moving trajectory of the corner point, the relative displacement distance, and the like.
可选的，目标跟踪算法可以采用金字塔LK光流跟踪算法，通过获取该目标图像的图像金字塔，然后通过金字塔LK光流跟踪算法跟踪图像金字塔中的角点的光流来跟踪角点的移动。Optionally, the target tracking algorithm may adopt a pyramid LK optical flow tracking algorithm, by acquiring an image pyramid of the target image, and then tracking the optical flow of the corner points in the image pyramid by the pyramid LK optical flow tracking algorithm to track the movement of the corner points.
该步骤可以是在电子设备的第一处理单元中实现，该第一处理单元可以是用于矢量运算的处理单元，如专职矢量运算的矢量运算处理器，例如GPU(Graphics Processing Unit，视觉处理器)。因为跟踪算法相对简单，但是有大量的像素级运算，因此通过矢量运算处理器运行101步骤时，可以大大提高运算效率。当然，除了矢量运算处理器，该步骤还可以采用CPU(Central Processing Unit，核心处理器)或者DSP(Digital Signal Processing，数字信号处理器)等器件运行计算。This step may be implemented in a first processing unit of the electronic device, and the first processing unit may be a processing unit for vector operations, such as a full-time vector operation processor, such as a GPU (Graphics Processing Unit, visual processor). ). Because the tracking algorithm is relatively simple, but has a large number of pixel-level operations, when the 101 steps are run by the vector operation processor, the operation efficiency can be greatly improved. Of course, in addition to the vector operation processor, this step may also use a CPU (Central Processing Unit) or a DSP (Digital Signal Processing) to perform calculations.
102、通过第二处理单元，对目标图像中的目标物体进行检测识别，确定目标物体对应的图像范围。102. Detect and identify a target object in the target image through a second processing unit, and determine an image range corresponding to the target object.
其中，该目标物体可以是一个或者多个，例如人体、车辆或者其他障碍物等。识别目标物体，可以通过预设的识别算法实现，例如神经网络算法等，具体识别算法可以根据实际情况进行选用。The target object may be one or more, such as a human body, a vehicle, or other obstacles. Recognizing the target object can be realized by a preset recognition algorithm, such as a neural network algorithm, etc. The specific recognition algorithm can be selected according to the actual situation.
在一些实施例中，确定目标物体对应的图像范围，可以预先通过识别算法将目标物体进行标示，并将标示出的物体的大致轮廓范围或者在目标图像中的相对位置用检测框来进行框选，以确定目标物体对应的图像范围。In some embodiments, to determine the image range corresponding to the target object, the target object can be marked in advance by a recognition algorithm, and the approximate outline range of the marked object or the relative position in the target image can be selected using a detection frame. To determine the image range corresponding to the target object.
该步骤可以是在电子设备的第二处理单元中实现，该第二处理单元与第一处理单元互为相对独立，使得该步骤102与步骤101处于不同的进程状态，该第二处理单元可以是通用运算处理器，例如采用CPU(Central Processing Unit，核心处理器)或者DSP(Digital Signal Processing，数字信号处理器)等器件运行计算This step may be implemented in a second processing unit of the electronic device. The second processing unit and the first processing unit are relatively independent from each other, so that step 102 and step 101 are in different process states. The second processing unit may be General-purpose computing processors, such as CPU (Central Processing Unit) or DSP (Digital Signal Processing)
103、通过第三处理单元，根据角点的跟踪数据及目标物体的图像范围，确定目标物体的目标范围。103. A third processing unit determines a target range of the target object according to the tracking data of the corner points and the image range of the target object.
其中，目标范围可以是该目标物体在图像中所占用的范围，例如，若目标范围是人体，则该目标范围框定了人体的最大宽度以及最大高度。The target range may be a range occupied by the target object in the image. For example, if the target range is a human body, the target range defines a maximum width and a maximum height of the human body.
在一些实施例中，通过角点的跟踪数据，可以获得该目标图像中各个角点的位移情况。将所检测到的目标物体的图像范围与各个角点的位移情况进行结合，可以通过目标物体的图像范围框选出在该图像范围内的角点，进而得到目标物体在图像范围内的角点的位移情况。In some embodiments, the tracking data of the corner points can be used to obtain the displacement of each corner point in the target image. Combining the image range of the detected target object with the displacement of each corner point, the corner points within the image range can be selected through the image range box of the target object, and the corner points of the target object within the image range can be obtained The displacement situation.
然后，将上述目标物体的图像范围以及目标物体范围内的角点的位移情况进行加权，从而获得目标物体的目标范围。Then, the image range of the target object and the displacement of the corner points within the target object range are weighted to obtain the target range of the target object.
该步骤可以在电子设备的第三处理单元中实现，该第三处理单元与该第二处理单元、第一处理单元互为相对独立，使得该步骤103与步骤102以及步骤101处于不同的进程状态，以将任务分配到不同核心实现数据处理效率的提高。当然，该第三处理单元和第二处理单元还可以为同一处理单元。This step may be implemented in a third processing unit of the electronic device. The third processing unit is relatively independent of the second processing unit and the first processing unit, so that step 103 and step 102 and step 101 are in different process states. In order to allocate tasks to different cores to improve data processing efficiency. Of course, the third processing unit and the second processing unit may also be the same processing unit.
在现有技术中，若需要获得目标物体所在范围内的角点，需要分别对不同的目标物体进行图像金字塔的构建，并结合目标物体的图像范围进行角点识别，使得算法需要多次构建目标图像的图像金字塔。若采用单一线程对目标物体进行识别，然后构建该目标图像的图像金字塔，并通过图像金字塔获得跟踪数据，其时间复杂度为O(N*Log(M)*(M*n^2+n^3))，其中N为检测到的目标物体个数，M为目标物体的平均尺寸，n一般为2。从中可以发现由 于是乘积的关系，因此当图像中目标较多时金字塔的构建耗时对整体时间复杂度的影响较大。In the prior art, if it is necessary to obtain the corner points within the range of the target object, it is necessary to separately construct the image pyramid of different target objects, and combine the corner points of the target object to identify the corner points, so that the algorithm needs to construct the target multiple times. Image of the image pyramid. If a single thread is used to identify the target object, then an image pyramid of the target image is constructed, and the tracking data is obtained through the image pyramid. The time complexity is O (N * Log (M) * (M * n ^ 2 + n ^ 3)), where N is the number of detected target objects, M is the average size of the target objects, and n is generally two. It can be found that the relationship between the products is thus, so when there are many targets in the image, the construction time of the pyramid has a great impact on the overall time complexity.
但是，当通过多核处理单元分别执行上述步骤101-103，只需对目标图像构建一次图像金字塔，并对目标图像的图像金字塔进行整体的角点识别，利用所识别到的目标物体的图像范围框选出所需的角点进行跟踪即可，可以使得其时间复杂度为O(Log(W)*(W*n^2+n^3))+O(W)，其中N为检测到的目标物体个数，M为目标物体的平均尺寸，n一般为2，W为ROI区域的大小，可知时间复杂度减小，数据处理效率大幅提高。However, when the above steps 101-103 are performed by a multi-core processing unit, only an image pyramid needs to be constructed once for the target image, and the overall corner recognition of the image pyramid of the target image is performed, and the image range frame of the identified target object is used. Select the required corner points for tracking, which can make its time complexity O (Log (W) * (W * n ^ 2 + n ^ 3)) + O (W), where N is the detected The number of target objects, M is the average size of the target object, n is generally 2, and W is the size of the ROI region. It can be seen that the time complexity is reduced and the data processing efficiency is greatly improved.
由上可知，通过将跟踪数据以及图像范围识别分配给不同的处理单元进行处理，并将其进行融合获得目标物体的目标范围，可以有效地利用多核处理器的性能提高数据处理效率，更好地应对复杂场景。As can be seen from the above, by assigning tracking data and image range recognition to different processing units for processing and fusing them to obtain the target range of the target object, the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better. Deal with complex scenarios.
请参阅图2，图中示出了本申请实施例提供的获得角点跟踪数据的实现流程。Please refer to FIG. 2, which illustrates an implementation process of obtaining corner tracking data provided by an embodiment of the present application.
如图2所示，所述获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据，包括：As shown in FIG. 2, the acquiring multiple corner points of the target image, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points includes:
201、获取目标图像的预设区域。201. Obtain a preset area of a target image.
202、构建预设区域的图像金字塔。202. Construct an image pyramid of a preset area.
203、根据图像金字塔确定多个角点。203. Determine a plurality of corner points according to the image pyramid.
204、对角点在多帧目标图像中的相应位置进行跟踪，得到角点的跟踪数据。204. Track the corresponding positions of the corner points in the multi-frame target image to obtain the tracking data of the corner points.
其中，预设区域可以是人工预设的目标图像中的某个区域或者整个区域，以减少目标图像的冗余数据，提高算法效率。例如，预设区域可以是位于目标图像中部的三分之二高度/宽度的图像区域。或者是通过识别目标图像中本车辆可能的移动方向，根据该移动方向确定预设区域在目标图像中位置。当然，除了上述预设方式，具体还可以根据实际需要对其区域的大小、位置等进行设定。The preset area may be a certain area or the entire area of the target image manually preset to reduce redundant data of the target image and improve the efficiency of the algorithm. For example, the preset area may be a two-thirds height / width image area located in the middle of the target image. Or by identifying a possible moving direction of the host vehicle in the target image, and determining the position of the preset area in the target image according to the moving direction. Of course, in addition to the preset methods described above, the size and position of the region can also be set according to actual needs.
在构建图像金字塔的过程中，可以对该目标图像构建4层或更多层的图像金字塔，并对这些金字塔中的图像通过算法提取角点，获得角点的位置。然后对上述角点在多帧目标图像中的相应位置进行跟踪，即可得到角点的跟踪数据。In the process of constructing an image pyramid, an image pyramid of 4 or more layers can be constructed for the target image, and the corners of the images in these pyramids are algorithmically extracted to obtain the positions of the corners. Then, the corresponding positions of the corner points in the multi-frame target image are tracked to obtain the tracking data of the corner points.
其中，跟踪算法可以是fast9角点跟踪算法，以提供较为准确地跟踪数据。当然，除此之外还可以采用其他方式的跟踪算法。Among them, the tracking algorithm may be a fast9 corner tracking algorithm to provide more accurate tracking data. Of course, other tracking algorithms can also be used.
在一些实施例中，对角点在多帧目标图像中的相应位置进行跟踪，得到角点的跟踪数据，包括：In some embodiments, tracking the corresponding positions of the corner points in the multi-frame target image to obtain the tracking data of the corner points includes:
分别对角点进行前向金字塔LK跟踪及反向金字塔LK跟踪，获得初步跟踪数据；对初步跟踪数据进行交叉验证，得到多个角点中满足预设条件的角点的跟踪数据。The forward pyramid LK tracking and the reverse pyramid LK tracking are respectively performed on the corner points to obtain preliminary tracking data; and the preliminary tracking data is cross-validated to obtain the tracking data of the corner points that meet the preset conditions from multiple corner points.
例如，如果某个角点持续几帧都出现检测丢失，则认为跟踪失效，将该角点的相关 参数删除。For example, if a corner is lost for several frames, the tracking is considered invalid, and the relevant parameters of the corner are deleted.
上述方式可以大大提高角点的跟踪数据的准确率，通过对初步跟踪数据进行交叉验证，可以将跟踪效果较差的角点进行去除，进一步降低资源浪费，提高数据处理效率。The above method can greatly improve the accuracy of tracking data of corner points. Through cross-validation of preliminary tracking data, corner points with poor tracking results can be removed, further reducing waste of resources, and improving data processing efficiency.
在一些实施例中，请参阅图3，图中示出了本申请实施例提供的确定目标范围的实现流程。In some embodiments, please refer to FIG. 3, which illustrates an implementation process of determining a target range provided by an embodiment of the present application.
如图3所示，所述根据角点的跟踪数据及目标物体的图像范围，确定目标物体的目标范围，包括：As shown in FIG. 3, determining the target range of the target object based on the tracking data of the corner points and the image range of the target object includes:
301、根据角点的跟踪数据及目标物体的图像范围，确定目标物体对应的目标角点。301. Determine a target corner point corresponding to a target object according to the tracking data of the corner points and the image range of the target object.
通过将目标图像中的所有角点的跟踪数据结合目标物体的图像范围，可以确定目标物体的图像范围内的角点，将该图像范围内的角点定义为目标角点。By combining the tracking data of all corner points in the target image with the image range of the target object, the corner points in the image range of the target object can be determined, and the corner points in the image range are defined as the target corner points.
例如，获取到目标物体的图像范围A，若该目标图像中有角点B以及角点C位于该图像范围A内，则可以将角点B、C作为目标角点。若存在N个目标物体的图像范围，则可以将该N个目标物体的图像范围与相应范围内的角点进行关联。For example, if an image range A of a target object is obtained, and if there are corner points B and C in the target image within the image range A, the corner points B and C may be used as the target corner points. If there are image ranges of N target objects, the image ranges of the N target objects can be associated with the corner points in the corresponding range.
302、根据目标角点的跟踪数据以及目标物体的图像范围，确定目标物体的目标范围。302. Determine a target range of the target object according to the tracking data of the target corner point and the image range of the target object.
当获取到目标角点后，可以根据该角点的跟踪情况以及图像范围的位置变化，确定目标物体较为准确的移动情况以及形态变化情况(例如车辆转弯、人体侧身走动等)，进而确定目标物体的目标范围。After obtaining the target corner point, you can determine the more accurate movement and morphological change of the target object (such as vehicle turning, human body moving sideways, etc.) according to the tracking situation of the corner point and the position change of the image range, and then determine the target object. Target range.
具体的，可以通过代表目标范围的跟踪框来框选出目标物体。该跟踪框的位移值可以由下一帧的角点坐标与上一帧的角点坐标的位移均值来得到，由下一帧该目标角点的间距离均值可以得到跟踪框的尺寸变化。Specifically, the target object can be frame-selected through a tracking frame representing the target range. The displacement value of the tracking frame can be obtained by the average displacement of the corner coordinates of the next frame and the corner coordinates of the previous frame, and the size change of the tracking frame can be obtained from the average distance between the target corners of the next frame.
例如，若图像范围A是人体的大致位置，角点B、C分别是人体的左右手的特征点，可以通过图像范围A的位置变化结合角点B、C的位置变化，利用跟踪框的高度/宽度来代表获知人体此时的体态以及移动情况，该跟踪框的长度/宽度可以根据该人体的体态以及移动情况的变化而变化。For example, if the image range A is the approximate position of the human body, and the corner points B and C are characteristic points of the left and right hands of the human body respectively, the position change of the image range A can be combined with the position changes of the corner points B and C to use the height of the tracking frame The width represents the body posture and movement of the human body at this time. The length / width of the tracking frame can be changed according to the body posture and movement of the human body.
通过所识别出的目标物体的图像范围，框选出所需要的角点作为目标角点，并对该目标角点进行跟踪，使得设备无需多次执行图像范围识别及在图像范围内对角点识别的动作，大大降低了该实施例的时间复杂度，提高了数据处理效率。Based on the image range of the identified target object, the required corner point is selected as the target corner point and the target corner point is tracked, so that the device does not need to perform image range recognition multiple times and diagonal points within the image range. The recognition action greatly reduces the time complexity of this embodiment and improves the data processing efficiency.
在一些实施例中，为了提高目标范围的准确度，步骤302中可以包括：In some embodiments, in order to improve the accuracy of the target range, step 302 may include:
确定角点的跟踪范围与目标物体的图像范围的置信度；根据跟踪范围与图像范围的置信度，分别按预设的加权值确定目标物体的目标范围。The confidence level of the tracking range of the corner point and the image range of the target object is determined; according to the confidence level of the tracking range and the image range, the target range of the target object is determined according to preset weighting values.
其中，置信度可以通过常规算法获得。角点的跟踪范围也即角点在目标图像中所占 的大致区域，该大致区域可以是通过跟踪框进行框定。Among them, the confidence can be obtained by conventional algorithms. The tracking range of the corner points is the approximate area occupied by the corner points in the target image. The approximate area may be framed by a tracking frame.
具体的，若角点的跟踪范围的置信度较高，而目标物体的图像范围的置信度较低，可以以角点的跟踪范围为准来结合预设的加权值确定目标物体的目标范围；若角点的跟踪范围的置信度较低，而目标物体的图像范围的置信度较高，可以目标物体的图像范围为准来结合预设的加权值确定目标物体的目标范围。Specifically, if the confidence level of the tracking range of the corner point is high and the confidence level of the image range of the target object is low, the target range of the target object may be determined based on the tracking range of the corner point in combination with a preset weight value; If the confidence of the tracking range of the corner point is low and the confidence of the image range of the target object is high, the target range of the target object may be determined based on the image range of the target object in combination with a preset weight value.
可以理解的，具体的加权方式可以根据实际情况而定。Understandably, the specific weighting method can be determined according to the actual situation.
将目标角点的跟踪数据与目标物体的图像范围进行结合，可以利用图像范围框选出所需的角点，无需对每一目标物体进行框选后再执行繁复的角点识别，使得实施例的时间复杂度为线型时间复杂度，提高运算效率。Combining the tracking data of the target corner points with the image range of the target object, you can use the image range frame to select the required corner points, and it is not necessary to perform frame corner selection for each target object before performing complicated corner point recognition. The time complexity is linear time complexity, which improves the operation efficiency.
请参阅图4，图中示出了本申请实施例提供的确定目标角点的实现流程。Please refer to FIG. 4, which illustrates an implementation process of determining a target corner point provided by an embodiment of the present application.
如图4所示，所述根据角点的跟踪数据及目标物体的图像范围，确定目标物体对应的目标角点，包括：As shown in FIG. 4, determining the target corner point corresponding to the target object based on the tracking data of the corner points and the image range of the target object includes:
401、获取目标图像与图像范围之间预设的映射关系。401. Obtain a preset mapping relationship between a target image and an image range.
402、根据映射关系，确定图像范围在目标图像中的多个映射像素。402. Determine a plurality of mapped pixels of the image range in the target image according to the mapping relationship.
403、根据映射像素确定图像范围在目标图像中所围成的像素范围。403. Determine a pixel range enclosed by the image range in the target image according to the mapped pixels.
404、将位于像素范围内的角点作为目标角点。404. Use a corner point located in the pixel range as a target corner point.
其中，图像范围可以为代表相对位置的参数，包括目标物体的大致轮廓范围，通过建立该图像范围与该目标图像的像素位置的映射关系，可以确定该目标物体的轮廓或者相对位置在目标图像中所映射的像素位置。The image range may be a parameter representing the relative position, including the approximate contour range of the target object. By establishing a mapping relationship between the image range and the pixel position of the target image, the contour or relative position of the target object may be determined in the target image. The mapped pixel position.
当获得该图像范围对应的映射像素后，可以从中获得该图像范围在目标图像中所围成的像素范围，从而更好地确定目标物体在目标图像中的位置，进而利用该目标物体在目标图像中的大致轮廓范围框选出相应的角点作为目标角点，以实现将图像范围与角点的跟踪数据进行结合，获得目标物体的图像范围内角点的跟踪数据。图4中的实施例通过映射关系快速确定图像范围内的角点，从而只需占用较少的资源即可获得对目标物体较好的跟踪效果，提升计算效率。After obtaining the mapped pixels corresponding to the image range, the pixel range enclosed by the image range in the target image can be obtained from it, so as to better determine the position of the target object in the target image, and then use the target object in the target image The corresponding corner point is selected as the target corner point in the approximate contour range box in the method to achieve the combination of the image range and the tracking data of the corner point to obtain the tracking data of the corner point within the image range of the target object. The embodiment in FIG. 4 quickly determines the corner points in the image range through the mapping relationship, so that only a small amount of resources are required to obtain a better tracking effect on the target object, and the calculation efficiency is improved.
在一些实施例中，为了可以进一步提高效率，在所述获取目标图像与图像范围之间预设的映射关系之前，还包括：In some embodiments, in order to further improve efficiency, before the obtaining a preset mapping relationship between a target image and an image range, the method further includes:
通过所述第一处理单元，构建目标图像中的像素与所述图像范围之间的映射关系。该第一处理单元为矢量运算处理器，通过矢量运算处理器构建目标图像中的像素与所述图像范围之间的映射关系，可以利用并行运算优势大大提高处理效率。A mapping relationship between pixels in the target image and the image range is constructed by the first processing unit. The first processing unit is a vector operation processor. The vector operation processor is used to construct a mapping relationship between pixels in the target image and the image range, and the processing efficiency can be greatly improved by utilizing the advantages of parallel operations.
请参阅图5，图中示出了本申请实施例提供的电子设备的结构框架。Please refer to FIG. 5, which illustrates a structural framework of an electronic device according to an embodiment of the present application.
如图5所示，该电子设备50包括至少三个处理单元，其中：As shown in FIG. 5, the electronic device 50 includes at least three processing units, where:
该第一处理单元51，用于获取目标图像的多个角点，对角点在多帧目标图像中的相应位置进行跟踪，得到角点的跟踪数据；The first processing unit 51 is configured to acquire a plurality of corner points of a target image, and track corresponding positions of the corner points in the multi-frame target image to obtain tracking data of the corner points;
该第二处理单元52，用于对目标图像中的目标物体进行检测，确定目标物体对应的图像范围；The second processing unit 52 is configured to detect a target object in a target image and determine an image range corresponding to the target object;
该第三处理单元53，用于根据角点的跟踪数据及目标物体的图像范围，确定目标物体的目标范围。The third processing unit 53 is configured to determine a target range of the target object based on the tracking data of the corner points and the image range of the target object.
在该电子设备50中，该电子设备50可以是车载的电子设备50，如ADAS(Advanced Driver Assistance System，高级驾驶辅助***)。该电子设备50还可以包括存储器。其中，处理单元与存储器电性连接。In the electronic device 50, the electronic device 50 may be an in-vehicle electronic device 50, such as an ADAS (Advanced Driver Assistance System). The electronic device 50 may further include a memory. The processing unit is electrically connected to the memory.
处理单元是电子设备50的控制中心，利用各种接口和线路连接整个电子设备50的各个部分，通过运行或加载存储在存储器内的计算机程序，以及调用存储在存储器内的数据，执行电子设备50的各种功能和处理数据，从而对电子设备50进行整体监控。The processing unit is the control center of the electronic device 50. It connects various parts of the entire electronic device 50 using various interfaces and lines, and executes the electronic device 50 by running or loading a computer program stored in the memory and calling data stored in the memory. Various functions and processing data, so as to monitor the electronic device 50 as a whole.
在一些实施例中，该第一处理单元51是矢量运算处理器。In some embodiments, the first processing unit 51 is a vector operation processor.
在本实施例中，电子设备50中的处理器单元会按照如下的步骤，将一个或一个以上的计算机程序的进程对应的指令加载到存储器中，并由处理单元来运行存储在存储器中的计算机程序，从而实现各种功能，如：In this embodiment, the processor unit in the electronic device 50 loads the instructions corresponding to the process of one or more computer programs into the memory according to the following steps, and the processing unit runs the computer stored in the memory. Programs to implement various functions, such as:
通过第一处理单元51，获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；通过第二处理单元52，对所述目标图像中的目标物体进行检测识别，确定所述目标物体对应的图像范围；通过第三处理单元53，根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。Through the first processing unit 51, multiple corner points of the target image are acquired, and corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points; through the second processing, A unit 52 detects and identifies a target object in the target image to determine an image range corresponding to the target object; and a third processing unit 53 according to the tracking data of the corner point and the image range of the target object, A target range of the target object is determined.
在一些实施例中，还提供了一种存储介质，该存储介质中存储有多条指令，该指令适于由处理单元加载以执行上述任一目标跟踪处理方法，例如：In some embodiments, a storage medium is also provided. The storage medium stores multiple instructions, and the instructions are adapted to be loaded by a processing unit to execute any one of the foregoing target tracking processing methods, for example:
通过第一处理单元51，获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；通过第二处理单元52，对所述目标图像中的目标物体进行检测识别，确定所述目标物体对应的图像范围；通过第三处理单元53，根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。Through the first processing unit 51, multiple corner points of the target image are acquired, and corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points; through the second processing, A unit 52 detects and identifies a target object in the target image to determine an image range corresponding to the target object; and a third processing unit 53 according to the tracking data of the corner point and the image range of the target object, A target range of the target object is determined.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、磁盘或光盘等。A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be implemented by a program instructing related hardware. The program may be stored in a computer-readable storage medium. The storage medium may include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks, etc.
本申请实施例中，所述电子设备与上文实施例中的一种目标跟踪处理方法属于同一构思，在所述电子设备上可以运行所述目标跟踪处理方法实施例中提供的任一方法步骤，其具体实现过程详见所述目标跟踪处理方法实施例，并可以采用任意结合形成本申请的可选实施例，此处不再赘述。In the embodiment of the present application, the electronic device and the target tracking processing method in the foregoing embodiment belong to the same concept, and any method step provided in the target tracking processing method embodiment can be run on the electronic device. For the specific implementation process, please refer to the embodiment of the target tracking processing method, and any combination may be used to form an optional embodiment of the present application, which will not be repeated here.
上面结合附图对本发明的实施方式作了详细说明，但是本发明并不限于上述实施方式，在本领域普通技术人员所具备的知识范围内，还可以在不脱离本发明宗旨的前提下作出各种变化。The embodiments of the present invention have been described in detail above with reference to the drawings, but the present invention is not limited to the above embodiments, and within the scope of knowledge possessed by a person of ordinary skill in the art, various embodiments can be made without departing from the spirit of the present invention Kind of change.

Claims (10)

  1. 一种目标跟踪处理方法，应用于包括至少三个处理单元的电子设备，其特征在于，所述方法包括：An object tracking processing method applied to an electronic device including at least three processing units, wherein the method includes:
    通过第一处理单元，获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；Obtaining a plurality of corner points of the target image through a first processing unit, and tracking corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points;
    通过第二处理单元，对所述目标图像中的目标物体进行检测识别，确定所述目标物体对应的图像范围；Detecting and identifying a target object in the target image by a second processing unit, and determining an image range corresponding to the target object;
    通过第三处理单元，根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。A third processing unit determines a target range of the target object based on the tracking data of the corner points and an image range of the target object.
  2. 如权利要求1所述的目标跟踪处理方法，其特征在于，所述获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据，包括：The target tracking processing method according to claim 1, wherein the acquiring multiple corner points of the target image, and tracking the corresponding positions of the corner points in the target image in multiple frames to obtain all Tracking data for corner points, including:
    获取所述目标图像的预设区域；Acquiring a preset area of the target image;
    构建所述预设区域的图像金字塔；Constructing an image pyramid of the preset area;
    根据所述图像金字塔确定多个角点；Determining a plurality of corner points according to the image pyramid;
    对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据。The corresponding positions of the corner points in the target image in multiple frames are tracked to obtain tracking data of the corner points.
  3. 如权利要求2所述的目标跟踪处理方法，其特征在于，所述对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据，包括：The target tracking processing method according to claim 2, wherein the tracking of the corresponding positions of the corner points in the target image in multiple frames to obtain the tracking data of the corner points comprises:
    分别对所述角点进行前向金字塔LK跟踪及反向金字塔LK跟踪，获得初步跟踪数据；Perform forward pyramid LK tracking and reverse pyramid LK tracking on the corner points, respectively, to obtain preliminary tracking data;
    对所述初步跟踪数据进行交叉验证，得到所述多个角点中满足预设条件的角点的跟踪数据。Cross-validation is performed on the preliminary tracking data to obtain tracking data of corner points that satisfy a preset condition among the multiple corner points.
  4. 如权利要求1所述的目标跟踪处理方法，其特征在于，所述根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围，包括：The target tracking processing method according to claim 1, wherein determining the target range of the target object based on the tracking data of the corner points and the image range of the target object comprises:
    根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体对应的目标角点；Determining a target corner point corresponding to the target object according to the tracking data of the corner point and the image range of the target object;
    根据所述目标角点的跟踪数据以及所述目标物体的图像范围，确定所述目标物体的目标范围。A target range of the target object is determined according to the tracking data of the target corner point and an image range of the target object.
  5. 如权利要求4所述的目标跟踪处理方法，其特征在于，所述角点的跟踪数据包括所述角点集合中每一所述角点对应的跟踪范围；The target tracking processing method according to claim 4, wherein the tracking data of the corner points includes a tracking range corresponding to each of the corner points in the set of corner points;
    所述根据所述目标角点的跟踪数据以及所述目标物体的图像范围，确定所述目标物体的目标范围，包括：The determining the target range of the target object based on the tracking data of the target corner point and the image range of the target object includes:
    确定所述角点的跟踪范围与所述目标物体的图像范围的置信度；Determining the confidence between the tracking range of the corner point and the image range of the target object;
    根据所述跟踪范围与所述图像范围的置信度，分别按预设的加权值确定所述目标物体的目标范围。According to the confidence of the tracking range and the image range, a target range of the target object is determined according to a preset weighting value, respectively.
  6. 如权利要求4所述的目标跟踪处理方法，其特征在于，所述根据所述角点的跟踪数据及 所述目标物体的图像范围，确定所述目标物体对应的目标角点，包括：The target tracking processing method according to claim 4, wherein determining the target corner point corresponding to the target object based on the tracking data of the corner point and the image range of the target object comprises:
    获取所述目标图像与所述图像范围之间预设的映射关系；Acquiring a preset mapping relationship between the target image and the image range;
    根据所述映射关系，确定所述图像范围在所述目标图像中的多个映射像素；Determining a plurality of mapped pixels of the image range in the target image according to the mapping relationship;
    根据所述映射像素确定所述图像范围在所述目标图像中所围成的像素范围；Determining a pixel range enclosed by the image range in the target image according to the mapped pixels;
    将位于所述像素范围内的角点作为目标角点。A corner point located within the pixel range is used as a target corner point.
  7. 如权利要求6所述的目标跟踪处理方法，其特征在于，在所述获取所述目标图像与所述图像范围之间预设的映射关系之前，还包括：The method for processing target tracking according to claim 6, before the acquiring a preset mapping relationship between the target image and the image range, further comprising:
    通过所述第一处理单元，构建所述目标图像中的像素与所述图像范围之间的映射关系。A mapping relationship between pixels in the target image and the image range is constructed by the first processing unit.
  8. 如权利要求1-7任意一项所述的目标跟踪处理方法，其特征在于，所述第一处理单元为矢量运算处理器。The target tracking processing method according to any one of claims 1 to 7, wherein the first processing unit is a vector operation processor.
  9. 一种电子设备，其特征在于，所述电子设备包括至少三个处理单元，其中：An electronic device, characterized in that the electronic device includes at least three processing units, wherein:
    所述第一处理单元，用于获取所述目标图像的多个角点，对所述角点在多帧所述目标图像中的相应位置进行跟踪，得到所述角点的跟踪数据；The first processing unit is configured to obtain a plurality of corner points of the target image, track corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;
    所述第二处理单元，用于对所述目标图像中的目标物体进行检测，确定所述目标物体对应的图像范围；The second processing unit is configured to detect a target object in the target image, and determine an image range corresponding to the target object;
    所述第三处理单元，用于根据所述角点的跟踪数据及所述目标物体的图像范围，确定所述目标物体的目标范围。The third processing unit is configured to determine a target range of the target object based on the tracking data of the corner points and an image range of the target object.
  10. 如权利要求9所述的电子设备，其特征在于，所述第一处理单元为矢量运算处理器。The electronic device according to claim 9, wherein the first processing unit is a vector operation processor.
PCT/CN2018/118621 2018-09-30 2018-11-30 Target tracking processing method and electronic device WO2020062546A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811155660.1 2018-09-30
CN201811155660.1A CN109509212B (en) 2018-09-30 2018-09-30 Target tracking processing method and electronic equipment

Publications (1)

Publication Number Publication Date
WO2020062546A1 true WO2020062546A1 (en) 2020-04-02

Family

ID=65746254

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/118621 WO2020062546A1 (en) 2018-09-30 2018-11-30 Target tracking processing method and electronic device

Country Status (2)

Country Link
CN (1) CN109509212B (en)
WO (1) WO2020062546A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109978801B (en) * 2019-03-25 2021-11-16 联想(北京)有限公司 Image processing method and image processing device
CN110120063A (en) * 2019-04-23 2019-08-13 深圳市道通智能航空技术有限公司 A kind of target following processing method based on multiprocessor
CN110288702B (en) * 2019-06-28 2021-05-18 联想(北京)有限公司 Data processing method and electronic equipment
CN113808159A (en) * 2021-01-04 2021-12-17 北京沃东天骏信息技术有限公司 Target tracking method and device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521842A (en) * 2011-11-28 2012-06-27 杭州海康威视数字技术股份有限公司 Method and device for detecting fast movement
CN105261042A (en) * 2015-10-19 2016-01-20 华为技术有限公司 Optical flow estimation method and apparatus
CN105578034A (en) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 Control method, control device and system for carrying out tracking shooting for object
CN107909600A (en) * 2017-11-04 2018-04-13 南京奇蛙智能科技有限公司 The unmanned plane real time kinematics target classification and detection method of a kind of view-based access control model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521842A (en) * 2011-11-28 2012-06-27 杭州海康威视数字技术股份有限公司 Method and device for detecting fast movement
CN105261042A (en) * 2015-10-19 2016-01-20 华为技术有限公司 Optical flow estimation method and apparatus
CN105578034A (en) * 2015-12-10 2016-05-11 深圳市道通智能航空技术有限公司 Control method, control device and system for carrying out tracking shooting for object
CN107909600A (en) * 2017-11-04 2018-04-13 南京奇蛙智能科技有限公司 The unmanned plane real time kinematics target classification and detection method of a kind of view-based access control model

Also Published As

Publication number Publication date
CN109509212A (en) 2019-03-22
CN109509212B (en) 2023-11-24

Similar Documents

Publication Publication Date Title
US11643076B2 (en) Forward collision control method and apparatus, electronic device, program, and medium
WO2020062546A1 (en) Target tracking processing method and electronic device
CN108446585B (en) Target tracking method and device, computer equipment and storage medium
CN108062536B (en) Detection method and device and computer storage medium
CN110443210B (en) Pedestrian tracking method and device and terminal
CN109543641B (en) Multi-target duplicate removal method for real-time video, terminal equipment and storage medium
CN110363817B (en) Target pose estimation method, electronic device, and medium
CN111062239A (en) Human body target detection method and device, computer equipment and storage medium
CN106203423B (en) Weak structure perception visual target tracking method fusing context detection
US20220343687A1 (en) Gesture recognition method and apparatus, and storage medium
WO2022252642A1 (en) Behavior posture detection method and apparatus based on video image, and device and medium
CN111047626A (en) Target tracking method and device, electronic equipment and storage medium
JP7499280B2 (en) Method and system for monocular depth estimation of a person - Patents.com
CN111354022B (en) Target Tracking Method and System Based on Kernel Correlation Filtering
US20220262093A1 (en) Object detection method and system, and non-transitory computer-readable medium
Tang et al. Multiple-kernel adaptive segmentation and tracking (MAST) for robust object tracking
WO2023083030A1 (en) Posture recognition method and related device
JP2020109644A (en) Fall detection method, fall detection apparatus, and electronic device
WO2020019353A1 (en) Tracking control method, apparatus, and computer-readable storage medium
Amamra et al. Real-time multiview data fusion for object tracking with RGBD sensors
US11314968B2 (en) Information processing apparatus, control method, and program
US20210216829A1 (en) Object likelihood estimation device, method, and program
CN110033474B (en) Target detection method, target detection device, computer equipment and storage medium
US20160379087A1 (en) Method for determining a similarity value between a first image and a second image
Zhou et al. Roadnet: An 80-mw hardware accelerator for road detection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18935125

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18935125

Country of ref document: EP

Kind code of ref document: A1