WO2022096027A1 - Garage space tracking method and apparatus - Google Patents

Garage space tracking method and apparatus Download PDF

Info

Publication number
WO2022096027A1
WO2022096027A1 PCT/CN2021/139527 CN2021139527W WO2022096027A1 WO 2022096027 A1 WO2022096027 A1 WO 2022096027A1 CN 2021139527 W CN2021139527 W CN 2021139527W WO 2022096027 A1 WO2022096027 A1 WO 2022096027A1
Authority
WO
WIPO (PCT)
Prior art keywords
tracking
storage location
eye view
area
corner point
Prior art date
Application number
PCT/CN2021/139527
Other languages
French (fr)
Chinese (zh)
Inventor
顾竟潇
霍璐
张立阳
王曦
宋健明
Original Assignee
天津天瞳威势电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 天津天瞳威势电子科技有限公司 filed Critical 天津天瞳威势电子科技有限公司
Publication of WO2022096027A1 publication Critical patent/WO2022096027A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/806—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • tracking the storage space and automatically parking is a dynamic process that is easily affected by various scene factors.
  • the driving speed is low, and it is accompanied by a large number of steering wheel rotations, starting and stopping, shifting and other steps.
  • the tracking work of the wheel speedometer will deviate and accumulate continuously, making it easy to park. A misalignment or even a crash occurs.
  • the present invention provides a storage location tracking method and device, and the technical solutions are as follows:
  • a storage location tracking method comprises:
  • the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time is composed of multiple layers of image areas centered on the corner of the storage location
  • the area of the image area of the upper layer is smaller than the area of the image area of the lower layer in the adjacent two-layer image areas;
  • the corner point of the warehouse location is tracked in the target image area of the first panoramic bird's-eye view to obtain the visual tracking result of the corner point of the warehouse location.
  • the process of determining the target image area includes:
  • the image regions of each layer are used as candidate image regions
  • the movement trend includes movement direction and movement distance
  • the movement trend of the main body is a movement trend in which the movement direction is the same and the proportion of the movement distance belonging to the same range is greater than a preset threshold;
  • the candidate image area is used as the target image area
  • the visual tracking result of the corner point of the warehouse location obtained by tracking the corner point of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method includes:
  • the movement trajectory of the storage location corner point is determined based on the moving direction and the moving distance of the storage location corner point.
  • the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
  • the tracking result of the wheel speed of the corner point of the storage location is used as the actual tracking result.
  • the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
  • the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
  • the gradient of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the warehouse and the center of the rear axle of the vehicle is less than or equal to If the distance threshold is preset, the visual tracking result of the corner point of the warehouse is used as the actual tracking result.
  • the method further includes:
  • the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used to correct the location angle Point wheel speed tracking results.
  • the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
  • the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to a preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than a preset distance threshold , and double-check the visual tracking result and the wheel speed tracking result of the corner point of the storage location to obtain the actual tracking result.
  • a storage location tracking device includes:
  • the image acquisition module is used to acquire panoramic bird's-eye views of the vehicle at different times;
  • the tracking area determination module is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers with the storage location.
  • the image area with the corner point as the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
  • a storage location tracking module is used to track the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is A first-level image area with the smallest area that can detect the corners of the storage location in the corresponding area of the first panoramic bird's-eye view; obtain the wheel speed tracking results of the corners of the storage location, and process the storage location by processing the storage location.
  • the visual tracking results and wheel speed tracking results of the corner points are used to obtain the actual tracking results.
  • the process of determining the target image area by the storage location tracking module includes:
  • the image regions of each layer are taken as candidate image regions in sequence from the upper layer to the lower layer; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are respectively divided into multiple each sub-region of the first panoramic bird's-eye view and the gray value probability histogram distribution of each sub-region of the second panoramic bird's-eye view are counted separately; for each sub-region of the second panoramic bird's-eye view, based on The gray value probability histogram distribution calculates the similarity between it and each sub-region of the first panoramic bird's-eye view, and selects a sub-region whose similarity meets the preset matching condition to form a region pair;
  • the movement trend includes the movement direction and the movement distance; determine whether there is a movement trend of the main body in the statistical results, and the movement trend of the main body is the movement trend in which the movement direction is the same and the movement distance belongs to the same range and the proportion is greater than the preset threshold;
  • the candidate image area is
  • the storage location tracking module tracks the storage location corner points in the target image area of the first panoramic bird's-eye view based on an optical flow tracking method to obtain visual tracking results of the storage location corner points, including:
  • the movement trend of each area pair belongs to the target area pair of the main body movement trend;
  • the movement distance of the corner point of the storage location is determined;
  • the movement trajectory of the corner point of the storage location is determined based on the movement direction and the movement distance of the corner point of the storage location.
  • the invention provides a storage location tracking method and device.
  • the visual detection method is used to locate the locked storage location in the automatic parking process in real time, so as to correct the cumulative error problem caused by the wheel speed tracking, and greatly improve one time
  • the probability of parking and parking in the center improves the user experience.
  • Fig. 1 is the method flow chart of the storage location tracking method that the embodiment of the present invention provides
  • FIG. 3 is a schematic diagram of optical flow tracking provided by an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of sub-region pair matching provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a parking scene provided by an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a storage location tracking device according to an embodiment of the present invention.
  • the automatic parking function is turned on.
  • the vehicle starts the automatic parking process.
  • the vehicle is equipped with an accurate wheel speedometer to calculate the speed and acceleration of the vehicle, and then calculate the unit time. The distance traveled down, and use this to locate the relative distance between the moving vehicle and the storage location.
  • the work of the wheel speedometer does not depend on other information. Deviations will occur in bad scenarios such as slopes on the ground or poor road conditions, causing deviations in the parking space and even the danger of a collision.
  • an embodiment of the present invention provides a storage location tracking method, and the method includes: Follow the steps below:
  • the vehicle is equipped with a vehicle-mounted 360-degree panoramic camera. Based on the vehicle-mounted 360-degree panoramic camera, an image of the environment where the vehicle is located can be obtained, and then scene recognition is performed on the image to obtain information about the environment where the vehicle is located, including road conditions and obstacles. objects, and ground markings.
  • the image coordinate system of the panoramic bird's-eye view can be obtained by converting the vehicle coordinate system to the image coordinate system of the panoramic bird's-eye view, and the image of the environment where the vehicle is located is mapped to the image coordinate system of the panoramic bird's-eye view to obtain the panoramic bird's eye view of the vehicle at different times.
  • S20 for the first panoramic bird's-eye view at the current time, determine the tracking area of the corner point of the warehouse location in the second panoramic bird's-eye view image that is closest to the current time, and the tracking area is composed of multiple layers of image areas centered on the corner of the warehouse location, And the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area.
  • a unit interval for storage location tracking can be preset, and the storage location detection is performed on the first frame of panoramic bird's-eye view in each unit interval to determine whether there is a storage location in the frame, and the second frame and For other subsequent frames of panoramic bird's-eye view, the optical flow tracking method is used to track the corners of the warehouse.
  • the unit interval can take the number of frames of the panoramic bird's-eye view as the dimension (for example, every 8 frames of the panoramic bird's-eye view is a unit interval), or the duration of the panoramic bird's-eye view can be the dimension (for example, the panoramic bird's eye view in every 30S is one unit interval) unit interval), which is not limited in this embodiment of the present invention.
  • a certain frame of panoramic bird's-eye view is based on the previous frame of panoramic bird's-eye view as the benchmark for storage location tracking.
  • each storage location has four corner points, and the essence of tracking the storage location is to track the four corner points of the storage location. Therefore, the embodiment of the present invention selects the previous frame of the panoramic bird's-eye view, that is, the second panoramic bird's-eye view has been detected. The positions of the corners of the four warehouse locations that come out are then used directly.
  • the four corners of the warehouse in the second panoramic bird's-eye view are used as the tracking start point set, and each corner of the warehouse is centered, and a multi-layer image area is formed in the form of a pyramid, which is used as the tracking of the corner of the warehouse. area.
  • a multi-layer image area is formed in the form of a pyramid, which is used as the tracking of the corner of the warehouse. area.
  • the embodiment of the present invention can also perform self-adaptive adjustment.
  • the vehicle speed can be used as the adjustment basis, for example:
  • the corner points of the warehouse location are tracked in the target image area of the first panoramic bird's-eye view to obtain a visual tracking result of the corner points of the warehouse location.
  • a new frame of panoramic bird's-eye view uses the tracking area of the previous frame of panoramic bird's-eye view, and uses the corner points of the storage location in the tracking area as feature points to perform optical flow tracking to form a new frame of panoramic bird's-eye view.
  • the tracking point set is obtained, and the movement trajectory of the corner point of the location is obtained, including the movement direction and movement distance.
  • the trajectories of all points in the tracking point set are fused to obtain the position and direction changes of the locked storage location relative to the vehicle during the parking process.
  • the new tracking point set means that the positions of the original four corner points of the warehouse location will change after completing one corner point tracking of the warehouse location, so the new location replaces the original location, forming a new tracking point. set. Therefore, what is updated is the position of the corner point of the storage location, and there are still 4 points in the point set.
  • optical flow tracking refers to tracking the small image changes of the target during the translation process of the two frames of images before and after, and using this to determine the position of the target on the next frame.
  • the specific implementation principle of tracking the corner points of the warehouse location based on the optical flow tracking method is to use the tracking area of the corner points of the warehouse location, and for each layer of image areas, determine that the two frames of panoramic bird's-eye views before and after are in the area.
  • the position with the smallest grayscale difference and a high degree of similarity is used as the candidate position, and the position with the smallest grayscale difference is further selected from all the candidate positions as the target position, which is the position of the corner point of the storage location in the next frame of panoramic bird's-eye view.
  • the determination process of the target image area includes the following steps:
  • the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately
  • the first panoramic bird's-eye view and the second panoramic bird's-eye view are subjected to image grayscale processing, and then the gray value histogram distribution of each sub-area of the A' area and each sub-area of the A area is counted respectively. It can be expressed in the form of a distribution histogram, that is, the gray value with the abscissa as 0, 1...255, and the ordinate as the probability of occurrence of the gray value. Furthermore, the histogram distribution of the grayscale values of each sub-region of the A' region and the histogram of the grayscale value of each sub-region of the A region are converted into 1*256-bit feature vectors.
  • a sub-region with the similarity of the A' region greater than the similarity threshold and the maximum similarity is selected to form a sub-region pair with the sub-region of the A region.
  • a pair of sub-areas indicated by arrows in FIG. 4 is a pair of sub-areas.
  • the moving trend of each sub-region pair that is, the moving direction and the moving distance
  • the moving trend of the main body can be found from it, and the moving trend of the main body can be at least three. Two-thirds of the sub-regions have the same moving trend, that is, the moving direction is the same and the moving distance is within a certain range.
  • step S30 the following steps can be adopted for "tracking the corner points of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the warehouse location":
  • the area pairs whose movement trend does not belong to the main body movement trend can be deleted.
  • the average value of the moving distance is calculated for all the reserved target area pairs, and the obtained result is the moving distance of the corner point of the storage location.
  • the moving direction of the target area pair is the moving direction of the corner point of the storage location.
  • the wheel speed tracking is always performed during the parking process, so the wheel speed tracking result and the visual tracking result of the corner point of the storage location can be simultaneously obtained when the storage location tracking is performed.
  • the embodiment of the present invention can switch wheel speed tracking and visual tracking in combination with the visual tracking result and the parking scene, so as to ensure the parking continuity.
  • a double check method can also be used to ensure the tracking accuracy. specific:
  • Visual tracking may be affected by exceeding the frame or being occluded.
  • the visual inspection cannot detect the storage position, or the storage position does not exceed the frame but is blocked, resulting in at least 2 corners of the storage position not being tracked. Or visual tracking has no output due to other environmental reasons. In this case, considering the parking continuity, the wheel speed tracking result is used as the actual tracking result.
  • the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used as the actual tracking result.
  • the visual tracking successfully outputs the moving trajectories of at least three corner points of the warehouse, and the road slope is greater than 3°, because the error of the wheel speed tracking is very large at this time, the visual tracking result is used as the actual tracking result.
  • the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to the preset distance threshold, then The visual tracking result of the corner point is used as the actual tracking result.
  • Visual tracking successfully outputs the moving trajectories of at least three corner points of the storage location, and the road slope is less than or equal to 3°, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to 3 meters. At this time, the visual effect is the best, so the visual tracking result is used as the Actual tracking results.
  • the distance from the storage location to the center of the rear axle of the vehicle is the straight-line distance from the midpoint of the line segment from the storage location to the near end of the vehicle to the center of the rear axle of the vehicle.
  • the visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold.
  • the actual tracking results are obtained by double-checking the visual tracking results and the wheel speed tracking results.
  • the distance between the storage location and the center of the rear axle of the vehicle is the straight-line distance from the midpoint A of the line segment between the storage location and the near end of the vehicle to the center O of the rear axle, and the length is D1.
  • the actual tracking result visual tracking result* ⁇ +wheel speed tracking result*(1- ⁇ ).
  • the visual tracking of the present invention is only processed within a certain range around the locked storage location, and the feature point tracking method is adopted, which can effectively perform tracking under various working conditions such as rotation, bumping, and deformation, and is stable and efficient. Therefore, the locked storage space in automatic parking can be tracked in real time, and the accumulated error of wheel speed tracking can be corrected, so as to realize automatic and precise parking.
  • the storage space may exceed the image frame or be occluded.
  • the tracking method will switch to wheel speed tracking.
  • the warehouse position returns to the screen or gets closer and closer to the car, the tracking method is switched from wheel speed to visual tracking, and double-checking is performed to achieve seamless connection.
  • the wheel speed tracking is corrected once within a period of time, so as to keep the error of the corrected wheel speed information to a minimum, and the track deviation will not occur due to the accumulation of errors.
  • the present invention is a method that can track and correct the parking route in real time during the automatic parking process. It can effectively improve the parking accuracy and success rate.
  • the invention has clear structure, concise method, good real-time performance and strong robustness.
  • an embodiment of the present invention provides a device for executing the above storage location tracking method.
  • the schematic structural diagram of the device is shown in FIG. 6 , including:
  • the image acquisition module 10 is used to acquire panoramic bird's-eye views of the vehicle at different times;
  • the tracking area determination module 20 is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time.
  • the image area in the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
  • the storage location tracking module 30 is used for tracking the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is the smallest in area and can be located in the first panoramic bird's eye view.
  • the process of determining the target image area by the location tracking module 30 includes:
  • the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately
  • the storage location tracking module 30 tracks the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the storage location, including:
  • the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
  • the wheel speed tracking result of the corner of the warehouse is used as the actual tracking result.
  • the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
  • the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
  • the storage location angle The visual tracking result of the point is used as the actual tracking result.
  • the location tracking module 30 is also used for:
  • the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, use the visual tracking result of the corner point of the warehouse location to correct the wheel speed tracking result of the corner point of the warehouse location.
  • the storage location tracking module 30 obtains the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the storage location, including:
  • the visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold.
  • the actual tracking results are obtained by double-checking the tracking results and the wheel speed tracking results.
  • the storage space tracking device uses the visual detection method to locate the locked storage space in the automatic parking process in real time, thereby correcting the cumulative error problem caused by wheel speed tracking, and greatly improving one parking time.
  • the probability of car entry and center parking improves user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

A garage space tracking method and an apparatus, the method comprising: obtaining a panoramic bird's eye view image of a vehicle at different times; and for a first panoramic bird's eye view image at a current time, determining a tracking region of a garage space corner point in a second panoramic bird's eye view image closest to the current time; on the basis of an optical flow tracking method, tracking the garage space corner point in a target image region of the first panoramic bird's eye view image to obtain a visual tracking result for the garage space corner point, the target image region being the one image region layer having the smallest area, and in which the garage space corner point can be detected in the corresponding region in the first panoramic bird's eye view image; and obtaining a wheel speed tracking result of the garage space corner point, and obtaining an actual tracking result by means of processing the visual tracking result and the wheel speed tracking result of the garage space corner point. In the present invention, a visual detection method is used to localize in real time a garage space that has been locked in a vehicle parking process, thus correcting the issue of accumulated calculation errors involved in wheel speed tracking, and greatly improving the probability of parking correctly the first time and stopping in the center of the space, thus improving user experience.

Description

一种库位跟踪方法及装置A storage location tracking method and device 技术领域technical field
本申请要求于2020年11月4日提交中国专利局、申请号为202011212958.9、发明名称为“一种库位跟踪方法及装置”的国内申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the domestic application with the application number of 202011212958.9 and the invention titled "A method and device for warehouse tracking", which was filed with the China Patent Office on November 4, 2020, the entire contents of which are incorporated into this application by reference .
背景技术Background technique
目前市面上很多智能车辆所带的泊车辅助***都具有自动泊车功能，其主要工作过程为探测库位、锁定库位、跟踪库位并自动泊车、完成泊车这四个步骤。At present, many parking assist systems on smart vehicles on the market have automatic parking functions.
其中，跟踪库位并自动泊车是一个容易受到各种场景因素影响的动态过程。这一过程行车速度低、并伴随着大量的方向盘转动、启动停止、换挡等步骤，一旦地面存在坡度或路况不好，轮速计的跟踪工作就会发生偏差并不断积累，很容易泊车入位发生偏移甚至是出现撞车的情况。Among them, tracking the storage space and automatically parking is a dynamic process that is easily affected by various scene factors. In this process, the driving speed is low, and it is accompanied by a large number of steering wheel rotations, starting and stopping, shifting and other steps. Once there is a slope on the ground or the road conditions are not good, the tracking work of the wheel speedometer will deviate and accumulate continuously, making it easy to park. A misalignment or even a crash occurs.
发明内容SUMMARY OF THE INVENTION
有鉴于此，为解决上述问题，本发明提供一种库位跟踪方法及装置，技术方案如下：In view of this, in order to solve the above problems, the present invention provides a storage location tracking method and device, and the technical solutions are as follows:
一种库位跟踪方法，所述方法包括：A storage location tracking method, the method comprises:
获取车辆在不同时间下的全景鸟瞰图；Obtain panoramic bird's-eye views of the vehicle at different times;
针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积；For the first panoramic bird's-eye view at the current time, determine the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers of image areas centered on the corner of the storage location The area of the image area of the upper layer is smaller than the area of the image area of the lower layer in the adjacent two-layer image areas;
基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层 图像区域；Based on the optical flow tracking method, the corner point of the warehouse location is tracked in the target image area of the first panoramic bird's-eye view to obtain the visual tracking result of the corner point of the warehouse location. A layer of image area where the corner point of the storage location is detected in the corresponding area of the first panoramic bird's-eye view;
获取所述库位角点的轮速跟踪结果，并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。Acquire the wheel speed tracking result of the corner point of the storage location, and obtain the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the storage location.
优选的，所述目标图像区域的确定过程，包括：Preferably, the process of determining the target image area includes:
按照上层到下层的次序依次将各层图像区域作为候选图像区域；According to the order from the upper layer to the lower layer, the image regions of each layer are used as candidate image regions;
分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域；respectively dividing the candidate image area of the first panoramic bird's-eye view and the candidate image area of the second panoramic bird's-eye view into a plurality of sub-regions;
分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布；Statistics of the gray value probability histogram distribution of each sub-region of the first panoramic bird's-eye view and each sub-region of the second panoramic bird's-eye view;
针对所述第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；For each sub-region of the second panoramic bird's-eye view, calculate the similarity between it and each sub-region of the first panoramic bird's-eye view based on the probability histogram distribution of gray values, and select a sub-region whose similarity meets the preset matching condition Regions form Region Pairs;
统计各区域对的移动趋势，所述移动趋势包括移动方向和移动距离；Statistics on the movement trend of each area pair, the movement trend includes movement direction and movement distance;
判断统计结果中是否存在主体移动趋势，所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；Judging whether there is a movement trend of the main body in the statistical result, the movement trend of the main body is a movement trend in which the movement direction is the same and the proportion of the movement distance belonging to the same range is greater than a preset threshold;
若是，则将所述候选图像区域作为目标图像区域；If so, the candidate image area is used as the target image area;
若否，则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。If not, execute the step of sequentially taking the image regions of each layer as candidate image regions in the order from the upper layer to the lower layer, and finish when all the image regions are traversed.
优选的，所述基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，包括：Preferably, the visual tracking result of the corner point of the warehouse location obtained by tracking the corner point of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method includes:
确定各区域对中移动趋势属于所述主体移动趋势的目标区域对；Determine that the movement trend of each area pair belongs to the target area pair of the movement trend of the subject;
将所述目标区域对的移动方向作为所述库位角点的移动方向，将所述目标区域对的移动距离的均值作为所述库位角点的移动距离；Taking the moving direction of the pair of target areas as the moving direction of the corners of the storage location, and taking the average value of the moving distances of the pair of target areas as the moving distance of the corners of the storage location;
基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨 迹。The movement trajectory of the storage location corner point is determined based on the moving direction and the moving distance of the storage location corner point.
优选的，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：Preferably, the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
如果所述库位角点的视觉跟踪结果表征库位跟踪失败，则将所述库位角点的轮速跟踪结果作为实际跟踪结果。If the visual tracking result of the corner point of the storage location indicates that the storage location tracking fails, the tracking result of the wheel speed of the corner point of the storage location is used as the actual tracking result.
优选的，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：Preferably, the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于预设坡度阈值，则将所述库位角点的视觉跟踪结果作为实际跟踪结果；If the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
或者or
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于所述预设坡度阈值、且所述库位与所述车辆的后轴中心的距离小于等于预设距离阈值，则将所述库位角点的视觉跟踪结果作为实际跟踪结果。If the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, the gradient of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the warehouse and the center of the rear axle of the vehicle is less than or equal to If the distance threshold is preset, the visual tracking result of the corner point of the warehouse is used as the actual tracking result.
优选的，所述方法还包括：Preferably, the method further includes:
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于所述预设坡度阈值，使用所述库位角点的视觉跟踪结果修正所述库位角点的轮速跟踪结果。If the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used to correct the location angle Point wheel speed tracking results.
优选的，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：Preferably, the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于预设坡度阈值、且所述库位与所述车辆的后轴中心的距离大于预设距离阈值，对所述库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。The visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to a preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than a preset distance threshold , and double-check the visual tracking result and the wheel speed tracking result of the corner point of the storage location to obtain the actual tracking result.
一种库位跟踪装置，所述装置包括：A storage location tracking device, the device includes:
图像获取模块，用于获取车辆在不同时间下的全景鸟瞰图；The image acquisition module is used to acquire panoramic bird's-eye views of the vehicle at different times;
跟踪区域确定模块，用于针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积；The tracking area determination module is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers with the storage location. The image area with the corner point as the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
库位跟踪模块，用于基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域；获取所述库位角点的轮速跟踪结果，并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。A storage location tracking module is used to track the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is A first-level image area with the smallest area that can detect the corners of the storage location in the corresponding area of the first panoramic bird's-eye view; obtain the wheel speed tracking results of the corners of the storage location, and process the storage location by processing the storage location. The visual tracking results and wheel speed tracking results of the corner points are used to obtain the actual tracking results.
优选的，所述库位跟踪模块确定目标图像区域的过程，包括：Preferably, the process of determining the target image area by the storage location tracking module includes:
按照上层到下层的次序依次将各层图像区域作为候选图像区域；分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域；分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布；针对所述第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；统计各区域对的移动趋势，所述移动趋势包括移动方向和移动距离；判断统计结果中是否存在主体移动趋势，所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；若是，则将所述候选图像区域作为目标图像区域；若否，则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。The image regions of each layer are taken as candidate image regions in sequence from the upper layer to the lower layer; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are respectively divided into multiple each sub-region of the first panoramic bird's-eye view and the gray value probability histogram distribution of each sub-region of the second panoramic bird's-eye view are counted separately; for each sub-region of the second panoramic bird's-eye view, based on The gray value probability histogram distribution calculates the similarity between it and each sub-region of the first panoramic bird's-eye view, and selects a sub-region whose similarity meets the preset matching condition to form a region pair; The movement trend includes the movement direction and the movement distance; determine whether there is a movement trend of the main body in the statistical results, and the movement trend of the main body is the movement trend in which the movement direction is the same and the movement distance belongs to the same range and the proportion is greater than the preset threshold; The candidate image area is used as the target image area; if not, the process of sequentially taking the image areas of each layer as the candidate image area in the order from the upper layer to the lower layer is performed until the traversal of all the image areas ends.
优选的，所述库位跟踪模块基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，包括：Preferably, the storage location tracking module tracks the storage location corner points in the target image area of the first panoramic bird's-eye view based on an optical flow tracking method to obtain visual tracking results of the storage location corner points, including:
确定各区域对中移动趋势属于所述主体移动趋势的目标区域对；将所述目标区域对的移动方向作为所述库位角点的移动方向，将所述目标区域对的移动距离的均值作为所述库位角点的移动距离；基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。It is determined that the movement trend of each area pair belongs to the target area pair of the main body movement trend; The movement distance of the corner point of the storage location is determined; the movement trajectory of the corner point of the storage location is determined based on the movement direction and the movement distance of the corner point of the storage location.
本发明提供一种库位跟踪方法及装置，采用视觉检测的方法对自动泊车过程中已经锁定好的库位进行实时定位，从而修正轮速跟踪所带来的累计误差问题，大幅度提高一次泊车入位、以及居中停靠的概率，提高用户体验。The invention provides a storage location tracking method and device. The visual detection method is used to locate the locked storage location in the automatic parking process in real time, so as to correct the cumulative error problem caused by the wheel speed tracking, and greatly improve one time The probability of parking and parking in the center improves the user experience.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work.
图1为本发明实施例提供的库位跟踪方法的方法流程图；Fig. 1 is the method flow chart of the storage location tracking method that the embodiment of the present invention provides;
图2为本发明实施例提供的全景鸟瞰图示例；2 is an example of a panoramic bird's-eye view provided by an embodiment of the present invention;
图3为本发明实施例提供的光流跟踪示意图；FIG. 3 is a schematic diagram of optical flow tracking provided by an embodiment of the present invention;
图4为本发明实施例提供的子区域对匹配示意图；FIG. 4 is a schematic diagram of sub-region pair matching provided by an embodiment of the present invention;
图5为本发明实施例提供的泊车场景示意图；5 is a schematic diagram of a parking scene provided by an embodiment of the present invention;
图6为本发明实施例提供的库位跟踪装置的结构示意图。FIG. 6 is a schematic structural diagram of a storage location tracking device according to an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
为使本发明的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本发明作进一步详细的说明。In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.
现阶段，自动泊车功能开启，一旦库位被锁定，路径规划完成，车辆就开始了自动泊车过程，车辆中配置有精确的轮速计来计算车辆行驶的速度、加速度，然后计算单位时间下行驶的距离，并以此来定位运动中的车辆与库位的相对距离。但是轮速计的工作是不依赖其他信息的，在地面存在坡度或路况不好等恶劣场景下就会发生偏差，使得泊车入位发生偏移甚至是撞车的危险。At this stage, the automatic parking function is turned on. Once the storage space is locked and the path planning is completed, the vehicle starts the automatic parking process. The vehicle is equipped with an accurate wheel speedometer to calculate the speed and acceleration of the vehicle, and then calculate the unit time. The distance traveled down, and use this to locate the relative distance between the moving vehicle and the storage location. However, the work of the wheel speedometer does not depend on other information. Deviations will occur in bad scenarios such as slopes on the ground or poor road conditions, causing deviations in the parking space and even the danger of a collision.
最常见的解决方法是泊车进入到车位一定位置后，再次重新探测停车位并锁定新车位，再进行重新规划，虽然最终成功几率很大，但是通常需要重新规划并调整车辆姿态3次以上才能够泊正车辆，这不仅浪费大量时间，而且当路况复杂时，会带来了极大的不方便，甚至路况都没有重新规划调整车辆姿态的空间。The most common solution is to park and enter a certain position of the parking space, re-detect the parking space and lock the new space, and then re-plan. Although the final success rate is very high, it usually needs to be re-planned and adjusted for more than 3 times. Being able to park the vehicle rightly not only wastes a lot of time, but also brings great inconvenience when the road conditions are complex.
为解决以上问题，本发明采用视觉定位跟踪库位的方案修正轮速计的轮速跟踪结果，参见图1所示的方法流程图，本发明实施例提供一种库位跟踪方法，该方法包括如下步骤：In order to solve the above problems, the present invention adopts the solution of visual positioning and tracking the storage location to correct the wheel speed tracking result of the wheel speedometer. Referring to the method flowchart shown in FIG. 1 , an embodiment of the present invention provides a storage location tracking method, and the method includes: Follow the steps below:
S10，获取车辆在不同时间下的全景鸟瞰图。S10, obtain panoramic bird's-eye views of the vehicle at different times.
本发明实施例中，车辆中配置有车载360度全景摄像头，基于该车载360度全景摄像头能够获得车辆所在环境的影像，进而对该影像进行场景识别，获得车辆所在环境的信息，包括路况、障碍物、以及地面标识。In the embodiment of the present invention, the vehicle is equipped with a vehicle-mounted 360-degree panoramic camera. Based on the vehicle-mounted 360-degree panoramic camera, an image of the environment where the vehicle is located can be obtained, and then scene recognition is performed on the image to obtain information about the environment where the vehicle is located, including road conditions and obstacles. objects, and ground markings.
进一步，本发明在获得可以通过车辆坐标系到全景鸟瞰图的图像坐标系的转换，将车辆所在环境的影像映射到全景鸟瞰图的图像坐标系中，获得车辆在不同时间下的全景鸟瞰图。Further, in the present invention, the image coordinate system of the panoramic bird's-eye view can be obtained by converting the vehicle coordinate system to the image coordinate system of the panoramic bird's-eye view, and the image of the environment where the vehicle is located is mapped to the image coordinate system of the panoramic bird's-eye view to obtain the panoramic bird's eye view of the vehicle at different times.
参见图2所示的全景鸟瞰图示例。图像坐标系原点(0,0)在左上，车辆中心坐标为(500,325)，其他距离信息如图，单位为像素。假设，这里每个像素代表实际的距离为2厘米，所以400像素就代表车辆左侧有800cm 的距离范围。同理，当我们在图像上检测到库位后，就可以通过1像素＝2厘米的比例进行转换，以实现将库位位置从车辆坐标系转换到全景鸟瞰图的图像坐标系中。See Figure 2 for an example of a panoramic bird's-eye view. The origin of the image coordinate system (0,0) is on the upper left, the vehicle center coordinate is (500,325), and other distance information is shown in the figure, and the unit is pixel. Assuming that each pixel here represents an actual distance of 2 cm, so 400 pixels represents a distance range of 800 cm on the left side of the vehicle. In the same way, when we detect the warehouse location on the image, we can convert it with a ratio of 1 pixel = 2 cm to convert the location of the warehouse location from the vehicle coordinate system to the image coordinate system of the panoramic bird's-eye view.
S20，针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，跟踪区域由多层以库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积。S20, for the first panoramic bird's-eye view at the current time, determine the tracking area of the corner point of the warehouse location in the second panoramic bird's-eye view image that is closest to the current time, and the tracking area is composed of multiple layers of image areas centered on the corner of the warehouse location, And the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area.
本发明实施例中，可以预先设置一个库位跟踪的单位区间，对每个单位区间内的第一帧全景鸟瞰图进行库位检测，以判定其画幅中是否存在库位，对第二帧以及后续的其他帧全景鸟瞰图，则以光流跟踪法进行库位角点的跟踪。当然，该单位区间可以以全景鸟瞰图的帧数为维度(比如每8帧全景鸟瞰图为一个单位区间)、还可以以全景鸟瞰图的时长为维度(比如每30S内的全景鸟瞰图为一个单位区间)进行设置，本发明实施例对此不做限定。In the embodiment of the present invention, a unit interval for storage location tracking can be preset, and the storage location detection is performed on the first frame of panoramic bird's-eye view in each unit interval to determine whether there is a storage location in the frame, and the second frame and For other subsequent frames of panoramic bird's-eye view, the optical flow tracking method is used to track the corners of the warehouse. Of course, the unit interval can take the number of frames of the panoramic bird's-eye view as the dimension (for example, every 8 frames of the panoramic bird's-eye view is a unit interval), or the duration of the panoramic bird's-eye view can be the dimension (for example, the panoramic bird's eye view in every 30S is one unit interval) unit interval), which is not limited in this embodiment of the present invention.
而在一个单位区间，某一帧全景鸟瞰图是以其前一帧全景鸟瞰图作为库位跟踪的基准的。In a unit interval, a certain frame of panoramic bird's-eye view is based on the previous frame of panoramic bird's-eye view as the benchmark for storage location tracking.
参见图3所示的光流跟踪示意图。每个库位都拥有四个角点，跟踪库位的本质就是对其四个库位角点的跟踪，因此本发明实施例选取前一帧全景鸟瞰图，即第二全景鸟瞰图中已经检测出来的四个库位角点的位置，然后直接使用。See the schematic diagram of optical flow tracking shown in Figure 3. Each storage location has four corner points, and the essence of tracking the storage location is to track the four corner points of the storage location. Therefore, the embodiment of the present invention selects the previous frame of the panoramic bird's-eye view, that is, the second panoramic bird's-eye view has been detected. The positions of the corners of the four warehouse locations that come out are then used directly.
将第二全景鸟瞰图中库位的四个角点作为跟踪起始点集，以每个库位角点为中心，采用金字塔的方式形成多层图像区域，将其作为该库位角点的跟踪区域。以三层图像区域为例，如图3中三层虚线框所示，以其中的库位角点为中心，按照不同像素距离取正方形范围，这样就可以得到多个不同面积的图像区域，从大到小就形成了图像金字塔，多个图像区域就形成了多层区域。The four corners of the warehouse in the second panoramic bird's-eye view are used as the tracking start point set, and each corner of the warehouse is centered, and a multi-layer image area is formed in the form of a pyramid, which is used as the tracking of the corner of the warehouse. area. Taking the three-layer image area as an example, as shown in the three-layer dashed box in Figure 3, taking the corner point of the warehouse as the center, taking the square area according to different pixel distances, so that multiple image areas of different areas can be obtained. From large to small, an image pyramid is formed, and multiple image areas form a multi-layered area.
此外，对于跟踪区域，本发明实施例还可以进行自适应调节，具体可 以以车速作为调节依据，例如：In addition, for the tracking area, the embodiment of the present invention can also perform self-adaptive adjustment. Specifically, the vehicle speed can be used as the adjustment basis, for example:
当车速≤20km/h时，跟踪区域的多层图像区域采用采用默认设定值，即共计三层，每一层大小为30*30，45*45，60*60(单位为厘米)；当车速＞20km/h(根据要求自动泊车功能开启车速不超过35km/h)，以20作为基准，每一层图像区域的宽和高均乘以相应的车速放大系数。假设，车速为30km/h时，车速放大系数为30/20＝1.5，此时跟踪区域的三层图像区域分别为45*45,68*68,90*90。When the vehicle speed is ≤20km/h, the multi-layer image area of the tracking area adopts the default setting value, that is, there are three layers in total, and the size of each layer is 30*30, 45*45, 60*60 (unit is cm); when Vehicle speed > 20km/h (auto-parking function is turned on according to requirements, the vehicle speed should not exceed 35km/h), with 20 as the benchmark, the width and height of each layer of image area are multiplied by the corresponding vehicle speed amplification factor. Assuming that when the vehicle speed is 30km/h, the vehicle speed amplification factor is 30/20=1.5, and the three-layer image areas of the tracking area are 45*45, 68*68, 90*90 respectively.
S30，基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果，目标图像区域为面积最小的、能够在第一全景鸟瞰图的相应区域内检测到库位角点的一层图像区域。S30, based on the optical flow tracking method, the corner points of the warehouse location are tracked in the target image area of the first panoramic bird's-eye view to obtain a visual tracking result of the corner points of the warehouse location. A layer of image area where the corners of the warehouse location are detected in the area.
本发明实施例中，当自动泊车运行后，新的一帧全景鸟瞰图使用前一帧全景鸟瞰图的跟踪区域，将跟踪区域内的库位角点作为特征点进行光流跟踪，形成新的跟踪点集，获得该库位角点的移动轨迹，包含移动方向和移动距离。将跟踪点集所有点的轨迹进行融合处理，可以得到泊车过程中被锁定的库位相对于车辆的位置与方向变化。In the embodiment of the present invention, after the automatic parking operation, a new frame of panoramic bird's-eye view uses the tracking area of the previous frame of panoramic bird's-eye view, and uses the corner points of the storage location in the tracking area as feature points to perform optical flow tracking to form a new frame of panoramic bird's-eye view. The tracking point set is obtained, and the movement trajectory of the corner point of the location is obtained, including the movement direction and movement distance. The trajectories of all points in the tracking point set are fused to obtain the position and direction changes of the locked storage location relative to the vehicle during the parking process.
其中，新的跟踪点集是指完成一次库位角点跟踪后，原来4个库位角点的位置就会发生了变化，所以新的位置就替换了原来的位置，形成了新的跟踪点集。因此，更新的是库位角点的位置，点集里依然是4个点。Among them, the new tracking point set means that the positions of the original four corner points of the warehouse location will change after completing one corner point tracking of the warehouse location, so the new location replaces the original location, forming a new tracking point. set. Therefore, what is updated is the position of the corner point of the storage location, and there are still 4 points in the point set.
需要说明的是，光流跟踪是指跟踪前后两帧图像在平移过程中目标发生的微小图像变化，并以此确定目标在后一帧上的位置。而在本发明实施例中，基于光流跟踪法跟踪库位角点的具体实现原理是，使用库位角点的跟踪区域，针对各层图像区域，确定前后两帧全景鸟瞰图在该区域中灰度差最小、且高度相似的位置作为候选位置，进一步从所有候选位置中选取灰度差最小的位置作为目标位置，该位置即为库位角点在后一帧全景鸟瞰图的位置。It should be noted that optical flow tracking refers to tracking the small image changes of the target during the translation process of the two frames of images before and after, and using this to determine the position of the target on the next frame. In the embodiment of the present invention, the specific implementation principle of tracking the corner points of the warehouse location based on the optical flow tracking method is to use the tracking area of the corner points of the warehouse location, and for each layer of image areas, determine that the two frames of panoramic bird's-eye views before and after are in the area. The position with the smallest grayscale difference and a high degree of similarity is used as the candidate position, and the position with the smallest grayscale difference is further selected from all the candidate positions as the target position, which is the position of the corner point of the storage location in the next frame of panoramic bird's-eye view.
具体实现过程中，目标图像区域的确定过程，包括如下步骤：In the specific implementation process, the determination process of the target image area includes the following steps:
按照上层到下层的次序依次将各层图像区域作为候选图像区域；分别 将第一全景鸟瞰图的候选图像区域、以及第二全景鸟瞰图的候选图像区域划分为多个子区域；分别统计第一全景鸟瞰图的各子区域、第二全景鸟瞰图的各子区域的灰度值概率直方分布；针对第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；统计各区域对的移动趋势，移动趋势包括移动方向和移动距离；判断统计结果中是否存在主体移动趋势，主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；若是，则将候选图像区域作为目标图像区域；若否，则执行按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。According to the order from the upper layer to the lower layer, the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately The gray value probability distribution of each sub-region of the bird's-eye view and each sub-region of the second panoramic bird's-eye view; for each sub-region of the second panoramic bird's-eye view, calculate its and the first panoramic bird's-eye view based on the gray value probability histogram distribution and select a sub-region whose similarity meets the preset matching conditions to form a region pair; count the movement trend of each region pair, which includes the movement direction and movement distance; determine whether there is subject movement in the statistical results trend, the movement trend of the main body is the movement trend of the same moving direction and the proportion of the moving distance belonging to the same range is greater than the preset threshold; if so, the candidate image area is used as the target image area; if not, the execution is performed in the order from upper to lower The image areas of each layer are used as candidate image areas, and it ends when all image areas are traversed.
参见图4所示的子区域对匹配示意图。以第二全景鸟瞰图在库位角点周围所选取的一层图像区域作为候选图像区域为例，在第一全景鸟瞰图上也存在一个大小和位置相同的区域。将第二全景鸟瞰图的候选图像区域记为A、第一全景鸟瞰图的候选图像区域记为A’，则在A区域和A’区域分别划分出多个大小相同的子区域。Refer to the schematic diagram of sub-region pair matching shown in FIG. 4 . Taking a layer of image area selected around the corner of the warehouse location as a candidate image area in the second panoramic bird's-eye view as an example, there is also an area with the same size and position on the first panoramic bird's-eye view. Denote the candidate image area of the second panoramic bird's-eye view as A, and the candidate image area of the first panoramic bird's-eye view as A', then the A area and the A' area are respectively divided into multiple sub-areas of the same size.
首先对第一全景鸟瞰图和第二全景鸟瞰图进行图像灰度化处理，然后分别统计A’区域的各子区域和A区域的各子区域的灰度值直方分布，该灰度值直方分布可以以分布直方图的形式体现，即横坐标为0，1…255的灰度值、纵坐标为灰度值出现的概率。进而，将A’区域的各子区域的灰度值直方分布和A区域的各子区域的灰度值直方分布转换为1*256位特征向量。First, the first panoramic bird's-eye view and the second panoramic bird's-eye view are subjected to image grayscale processing, and then the gray value histogram distribution of each sub-area of the A' area and each sub-area of the A area is counted respectively. It can be expressed in the form of a distribution histogram, that is, the gray value with the abscissa as 0, 1...255, and the ordinate as the probability of occurrence of the gray value. Furthermore, the histogram distribution of the grayscale values of each sub-region of the A' region and the histogram of the grayscale value of each sub-region of the A region are converted into 1*256-bit feature vectors.
针对A区域中的每一个子区域，计算其与A’区域的各子区域的相似度，具体的，使用两个特征向量进行余弦距离的计算，距离越趋于0表示两个子区域的相似度越高、越趋于1表示相似度很低。通过设置相似度阈值，选择A’区域的相似度大于相似度阈值、且相似度最大的一个子区域与该A区域的子区域构成子区域对。如图4中箭头所指示的一对子区域，即为一个子区域对。For each sub-area in the A area, calculate the similarity with each sub-area in the A' area. Specifically, two feature vectors are used to calculate the cosine distance. The closer the distance is to 0, the similarity of the two sub-areas. The higher the value, the closer it is to 1, the lower the similarity. By setting the similarity threshold, a sub-region with the similarity of the A' region greater than the similarity threshold and the maximum similarity is selected to form a sub-region pair with the sub-region of the A region. A pair of sub-areas indicated by arrows in FIG. 4 is a pair of sub-areas.
进一步，对于在某一层图像区域所获得的子区域对，可以统计各子区 域对的移动趋势，即移动方向和移动距离，并从中寻找主体移动趋势，其中主体移动趋势可以是至少占比三分之二的子区域对移动趋势都相同，即移动方向相同、移动距离均属于某一范围内。Further, for the sub-region pairs obtained in an image area of a certain layer, the moving trend of each sub-region pair, that is, the moving direction and the moving distance, can be counted, and the moving trend of the main body can be found from it, and the moving trend of the main body can be at least three. Two-thirds of the sub-regions have the same moving trend, that is, the moving direction is the same and the moving distance is within a certain range.
当然，如果某一层图像区域没有获得子区域对/没有找到主体移动趋势则表示A’区域中不存在库位角点，此时再选择面积较大的一层图像区域作为候选图像区域，并再次执行上述步骤，直到遍历完跟踪区域全部层的图像区域为止，如果在面积最大的一层图像区域也没有获得子区域对/没有找到主体移动趋势，则表示视觉跟踪失败。Of course, if there is no sub-area pair in a certain layer of image area/the main body movement trend is not found, it means that there is no corner point in the A' area. At this time, a layer of image area with a larger area is selected as the candidate image area, and Perform the above steps again until the image areas of all layers of the tracking area are traversed. If no sub-area pair is obtained in the image area of the layer with the largest area/no movement trend of the subject is found, it means that the visual tracking fails.
在此基础上，步骤S30中“基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果”可以采用如下步骤：On this basis, in step S30, the following steps can be adopted for "tracking the corner points of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the warehouse location":
确定各区域对中移动趋势属于主体移动趋势的目标区域对；将目标区域对的移动方向作为库位角点的移动方向，将目标区域对的移动距离的均值作为库位角点的移动距离；基于库位角点的移动方向和移动距离确定库位角点的移动轨迹。Determine that the movement trend of each area pair belongs to the target area pair of the main body movement trend; take the movement direction of the target area pair as the movement direction of the corner point of the storage location, and take the average value of the moving distance of the target area pair as the movement distance of the corner point of the storage location; The moving direction and moving distance of the corner point of the storage location determine the movement track of the corner point of the storage location.
本发明实施例中，可以删除移动趋势不属于主体移动趋势的区域对，删除完成后，所有保留的目标区域对进行移动距离的均值计算，所得结果即为库位角点的移动距离。此外，目标区域对的移动方向即为库位角点的移动方向。由此，即可获得库位在连续全景鸟瞰图中的位置变化，从而得到最新位置。In the embodiment of the present invention, the area pairs whose movement trend does not belong to the main body movement trend can be deleted. After the deletion is completed, the average value of the moving distance is calculated for all the reserved target area pairs, and the obtained result is the moving distance of the corner point of the storage location. In addition, the moving direction of the target area pair is the moving direction of the corner point of the storage location. In this way, the position change of the warehouse location in the continuous panoramic bird's-eye view can be obtained, so as to obtain the latest position.
S40，获取库位角点的轮速跟踪结果，并通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。S40, acquiring the wheel speed tracking result of the corner point of the warehouse, and obtaining the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the warehouse.
本发明实施例中，轮速跟踪是在泊车过程中一直进行的工作，因此在进行库位跟踪时可以同时获得库位角点的轮速跟踪结果和视觉跟踪结果。In the embodiment of the present invention, the wheel speed tracking is always performed during the parking process, so the wheel speed tracking result and the visual tracking result of the corner point of the storage location can be simultaneously obtained when the storage location tracking is performed.
对于轮速跟踪，由于轮速计都存在相应的误差，运动一定距离会便会造成误差累计，而轮速计不会自动修正；对于视觉跟踪，其有可能受到库位超出画面或者被遮挡的影响，造成结果不准确。For wheel speed tracking, since there are corresponding errors in the wheel speedometer, moving a certain distance will cause the error to accumulate, but the wheel speedometer will not be automatically corrected; for visual tracking, it may be affected by the storage space exceeding the screen or being blocked impact, resulting in inaccurate results.
对此，本发明实施例可以结合视觉跟踪结果和泊车场景切换轮速跟踪和视觉跟踪，保证泊车连续性。此外，还可以使用双校验的方式保证跟踪的精确性。具体的：In this regard, the embodiment of the present invention can switch wheel speed tracking and visual tracking in combination with the visual tracking result and the parking scene, so as to ensure the parking continuity. In addition, a double check method can also be used to ensure the tracking accuracy. specific:
1)如果库位角点的视觉跟踪结果表征库位跟踪失败，则将库位角点的轮速跟踪结果作为实际跟踪结果。1) If the visual tracking result of the corner of the warehouse indicates that the tracking of the warehouse has failed, the wheel speed tracking result of the corner of the warehouse is used as the actual tracking result.
视觉跟踪可能会受到超出画面或被遮挡的影响，在库位超出画幅、视觉检测无法检测到库位，或者库位未超出画幅、但被遮挡导致至少2个库位角点没有被跟踪到，或者由于其他环境原因视觉跟踪无输出，此时考虑到泊车连续性，采用轮速跟踪结果作为实际跟踪结果。Visual tracking may be affected by exceeding the frame or being occluded. When the storage location exceeds the frame, the visual inspection cannot detect the storage position, or the storage position does not exceed the frame but is blocked, resulting in at least 2 corners of the storage position not being tracked. Or visual tracking has no output due to other environmental reasons. In this case, considering the parking continuity, the wheel speed tracking result is used as the actual tracking result.
2)如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值，则将库位角点的视觉跟踪结果作为实际跟踪结果。2) If the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used as the actual tracking result.
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度大于3°，则由于此时轮速跟踪的误差非常大，因此采用视觉跟踪结果作为实际跟踪结果。If the visual tracking successfully outputs the moving trajectories of at least three corner points of the warehouse, and the road slope is greater than 3°, because the error of the wheel speed tracking is very large at this time, the visual tracking result is used as the actual tracking result.
进一步，使用库位角点的视觉跟踪结果修正库位角点的轮速跟踪结果。通过对比轮速跟踪结果和视觉跟踪结果，如果检测到轮速跟踪连续多帧的误差都超过某一阈值(大量测试后的统计学结果)，则此时即可确定轮速跟踪出现问题，因此采用视觉跟踪结果对轮速跟踪结果进行位置修正，以消除轮速跟踪累计的误差，使得泊车过程中可以随时调整位姿，保证行车轨迹能够得到实时修正，从而提高一次泊车完成的成功率和准确率。Further, use the visual tracking result of the corner of the warehouse to correct the result of the wheel speed tracking of the corner of the warehouse. By comparing the wheel speed tracking results and the visual tracking results, if it is detected that the errors of multiple consecutive frames of wheel speed tracking exceed a certain threshold (statistical results after a large number of tests), then it can be determined that there is a problem with the wheel speed tracking, so Use the visual tracking result to correct the position of the wheel speed tracking result to eliminate the accumulated error of the wheel speed tracking, so that the position and attitude can be adjusted at any time during the parking process to ensure that the driving trajectory can be corrected in real time, thereby improving the success rate of one parking completion. and accuracy.
3)如果库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离小于等于预设距离阈值，则将库位角点的视觉跟踪结果作为实际跟踪结果。3) If the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to the preset distance threshold, then The visual tracking result of the corner point is used as the actual tracking result.
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度小于等于3°、库位与车辆后轴中心的距离小于等于3米，此时视觉效果最好，因此采用视觉跟踪结果作为实际跟踪结果。Visual tracking successfully outputs the moving trajectories of at least three corner points of the storage location, and the road slope is less than or equal to 3°, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to 3 meters. At this time, the visual effect is the best, so the visual tracking result is used as the Actual tracking results.
需要说明的是，库位距离车辆后轴中心的距离即为库位距离车辆近端的线段中点到车辆后轴中心的直线距离。It should be noted that the distance from the storage location to the center of the rear axle of the vehicle is the straight-line distance from the midpoint of the line segment from the storage location to the near end of the vehicle to the center of the rear axle of the vehicle.
4)库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离大于预设距离阈值，对库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。4) The visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold. The actual tracking results are obtained by double-checking the visual tracking results and the wheel speed tracking results.
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度小于等于3°、库位与车辆后轴中心的距离大于3米，此时采用双校验结果，即确定视觉跟踪结果的可靠度比例系数α、以及轮速跟踪结果的可靠度比例系数1-α。α计算方法如下：Visual tracking successfully outputs the movement trajectories of at least three corner points of the storage location, and the road slope is less than or equal to 3°, and the distance between the storage location and the center of the rear axle of the vehicle is greater than 3 meters. At this time, the double verification result is used, that is, the visual tracking result is determined. The reliability proportional coefficient α, and the reliability proportional coefficient 1-α of the wheel speed tracking result. The calculation method of α is as follows:
参见图5所示的泊车场景示意图。当前时间下的第一全景鸟瞰图中，库位与车辆后轴中心的距离即库位距离车近端的线段中点A到后轴中心O直线距离，长度为D1，此时D1＞3米，取其差值D1’＝(D1-3)；沿OA向做远离车辆方向的延长线，交第一全景鸟瞰图画幅边缘的交点为P，读取AP的长度为D2，则α＝1.0–D1’/D2。Refer to the schematic diagram of the parking scene shown in FIG. 5 . In the first panoramic bird's-eye view at the current time, the distance between the storage location and the center of the rear axle of the vehicle is the straight-line distance from the midpoint A of the line segment between the storage location and the near end of the vehicle to the center O of the rear axle, and the length is D1. At this time, D1 > 3 meters , take the difference D1'=(D1-3); make an extension line along the OA direction away from the vehicle direction, the intersection point with the edge of the first panoramic bird's-eye view picture is P, and the length of the read AP is D2, then α=1.0 –D1'/D2.
因此，采用双校验结果时，实际跟踪结果＝视觉跟踪结果*α+轮速跟踪结果*(1-α)。Therefore, when the double verification result is adopted, the actual tracking result=visual tracking result*α+wheel speed tracking result*(1-α).
本发明的视觉跟踪仅针对被锁定库位周围一定范围内进行处理，并且采取特征点跟踪方式，可以有效进行旋转、颠簸、形变等各种工况下的跟踪，稳定且高效。从而可以对自动泊车中被锁定的库位进行实时跟踪，并可以对轮速跟踪的累积误差进行修正，从而实现自动精确泊车入位。The visual tracking of the present invention is only processed within a certain range around the locked storage location, and the feature point tracking method is adopted, which can effectively perform tracking under various working conditions such as rotation, bumping, and deformation, and is stable and efficient. Therefore, the locked storage space in automatic parking can be tracked in real time, and the accumulated error of wheel speed tracking can be corrected, so as to realize automatic and precise parking.
由于在整个泊车过程中，库位可能会超出图像画面或被遮挡，当跟踪点集的跟踪置信度降低时，跟踪方式将切换到轮速跟踪。当库位重新回到画面中或者距离车越来越近，跟踪方式从轮速切换到视觉跟踪，并进行双校验，从而实现无缝衔接。During the entire parking process, the storage space may exceed the image frame or be occluded. When the tracking confidence of the tracking point set decreases, the tracking method will switch to wheel speed tracking. When the warehouse position returns to the screen or gets closer and closer to the car, the tracking method is switched from wheel speed to visual tracking, and double-checking is performed to achieve seamless connection.
此外，由于视觉跟踪是包含图像信息的，所以一段时间内对轮速跟踪进行一次轨迹修正，从而保持修正后的轮速信息误差最小，不会因为误差 累积而导致出现轨迹偏离。In addition, since the visual tracking contains image information, the wheel speed tracking is corrected once within a period of time, so as to keep the error of the corrected wheel speed information to a minimum, and the track deviation will not occur due to the accumulation of errors.
综上，本发明是一种可以在自动泊车过程中实时跟踪并修正泊车路线的方法。能够有效提高泊车精度及成功率。本发明结构清晰、方法简洁、实时性好、鲁棒性强。To sum up, the present invention is a method that can track and correct the parking route in real time during the automatic parking process. It can effectively improve the parking accuracy and success rate. The invention has clear structure, concise method, good real-time performance and strong robustness.
基于以上实施例提供的库位跟踪方法，本发明实施例则提供一种执行上述库位跟踪方法的装置，该装置的结构示意图如图6所示，包括：Based on the storage location tracking method provided by the above embodiment, an embodiment of the present invention provides a device for executing the above storage location tracking method. The schematic structural diagram of the device is shown in FIG. 6 , including:
图像获取模块10，用于获取车辆在不同时间下的全景鸟瞰图；The image acquisition module 10 is used to acquire panoramic bird's-eye views of the vehicle at different times;
跟踪区域确定模块20，用于针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，跟踪区域由多层以库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积；The tracking area determination module 20 is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time. The image area in the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
库位跟踪模块30，用于基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果，目标图像区域为面积最小的、能够在第一全景鸟瞰图的相应区域内检测到库位角点的一层图像区域；获取库位角点的轮速跟踪结果，并通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。The storage location tracking module 30 is used for tracking the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is the smallest in area and can be located in the first panoramic bird's eye view. A first-level image area where the corner points of the warehouse location are detected in the corresponding area of the panoramic bird's-eye view; the wheel speed tracking results of the corner points of the warehouse location are obtained, and the actual tracking is obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse location. result.
可选的，库位跟踪模块30确定目标图像区域的过程，包括：Optionally, the process of determining the target image area by the location tracking module 30 includes:
按照上层到下层的次序依次将各层图像区域作为候选图像区域；分别将第一全景鸟瞰图的候选图像区域、以及第二全景鸟瞰图的候选图像区域划分为多个子区域；分别统计第一全景鸟瞰图的各子区域、第二全景鸟瞰图的各子区域的灰度值概率直方分布；针对第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；统计各区域对的移动趋势，移动趋势包括移动方向和移动距离；判断统计结果中是否存在主体移动趋势，主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；若是，则将候选图像区域作为目标图像 区域；若否，则执行按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。According to the order from the upper layer to the lower layer, the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately The gray value probability distribution of each sub-region of the bird's-eye view and each sub-region of the second panoramic bird's-eye view; for each sub-region of the second panoramic bird's-eye view, calculate its and the first panoramic bird's-eye view based on the gray value probability histogram distribution and select a sub-region whose similarity meets the preset matching conditions to form a region pair; count the movement trend of each region pair, which includes the movement direction and movement distance; determine whether there is subject movement in the statistical results trend, the movement trend of the main body is the movement trend of the same moving direction and the proportion of the moving distance belonging to the same range is greater than the preset threshold; if so, the candidate image area is used as the target image area; if not, the execution is performed in the order from upper to lower The image areas of each layer are used as candidate image areas, and it ends when all image areas are traversed.
可选的，库位跟踪模块30基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果，包括：Optionally, the storage location tracking module 30 tracks the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the storage location, including:
确定各区域对中移动趋势属于主体移动趋势的目标区域对；将目标区域对的移动方向作为库位角点的移动方向，将目标区域对的移动距离的均值作为库位角点的移动距离；基于库位角点的移动方向和移动距离确定库位角点的移动轨迹。Determine that the movement trend of each area pair belongs to the target area pair of the main body movement trend; take the movement direction of the target area pair as the movement direction of the corner point of the storage location, and take the average value of the moving distance of the target area pair as the movement distance of the corner point of the storage location; The moving direction and moving distance of the corner point of the storage location determine the movement track of the corner point of the storage location.
可选的，库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：Optionally, the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
如果库位角点的视觉跟踪结果表征库位跟踪失败，则将库位角点的轮速跟踪结果作为实际跟踪结果。If the visual tracking result of the corner of the warehouse indicates that the tracking of the warehouse has failed, the wheel speed tracking result of the corner of the warehouse is used as the actual tracking result.
可选的，库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：Optionally, the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值，则将库位角点的视觉跟踪结果作为实际跟踪结果；If the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
或者or
如果库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离小于等于预设距离阈值，则将库位角点的视觉跟踪结果作为实际跟踪结果。If the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to the preset distance threshold, the storage location angle The visual tracking result of the point is used as the actual tracking result.
可选的，库位跟踪模块30还用于：Optionally, the location tracking module 30 is also used for:
如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值，使用库位角点的视觉跟踪结果修正库位角点的轮速跟踪结果。If the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, use the visual tracking result of the corner point of the warehouse location to correct the wheel speed tracking result of the corner point of the warehouse location.
可选的，库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟 踪结果得到实际跟踪结果，包括：Optionally, the storage location tracking module 30 obtains the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the storage location, including:
库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离大于预设距离阈值，对库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。The visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold. The actual tracking results are obtained by double-checking the tracking results and the wheel speed tracking results.
本发明实施例提供的库位跟踪装置，采用视觉检测的方法对自动泊车过程中已经锁定好的库位进行实时定位，从而修正轮速跟踪所带来的累计误差问题，大幅度提高一次泊车入位、以及居中停靠的概率，提高用户体验。The storage space tracking device provided by the embodiment of the present invention uses the visual detection method to locate the locked storage space in the automatic parking process in real time, thereby correcting the cumulative error problem caused by wheel speed tracking, and greatly improving one parking time. The probability of car entry and center parking improves user experience.
以上对本发明所提供的一种库位跟踪方法及装置进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。The storage location tracking method and device provided by the present invention have been introduced in detail above. The principles and implementations of the present invention are described with specific examples in this paper. The descriptions of the above embodiments are only used to help understand the present invention. method and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. Invention limitations.
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, refer to each other Can. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.
还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备所固有的要素，或者是还包括为这些过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should also be noted that in this document, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply those entities or operations There is no such actual relationship or order between them. Furthermore, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article, or device of a list of elements is included, inherent to, or is also included for, those processes. , method, article or device inherent elements. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使 用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下，在其它实施例中实现。因此，本发明将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

  1. 一种库位跟踪方法，其特征在于，所述方法包括：A storage location tracking method, characterized in that the method comprises:
    获取车辆在不同时间下的全景鸟瞰图；Obtain panoramic bird's-eye views of the vehicle at different times;
    针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积；For the first panoramic bird's-eye view at the current time, determine the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers of image areas centered on the corner of the storage location The area of the image area of the upper layer is smaller than the area of the image area of the lower layer in the adjacent two-layer image areas;
    基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域；Based on the optical flow tracking method, the corner point of the warehouse location is tracked in the target image area of the first panoramic bird's-eye view to obtain the visual tracking result of the corner point of the warehouse location. A layer of image area where the corner point of the storage location is detected in the corresponding area of the first panoramic bird's-eye view;
    获取所述库位角点的轮速跟踪结果，并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。Acquire the wheel speed tracking result of the corner point of the storage location, and obtain the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the storage location.
  2. 根据权利要求1所述的方法，其特征在于，所述目标图像区域的确定过程，包括：The method according to claim 1, wherein the process of determining the target image area comprises:
    按照上层到下层的次序依次将各层图像区域作为候选图像区域；According to the order from the upper layer to the lower layer, the image regions of each layer are used as candidate image regions;
    分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域；respectively dividing the candidate image area of the first panoramic bird's-eye view and the candidate image area of the second panoramic bird's-eye view into a plurality of sub-regions;
    分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布；Statistics of the gray value probability histogram distribution of each sub-region of the first panoramic bird's-eye view and each sub-region of the second panoramic bird's-eye view;
    针对所述第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；For each sub-region of the second panoramic bird's-eye view, calculate the similarity between it and each sub-region of the first panoramic bird's-eye view based on the probability histogram distribution of gray values, and select a sub-region whose similarity meets the preset matching condition Regions form Region Pairs;
    统计各区域对的移动趋势，所述移动趋势包括移动方向和移动距离；Statistics on the movement trend of each area pair, the movement trend includes movement direction and movement distance;
    判断统计结果中是否存在主体移动趋势，所述主体移动趋势为移动方 向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；Judging whether there is a movement trend of the main body in the statistical result, the movement trend of the main body is the movement trend of the same movement direction and the proportion of the movement distance belonging to the same range greater than the preset threshold;
    若是，则将所述候选图像区域作为目标图像区域；If so, the candidate image area is used as the target image area;
    若否，则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。If not, execute the step of sequentially taking the image regions of each layer as candidate image regions in the order from the upper layer to the lower layer, and finish when all the image regions are traversed.
  3. 根据权利要求2所述的方法，其特征在于，所述基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，包括：The method according to claim 2, wherein the visual tracking of the corner points of the storage location is obtained by tracking the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on an optical flow tracking method Results, including:
    确定各区域对中移动趋势属于所述主体移动趋势的目标区域对；Determine that the movement trend of each area pair belongs to the target area pair of the movement trend of the subject;
    将所述目标区域对的移动方向作为所述库位角点的移动方向，将所述目标区域对的移动距离的均值作为所述库位角点的移动距离；Taking the moving direction of the pair of target areas as the moving direction of the corners of the storage location, and taking the average value of the moving distances of the pair of target areas as the moving distance of the corners of the storage location;
    基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。The movement trajectory of the storage location corner point is determined based on the moving direction and the moving distance of the storage location corner point.
  4. 根据权利要求1所述的方法，其特征在于，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：The method according to claim 1, wherein the obtaining an actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the warehouse, comprising:
    如果所述库位角点的视觉跟踪结果表征库位跟踪失败，则将所述库位角点的轮速跟踪结果作为实际跟踪结果。If the visual tracking result of the corner point of the storage location indicates that the storage location tracking fails, the tracking result of the wheel speed of the corner point of the storage location is used as the actual tracking result.
  5. 根据权利要求1所述的方法，其特征在于，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：The method according to claim 1, wherein the obtaining an actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the warehouse, comprising:
    如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于预设坡度阈值，则将所述库位角点的视觉跟踪结果作为实际跟踪结果；If the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
    或者or
    如果所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于所述预设坡度阈值、且所述库位与所述车辆的后轴中心的距离小于等于预设距离阈值，则将所述库位角点的视觉跟踪结果作为 实际跟踪结果。If the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, the gradient of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the warehouse and the center of the rear axle of the vehicle is less than or equal to If the distance threshold is preset, the visual tracking result of the corner point of the warehouse is used as the actual tracking result.
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：The method according to claim 5, wherein the method further comprises:
    如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于所述预设坡度阈值，使用所述库位角点的视觉跟踪结果修正所述库位角点的轮速跟踪结果。If the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used to correct the location angle Point wheel speed tracking results.
  7. 根据权利要求1所述的方法，其特征在于，所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果，包括：The method according to claim 1, wherein the obtaining an actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the warehouse, comprising:
    所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于预设坡度阈值、且所述库位与所述车辆的后轴中心的距离大于预设距离阈值，对所述库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。The visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to a preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than a preset distance threshold. , and double-check the visual tracking result and the wheel speed tracking result of the corner point of the storage location to obtain the actual tracking result.
  8. 一种库位跟踪装置，其特征在于，所述装置包括：A storage location tracking device, characterized in that the device comprises:
    图像获取模块，用于获取车辆在不同时间下的全景鸟瞰图；The image acquisition module is used to acquire panoramic bird's-eye views of the vehicle at different times;
    跟踪区域确定模块，用于针对当前时间下的第一全景鸟瞰图，确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域，所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积；The tracking area determination module is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers with the storage location. The image area with the corner point as the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
    库位跟踪模块，用于基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域；获取所述库位角点的轮速跟踪结果，并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。A storage location tracking module is used to track the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is A first-level image area with the smallest area that can detect the corners of the storage location in the corresponding area of the first panoramic bird's-eye view; obtain the wheel speed tracking results of the corners of the storage location, and process the storage location by processing the storage location. The visual tracking results and wheel speed tracking results of the corner points are used to obtain the actual tracking results.
  9. 根据权利要求8所述的装置，其特征在于，所述库位跟踪模块确定目标图像区域的过程，包括：The device according to claim 8, wherein the process of determining the target image area by the storage location tracking module comprises:
    按照上层到下层的次序依次将各层图像区域作为候选图像区域；分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的 所述候选图像区域划分为多个子区域；分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布；针对所述第二全景鸟瞰图的每个子区域，基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度，并选取相似度符合预设匹配条件的一个子区域构成区域对；统计各区域对的移动趋势，所述移动趋势包括移动方向和移动距离；判断统计结果中是否存在主体移动趋势，所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势；若是，则将所述候选图像区域作为目标图像区域；若否，则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域，直到遍历完所有图像区域时结束。The image regions of each layer are taken as candidate image regions in sequence from the upper layer to the lower layer; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are respectively divided into multiple each sub-region of the first panoramic bird's-eye view and the gray value probability histogram distribution of each sub-region of the second panoramic bird's-eye view are counted separately; for each sub-region of the second panoramic bird's-eye view, based on The gray value probability histogram distribution calculates the similarity between it and each sub-region of the first panoramic bird's-eye view, and selects a sub-region whose similarity meets the preset matching condition to form a region pair; The movement trend includes the movement direction and the movement distance; determine whether there is a movement trend of the main body in the statistical results, and the movement trend of the main body is the movement trend of the same movement direction and the movement distance belonging to the same range that the proportion is greater than the preset threshold; The candidate image area is used as the target image area; if not, the process of sequentially taking the image areas of each layer as the candidate image area in the order from the upper layer to the lower layer is performed until the traversal of all the image areas ends.
  10. 根据权利要求9所述的装置，其特征在于，所述库位跟踪模块基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果，包括：The device according to claim 9, wherein the storage location tracking module tracks the storage location corner points in the target image area of the first panoramic bird's-eye view based on an optical flow tracking method to obtain the storage location angle Visual tracking results for points, including:
    确定各区域对中移动趋势属于所述主体移动趋势的目标区域对；将所述目标区域对的移动方向作为所述库位角点的移动方向，将所述目标区域对的移动距离的均值作为所述库位角点的移动距离；基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。It is determined that the movement trend of each area pair belongs to the target area pair of the main body movement trend; The movement distance of the corner point of the storage location is determined; the movement trajectory of the corner point of the storage location is determined based on the movement direction and the movement distance of the corner point of the storage location.
PCT/CN2021/139527 2020-11-04 2021-12-20 Garage space tracking method and apparatus WO2022096027A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011212958.9 2020-11-04
CN202011212958.9A CN112037265B (en) 2020-11-04 2020-11-04 Library bit tracking method and device

Publications (1)

Publication Number Publication Date
WO2022096027A1 true WO2022096027A1 (en) 2022-05-12

Family

ID=73573154

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/139527 WO2022096027A1 (en) 2020-11-04 2021-12-20 Garage space tracking method and apparatus

Country Status (2)

Country Link
CN (1) CN112037265B (en)
WO (1) WO2022096027A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112037265B (en) * 2020-11-04 2021-02-02 天津天瞳威势电子科技有限公司 Library bit tracking method and device
CN112356831B (en) * 2021-01-12 2021-04-20 天津天瞳威势电子科技有限公司 Warehouse location tracking method and warehouse location tracking system
CN113066306B (en) * 2021-03-23 2022-07-08 超级视线科技有限公司 Management method and device for roadside parking
CN113963034A (en) * 2021-10-22 2022-01-21 长春一汽富晟集团有限公司 Multi-parking space target tracking method based on vision
CN115601271B (en) * 2022-11-29 2023-03-24 上海仙工智能科技有限公司 Visual information anti-shake method, storage warehouse location state management method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105374049A (en) * 2014-09-01 2016-03-02 浙江宇视科技有限公司 Multi-angle-point tracking method based on sparse optical flow method and apparatus thereof
CN109443348A (en) * 2018-09-25 2019-03-08 同济大学 It is a kind of based on the underground garage warehouse compartment tracking for looking around vision and inertial navigation fusion
US20200042805A1 (en) * 2017-09-19 2020-02-06 Jvckenwood Corporation Display control device, display control system, display control method, and non-transitory storage medium
CN111016918A (en) * 2018-10-10 2020-04-17 上海汽车集团股份有限公司 Library position detection method and device and model training device
CN111508260A (en) * 2019-01-30 2020-08-07 上海欧菲智能车联科技有限公司 Vehicle parking space detection method, device and system
CN112037265A (en) * 2020-11-04 2020-12-04 天津天瞳威势电子科技有限公司 Library bit tracking method and device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881645B (en) * 2015-05-26 2018-09-14 南京通用电器有限公司 The vehicle front mesh object detection method of feature based point mutual information and optical flow method
CN105825525A (en) * 2016-03-16 2016-08-03 中山大学 TLD target tracking method and device based on Mean-shift model optimization
CN106004515B (en) * 2016-05-12 2018-04-10 广州橙行智动汽车科技有限公司 Method for controlling driving speed and system for electric automobile automatic parking
CN109697860A (en) * 2017-10-20 2019-04-30 上海欧菲智能车联科技有限公司 Parking stall measure and tracking system and method and vehicle
CN108764216A (en) * 2018-07-11 2018-11-06 天津天瞳威势电子科技有限公司 A kind of traffic lights recognition methods of view-based access control model and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105374049A (en) * 2014-09-01 2016-03-02 浙江宇视科技有限公司 Multi-angle-point tracking method based on sparse optical flow method and apparatus thereof
US20200042805A1 (en) * 2017-09-19 2020-02-06 Jvckenwood Corporation Display control device, display control system, display control method, and non-transitory storage medium
CN109443348A (en) * 2018-09-25 2019-03-08 同济大学 It is a kind of based on the underground garage warehouse compartment tracking for looking around vision and inertial navigation fusion
CN111016918A (en) * 2018-10-10 2020-04-17 上海汽车集团股份有限公司 Library position detection method and device and model training device
CN111508260A (en) * 2019-01-30 2020-08-07 上海欧菲智能车联科技有限公司 Vehicle parking space detection method, device and system
CN112037265A (en) * 2020-11-04 2020-12-04 天津天瞳威势电子科技有限公司 Library bit tracking method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG, PENGFEI: " Research on Parking Slot Detection Technology Based on Panoramic Around View System", SCIENCE & ENGINEERING (B), CHINA MASTER’S THESES FULL-TEXT DATABASE, 1 December 2018 (2018-12-01), pages 1 - 80, XP055929074, [retrieved on 20220608] *

Also Published As

Publication number Publication date
CN112037265A (en) 2020-12-04
CN112037265B (en) 2021-02-02

Similar Documents

Publication Publication Date Title
WO2022096027A1 (en) Garage space tracking method and apparatus
US11240471B2 (en) Road vertical contour detection
US9256791B2 (en) Road vertical contour detection
US8259174B2 (en) Camera auto-calibration by horizon estimation
JP4919036B2 (en) Moving object recognition device
JP5966747B2 (en) Vehicle travel control apparatus and method
US7899211B2 (en) Object detecting system and object detecting method
WO2019116958A1 (en) Onboard environment recognition device
US8594378B2 (en) 3D object detecting apparatus and 3D object detecting method
US11403767B2 (en) Method and apparatus for detecting a trailer, tow-ball, and coupler for trailer hitch assistance and jackknife prevention
JP2002197444A (en) Run lane recognition device for vehicle
LU502288B1 (en) Method and system for detecting position relation between vehicle and lane line, and storage medium
US8730325B2 (en) Traveling lane detector
Lopez et al. Detection of lane markings based on ridgeness and RANSAC
CN115578470B (en) Monocular vision positioning method and device, storage medium and electronic equipment
Rasmussen RoadCompass: following rural roads with vision+ ladar using vanishing point tracking
CN114141055B (en) Parking space detection device and method of intelligent parking system
WO2022133986A1 (en) Accuracy estimation method and system
Kobayashi et al. Wide Angle Multi-Shift Stereo Camera with Monocular Vision
JP3951734B2 (en) Vehicle external recognition device
CN114998849B (en) Traffic flow element sensing and positioning method based on road-side monocular camera and application thereof
CN114274948A (en) Automatic parking method and device based on 360-degree panorama
McFall et al. Visual lane detection algorithm using the perspective transform
JP2000090240A (en) Method and device for detecting vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21888719

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.10.2023)