WO2022096027A1 - 一种库位跟踪方法及装置 - Google Patents
一种库位跟踪方法及装置 Download PDFInfo
- Publication number
- WO2022096027A1 WO2022096027A1 PCT/CN2021/139527 CN2021139527W WO2022096027A1 WO 2022096027 A1 WO2022096027 A1 WO 2022096027A1 CN 2021139527 W CN2021139527 W CN 2021139527W WO 2022096027 A1 WO2022096027 A1 WO 2022096027A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tracking
- storage location
- eye view
- area
- corner point
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 80
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 claims abstract description 109
- 230000000007 visual effect Effects 0.000 claims abstract description 87
- 230000008569 process Effects 0.000 claims abstract description 26
- 230000003287 optical effect Effects 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 17
- 240000004050 Pentaglottis sempervirens Species 0.000 claims description 101
- 238000001514 detection method Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000003321 amplification Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/60—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
- B60R2300/607—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/806—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Definitions
- tracking the storage space and automatically parking is a dynamic process that is easily affected by various scene factors.
- the driving speed is low, and it is accompanied by a large number of steering wheel rotations, starting and stopping, shifting and other steps.
- the tracking work of the wheel speedometer will deviate and accumulate continuously, making it easy to park. A misalignment or even a crash occurs.
- the present invention provides a storage location tracking method and device, and the technical solutions are as follows:
- a storage location tracking method comprises:
- the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time is composed of multiple layers of image areas centered on the corner of the storage location
- the area of the image area of the upper layer is smaller than the area of the image area of the lower layer in the adjacent two-layer image areas;
- the corner point of the warehouse location is tracked in the target image area of the first panoramic bird's-eye view to obtain the visual tracking result of the corner point of the warehouse location.
- the process of determining the target image area includes:
- the image regions of each layer are used as candidate image regions
- the movement trend includes movement direction and movement distance
- the movement trend of the main body is a movement trend in which the movement direction is the same and the proportion of the movement distance belonging to the same range is greater than a preset threshold;
- the candidate image area is used as the target image area
- the visual tracking result of the corner point of the warehouse location obtained by tracking the corner point of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method includes:
- the movement trajectory of the storage location corner point is determined based on the moving direction and the moving distance of the storage location corner point.
- the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
- the tracking result of the wheel speed of the corner point of the storage location is used as the actual tracking result.
- the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
- the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
- the gradient of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the warehouse and the center of the rear axle of the vehicle is less than or equal to If the distance threshold is preset, the visual tracking result of the corner point of the warehouse is used as the actual tracking result.
- the method further includes:
- the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used to correct the location angle Point wheel speed tracking results.
- the actual tracking results are obtained by processing the visual tracking results and the wheel speed tracking results of the corner points of the warehouse, including:
- the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to a preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than a preset distance threshold , and double-check the visual tracking result and the wheel speed tracking result of the corner point of the storage location to obtain the actual tracking result.
- a storage location tracking device includes:
- the image acquisition module is used to acquire panoramic bird's-eye views of the vehicle at different times;
- the tracking area determination module is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time, and the tracking area is composed of multiple layers with the storage location.
- the image area with the corner point as the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
- a storage location tracking module is used to track the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is A first-level image area with the smallest area that can detect the corners of the storage location in the corresponding area of the first panoramic bird's-eye view; obtain the wheel speed tracking results of the corners of the storage location, and process the storage location by processing the storage location.
- the visual tracking results and wheel speed tracking results of the corner points are used to obtain the actual tracking results.
- the process of determining the target image area by the storage location tracking module includes:
- the image regions of each layer are taken as candidate image regions in sequence from the upper layer to the lower layer; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are respectively divided into multiple each sub-region of the first panoramic bird's-eye view and the gray value probability histogram distribution of each sub-region of the second panoramic bird's-eye view are counted separately; for each sub-region of the second panoramic bird's-eye view, based on The gray value probability histogram distribution calculates the similarity between it and each sub-region of the first panoramic bird's-eye view, and selects a sub-region whose similarity meets the preset matching condition to form a region pair;
- the movement trend includes the movement direction and the movement distance; determine whether there is a movement trend of the main body in the statistical results, and the movement trend of the main body is the movement trend in which the movement direction is the same and the movement distance belongs to the same range and the proportion is greater than the preset threshold;
- the candidate image area is
- the storage location tracking module tracks the storage location corner points in the target image area of the first panoramic bird's-eye view based on an optical flow tracking method to obtain visual tracking results of the storage location corner points, including:
- the movement trend of each area pair belongs to the target area pair of the main body movement trend;
- the movement distance of the corner point of the storage location is determined;
- the movement trajectory of the corner point of the storage location is determined based on the movement direction and the movement distance of the corner point of the storage location.
- the invention provides a storage location tracking method and device.
- the visual detection method is used to locate the locked storage location in the automatic parking process in real time, so as to correct the cumulative error problem caused by the wheel speed tracking, and greatly improve one time
- the probability of parking and parking in the center improves the user experience.
- Fig. 1 is the method flow chart of the storage location tracking method that the embodiment of the present invention provides
- FIG. 3 is a schematic diagram of optical flow tracking provided by an embodiment of the present invention.
- FIG. 4 is a schematic diagram of sub-region pair matching provided by an embodiment of the present invention.
- FIG. 5 is a schematic diagram of a parking scene provided by an embodiment of the present invention.
- FIG. 6 is a schematic structural diagram of a storage location tracking device according to an embodiment of the present invention.
- the automatic parking function is turned on.
- the vehicle starts the automatic parking process.
- the vehicle is equipped with an accurate wheel speedometer to calculate the speed and acceleration of the vehicle, and then calculate the unit time. The distance traveled down, and use this to locate the relative distance between the moving vehicle and the storage location.
- the work of the wheel speedometer does not depend on other information. Deviations will occur in bad scenarios such as slopes on the ground or poor road conditions, causing deviations in the parking space and even the danger of a collision.
- an embodiment of the present invention provides a storage location tracking method, and the method includes: Follow the steps below:
- the vehicle is equipped with a vehicle-mounted 360-degree panoramic camera. Based on the vehicle-mounted 360-degree panoramic camera, an image of the environment where the vehicle is located can be obtained, and then scene recognition is performed on the image to obtain information about the environment where the vehicle is located, including road conditions and obstacles. objects, and ground markings.
- the image coordinate system of the panoramic bird's-eye view can be obtained by converting the vehicle coordinate system to the image coordinate system of the panoramic bird's-eye view, and the image of the environment where the vehicle is located is mapped to the image coordinate system of the panoramic bird's-eye view to obtain the panoramic bird's eye view of the vehicle at different times.
- S20 for the first panoramic bird's-eye view at the current time, determine the tracking area of the corner point of the warehouse location in the second panoramic bird's-eye view image that is closest to the current time, and the tracking area is composed of multiple layers of image areas centered on the corner of the warehouse location, And the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area.
- a unit interval for storage location tracking can be preset, and the storage location detection is performed on the first frame of panoramic bird's-eye view in each unit interval to determine whether there is a storage location in the frame, and the second frame and For other subsequent frames of panoramic bird's-eye view, the optical flow tracking method is used to track the corners of the warehouse.
- the unit interval can take the number of frames of the panoramic bird's-eye view as the dimension (for example, every 8 frames of the panoramic bird's-eye view is a unit interval), or the duration of the panoramic bird's-eye view can be the dimension (for example, the panoramic bird's eye view in every 30S is one unit interval) unit interval), which is not limited in this embodiment of the present invention.
- a certain frame of panoramic bird's-eye view is based on the previous frame of panoramic bird's-eye view as the benchmark for storage location tracking.
- each storage location has four corner points, and the essence of tracking the storage location is to track the four corner points of the storage location. Therefore, the embodiment of the present invention selects the previous frame of the panoramic bird's-eye view, that is, the second panoramic bird's-eye view has been detected. The positions of the corners of the four warehouse locations that come out are then used directly.
- the four corners of the warehouse in the second panoramic bird's-eye view are used as the tracking start point set, and each corner of the warehouse is centered, and a multi-layer image area is formed in the form of a pyramid, which is used as the tracking of the corner of the warehouse. area.
- a multi-layer image area is formed in the form of a pyramid, which is used as the tracking of the corner of the warehouse. area.
- the embodiment of the present invention can also perform self-adaptive adjustment.
- the vehicle speed can be used as the adjustment basis, for example:
- the corner points of the warehouse location are tracked in the target image area of the first panoramic bird's-eye view to obtain a visual tracking result of the corner points of the warehouse location.
- a new frame of panoramic bird's-eye view uses the tracking area of the previous frame of panoramic bird's-eye view, and uses the corner points of the storage location in the tracking area as feature points to perform optical flow tracking to form a new frame of panoramic bird's-eye view.
- the tracking point set is obtained, and the movement trajectory of the corner point of the location is obtained, including the movement direction and movement distance.
- the trajectories of all points in the tracking point set are fused to obtain the position and direction changes of the locked storage location relative to the vehicle during the parking process.
- the new tracking point set means that the positions of the original four corner points of the warehouse location will change after completing one corner point tracking of the warehouse location, so the new location replaces the original location, forming a new tracking point. set. Therefore, what is updated is the position of the corner point of the storage location, and there are still 4 points in the point set.
- optical flow tracking refers to tracking the small image changes of the target during the translation process of the two frames of images before and after, and using this to determine the position of the target on the next frame.
- the specific implementation principle of tracking the corner points of the warehouse location based on the optical flow tracking method is to use the tracking area of the corner points of the warehouse location, and for each layer of image areas, determine that the two frames of panoramic bird's-eye views before and after are in the area.
- the position with the smallest grayscale difference and a high degree of similarity is used as the candidate position, and the position with the smallest grayscale difference is further selected from all the candidate positions as the target position, which is the position of the corner point of the storage location in the next frame of panoramic bird's-eye view.
- the determination process of the target image area includes the following steps:
- the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately
- the first panoramic bird's-eye view and the second panoramic bird's-eye view are subjected to image grayscale processing, and then the gray value histogram distribution of each sub-area of the A' area and each sub-area of the A area is counted respectively. It can be expressed in the form of a distribution histogram, that is, the gray value with the abscissa as 0, 1...255, and the ordinate as the probability of occurrence of the gray value. Furthermore, the histogram distribution of the grayscale values of each sub-region of the A' region and the histogram of the grayscale value of each sub-region of the A region are converted into 1*256-bit feature vectors.
- a sub-region with the similarity of the A' region greater than the similarity threshold and the maximum similarity is selected to form a sub-region pair with the sub-region of the A region.
- a pair of sub-areas indicated by arrows in FIG. 4 is a pair of sub-areas.
- the moving trend of each sub-region pair that is, the moving direction and the moving distance
- the moving trend of the main body can be found from it, and the moving trend of the main body can be at least three. Two-thirds of the sub-regions have the same moving trend, that is, the moving direction is the same and the moving distance is within a certain range.
- step S30 the following steps can be adopted for "tracking the corner points of the warehouse location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the warehouse location":
- the area pairs whose movement trend does not belong to the main body movement trend can be deleted.
- the average value of the moving distance is calculated for all the reserved target area pairs, and the obtained result is the moving distance of the corner point of the storage location.
- the moving direction of the target area pair is the moving direction of the corner point of the storage location.
- the wheel speed tracking is always performed during the parking process, so the wheel speed tracking result and the visual tracking result of the corner point of the storage location can be simultaneously obtained when the storage location tracking is performed.
- the embodiment of the present invention can switch wheel speed tracking and visual tracking in combination with the visual tracking result and the parking scene, so as to ensure the parking continuity.
- a double check method can also be used to ensure the tracking accuracy. specific:
- Visual tracking may be affected by exceeding the frame or being occluded.
- the visual inspection cannot detect the storage position, or the storage position does not exceed the frame but is blocked, resulting in at least 2 corners of the storage position not being tracked. Or visual tracking has no output due to other environmental reasons. In this case, considering the parking continuity, the wheel speed tracking result is used as the actual tracking result.
- the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner point of the warehouse location is used as the actual tracking result.
- the visual tracking successfully outputs the moving trajectories of at least three corner points of the warehouse, and the road slope is greater than 3°, because the error of the wheel speed tracking is very large at this time, the visual tracking result is used as the actual tracking result.
- the visual tracking result of the corner point of the storage location indicates that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to the preset distance threshold, then The visual tracking result of the corner point is used as the actual tracking result.
- Visual tracking successfully outputs the moving trajectories of at least three corner points of the storage location, and the road slope is less than or equal to 3°, and the distance between the storage location and the center of the rear axle of the vehicle is less than or equal to 3 meters. At this time, the visual effect is the best, so the visual tracking result is used as the Actual tracking results.
- the distance from the storage location to the center of the rear axle of the vehicle is the straight-line distance from the midpoint of the line segment from the storage location to the near end of the vehicle to the center of the rear axle of the vehicle.
- the visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold.
- the actual tracking results are obtained by double-checking the visual tracking results and the wheel speed tracking results.
- the distance between the storage location and the center of the rear axle of the vehicle is the straight-line distance from the midpoint A of the line segment between the storage location and the near end of the vehicle to the center O of the rear axle, and the length is D1.
- the actual tracking result visual tracking result* ⁇ +wheel speed tracking result*(1- ⁇ ).
- the visual tracking of the present invention is only processed within a certain range around the locked storage location, and the feature point tracking method is adopted, which can effectively perform tracking under various working conditions such as rotation, bumping, and deformation, and is stable and efficient. Therefore, the locked storage space in automatic parking can be tracked in real time, and the accumulated error of wheel speed tracking can be corrected, so as to realize automatic and precise parking.
- the storage space may exceed the image frame or be occluded.
- the tracking method will switch to wheel speed tracking.
- the warehouse position returns to the screen or gets closer and closer to the car, the tracking method is switched from wheel speed to visual tracking, and double-checking is performed to achieve seamless connection.
- the wheel speed tracking is corrected once within a period of time, so as to keep the error of the corrected wheel speed information to a minimum, and the track deviation will not occur due to the accumulation of errors.
- the present invention is a method that can track and correct the parking route in real time during the automatic parking process. It can effectively improve the parking accuracy and success rate.
- the invention has clear structure, concise method, good real-time performance and strong robustness.
- an embodiment of the present invention provides a device for executing the above storage location tracking method.
- the schematic structural diagram of the device is shown in FIG. 6 , including:
- the image acquisition module 10 is used to acquire panoramic bird's-eye views of the vehicle at different times;
- the tracking area determination module 20 is used to determine, for the first panoramic bird's-eye view at the current time, the tracking area of the corner point of the storage location in the second panoramic bird's-eye view that is closest to the current time.
- the image area in the center is formed, and the area of the upper image area in the adjacent two image areas is smaller than the area of the lower image area;
- the storage location tracking module 30 is used for tracking the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking result of the corner points of the storage location, and the target image area is the smallest in area and can be located in the first panoramic bird's eye view.
- the process of determining the target image area by the location tracking module 30 includes:
- the image regions of each layer are taken as candidate image regions; the candidate image regions of the first panoramic bird's-eye view and the candidate image regions of the second panoramic bird's-eye view are divided into multiple sub-regions; the first panorama is counted separately
- the storage location tracking module 30 tracks the corner points of the storage location in the target image area of the first panoramic bird's-eye view based on the optical flow tracking method to obtain the visual tracking results of the corner points of the storage location, including:
- the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
- the wheel speed tracking result of the corner of the warehouse is used as the actual tracking result.
- the storage location tracking module 30 obtains actual tracking results by processing the visual tracking results and the wheel speed tracking results of the corner points of the storage location, including:
- the visual tracking result of the corner point of the warehouse indicates that the tracking of the warehouse is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, the visual tracking result of the corner of the warehouse is used as the actual tracking result;
- the storage location angle The visual tracking result of the point is used as the actual tracking result.
- the location tracking module 30 is also used for:
- the visual tracking result of the corner point of the warehouse location indicates that the location tracking is successful, and the slope of the road where the vehicle is located is greater than the preset gradient threshold, use the visual tracking result of the corner point of the warehouse location to correct the wheel speed tracking result of the corner point of the warehouse location.
- the storage location tracking module 30 obtains the actual tracking result by processing the visual tracking result and the wheel speed tracking result of the corner point of the storage location, including:
- the visual tracking results of the corner points of the storage location indicate that the tracking of the storage location is successful, the slope of the road where the vehicle is located is less than or equal to the preset gradient threshold, and the distance between the storage location and the center of the rear axle of the vehicle is greater than the preset distance threshold.
- the actual tracking results are obtained by double-checking the tracking results and the wheel speed tracking results.
- the storage space tracking device uses the visual detection method to locate the locked storage space in the automatic parking process in real time, thereby correcting the cumulative error problem caused by wheel speed tracking, and greatly improving one parking time.
- the probability of car entry and center parking improves user experience.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
Abstract
本发明提供一种库位跟踪方法及装置,方法包括:获取车辆在不同时间下的全景鸟瞰图;针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域;基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果,目标图像区域为面积最小的、能够在第一全景鸟瞰图的相应区域内检测到库位角点的一层图像区域;获取库位角点的轮速跟踪结果,并通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。本发明采用视觉检测的方法对自动泊车过程中已经锁定好的库位进行实时定位,从而修正轮速跟踪所带来的累计误差问题,大幅度提高一次泊车入位、以及居中停靠的概率,提高用户体验。
Description
本申请要求于2020年11月4日提交中国专利局、申请号为202011212958.9、发明名称为“一种库位跟踪方法及装置”的国内申请的优先权,其全部内容通过引用结合在本申请中。
目前市面上很多智能车辆所带的泊车辅助***都具有自动泊车功能,其主要工作过程为探测库位、锁定库位、跟踪库位并自动泊车、完成泊车这四个步骤。
其中,跟踪库位并自动泊车是一个容易受到各种场景因素影响的动态过程。这一过程行车速度低、并伴随着大量的方向盘转动、启动停止、换挡等步骤,一旦地面存在坡度或路况不好,轮速计的跟踪工作就会发生偏差并不断积累,很容易泊车入位发生偏移甚至是出现撞车的情况。
发明内容
有鉴于此,为解决上述问题,本发明提供一种库位跟踪方法及装置,技术方案如下:
一种库位跟踪方法,所述方法包括:
获取车辆在不同时间下的全景鸟瞰图;
针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积;
基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层 图像区域;
获取所述库位角点的轮速跟踪结果,并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
优选的,所述目标图像区域的确定过程,包括:
按照上层到下层的次序依次将各层图像区域作为候选图像区域;
分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域;
分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布;
针对所述第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;
统计各区域对的移动趋势,所述移动趋势包括移动方向和移动距离;
判断统计结果中是否存在主体移动趋势,所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;
若是,则将所述候选图像区域作为目标图像区域;
若否,则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
优选的,所述基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,包括:
确定各区域对中移动趋势属于所述主体移动趋势的目标区域对;
将所述目标区域对的移动方向作为所述库位角点的移动方向,将所述目标区域对的移动距离的均值作为所述库位角点的移动距离;
基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨 迹。
优选的,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:
如果所述库位角点的视觉跟踪结果表征库位跟踪失败,则将所述库位角点的轮速跟踪结果作为实际跟踪结果。
优选的,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于预设坡度阈值,则将所述库位角点的视觉跟踪结果作为实际跟踪结果;
或者
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于所述预设坡度阈值、且所述库位与所述车辆的后轴中心的距离小于等于预设距离阈值,则将所述库位角点的视觉跟踪结果作为实际跟踪结果。
优选的,所述方法还包括:
如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于所述预设坡度阈值,使用所述库位角点的视觉跟踪结果修正所述库位角点的轮速跟踪结果。
优选的,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:
所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于预设坡度阈值、且所述库位与所述车辆的后轴中心的距离大于预设距离阈值,对所述库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。
一种库位跟踪装置,所述装置包括:
图像获取模块,用于获取车辆在不同时间下的全景鸟瞰图;
跟踪区域确定模块,用于针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积;
库位跟踪模块,用于基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域;获取所述库位角点的轮速跟踪结果,并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
优选的,所述库位跟踪模块确定目标图像区域的过程,包括:
按照上层到下层的次序依次将各层图像区域作为候选图像区域;分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域;分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布;针对所述第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;统计各区域对的移动趋势,所述移动趋势包括移动方向和移动距离;判断统计结果中是否存在主体移动趋势,所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;若是,则将所述候选图像区域作为目标图像区域;若否,则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
优选的,所述库位跟踪模块基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,包括:
确定各区域对中移动趋势属于所述主体移动趋势的目标区域对;将所述目标区域对的移动方向作为所述库位角点的移动方向,将所述目标区域对的移动距离的均值作为所述库位角点的移动距离;基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。
本发明提供一种库位跟踪方法及装置,采用视觉检测的方法对自动泊车过程中已经锁定好的库位进行实时定位,从而修正轮速跟踪所带来的累计误差问题,大幅度提高一次泊车入位、以及居中停靠的概率,提高用户体验。
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。
图1为本发明实施例提供的库位跟踪方法的方法流程图;
图2为本发明实施例提供的全景鸟瞰图示例;
图3为本发明实施例提供的光流跟踪示意图;
图4为本发明实施例提供的子区域对匹配示意图;
图5为本发明实施例提供的泊车场景示意图;
图6为本发明实施例提供的库位跟踪装置的结构示意图。
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。
为使本发明的上述目的、特征和优点能够更加明显易懂,下面结合附图和具体实施方式对本发明作进一步详细的说明。
现阶段,自动泊车功能开启,一旦库位被锁定,路径规划完成,车辆就开始了自动泊车过程,车辆中配置有精确的轮速计来计算车辆行驶的速度、加速度,然后计算单位时间下行驶的距离,并以此来定位运动中的车辆与库位的相对距离。但是轮速计的工作是不依赖其他信息的,在地面存在坡度或路况不好等恶劣场景下就会发生偏差,使得泊车入位发生偏移甚至是撞车的危险。
最常见的解决方法是泊车进入到车位一定位置后,再次重新探测停车位并锁定新车位,再进行重新规划,虽然最终成功几率很大,但是通常需要重新规划并调整车辆姿态3次以上才能够泊正车辆,这不仅浪费大量时间,而且当路况复杂时,会带来了极大的不方便,甚至路况都没有重新规划调整车辆姿态的空间。
为解决以上问题,本发明采用视觉定位跟踪库位的方案修正轮速计的轮速跟踪结果,参见图1所示的方法流程图,本发明实施例提供一种库位跟踪方法,该方法包括如下步骤:
S10,获取车辆在不同时间下的全景鸟瞰图。
本发明实施例中,车辆中配置有车载360度全景摄像头,基于该车载360度全景摄像头能够获得车辆所在环境的影像,进而对该影像进行场景识别,获得车辆所在环境的信息,包括路况、障碍物、以及地面标识。
进一步,本发明在获得可以通过车辆坐标系到全景鸟瞰图的图像坐标系的转换,将车辆所在环境的影像映射到全景鸟瞰图的图像坐标系中,获得车辆在不同时间下的全景鸟瞰图。
参见图2所示的全景鸟瞰图示例。图像坐标系原点(0,0)在左上,车辆中心坐标为(500,325),其他距离信息如图,单位为像素。假设,这里每个像素代表实际的距离为2厘米,所以400像素就代表车辆左侧有800cm 的距离范围。同理,当我们在图像上检测到库位后,就可以通过1像素=2厘米的比例进行转换,以实现将库位位置从车辆坐标系转换到全景鸟瞰图的图像坐标系中。
S20,针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,跟踪区域由多层以库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积。
本发明实施例中,可以预先设置一个库位跟踪的单位区间,对每个单位区间内的第一帧全景鸟瞰图进行库位检测,以判定其画幅中是否存在库位,对第二帧以及后续的其他帧全景鸟瞰图,则以光流跟踪法进行库位角点的跟踪。当然,该单位区间可以以全景鸟瞰图的帧数为维度(比如每8帧全景鸟瞰图为一个单位区间)、还可以以全景鸟瞰图的时长为维度(比如每30S内的全景鸟瞰图为一个单位区间)进行设置,本发明实施例对此不做限定。
而在一个单位区间,某一帧全景鸟瞰图是以其前一帧全景鸟瞰图作为库位跟踪的基准的。
参见图3所示的光流跟踪示意图。每个库位都拥有四个角点,跟踪库位的本质就是对其四个库位角点的跟踪,因此本发明实施例选取前一帧全景鸟瞰图,即第二全景鸟瞰图中已经检测出来的四个库位角点的位置,然后直接使用。
将第二全景鸟瞰图中库位的四个角点作为跟踪起始点集,以每个库位角点为中心,采用金字塔的方式形成多层图像区域,将其作为该库位角点的跟踪区域。以三层图像区域为例,如图3中三层虚线框所示,以其中的库位角点为中心,按照不同像素距离取正方形范围,这样就可以得到多个不同面积的图像区域,从大到小就形成了图像金字塔,多个图像区域就形成了多层区域。
此外,对于跟踪区域,本发明实施例还可以进行自适应调节,具体可 以以车速作为调节依据,例如:
当车速≤20km/h时,跟踪区域的多层图像区域采用采用默认设定值,即共计三层,每一层大小为30*30,45*45,60*60(单位为厘米);当车速>20km/h(根据要求自动泊车功能开启车速不超过35km/h),以20作为基准,每一层图像区域的宽和高均乘以相应的车速放大系数。假设,车速为30km/h时,车速放大系数为30/20=1.5,此时跟踪区域的三层图像区域分别为45*45,68*68,90*90。
S30,基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果,目标图像区域为面积最小的、能够在第一全景鸟瞰图的相应区域内检测到库位角点的一层图像区域。
本发明实施例中,当自动泊车运行后,新的一帧全景鸟瞰图使用前一帧全景鸟瞰图的跟踪区域,将跟踪区域内的库位角点作为特征点进行光流跟踪,形成新的跟踪点集,获得该库位角点的移动轨迹,包含移动方向和移动距离。将跟踪点集所有点的轨迹进行融合处理,可以得到泊车过程中被锁定的库位相对于车辆的位置与方向变化。
其中,新的跟踪点集是指完成一次库位角点跟踪后,原来4个库位角点的位置就会发生了变化,所以新的位置就替换了原来的位置,形成了新的跟踪点集。因此,更新的是库位角点的位置,点集里依然是4个点。
需要说明的是,光流跟踪是指跟踪前后两帧图像在平移过程中目标发生的微小图像变化,并以此确定目标在后一帧上的位置。而在本发明实施例中,基于光流跟踪法跟踪库位角点的具体实现原理是,使用库位角点的跟踪区域,针对各层图像区域,确定前后两帧全景鸟瞰图在该区域中灰度差最小、且高度相似的位置作为候选位置,进一步从所有候选位置中选取灰度差最小的位置作为目标位置,该位置即为库位角点在后一帧全景鸟瞰图的位置。
具体实现过程中,目标图像区域的确定过程,包括如下步骤:
按照上层到下层的次序依次将各层图像区域作为候选图像区域;分别 将第一全景鸟瞰图的候选图像区域、以及第二全景鸟瞰图的候选图像区域划分为多个子区域;分别统计第一全景鸟瞰图的各子区域、第二全景鸟瞰图的各子区域的灰度值概率直方分布;针对第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;统计各区域对的移动趋势,移动趋势包括移动方向和移动距离;判断统计结果中是否存在主体移动趋势,主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;若是,则将候选图像区域作为目标图像区域;若否,则执行按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
参见图4所示的子区域对匹配示意图。以第二全景鸟瞰图在库位角点周围所选取的一层图像区域作为候选图像区域为例,在第一全景鸟瞰图上也存在一个大小和位置相同的区域。将第二全景鸟瞰图的候选图像区域记为A、第一全景鸟瞰图的候选图像区域记为A’,则在A区域和A’区域分别划分出多个大小相同的子区域。
首先对第一全景鸟瞰图和第二全景鸟瞰图进行图像灰度化处理,然后分别统计A’区域的各子区域和A区域的各子区域的灰度值直方分布,该灰度值直方分布可以以分布直方图的形式体现,即横坐标为0,1…255的灰度值、纵坐标为灰度值出现的概率。进而,将A’区域的各子区域的灰度值直方分布和A区域的各子区域的灰度值直方分布转换为1*256位特征向量。
针对A区域中的每一个子区域,计算其与A’区域的各子区域的相似度,具体的,使用两个特征向量进行余弦距离的计算,距离越趋于0表示两个子区域的相似度越高、越趋于1表示相似度很低。通过设置相似度阈值,选择A’区域的相似度大于相似度阈值、且相似度最大的一个子区域与该A区域的子区域构成子区域对。如图4中箭头所指示的一对子区域,即为一个子区域对。
进一步,对于在某一层图像区域所获得的子区域对,可以统计各子区 域对的移动趋势,即移动方向和移动距离,并从中寻找主体移动趋势,其中主体移动趋势可以是至少占比三分之二的子区域对移动趋势都相同,即移动方向相同、移动距离均属于某一范围内。
当然,如果某一层图像区域没有获得子区域对/没有找到主体移动趋势则表示A’区域中不存在库位角点,此时再选择面积较大的一层图像区域作为候选图像区域,并再次执行上述步骤,直到遍历完跟踪区域全部层的图像区域为止,如果在面积最大的一层图像区域也没有获得子区域对/没有找到主体移动趋势,则表示视觉跟踪失败。
在此基础上,步骤S30中“基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果”可以采用如下步骤:
确定各区域对中移动趋势属于主体移动趋势的目标区域对;将目标区域对的移动方向作为库位角点的移动方向,将目标区域对的移动距离的均值作为库位角点的移动距离;基于库位角点的移动方向和移动距离确定库位角点的移动轨迹。
本发明实施例中,可以删除移动趋势不属于主体移动趋势的区域对,删除完成后,所有保留的目标区域对进行移动距离的均值计算,所得结果即为库位角点的移动距离。此外,目标区域对的移动方向即为库位角点的移动方向。由此,即可获得库位在连续全景鸟瞰图中的位置变化,从而得到最新位置。
S40,获取库位角点的轮速跟踪结果,并通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
本发明实施例中,轮速跟踪是在泊车过程中一直进行的工作,因此在进行库位跟踪时可以同时获得库位角点的轮速跟踪结果和视觉跟踪结果。
对于轮速跟踪,由于轮速计都存在相应的误差,运动一定距离会便会造成误差累计,而轮速计不会自动修正;对于视觉跟踪,其有可能受到库位超出画面或者被遮挡的影响,造成结果不准确。
对此,本发明实施例可以结合视觉跟踪结果和泊车场景切换轮速跟踪和视觉跟踪,保证泊车连续性。此外,还可以使用双校验的方式保证跟踪的精确性。具体的:
1)如果库位角点的视觉跟踪结果表征库位跟踪失败,则将库位角点的轮速跟踪结果作为实际跟踪结果。
视觉跟踪可能会受到超出画面或被遮挡的影响,在库位超出画幅、视觉检测无法检测到库位,或者库位未超出画幅、但被遮挡导致至少2个库位角点没有被跟踪到,或者由于其他环境原因视觉跟踪无输出,此时考虑到泊车连续性,采用轮速跟踪结果作为实际跟踪结果。
2)如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值,则将库位角点的视觉跟踪结果作为实际跟踪结果。
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度大于3°,则由于此时轮速跟踪的误差非常大,因此采用视觉跟踪结果作为实际跟踪结果。
进一步,使用库位角点的视觉跟踪结果修正库位角点的轮速跟踪结果。通过对比轮速跟踪结果和视觉跟踪结果,如果检测到轮速跟踪连续多帧的误差都超过某一阈值(大量测试后的统计学结果),则此时即可确定轮速跟踪出现问题,因此采用视觉跟踪结果对轮速跟踪结果进行位置修正,以消除轮速跟踪累计的误差,使得泊车过程中可以随时调整位姿,保证行车轨迹能够得到实时修正,从而提高一次泊车完成的成功率和准确率。
3)如果库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离小于等于预设距离阈值,则将库位角点的视觉跟踪结果作为实际跟踪结果。
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度小于等于3°、库位与车辆后轴中心的距离小于等于3米,此时视觉效果最好,因此采用视觉跟踪结果作为实际跟踪结果。
需要说明的是,库位距离车辆后轴中心的距离即为库位距离车辆近端的线段中点到车辆后轴中心的直线距离。
4)库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离大于预设距离阈值,对库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。
视觉跟踪成功输出至少三个库位角点的移动轨迹、且路面坡度小于等于3°、库位与车辆后轴中心的距离大于3米,此时采用双校验结果,即确定视觉跟踪结果的可靠度比例系数α、以及轮速跟踪结果的可靠度比例系数1-α。α计算方法如下:
参见图5所示的泊车场景示意图。当前时间下的第一全景鸟瞰图中,库位与车辆后轴中心的距离即库位距离车近端的线段中点A到后轴中心O直线距离,长度为D1,此时D1>3米,取其差值D1’=(D1-3);沿OA向做远离车辆方向的延长线,交第一全景鸟瞰图画幅边缘的交点为P,读取AP的长度为D2,则α=1.0–D1’/D2。
因此,采用双校验结果时,实际跟踪结果=视觉跟踪结果*α+轮速跟踪结果*(1-α)。
本发明的视觉跟踪仅针对被锁定库位周围一定范围内进行处理,并且采取特征点跟踪方式,可以有效进行旋转、颠簸、形变等各种工况下的跟踪,稳定且高效。从而可以对自动泊车中被锁定的库位进行实时跟踪,并可以对轮速跟踪的累积误差进行修正,从而实现自动精确泊车入位。
由于在整个泊车过程中,库位可能会超出图像画面或被遮挡,当跟踪点集的跟踪置信度降低时,跟踪方式将切换到轮速跟踪。当库位重新回到画面中或者距离车越来越近,跟踪方式从轮速切换到视觉跟踪,并进行双校验,从而实现无缝衔接。
此外,由于视觉跟踪是包含图像信息的,所以一段时间内对轮速跟踪进行一次轨迹修正,从而保持修正后的轮速信息误差最小,不会因为误差 累积而导致出现轨迹偏离。
综上,本发明是一种可以在自动泊车过程中实时跟踪并修正泊车路线的方法。能够有效提高泊车精度及成功率。本发明结构清晰、方法简洁、实时性好、鲁棒性强。
基于以上实施例提供的库位跟踪方法,本发明实施例则提供一种执行上述库位跟踪方法的装置,该装置的结构示意图如图6所示,包括:
图像获取模块10,用于获取车辆在不同时间下的全景鸟瞰图;
跟踪区域确定模块20,用于针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,跟踪区域由多层以库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积;
库位跟踪模块30,用于基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果,目标图像区域为面积最小的、能够在第一全景鸟瞰图的相应区域内检测到库位角点的一层图像区域;获取库位角点的轮速跟踪结果,并通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
可选的,库位跟踪模块30确定目标图像区域的过程,包括:
按照上层到下层的次序依次将各层图像区域作为候选图像区域;分别将第一全景鸟瞰图的候选图像区域、以及第二全景鸟瞰图的候选图像区域划分为多个子区域;分别统计第一全景鸟瞰图的各子区域、第二全景鸟瞰图的各子区域的灰度值概率直方分布;针对第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;统计各区域对的移动趋势,移动趋势包括移动方向和移动距离;判断统计结果中是否存在主体移动趋势,主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;若是,则将候选图像区域作为目标图像 区域;若否,则执行按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
可选的,库位跟踪模块30基于光流跟踪法在第一全景鸟瞰图的目标图像区域内跟踪库位角点得到库位角点的视觉跟踪结果,包括:
确定各区域对中移动趋势属于主体移动趋势的目标区域对;将目标区域对的移动方向作为库位角点的移动方向,将目标区域对的移动距离的均值作为库位角点的移动距离;基于库位角点的移动方向和移动距离确定库位角点的移动轨迹。
可选的,库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:
如果库位角点的视觉跟踪结果表征库位跟踪失败,则将库位角点的轮速跟踪结果作为实际跟踪结果。
可选的,库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:
如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值,则将库位角点的视觉跟踪结果作为实际跟踪结果;
或者
如果库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离小于等于预设距离阈值,则将库位角点的视觉跟踪结果作为实际跟踪结果。
可选的,库位跟踪模块30还用于:
如果库位角点的视觉跟踪结果表征库位跟踪成功、且车辆所在路面的坡度大于预设坡度阈值,使用库位角点的视觉跟踪结果修正库位角点的轮速跟踪结果。
可选的,库位跟踪模块30通过处理库位角点的视觉跟踪结果和轮速跟 踪结果得到实际跟踪结果,包括:
库位角点的视觉跟踪结果表征库位跟踪成功、车辆所在路面的坡度小于等于预设坡度阈值、且库位与车辆的后轴中心的距离大于预设距离阈值,对库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。
本发明实施例提供的库位跟踪装置,采用视觉检测的方法对自动泊车过程中已经锁定好的库位进行实时定位,从而修正轮速跟踪所带来的累计误差问题,大幅度提高一次泊车入位、以及居中停靠的概率,提高用户体验。
以上对本发明所提供的一种库位跟踪方法及装置进行了详细介绍,本文中应用了具体个例对本发明的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本发明的方法及其核心思想;同时,对于本领域的一般技术人员,依据本发明的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本发明的限制。
需要说明的是,本说明书中的各个实施例均采用递进的方式描述,每个实施例重点说明的都是与其他实施例的不同之处,各个实施例之间相同相似的部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。
还需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备所固有的要素,或者是还包括为这些过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
对所公开的实施例的上述说明,使本领域专业技术人员能够实现或使 用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的,本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下,在其它实施例中实现。因此,本发明将不会被限制于本文所示的这些实施例,而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。
Claims (10)
- 一种库位跟踪方法,其特征在于,所述方法包括:获取车辆在不同时间下的全景鸟瞰图;针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积;基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域;获取所述库位角点的轮速跟踪结果,并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
- 根据权利要求1所述的方法,其特征在于,所述目标图像区域的确定过程,包括:按照上层到下层的次序依次将各层图像区域作为候选图像区域;分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的所述候选图像区域划分为多个子区域;分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布;针对所述第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;统计各区域对的移动趋势,所述移动趋势包括移动方向和移动距离;判断统计结果中是否存在主体移动趋势,所述主体移动趋势为移动方 向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;若是,则将所述候选图像区域作为目标图像区域;若否,则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
- 根据权利要求2所述的方法,其特征在于,所述基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,包括:确定各区域对中移动趋势属于所述主体移动趋势的目标区域对;将所述目标区域对的移动方向作为所述库位角点的移动方向,将所述目标区域对的移动距离的均值作为所述库位角点的移动距离;基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。
- 根据权利要求1所述的方法,其特征在于,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:如果所述库位角点的视觉跟踪结果表征库位跟踪失败,则将所述库位角点的轮速跟踪结果作为实际跟踪结果。
- 根据权利要求1所述的方法,其特征在于,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于预设坡度阈值,则将所述库位角点的视觉跟踪结果作为实际跟踪结果;或者如果所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于所述预设坡度阈值、且所述库位与所述车辆的后轴中心的距离小于等于预设距离阈值,则将所述库位角点的视觉跟踪结果作为 实际跟踪结果。
- 根据权利要求5所述的方法,其特征在于,所述方法还包括:如果所述库位角点的视觉跟踪结果表征库位跟踪成功、且所述车辆所在路面的坡度大于所述预设坡度阈值,使用所述库位角点的视觉跟踪结果修正所述库位角点的轮速跟踪结果。
- 根据权利要求1所述的方法,其特征在于,所述通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果,包括:所述库位角点的视觉跟踪结果表征库位跟踪成功、所述车辆所在路面的坡度小于等于预设坡度阈值、且所述库位与所述车辆的后轴中心的距离大于预设距离阈值,对所述库位角点的视觉跟踪结果和轮速跟踪结果进行双校验得到实际跟踪结果。
- 一种库位跟踪装置,其特征在于,所述装置包括:图像获取模块,用于获取车辆在不同时间下的全景鸟瞰图;跟踪区域确定模块,用于针对当前时间下的第一全景鸟瞰图,确定距离当前时间最近的第二全景鸟瞰图中库位角点的跟踪区域,所述跟踪区域由多层以所述库位角点为中心的图像区域构成、且相邻两层图像区域中上层图像区域的面积小于下层图像区域的面积;库位跟踪模块,用于基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,所述目标图像区域为面积最小的、能够在所述第一全景鸟瞰图的相应区域内检测到所述库位角点的一层图像区域;获取所述库位角点的轮速跟踪结果,并通过处理所述库位角点的视觉跟踪结果和轮速跟踪结果得到实际跟踪结果。
- 根据权利要求8所述的装置,其特征在于,所述库位跟踪模块确定目标图像区域的过程,包括:按照上层到下层的次序依次将各层图像区域作为候选图像区域;分别将所述第一全景鸟瞰图的所述候选图像区域、以及所述第二全景鸟瞰图的 所述候选图像区域划分为多个子区域;分别统计所述第一全景鸟瞰图的各子区域、所述第二全景鸟瞰图的各子区域的灰度值概率直方分布;针对所述第二全景鸟瞰图的每个子区域,基于灰度值概率直方分布计算其与所述第一全景鸟瞰图的各子区域间相似度,并选取相似度符合预设匹配条件的一个子区域构成区域对;统计各区域对的移动趋势,所述移动趋势包括移动方向和移动距离;判断统计结果中是否存在主体移动趋势,所述主体移动趋势为移动方向相同、移动距离属于同一范围的占比大于预设阈值的移动趋势;若是,则将所述候选图像区域作为目标图像区域;若否,则执行所述按照上层到下层的次序依次将各层图像区域作为候选图像区域,直到遍历完所有图像区域时结束。
- 根据权利要求9所述的装置,其特征在于,所述库位跟踪模块基于光流跟踪法在所述第一全景鸟瞰图的目标图像区域内跟踪所述库位角点得到所述库位角点的视觉跟踪结果,包括:确定各区域对中移动趋势属于所述主体移动趋势的目标区域对;将所述目标区域对的移动方向作为所述库位角点的移动方向,将所述目标区域对的移动距离的均值作为所述库位角点的移动距离;基于所述库位角点的移动方向和移动距离确定所述库位角点的移动轨迹。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011212958.9A CN112037265B (zh) | 2020-11-04 | 2020-11-04 | 一种库位跟踪方法及装置 |
CN202011212958.9 | 2020-11-04 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022096027A1 true WO2022096027A1 (zh) | 2022-05-12 |
Family
ID=73573154
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2021/139527 WO2022096027A1 (zh) | 2020-11-04 | 2021-12-20 | 一种库位跟踪方法及装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112037265B (zh) |
WO (1) | WO2022096027A1 (zh) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112037265B (zh) * | 2020-11-04 | 2021-02-02 | 天津天瞳威势电子科技有限公司 | 一种库位跟踪方法及装置 |
CN112356831B (zh) * | 2021-01-12 | 2021-04-20 | 天津天瞳威势电子科技有限公司 | 一种库位跟踪方法及库位跟踪*** |
CN113066306B (zh) * | 2021-03-23 | 2022-07-08 | 超级视线科技有限公司 | 一种路侧停车的管理方法及装置 |
CN113963034A (zh) * | 2021-10-22 | 2022-01-21 | 长春一汽富晟集团有限公司 | 一种基于视觉的多车位目标跟踪方法 |
CN115601271B (zh) * | 2022-11-29 | 2023-03-24 | 上海仙工智能科技有限公司 | 一种视觉信息防抖方法、仓储库位状态管理方法及*** |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105374049A (zh) * | 2014-09-01 | 2016-03-02 | 浙江宇视科技有限公司 | 一种基于稀疏光流法的多角点跟踪方法及装置 |
CN109443348A (zh) * | 2018-09-25 | 2019-03-08 | 同济大学 | 一种基于环视视觉和惯导融合的地下车库库位跟踪方法 |
US20200042805A1 (en) * | 2017-09-19 | 2020-02-06 | Jvckenwood Corporation | Display control device, display control system, display control method, and non-transitory storage medium |
CN111016918A (zh) * | 2018-10-10 | 2020-04-17 | 上海汽车集团股份有限公司 | 一种库位检测方法、装置和模型训练装置 |
CN111508260A (zh) * | 2019-01-30 | 2020-08-07 | 上海欧菲智能车联科技有限公司 | 车辆停车位检测方法、装置和*** |
CN112037265A (zh) * | 2020-11-04 | 2020-12-04 | 天津天瞳威势电子科技有限公司 | 一种库位跟踪方法及装置 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104881645B (zh) * | 2015-05-26 | 2018-09-14 | 南京通用电器有限公司 | 基于特征点互信息量和光流法的车辆前方目标的检测方法 |
CN105825525A (zh) * | 2016-03-16 | 2016-08-03 | 中山大学 | 一种基于Mean-shift模型优化的TLD目标跟踪方法及其装置 |
CN106004515B (zh) * | 2016-05-12 | 2018-04-10 | 广州橙行智动汽车科技有限公司 | 用于电动汽车自动泊车的车速控制方法及*** |
CN109697860A (zh) * | 2017-10-20 | 2019-04-30 | 上海欧菲智能车联科技有限公司 | 车位检测和跟踪***及方法及车辆 |
CN108764216A (zh) * | 2018-07-11 | 2018-11-06 | 天津天瞳威势电子科技有限公司 | 一种基于视觉的交通信号灯识别方法及装置 |
-
2020
- 2020-11-04 CN CN202011212958.9A patent/CN112037265B/zh active Active
-
2021
- 2021-12-20 WO PCT/CN2021/139527 patent/WO2022096027A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105374049A (zh) * | 2014-09-01 | 2016-03-02 | 浙江宇视科技有限公司 | 一种基于稀疏光流法的多角点跟踪方法及装置 |
US20200042805A1 (en) * | 2017-09-19 | 2020-02-06 | Jvckenwood Corporation | Display control device, display control system, display control method, and non-transitory storage medium |
CN109443348A (zh) * | 2018-09-25 | 2019-03-08 | 同济大学 | 一种基于环视视觉和惯导融合的地下车库库位跟踪方法 |
CN111016918A (zh) * | 2018-10-10 | 2020-04-17 | 上海汽车集团股份有限公司 | 一种库位检测方法、装置和模型训练装置 |
CN111508260A (zh) * | 2019-01-30 | 2020-08-07 | 上海欧菲智能车联科技有限公司 | 车辆停车位检测方法、装置和*** |
CN112037265A (zh) * | 2020-11-04 | 2020-12-04 | 天津天瞳威势电子科技有限公司 | 一种库位跟踪方法及装置 |
Non-Patent Citations (1)
Title |
---|
WANG, PENGFEI: " Research on Parking Slot Detection Technology Based on Panoramic Around View System", SCIENCE & ENGINEERING (B), CHINA MASTER’S THESES FULL-TEXT DATABASE, 1 December 2018 (2018-12-01), pages 1 - 80, XP055929074, [retrieved on 20220608] * |
Also Published As
Publication number | Publication date |
---|---|
CN112037265A (zh) | 2020-12-04 |
CN112037265B (zh) | 2021-02-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022096027A1 (zh) | 一种库位跟踪方法及装置 | |
US11240471B2 (en) | Road vertical contour detection | |
US9256791B2 (en) | Road vertical contour detection | |
US8259174B2 (en) | Camera auto-calibration by horizon estimation | |
JP4919036B2 (ja) | 移動物体認識装置 | |
WO2019116958A1 (ja) | 車載環境認識装置 | |
JP5966747B2 (ja) | 車両走行制御装置及びその方法 | |
US7899211B2 (en) | Object detecting system and object detecting method | |
WO2018225446A1 (ja) | 地図変化点検出装置 | |
US8594378B2 (en) | 3D object detecting apparatus and 3D object detecting method | |
US11403767B2 (en) | Method and apparatus for detecting a trailer, tow-ball, and coupler for trailer hitch assistance and jackknife prevention | |
JP2002197444A (ja) | 車両の走行路認識装置 | |
LU502288B1 (en) | Method and system for detecting position relation between vehicle and lane line, and storage medium | |
US8730325B2 (en) | Traveling lane detector | |
Lopez et al. | Detection of lane markings based on ridgeness and RANSAC | |
CN115578470B (zh) | 一种单目视觉定位方法、装置、存储介质和电子设备 | |
Rasmussen | RoadCompass: following rural roads with vision+ ladar using vanishing point tracking | |
JP2005170290A (ja) | 障害物検出装置 | |
CN114141055B (zh) | 一种智能泊车***的泊车位检测装置和检测方法 | |
WO2022133986A1 (en) | Accuracy estimation method and system | |
Kobayashi et al. | Wide Angle Multi-Shift Stereo Camera with Monocular Vision | |
JP3951734B2 (ja) | 車両用外界認識装置 | |
CN114998849B (zh) | 一种基于路端单目相机的交通流要素感知与定位方法及其应用 | |
CN114274948A (zh) | 一种基于360度全景的自动泊车方法及装置 | |
JP2000090240A (ja) | 車両検出方法及び車両検出装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21888719 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.10.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 21888719 Country of ref document: EP Kind code of ref document: A1 |