WO2022193448A1 - Positioning method and apparatus, electronic device, and storage medium - Google Patents

Positioning method and apparatus, electronic device, and storage medium Download PDF

Info

Publication number
WO2022193448A1
WO2022193448A1 PCT/CN2021/098697 CN2021098697W WO2022193448A1 WO 2022193448 A1 WO2022193448 A1 WO 2022193448A1 CN 2021098697 W CN2021098697 W CN 2021098697W WO 2022193448 A1 WO2022193448 A1 WO 2022193448A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature point
pavement
road surface
point set
pavement element
Prior art date
Application number
PCT/CN2021/098697
Other languages
French (fr)
Chinese (zh)
Inventor
唐庆
王潇峰
刘余钱
Original Assignee
上海商汤临港智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤临港智能科技有限公司 filed Critical 上海商汤临港智能科技有限公司
Priority to KR1020227021876A priority Critical patent/KR20220131378A/en
Publication of WO2022193448A1 publication Critical patent/WO2022193448A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually

Definitions

  • the present disclosure relates to the field of computer vision technology, and in particular, to a positioning method, an apparatus, an electronic device, and a storage medium.
  • a homography matrix is required. Based on the pixel coordinates of the captured target object in the pixel coordinate system, and the homography matrix, the target object in the world coordinate system can be obtained. in world coordinates. Therefore, the accuracy of the homography matrix directly affects the accuracy of the positioning results.
  • the embodiments of the present disclosure provide at least a positioning method, an apparatus, an electronic device, and a storage medium.
  • an embodiment of the present disclosure provides a positioning method, including: acquiring a current road surface image collected by a target vehicle and a prefabricated map of the current location of the target vehicle; The first world coordinate of the road surface element; based on the prefabricated map, the second world coordinate of the road surface element is determined; according to the first world coordinate and the second world coordinate of the road surface element, the target collected by the target vehicle is determined. The target world coordinate of the target object in the image.
  • the historical positioning parameters used for positioning in the target vehicle are no longer accurate due to mechanical vibrations or uneven driving surfaces.
  • the historical homography matrix of the target vehicle may not be accurate. No matter how accurate it is, there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, so as to improve the accuracy of the positioning parameters, thereby improving the positioning of the target object. the accuracy of the results.
  • the positioning method before acquiring the current road image collected by the target vehicle and the pre-made map of the current location of the target vehicle, the positioning method further includes: responding to a triggering operation of a calibration button for the target vehicle, or receiving to the calibration command to calibrate the target vehicle.
  • the target vehicle after detecting that there is a trigger operation of the calibration button for the target vehicle, or after receiving a calibration instruction for calibrating the target vehicle, the target vehicle can be calibrated first, so as to improve the accuracy of the target vehicle for positioning.
  • Current localization parameters such as improving the homography matrix of the target vehicle.
  • the determining, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image includes: acquiring pixel coordinates of the road surface element in the current road surface image; A first world coordinate of the road surface element is determined based on the pixel coordinates of the road surface element and a predetermined historical homography matrix of the acquisition device.
  • the homography matrix can reflect the transformation relationship between the image coordinate system corresponding to the road surface image and the world coordinate system corresponding to the target vehicle.
  • the history homography matrix of quickly determines the first world coordinates of pavement elements in the world coordinate system.
  • the obtaining the pixel coordinates of the road surface element in the current road surface image includes: performing semantic segmentation on the current road surface image to obtain at least one road surface element included in the current road surface image , and the pavement element category of each pavement element; according to the pavement element category of each pavement element, extract the pixel coordinates of the first feature point set contained in the pavement element from the current pavement image; The pixel coordinates of the set are taken as the pixel coordinates of the road surface elements corresponding to the first feature point set.
  • the pavement elements included in the pavement image can be quickly extracted according to the semantic segmentation method, and the first feature points included in the pavement elements in the pavement image can be quickly extracted according to the preset pavement element categories Set the pixel coordinates to get the pixel coordinates of the pavement element.
  • each pavement element extracting the pixel coordinates of the first feature point set included in the pavement element from the current pavement image, including: in the pavement element being In the case of the first type of pavement element, extract a plurality of first feature points on the contour of the pavement element to obtain the first feature point set of the pavement element; in the case of the pavement element of the second type of pavement element, in the The first feature point is extracted from the center line of the pavement element according to the set distance interval, and the first feature point set of the pavement element is obtained; in the pixel coordinate system corresponding to the current pavement image, the first feature point set is obtained.
  • the pixel coordinates of each first feature point of is
  • the feature point extraction can be performed according to the specific pavement element category of the pavement element, so as to quickly obtain the first feature point set included in each pavement element, and the third The pixel coordinates of each first feature point in a feature point set.
  • the determining the second world coordinates of the pavement element based on a prefabricated map includes: acquiring a pre-stored map in which the second feature point set of the pavement element is contained in the prefabricated map location information; based on the map location information of the second feature point set of the pavement element and the pose data of the target vehicle in the road scene represented by the prefabricated map, determine the location of the second feature point set of the pavement element World coordinates; take the world coordinates of the second feature point set as the second world coordinates of the road surface element corresponding to the second feature point set.
  • the location of the second feature point set included in the pavement element can be quickly determined. world coordinates to get the second world coordinates of the pavement element.
  • the determining, according to the first world coordinates and the second world coordinates of the road surface element, the target world coordinates of the target object in the target image collected by the target vehicle includes: based on the The first world coordinate and the second world coordinate of the road surface element, the historical homography matrix is adjusted to obtain the current homography matrix of the target vehicle; based on the target image, it is determined that the target object is in the pixel coordinates in the target image; based on the pixel coordinates of the target object and the current homography matrix, determine the target world coordinates of the target object.
  • the historical homography matrix of the target vehicle can be corrected according to the difference between the first world coordinate and the second world coordinate of the road surface element, so as to obtain the current homography matrix with high accuracy, and further based on The current homography matrix with high accuracy and the pixel coordinates of the target object can obtain the target world coordinates with high accuracy for determining the target object.
  • the positioning method further includes: determining the target based on the target world coordinates of the target object and the position information of a preset position point The distance between the object and the preset position point.
  • the current homography matrix can be used to accurately determine the target world coordinates of the target object, and then the distance between the prefabricated position point and the target object can be obtained with high accuracy .
  • the historical homography matrix is adjusted based on the first world coordinate and the second world coordinate of the road surface element to obtain the current homography matrix of the target vehicle
  • the method includes: determining the first feature point set and the second feature point of the same pavement element based on the world coordinates of the first feature point set of the pavement element and the world coordinates of the second feature point set of the pavement element
  • the target affine matrix between sets; based on the target affine matrix, the historical homography matrix is modified to obtain the current homography matrix of the target vehicle.
  • a target affine matrix for representing the gap between the historical homography matrix and the current homography matrix can be obtained.
  • the historical homography matrix can be modified based on the target affine matrix, so as to obtain the current homography matrix with higher accuracy.
  • the pavement element includes a plurality of elements
  • the pavement element is determined based on the world coordinates of the first feature point set of the pavement element and the second feature point set of the pavement element
  • the target affine matrix between the first feature point set and the second feature point set includes: according to the current affine matrix, for the world of the second feature point set of each road surface element in the plurality of road surface elements The coordinates are updated, and the updated world coordinates of the second feature point set of the road surface element are determined;
  • the current affine matrix is an initial preset affine matrix or an affine matrix that has been adjusted at least once for the initial preset affine matrix Affine matrix; based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated world coordinates of the second feature point set of the pavement element and the first feature point set of the pavement element.
  • a residual value between the world coordinates of a feature point set based on the residual values of the plurality of pavement elements, after adjusting the current affine matrix, return to determine the update of the second feature point set of the pavement element After the steps of the world coordinates, the target affine matrix is obtained until the preset cut-off condition is met.
  • the affine matrix may be adjusted multiple times until a target affine matrix with higher accuracy is obtained, so that the current single target affine matrix with higher accuracy can be obtained subsequently based on the target affine matrix with higher accuracy. Responsiveness Matrix.
  • the pavement element includes a first type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, The positioning method further includes: for each first-type pavement element, extracting each second feature point from the first feature point set of the first-type pavement element and the second feature point set of the first-type pavement element The matched first feature point; the updated second feature point set of the pavement element is determined based on the updated world coordinates of the second feature point set of the pavement element and the world coordinate of the first feature point set of the pavement element.
  • the residual value between the world coordinates of and the world coordinates of the first feature point set including: for each first type of pavement element, based on each second feature point in the second feature point set of the first type of pavement element The updated world coordinates and the world coordinates of the first feature point matched with the second feature point, determine the distance between the second feature point and the matched first feature point; The distances between the second feature points and the matched first feature points are summed to obtain the residual value corresponding to the first type of road surface element.
  • the distance of each second feature point included in each first type of pavement element and the matched first feature point in the world coordinate system can be used to determine the passing history
  • the pavement element further includes a second type of pavement element, before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element , the positioning method further includes: for each second-type pavement element, according to the world coordinates of the first feature point set of the second-type pavement element, determining the fitting of the first feature point set of the second-type pavement element straight line equation; the updated world coordinates of the second feature point set of the pavement element and the The residual value between the world coordinates of the first feature point set, including: for each second-type road surface element, the updated world based on each second feature point in the second feature point set of the second-type road surface element Coordinates and the fitting line equation corresponding to the second type of pavement element, determine the distance between the second feature point and the fitted straight line corresponding to the second type of pavement element; The distance between the two feature points and the fitting straight line corresponding to the second type of road surface element determines the residual value corresponding to the second type of road surface element.
  • the distance between each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element can be determined.
  • the residual value between the first world coordinate of the second type of pavement element determined by the historical homography matrix and the second world coordinate of the second type of pavement element, so that the target simulation can be determined by adjusting the residual value subsequently. shot matrix.
  • the positioning method before determining the residual value corresponding to the second type of pavement element, further includes: based on the world coordinates of each second feature point included in the second type of pavement element, Obtain the distance weight corresponding to each second feature point of the second type of pavement element; the distance weight based on each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element and determining the residual value corresponding to the second type of pavement element, including: based on the distance weight corresponding to each second feature point included in the second type of pavement element, each The distance between the two feature points and the fitted straight line corresponding to the second type of road surface element is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
  • the pavement element when the pavement element includes both the first type of pavement element and the second type of pavement element, considering that the area far from the target vehicle contains the first type of pavement element, the number of effective feature points is small, so it can be Adjust the distance weights corresponding to the second feature points of the second type of pavement elements at different positions of the distance collection device to balance the problem of unbalanced effective feature points contained in the far and near areas of the target vehicle, so that the obtained current homography matrix is measured and The conversion relationship between the target vehicle and the road surface in different regions and the corresponding images has high robustness and accuracy.
  • an embodiment of the present disclosure provides a positioning device, including: an acquisition module for acquiring a current road image collected by a target vehicle and a pre-made map of the current location of the target vehicle; a first determination module for acquiring the current road surface image, and determine the first world coordinates of the road surface elements in the current road surface image; the second determination module is used to determine the second world coordinates of the road surface elements based on the prefabricated map; the third determination module, It is used for determining the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements.
  • embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing The processor and the memory communicate through a bus, and the machine-readable instructions execute the steps of the positioning method according to the first aspect when the machine-readable instructions are executed by the processor.
  • an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the positioning method according to the first aspect are executed .
  • the present disclosure provides a computer program comprising computer readable code, when the computer readable code is executed in an electronic device, a processor in the electronic device performs the positioning according to the first aspect steps of the method.
  • FIG. 1 shows a flowchart of a positioning method provided by an embodiment of the present disclosure
  • FIG. 2 shows a flowchart of a method for determining a first world coordinate of a road surface element provided by an embodiment of the present disclosure
  • FIG. 3 shows a flowchart of a method for determining the target world coordinates of a target object provided by an embodiment of the present disclosure
  • FIG. 4 shows a flowchart of a method for determining a current homography matrix of a target vehicle provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram showing the projected position of the same pavement element in the world coordinate system obtained by different methods provided by an embodiment of the present disclosure
  • FIG. 6 shows a flowchart of a method for determining a target affine matrix provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic diagram of a positioning device provided by an embodiment of the present disclosure.
  • FIG. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
  • visual positioning can be carried out by relying on images collected by acquisition equipment, such as cameras or monocular cameras.
  • acquisition equipment such as cameras or monocular cameras.
  • the principle of visual positioning relying on acquisition equipment is to determine that the target object captured by the acquisition equipment is in the pixel coordinate system. Then, based on the homography matrix of the acquisition device, the world coordinates of the target object in the world coordinate system are determined. Therefore, the accuracy of the homography matrix directly affects the accuracy of the positioning result.
  • the homography matrix of the acquisition device on the vehicle can be obtained by pre-calibrating the positional relationship between the acquisition device and the vehicle. After the acquisition device on the vehicle is calibrated, the location of the target object can be determined according to the homography matrix.
  • the relative position of the acquisition device and the vehicle may change due to the mechanical vibration caused by long-term driving or when the vehicle is driven on an uneven interior, making the historically determined homography matrix no longer applicable. , which affects the accuracy of positioning.
  • the following embodiments of the present disclosure provide a method for positioning a target object during vehicle driving.
  • the present disclosure provides a positioning method.
  • the historical positioning parameters used for positioning in the target vehicle may become inaccurate due to mechanical vibrations or uneven driving surfaces.
  • the historical homography matrix is no longer accurate, so that there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, thereby improving the accuracy of the positioning parameters.
  • the accuracy of the target object localization result can be improved.
  • the execution subject of the positioning method provided by the embodiment of the present disclosure is generally a computer device with a certain computing capability, such as a computer device.
  • the terminal equipment may be user equipment (User Equipment, UE), mobile equipment, user terminal, computing equipment, vehicle-mounted equipment, etc.
  • the positioning method may be implemented by the processor invoking computer-readable instructions stored in the memory.
  • the positioning method can be applied to a processor in a vehicle-mounted device, and the positioning method includes steps S101-S104, wherein:
  • the target vehicle may be provided with a capture device for capturing road images
  • the capture device may include a monocular camera, or a monocular camera, or a camera for capturing color images, grayscale images, or depth images.
  • the acquisition device can capture road images of the road at preset time intervals.
  • the prefabricated map may be a pre-built scene map that matches the road scene where the target vehicle is driving, and the elements in the prefabricated map and the same element in the road scene where the target vehicle is driving may be presented 1:1 in the same coordinate system.
  • the map location information of the feature points contained in each road surface element in the prefabricated map can be stored in the prefabricated map.
  • the feature points of each road surface element are stored in the prefabricated map.
  • the map location information is 1, each pavement element can store a small amount of map location information of points that can characterize the geometric shape features.
  • the road surface elements include road surface signs used to indicate the target vehicle, and may specifically include pointing signs, dotted line segments, zebra crossings, solid lines, stop lines, etc. Pointing signs and going straight signs, etc.
  • the first world coordinates of the pavement element may be represented by the world coordinates of the feature points contained in the pavement element in the world coordinate system, wherein the feature points of the pavement element may include points that can characterize the geometrical features of the pavement element, such as For pavement elements whose complete outlines can be extracted, multiple points that constitute the outlines of pavement elements can be used as feature points, such as pointing signs, each line segment in the dotted line segment, zebra crossings, etc.; for pavement elements whose complete outlines cannot be extracted, For example, the solid line lane line, the solid line stop line, etc., because the lane line and the stop line are generally straight lines, so multiple points representing the road surface element as a straight line feature can be extracted as feature points.
  • the pose data of the target vehicle in the road scene indicated by the prefabricated map can be determined based on the current road surface image, so as to determine that the road surface elements contained in the location of the target vehicle are constructed with the target vehicle as the origin.
  • the second world coordinate in the world coordinate system of the The map location information in the prefabricated map and the pose data of the target vehicle in the road scene indicated by the prefabricated map are determined, as described later.
  • the plurality of feature points contained in each road surface element extracted based on the current road surface image may be recorded as the first feature point set of the road surface element, A plurality of feature points included in the road surface element pre-stored based on the pre-made map is recorded as a second feature point set.
  • the homography matrix is first introduced.
  • the property matrix refers to the homography matrix of the acquisition device on the target vehicle.
  • the homography matrix can be used to represent the transformation relationship between the image coordinate system corresponding to the road surface image acquired by the acquisition device and the world coordinate system where the target vehicle is located.
  • the world coordinates of the first feature point set in the world coordinate system can be determined.
  • the historical homography matrix may be the initial homography matrix determined during the initial calibration process for the acquisition device, or may be the historical homography matrix obtained after the last adjustment, wherein the obtained value after the last adjustment
  • the method of determining the historical homography matrix is similar to the method of determining the current homography matrix, and the specific process is described later.
  • the initial homography matrix may be a homography matrix that is calibrated before the target vehicle travels, where the homography matrix that is calibrated before the target vehicle travels may be calibrated by a currently known calibration method, such as when stationary.
  • the reference object is placed on the road surface of the device, and the homography matrix of the acquisition device on the stationary target vehicle is determined by the world coordinates of the reference object in the world coordinate system and the pixel coordinates of the reference object in the image acquired by the acquisition device.
  • the historical homography matrix is no longer accurate, there will be a gap between the first world coordinates and the second world coordinates of the same road surface element, and the historical homography matrix can be corrected based on the gap to obtain The current homography matrix with high accuracy.
  • the target object in the collected target image can be located based on the current homography matrix in the current time period, for example, the homography matrix is performed every 5 minutes. Calibration, so that between the current calibration and the next calibration period, the target world coordinates of the target object in the target image can be determined through the current homography matrix and the pixel coordinates of the target object in the target image.
  • the target image here may be a current road image
  • the target object may refer to a preset target object that needs to be positioned, such as obstacles such as vehicles, pedestrians, trees, and railings in the target image.
  • the historical positioning parameters used for positioning in the target vehicle are no longer accurate due to mechanical vibrations or uneven driving surfaces.
  • the historical homography matrix of the target vehicle may not be accurate. No matter how accurate it is, there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, so as to improve the accuracy of the positioning parameters, thereby improving the positioning of the target object. the accuracy of the results.
  • the positioning method provided by the embodiment of the present disclosure further includes:
  • a calibration instruction for calibrating the target vehicle is received.
  • the target vehicle may be an automatic driving vehicle.
  • the target vehicle in addition to correcting the homography matrix according to a preset time interval, it is also possible to detect that there is a trigger for the calibration button on the target vehicle. After the operation, start to correct the homography matrix.
  • a trigger for the calibration button on the target vehicle After the operation, start to correct the homography matrix.
  • there is a software button or physical button on the target vehicle that triggers the target vehicle to start correcting the homography matrix.
  • the user This calibration button can be triggered.
  • the target vehicle may also be provided with a communication system that maintains a communication connection with the cloud or the server, and can receive a calibration command sent by the cloud or the server through the communication system to calibrate the target vehicle, that is, to correct the homography matrix. After that, start to acquire the current road image and prefabricated map collected by the target vehicle to complete the process of correcting the historical homography matrix.
  • the target vehicle after detecting that there is a trigger operation of the calibration button for the target vehicle, or after receiving a calibration instruction for calibrating the target vehicle, the target vehicle can be calibrated first, so as to improve the accuracy of the target vehicle for positioning.
  • Current localization parameters such as improving the homography matrix of the target vehicle.
  • target detection may be performed on the current road surface image to determine the road surface elements contained in the current road surface image, and further according to the above method, each of the first feature point sets contained in each road surface element is extracted.
  • the i-th first feature point can be determined according to the following formula (1):
  • H represents the historical homography matrix
  • the homography matrix can reflect the transformation relationship between the image coordinate system corresponding to the road surface image and the world coordinate system corresponding to the target vehicle.
  • the history homography matrix of quickly determines the first world coordinates of pavement elements in the world coordinate system.
  • S2011 performing semantic segmentation on the road surface image to obtain at least one road surface element included in the road surface image and the road surface element category of each road surface element;
  • the pavement elements can be divided into the first type of pavement elements and the second type of pavement elements, where the first type of pavement elements refers to the pavement elements that can extract contours, such as the above The mentioned pointing signs, virtual segments and zebra crossings, the second type of pavement elements refer to linear pavement elements whose contours cannot be extracted, such as the above-mentioned solid lines and stop lines.
  • the pavement elements collected by the collection device may be semantically segmented through a pre-trained semantic segmentation model, to obtain at least one pavement element included in the pavement image, and the pavement element category of each pavement element.
  • the pixel coordinates of the first feature point set included in the pavement element can be extracted from the pavement image according to the pavement element category of the pavement element.
  • the extracted pixel coordinates of the first feature point set included in each road surface element are taken as the pixel coordinates of the road surface element.
  • the pavement elements included in the pavement image can be quickly extracted according to the semantic segmentation method, and the first feature points included in the pavement elements in the pavement image can be quickly extracted according to the preset pavement element categories Set the pixel coordinates to get the pixel coordinates of the pavement element.
  • first feature points can be extracted on the contours of such pavement elements, and the connecting lines of these first feature points can constitute the pavement.
  • the contour of the element is obtained to obtain the first feature point set of the pavement element.
  • the first feature points can be extracted on the center line of such pavement elements according to a set distance interval, for example, one feature point is extracted every n pixel points.
  • the first feature point in this way, the first feature point set of the road surface element can be extracted.
  • the pixel coordinates of each first feature point in the first feature point set may be extracted in the pixel coordinate system corresponding to the pavement image element.
  • the feature point extraction can be performed according to the specific pavement element category of the pavement element, so as to quickly obtain the first feature point set included in each pavement element, and the third The pixel coordinates of each first feature point in a feature point set.
  • the prefabricated map may be constructed according to pre-collected road scene images included in the road scene where the target vehicle is traveling.
  • a three-dimensional scene map corresponding to the road scene may be constructed in advance, and the three-dimensional scene map corresponding to the road scene may be stored in advance
  • the second feature points of the pavement elements in the road set the map position information in the prefabricated map.
  • the pose data of the target vehicle in the prefabricated map can be determined after aligning the prefabricated map and the road scene. , and then based on the map position information of the second feature point set of the pavement element in the prefabricated map and the pose data of the target vehicle in the prefabricated map, determine the second feature point of the pavement element in the world coordinate system established according to the target vehicle set of world coordinates.
  • the pose data of the target vehicle in the road scene may include the position coordinates and orientation of the target vehicle in the road scene, and the pose data may be based on road images collected by the collection device and real-time positioning and map construction. Mapping, SLAM), or can also be determined based on the integrated navigation device installed on the acquisition device, which is not specifically limited here.
  • the location of the second feature point set included in the pavement element can be quickly determined. world coordinates to get the second world coordinates of the pavement element.
  • the historical homography matrix is accurate, based on the historical homography matrix of the acquisition device, the world coordinate system of the matching feature points in the first feature point set and the second feature point set of the same road surface element is obtained. The coordinates should be the same. If the matched feature points in the first feature point set and the second feature point set of the same pavement element have different world coordinates in the world coordinate system, it can be determined that the historical homography matrix is no longer accurate. At the same time, the historical homography matrix can be adjusted based on the world coordinates of the first feature point set and the second feature point set of the same road surface element in the world coordinate system to obtain the current homography matrix of the target vehicle.
  • the detection frame corresponding to the target object can be obtained based on the image recognition technology, and the center position point of the tangent between the detection frame and the ground is used as the position point that can characterize the target object in the target image, and then the detection frame is extracted.
  • the pixel coordinates of the position point are used as the pixel coordinates of the target object in the image coordinate system.
  • the historical homography matrix of the target vehicle can be corrected according to the difference between the first world coordinates and the second world coordinates of the road surface elements, so as to obtain the current homography matrix with higher accuracy, and further based on The current homography matrix with high accuracy and the pixel coordinates of the target object can obtain the target world coordinates with high accuracy for determining the target object.
  • the positioning method further includes:
  • the distance between the target object and the preset position point is determined.
  • the preset position point here can be the projection of the center point of the front axle of the target vehicle on the ground, or the projection of the center of the vehicle body of the target vehicle on the ground, which is used as the world coordinate system.
  • the origin of the origin is known
  • the coordinates of the origin in the world coordinate system are known
  • the preset position point can be used as the vehicle ranging point corresponding to the distance between the target object and the target vehicle.
  • the Euclidean distance can be calculated according to the determined world coordinates of the target object in the world coordinate system and the world coordinates of the preset position point, so as to determine the distance between the target object and the target vehicle.
  • the current homography matrix can be used to accurately determine the target world coordinates of the target object, and then the distance between the prefabricated position point and the target object can be obtained with high accuracy .
  • the projected position of the same road surface element in the world coordinate system will deviate from the position in the world coordinate system determined based on the pre-made map, as shown in Figure 5.
  • the dotted line represents the position of the first world coordinate of the pavement element in the world coordinate system determined based on the historical homography matrix
  • the solid line represents the first world coordinate of the pavement element determined based on the pre-made map in the world coordinate system.
  • the position represented by the two world coordinates shows that when the historical homography matrix is no longer accurate, there is a certain deviation between the two.
  • the embodiment of the present disclosure proposes to represent the geometric shape of the pavement element by the feature points of the pavement element. Therefore, the pavement element in the world coordinate system can be represented by the world coordinates of the feature point set contained in the pavement element in the world coordinate system. position below.
  • the two projection results of the same road surface element in the world coordinate system can be obtained by affine Transform to convert, such as L1 and L2, M1 and M2, H1 and H2, W1 and W2, and S1 and S2 respectively represent the two projection results of the same road surface element in the world coordinate system
  • the target affine matrix can represent the same road surface
  • the conversion relationship between the two projection results of the element can indicate how the position of the pavement element L1 in the world coordinate system has changed, and it can coincide with the position of the pavement element L2 in the world coordinate system.
  • the position of the feature point in the world coordinate system represents the position of the pavement element in the world coordinate system. Therefore, it is proposed here that the same pavement element can be determined based on the world coordinates of the first feature point set and the second feature point set in the world coordinate system.
  • the target affine matrix between the first feature point set and the second feature point set is proposed here.
  • the pavement elements obtained by extracting the pavement image may include multiple ones, and the world coordinates of the second feature point set of the pavement elements in the world coordinate system determined according to the prefabricated map also include multiple second feature point sets of the pavement elements. Therefore, before determining the target affine matrix, it is necessary to determine the first feature point set of each pavement element and the second feature point set of each pavement element in the world coordinate system to determine the first feature point set of the same pavement element. A feature point set and a second feature point set.
  • the nearest neighbor method can be used to determine the first feature point set and the second feature point set of the same pavement element, and for the first type of pavement element, such as a pointing sign, it can be determined according to the world coordinate of the center of gravity of each pavement element.
  • the pavement element with the closest distance to the pavement element is regarded as the same pavement element.
  • the pointing signs M1 and M2 in the above Figure 5 are the same first type of pavement element; for the second type of pavement element, such as a straight pavement element, the distance between the straight lines
  • the two nearest straight lines are regarded as the same road surface element.
  • the straight line L1 and the straight line L2 are the same second type of road surface element.
  • the historical homography matrix can be modified based on the target affine matrix. Specifically, the following formula (2) can be used to determine the current homography matrix:
  • H new represents the current homography matrix of the acquisition device
  • A represents the target affine matrix
  • H old represents the historical homography matrix
  • a target affine matrix for representing the gap between the historical homography matrix and the current homography matrix can be obtained.
  • the historical homography matrix can be modified based on the target affine matrix, so as to obtain the current homography matrix with higher accuracy.
  • the first feature point set and the second feature of the same pavement element are determined based on the world coordinates of the first feature point set of the pavement element and the second feature point set of the pavement element.
  • the target affine matrix between point sets includes the following S501-S503:
  • the affine matrix is an initial preset affine matrix or an affine matrix adjusted at least once for the initial preset affine matrix.
  • the initial preset affine matrix may be set according to experience, and further the initial preset affine matrix may be adjusted according to the methods of S501 to S503 until the target affine matrix is obtained.
  • the world coordinates of each second feature point in the second feature point set of each pavement element may be updated according to the current affine matrix. Update, and then obtain the updated world coordinates of the second feature point.
  • the residual values are determined in different ways.
  • the first type of pavement elements in which points on the contour are extracted as feature points the first type of pavement element can be determined according to the distance between two matched points.
  • the second type of pavement element that extracts the point on the center line as the feature point the residual value of the second type of pavement element can be determined according to the distance from the point to the line.
  • the process. described below please refer to the process. described below.
  • the sum of the residual values corresponding to the multiple pavement elements may be determined, and then according to the sum of the residual values, the current simulat Adjust the affine matrix, take the adjusted affine matrix as the current affine matrix, and return to S501 to restart the update of the world coordinates of the second feature point set of each pavement element in the plurality of pavement elements until the preset deadline is met After the condition, the target affine matrix is obtained.
  • the preset cutoff conditions may include one or more of the following:
  • the affine matrix may be adjusted multiple times until a target affine matrix with higher accuracy is obtained, so that the current single target affine matrix with higher accuracy can be obtained subsequently based on the target affine matrix with higher accuracy. Responsiveness Matrix.
  • the pavement elements include the first type of pavement elements, such as the above-mentioned directional signs, dashed line segments and zebra crossings that can be extracted to contour points.
  • a first feature point matching each second feature point in the second feature point set of the first-type pavement element is extracted from the first feature point set of the first-type pavement element.
  • the number of the second feature points included in the second feature point set is small, after obtaining the world coordinates of the second feature point set included in the first type of road surface elements in the world coordinate system, The number of second feature points is increased according to the up-sampling method, and then the first feature point matching each second feature point is determined in the first feature point set. ) to determine a first feature point matching each of the second feature points in the first feature point set.
  • the residual value between the world coordinates and the world coordinates of the first feature point set includes the following S5021 to S5022:
  • the distance between the second feature point and the matched first feature point may be determined by the Euclidean distance calculation formula.
  • each second feature point contained in each first type of pavement element can be summed to obtain the corresponding first type of pavement element. residual value.
  • the distance of each second feature point included in each first type of pavement element and the matched first feature point in the world coordinate system can be used to determine the passing history
  • the pavement element further includes a second type of pavement element, for example, in addition to the first type of pavement element, it also includes the above-mentioned solid line and stop line whose contour points cannot be extracted from the pavement image,
  • the positioning method provided by the embodiment of the present disclosure further includes:
  • the fitting line equation of the first feature point set of the second-type pavement element is determined.
  • the second type of pavement elements as the pavement elements for extracting feature points for the centerline as an example, such as the L and S types of pavement elements in Figure 5, such pavement elements cannot obtain the complete outline of the pavement element, but
  • the first feature point is extracted at the set interval on the center line of the pavement element to obtain the first feature point set.
  • a plurality of first feature points in the first feature point set of the second type of pavement element can be linearly fitted to obtain the second type of pavement
  • the adjacent set number of The first feature point is fitted with a straight line to obtain a plurality of fitted straight lines with different curvatures, and a fitted straight line equation corresponding to each segment of the fitted straight line.
  • the residual value between the world coordinates and the world coordinates of the first feature point set includes the following S5023 to S5024:
  • each second feature point in the second feature point set of the second type of pavement element corresponds to the second type of pavement element based on the distance formula from the point to the straight line.
  • the distance between the fitted straight lines for example, the pavement image includes n second-type pavement elements, and the second feature point set of each second-type pavement element contains m second-type feature points, so that for each second-type pavement element Pavement elements, m distances can be determined.
  • each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element can be summed to obtain the second type of pavement element.
  • the residual value corresponding to the pavement element can be summed to obtain the second type of pavement element.
  • the distance between each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element can be determined.
  • the residual value between the first world coordinate of the second type of pavement element determined by the historical homography matrix and the second world coordinate of the second type of pavement element, so that the target simulation can be determined by adjusting the residual value subsequently. shot matrix.
  • the positioning method when the road surface image contains both the first type of road surface element and the second type of road surface element, consider the first feature contained in the road surface image of the first type of road surface element in the area far from the acquisition device.
  • the number of points is small, or the first type of feature points cannot be extracted, that is, the area far away from the collection device contains fewer effective feature points.
  • the homography matrix is adjusted, in order to obtain the current homography matrix with higher accuracy, the sum of the residual values corresponding to the pavement elements (the sum of the residual values of the first type of pavement elements and the second type of pavement elements) is determined.
  • the weight of the second feature point of the second type of pavement element that is far away from the collection device can be increased. Therefore, before determining the residual value corresponding to the second type of pavement element, the positioning method provided by the embodiment of the present disclosure further includes: :
  • the distance weight corresponding to each second feature point can be determined by the following formula (3):
  • ⁇ k represents the distance weight corresponding to the k-th second feature point
  • d(P k ) is the Euclidean distance from the k-th second feature point P k of the world coordinate system to the origin of the world coordinate system
  • ⁇ represents the preset distance
  • the residual value corresponding to the second type of pavement element when determining the residual value corresponding to the second type of pavement element based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element, include:
  • the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
  • the distance weight corresponding to each second feature point can be determined according to the above formula (3), so that when determining the residual value corresponding to the second type of pavement element, each of the first pavement elements can be used.
  • the distance weight corresponding to the second feature point is weighted and summed to the distance between the second feature point and the fitting straight line corresponding to the second type of road surface element, so that the residual value is more accurate.
  • the pavement element when the pavement element includes both the first type of pavement element and the second type of pavement element, considering that the area far from the target vehicle contains the first type of pavement element, the number of effective feature points is small, so it can be Adjust the distance weights corresponding to the second feature points of the second type of pavement elements at different positions of the distance collection device to balance the problem of unbalanced effective feature points contained in the far and near areas of the target vehicle, so that the obtained current homography matrix is measured and The conversion relationship between the target vehicle and the road surface in different regions and the corresponding images has high robustness and accuracy.
  • the target affine matrix can be determined by the following formula (4):
  • E represents the sum of the residual values of multiple road surface elements
  • Q represents the total number of second-type road surface elements contained in the road surface image
  • i represents the i-th second-type road surface element
  • k represents the i-th second-type road surface element
  • P k represents the world coordinate of the kth second feature point of the ith second type of pavement element in the world coordinate system;
  • A represents the current affine matrix;
  • l i represents the ith
  • h(P k ,A) represents the result obtained by updating the world coordinate of the kth second feature point of the ith second type of pavement element based on the current affine matrix
  • D l (h(P k ,A),l i ) represents the updated world coordinates based on the kth second feature point of the ith second type road surface element and the ith second feature point
  • the world coordinate in the coordinate system; p j represents the world coordinate of the first feature point in the world coordinate system that matches the j-th second feature point of the j-th first-type road surface element; h(P j , A) represents The updated world coordinates are obtained by updating the world coordinates of the kth second feature point of the jth first type pavement element based on the current current affine matrix; D p (h(P j ,A),p j ) indicates that based on the updated world coordinates of the kth second feature point of the jth first type pavement element and the world coordinate of the matched first feature point, the kth second feature point of the jth first type pavement element is determined.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the embodiment of the present disclosure also provides a positioning device corresponding to the positioning method. Since the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned positioning method of the embodiment of the present disclosure, the implementation of the device can refer to the method implementation, and the repetition will not be repeated.
  • the positioning device 600 includes:
  • An acquisition module 601 configured to acquire a current road image collected by the target vehicle and a prefabricated map of the current location of the target vehicle;
  • a first determining module 602 configured to determine, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image;
  • the second determining module 603 is configured to determine the second world coordinates of the road surface element based on the prefabricated map
  • the third determining module 604 is configured to determine the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements.
  • the acquiring module 601 before acquiring the current road surface image collected by the target vehicle and the prefabricated map of the current location of the target vehicle, the acquiring module 601 is further configured to:
  • a calibration instruction for calibrating the target vehicle is received.
  • the method when the first determining module 602 is used to determine the first world coordinates of the road surface element in the current road surface image based on the current road surface image, the method includes:
  • the first world coordinates of the pavement elements are determined based on the pixel coordinates of the pavement elements and the predetermined historical homography matrix of the acquisition device.
  • the method when the first determining module 602 is used to acquire the pixel coordinates of the road surface elements in the current road surface image, the method includes:
  • each pavement element extracts the pixel coordinates of the first feature point set contained in the pavement element in the current pavement image
  • the pixel coordinates of the first feature point set are used as the pixel coordinates of the road surface element corresponding to the first feature point set.
  • the method includes:
  • the pavement element is the first type of pavement element, extract a plurality of first feature points on the outline of the pavement element to obtain a first feature point set of the pavement element;
  • the pavement element is the second type of pavement element, extract the first feature points on the center line of the pavement element according to the set distance interval, and obtain the first feature point set of the pavement element;
  • the pixel coordinates of each first feature point in the first feature point set are acquired.
  • the method when the second determining module 602 is used to determine the second world coordinates of the road surface element based on the prefabricated map, the method includes:
  • the world coordinate of the second feature point set is used as the second world coordinate of the road surface element corresponding to the second feature point set.
  • the method includes:
  • the target world coordinates of the target object are determined.
  • the third determining module 604 is further configured to:
  • the distance between the target object and the preset position point is determined.
  • the third determination module 604 is used to adjust the historical homography matrix based on the first world coordinates and the second world coordinates of the road surface elements to obtain the current homography matrix of the target vehicle. ,include:
  • the historical homography matrix is modified to obtain the current homography matrix of the target vehicle.
  • the pavement element includes multiple, and the third determination module 604 is used to determine the same
  • the target affine matrix between the first feature point set and the second feature point set of the pavement element includes:
  • the current affine matrix is an initial preset affine matrix or an affine matrix adjusted at least once for the initial preset affine matrix;
  • the pavement element includes the first type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, the third determining module 604 further Used for:
  • each first-type pavement element For each first-type pavement element, extracting a first feature point matching each second feature point in the second feature point set of the first-type pavement element from the first feature point set of the first-type pavement element;
  • the third determination module 604 is configured to determine the updated second feature point set of the pavement element based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element When the residual value between the world coordinates and the world coordinates of the first feature point set, includes:
  • each first-type pavement element For each first-type pavement element, based on the updated world coordinates of each second feature point in the second feature point set of the first-type pavement element and the world coordinates of the first feature point matching the second feature point , determine the distance between the second feature point and the matched first feature point;
  • the pavement element further includes a second type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, the third determining module 604 Also used for:
  • the third determination module 604 is configured to determine the updated second feature point set of the pavement element based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element When the residual value between the world coordinates and the world coordinates of the first feature point set, includes:
  • the residual value corresponding to the second type of pavement element is determined.
  • the third determining module 604 is further configured to:
  • the third determination module 604 is used to determine the residual corresponding to the second type of pavement element based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element difference, including:
  • the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
  • an embodiment of the present disclosure further provides an electronic device 700 .
  • the schematic structural diagram of the electronic device 700 provided by the embodiment of the present disclosure includes:
  • the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the following instructions : Obtain the current road image collected by the target vehicle and a pre-made map of the current location of the target vehicle; based on the current road image, determine the first world coordinates of the road elements in the current road image; based on the pre-made map, determine the second world of the road elements Coordinates: According to the first world coordinates and the second world coordinates of the road surface elements, determine the target world coordinates of the target object in the target image collected by the target vehicle.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the positioning method in the foregoing method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the positioning methods described in the foregoing method embodiments.
  • the computer program product carries program codes
  • the instructions included in the program codes can be used to execute the steps of the positioning methods described in the foregoing method embodiments.
  • the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Exposure And Positioning Against Photoresist Photosensitive Materials (AREA)
  • Automatic Disk Changers (AREA)
  • Display Devices Of Pinball Game Machines (AREA)

Abstract

Provided in the present disclosure are a positioning method and apparatus, an electronic device, and a storage medium, the positioning method comprising: acquiring current road surface images collected by a target vehicle and a prefabricated map of the current location of the target vehicle; on the basis of the current road surface images, determining first world coordinates of a road surface element in the current road surface images; on the basis of the prefabricated map, determining second world coordinates of the road surface element; and, on the basis of the first world coordinates and the second world coordinates of the road surface element, determining target world coordinates of a target object in target images collected by the target vehicle.

Description

一种定位方法、装置、电子设备及存储介质A positioning method, device, electronic device and storage medium
本公开要求在2021年03月19日提交中国专利局、申请号为202110295186.8、申请名称为“一种定位方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。This disclosure claims the priority of the Chinese patent application with the application number of 202110295186.8 and the application title of "a positioning method, device, electronic device and storage medium" filed with the China Patent Office on March 19, 2021, the entire contents of which are by reference Incorporated in this disclosure.
技术领域technical field
本公开涉及计算机视觉技术领域，具体而言，涉及一种定位方法、装置、电子设备及存储介质。The present disclosure relates to the field of computer vision technology, and in particular, to a positioning method, an apparatus, an electronic device, and a storage medium.
背景技术Background technique
随着人工智能技术的迅速发展，传统工业与信息技术结合，为人们的生活带来便利，比如将汽车行业与信息技术结合，可以产生能够自动驾驶的智能汽车，智能汽车在自动驾驶过程中，视觉定位是非常重要的环节。在视觉定位中，单目视觉定位技术相对于多目视觉测距技术具有成本低廉、***安装简单、稳定性好等特点，因而被广泛采用。With the rapid development of artificial intelligence technology, the combination of traditional industry and information technology brings convenience to people's lives. For example, the combination of the automobile industry and information technology can produce smart cars that can drive automatically. Visual positioning is a very important part. In visual positioning, monocular visual positioning technology has the characteristics of low cost, simple system installation and good stability compared with multi-eye visual ranging technology, so it is widely used.
在单目视觉定位中，需要用到单应性矩阵(homography matrix)，基于拍摄的目标对象体在像素坐标系中的像素坐标，以及该单应性矩阵，可以得到目标对象体在世界坐标系中的世界坐标。因此，单应性矩阵的准确性直接影响定位结果的精确性。In monocular vision positioning, a homography matrix is required. Based on the pixel coordinates of the captured target object in the pixel coordinate system, and the homography matrix, the target object in the world coordinate system can be obtained. in world coordinates. Therefore, the accuracy of the homography matrix directly affects the accuracy of the positioning results.
发明内容SUMMARY OF THE INVENTION
本公开实施例至少提供一种定位方法、装置、电子设备及存储介质。The embodiments of the present disclosure provide at least a positioning method, an apparatus, an electronic device, and a storage medium.
第一方面，本公开实施例提供了一种定位方法，包括：获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标；基于所述预制地图，确定所述路面元素的第二世界坐标；根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中的目标对象的目标世界坐标。In a first aspect, an embodiment of the present disclosure provides a positioning method, including: acquiring a current road surface image collected by a target vehicle and a prefabricated map of the current location of the target vehicle; The first world coordinate of the road surface element; based on the prefabricated map, the second world coordinate of the road surface element is determined; according to the first world coordinate and the second world coordinate of the road surface element, the target collected by the target vehicle is determined. The target world coordinate of the target object in the image.
本公开实施例中，随着目标车辆的行驶，因为机械振动或者行驶路面的不平整，会导致目标车辆中用于定位的历史定位参数不再准确，比如导致目标车辆的历史单应性矩阵不再准确，从而使得第一世界坐标和第二世界坐标会存在一定的差距，这样可以根据该差距对目标车辆的当前定位参数进行调整，从而提高定位参数的准确度，从而可以提高对目标对象定位结果的准确度。In the embodiment of the present disclosure, as the target vehicle travels, the historical positioning parameters used for positioning in the target vehicle are no longer accurate due to mechanical vibrations or uneven driving surfaces. For example, the historical homography matrix of the target vehicle may not be accurate. No matter how accurate it is, there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, so as to improve the accuracy of the positioning parameters, thereby improving the positioning of the target object. the accuracy of the results.
在一种实施方式中，在获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图之前，所述定位方法还包括：响应于针对目标车辆的标定按键的触发操作，或者，接收到针对目标车辆进行标定的标定指令。In an embodiment, before acquiring the current road image collected by the target vehicle and the pre-made map of the current location of the target vehicle, the positioning method further includes: responding to a triggering operation of a calibration button for the target vehicle, or receiving to the calibration command to calibrate the target vehicle.
本公开实施例中，可以在检测到存在针对目标车辆的标定按键的触发操作，或者接收到针对目标车辆进行标定的标定指令后，首先对目标车辆进行标定，以提高目标车辆用于进行定位的当前定位参数，比如提高目标车辆的单应性矩阵。In the embodiment of the present disclosure, after detecting that there is a trigger operation of the calibration button for the target vehicle, or after receiving a calibration instruction for calibrating the target vehicle, the target vehicle can be calibrated first, so as to improve the accuracy of the target vehicle for positioning. Current localization parameters, such as improving the homography matrix of the target vehicle.
在一种实施方式中，所述基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标，包括：获取所述当前路面图像中的所述路面元素的像素坐标；基于所述路面元素的像素坐标以及预先确定的所述采集设备的历史单应性矩阵，确定所述路面元素的第一世界坐标。In one embodiment, the determining, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image includes: acquiring pixel coordinates of the road surface element in the current road surface image; A first world coordinate of the road surface element is determined based on the pixel coordinates of the road surface element and a predetermined historical homography matrix of the acquisition device.
本公开实施例中，单应性矩阵可以反映路面图像对应的图像坐标系和目标车辆对应的世界坐标系之间的转换关系，因此基于路面元素在当前路面图像中的像素坐标，可以基于预先确定的历史单应性矩阵，快速确定路面元素在世界坐标系下的第一世界坐标。In the embodiment of the present disclosure, the homography matrix can reflect the transformation relationship between the image coordinate system corresponding to the road surface image and the world coordinate system corresponding to the target vehicle. The history homography matrix of , quickly determines the first world coordinates of pavement elements in the world coordinate system.
在一种可能的实施方式中，所述获取所述当前路面图像中所述路面元素的像素坐标，包括：对所述当前路面图像进行语义分割，得到所述当前路面图像包含的至少一个路面元素，以及每个路面元素的路面元素类别；按照每个路面元素的路面元素类别，在所述当前路面图像中提取该路面元素包含的第一特征点集的像素坐标；将所述第一特征点集的像素坐标作为所述第一特征点集对应的路面元素的像素坐标。In a possible implementation manner, the obtaining the pixel coordinates of the road surface element in the current road surface image includes: performing semantic segmentation on the current road surface image to obtain at least one road surface element included in the current road surface image , and the pavement element category of each pavement element; according to the pavement element category of each pavement element, extract the pixel coordinates of the first feature point set contained in the pavement element from the current pavement image; The pixel coordinates of the set are taken as the pixel coordinates of the road surface elements corresponding to the first feature point set.
本公开实施例中，可有按照语义分割方式，快速提取出路面图像中包含的路面元素，并可以按照预先设定的路面元素类别，快速提取到路面图像中的路面元素包含的第一特征点集的像素坐标，得到路面元素的像素坐标。In the embodiment of the present disclosure, the pavement elements included in the pavement image can be quickly extracted according to the semantic segmentation method, and the first feature points included in the pavement elements in the pavement image can be quickly extracted according to the preset pavement element categories Set the pixel coordinates to get the pixel coordinates of the pavement element.
在一种可能的实施方式中，所述按照每个路面元素的路面元素类别，在所述当前路面图像中提取该路面元素包含的第一特征点集的像素坐标，包括：在该路面元素为第一类路面元素的情况下，在该路面元素的轮廓上提取多个第一特征点，得到该路面元素的第一特征点集；在该路面元素为第二类路面元素的情况下，在该路面元素的中心线上按照设定距离间隔提取第一特征点，得到该路面元素的第一特征点集；在所述当前路面图像对应的像素坐标系中，获取所述第一特征点集中的每个第一特征点的像素坐标。In a possible implementation manner, according to the pavement element category of each pavement element, extracting the pixel coordinates of the first feature point set included in the pavement element from the current pavement image, including: in the pavement element being In the case of the first type of pavement element, extract a plurality of first feature points on the contour of the pavement element to obtain the first feature point set of the pavement element; in the case of the pavement element of the second type of pavement element, in the The first feature point is extracted from the center line of the pavement element according to the set distance interval, and the first feature point set of the pavement element is obtained; in the pixel coordinate system corresponding to the current pavement image, the first feature point set is obtained. The pixel coordinates of each first feature point of .
本公开实施例中，在针对每个路面元素进行特征点提取时，可以按照该路面元素具体的路面元素类别进行特征点提取，从而快速得到每种路面元素包含的第一特征点集，以及第一特征点集中各第一特征点的像素坐标。In the embodiment of the present disclosure, when extracting feature points for each pavement element, the feature point extraction can be performed according to the specific pavement element category of the pavement element, so as to quickly obtain the first feature point set included in each pavement element, and the third The pixel coordinates of each first feature point in a feature point set.
在一种可能的实施方式中，所述基于预制地图，确定所述路面元素的第二世界坐标，包括：获取预先存储的所述路面元素的第二特征点集在所述预制地图中的地图位置信息；基于所述路面元素的第二特征点集的地图位置信息和所述目标车辆在所述预制地图表征的道路场景中的位姿数据，确定所述路面元素的第二特征点集的世界坐标；将所述第二特征点集的世界坐标作为所述第二特征点集对应的路面元素的第二世界坐标。In a possible implementation manner, the determining the second world coordinates of the pavement element based on a prefabricated map includes: acquiring a pre-stored map in which the second feature point set of the pavement element is contained in the prefabricated map location information; based on the map location information of the second feature point set of the pavement element and the pose data of the target vehicle in the road scene represented by the prefabricated map, determine the location of the second feature point set of the pavement element World coordinates; take the world coordinates of the second feature point set as the second world coordinates of the road surface element corresponding to the second feature point set.
本公开实施例中，通过预先存储的路面元素在预制地图中的地图位置信息以及目标车辆在预制地图表征的道路场景中的位姿数据，可以快速确定出路面元素包含的第二特征点集的世界坐标，从而得到路面元素的第二世界坐标。In the embodiment of the present disclosure, by using the pre-stored map position information of the pavement element in the prefabricated map and the pose data of the target vehicle in the road scene represented by the prefabricated map, the location of the second feature point set included in the pavement element can be quickly determined. world coordinates to get the second world coordinates of the pavement element.
在一种可能的实施方式中，所述根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中的目标对象的目标世界坐标，包括：基于所述路面元素的第一世界坐标和第二世界坐标，对所述历史单应性矩阵进行调整，得到所述目标车辆的当前单应性矩阵；基于所述目标图像，确定所述目标对象在所述目标图像中的像素坐标；基于所述目标对象的像素坐标和所 述当前单应性矩阵，确定所述目标对象的目标世界坐标。In a possible implementation manner, the determining, according to the first world coordinates and the second world coordinates of the road surface element, the target world coordinates of the target object in the target image collected by the target vehicle includes: based on the The first world coordinate and the second world coordinate of the road surface element, the historical homography matrix is adjusted to obtain the current homography matrix of the target vehicle; based on the target image, it is determined that the target object is in the pixel coordinates in the target image; based on the pixel coordinates of the target object and the current homography matrix, determine the target world coordinates of the target object.
本公开实施例中，可以根据路面元素的第一世界坐标和第二世界坐标之间的差距，对目标车辆的历史单应性矩阵修正，得到准确度较高的当前单应性矩阵，进一步基于准确度较高的当前单应性矩阵以及目标对象的像素坐标，可以得到确定目标对象准确度较高的目标世界坐标。In the embodiment of the present disclosure, the historical homography matrix of the target vehicle can be corrected according to the difference between the first world coordinate and the second world coordinate of the road surface element, so as to obtain the current homography matrix with high accuracy, and further based on The current homography matrix with high accuracy and the pixel coordinates of the target object can obtain the target world coordinates with high accuracy for determining the target object.
在一种可能的实施方式中，在确定所述目标对象的目标世界坐标之后，所述定位方法还包括：基于所述目标对象的目标世界坐标以及预设位置点的位置信息，确定所述目标对象与所述预设位置点之间的距离。In a possible implementation manner, after determining the target world coordinates of the target object, the positioning method further includes: determining the target based on the target world coordinates of the target object and the position information of a preset position point The distance between the object and the preset position point.
本公开实施例得到目标车辆的当前单应性矩阵后，能够利用该当前单应性矩阵准确地确定目标对象的目标世界坐标，进而可以得到预制位置点与目标对象之间准确度较高的距离。After obtaining the current homography matrix of the target vehicle in the embodiment of the present disclosure, the current homography matrix can be used to accurately determine the target world coordinates of the target object, and then the distance between the prefabricated position point and the target object can be obtained with high accuracy .
在一种可能的实施方式中，所述基于所述路面元素的第一世界坐标和第二世界坐标，对所述历史单应性矩阵进行调整，得到所述目标车辆的当前单应性矩阵，包括：基于所述路面元素的第一特征点集的世界坐标和所述路面元素的第二特征点集的世界坐标，确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵；基于所述目标仿射矩阵，对所述历史单应性矩阵进行修正，得到所述目标车辆的当前单应性矩阵。In a possible implementation manner, the historical homography matrix is adjusted based on the first world coordinate and the second world coordinate of the road surface element to obtain the current homography matrix of the target vehicle, The method includes: determining the first feature point set and the second feature point of the same pavement element based on the world coordinates of the first feature point set of the pavement element and the world coordinates of the second feature point set of the pavement element The target affine matrix between sets; based on the target affine matrix, the historical homography matrix is modified to obtain the current homography matrix of the target vehicle.
本公开实施例中，提出通过确定同一路面元素的第一特征点集和第二特征点集，可以得到用于表示历史单应性矩阵和当前单应性矩阵之间的差距的目标仿射矩阵，因此可以基于该目标仿射矩阵来对历史单应性矩阵进行修正，从而得到准确度较高的当前单应性矩阵。In the embodiment of the present disclosure, it is proposed that by determining the first feature point set and the second feature point set of the same pavement element, a target affine matrix for representing the gap between the historical homography matrix and the current homography matrix can be obtained. , so the historical homography matrix can be modified based on the target affine matrix, so as to obtain the current homography matrix with higher accuracy.
在一种可能的实施方式中，所述路面元素包含多个，所述基于所述路面元素的第一特征点集和所述路面元素的第二特征点集的世界坐标，确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵，包括：按照当前仿射矩阵，对多个路面元素中的每个路面元素的第二特征点集的世界坐标进行更新，确定该路面元素的第二特征点集的更新后的世界坐标；所述当前仿射矩阵为初始预设仿射矩阵或者为针对所述初始预设仿射矩阵调整过至少一次的仿射矩阵；基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值；基于所述多个路面元素的残差值，对所述当前仿射矩阵进行调整后，返回确定该路面元素的第二特征点集的更新后的世界坐标的步骤，直至满足预设截止条件后，得到所述目标仿射矩阵。In a possible implementation manner, the pavement element includes a plurality of elements, and the pavement element is determined based on the world coordinates of the first feature point set of the pavement element and the second feature point set of the pavement element The target affine matrix between the first feature point set and the second feature point set includes: according to the current affine matrix, for the world of the second feature point set of each road surface element in the plurality of road surface elements The coordinates are updated, and the updated world coordinates of the second feature point set of the road surface element are determined; the current affine matrix is an initial preset affine matrix or an affine matrix that has been adjusted at least once for the initial preset affine matrix Affine matrix; based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated world coordinates of the second feature point set of the pavement element and the first feature point set of the pavement element. A residual value between the world coordinates of a feature point set; based on the residual values of the plurality of pavement elements, after adjusting the current affine matrix, return to determine the update of the second feature point set of the pavement element After the steps of the world coordinates, the target affine matrix is obtained until the preset cut-off condition is met.
本公开实施例中，提出可以对仿射矩阵进行多次调整，直至得到准确度较高的目标仿射矩阵，以便后续可以基于准确度较高的目标仿射矩阵得到准确度较高的当前单应性矩阵。In the embodiment of the present disclosure, it is proposed that the affine matrix may be adjusted multiple times until a target affine matrix with higher accuracy is obtained, so that the current single target affine matrix with higher accuracy can be obtained subsequently based on the target affine matrix with higher accuracy. Responsiveness Matrix.
在一种可能的实施方式中，所述路面元素包括第一类路面元素，在确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵之前，所述定位方法还包括：针对每个第一类路面元素，在该第一类路面元素的第一特征点集中提取与该第一类路面元素的第二特征点集中的每个第二特征点匹配的第一特征点；所述基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值，包括：针对每个第一类路面元素，基于该第一类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和与该第二特征点匹配的第一特征点的世界坐标，确定 该第二特征点和匹配的第一特征点之间的距离；对该第一类路面元素包含的每个第二特征点和匹配的第一特征点之间的距离进行求和，得到该第一类路面元素对应的残差值。In a possible implementation manner, the pavement element includes a first type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, The positioning method further includes: for each first-type pavement element, extracting each second feature point from the first feature point set of the first-type pavement element and the second feature point set of the first-type pavement element The matched first feature point; the updated second feature point set of the pavement element is determined based on the updated world coordinates of the second feature point set of the pavement element and the world coordinate of the first feature point set of the pavement element. The residual value between the world coordinates of and the world coordinates of the first feature point set, including: for each first type of pavement element, based on each second feature point in the second feature point set of the first type of pavement element The updated world coordinates and the world coordinates of the first feature point matched with the second feature point, determine the distance between the second feature point and the matched first feature point; The distances between the second feature points and the matched first feature points are summed to obtain the residual value corresponding to the first type of road surface element.
本公开实施例中，提出针对第一类路面元素，可以通过每个第一类路面元素包含的每个第二特征点和匹配的第一特征点在世界坐标系中的距离，来确定通过历史单应性矩阵确定的第一类路面元素的第一世界坐标和该第一类路面元素的第二世界坐标之间的残差值，以便后续通过对该残差值进行调整确定目标仿射矩阵。In the embodiment of the present disclosure, it is proposed that for the first type of pavement element, the distance of each second feature point included in each first type of pavement element and the matched first feature point in the world coordinate system can be used to determine the passing history The residual value between the first world coordinate of the first type of pavement element determined by the homography matrix and the second world coordinate of the first type of pavement element, so that the target affine matrix can be determined by adjusting the residual value subsequently .
在一种可能的实施方式中，所述路面元素还包括第二类路面元素，在确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵之前，所述定位方法还包括：针对每个第二类路面元素，按照该第二类路面元素的第一特征点集的世界坐标，确定该第二类路面元素的第一特征点集的拟合直线方程；所述基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值，包括：针对每个第二类路面元素，基于该第二类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和该第二类路面元素对应的拟合直线方程，确定该第二特征点和该第二类路面元素对应的拟合直线之间的距离；基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值。In a possible implementation manner, the pavement element further includes a second type of pavement element, before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element , the positioning method further includes: for each second-type pavement element, according to the world coordinates of the first feature point set of the second-type pavement element, determining the fitting of the first feature point set of the second-type pavement element straight line equation; the updated world coordinates of the second feature point set of the pavement element and the The residual value between the world coordinates of the first feature point set, including: for each second-type road surface element, the updated world based on each second feature point in the second feature point set of the second-type road surface element Coordinates and the fitting line equation corresponding to the second type of pavement element, determine the distance between the second feature point and the fitted straight line corresponding to the second type of pavement element; The distance between the two feature points and the fitting straight line corresponding to the second type of road surface element determines the residual value corresponding to the second type of road surface element.
本公开实施例中，提出针对第二类路面元素，可以通过每个第二类路面元素包含的每个第二特征点到该第二类路面元素对应的拟合直线之间的距离，来确定通过历史单应性矩阵确定的第二类路面元素的第一世界坐标和该第二类路面元素的第二世界坐标之间的残差值，以便后续通过对该残差值进行调整确定目标仿射矩阵。In the embodiment of the present disclosure, it is proposed that for the second type of pavement element, the distance between each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element can be determined. The residual value between the first world coordinate of the second type of pavement element determined by the historical homography matrix and the second world coordinate of the second type of pavement element, so that the target simulation can be determined by adjusting the residual value subsequently. shot matrix.
在一种可能的实施方式中，在确定该第二类路面元素对应的残差值之前，所述定位方法还包括：基于该第二类路面元素包含的每个第二特征点的世界坐标，获取该第二类路面元素的每个第二特征点对应的距离权重；所述基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值，包括：基于该第二类路面元素包含的每个第二特征点对应的距离权重，对该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离进行加权求和，得到该第二类路面元素对应的残差值。In a possible implementation manner, before determining the residual value corresponding to the second type of pavement element, the positioning method further includes: based on the world coordinates of each second feature point included in the second type of pavement element, Obtain the distance weight corresponding to each second feature point of the second type of pavement element; the distance weight based on each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element and determining the residual value corresponding to the second type of pavement element, including: based on the distance weight corresponding to each second feature point included in the second type of pavement element, each The distance between the two feature points and the fitted straight line corresponding to the second type of road surface element is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
本公开实施例中，在路面元素同时包含第一类路面元素和第二类路面元素时，考虑到距离目标车辆较远的区域第一类路面元素包含的有效特征点的数量较少，因此可以调整第二类路面元素在距离采集设备不同位置的第二特征点对应的距离权重，来平衡距离目标车辆远近区域包含的有效特征点不均衡的问题，从而使得得到的当前单应矩阵在衡量与目标车辆距离不同区域的路面和对应的图像之间的转换关系时均具有较高的鲁棒性和精度。In the embodiment of the present disclosure, when the pavement element includes both the first type of pavement element and the second type of pavement element, considering that the area far from the target vehicle contains the first type of pavement element, the number of effective feature points is small, so it can be Adjust the distance weights corresponding to the second feature points of the second type of pavement elements at different positions of the distance collection device to balance the problem of unbalanced effective feature points contained in the far and near areas of the target vehicle, so that the obtained current homography matrix is measured and The conversion relationship between the target vehicle and the road surface in different regions and the corresponding images has high robustness and accuracy.
第二方面，本公开实施例提供了一种定位装置，包括：获取模块，用于获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；第一确定模块，用于基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标；第二确定模块，用于基于所述预制地图，确定所述路面元素的第二世界坐标；第三确定模块，用于根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中目标对象的目标世界坐标。In a second aspect, an embodiment of the present disclosure provides a positioning device, including: an acquisition module for acquiring a current road image collected by a target vehicle and a pre-made map of the current location of the target vehicle; a first determination module for acquiring the current road surface image, and determine the first world coordinates of the road surface elements in the current road surface image; the second determination module is used to determine the second world coordinates of the road surface elements based on the prefabricated map; the third determination module, It is used for determining the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements.
第三方面，本公开实施例提供了一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如第一方面所述的定位方法的步骤。In a third aspect, embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing The processor and the memory communicate through a bus, and the machine-readable instructions execute the steps of the positioning method according to the first aspect when the machine-readable instructions are executed by the processor.
第四方面，本公开实施例提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如第一方面所述的定位方法的步骤。In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the positioning method according to the first aspect are executed .
第五方面，本公开提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行如第一方面所述的定位方法的步骤。In a fifth aspect, the present disclosure provides a computer program comprising computer readable code, when the computer readable code is executed in an electronic device, a processor in the electronic device performs the positioning according to the first aspect steps of the method.
为使本公开的上述目的、特征和优点能更明显易懂，下文特举实施例，并配合所附附图，作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the following specific embodiments are given and described in detail in conjunction with the accompanying drawings.
附图说明Description of drawings
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.
图1示出了本公开实施例所提供的一种定位方法的流程图；FIG. 1 shows a flowchart of a positioning method provided by an embodiment of the present disclosure;
图2示出了本公开实施例所提供的一种确定路面元素的第一世界坐标的方法流程图；FIG. 2 shows a flowchart of a method for determining a first world coordinate of a road surface element provided by an embodiment of the present disclosure;
图3示出了本公开实施例所提供的一种确定目标对象的目标世界坐标的方法流程图；3 shows a flowchart of a method for determining the target world coordinates of a target object provided by an embodiment of the present disclosure;
图4示出了本公开实施例所提供的一种确定目标车辆的当前单应性矩阵的方法流程图；FIG. 4 shows a flowchart of a method for determining a current homography matrix of a target vehicle provided by an embodiment of the present disclosure;
图5示出了本公开实施例所提供的同一路面元素通过不同方式得到的在世界坐标系下的投影位置示意图；FIG. 5 is a schematic diagram showing the projected position of the same pavement element in the world coordinate system obtained by different methods provided by an embodiment of the present disclosure;
图6示出了本公开实施例提供的一种确定目标仿射矩阵的方法流程图FIG. 6 shows a flowchart of a method for determining a target affine matrix provided by an embodiment of the present disclosure
图7示出了本公开实施例所提供的一种定位装置的示意图；FIG. 7 shows a schematic diagram of a positioning device provided by an embodiment of the present disclosure;
图8示出了本公开实施例所提供的一种电子设备的示意图。FIG. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
具体实施方式Detailed ways
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.
本文中术语“和/或”，仅仅是描述一种关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。The term "and/or" in this paper only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: the existence of A alone, the existence of A and B at the same time, the existence of B alone. a situation. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.
在自动驾驶领域、机器人领域，可以依靠采集设备采集的图像进行视觉定位，比如摄像机或者单目相机来进行视觉定位，依靠采集设备进行视觉定位的原理是确定采集设备拍摄的目标对象在像素坐标系中的像素坐标，然后基于采集设备的单应性矩阵，确定目标对象在世界坐标系下的世界坐标，因此，单应性矩阵的准确性直接影响定位结果的精确性。车辆上的采集设备的单应性矩阵可以通过采集设备与车辆的位置关系进行预先标定得到，在对车辆上的采集设备进行标定后，即可以按照该单应性矩阵来确定目标对象的位置，但是在车辆行驶过程中，由于长期行驶带来的机械振动或者车辆行驶在不平整的里面上时，可能会造成采集设备与车辆的相对位置发生改变，使得历史确定的单应性矩阵不再适用，影响定位的准确度，针对此，本公开以下实施例提供了一种对在车辆行驶过程中对目标对象进行定位的方法。In the field of autonomous driving and robotics, visual positioning can be carried out by relying on images collected by acquisition equipment, such as cameras or monocular cameras. The principle of visual positioning relying on acquisition equipment is to determine that the target object captured by the acquisition equipment is in the pixel coordinate system. Then, based on the homography matrix of the acquisition device, the world coordinates of the target object in the world coordinate system are determined. Therefore, the accuracy of the homography matrix directly affects the accuracy of the positioning result. The homography matrix of the acquisition device on the vehicle can be obtained by pre-calibrating the positional relationship between the acquisition device and the vehicle. After the acquisition device on the vehicle is calibrated, the location of the target object can be determined according to the homography matrix. However, during the driving process of the vehicle, the relative position of the acquisition device and the vehicle may change due to the mechanical vibration caused by long-term driving or when the vehicle is driven on an uneven interior, making the historically determined homography matrix no longer applicable. , which affects the accuracy of positioning. In view of this, the following embodiments of the present disclosure provide a method for positioning a target object during vehicle driving.
基于上述研究，本公开提供了一种定位方法，随着目标车辆的行驶，因为机械振动或者行驶路面的不平整，会导致目标车辆中用于定位的历史定位参数不再准确，比如导致目标车辆的历史单应性矩阵不再准确，从而使得第一世界坐标和第二世界坐标会存在一定的差距，这样可以根据该差距对目标车辆的当前定位参数进行调整，从而提高定位参数的准确度，从而可以提高对目标对象定位结果的准确度。Based on the above research, the present disclosure provides a positioning method. As the target vehicle travels, the historical positioning parameters used for positioning in the target vehicle may become inaccurate due to mechanical vibrations or uneven driving surfaces. The historical homography matrix is no longer accurate, so that there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, thereby improving the accuracy of the positioning parameters. Thus, the accuracy of the target object localization result can be improved.
为便于对本实施例进行理解，首先对本公开实施例所公开的一种定位方法进行详细介绍，本公开实施例所提供的定位方法的执行主体一般为具有一定计算能力的计算机设备，该计算机设备例如包括：终端设备或服务器或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、计算设备、车载设备等。在一些可能的实现方式中，该定位方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。In order to facilitate the understanding of this embodiment, a positioning method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the positioning method provided by the embodiment of the present disclosure is generally a computer device with a certain computing capability, such as a computer device. Including: terminal equipment or server or other processing equipment, the terminal equipment may be user equipment (User Equipment, UE), mobile equipment, user terminal, computing equipment, vehicle-mounted equipment, etc. In some possible implementations, the positioning method may be implemented by the processor invoking computer-readable instructions stored in the memory.
参见图1所示，为本公开实施例提供的定位方法的流程图，该定位方法可以应用于车载设备中的处理器，该定位方法包括步骤S101～S104，其中：Referring to FIG. 1, which is a flowchart of a positioning method provided by an embodiment of the present disclosure, the positioning method can be applied to a processor in a vehicle-mounted device, and the positioning method includes steps S101-S104, wherein:
S101，获取目标车辆采集的当前路面图像和目标车辆所处位置的预制地图。S101 , acquiring a current road image collected by a target vehicle and a prefabricated map of the location of the target vehicle.
示例性地，目标车辆上可以设置有用于采集路面图像的采集设备，采集设备可以包括单目摄像机，或者单目照相机，也可以包含用于采集彩色图像、灰度图像或者深度图像的摄像头，当目标车辆在路面中行驶时，采集设备可以按照预设的时间间隔拍摄路面的路面图像。Exemplarily, the target vehicle may be provided with a capture device for capturing road images, and the capture device may include a monocular camera, or a monocular camera, or a camera for capturing color images, grayscale images, or depth images. When the target vehicle is driving on the road, the acquisition device can capture road images of the road at preset time intervals.
示例性地，预制地图可以是预先构建的与目标车辆行驶的道路场景匹配的场景地图，该预制地图中元素与目标车辆行驶的道路场景中的同一元素在相同坐标系下可以按照1:1呈现的，另外，在构建该预制地图时可以在该预制地图中存储各个路面元素包含的特征点在预制地图中的地图位置信息，为了节省存储空间，在存储各个路面元素的特征点在预制地图中的地图位置信息时，每个路面元素可以存储少量的能够表征几何形状特征的点的地图位置信息。Exemplarily, the prefabricated map may be a pre-built scene map that matches the road scene where the target vehicle is driving, and the elements in the prefabricated map and the same element in the road scene where the target vehicle is driving may be presented 1:1 in the same coordinate system. In addition, when the prefabricated map is constructed, the map location information of the feature points contained in each road surface element in the prefabricated map can be stored in the prefabricated map. In order to save storage space, the feature points of each road surface element are stored in the prefabricated map. When the map location information is 1, each pavement element can store a small amount of map location information of points that can characterize the geometric shape features.
S102，基于当前路面图像，确定当前路面图像中的路面元素的第一世界坐标。S102, based on the current road surface image, determine the first world coordinates of the road surface element in the current road surface image.
示例性地，路面元素包括用于对目标车辆进行指示的路面标志，具体可以包括指向标志、虚线段、斑马线、实线、停止线等，其中，指向标志比如可以包含左转指向标志、右转指向标志和直行标志等。Exemplarily, the road surface elements include road surface signs used to indicate the target vehicle, and may specifically include pointing signs, dotted line segments, zebra crossings, solid lines, stop lines, etc. Pointing signs and going straight signs, etc.
示例性地，路面元素的第一世界坐标可以通过路面元素包含的特征点在世界坐标系中的世界坐标来表示，其中，路面元素的特征点可以包括能够表征路面元素几何形状特征的点，比如针对可以提取到完整轮廓的路面元素，可以将构成路面元素的轮廓的多个点作为特征点，比如指向标志、虚线段中的每个线段、斑马线等；针对无法提取到完整轮廓的路面元素，比如实线的车道线、实线的停止线等，因为车道线、停止线一般为直线，因此可以提取表示该路面元素作为直线特征的多个点作为特征点。Exemplarily, the first world coordinates of the pavement element may be represented by the world coordinates of the feature points contained in the pavement element in the world coordinate system, wherein the feature points of the pavement element may include points that can characterize the geometrical features of the pavement element, such as For pavement elements whose complete outlines can be extracted, multiple points that constitute the outlines of pavement elements can be used as feature points, such as pointing signs, each line segment in the dotted line segment, zebra crossings, etc.; for pavement elements whose complete outlines cannot be extracted, For example, the solid line lane line, the solid line stop line, etc., because the lane line and the stop line are generally straight lines, so multiple points representing the road surface element as a straight line feature can be extracted as feature points.
S103，基于预制地图，确定路面元素的第二世界坐标。S103, based on the prefabricated map, determine the second world coordinates of the road surface element.
示例性地，在获取到当前路面图像后，可以基于当前路面图像确定目标车辆在预制地图指示的道路场景中的位姿数据，从而确定目标车辆所在位置包含的路面元素在以目标车辆为原点构建的世界坐标系下的第二世界坐标，具体地，每个路面元素的第二世界坐标具体指预先存储的该路面元素包含的特征点在世界坐标系下的世界坐标，可以根据该路面元素在预制地图中的地图位置信息以及目标车辆在预制地图指示的道路场景中的位姿数据进行确定，详见后文。Exemplarily, after the current road surface image is acquired, the pose data of the target vehicle in the road scene indicated by the prefabricated map can be determined based on the current road surface image, so as to determine that the road surface elements contained in the location of the target vehicle are constructed with the target vehicle as the origin. The second world coordinate in the world coordinate system of the The map location information in the prefabricated map and the pose data of the target vehicle in the road scene indicated by the prefabricated map are determined, as described later.
示例性地，在按照预设时间间隔获取路面图像时，针对当前路面图像，可以将基于当前路面图像提取的每个路面元素包含的多个特征点记为该路面元素的第一特征点集，将基于预制地图预先存储的该路面元素包含的多个特征点记为第二特征点集。Exemplarily, when acquiring the road surface image according to the preset time interval, for the current road surface image, the plurality of feature points contained in each road surface element extracted based on the current road surface image may be recorded as the first feature point set of the road surface element, A plurality of feature points included in the road surface element pre-stored based on the pre-made map is recorded as a second feature point set.
示例性地，上述提到的世界坐标系可以按照以下方式预先建立：Exemplarily, the above-mentioned world coordinate system can be pre-established in the following manner:
以车辆的前轴中心点或者车体中心在地面的映射点为原点，以车辆的前进方向为X轴，以垂直于车辆前进方向的方向为Y轴，以指向天空的方向为Z轴建立世界坐标系，在得到世界坐标系后，具体如何获取根据预制地图确定的在世界坐标系下路面元素的第二特征点集的世界坐标，将在后文进行详细介绍。Take the center point of the front axle of the vehicle or the mapping point of the center of the vehicle body on the ground as the origin, the forward direction of the vehicle as the X axis, the direction perpendicular to the forward direction of the vehicle as the Y axis, and the direction pointing to the sky as the Z axis to build the world Coordinate system, after obtaining the world coordinate system, how to obtain the world coordinates of the second feature point set of the pavement element in the world coordinate system determined according to the prefabricated map will be described in detail later.
S104，基于路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中的目标对象的目标世界坐标。S104 , based on the first world coordinates and the second world coordinates of the road surface elements, determine the target world coordinates of the target object in the target image collected by the target vehicle.
示例性地，在基于路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中的目标对象的目标世界坐标之前，首先对单应性矩阵进行介绍，目标车辆的单应性矩阵是指目标车辆上的采集设备的单应性矩阵，单应性矩阵可以用来表示采集设备采集的路面图像对应的图像坐标系和目标车辆所在的世界坐标系之间的转换关系，在采集到当前路面图像后，基于当前路面图像中的路面元素包含的第一特征点集的像素坐标，以及历史单应性矩阵，可以确定第一特征点集在世界坐标系下的世界坐标。Exemplarily, before determining the target world coordinates of the target object in the target image collected by the target vehicle based on the first world coordinates and the second world coordinates of the road surface elements, the homography matrix is first introduced. The property matrix refers to the homography matrix of the acquisition device on the target vehicle. The homography matrix can be used to represent the transformation relationship between the image coordinate system corresponding to the road surface image acquired by the acquisition device and the world coordinate system where the target vehicle is located. After the current road surface image is collected, based on the pixel coordinates of the first feature point set included in the road surface elements in the current road surface image and the historical homography matrix, the world coordinates of the first feature point set in the world coordinate system can be determined.
示例性地，历史单应性矩阵可以是针对采集设备进行初始标定过程中确定的初始单应性矩阵，也可以是上一次调整后得到的历史单应性矩阵，其中，上一次调整后得到的历史单应性矩阵的确定方式与确定当前单应性矩阵的方式相似，具体过程详见后文。Exemplarily, the historical homography matrix may be the initial homography matrix determined during the initial calibration process for the acquisition device, or may be the historical homography matrix obtained after the last adjustment, wherein the obtained value after the last adjustment The method of determining the historical homography matrix is similar to the method of determining the current homography matrix, and the specific process is described later.
示例性地，初始单应性矩阵可以是在目标车辆行驶之前进行标定的单应性矩阵，这里目标车辆 行驶之前进行标定的单应性矩阵可以通过目前已知的标定方式进行标定，比如在静止的路面上摆放参照物，通过参照物在世界坐标系中的世界坐标，以及参照物在采集设备获取的图像中的像素坐标，确定该采集设备在静止的目标车辆上的单应性矩阵。Exemplarily, the initial homography matrix may be a homography matrix that is calibrated before the target vehicle travels, where the homography matrix that is calibrated before the target vehicle travels may be calibrated by a currently known calibration method, such as when stationary. The reference object is placed on the road surface of the device, and the homography matrix of the acquisition device on the stationary target vehicle is determined by the world coordinates of the reference object in the world coordinate system and the pixel coordinates of the reference object in the image acquired by the acquisition device.
示例性地，在历史单应性矩阵不再准确的情况下，同一路面元素的第一世界坐标和第二世界坐标之间会存在差距，可以基于该差距对历史单应性矩阵进行修正，得到准确度较高的当前单应性矩阵。Exemplarily, in the case where the historical homography matrix is no longer accurate, there will be a gap between the first world coordinates and the second world coordinates of the same road surface element, and the historical homography matrix can be corrected based on the gap to obtain The current homography matrix with high accuracy.
进一步在得到采集设备的当前单应性矩阵后，可以在当前时间段内基于当前单应性矩阵来对采集到的目标图像中的目标对象进行定位，比如每隔5min对单应性矩阵进行一次校准，这样在当前校准后到下一个校准期之间可以通过当前单应性矩阵和目标图像中的目标对象的像素坐标，确定目标图像中的目标对象的目标世界坐标。Further, after the current homography matrix of the acquisition device is obtained, the target object in the collected target image can be located based on the current homography matrix in the current time period, for example, the homography matrix is performed every 5 minutes. Calibration, so that between the current calibration and the next calibration period, the target world coordinates of the target object in the target image can be determined through the current homography matrix and the pixel coordinates of the target object in the target image.
示例性地，这里的目标图像可以为当前路面图像，目标对象可以指预先设定好的需要进行定位的目标对象，比如目标图像中的车辆、行人、树木、栏杆等障碍物。Exemplarily, the target image here may be a current road image, and the target object may refer to a preset target object that needs to be positioned, such as obstacles such as vehicles, pedestrians, trees, and railings in the target image.
本公开实施例中，随着目标车辆的行驶，因为机械振动或者行驶路面的不平整，会导致目标车辆中用于定位的历史定位参数不再准确，比如导致目标车辆的历史单应性矩阵不再准确，从而使得第一世界坐标和第二世界坐标会存在一定的差距，这样可以根据该差距对目标车辆的当前定位参数进行调整，从而提高定位参数的准确度，从而可以提高对目标对象定位结果的准确度。In the embodiment of the present disclosure, as the target vehicle travels, the historical positioning parameters used for positioning in the target vehicle are no longer accurate due to mechanical vibrations or uneven driving surfaces. For example, the historical homography matrix of the target vehicle may not be accurate. No matter how accurate it is, there will be a certain gap between the first world coordinates and the second world coordinates, so that the current positioning parameters of the target vehicle can be adjusted according to the gap, so as to improve the accuracy of the positioning parameters, thereby improving the positioning of the target object. the accuracy of the results.
下面将结合具体实施例对上述S101～S104进行具体介绍。The foregoing S101 to S104 will be described in detail below with reference to specific embodiments.
在一种实施方式中，在获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图之前，本公开实施例提供的定位方法还包括：In one embodiment, before acquiring the current road image collected by the target vehicle and the prefabricated map of the current location of the target vehicle, the positioning method provided by the embodiment of the present disclosure further includes:
响应于针对目标车辆的标定按键的触发操作，或者，接收到针对目标车辆进行标定的标定指令。In response to a triggering operation of a calibration button for the target vehicle, or, a calibration instruction for calibrating the target vehicle is received.
示例性地，目标车辆可以为自动驾驶车辆，在行驶过程中，除了可以按照预先设定的时间间隔对单应性矩阵进行修正外，还可以在检测到存在针对目标车辆上的标定按键的触发操作后再开始对单应性矩阵进行修正，比如目标车辆上设置有触发目标车辆开始对单应性矩阵进行修正的软件按键或者物理按键，在需要对单应性矩阵进行修正的情况下，用户可以触发该标定按键。Exemplarily, the target vehicle may be an automatic driving vehicle. During the driving process, in addition to correcting the homography matrix according to a preset time interval, it is also possible to detect that there is a trigger for the calibration button on the target vehicle. After the operation, start to correct the homography matrix. For example, there is a software button or physical button on the target vehicle that triggers the target vehicle to start correcting the homography matrix. When the homography matrix needs to be corrected, the user This calibration button can be triggered.
或者，目标车辆上还可以设置有与云端或者服务器保持通信连接的通信***，可以在通过该通信***接收到云端或者服务器发送的对目标车辆进行标定，即对单应性矩阵进行修正的标定指令后，开始获取目标车辆采集的当前路面图像和预制地图完成对历史单应性矩阵的修正过程。Alternatively, the target vehicle may also be provided with a communication system that maintains a communication connection with the cloud or the server, and can receive a calibration command sent by the cloud or the server through the communication system to calibrate the target vehicle, that is, to correct the homography matrix. After that, start to acquire the current road image and prefabricated map collected by the target vehicle to complete the process of correcting the historical homography matrix.
本公开实施例中，可以在检测到存在针对目标车辆的标定按键的触发操作，或者接收到针对目标车辆进行标定的标定指令后，首先对目标车辆进行标定，以提高目标车辆用于进行定位的当前定位参数，比如提高目标车辆的单应性矩阵。In the embodiment of the present disclosure, after detecting that there is a trigger operation of the calibration button for the target vehicle, or after receiving a calibration instruction for calibrating the target vehicle, the target vehicle can be calibrated first, so as to improve the accuracy of the target vehicle for positioning. Current localization parameters, such as improving the homography matrix of the target vehicle.
针对上述S102，在基于当前路面图像，确定当前路面图像中的路面元素的第一世界坐标时，如图2所示，可以包括以下S201～S202：For the above S102, when determining the first world coordinates of the road surface elements in the current road surface image based on the current road surface image, as shown in FIG. 2, the following S201-S202 may be included:
S201，获取当前路面图像中的路面元素的像素坐标；S201, acquiring pixel coordinates of road surface elements in the current road surface image;
S202，基于路面元素的像素坐标以及预先确定的采集设备的历史单应性矩阵，确定路面元素的第一世界坐标。S202 , based on the pixel coordinates of the pavement elements and the predetermined historical homography matrix of the acquisition device, determine the first world coordinates of the pavement elements.
示例性地，在获取到当前路面图像后，可以对当前路面图像进行目标检测，确定当前路面图像中包含的路面元素，进一步按照上述方式，提取每个路面元素包含的第一特征点集中每个特征点在当前路面图像对应的图像坐标系中的像素坐标。Exemplarily, after the current road surface image is acquired, target detection may be performed on the current road surface image to determine the road surface elements contained in the current road surface image, and further according to the above method, each of the first feature point sets contained in each road surface element is extracted. The pixel coordinates of the feature point in the image coordinate system corresponding to the current road image.
示例性地，在提取到在路面图像中路面元素的第一特征点集中的第i个第一特征点的像素坐标(u i,v i)后，可以根据以下公式(1)确定该第i个第一特征点在世界坐标系下的世界坐标(X i,Y i)： Exemplarily, after the pixel coordinates (u i , v i ) of the i-th first feature point in the first feature point set of road surface elements in the road surface image are extracted, the i-th first feature point can be determined according to the following formula (1): The world coordinates (X i ,Y i ) of the first feature point in the world coordinate system:
Figure PCTCN2021098697-appb-000001
Figure PCTCN2021098697-appb-000001
其中，H表示历史单应性矩阵。where H represents the historical homography matrix.
本公开实施例中，单应性矩阵可以反映路面图像对应的图像坐标系和目标车辆对应的世界坐标系之间的转换关系，因此基于路面元素在当前路面图像中的像素坐标，可以基于预先确定的历史单应性矩阵，快速确定路面元素在世界坐标系下的第一世界坐标。In the embodiment of the present disclosure, the homography matrix can reflect the transformation relationship between the image coordinate system corresponding to the road surface image and the world coordinate system corresponding to the target vehicle. The history homography matrix of , quickly determines the first world coordinates of pavement elements in the world coordinate system.
具体地，针对上述S201，在获取当前路面图像中路面元素的像素坐标时，包括以下S2011～S2013：Specifically, for the above S201, when acquiring the pixel coordinates of the road surface elements in the current road surface image, the following steps S2011 to S2013 are included:
S2011，对路面图像进行语义分割，得到路面图像包含的至少一个路面元素，以及每个路面元素的路面元素类别；S2011, performing semantic segmentation on the road surface image to obtain at least one road surface element included in the road surface image and the road surface element category of each road surface element;
S2012，按照每个路面元素的路面元素类别，在路面图像中提取该路面元素包含的第一特征点集的像素坐标；S2012, according to the pavement element category of each pavement element, extract the pixel coordinates of the first feature point set included in the pavement element from the pavement image;
S2013，将第一特征点集的像素坐标作为第一特征点集对应的路面元素的像素坐标。S2013, taking the pixel coordinates of the first feature point set as the pixel coordinates of the road surface element corresponding to the first feature point set.
示例性地，路面元素类别可以包含两种，一种是可以提取到轮廓的路面元素，比如上述提到的指向标志、虚线段和斑马线，另一种是无法提取到轮廓的路面元素，比如上述提到的实线、停止线，为了便于描述，这里可以将路面元素分为第一类路面元素和第二类路面元素，其中第一类路面元素指的是可以提取轮廓的路面元素，比如上述提到的指向标志、虚拟段和斑马线，第二类路面元素指无法提取到轮廓的线型路面元素，比如上述提到的实线和停止线。Exemplarily, there are two types of pavement element categories, one is pavement elements whose outlines can be extracted, such as the above-mentioned directional signs, dotted line segments and zebra crossings, and the other is pavement elements whose outlines cannot be extracted, such as the above-mentioned pavement elements. The mentioned solid line and stop line, for the convenience of description, the pavement elements can be divided into the first type of pavement elements and the second type of pavement elements, where the first type of pavement elements refers to the pavement elements that can extract contours, such as the above The mentioned pointing signs, virtual segments and zebra crossings, the second type of pavement elements refer to linear pavement elements whose contours cannot be extracted, such as the above-mentioned solid lines and stop lines.
示例性地，可以通过预先训练的语义分割模型对采集设备采集的路面元素进行语义分割，得到路面图像中包含的至少一个路面元素，以及每个路面元素的路面元素类别。Exemplarily, the pavement elements collected by the collection device may be semantically segmented through a pre-trained semantic segmentation model, to obtain at least one pavement element included in the pavement image, and the pavement element category of each pavement element.
进一步地，在确定出每个路面元素的路面元素类别后，可以按照该路面元素的路面元素类别，在路面图像中提取该路面元素包含的第一特征点集的像素坐标。Further, after the pavement element category of each pavement element is determined, the pixel coordinates of the first feature point set included in the pavement element can be extracted from the pavement image according to the pavement element category of the pavement element.
进一步地，将提取得到的每个路面元素包含的第一特征点集的像素坐标，作为该路面元素的像素坐标。Further, the extracted pixel coordinates of the first feature point set included in each road surface element are taken as the pixel coordinates of the road surface element.
本公开实施例中，可有按照语义分割方式，快速提取出路面图像中包含的路面元素，并可以按照预先设定的路面元素类别，快速提取到路面图像中的路面元素包含的第一特征点集的像素坐标，得到路面元素的像素坐标。In the embodiment of the present disclosure, the pavement elements included in the pavement image can be quickly extracted according to the semantic segmentation method, and the first feature points included in the pavement elements in the pavement image can be quickly extracted according to the preset pavement element categories Set the pixel coordinates to get the pixel coordinates of the pavement element.
具体地，针对上述2012，在按照每个路面元素的路面元素类别，在当前路面图像中提取该路面元素包含的第一特征点集的像素坐标时，包括以下S20121～S20123：Specifically, for the above 2012, when extracting the pixel coordinates of the first feature point set included in the road surface element in the current road surface image according to the road surface element category of each road surface element, the following steps S20121 to S20123 are included:
S20121，在该路面元素为第一类路面元素的情况下，在该路面元素的轮廓上提取多个第一特征点，得到该路面元素的第一特征点集。S20121 , in the case that the pavement element is the first type of pavement element, extract a plurality of first feature points on the outline of the pavement element to obtain a first feature point set of the pavement element.
示例性地，针对可以提取到完整轮廓的路面元素，比如指向标志、虚线段和斑马线，可以在这类路面元素的轮廓上提取第一特征点，这些第一特征点的连线能够构成该路面元素的轮廓，从而得到路面元素的第一特征点集。Exemplarily, for pavement elements whose complete contours can be extracted, such as pointing signs, dotted line segments and zebra crossings, first feature points can be extracted on the contours of such pavement elements, and the connecting lines of these first feature points can constitute the pavement. The contour of the element is obtained to obtain the first feature point set of the pavement element.
S20122，在该路面元素为第二类路面元素的情况下，在该路面元素的中心线上按照设定距离间隔提取第一特征点，得到该路面元素的第一特征点集。S20122 , in the case that the pavement element is a pavement element of the second type, extract first feature points on the centerline of the pavement element according to a set distance interval to obtain a first feature point set of the pavement element.
示例性地，针对无法提取到轮廓的路面元素，比如实现、停止线等，可以在这类路面元素的中心线上按照设定距离间隔提取第一特征点，比如每间隔n个像素点提取一个第一特征点，按照该方式可以提取到该路面元素的第一特征点集。Exemplarily, for pavement elements whose contours cannot be extracted, such as realizations, stop lines, etc., the first feature points can be extracted on the center line of such pavement elements according to a set distance interval, for example, one feature point is extracted every n pixel points. The first feature point, in this way, the first feature point set of the road surface element can be extracted.
S20123，在当前路面图像对应的像素坐标系中，获取第一特征点集中的每个第一特征点的像素坐标。S20123, in the pixel coordinate system corresponding to the current road surface image, obtain the pixel coordinates of each first feature point in the first feature point set.
示例性地，在提取到路面元素的第一特征点集后，可以在路面图像元素对应的像素坐标系中，提取第一特征点集中的每个第一特征点的像素坐标。Exemplarily, after the first feature point set of the pavement element is extracted, the pixel coordinates of each first feature point in the first feature point set may be extracted in the pixel coordinate system corresponding to the pavement image element.
本公开实施例中，在针对每个路面元素进行特征点提取时，可以按照该路面元素具体的路面元素类别进行特征点提取，从而快速得到每种路面元素包含的第一特征点集，以及第一特征点集中各第一特征点的像素坐标。In the embodiment of the present disclosure, when extracting feature points for each pavement element, the feature point extraction can be performed according to the specific pavement element category of the pavement element, so as to quickly obtain the first feature point set included in each pavement element, and the third The pixel coordinates of each first feature point in a feature point set.
在一种可能的实施方式中，针对上述S103，在基于预制地图，确定路面元素的第二世界坐标时，可以包括以下S1031～S1033：In a possible implementation manner, for the above S103, when determining the second world coordinates of the road surface element based on the prefabricated map, the following steps S1031 to S1033 may be included:
S1031，获取预先存储的路面元素的第二特征点集在预制地图中的地图位置信息；S1031, obtaining pre-stored map position information of the second feature point set of the road surface element in the pre-made map;
S1032，基于路面元素的第二特征点集的地图位置信息和目标车辆在预制地图表征的道路场景中的位姿数据，确定路面元素的第二特征点集的世界坐标；S1032, determining the world coordinates of the second feature point set of the pavement element based on the map position information of the second feature point set of the pavement element and the pose data of the target vehicle in the road scene represented by the prefabricated map;
S1033，将第二特征点集的世界坐标作为第二特征点集对应的路面元素的第二世界坐标。S1033 , taking the world coordinates of the second feature point set as the second world coordinates of the road surface element corresponding to the second feature point set.
示例性地，预制地图可以是根据预先采集的目标车辆行驶的道路场景包含的道路场景图像构建的，比如可以预先构建道路场景对应的三维场景地图，在道路场景对应的三维场景地图中可以预先存储道路中的路面元素的第二特征点集在预制地图中的地图位置信息。Exemplarily, the prefabricated map may be constructed according to pre-collected road scene images included in the road scene where the target vehicle is traveling. For example, a three-dimensional scene map corresponding to the road scene may be constructed in advance, and the three-dimensional scene map corresponding to the road scene may be stored in advance The second feature points of the pavement elements in the road set the map position information in the prefabricated map.
考虑到预制地图和道路场景中的同一路面元素在相同的坐标系中可以按照1：1进行呈现，这里可以在将预制地图和道路场景对齐后，确定出目标车辆在预制地图中的位姿数据，进而可以基于路面元素的第二特征点集在预制地图中的地图位置信息以及目标车辆在预制地图中的位姿数据，确定在根据目标车辆建立的世界坐标系中路面元素的第二特征点集的世界坐标。Considering that the same road surface elements in the prefabricated map and the road scene can be presented 1:1 in the same coordinate system, the pose data of the target vehicle in the prefabricated map can be determined after aligning the prefabricated map and the road scene. , and then based on the map position information of the second feature point set of the pavement element in the prefabricated map and the pose data of the target vehicle in the prefabricated map, determine the second feature point of the pavement element in the world coordinate system established according to the target vehicle set of world coordinates.
示例性地，目标车辆在道路场景中的位姿数据可以包括目标车辆在道路场景中的位置坐标和朝向，该位姿数据可以基于采集设备采集的路面图像以及即时定位与地图构建(Simultaneous Localization And Mapping，SLAM)的方式确定，也可以基于采集设备上安装的组合导航设备来确定，在此不做具体限定。Exemplarily, the pose data of the target vehicle in the road scene may include the position coordinates and orientation of the target vehicle in the road scene, and the pose data may be based on road images collected by the collection device and real-time positioning and map construction. Mapping, SLAM), or can also be determined based on the integrated navigation device installed on the acquisition device, which is not specifically limited here.
本公开实施例中，通过预先存储的路面元素在预制地图中的地图位置信息以及目标车辆在预制 地图表征的道路场景中的位姿数据，可以快速确定出路面元素包含的第二特征点集的世界坐标，从而得到路面元素的第二世界坐标。In the embodiment of the present disclosure, by using the pre-stored map position information of the pavement element in the prefabricated map and the pose data of the target vehicle in the road scene represented by the prefabricated map, the location of the second feature point set included in the pavement element can be quickly determined. world coordinates to get the second world coordinates of the pavement element.
在一种可能的实施方式中，针对上述S104，在根据路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中的目标对象的目标世界坐标时，如图3所示，包括以下S301～S303：In a possible implementation, for the above S104, when determining the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements, as shown in FIG. 3 , including the following S301 to S303:
S301，基于路面元素的第一世界坐标和第二世界坐标，对历史单应性矩阵进行调整，得到目标车辆的当前单应性矩阵。S301 , based on the first world coordinates and the second world coordinates of the road surface elements, adjust the historical homography matrix to obtain the current homography matrix of the target vehicle.
示例性地，如果历史单应性矩阵是准确的，基于采集设备的历史单应性矩阵得到同一路面元素的第一特征点集和第二特征点集中匹配的特征点在世界坐标系下的世界坐标应该相同，在同一路面元素的第一特征点集和第二特征点集中匹配的特征点在世界坐标系下的世界坐标不相同的情况下，可以确定历史单应性矩阵不再准确，此时可以基于同一路面元素的第一特征点集和第二特征点集在世界坐标系下的世界坐标，对历史单应性矩阵进行调整，得到目标车辆的当前单应性矩阵。Exemplarily, if the historical homography matrix is accurate, based on the historical homography matrix of the acquisition device, the world coordinate system of the matching feature points in the first feature point set and the second feature point set of the same road surface element is obtained. The coordinates should be the same. If the matched feature points in the first feature point set and the second feature point set of the same pavement element have different world coordinates in the world coordinate system, it can be determined that the historical homography matrix is no longer accurate. At the same time, the historical homography matrix can be adjusted based on the world coordinates of the first feature point set and the second feature point set of the same road surface element in the world coordinate system to obtain the current homography matrix of the target vehicle.
S302，基于目标图像，确定目标对象在目标图像中的像素坐标。S302, based on the target image, determine the pixel coordinates of the target object in the target image.
示例性地，得到目标图像后，可以基于图像识别技术，得到目标对象对应的检测框，将检测框与地面的切线的中心位置点作为可以表征目标对象在目标图像中的位置点，然后提取该位置点的像素坐标作为目标对象在图像坐标系下的像素坐标。Exemplarily, after the target image is obtained, the detection frame corresponding to the target object can be obtained based on the image recognition technology, and the center position point of the tangent between the detection frame and the ground is used as the position point that can characterize the target object in the target image, and then the detection frame is extracted. The pixel coordinates of the position point are used as the pixel coordinates of the target object in the image coordinate system.
S303，基于目标对象的像素坐标和当前单应性矩阵，确定目标对象的目标世界坐标。S303 , based on the pixel coordinates of the target object and the current homography matrix, determine the target world coordinates of the target object.
在得到目标对象在图像坐标系下的像素坐标后，将该目标对象在图像坐标系下的像素坐标和当前单应性矩阵输入像素坐标和世界坐标的转换方程中，即可以得到目标对象在世界坐标系下的世界坐标。After obtaining the pixel coordinates of the target object in the image coordinate system, input the pixel coordinates of the target object in the image coordinate system and the current homography matrix into the conversion equation of pixel coordinates and world coordinates, that is, the target object in the world can be obtained. World coordinates in the coordinate system.
本公开实施例中，可以根据路面元素的第一世界坐标和第二世界坐标之间的差距，对目标车辆的历史单应性矩阵修正，得到准确度较高的当前单应性矩阵，进一步基于准确度较高的当前单应性矩阵以及目标对象的像素坐标，可以得到确定目标对象准确度较高的目标世界坐标。In the embodiment of the present disclosure, the historical homography matrix of the target vehicle can be corrected according to the difference between the first world coordinates and the second world coordinates of the road surface elements, so as to obtain the current homography matrix with higher accuracy, and further based on The current homography matrix with high accuracy and the pixel coordinates of the target object can obtain the target world coordinates with high accuracy for determining the target object.
在一种可能的实施方式中，在确定目标对象的目标世界坐标之后，定位方法还包括：In a possible implementation manner, after determining the target world coordinates of the target object, the positioning method further includes:
基于目标对象的目标世界坐标以及预设位置点的位置信息，确定目标对象与预设位置点之间的距离。Based on the target world coordinates of the target object and the position information of the preset position point, the distance between the target object and the preset position point is determined.
示例性地，以目标车辆为例，这里的预设位置点可以是目标车辆的车前轴中心点在地面的投影，也可以是目标车辆的车体中心在地面的投影，其作为世界坐标系的原点时，该原点的在世界坐标系中的坐标为已知的，可以将该预设位置点作为在测量目标对象与目标车辆的距离时对应的车辆测距点。Exemplarily, taking the target vehicle as an example, the preset position point here can be the projection of the center point of the front axle of the target vehicle on the ground, or the projection of the center of the vehicle body of the target vehicle on the ground, which is used as the world coordinate system. When the origin of the origin is known, the coordinates of the origin in the world coordinate system are known, and the preset position point can be used as the vehicle ranging point corresponding to the distance between the target object and the target vehicle.
具体地，可以根据上述确定的目标对象在世界坐标系下的世界坐标以及预设位置点的世界坐标，计算两者的欧式距离，即可以确定目标对象与目标车辆之间的距离。Specifically, the Euclidean distance can be calculated according to the determined world coordinates of the target object in the world coordinate system and the world coordinates of the preset position point, so as to determine the distance between the target object and the target vehicle.
本公开实施例得到目标车辆的当前单应性矩阵后，能够利用该当前单应性矩阵准确地确定目标对象的目标世界坐标，进而可以得到预制位置点与目标对象之间准确度较高的距离。After obtaining the current homography matrix of the target vehicle in the embodiment of the present disclosure, the current homography matrix can be used to accurately determine the target world coordinates of the target object, and then the distance between the prefabricated position point and the target object can be obtained with high accuracy .
在一种实施方式中，针对上述S301，在基于路面元素的第一世界坐标和第二世界坐标，对历史单应性矩阵进行调整，得到目标车辆的当前单应性矩阵时，如图4所示，包括以下S401～S402：In an embodiment, for the above S301, when the historical homography matrix is adjusted based on the first world coordinates and the second world coordinates of the road surface elements to obtain the current homography matrix of the target vehicle, as shown in FIG. 4 display, including the following S401-S402:
S401，基于路面元素的第一特征点集的世界坐标和路面元素的第二特征点集的世界坐标，确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵；S401, based on the world coordinates of the first feature point set of the pavement element and the world coordinates of the second feature point set of the pavement element, determine the target affine matrix between the first feature point set and the second feature point set of the same pavement element ;
S402，基于目标仿射矩阵，对历史单应性矩阵进行修正，得到目标车辆的当前单应性矩阵。S402, based on the target affine matrix, correct the historical homography matrix to obtain the current homography matrix of the target vehicle.
示例性地，在历史单应性矩阵的不再准确的情况下，同一路面元素投影至世界坐标系下的位置相比基于预制地图确定的在世界坐标系下的位置会出现偏差，如图5所示，虚线表示的是基于历史单应性矩阵确定的路面元素在世界坐标系下的第一世界坐标表示的位置，实线表示的是基于预制地图确定的路面元素在世界坐标系下的第二世界坐标表示的位置，可见在历史单应性矩阵不再准确时，两者之间存在一定的偏差。Exemplarily, when the historical homography matrix is no longer accurate, the projected position of the same road surface element in the world coordinate system will deviate from the position in the world coordinate system determined based on the pre-made map, as shown in Figure 5. As shown, the dotted line represents the position of the first world coordinate of the pavement element in the world coordinate system determined based on the historical homography matrix, and the solid line represents the first world coordinate of the pavement element determined based on the pre-made map in the world coordinate system. The position represented by the two world coordinates shows that when the historical homography matrix is no longer accurate, there is a certain deviation between the two.
示例性地，本公开实施例提出通过路面元素的特征点来表示路面元素的几何形状，因此，可以通过路面元素包含的特征点集在世界坐标系下的世界坐标来表示路面元素在世界坐标系下的位置。Exemplarily, the embodiment of the present disclosure proposes to represent the geometric shape of the pavement element by the feature points of the pavement element. Therefore, the pavement element in the world coordinate system can be represented by the world coordinates of the feature point set contained in the pavement element in the world coordinate system. position below.
示例性地，由图5可见，同一路面元素在世界坐标系下的两种投影结果(通过历史单应性矩阵进行投影得到以及通过目标车辆在预制地图中的位姿投影得到)可以通过仿射变换进行转换，比如L1和L2、M1和M2，H1和H2、W1和W2、以及S1和S2分别表示同一路面元素在世界坐标系下的两种投影结果，目标仿射矩阵能够表示出同一路面元素的两种投影结果之间的转换关系，比如可以表示路面元素L1在世界坐标系中的位置经过怎样的变化，能够与路面元素L2在世界坐标系中的位置重合，考虑到通过路面元素的特征点在世界坐标系中的位置表示该路面元素在世界坐标系中的位置，因此这里提出可以基于第一特征点集和第二特征点集在世界坐标系下的世界坐标，确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵。Exemplarily, it can be seen from Fig. 5 that the two projection results of the same road surface element in the world coordinate system (obtained by the historical homography matrix projection and by the pose projection of the target vehicle in the pre-made map) can be obtained by affine Transform to convert, such as L1 and L2, M1 and M2, H1 and H2, W1 and W2, and S1 and S2 respectively represent the two projection results of the same road surface element in the world coordinate system, the target affine matrix can represent the same road surface The conversion relationship between the two projection results of the element, for example, can indicate how the position of the pavement element L1 in the world coordinate system has changed, and it can coincide with the position of the pavement element L2 in the world coordinate system. The position of the feature point in the world coordinate system represents the position of the pavement element in the world coordinate system. Therefore, it is proposed here that the same pavement element can be determined based on the world coordinates of the first feature point set and the second feature point set in the world coordinate system. The target affine matrix between the first feature point set and the second feature point set.
示例性地，通过路面图像提取得到的路面元素可以包括多个，根据预制地图确定的在世界坐标系下路面元素的第二特征点集的世界坐标同样包括多个路面元素的第二特征点集的世界坐标，因此在确定目标仿射矩阵之前，需要先基于各路面元素的第一特征点集和各路面元素的第二特征点集在世界坐标系下的世界坐标，确定同一路面元素的第一特征点集和第二特征点集。Exemplarily, the pavement elements obtained by extracting the pavement image may include multiple ones, and the world coordinates of the second feature point set of the pavement elements in the world coordinate system determined according to the prefabricated map also include multiple second feature point sets of the pavement elements. Therefore, before determining the target affine matrix, it is necessary to determine the first feature point set of each pavement element and the second feature point set of each pavement element in the world coordinate system to determine the first feature point set of the same pavement element. A feature point set and a second feature point set.
示例性地，可以采用最近邻法来确定同一路面元素的第一特征点集和第二特征点集，针对第一类路面元素，比如指向标志，可以根据每个路面元素的重心的世界坐标确定与该路面元素距离最近的路面元素作为同一路面元素，比如上述图5中指向标志M1和M2为同一第一类路面元素；针对第二类路面元素，比如直线型路面元素，可以将直线间距离将最近的两条直线作为同一路面元素，比如上述图5中，直线L1和直线L2为同一第二类路面元素。Exemplarily, the nearest neighbor method can be used to determine the first feature point set and the second feature point set of the same pavement element, and for the first type of pavement element, such as a pointing sign, it can be determined according to the world coordinate of the center of gravity of each pavement element. The pavement element with the closest distance to the pavement element is regarded as the same pavement element. For example, the pointing signs M1 and M2 in the above Figure 5 are the same first type of pavement element; for the second type of pavement element, such as a straight pavement element, the distance between the straight lines The two nearest straight lines are regarded as the same road surface element. For example, in the above FIG. 5 , the straight line L1 and the straight line L2 are the same second type of road surface element.
进一步地，在得到目标仿射矩阵后，可以基于该目标仿射矩阵对历史单应性矩阵进行修正，具体可以使用以下公式(2)来确定当前单应性矩阵：Further, after the target affine matrix is obtained, the historical homography matrix can be modified based on the target affine matrix. Specifically, the following formula (2) can be used to determine the current homography matrix:
H new＝AH old             (2)； H new = AH old (2);
其中，H new表示采集设备的当前单应性矩阵；A表示目标仿射矩阵；H old表示历史单应性矩阵。 Wherein, H new represents the current homography matrix of the acquisition device; A represents the target affine matrix; H old represents the historical homography matrix.
本公开实施例中，提出通过确定同一路面元素的第一特征点集和第二特征点集，可以得到用于表示历史单应性矩阵和当前单应性矩阵之间的差距的目标仿射矩阵，因此可以基于该目标仿射矩阵来对历史单应性矩阵进行修正，从而得到准确度较高的当前单应性矩阵。In the embodiment of the present disclosure, it is proposed that by determining the first feature point set and the second feature point set of the same pavement element, a target affine matrix for representing the gap between the historical homography matrix and the current homography matrix can be obtained. , so the historical homography matrix can be modified based on the target affine matrix, so as to obtain the current homography matrix with higher accuracy.
此外，使用基于世界坐标系下的仿射变换更新来间接更新单应矩阵的方法，能便捷稳定地基于上一次调整后的历史单应矩阵对其进行迭代更新，从而得到准确度较高的目标单应性矩阵。In addition, using the method of indirectly updating the homography matrix based on the affine transformation update in the world coordinate system, it can be easily and stably updated based on the historical homography matrix after the last adjustment, so as to obtain a target with high accuracy Homography matrix.
具体地，路面元素包含多个，针对上述S401，在基于路面元素的第一特征点集和路面元素的第二特征点集的世界坐标，确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵时，如图6所示，包括以下S501～S503：Specifically, there are multiple pavement elements. For the above S401, the first feature point set and the second feature of the same pavement element are determined based on the world coordinates of the first feature point set of the pavement element and the second feature point set of the pavement element. The target affine matrix between point sets, as shown in Figure 6, includes the following S501-S503:
S501，按照当前仿射矩阵，对多个路面元素中的每个路面元素的第二特征点集的世界坐标进行更新，确定该路面元素的第二特征点集的更新后的世界坐标；当前仿射矩阵为初始预设仿射矩阵或者为针对初始预设仿射矩阵调整过至少一次的仿射矩阵。S501, according to the current affine matrix, update the world coordinates of the second feature point set of each pavement element in the plurality of pavement elements, and determine the updated world coordinates of the second feature point set of the pavement element; The affine matrix is an initial preset affine matrix or an affine matrix adjusted at least once for the initial preset affine matrix.
示例性地，初始预设仿射矩阵可以根据经验设定，进一步可以按照S501～S503的方式对该初始预设仿射矩阵进行调整，直至得到目标仿射矩阵。Exemplarily, the initial preset affine matrix may be set according to experience, and further the initial preset affine matrix may be adjusted according to the methods of S501 to S503 until the target affine matrix is obtained.
示例性地，在对每个路面元素的第二特征点集的世界坐标进行更新时，可以按照当前仿射矩阵对每个路面元素的第二特征点集中的每个第二特征点的世界坐标进行更新，然后得到该第二特征点更新后的世界坐标。Exemplarily, when updating the world coordinates of the second feature point set of each pavement element, the world coordinates of each second feature point in the second feature point set of each pavement element may be updated according to the current affine matrix. Update, and then obtain the updated world coordinates of the second feature point.
S502，基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值。S502, based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated world coordinates and the first feature of the second feature point set of the pavement element The residual value between the world coordinates of the point set.
示例性地，针对不同类型的路面元素，确定残差值的方式不同，针对提取轮廓上的点作为特征点的第一类路面元素，可以按照两个匹配的点之间的距离来确定该第一类路面元素的残差值，针对提取中心线上的点作为特征点的第二类路面元素，可以按照点到线的距离来确定该第二类路面元素的残差值，具体过程详见下文描述。Exemplarily, for different types of pavement elements, the residual values are determined in different ways. For the first type of pavement elements in which points on the contour are extracted as feature points, the first type of pavement element can be determined according to the distance between two matched points. The residual value of a type of pavement element. For the second type of pavement element that extracts the point on the center line as the feature point, the residual value of the second type of pavement element can be determined according to the distance from the point to the line. For details, please refer to the process. described below.
S503，基于多个路面元素的残差值，对当前仿射矩阵进行调整后，返回确定该路面元素的第二特征点集的更新后的世界坐标信息的步骤，直至满足预设截止条件后，得到目标仿射矩阵。S503, after adjusting the current affine matrix based on the residual values of the plurality of pavement elements, return to the step of determining the updated world coordinate information of the second feature point set of the pavement element, until the preset cut-off condition is met, Get the target affine matrix.
示例性地，在基于当前仿射矩阵得到多个路面元素中每个路面元素对应的残差值后，可以确定多个路面元素对应的残差值总和，然后根据该残差值总和对当前仿射矩阵进行调整，将调整后的仿射矩阵作为当前仿射矩阵，返回S501重新开始对多个路面元素中的每个路面元素的第二特征点集的世界坐标进行更新，直至满足预设截止条件后，得到目标仿射矩阵。Exemplarily, after obtaining the residual value corresponding to each pavement element among the multiple pavement elements based on the current affine matrix, the sum of the residual values corresponding to the multiple pavement elements may be determined, and then according to the sum of the residual values, the current simulat Adjust the affine matrix, take the adjusted affine matrix as the current affine matrix, and return to S501 to restart the update of the world coordinates of the second feature point set of each pavement element in the plurality of pavement elements until the preset deadline is met After the condition, the target affine matrix is obtained.
示例性地，预设截止条件可以包括以下一种或多种：Exemplarily, the preset cutoff conditions may include one or more of the following:
(1)调整次数达到预设调整次数；(1) The adjustment times reach the preset adjustment times;
(2)多个路面元素的残差值总和小于预设残差值阈值；(2) The sum of the residual values of multiple road surface elements is less than the preset residual value threshold;
(3)多个路面元素的当前残差值总和相比上一次调整后得到的残差值总和的变化量小于预设变化量。(3) The change amount of the sum of the current residual values of the plurality of road surface elements compared with the sum of the residual values obtained after the last adjustment is smaller than the preset change amount.
本公开实施例中，提出可以对仿射矩阵进行多次调整，直至得到准确度较高的目标仿射矩阵，以便后续可以基于准确度较高的目标仿射矩阵得到准确度较高的当前单应性矩阵。In the embodiment of the present disclosure, it is proposed that the affine matrix may be adjusted multiple times until a target affine matrix with higher accuracy is obtained, so that the current single target affine matrix with higher accuracy can be obtained subsequently based on the target affine matrix with higher accuracy. Responsiveness Matrix.
在一种实施方式中，路面元素包括第一类路面元素，比如上述提到的可以提取到轮廓点的指向标志、虚线段和斑马线，在确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵之前，本公开实施例提供的定位方法还包括：In one embodiment, the pavement elements include the first type of pavement elements, such as the above-mentioned directional signs, dashed line segments and zebra crossings that can be extracted to contour points. After determining the first feature point set and the second feature of the same pavement element Before the target affine matrix between the point sets, the positioning method provided by the embodiment of the present disclosure further includes:
针对每个第一类路面元素，在该第一类路面元素的第一特征点集中提取与该第一类路面元素的第二特征点集中的每个第二特征点匹配的第一特征点。For each first-type pavement element, a first feature point matching each second feature point in the second feature point set of the first-type pavement element is extracted from the first feature point set of the first-type pavement element.
示例性地，在第二特征点集中包含的第二特征点的个数较少时，还可以在得到在世界坐标系下第一类路面元素包含的第二特征点集的世界坐标后，再根据上采样的方式增加第二特征点的数量，然后在第一特征点集中确定与每个第二特征点匹配的第一特征点，示例性地，可以根据ICP算法(Iterative Closest Point迭代最近点)确定出在第一特征点集中确定与每个第二特征点匹配的第一特征点。Exemplarily, when the number of the second feature points included in the second feature point set is small, after obtaining the world coordinates of the second feature point set included in the first type of road surface elements in the world coordinate system, The number of second feature points is increased according to the up-sampling method, and then the first feature point matching each second feature point is determined in the first feature point set. ) to determine a first feature point matching each of the second feature points in the first feature point set.
具体地，针对上述S502，在基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值时，包括以下S5021～S5022：Specifically, for the above S502, based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated second feature point set of the pavement element. The residual value between the world coordinates and the world coordinates of the first feature point set includes the following S5021 to S5022:
S5021，针对每个第一类路面元素，基于该第一类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和与该第二特征点匹配的第一特征点的世界坐标，确定该第二特征点和匹配的第一特征点之间的距离。S5021, for each first-type pavement element, based on the updated world coordinates of each second feature point in the second feature point set of the first-type pavement element and the first feature point matching the second feature point The world coordinate, which determines the distance between the second feature point and the matched first feature point.
示例性地，这里确定该第二特征点和匹配的第一特征点之间的距离可以通过欧式距离计算公式确定。Exemplarily, the distance between the second feature point and the matched first feature point may be determined by the Euclidean distance calculation formula.
S5022，对该第一类路面元素包含的每个第二特征点和匹配的第一特征点之间的距离进行求和，得到该第一类路面元素对应的残差值。S5022 , summing the distances between each second feature point included in the first type of pavement element and the matched first feature point to obtain a residual value corresponding to the first type of pavement element.
进一步地，在得到每个第一类路面元素包含的每个第二特征点和匹配的第一特征点之间的距离后，可以对这些距离进行求和，得到该第一类路面元素对应的残差值。Further, after obtaining the distance between each second feature point contained in each first type of pavement element and the matched first feature point, these distances can be summed to obtain the corresponding first type of pavement element. residual value.
本公开实施例中，提出针对第一类路面元素，可以通过每个第一类路面元素包含的每个第二特征点和匹配的第一特征点在世界坐标系中的距离，来确定通过历史单应性矩阵确定的第一类路面元素的第一世界坐标和该第一类路面元素的第二世界坐标之间的残差值，以便后续通过对该残差值进行调整确定目标仿射矩阵。In the embodiment of the present disclosure, it is proposed that for the first type of pavement element, the distance of each second feature point included in each first type of pavement element and the matched first feature point in the world coordinate system can be used to determine the passing history The residual value between the first world coordinate of the first type of pavement element determined by the homography matrix and the second world coordinate of the first type of pavement element, so that the target affine matrix can be determined by adjusting the residual value subsequently .
在一种可能的实施方式中，路面元素还包括第二类路面元素，比如除了包含第一类路面元素外，还包括上述提到无法在路面图像中提取到轮廓点的实线和停止线，在确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵之前，本公开实施例提供的定位方法还包括：In a possible implementation manner, the pavement element further includes a second type of pavement element, for example, in addition to the first type of pavement element, it also includes the above-mentioned solid line and stop line whose contour points cannot be extracted from the pavement image, Before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, the positioning method provided by the embodiment of the present disclosure further includes:
针对每个第二类路面元素，按照该第二类路面元素的第一特征点集的世界坐标，确定该第二类路面元素的第一特征点集的拟合直线方程。For each second-type pavement element, according to the world coordinates of the first feature point set of the second-type pavement element, the fitting line equation of the first feature point set of the second-type pavement element is determined.
示例性地，以第二类路面元素为针对中心线进行特征点提取的路面元素为例，比如图5中L和S类型的路面元素，这类路面元素无法获取到路面元素完整的轮廓，可以在该路面元素的中心线上按照设定间隔提取第一特征点，得到第一特征点集，在基于历史单应性矩阵，得到该第二类路面元素的第一特征点集的世界坐标后，可以根据该第二类路面元素的第一特征点集的世界坐标，对该第二类路面元素的第一特征点集中的多个第一特征点进行线性拟合，得到该第二类路面元素的第一特征点集的拟合直线以及该拟合直线的拟合直线方程。Illustratively, take the second type of pavement elements as the pavement elements for extracting feature points for the centerline as an example, such as the L and S types of pavement elements in Figure 5, such pavement elements cannot obtain the complete outline of the pavement element, but The first feature point is extracted at the set interval on the center line of the pavement element to obtain the first feature point set. After obtaining the world coordinates of the first feature point set of the second type of pavement element based on the historical homography matrix , according to the world coordinates of the first feature point set of the second type of pavement element, a plurality of first feature points in the first feature point set of the second type of pavement element can be linearly fitted to obtain the second type of pavement The fitted straight line of the first feature point set of the element and the fitted straight line equation of the fitted straight line.
特别地，针对第二类路面元素为曲线的情况下，在对第二类路面元素的第一特征点集中的多个 第一特征点进行线性拟合时，可以按照相邻设定个数的第一特征点进行直线拟合，得到多个曲率不同的拟合直线，以及每段拟合直线对应的拟合直线方程。In particular, in the case where the second type of pavement element is a curve, when performing linear fitting on a plurality of first feature points in the first feature point set of the second type of pavement element, the adjacent set number of The first feature point is fitted with a straight line to obtain a plurality of fitted straight lines with different curvatures, and a fitted straight line equation corresponding to each segment of the fitted straight line.
具体地，针对上述S502，在基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值时，包括以下S5023～S5024：Specifically, for the above S502, based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated second feature point set of the pavement element. The residual value between the world coordinates and the world coordinates of the first feature point set includes the following S5023 to S5024:
S5023，针对每个第二类路面元素，基于该第二类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和该第二类路面元素对应的拟合直线方程，确定该第二特征点和该第二类路面元素对应的拟合直线之间的距离；S5023, for each second-type pavement element, based on the updated world coordinates of each second feature point in the second feature point set of the second-type pavement element and the fitting line equation corresponding to the second-type pavement element, determining the distance between the second feature point and the fitted straight line corresponding to the second type of road surface element;
S5024，基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值。S5024 , based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element, determine a residual value corresponding to the second type of pavement element.
示例性地，针对每个第一路面元素，可以基于点到直线的距离公式，确定出该第二类路面元素的第二特征点集中的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，比如路面图像中包括n个第二类路面元素，每个第二类路面元素的第二特征点集中包含m个第二特征点，这样针对每个第二类路面元素，可以确定出m个距离。Exemplarily, for each first pavement element, it may be determined that each second feature point in the second feature point set of the second type of pavement element corresponds to the second type of pavement element based on the distance formula from the point to the straight line. The distance between the fitted straight lines, for example, the pavement image includes n second-type pavement elements, and the second feature point set of each second-type pavement element contains m second-type feature points, so that for each second-type pavement element Pavement elements, m distances can be determined.
进一步地，在得到每个第二类路面元素包含的各第二特征点和该第二类路面元素对应的拟合直线之间的距离后，可以对这些距离进行求和，得到该第二类路面元素对应的残差值。Further, after obtaining the distance between each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element, these distances can be summed to obtain the second type of pavement element. The residual value corresponding to the pavement element.
本公开实施例中，提出针对第二类路面元素，可以通过每个第二类路面元素包含的每个第二特征点到该第二类路面元素对应的拟合直线之间的距离，来确定通过历史单应性矩阵确定的第二类路面元素的第一世界坐标和该第二类路面元素的第二世界坐标之间的残差值，以便后续通过对该残差值进行调整确定目标仿射矩阵。In the embodiment of the present disclosure, it is proposed that for the second type of pavement element, the distance between each second feature point included in each second type of pavement element and the fitted straight line corresponding to the second type of pavement element can be determined. The residual value between the first world coordinate of the second type of pavement element determined by the historical homography matrix and the second world coordinate of the second type of pavement element, so that the target simulation can be determined by adjusting the residual value subsequently. shot matrix.
示例性地，在路面图像中同时包含第一类路面元素和第二类路面元素的情况下，考虑到距离采集设备较远的区域中的第一类路面元素在路面图像中包含的第一特征点的数量较少，或者无法提取第一类特征点，即距离采集设备较远的区域包含的有效特征点的数量较少，因此，在基于第一类路面元素和第二类路面元素对历史单应性矩阵进行调整时，为了得到准确度更高的当前单应性矩阵，在确定路面元素对应的残差值总和(第一类路面元素和第二类路面元素的残差值之和)时，可以提高与采集设备距离较远的第二类路面元素的第二特征点的权重，因此在确定该第二类路面元素对应的残差值之前，本公开实施例提供的定位方法还包括：Exemplarily, when the road surface image contains both the first type of road surface element and the second type of road surface element, consider the first feature contained in the road surface image of the first type of road surface element in the area far from the acquisition device. The number of points is small, or the first type of feature points cannot be extracted, that is, the area far away from the collection device contains fewer effective feature points. When the homography matrix is adjusted, in order to obtain the current homography matrix with higher accuracy, the sum of the residual values corresponding to the pavement elements (the sum of the residual values of the first type of pavement elements and the second type of pavement elements) is determined. , the weight of the second feature point of the second type of pavement element that is far away from the collection device can be increased. Therefore, before determining the residual value corresponding to the second type of pavement element, the positioning method provided by the embodiment of the present disclosure further includes: :
基于该第二类路面元素包含的每个第二特征点的世界坐标，获取该第二类路面元素的每个第二特征点对应的距离权重。Based on the world coordinates of each second feature point included in the second type of pavement element, a distance weight corresponding to each second feature point of the second type of pavement element is obtained.
示例性地，可以通过以下公式(3)确定每个第二特征点对应的距离权重：Exemplarily, the distance weight corresponding to each second feature point can be determined by the following formula (3):
Figure PCTCN2021098697-appb-000002
Figure PCTCN2021098697-appb-000002
其中，λ k表示第k个第二特征点对应的距离权重；d(P k)为世界坐标系第k个第二特征点P k离世界坐标系原点的欧式距离；Δ表示预设距离。 Among them, λ k represents the distance weight corresponding to the k-th second feature point; d(P k ) is the Euclidean distance from the k-th second feature point P k of the world coordinate system to the origin of the world coordinate system; Δ represents the preset distance.
具体地，在基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值时，包括：Specifically, when determining the residual value corresponding to the second type of pavement element based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element, include:
基于该第二类路面元素包含的每个第二特征点对应的距离权重，对该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离进行加权求和，得到该第二类路面元素对应的残差值。Based on the distance weight corresponding to each second feature point included in the second type of pavement element, the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element The distance is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
示例性地，可以按照上述公式(3)确定出每个第二特征点对应的距离权重，这样在确定该第二类路面元素对应的残差值时，可以通过该第一路面元素包含的每个第二特征点对应的距离权重，对该第二特征点和该第二类路面元素对应的拟合直线之间的距离进行加权求和，从而使得残差值更加准确。Exemplarily, the distance weight corresponding to each second feature point can be determined according to the above formula (3), so that when determining the residual value corresponding to the second type of pavement element, each of the first pavement elements can be used. The distance weight corresponding to the second feature point is weighted and summed to the distance between the second feature point and the fitting straight line corresponding to the second type of road surface element, so that the residual value is more accurate.
本公开实施例中，在路面元素同时包含第一类路面元素和第二类路面元素时，考虑到距离目标车辆较远的区域第一类路面元素包含的有效特征点的数量较少，因此可以调整第二类路面元素在距离采集设备不同位置的第二特征点对应的距离权重，来平衡距离目标车辆远近区域包含的有效特征点不均衡的问题，从而使得得到的当前单应矩阵在衡量与目标车辆距离不同区域的路面和对应的图像之间的转换关系时均具有较高的鲁棒性和精度。In the embodiment of the present disclosure, when the pavement element includes both the first type of pavement element and the second type of pavement element, considering that the area far from the target vehicle contains the first type of pavement element, the number of effective feature points is small, so it can be Adjust the distance weights corresponding to the second feature points of the second type of pavement elements at different positions of the distance collection device to balance the problem of unbalanced effective feature points contained in the far and near areas of the target vehicle, so that the obtained current homography matrix is measured and The conversion relationship between the target vehicle and the road surface in different regions and the corresponding images has high robustness and accuracy.
综上，在路面图像中同时包含多个第一类路面元素和多个第二类路面元素时，可以通过以下公式(4)来确定目标仿射矩阵：To sum up, when the road surface image contains multiple first-type pavement elements and multiple second-type pavement elements at the same time, the target affine matrix can be determined by the following formula (4):
Figure PCTCN2021098697-appb-000003
Figure PCTCN2021098697-appb-000003
其中，E表示多个路面元素的残差值总和；Q表示路面图像中包含的第二类路面元素的总数；i表示第i个第二类路面元素；k表示第i个第二类路面元素中的第k个第二特征点；P k表示第i个第二类路面元素的第k个第二特征点在世界坐标系下的世界坐标；A表示当前仿射矩阵；l i表示第i个第二类路面元素对应的拟合直线；h(P k,A)表示基于当前仿射矩阵对第i个第二类路面元素的第k个第二特征点的世界坐标进行更新后得到的更新后的世界坐标；D l(h(P k,A),l i)表示基于第i个第二类路面元素的第k个第二特征点更新后的世界坐标和该第i个第二类路面元素对应的拟合直线方程，确定的第i个第二类路面元素的第k个第二特征点和第i个第二类路面元素对应的拟合直线之间的距离。 Among them, E represents the sum of the residual values of multiple road surface elements; Q represents the total number of second-type road surface elements contained in the road surface image; i represents the i-th second-type road surface element; k represents the i-th second-type road surface element The kth second feature point in ; P k represents the world coordinate of the kth second feature point of the ith second type of pavement element in the world coordinate system; A represents the current affine matrix; l i represents the ith The fitted straight line corresponding to the second type of pavement element; h(P k ,A) represents the result obtained by updating the world coordinate of the kth second feature point of the ith second type of pavement element based on the current affine matrix The updated world coordinates; D l (h(P k ,A),l i ) represents the updated world coordinates based on the kth second feature point of the ith second type road surface element and the ith second feature point The fitting line equation corresponding to the pavement-like element corresponds to the determined distance between the k-th second feature point of the i-th second-type pavement element and the fitting line corresponding to the i-th second-type pavement element.
其中，P表示路面图像中包含的第一类路面元素的总数；j表示第j个第一类路面元素；P j表示第j个第一类路面元素上的第j个第二特征点在世界坐标系下的世界坐标；p j表示与第j个第一类路面元素的第j个第二特征点匹配的第一特征点在世界坐标系下的世界坐标；h(P j,A)表示基于当前当前仿射矩阵对第j个第一类路面元素的第k个第二特征点的世界坐标进行更新后得到得更新后的世界坐标；D p(h(P j,A),p j)表示基于第j个第一类路面元素的第k个第二特征点更新后的世界坐标和匹配的第一特征点的世界坐标，确定第j个第一类路面元素的第k个第二特征点和匹配的第一特征点之间的距离。 Among them, P represents the total number of first-type road surface elements contained in the road surface image; j represents the j-th first-type road surface element; P j represents the j-th second feature point on the j-th first-type road surface element in the world The world coordinate in the coordinate system; p j represents the world coordinate of the first feature point in the world coordinate system that matches the j-th second feature point of the j-th first-type road surface element; h(P j , A) represents The updated world coordinates are obtained by updating the world coordinates of the kth second feature point of the jth first type pavement element based on the current current affine matrix; D p (h(P j ,A),p j ) indicates that based on the updated world coordinates of the kth second feature point of the jth first type pavement element and the world coordinate of the matched first feature point, the kth second feature point of the jth first type pavement element is determined. The distance between the feature point and the matched first feature point.
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
基于同一技术构思，本公开实施例中还提供了与定位方法对应的定位装置，由于本公开实施例 中的装置解决问题的原理与本公开实施例上述定位方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。Based on the same technical concept, the embodiment of the present disclosure also provides a positioning device corresponding to the positioning method. Since the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned positioning method of the embodiment of the present disclosure, the implementation of the device can refer to the method implementation, and the repetition will not be repeated.
参照图7所示，为本公开实施例提供的一种定位装置600的示意图，该定位装置600包括：Referring to FIG. 7 , which is a schematic diagram of a positioning device 600 according to an embodiment of the present disclosure, the positioning device 600 includes:
获取模块601，用于获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；An acquisition module 601, configured to acquire a current road image collected by the target vehicle and a prefabricated map of the current location of the target vehicle;
第一确定模块602，用于基于当前路面图像，确定当前路面图像中的路面元素的第一世界坐标；a first determining module 602, configured to determine, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image;
第二确定模块603，用于基于预制地图，确定路面元素的第二世界坐标；The second determining module 603 is configured to determine the second world coordinates of the road surface element based on the prefabricated map;
第三确定模块604，用于根据路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中目标对象的目标世界坐标。The third determining module 604 is configured to determine the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements.
在一种可能的实施方式中，在获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图之前，获取模块601还用于：In a possible implementation manner, before acquiring the current road surface image collected by the target vehicle and the prefabricated map of the current location of the target vehicle, the acquiring module 601 is further configured to:
响应于针对目标车辆的标定按键的触发操作，或者，接收到针对目标车辆进行标定的标定指令。In response to a triggering operation of a calibration button for the target vehicle, or, a calibration instruction for calibrating the target vehicle is received.
在一种可能的实施方式中，第一确定模块602在用于基于当前路面图像，确定当前路面图像中的路面元素的第一世界坐标时，包括：In a possible implementation manner, when the first determining module 602 is used to determine the first world coordinates of the road surface element in the current road surface image based on the current road surface image, the method includes:
获取当前路面图像中的路面元素的像素坐标；Get the pixel coordinates of the pavement elements in the current pavement image;
基于路面元素的像素坐标以及预先确定的采集设备的历史单应性矩阵，确定路面元素的第一世界坐标。The first world coordinates of the pavement elements are determined based on the pixel coordinates of the pavement elements and the predetermined historical homography matrix of the acquisition device.
在一种可能的实施方式中，第一确定模块602在用于获取当前路面图像中的路面元素的像素坐标时，包括：In a possible implementation manner, when the first determining module 602 is used to acquire the pixel coordinates of the road surface elements in the current road surface image, the method includes:
对当前路面图像进行语义分割，得到当前路面图像包含的至少一个路面元素，以及每个路面元素的路面元素类别；Semantically segment the current pavement image to obtain at least one pavement element contained in the current pavement image and the pavement element category of each pavement element;
按照每个路面元素的路面元素类别，在当前路面图像中提取该路面元素包含的第一特征点集的像素坐标；According to the pavement element category of each pavement element, extract the pixel coordinates of the first feature point set contained in the pavement element in the current pavement image;
将第一特征点集的像素坐标作为第一特征点集对应的路面元素的像素坐标。The pixel coordinates of the first feature point set are used as the pixel coordinates of the road surface element corresponding to the first feature point set.
在一种可能的实施方式中，第一确定模块602在用于按照每个路面元素的路面元素类别，在当前路面图像中提取该路面元素包含的第一特征点集的像素坐标时，包括：In a possible implementation manner, when the first determining module 602 is configured to extract the pixel coordinates of the first feature point set included in the pavement element in the current pavement image according to the pavement element category of each pavement element, the method includes:
在该路面元素为第一类路面元素的情况下，在该路面元素的轮廓上提取多个第一特征点，得到该路面元素的第一特征点集；In the case that the pavement element is the first type of pavement element, extract a plurality of first feature points on the outline of the pavement element to obtain a first feature point set of the pavement element;
在该路面元素为第二类路面元素的情况下，在该路面元素的中心线上按照设定距离间隔提取第一特征点，得到该路面元素的第一特征点集；In the case that the pavement element is the second type of pavement element, extract the first feature points on the center line of the pavement element according to the set distance interval, and obtain the first feature point set of the pavement element;
在当前路面图像对应的像素坐标系中，获取第一特征点集中的每个第一特征点的像素坐标。In the pixel coordinate system corresponding to the current road surface image, the pixel coordinates of each first feature point in the first feature point set are acquired.
在一种可能的实施方式中，第二确定模块602在用于基于预制地图，确定路面元素的第二世界坐标时，包括：In a possible implementation manner, when the second determining module 602 is used to determine the second world coordinates of the road surface element based on the prefabricated map, the method includes:
获取预先存储的路面元素的第二特征点集在预制地图中的地图位置信息；obtaining the map position information of the second feature point set of the pre-stored pavement element in the pre-made map;
基于路面元素的第二特征点集的地图位置信息和目标车辆在预制地图表征的道路场景中的位姿数据，确定路面元素的第二特征点集的世界坐标；Determine the world coordinates of the second feature point set of the pavement element based on the map position information of the second feature point set of the pavement element and the pose data of the target vehicle in the road scene represented by the prefabricated map;
将第二特征点集的世界坐标作为第二特征点集对应的路面元素的第二世界坐标。The world coordinate of the second feature point set is used as the second world coordinate of the road surface element corresponding to the second feature point set.
在一种可能的实施方式中，第三确定模块604在用于根据路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中的目标对象的目标世界坐标时，包括：In a possible implementation manner, when the third determining module 604 is used to determine the target world coordinates of the target object in the target image collected by the target vehicle according to the first world coordinates and the second world coordinates of the road surface elements, the method includes:
基于路面元素的第一世界坐标和第二世界坐标，对历史单应性矩阵进行调整，得到目标车辆的当前单应性矩阵；Based on the first world coordinates and the second world coordinates of the road surface elements, adjust the historical homography matrix to obtain the current homography matrix of the target vehicle;
基于目标图像，确定目标对象在目标图像中的像素坐标；Based on the target image, determine the pixel coordinates of the target object in the target image;
基于目标对象的像素坐标和当前单应性矩阵，确定目标对象的目标世界坐标。Based on the pixel coordinates of the target object and the current homography matrix, the target world coordinates of the target object are determined.
在一种可能的实施方式中，在确定目标对象的目标世界坐标之后，第三确定模块604还用于：In a possible implementation manner, after determining the target world coordinates of the target object, the third determining module 604 is further configured to:
基于目标对象的目标世界坐标以及预设位置点的位置信息，确定目标对象与预设位置点之间的距离。Based on the target world coordinates of the target object and the position information of the preset position point, the distance between the target object and the preset position point is determined.
在一种可能的实施方式中国，第三确定模块604在用于基于路面元素的第一世界坐标和第二世界坐标，对历史单应性矩阵进行调整，得到目标车辆的当前单应性矩阵时，包括：In a possible implementation manner, the third determination module 604 is used to adjust the historical homography matrix based on the first world coordinates and the second world coordinates of the road surface elements to obtain the current homography matrix of the target vehicle. ,include:
基于路面元素的第一特征点集的世界坐标和路面元素的第二特征点集的世界坐标，确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵；Based on the world coordinates of the first feature point set of the pavement element and the world coordinates of the second feature point set of the pavement element, determine the target affine matrix between the first feature point set and the second feature point set of the same pavement element;
基于目标仿射矩阵，对历史单应性矩阵进行修正，得到目标车辆的当前单应性矩阵。Based on the target affine matrix, the historical homography matrix is modified to obtain the current homography matrix of the target vehicle.
在一种可能的实施方式中，路面元素包含多个，第三确定模块604在用于基于路面元素的第一特征点集的世界坐标和路面元素的第二特征点集的世界坐标，确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵时，包括：In a possible implementation manner, the pavement element includes multiple, and the third determination module 604 is used to determine the same When the target affine matrix between the first feature point set and the second feature point set of the pavement element, includes:
按照当前仿射矩阵，对多个路面元素中的每个路面元素的第二特征点集的世界坐标进行更新，确定该路面元素的第二特征点集的更新后的世界坐标；当前仿射矩阵为初始预设仿射矩阵或者为针对初始预设仿射矩阵调整过至少一次的仿射矩阵；According to the current affine matrix, the world coordinates of the second feature point set of each road surface element in the plurality of road surface elements are updated, and the updated world coordinates of the second feature point set of the road surface element are determined; the current affine matrix is an initial preset affine matrix or an affine matrix adjusted at least once for the initial preset affine matrix;
基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值；Based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated world coordinates and the first feature point set of the second feature point set of the pavement element The residual value between the world coordinates of ;
基于多个路面元素的残差值，对当前仿射矩阵进行调整后，返回确定该路面元素的第二特征点集的更新后的世界坐标的步骤，直至满足预设截止条件后，得到目标仿射矩阵。Based on the residual values of multiple pavement elements, after adjusting the current affine matrix, return to the step of determining the updated world coordinates of the second feature point set of the pavement element, until the preset cutoff condition is met, and the target affine matrix is obtained. shot matrix.
在一种可能的实施方式中，路面元素包括第一类路面元素，在确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵之前，第三确定模块604还用于：In a possible implementation, the pavement element includes the first type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, the third determining module 604 further Used for:
针对每个第一类路面元素，在该第一类路面元素的第一特征点集中提取与该第一类路面元素的第二特征点集中的每个第二特征点匹配的第一特征点；For each first-type pavement element, extracting a first feature point matching each second feature point in the second feature point set of the first-type pavement element from the first feature point set of the first-type pavement element;
第三确定模块604在用于基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值时，包括：The third determination module 604 is configured to determine the updated second feature point set of the pavement element based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element When the residual value between the world coordinates and the world coordinates of the first feature point set, includes:
针对每个第一类路面元素，基于该第一类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和与该第二特征点匹配的第一特征点的世界坐标，确定该第二特征点和匹配的第一特征点之间的距离；For each first-type pavement element, based on the updated world coordinates of each second feature point in the second feature point set of the first-type pavement element and the world coordinates of the first feature point matching the second feature point , determine the distance between the second feature point and the matched first feature point;
对该第一类路面元素包含的每个第二特征点和匹配的第一特征点之间的距离进行求和，得到该第一类路面元素对应的残差值。Summing the distances between each second feature point included in the first type of pavement element and the matched first feature point to obtain a residual value corresponding to the first type of pavement element.
在一种可能的实施方式中，路面元素还包括第二类路面元素，在确定同一路面元素的第一特征点集和第二特征点集之间的目标仿射矩阵之前，第三确定模块604还用于：In a possible implementation, the pavement element further includes a second type of pavement element, and before determining the target affine matrix between the first feature point set and the second feature point set of the same pavement element, the third determining module 604 Also used for:
针对每个第二类路面元素，按照该第二类路面元素的第一特征点集的世界坐标，确定该第二类路面元素的第一特征点集的拟合直线方程；For each second-type pavement element, according to the world coordinates of the first feature point set of the second-type pavement element, determine the fitted line equation of the first feature point set of the second-type pavement element;
第三确定模块604在用于基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值时，包括：The third determination module 604 is configured to determine the updated second feature point set of the pavement element based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element When the residual value between the world coordinates and the world coordinates of the first feature point set, includes:
针对每个第二类路面元素，基于该第二类路面元素的第二特征点集中的每个第二特征点更新后的世界坐标和该第二类路面元素对应的拟合直线方程，确定该第二特征点和该第二类路面元素对应的拟合直线之间的距离；For each second type of pavement element, determine the the distance between the second feature point and the fitted straight line corresponding to the second type of pavement element;
基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值。Based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element, the residual value corresponding to the second type of pavement element is determined.
在一种可能的实施方式中，在确定该第二类路面元素对应的残差值之前，第三确定模块604还用于：In a possible implementation manner, before determining the residual value corresponding to the second type of road surface element, the third determining module 604 is further configured to:
基于该第二类路面元素包含的每个第二特征点的世界坐标，获取该第二类路面元素的每个第二特征点对应的距离权重；Based on the world coordinates of each second feature point included in the second type of pavement element, obtain a distance weight corresponding to each second feature point of the second type of pavement element;
第三确定模块604在用于基于该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值时，包括：The third determination module 604 is used to determine the residual corresponding to the second type of pavement element based on the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element difference, including:
基于该第二类路面元素包含的每个第二特征点对应的距离权重，对该第二类路面元素包含的每个第二特征点和该第二类路面元素对应的拟合直线之间的距离进行加权求和，得到该第二类路面元素对应的残差值。Based on the distance weight corresponding to each second feature point included in the second type of pavement element, the distance between each second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element The distance is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
关于装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.
对应于图1中的定位方法，本公开实施例还提供了一种电子设备700，如图8所示，为本公开实施例提供的电子设备700结构示意图，包括：Corresponding to the positioning method in FIG. 1 , an embodiment of the present disclosure further provides an electronic device 700 . As shown in FIG. 8 , the schematic structural diagram of the electronic device 700 provided by the embodiment of the present disclosure includes:
处理器71、存储器72、和总线73；存储器72用于存储执行指令，包括内存721和外部存储器722；这里的内存721也称内存储器，用于暂时存放处理器71中的运算数据，以及与硬盘等外部存储器722交换的数据，处理器71通过内存721与外部存储器722进行数据交换，当电子设备700运行时，处理器71与存储器72之间通过总线73通信，使得处理器71执行以下指令：获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；基于当前路面图像，确定当前路面图像中的路面元素的第一世界坐标；基于预制地图，确定路面元素的第二世界坐标；根据路面元素的第一世界坐标、第二世界坐标，确定目标车辆采集的目标图像中的目标对象的目标世界坐标。The processor 71, the memory 72, and the bus 73; the memory 72 is used to store execution instructions, including the memory 721 and the external memory 722; the memory 721 here is also called internal memory, which is used to temporarily store the operation data in the processor 71, and For the data exchanged by the external memory 722 such as the hard disk, the processor 71 exchanges data with the external memory 722 through the memory 721. When the electronic device 700 is running, the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the following instructions : Obtain the current road image collected by the target vehicle and a pre-made map of the current location of the target vehicle; based on the current road image, determine the first world coordinates of the road elements in the current road image; based on the pre-made map, determine the second world of the road elements Coordinates: According to the first world coordinates and the second world coordinates of the road surface elements, determine the target world coordinates of the target object in the target image collected by the target vehicle.
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序， 该计算机程序被处理器运行时执行上述方法实施例中的定位方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the positioning method in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.
本公开实施例还提供一种计算机程序产品，该计算机程序产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的定位方法的步骤，具体可参见上述方法实施例，在此不再赘述。Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the positioning methods described in the foregoing method embodiments. For details, please refer to the foregoing method implementation. For example, it will not be repeated here.
其中，上述计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。Wherein, the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本公开所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims (17)

  1. 一种定位方法，其特征在于，包括：A positioning method, comprising:
    获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；Obtain the current road image collected by the target vehicle and a pre-made map of the current location of the target vehicle;
    基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标；determining, based on the current road surface image, a first world coordinate of a road surface element in the current road surface image;
    基于所述预制地图，确定所述路面元素的第二世界坐标；determining a second world coordinate of the road surface element based on the prefabricated map;
    根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中的目标对象的目标世界坐标。According to the first world coordinate and the second world coordinate of the road surface element, the target world coordinate of the target object in the target image collected by the target vehicle is determined.
  2. 根据权利要求1所述的定位方法，其特征在于，在获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图之前，所述定位方法还包括：The positioning method according to claim 1, wherein before acquiring the current road surface image collected by the target vehicle and the prefabricated map of the current location of the target vehicle, the positioning method further comprises:
    响应于针对目标车辆的标定按键的触发操作，或者，接收到针对目标车辆进行标定的标定指令。In response to a triggering operation of a calibration button for the target vehicle, or, a calibration instruction for calibrating the target vehicle is received.
  3. 根据权利要求1或2所述的定位方法，其特征在于，所述基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标，包括：The positioning method according to claim 1 or 2, wherein the determining, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image, comprising:
    获取所述当前路面图像中的所述路面元素的像素坐标；obtaining the pixel coordinates of the road surface element in the current road surface image;
    基于所述路面元素的像素坐标以及预先确定的所述采集设备的历史单应性矩阵，确定所述路面元素的第一世界坐标。A first world coordinate of the road surface element is determined based on the pixel coordinates of the road surface element and a predetermined historical homography matrix of the acquisition device.
  4. 根据权利要求3所述的定位方法，其特征在于，所述获取所述当前路面图像中的所述路面元素的像素坐标，包括：The positioning method according to claim 3, wherein the acquiring the pixel coordinates of the road surface element in the current road surface image comprises:
    对所述当前路面图像进行语义分割，得到所述当前路面图像包含的至少一个路面元素，以及至少一个路面元素的路面元素类别；Semantic segmentation is performed on the current road surface image to obtain at least one road surface element included in the current road surface image, and a road surface element category of at least one road surface element;
    针对任意一个路面元素，按照该路面元素的路面元素类别，在所述当前路面图像中提取该路面元素包含的第一特征点集的像素坐标；For any pavement element, according to the pavement element category of the pavement element, extract the pixel coordinates of the first feature point set included in the pavement element from the current pavement image;
    将所述第一特征点集的像素坐标作为所述第一特征点集对应的路面元素的像素坐标。The pixel coordinates of the first feature point set are used as the pixel coordinates of the road surface element corresponding to the first feature point set.
  5. 根据权利要求4所述的定位方法，其特征在于，所述针对任意一个路面元素，按照该路面元素的路面元素类别，在所述当前路面图像中提取该路面元素包含的第一特征点集的像素坐标，包括：The positioning method according to claim 4, wherein, for any pavement element, according to the pavement element category of the pavement element, the first feature point set included in the pavement element is extracted from the current pavement image. Pixel coordinates, including:
    在该路面元素为第一类路面元素的情况下，在该路面元素的轮廓上提取多个第一特征点，得到该路面元素的第一特征点集；In the case that the pavement element is the first type of pavement element, extract a plurality of first feature points on the outline of the pavement element to obtain a first feature point set of the pavement element;
    在该路面元素为第二类路面元素的情况下，在该路面元素的中心线上按照设定距离间隔提取第一特征点，得到该路面元素的第一特征点集；In the case that the pavement element is the second type of pavement element, extract the first feature points on the center line of the pavement element according to the set distance interval, and obtain the first feature point set of the pavement element;
    在所述当前路面图像对应的像素坐标系中，获取所述第一特征点集中的至少一个第一特征点的像素坐标。In the pixel coordinate system corresponding to the current road surface image, the pixel coordinates of at least one first feature point in the first feature point set are acquired.
  6. 根据权利要求1所述的定位方法，其特征在于，所述基于所述预制地图，确定所述路面元素的第二世界坐标，包括：The positioning method according to claim 1, wherein the determining the second world coordinates of the road surface element based on the prefabricated map comprises:
    获取预先存储的所述路面元素的第二特征点集在所述预制地图中的地图位置信息；acquiring pre-stored map location information of the second feature point set of the road surface element in the pre-made map;
    基于所述路面元素的第二特征点集的地图位置信息和所述目标车辆在所述预制地图表征的道路 场景中的位姿数据，确定所述路面元素的第二特征点集的世界坐标；Determine the world coordinates of the second feature point set of the pavement element based on the map position information of the second feature point set of the pavement element and the pose data of the target vehicle in the road scene represented by the pre-made map;
    将所述第二特征点集的世界坐标作为所述第二特征点集对应的路面元素的第二世界坐标。The world coordinate of the second feature point set is used as the second world coordinate of the road surface element corresponding to the second feature point set.
  7. 根据权利要求3所述的定位方法，其特征在于，所述根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中的目标对象的目标世界坐标，包括：The positioning method according to claim 3, wherein the target world coordinates of the target object in the target image collected by the target vehicle are determined according to the first world coordinates and the second world coordinates of the road surface elements, include:
    基于所述路面元素的第一世界坐标和第二世界坐标，对所述历史单应性矩阵进行调整，得到所述目标车辆的当前单应性矩阵；Adjusting the historical homography matrix based on the first world coordinate and the second world coordinate of the road surface element to obtain the current homography matrix of the target vehicle;
    基于所述目标图像，确定所述目标对象在所述目标图像中的像素坐标；Based on the target image, determine the pixel coordinates of the target object in the target image;
    基于所述目标对象的像素坐标和所述当前单应性矩阵，确定所述目标对象的目标世界坐标。Based on the pixel coordinates of the target object and the current homography matrix, the target world coordinates of the target object are determined.
  8. 根据权利要求1所述的定位方法，其特征在于，在确定所述目标对象的目标世界坐标之后，所述定位方法还包括：The positioning method according to claim 1, wherein after determining the target world coordinates of the target object, the positioning method further comprises:
    基于所述目标对象的目标世界坐标以及预设位置点的位置信息，确定所述目标对象与所述预设位置点之间的距离。Based on the target world coordinates of the target object and the position information of the preset position point, the distance between the target object and the preset position point is determined.
  9. 根据权利要求7所述的定位方法，其特征在于，所述基于所述路面元素的第一世界坐标和第二世界坐标，对所述历史单应性矩阵进行调整，得到所述目标车辆的当前单应性矩阵，包括：The positioning method according to claim 7, wherein the historical homography matrix is adjusted based on the first world coordinate and the second world coordinate of the road surface element to obtain the current position of the target vehicle. Homography matrices, including:
    基于所述路面元素的第一特征点集的世界坐标和所述路面元素的第二特征点集的世界坐标，确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵；Based on the world coordinates of the first feature point set of the pavement element and the world coordinates of the second feature point set of the pavement element, determine the difference between the first feature point set and the second feature point set of the same pavement element The target affine matrix between;
    基于所述目标仿射矩阵，对所述历史单应性矩阵进行修正，得到所述目标车辆的当前单应性矩阵。Based on the target affine matrix, the historical homography matrix is modified to obtain the current homography matrix of the target vehicle.
  10. 根据权利要求9所述的定位方法，其特征在于，所述路面元素包含多个，所述基于所述路面元素的第一特征点集的世界坐标和所述路面元素的第二特征点集的世界坐标，确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵，包括：The positioning method according to claim 9, wherein the pavement element comprises a plurality of, and the world coordinate based on the first feature point set of the pavement element and the second feature point set of the pavement element world coordinates, determine the target affine matrix between the first feature point set and the second feature point set of the same pavement element, including:
    按照当前仿射矩阵，对多个路面元素中的任意一个路面元素的第二特征点集的世界坐标进行更新，确定该路面元素的第二特征点集的更新后的世界坐标；所述当前仿射矩阵为初始预设仿射矩阵或者为针对所述初始预设仿射矩阵调整过至少一次的仿射矩阵；According to the current affine matrix, update the world coordinates of the second feature point set of any one of the multiple pavement elements, and determine the updated world coordinates of the second feature point set of the pavement element; the current simulation The affine matrix is an initial preset affine matrix or an affine matrix that has been adjusted at least once for the initial preset affine matrix;
    基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值；Based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element, determine the updated world coordinates and the first feature point set of the second feature point set of the pavement element The residual value between the world coordinates of ;
    基于所述多个路面元素的残差值，对所述当前仿射矩阵进行调整后，返回确定该路面元素的第二特征点集的更新后的世界坐标的步骤，直至满足预设截止条件后，得到所述目标仿射矩阵。After adjusting the current affine matrix based on the residual values of the plurality of pavement elements, return to the step of determining the updated world coordinates of the second feature point set of the pavement element until the preset cutoff condition is met , to obtain the target affine matrix.
  11. 根据权利要求10所述的定位方法，其特征在于，所述路面元素包括第一类路面元素，在确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵之前，所述定位方法还包括：The positioning method according to claim 10, wherein the pavement element comprises a first type of pavement element, and the target between the first feature point set and the second feature point set of the same pavement element is determined Before the affine matrix, the positioning method further includes:
    针对任意一个第一类路面元素，在该第一类路面元素的第一特征点集中提取与该第一类路面元素的第二特征点集中的任意一个第二特征点匹配的第一特征点；For any first type of pavement element, extract a first feature point matching any second feature point in the second feature point set of the first type of pavement element from the first feature point set of the first type of pavement element;
    所述基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值， 包括：The updated world coordinates and the first feature of the second feature point set of the pavement element are determined based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element The residual value between the world coordinates of the point set, including:
    针对任意一个第一类路面元素，基于该第一类路面元素的第二特征点集中的任意一个第二特征点更新后的世界坐标和与该第二特征点匹配的第一特征点的世界坐标，确定该第二特征点和匹配的第一特征点之间的距离；For any first-type pavement element, the updated world coordinate based on any second feature point in the second feature point set of the first-type pavement element and the world coordinate of the first feature point matching the second feature point , determine the distance between the second feature point and the matched first feature point;
    对该第一类路面元素包含的至少一个第二特征点和匹配的第一特征点之间的距离进行求和，得到该第一类路面元素对应的残差值。The distances between the at least one second feature point included in the first type of road surface element and the matched first characteristic point are summed to obtain a residual value corresponding to the first type of road surface element.
  12. 根据权利要求11所述的定位方法，其特征在于，所述路面元素还包括第二类路面元素，在确定同一路面元素的所述第一特征点集和所述第二特征点集之间的目标仿射矩阵之前，所述定位方法还包括：The positioning method according to claim 11, wherein the pavement element further comprises a second type of pavement element, and the difference between the first feature point set and the second feature point set determined for the same pavement element Before the target affine matrix, the positioning method further includes:
    针对任意一个第二类路面元素，按照该第二类路面元素的第一特征点集的世界坐标，确定该第二类路面元素的第一特征点集的拟合直线方程；For any second type of pavement element, according to the world coordinates of the first characteristic point set of the second type of pavement element, determine the fitted line equation of the first characteristic point set of the second type of pavement element;
    所述基于该路面元素的第二特征点集更新后的世界坐标和该路面元素的第一特征点集的世界坐标，确定该路面元素的第二特征点集更新后的世界坐标和第一特征点集的世界坐标之间的残差值，包括：The updated world coordinates and the first feature of the second feature point set of the pavement element are determined based on the updated world coordinates of the second feature point set of the pavement element and the world coordinates of the first feature point set of the pavement element The residual value between the world coordinates of the point set, including:
    针对任意一个第二类路面元素，基于该第二类路面元素的第二特征点集中的任意一个第二特征点更新后的世界坐标和该第二类路面元素对应的拟合直线方程，确定该第二特征点和该第二类路面元素对应的拟合直线之间的距离；For any second type of pavement element, based on the updated world coordinates of any second feature point in the second feature point set of the second type of pavement element and the fitted straight line equation corresponding to the second type of pavement element, determine the the distance between the second feature point and the fitted straight line corresponding to the second type of pavement element;
    基于该第二类路面元素包含的至少一个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值。Based on the distance between the at least one second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element, the residual value corresponding to the second type of pavement element is determined.
  13. 根据权利要求12所述的定位方法，其特征在于，在确定该第二类路面元素对应的残差值之前，所述定位方法还包括：The positioning method according to claim 12, wherein before determining the residual value corresponding to the second type of road surface element, the positioning method further comprises:
    基于该第二类路面元素包含的至少一个第二特征点的世界坐标，获取该第二类路面元素的至少一个第二特征点对应的距离权重；Obtaining a distance weight corresponding to at least one second feature point of the second type of pavement element based on the world coordinate of at least one second feature point included in the second type of pavement element;
    所述基于该第二类路面元素包含的至少一个第二特征点和该第二类路面元素对应的拟合直线之间的距离，确定该第二类路面元素对应的残差值，包括：The determining the residual value corresponding to the second type of pavement element based on the distance between at least one second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element includes:
    基于该第二类路面元素包含的至少一个第二特征点对应的距离权重，对该第二类路面元素包含的至少一个第二特征点和该第二类路面元素对应的拟合直线之间的距离进行加权求和，得到该第二类路面元素对应的残差值。Based on the distance weight corresponding to at least one second feature point included in the second type of pavement element, the distance between the at least one second feature point included in the second type of pavement element and the fitted straight line corresponding to the second type of pavement element The distance is weighted and summed to obtain the residual value corresponding to the second type of road surface element.
  14. 一种定位装置，其特征在于，包括：A positioning device, characterized in that it includes:
    获取模块，用于获取目标车辆采集的当前路面图像和目标车辆当前所处位置的预制地图；an acquisition module for acquiring the current road surface image collected by the target vehicle and a prefabricated map of the current location of the target vehicle;
    第一确定模块，用于基于所述当前路面图像，确定所述当前路面图像中的路面元素的第一世界坐标；a first determining module, configured to determine, based on the current road surface image, the first world coordinates of the road surface element in the current road surface image;
    第二确定模块，用于基于所述预制地图，确定所述路面元素的第二世界坐标；a second determining module, configured to determine the second world coordinates of the road surface element based on the prefabricated map;
    第三确定模块，用于根据所述路面元素的第一世界坐标、第二世界坐标，确定所述目标车辆采集的目标图像中目标对象的目标世界坐标。The third determining module is configured to determine, according to the first world coordinate and the second world coordinate of the road surface element, the target world coordinate of the target object in the target image collected by the target vehicle.
  15. 一种电子设备，其特征在于，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至13任一所述的定位方法的步骤。An electronic device, characterized in that it includes: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the memory are connected The machine-readable instructions perform the steps of the positioning method according to any one of claims 1 to 13 when the machine-readable instructions are executed by the processor.
  16. 一种计算机可读存储介质，其特征在于，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如权利要求1至13任一所述的定位方法的步骤。A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the positioning method according to any one of claims 1 to 13 are executed.
  17. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1-13中的任一权利要求所述的定位方法的步骤。A computer program, comprising computer readable code, when the computer readable code is run in an electronic device, a processor in the electronic device executes the code for implementing any one of claims 1-13. The steps of the positioning method.
PCT/CN2021/098697 2021-03-19 2021-06-07 Positioning method and apparatus, electronic device, and storage medium WO2022193448A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020227021876A KR20220131378A (en) 2021-03-19 2021-06-07 Positioning method, apparatus, electronic device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110295186.8A CN113052904B (en) 2021-03-19 2021-03-19 Positioning method, positioning device, electronic equipment and storage medium
CN202110295186.8 2021-03-19

Publications (1)

Publication Number Publication Date
WO2022193448A1 true WO2022193448A1 (en) 2022-09-22

Family

ID=76513561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/098697 WO2022193448A1 (en) 2021-03-19 2021-06-07 Positioning method and apparatus, electronic device, and storage medium

Country Status (3)

Country Link
KR (1) KR20220131378A (en)
CN (1) CN113052904B (en)
WO (1) WO2022193448A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704046A (en) * 2023-08-01 2023-09-05 北京积加科技有限公司 Cross-mirror image matching method and device
CN118050010A (en) * 2024-04-15 2024-05-17 小米汽车科技有限公司 Positioning method, device, vehicle, storage medium and program product for vehicle

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120166033A1 (en) * 2010-12-23 2012-06-28 Electronics And Telecommunications Research Institute Method and apparatus for detecting driving information of autonomous driving system
CN108805934A (en) * 2017-04-28 2018-11-13 华为技术有限公司 A kind of method for calibrating external parameters and device of vehicle-mounted vidicon
CN109300159A (en) * 2018-09-07 2019-02-01 百度在线网络技术（北京）有限公司 Method for detecting position, device, equipment, storage medium and vehicle
CN110155053A (en) * 2018-02-12 2019-08-23 三星电子株式会社 Method and apparatus for driving the information of vehicle is provided
CN112419423A (en) * 2020-10-30 2021-02-26 上海商汤临港智能科技有限公司 Calibration method, calibration device, electronic equipment and storage medium
CN112489136A (en) * 2020-11-30 2021-03-12 商汤集团有限公司 Calibration method, position determination method, device, electronic equipment and storage medium

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6581359B2 (en) * 2015-01-07 2019-09-25 キヤノン株式会社 Information processing apparatus, control method thereof, program, storage medium, and video camera
CN108303721B (en) * 2018-02-12 2020-04-03 北京经纬恒润科技有限公司 Vehicle positioning method and system
CN110567469B (en) * 2018-06-05 2021-07-20 北京市商汤科技开发有限公司 Visual positioning method and device, electronic equipment and system
CN111341097B (en) * 2020-02-13 2022-05-20 中交第一公路勘察设计研究院有限公司 Traffic data acquisition method based on unmanned aerial vehicle video
CN111524185A (en) * 2020-04-21 2020-08-11 上海商汤临港智能科技有限公司 Positioning method and device, electronic equipment and storage medium
CN111582079A (en) * 2020-04-24 2020-08-25 杭州鸿泉物联网技术股份有限公司 Lane positioning method and device based on computer vision

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120166033A1 (en) * 2010-12-23 2012-06-28 Electronics And Telecommunications Research Institute Method and apparatus for detecting driving information of autonomous driving system
CN108805934A (en) * 2017-04-28 2018-11-13 华为技术有限公司 A kind of method for calibrating external parameters and device of vehicle-mounted vidicon
CN110155053A (en) * 2018-02-12 2019-08-23 三星电子株式会社 Method and apparatus for driving the information of vehicle is provided
CN109300159A (en) * 2018-09-07 2019-02-01 百度在线网络技术（北京）有限公司 Method for detecting position, device, equipment, storage medium and vehicle
CN112419423A (en) * 2020-10-30 2021-02-26 上海商汤临港智能科技有限公司 Calibration method, calibration device, electronic equipment and storage medium
CN112489136A (en) * 2020-11-30 2021-03-12 商汤集团有限公司 Calibration method, position determination method, device, electronic equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116704046A (en) * 2023-08-01 2023-09-05 北京积加科技有限公司 Cross-mirror image matching method and device
CN116704046B (en) * 2023-08-01 2023-11-10 北京积加科技有限公司 Cross-mirror image matching method and device
CN118050010A (en) * 2024-04-15 2024-05-17 小米汽车科技有限公司 Positioning method, device, vehicle, storage medium and program product for vehicle

Also Published As

Publication number Publication date
CN113052904A (en) 2021-06-29
CN113052904B (en) 2022-12-13
KR20220131378A (en) 2022-09-27

Similar Documents

Publication Publication Date Title
WO2021004548A1 (en) Vehicle speed intelligent measurement method based on binocular stereo vision system
TWI722355B (en) Systems and methods for correcting a high-definition map based on detection of obstructing objects
EP3581890B1 (en) Method and device for positioning
WO2020237996A1 (en) Vehicle pose correction method and device
WO2022193448A1 (en) Positioning method and apparatus, electronic device, and storage medium
CN110738121A (en) front vehicle detection method and detection system
CN107741234A (en) The offline map structuring and localization method of a kind of view-based access control model
CN113903011B (en) Semantic map construction and positioning method suitable for indoor parking lot
WO2020043081A1 (en) Positioning technique
WO2021056341A1 (en) Lane line fusion method, lane line fusion apparatus, vehicle, and storage medium
WO2021208110A1 (en) Method for determining lane line recognition abnormal event, and lane line recognition apparatus and system
CN110992424B (en) Positioning method and system based on binocular vision
CN107808524B (en) Road intersection vehicle detection method based on unmanned aerial vehicle
EP4209755A1 (en) Positioning method and positioning apparatus of mobile device
CN113095154A (en) Three-dimensional target detection system and method based on millimeter wave radar and monocular camera
CN112067006B (en) Semantic-based visual positioning method and device
CN111161334A (en) Semantic map construction method based on deep learning
Laflamme et al. Driving datasets literature review
CN113673288B (en) Idle parking space detection method and device, computer equipment and storage medium
CN111126363B (en) Object recognition method and device for automatic driving vehicle
CN110864670B (en) Method and system for acquiring position of target obstacle
CN110415299B (en) Vehicle position estimation method based on set guideboard under motion constraint
CN111860084B (en) Image feature matching and positioning method and device and positioning system
CN111754388B (en) Picture construction method and vehicle-mounted terminal
CN107248171A (en) A kind of monocular vision odometer yardstick restoration methods based on triangulation

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022532079

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21931033

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21931033

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 23.02.2024)