CN115773765A

CN115773765A - Semantic positioning method combining dynamic and static characteristics, electronic device and medium

Info

Publication number: CN115773765A
Application number: CN202211477927.5A
Authority: CN
Inventors: 颜扬治; 许钰龙; 李凯; 张志萌
Original assignee: Ecarx Hubei Tech Co Ltd
Current assignee: Ecarx Hubei Tech Co Ltd
Priority date: 2022-11-23
Filing date: 2022-11-23
Publication date: 2023-03-10

Abstract

The invention provides a semantic positioning method combining dynamic and static characteristics, an electronic device and a machine-readable storage medium. The semantic locating method comprises the following steps: acquiring 3D sensing information of a road where a vehicle is located by a sensor in real time, and processing the 3D sensing information to obtain road semantic information, wherein the road semantic information comprises static semantic features and dynamic semantic features; generating road 3D observation data comprising static semantic features and dynamic semantic features based on the road semantic information; and (3) performing semantic information registration on the road 3D observation data and the pre-constructed feature map, and solving the optimal registration pose by a cost function optimization method, thereby realizing the positioning of the vehicle. The scheme of the invention creatively uses the dynamic information originally regarded as the noise item in the positioning process, namely, the dynamic and static characteristic information alignment is expanded from single static characteristic information alignment to the common alignment of the dynamic and static characteristic information, thereby improving the robustness and the precision of the positioning process.

Description

Semantic localization method, electronic device, and medium combining dynamic and static features

Technical Field

The invention relates to the technical field of automatic driving, in particular to a semantic positioning method combining dynamic and static characteristics, an electronic device and a machine-readable storage medium.

Background

The positioning technology is one of the basic and core technologies of robot application technologies such as automatic driving and the like, and provides position and attitude, namely attitude information for the robot. According to the positioning principle, the existing positioning technology can be divided into geometric positioning, dead reckoning, feature positioning and the like.

The geometric positioning depends on positioning facilities, is influenced by signal shielding, reflection and the like, and can fail in scenes such as tunnels, elevated buildings and the like. The limitation of dead reckoning is that positioning errors increase cumulatively as the estimated distance increases. The feature positioning technology is widely applied to the technical field of robots such as automatic driving. In the prior art, the static characteristics detected in real time in the driving process of a vehicle are aligned with a characteristic map in characteristic positioning selection, positioning is carried out, and dynamic objects need to be filtered. However, dynamic objects are ubiquitous in a driving environment, and in certain scenes, even the majority, such as congested scenes, result in a large amount of filtering effort. In other cases, for example, due to occlusion, the filtering of the dynamic object is difficult, which increases the calculation difficulty and affects the robustness and accuracy of the positioning.

Disclosure of Invention

In view of the above, a semantic locating method, an electronic device and a machine-readable storage medium combining dynamic and static features are proposed that overcome or at least partly solve the above mentioned problems.

An object of the present invention is to provide a semantic location method combining static features and dynamic features, so that the location process is more robust and has high accuracy.

A further object of the present invention is to increase road semantic information contained in road 3D observation data to be registered to further improve positioning accuracy.

In particular, according to an aspect of the embodiments of the present invention, there is provided a semantic location method combining dynamic and static features, including:

acquiring 3D sensing information of a road where a vehicle is located by a sensor in real time, and processing the 3D sensing information to obtain road semantic information, wherein the road semantic information comprises static semantic features and dynamic semantic features;

generating road 3D observation data comprising the static semantic features and the dynamic semantic features based on the road semantic information;

semantic information registration is carried out on the road 3D observation data and a pre-constructed feature map, and an optimal registration pose is obtained through a cost function optimization method, so that the vehicle is positioned, wherein the feature map comprises feature vector information and road rule information, the cost function is constructed on the basis of matching between static semantic features in the road 3D observation data and the feature vector information of the feature map and between dynamic semantic features in the road 3D observation data and the road rule information of the feature map, and the optimal registration pose enables the value of the cost function to be minimum.

Optionally, the step of acquiring, in real time, 3D sensing information of a road where a vehicle is located by a sensor, and processing the 3D sensing information to obtain road semantic information includes:

acquiring multi-frame 3D sensing information of a road where a vehicle is located by a sensor in real time, processing each frame of the 3D sensing information including segmentation and identification, and acquiring multi-frame road semantic information corresponding to the multi-frame 3D sensing information one to one;

the step of generating road 3D observation data including the static semantic features and the dynamic semantic features based on the road semantic information includes:

calculating the relative positions of the vehicles in multiple frames corresponding to the road semantic information one by one through a specified position calculation algorithm;

and splicing the multiple frames of the road semantic information together according to the multiple frames of the relative poses of the vehicles to obtain the road 3D observation data.

Optionally, the step of calculating, by a pose-specific calculation algorithm, a plurality of frames of vehicle relative poses corresponding to the plurality of frames of road semantic information one-to-one includes:

and calculating the relative pose of the vehicle relative to the vehicle pose at the appointed origin point when each frame of the 3D sensing information is acquired through a dead reckoning algorithm, so as to obtain a plurality of frames of vehicle relative poses corresponding to the plurality of frames of the road semantic information one by one.

Optionally, the step of splicing multiple frames of the road semantic information together to obtain the road 3D observation data according to the multiple frames of the relative pose of the vehicle includes:

calculating the relative pose between the road semantic information of each frame and the road semantic information of the latest frame according to the relative poses of the vehicles of the plurality of frames;

converting the road semantic information of each frame into the latest frame through each relative pose to obtain the converted road semantic information of each frame;

and accumulating all the converted road semantic information of each frame together to obtain the road 3D observation data.

Optionally, the step of registering semantic information of the road 3D observation data and a pre-constructed feature map, and obtaining an optimal registration pose by using a cost function optimization method includes:

constructing a static feature cost function, wherein the static feature cost function is equal to the sum of reprojection errors between each static semantic feature in the road 3D observation data and corresponding feature vector information in the feature map after registration pose transformation;

constructing a plurality of map areas of the road according to the road rule information of the feature map, and constructing a dynamic feature cost function based on dynamic semantic features of the road 3D observation data and the map areas, wherein the plurality of map areas comprise operable areas of different traffic participants and inoperable areas of the road except the operable areas, and the dynamic feature cost function is equal to the sum of matching degrees of each dynamic semantic feature in the road 3D observation data after registration pose transformation and the corresponding map area;

and taking the sum of the static characteristic cost function and the dynamic characteristic cost function as a cost function to be optimized, and solving the optimal registration pose which enables the value of the cost function to be optimized to be minimum.

Optionally, the matching degree between each dynamic semantic feature in the road 3D observation data and the corresponding map region after the registration pose transformation is determined by:

for each dynamic semantic feature, if the dynamic semantic feature falls into the correct corresponding map area after the registration pose transformation, determining the matching degree to be 1;

and if the dynamic semantic features fall into the wrong corresponding map area after the registration pose transformation, determining the matching degree to be 0.

Optionally, the travelable areas comprise at least one of a motorway, a non-motorway and a sidewalk, and each travelable area is provided with travelable direction information;

the non-operational area includes at least one of a green belt, a building, and a waterway.

Optionally, the feature map is constructed in advance by collecting road information through high-precision positioning equipment and a sensor;

the feature vector information comprises road surface object information and road surface identification information, the road surface object comprises at least one of a lamp post, a guideboard, a road edge and a guardrail, and the road surface identification comprises at least one of a solid line, a dotted line, an arrow and road surface characters;

the road regulation information comprises at least one of a motor vehicle lane, a non-motor vehicle lane, a road course and a road range.

According to another aspect of the embodiments of the present invention, there is also provided an electronic device, which includes a memory, a processor and a machine executable program stored in the memory and running on the processor, and when the processor executes the machine executable program, the aforementioned semantic locating method combining dynamic and static features is implemented.

According to yet another aspect of the embodiments of the present invention, there is further provided a machine-readable storage medium, on which a machine-executable program is stored, wherein the machine-executable program, when executed by a processor, implements the aforementioned semantic locating method combining dynamic and static features.

According to the semantic positioning method combining the dynamic and static characteristics, when a vehicle runs, the dynamic characteristics and the static characteristics acquired in real time are aligned with the pre-constructed characteristic map to realize positioning. Compared with the prior art, the scheme of the invention creatively uses the dynamic information originally regarded as the noise item in the positioning process, namely, the single static characteristic information alignment is expanded to the common alignment of the dynamic and static characteristic information, so that the robustness and the precision of the positioning process are improved.

Further, in the semantic positioning method combining dynamic and static characteristics, road semantic information corresponding to multiple frames of 3D sensing information of a road is spliced into one frame of road 3D observation data for registration with a characteristic map. Compared with single-frame 3D observation data, the road 3D observation data to be registered obtained by splicing the multi-frame road semantic information contains more road semantic information, so that the registration is more accurate, and the precision of the positioning process is further improved.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.

Drawings

Various additional advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 illustrates a flow diagram of a semantic location method that combines dynamic and static features in accordance with an embodiment of the present invention;

FIG. 2 illustrates a flow diagram of a semantic locating method that combines dynamic and static features in accordance with another embodiment of the present invention;

FIG. 3 shows a schematic diagram of a feature map according to an embodiment of the invention;

FIG. 4 illustrates a diagram of semantic targeting incorporating dynamic and static features according to an embodiment of the present invention;

fig. 5 shows a schematic block diagram of an electronic device according to an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The positioning technology is one of the basic and core technologies of robot application technologies such as automatic driving. According to the positioning principle, the positioning technology can be divided into geometric positioning, dead reckoning and feature positioning.

The geometric positioning is to measure the distance or angle of a reference device with a known position and then determine the position of the reference device through geometric calculation. The geometric positioning includes GNSS (Global Navigation Satellite System), UWB (Ultra Wide Band), bluetooth, 5G and other technologies, and provides absolute positioning information. The GNSS technology is most widely applied in the intelligent automobile application. The GNSS positioning is based on a satellite positioning technology and is divided into single-point positioning, differential GPS positioning, and RTK (Real-time kinematic) GPS positioning, wherein the single-point positioning provides a positioning accuracy of 3 to 10 meters, the differential GPS positioning provides a positioning accuracy of 0.5 to 2 meters, and the RTK GPS positioning provides a centimeter-level positioning accuracy. The limitation of geometric positioning is that the geometric positioning depends on positioning facilities, is influenced by signal shielding, reflection and the like, and fails in scenes such as tunnels, elevated buildings and the like.

Dead Reckoning (Dead Reckoning) is to calculate the position of the next moment according to the motion data of sensors such as an Inertial Measurement Unit (IMU) and a wheel speed meter from the position of the previous moment, and provides relative positioning information. The limitation of dead reckoning is that as the estimated distance increases, the positioning error will increase cumulatively.

The feature localization firstly obtains a plurality of features of the surrounding environment, such as base station ID, wifi fingerprint, image, lidar point cloud and the like. Then, the observation features are matched with a feature map established in advance, the position in the feature map is determined, and absolute positioning information can be provided. The direct factors affecting feature localization are the number, quality and discrimination of features. The limitation of feature positioning is that when the factors such as scene and environment affect the feature observation, the positioning accuracy and stability are reduced.

The general approach for feature location is to select static features, which are used to build a feature map in advance. And aligning the static features detected in real time with the feature map for positioning in the driving process of the vehicle. The reason for selecting static features (including lane markings, curbs, guardrails, buildings, etc.) for feature localization is that static features are stable in the environment, while dynamic objects (including moving and stationary motor vehicles, non-motor vehicles, pedestrians, etc.) are unstable and typically need to be filtered out to avoid interference with localization. However, dynamic objects are ubiquitous in a driving environment, in certain scenarios, even the majority, such as congested scenarios. In other cases, for example, due to occlusion, etc., it is difficult to filter out the dynamic objects, which increases the difficulty of feature location and also affects the robustness and accuracy of location.

To solve or at least partially solve the above technical problem, an embodiment of the present invention provides a semantic location method combining dynamic and static features. Firstly, acquiring dynamic and static characteristic information of a road through a sensor, and establishing a characteristic map in advance; and when the vehicle runs, aligning the dynamic and static characteristics acquired in real time with the characteristic map to realize positioning.

For ease of illustration, the coordinate definitions are first specified. In the present invention, a world coordinate system W is defined, which is in a Fixed relationship with the actual geographic location, for example, an Earth-Centered coordinate system ECEF (Earth-Centered, earth-Fixed) may be used. A carrier coordinate system B is also defined, which for a vehicle may also be referred to as a body coordinate system, and which is fixed to a fixed position of the carrier, such as the center of the rear axle of the vehicle. Vehicle pose, i.e. the body coordinate system, is the 6Dof (Degree of Freedom) pose WPB = TWB in the world coordinate system. A sensor coordinate system S, also called observation coordinate system, is then defined. The measurement data acquired by the sensor are all based on the sensor coordinate system. Usually, the sensor is fixed on the carrier and moves with the carrier in a rigid manner, so that there is a fixed conversion relation TBS between the sensor coordinate system and the carrier coordinate system, i.e. the sensor external parameter.

FIG. 1 is a flow diagram illustrating a semantic location method that combines dynamic and static features according to an embodiment of the present invention. Referring to fig. 1, the method may include at least the following steps S102 to S106.

Step S102, acquiring 3D sensing information of a road where a vehicle is located by a sensor in real time, and processing the 3D sensing information to obtain road semantic information, wherein the road semantic information comprises static semantic features and dynamic semantic features;

step S104, generating road 3D observation data comprising static semantic features and dynamic semantic features based on the road semantic information;

and S106, performing semantic information registration on the road 3D observation data and a pre-constructed feature map, and solving an optimal registration pose by a cost function optimization method so as to realize the positioning of the vehicle, wherein the feature map comprises feature vector information and road rule information, the cost function is constructed on the basis of the matching between the static semantic features in the road 3D observation data and the feature vector information of the feature map as well as the dynamic semantic features in the road 3D observation data and the road rule information of the feature map, and the optimal registration pose enables the value of the cost function to be minimum.

In the semantic positioning method combining the dynamic and static characteristics provided by the embodiment of the invention, when a vehicle runs, the dynamic characteristics and the static characteristics acquired in real time are aligned with the pre-constructed characteristic map to realize positioning. Compared with the prior art, the scheme of the invention creatively uses the dynamic information originally regarded as the noise item in the positioning process, namely, the single static characteristic information alignment is expanded to the common alignment of the dynamic and static characteristic information, so that the robustness and the precision of the positioning process are improved.

In the embodiment of the invention, a feature map needs to be established in advance for aligning with the dynamic and static features acquired in real time to realize the positioning of the vehicle. In some embodiments, the feature map is pre-constructed by collecting road information with high precision positioning equipment and sensors. The sensor may be a single sensor or a combination of multiple sensors, such as a camera, liDAR (Laser Detection and Ranging) or other sensor, etc.

The feature map includes feature vector information and road regulation information. The feature vector information is vector-based road feature information, and includes, but is not limited to, information of road surface objects storing at least one of light poles, road signs, road edges, guardrails, and the like on the road surface as vector information of points, lines, surfaces, and the like, and information of road surface identifications storing at least one of solid lines, broken lines, arrows, characters, and the like. The road regulation information includes, but is not limited to, at least one of a motorway, a non-motorway, a road heading, a road range, and the like.

In an embodiment of the present invention, 3D sensing information of a road is acquired in real time by a sensor equipped on a vehicle. The sensor may be, for example, a camera, liDAR or other sensor, or a combination of such sensors. By these sensors, 3D sensing information of the road is acquired. Then, road semantic information is acquired through a detection, segmentation and identification method. The identification method can adopt the existing identification technology. The road semantic information includes static semantic features and dynamic semantic features. In particular, the static semantic features may include lane markings, curbs, guardrails, buildings, and other stably existing roadway objects and markings. The dynamic semantic features may include moving and stationary motor vehicles, non-motor vehicles, pedestrians, etc. unstable dynamic objects.

Further, road 3D observation data including static semantic features and dynamic semantic features are generated based on the road semantic information.

As described above, those skilled in the art will recognize that, due to the fixed relationship between the sensor coordinate system and the vehicle body coordinate system, the road semantic information identified based on the 3D sensing information of the sensor should be converted into the vehicle body coordinate system by the sensor external parameter before the road semantic information obtained in real time is used for alignment with the feature map, so as to generate the road 3D observation data in the vehicle body coordinate system. Specifically, if road semantic information (may also be referred to as observation information) obtained based on the 3D sensing information of the sensor a is PA and the sensor external parameter is TBA, the road semantic information PB = TBA × PA in the vehicle body coordinate system.

In an embodiment of the present invention, the road 3D sensing data for registration may be obtained based on a single frame of 3D sensing information, and may also be obtained based on a plurality of frames of 3D sensing information.

In a preferred embodiment, the road 3D sensing data for registration is obtained based on 3D sensing information of a plurality of frames. Specifically, multi-frame 3D sensing information of a road where a vehicle is located by a sensor is obtained in real time, each frame of 3D sensing information is processed, and multi-frame road semantic information corresponding to the multi-frame 3D sensing information one to one is obtained. The processing of each frame of 3D sensing information may include segmentation and recognition, and may also include necessary detection. Furthermore, each frame of road semantic information can be converted into a vehicle body coordinate system through sensor external parameters.

Then, calculating the relative positions of the vehicles in multiple frames corresponding to the road semantic information of the multiple frames one by one through a specified position calculation algorithm; and splicing the multiple frames of road semantic information together according to the relative poses of the multiple frames of vehicles to obtain the road 3D observation data. In practical application, any algorithm capable of calculating the vehicle pose can be adopted as the designated pose calculation algorithm. In this embodiment, the road semantic information corresponding to the multi-frame 3D sensing information of the road is spliced into one frame of road 3D observation data for subsequent registration with the feature map. Compared with single-frame 3D observation data, the road 3D observation data to be registered obtained by splicing the multi-frame road semantic information contains more road semantic information, so that the registration is more accurate, and the precision of the positioning process is further improved.

In a preferred embodiment, a dead reckoning algorithm may be employed to calculate the relative pose of the vehicle. In this case, a sensor such as an IMU, a wheel speed meter, or a vehicle speed meter needs to be provided on the vehicle to acquire motion data required for dead reckoning. Specifically, the relative pose of the vehicle relative to the vehicle pose at the designated origin when each frame of 3D sensing information is acquired is calculated through a dead reckoning algorithm, so that the multi-frame vehicle relative pose corresponding to the multi-frame road semantic information one to one is obtained. The relative pose of the vehicle acquired by the dead reckoning DR refers to the relative pose from the point a to the point b in the DR coordinate system provided by the DR. The DR coordinate system is self-defined by DR, and generally the pose when the DR acquires the first frame of observation data can be taken as the origin. Specifically, assuming that the pose of the point a is Ta and the pose of the point b is Tb, the relative pose Tba between the point a and the point b = Ta-inverse Tb, where Ta-inverse refers to the inverse matrix of Ta. And a dead reckoning algorithm is adopted to obtain more stable vehicle relative pose data.

Furthermore, by the relative pose of the vehicle, multiple frames of road semantic information are spliced together to obtain road 3D observation data. Specifically, calculating the relative pose between the road semantic information of each frame and the road semantic information of the latest frame; converting the road semantic information of each frame into the latest frame through each relative pose so as to obtain the converted road semantic information of each frame; and accumulating all the converted road semantic information of each frame together to obtain road 3D observation data. The multi-frame road semantic information is represented as F1, F2 \8230, fn, the corresponding DR vehicle relative pose is T1, T2 \8230, tn, wherein Fn is the latest frame road semantic information, and the calculation process can be represented as follows: firstly, calculating the relative pose of each frame of road semantic information and the latest frame of road semantic information, and regarding the ith frame of road semantic information, the relative pose Tni = Ti-inverse Tn, wherein Ti-inverse is an inverse matrix of Ti; then, converting each frame of road semantic information into the latest frame through the relative pose of each frame of road semantic information and the latest frame of road semantic information, and converting the ith frame of road semantic information into the road semantic information nFi = Tni × Fi after the latest frame; and finally, directly accumulating all the road semantic information converted into the latest frame, namely acquiring spliced road 3D observation data.

After the road 3D observation data are obtained, semantic information registration is carried out on the road 3D observation data and the corresponding feature map, and the optimal registration pose TWB is obtained. And a cost function optimization method is adopted for solving the registration pose TWB. The cost function to be optimized comprises two parts, namely static characteristics and dynamic characteristics.

In a specific embodiment, the step of registering semantic information between the road 3D observation data and the pre-constructed feature map, and obtaining the optimal registration pose by using a cost function optimization method, may include:

constructing a static characteristic cost function, wherein the static characteristic cost function is equal to the sum of reprojection errors between each static semantic feature in the road 3D observation data and corresponding feature vector information in the feature map after registration pose transformation;

constructing a plurality of map areas of a road according to road rule information of a feature map, and constructing a dynamic feature cost function based on dynamic semantic features and the map areas of road 3D observation data, wherein the plurality of map areas comprise operable areas of different traffic participants and inoperable areas except the operable areas in the road, and the dynamic feature cost function is equal to the sum of matching degrees between each dynamic semantic feature in the road 3D observation data and the corresponding map area after registration pose transformation;

In the above step, the travelable areas of different traffic participants may include at least one of a motorway, a non-motorway, a sidewalk and the like, and each travelable area carries travelable direction information. The inoperable area may include at least one of greenbelts, buildings, waterways, etc. in the road environment.

Further, the matching degree between each dynamic semantic feature in the road 3D observation data and the corresponding map area after pose registration transformation is determined in the following way: for each dynamic semantic feature, if the dynamic semantic feature falls into a correct corresponding map area after pose registration transformation, determining the matching degree as 1; and if the dynamic semantic features fall into the wrong corresponding map area after the pose transformation is registered, determining the matching degree to be 0. For example, for a dynamic semantic feature identified as a motor vehicle, if the dynamic semantic feature falls on a motor vehicle lane after being transformed by the registration pose TWB, the matching degree is determined to be 1, and if the dynamic semantic feature falls on an inoperable area such as a sidewalk or a green belt after being transformed by the TWB, the matching degree is determined to be 0.

In the above, various implementation manners of each link of the embodiment shown in fig. 1 are introduced, and the implementation process of the semantic locating method combining dynamic and static features according to the present invention will be described in detail through specific embodiments.

FIG. 2 is a flow chart illustrating a semantic locating method that combines dynamic and static features according to an embodiment of the present invention. In the embodiment, the road information is acquired in advance through high-precision positioning equipment and a sensor, and the characteristic map is established in advance. In the construction stage of the feature map, the feature vector information and the road rule information are respectively stored in the feature map. FIG. 3 is a schematic diagram of a feature map. In the illustrated feature map, the feature vector information includes road surface object information such as road edges, and road surface identification information such as solid lines, broken lines, arrows, stop lines, zebra stripes, and characters. The road regulation information includes information such as a motorway, a non-motorway, a road course and the like.

As shown in fig. 2, the semantic location method combining dynamic and static features may include the following steps S202 to S216.

Step S202, multi-frame 3D sensing information of a road where a vehicle is located by a sensor is obtained in real time, processing including segmentation and identification is carried out on each frame of 3D sensing information, and multi-frame road semantic information corresponding to the multi-frame 3D sensing information one by one is obtained.

On the vehicle, a camera, lidar or other sensor or a combination of these sensors is provided. By these sensors, 3D sensing information of the road is acquired. Then, road semantic information is acquired through a detection, segmentation and identification method. The road semantic information includes static semantic features and dynamic semantic features.

And converting the road semantic information into a vehicle body coordinate system through sensor external parameters.

The multi-frame road semantic information obtained in the step is represented as F1, F2 \8230andFn, wherein Fn is the latest frame road semantic information.

And S204, calculating the relative position of the vehicle relative to the vehicle position at the appointed origin when each frame of 3D sensing information is obtained through a dead reckoning algorithm, so as to obtain the relative positions of the vehicles of multiple frames corresponding to the road semantic information of multiple frames one to one.

On the vehicle, sensors such as an inertia measurement unit, a wheel speed meter, or a vehicle speed meter are equipped. And acquiring the relative pose of the vehicle when each frame of 3D sensing information is calculated through dead reckoning. The relative pose estimation here refers to the relative pose from point a to point b provided by DR. Specifically, in the DR coordinate system, the pose of point a is Ta, and the pose of point b is Tb, then the relative pose Tba = Ta-inverse Tb between a and b can be obtained, where Ta-inverse refers to the inverse matrix of Ta.

The DR vehicle relative pose corresponding to the multi-frame road semantic information one by one obtained in the step is represented as T1, T2 \8230andTn.

And step S206, calculating the relative pose between the road semantic information of each frame and the road semantic information of the latest frame according to the relative poses of the vehicles of the plurality of frames.

And for the ith frame of road semantic information, the relative pose Tni = Ti-inverse Tn of the ith frame of road semantic information and the latest frame of road semantic information, wherein Ti-inverse is an inverse matrix of Ti.

And S208, converting the road semantic information of each frame into the latest frame through each relative pose so as to obtain the converted road semantic information of each frame.

For the ith frame of road semantic information, the conversion is made to the road semantic information nFi = Tni × Fi after the latest frame.

And step S210, accumulating all the converted road semantic information of each frame together to obtain road 3D observation data.

In the step, all the road semantic information converted into the latest frame is directly accumulated, namely spliced road 3D observation data is obtained.

And step S212, constructing a static characteristic cost function.

The static feature cost function is equal to the sum of reprojection errors between each static semantic feature in the road 3D observation data and corresponding feature vector information in the feature map after registration pose transformation. The formula shows that the static semantic features in the road 3D observation data are S1, S2, S3 \8230; \8230Sn, the feature vector information corresponding to the feature map is M1, M2, M3 \8230; mm, and the static feature cost function is:

F-static(TWB)＝SUM(DistStatic(TWB*Si，Mi))

where DistStatic (×) represents the reprojection error between a static semantic feature (may also be referred to as a static observation element) Si and the feature vector information (may also be referred to as a map element) Mi of the feature map, and SUM (×) represents the SUM of the reprojection errors for all static semantic features and the corresponding feature vector information. m and n may or may not be equal. Generally speaking, due to the influence of environment, occlusion and the like, the number of static observation elements acquired in real time may be less than the number of elements in the feature map, and therefore m is greater than or equal to n.

Step S214, a plurality of map areas of the road are constructed according to the road rule information of the feature map, and a dynamic feature cost function is constructed based on the dynamic semantic features of the road 3D observation data and the map areas.

In the step, road rule information of the characteristic map is obtained, operable areas Z1, Z2 and Z3 of different traffic participants are constructed, wherein the operable areas Z8230are \8230andthe operable areas Zg include but are not limited to: motorways, non-motorways, sidewalks, etc. And each operational area is accompanied by direction-travelable information. And the part outside the operable area is classified as an inoperable area Zx, which corresponds to an inoperable area such as green belts, buildings, water courses, etc. in the road environment.

Assuming that the dynamic semantic features in the road 3D observation data are D1, D2, D3 \8230; \8230andDk, the cost function of the dynamic features is as follows:

F-dynamic(TWB)＝SUM(DistZone(TWB*Di,Zi,Zx))

wherein DistZone (x) represents the matching degree of dynamic semantic features (also called dynamic observation elements) Di with corresponding map regions Zi/Zx, and SUM (x) represents the SUM of the matching degrees of all dynamic semantic features with corresponding map regions. The magnitude relationship between g and k is not limited.

Further, distZone (, s) may be represented as:

DistZone (TWB × Di, zi, zx) =1if TWB × Di in the correct zone Zi

=0if twb × di in the erroneous region Zj or Zx.

That is, if the dynamic semantic features fall into the correct corresponding map region after being transformed by the TWB, the score of the obtained matching degree is 1; and if the dynamic semantic features fall into wrong corresponding map areas after being transformed by the TWB, obtaining the score of the matching degree as 0.

And S216, taking the sum of the static characteristic cost function and the dynamic characteristic cost function as a cost function to be optimized, and solving the optimal registration pose which enables the value of the cost function to be optimized to be minimum, so as to realize the positioning of the vehicle.

The cost function to be optimized is expressed as:

F(TWB)＝F-static(TWB)+F-dynamic(TWB)

the optimization solution can be expressed as:

TWB＝argmin(F(TWB))

wherein, argmin (×) represents that the optimal TWB is found, so that the value of the cost function is minimized. Thereby obtaining a high-precision pose of the vehicle, i.e., by registration.

The order of steps S212 and S214 may be interchanged or performed simultaneously.

A schematic diagram of semantic targeting with dynamic and static features according to the present embodiment is shown in fig. 4. As shown in fig. 4, after registration, in addition to the alignment of the static semantic features with the feature vector information, the dynamic semantic features also fall into the corresponding map region. For example, a pedestrian falls on a zebra crossing, a non-motor vehicle falls on a non-motor vehicle lane, and a motor vehicle falls on a motor vehicle lane. Therefore, high-precision positioning of the self-vehicle is realized.

According to the embodiment, dynamic and static characteristic information is acquired through the multiple sensors and is aligned with the characteristic map established in advance, so that positioning is realized, and the positioning process is more robust and high-precision. In the map construction stage, the feature vector information and the road rule information are respectively stored in a map, and a feature map which can be aligned with the dynamic and static information is obtained. And relative pose information of the vehicle is obtained through a dead reckoning technology, and multi-frame dynamic and static characteristics are spliced together, so that 3D observation data of the road are obtained, and subsequent positioning is more reliable and robust.

Based on the same inventive concept, the embodiment of the invention also provides an electronic device 200. Referring to fig. 5, the electronic device 200 includes a memory 201, a processor 202, and a machine executable program 203 stored on the memory 201 and running on the processor 202, and the processor 202 implements the semantic locating method combining the dynamic and static features of any of the embodiments or the combination of the embodiments when executing the machine executable program 203.

Based on the same inventive concept, the embodiment of the invention also provides a machine-readable storage medium. The machine-readable storage medium has stored thereon a machine-executable program which, when executed by a processor, implements the semantic locating method combining dynamic and static features of any of the foregoing embodiments or combinations of embodiments.

It is clear to those skilled in the art that the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, further description is omitted here.

In addition, the functional units in the embodiments of the present invention may be physically independent of each other, two or more functional units may be integrated together, or all the functional units may be integrated in one processing unit. The integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

Those of ordinary skill in the art will understand that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computing device (e.g., a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention when the instructions are executed. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.

Alternatively, all or part of the steps of the method embodiments may be implemented by hardware (such as a personal computer, a server, or a network device) related to program instructions, which may be stored in a computer-readable storage medium, and when the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method according to the embodiments of the present invention.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments can be modified or some or all of the technical features can be equivalently replaced within the spirit and principle of the present invention; such modifications or substitutions do not depart from the scope of the present invention.

Claims

1. A semantic locating method combining dynamic and static characteristics comprises the following steps:

and semantic information registration is carried out on the road 3D observation data and a pre-constructed feature map, and an optimal registration pose is obtained through a cost function optimization method, so that the vehicle is positioned, wherein the feature map comprises feature vector information and road rule information, the cost function is constructed on the basis of matching between static semantic features in the road 3D observation data and the feature vector information of the feature map, and dynamic semantic features in the road 3D observation data and the road rule information of the feature map, and the optimal registration pose enables the value of the cost function to be minimum.

2. The semantic locating method according to claim 1, wherein 3D sensing information of a road where a vehicle is located by a sensor is acquired in real time, and the step of processing the 3D sensing information to obtain road semantic information comprises:

calculating the relative positions of the vehicles in multiple frames which are in one-to-one correspondence with the semantic information of the roads through a designated position calculation algorithm;

and splicing multiple frames of the road semantic information together according to the relative poses of the vehicles to obtain the road 3D observation data.

3. The semantic locating method according to claim 2, wherein the step of calculating the relative poses of the plurality of frames of vehicles corresponding to the semantic information of the roads one by one through a pose-specific calculation algorithm comprises:

4. The semantic locating method according to claim 2, wherein the step of splicing the road semantic information of multiple frames together to obtain the road 3D observation data according to the relative poses of the vehicles of the multiple frames comprises:

and accumulating all the converted road semantic information frames together to obtain the road 3D observation data.

5. The semantic locating method according to any one of claims 1 to 4, wherein semantic information registration is performed on the road 3D observation data and a pre-constructed feature map, and the step of obtaining an optimal registration pose through a cost function optimization method comprises the following steps:

6. The semantic localization method according to claim 5, wherein the matching degree between each dynamic semantic feature in the road 3D observation data and the corresponding map area after the registration pose transformation is determined by the following means:

for each dynamic semantic feature, if the dynamic semantic feature falls into the correct corresponding map area after pose registration transformation, determining the matching degree to be 1;

7. The semantic locating method according to claim 5, wherein the travelable areas comprise at least one of a motorway, a non-motorway and a sidewalk, and each travelable area carries travelable direction information;

8. The semantic positioning method according to claim 1, wherein the feature map is constructed in advance by collecting road information through a high-precision positioning device and a sensor;

the road regulation information comprises at least one of a motorway, a non-motorway, a road course and a road range.

9. An electronic device comprising a memory, a processor, and a machine-executable program stored on the memory and running on the processor, and the processor when executing the machine-executable program implements the semantic locating method according to any one of claims 1-8 in combination with dynamic and static features.

10. A machine readable storage medium having stored thereon a machine executable program which when executed by a processor implements the semantic locating method according to any one of claims 1 to 8 in combination with dynamic and static features.