CN115731215A

CN115731215A - Loop detection method and device, electronic equipment and storage medium

Info

Publication number: CN115731215A
Application number: CN202211516008.4A
Authority: CN
Inventors: 赵楠楠; 王东科; 张振林
Original assignee: China Automotive Innovation Corp
Current assignee: China Automotive Innovation Corp
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2023-03-03

Abstract

The invention provides a loop detection method, a loop detection device, electronic equipment and a storage medium, and relates to the field of automatic driving, wherein the method comprises the following steps: obtaining initial semantic point clouds to be matched and a plurality of local grid subgraphs of multi-resolution levels; performing rotation operation and translation operation on the initial semantic point cloud to be matched for preset times; carrying out occupation state matching on each frame of semantic point cloud to be matched based on a multi-resolution level grid map corresponding to the current local grid subgraph to obtain a matching result; under the condition that the matching result indicates that the target semantic point cloud to be matched exists, ending occupation state matching; and updating the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph when the occupation state matching is finished. The invention solves the problem of large-angle deflection of the semantic point cloud to be matched, so that the loop detection result is more accurate, the matching speed is increased, and the loop matching algorithm achieves real-time performance.

Description

Loop detection method and device, electronic equipment and storage medium

Technical Field

The invention relates to the field of automatic driving, in particular to a loop detection method and device, electronic equipment and a storage medium.

Background

In the automatic driving field, the instant positioning and map building are an indispensable link of the automatic driving automobile, so that the automatic driving automobile can be helped to sense the complex information of the road in advance, the real-time high-precision positioning information is provided, and the automobile can make a correct decision by combining with intelligent path planning. In the map building stage, the automatic driving automobile cannot accurately observe global information, and the introduction of map building accumulated errors cannot be avoided. In order to eliminate the accumulated error, the vehicle is generally driven for a period of time, and then driven to the area where the map has been constructed again to perform loop detection, so as to eliminate the accumulated error. In the prior art, point cloud and map matching is used as a core of loop detection, but most loop detection methods have the problems of large detection error and long matching search time.

Disclosure of Invention

In order to solve the above technical problems, the present invention provides a loop detection method, apparatus, electronic device and storage medium, wherein the scheme is as follows:

in a first aspect, a loop detection method is provided, including:

obtaining initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a grid map of a multi-resolution level;

selecting a current local grid subgraph from a plurality of local grid subgraphs; the current local grid subgraph is any unmatched local grid subgraph in the multiple local grid subgraphs;

performing rotation operation and translation operation on the initial semantic point cloud to be matched for preset times to obtain a semantic point cloud set to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

carrying out occupation state matching on each frame of semantic point clouds to be matched in the semantic point cloud set to be matched based on a multi-resolution hierarchy grid map corresponding to the current local grid subgraph to obtain a matching result;

under the condition that the matching result indicates that the semantic point cloud to be matched does not exist, the step of selecting the current local grid sub-graph from the plurality of local grid sub-graphs is executed until the matching result indicates that the semantic point cloud to be matched exists, and occupation state matching is finished;

and updating the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph when the occupation state matching is finished.

Optionally, the method further includes:

acquiring an initial global semantic map, and dividing the initial global semantic map into a plurality of local subgraphs;

performing mesh division on each local subgraph according to a first preset resolution ratio to generate a first-level mesh map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local subgraph;

and performing resolution reduction processing for preset times on the first-level grid map corresponding to each local sub-graph based on a preset resolution level to obtain each local grid sub-graph.

Optionally, after the selecting the current local mesh sub-graph from the multiple local mesh sub-graphs, the method further includes:

determining a first rotating histogram of the initial semantic point cloud to be matched and a second rotating histogram of the current local grid subgraph;

rotating the first rotating histogram for multiple times according to a preset unit rotating angle to obtain a first rotating histogram corresponding to each rotating angle;

respectively determining the correlation degree score of the first rotation histogram and the second rotation histogram corresponding to each rotation angle;

and determining a target first rotation histogram corresponding to the highest relevance score, and taking a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

Optionally, the performing rotation operation and translation operation on the initial semantic point cloud to be matched for a preset number of times to obtain a semantic point cloud set to be matched includes:

determining a rotation range of the rotation operation based on the target rotation angle; determining preset rotation times based on the rotation range;

acquiring a first geometric center of the initial semantic point cloud to be matched and a second geometric center of the current local grid subgraph;

determining a translation range of the translation operation based on the first geometric center and the second geometric center; determining a preset translation frequency based on the translation range;

and performing rotation operation of the preset rotation times on the initial semantic point cloud to be matched, and performing translation operation of the preset translation times during each rotation operation to obtain the semantic point cloud set to be matched.

Optionally, the occupancy state matching is performed on each frame of semantic point clouds to be matched in the semantic point cloud set to be matched based on the grid map of the multi-resolution hierarchy corresponding to the current local grid sub-graph, and obtaining a matching result includes:

carrying out occupation state matching on each frame of semantic point cloud to be matched in the semantic point cloud set to be matched and a lowest level grid map corresponding to the current local grid sub-graph, and determining a first matching degree score of each frame of semantic point cloud to be matched and the lowest level grid map;

and obtaining a matching result based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest-level grid map.

Optionally, obtaining a matching result based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest-level grid map includes:

determining a highest first matching degree score from the first matching degree scores corresponding to the semantic point clouds to be matched and the lowest-level grid map;

when the highest first matching degree score is smaller than a preset score threshold value, determining a matching result as a first matching result, wherein the first matching result indicates that no target semantic point cloud to be matched exists;

when the highest first matching degree score is not smaller than the preset score threshold value, carrying out occupation state matching on the semantic point cloud to be matched corresponding to the highest first matching degree score from the lowest-level grid map corresponding to the current local grid subgraph layer by layer to obtain a second matching degree score of the semantic point cloud to be matched corresponding to the highest first matching degree score and each level grid map; wherein the content of the first and second substances,

when the second matching degree score is smaller than the preset score threshold, stopping continuously occupying state matching, and determining that the matching result is the first matching result;

and when the semantic point cloud to be matched and the second matching degree score of each level of grid map are not less than the preset score threshold, determining that the matching result is a second matching result, wherein the second matching result indicates that the semantic point cloud to be matched corresponding to the highest first matching degree score is used as the target semantic point cloud to be matched.

Optionally, the updating the global semantic map based on the target semantic point cloud to be matched and the current local network sub-graph when the occupancy state matching is finished includes:

acquiring a rotation angle and a translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched;

determining the optimized pose information of the semantic point cloud to be matched of the target based on the rotation angle, the translation distance and the current local grid subgraph when the occupation state matching is finished;

and updating the global semantic map based on the optimized pose information of the target semantic point cloud to be matched.

Optionally, the method further includes:

splicing the panoramic images currently acquired by the mobile equipment to obtain an inverse perspective transformation image;

performing semantic segmentation on the inverse perspective transformation image to obtain a semantic image to be matched;

and carrying out coordinate mapping on the semantic image to be matched to obtain the initial semantic point cloud to be matched.

In a second aspect, a loop detection apparatus is provided, comprising:

the acquisition module is used for acquiring an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a grid map of a multi-resolution level;

the local grid subgraph selecting module is used for selecting the current local grid subgraph from the multiple local grid subgraphs; the current local grid subgraph is any unmatched local grid subgraph in the multiple local grid subgraphs;

the semantic point cloud conversion module to be matched is used for performing rotation operation and translation operation on the initial semantic point cloud to be matched for preset times to obtain a semantic point cloud set to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

the matching module is used for carrying out occupation state matching on each frame of semantic point clouds to be matched in the semantic point cloud sets to be matched based on a multi-resolution hierarchy grid map corresponding to the current local grid subgraph to obtain a matching result;

under the condition that the matching result indicates that the target semantic point cloud to be matched does not exist, the step of selecting the current local grid sub-graph from the plurality of local grid sub-graphs is executed until the matching result indicates that the target semantic point cloud to be matched exists, and occupation state matching is finished;

and the map optimization module is used for updating the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph when the occupation state matching is finished.

Optionally, the loop detection module further includes:

the system comprises an initial global semantic map acquisition module, a global semantic map generation module and a global semantic map generation module, wherein the initial global semantic map acquisition module is used for acquiring an initial global semantic map and dividing the initial global semantic map into a plurality of local subgraphs;

the mesh division module is used for carrying out mesh division on each local sub-graph according to a first preset resolution ratio to generate a first-level mesh map corresponding to each local sub-graph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local sub-image;

and the local grid subgraph generation module is used for performing resolution reduction processing on the first-level grid map corresponding to each local subgraph for preset times based on preset resolution levels to obtain each local grid subgraph.

Optionally, the loop detection module further includes:

a rotation histogram determination module, configured to determine a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid sub-graph;

the rotation conversion module is used for rotating the first rotation histogram for multiple times according to a preset unit rotation angle to obtain a first rotation histogram corresponding to each rotation angle;

a correlation score determining module, configured to determine a correlation score of the first rotation histogram and the second rotation histogram corresponding to each rotation angle respectively;

and the target rotation angle determining module is used for determining a target first rotation histogram corresponding to the highest relevancy score and taking the rotation angle corresponding to the target first rotation histogram as the target rotation angle.

Optionally, the to-be-matched semantic point cloud transforming module includes:

a rotation range determination unit configured to determine a rotation range of the rotation operation based on the target rotation angle; and determining a preset number of rotations based on the rotation range;

the geometric center determining unit is used for acquiring a first geometric center of the initial semantic point cloud to be matched and a second geometric center of the current local grid subgraph;

a translation range determination unit configured to determine a translation range of the translation operation based on the first geometric center and the second geometric center; determining a preset translation frequency based on the translation range;

and the semantic point cloud conversion unit to be matched is used for performing rotation operation of the preset rotation times on the initial semantic point cloud to be matched and performing translation operation of the preset translation times during each rotation operation to obtain the semantic point cloud set to be matched.

Optionally, the matching module includes:

the first matching degree score determining module is used for carrying out occupation state matching on each frame of semantic point cloud to be matched in the semantic point cloud set to be matched and the lowest-level grid map corresponding to the current local grid sub-graph and determining a first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest-level grid map;

and the matching result generation module is used for obtaining a matching result based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest-level grid map.

Optionally, the matching result generating module includes:

the first matching degree score determining unit is used for determining the highest first matching degree score from the first matching degree scores of each frame of semantic point cloud to be matched and the lowest level grid map;

the first matching result generating unit is used for determining the matching result as a first matching result when the highest first matching degree score is smaller than a preset score threshold value, and the first matching result indicates that the target semantic point cloud to be matched does not exist;

a second matching degree score determining unit, configured to perform occupancy state matching on the semantic point cloud to be matched corresponding to the highest first matching degree score layer by layer from the lowest-level grid map corresponding to the current local grid sub-graph when the highest first matching degree score is not smaller than the preset score threshold, so as to obtain a second matching degree score between the semantic point cloud to be matched corresponding to the highest first matching degree score and each level grid map; wherein, the first and the second end of the pipe are connected with each other,

a first matching result generating unit, configured to stop continuing to occupy state matching and determine that the matching result is the first matching result when the second matching degree score is smaller than the preset score threshold;

and the target semantic point cloud to be matched determining unit is used for determining that the matching result is a second matching result when the second matching degree scores of the semantic point cloud to be matched and each level of grid map are not less than the preset score threshold, and the second matching result indicates that the semantic point cloud to be matched corresponding to the highest first matching degree score is used as the target semantic point cloud to be matched.

Optionally, the map optimization module includes:

a pose transformation information acquisition unit for acquiring a rotation angle and a translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched;

the optimization pose information determining unit is used for determining the optimization pose information of the semantic point cloud to be matched based on the rotation angle and the translation distance and the current local grid subgraph when the occupation state matching is finished;

and the map updating unit is used for updating the global semantic map based on the optimized pose information of the semantic point cloud to be matched with the target.

Optionally, the loop detection device includes:

the inverse perspective transformation unit is used for splicing the panoramic images currently acquired by the mobile equipment to obtain inverse perspective transformation images;

the semantic segmentation unit is used for performing semantic segmentation on the inverse perspective transformation image to obtain a semantic image to be matched;

and the coordinate mapping unit is used for carrying out coordinate mapping on the semantic image to be matched to obtain the initial semantic point cloud to be matched.

In a third aspect, an electronic device is provided, which includes a processor and a memory, where at least one instruction or at least one program is stored in the memory, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the above method.

In a fourth aspect, a computer-readable storage medium is provided, where at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded by a processor and executed to implement the steps of the above method.

In a fifth aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium, the computer instructions being read by a processor of a computer device from the computer readable storage medium, the processor executing the computer instructions to cause the computer device to perform the steps of the method described above.

By adopting the technical scheme, the invention has the following beneficial effects:

in the loop detection process, the semantic point cloud to be matched is appropriately rotated and translated, so that the rotation and translation errors of the semantic point cloud to be matched and a global semantic map in the matching process are reduced or even eliminated, the problem of large-angle deflection of the semantic point cloud to be matched is solved, the loop detection result is more accurate, the global semantic map is divided into local grid subgraphs of multi-resolution levels, the semantic point cloud to be matched and the local grid subgraphs occupy state matching layer by layer, the matching speed is increased, and the loop matching algorithm achieves real-time performance.

Additional features and advantages of the invention will be set forth in the detailed description which follows.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that are required to be used in the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, wherein like reference numerals generally represent like parts. For a person skilled in the art, without inventive effort, further figures can be obtained from these figures.

Fig. 1 is a schematic flow chart of a loop detection method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an implementation process of an embodiment of a loopback detection method according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a loop detection apparatus according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a hardware structure of a server for executing the loop detection method according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic may be included in at least one implementation of the invention. In the description of the present invention, it is to be understood that the terms "upper", "lower", "top", "bottom", and the like, indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are only for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Moreover, the terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or described herein.

The loop detection method provided by the embodiment of the invention can be applied to the instant positioning and map building stage in automatic driving, and referring to fig. 1, a schematic flow chart of the loop detection method provided by the embodiment of the invention is shown. It is noted that the present specification provides the method steps as described in the examples or flowcharts, but may include more or less steps based on routine or non-inventive labor. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. In practice, the system apparatus or product may be implemented in a sequential or parallel manner (e.g., in the context of parallel processors or multi-threaded processing) according to the embodiments or methods shown in the figures. The loop detection method provided by the embodiment of the invention comprises the following steps:

s101, obtaining an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a grid map of a multi-resolution level;

in a possible manner, the method for generating the initial semantic point cloud to be matched includes:

Specifically, in the embodiment of the present invention, the mobile device may be a vehicle, the image acquisition unit on the vehicle acquires a frame of road image at preset time intervals or at preset distance intervals in a vehicle driving process, the semantic point cloud to be matched is obtained by processing a panoramic image acquired by a current frame, the panoramic image is spliced and then subjected to inverse perspective transformation processing to obtain an inverse perspective transformation image, the inverse perspective transformation image is subjected to semantic segmentation to obtain a semantic image to be matched, and the semantic image to be matched may include lane line information, ground identification information such as zebra stripes, no-parking areas, deceleration strips, and the like, and is used for matching in a loopback detection process. In another embodiment, the initial semantic point cloud to be matched can also be a laser point cloud, wherein a plane feature point is selected from the laser point cloud, and a two-dimensional coordinate of each plane feature point in a map coordinate system is obtained according to the pose of a vehicle when the current frame laser point cloud is collected, so that a two-dimensional semantic point cloud is obtained.

The global semantic map is a global map which is formed by performing semantic segmentation on historical frame images acquired by vehicles and splicing the segmented images, and can be a two-dimensional image or a point cloud map. The local grid subgraph is a multi-resolution-level grid map, each grid is marked with the probability occupied by the obstacles by adopting a sliding window algorithm, the obstacles comprise lane lines, zebra stripes, no-parking areas, deceleration strips and other ground identifications in the local grid subgraph and the semantic point cloud to be matched, and the grid occupation probability is used for calculating the matching degree when the semantic point cloud to be matched is matched with each local grid subgraph.

In a possible manner, the method for generating each partial grid graph includes:

performing mesh division on each local subgraph according to a first preset resolution ratio to generate a first-level mesh map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local sub-image;

Specifically, an initial global semantic map which is not divided is obtained, the initial global semantic map is divided into a plurality of local subgraphs, and the local subgraphs are regarded as feature points, so that the whole divided global semantic map can be regarded as a topological map, the local subgraphs are the feature points, and the constraint relation of the relative positions between the local subgraphs is lines. The local subgraphs and the local subgraphs keep a relationship through the relative position of the central point, but the relative position error is lower between the local subgraphs with similar positions, and the error is larger and larger along with the increase of the number of the local subgraphs, the increase of the moving distance of the robot, various random factors and the like, so that loop detection is carried out to correct the errors.

The method comprises the steps of carrying out meshing processing on local subgraphs by taking n centimeters as a unit, namely a first preset resolution to obtain a first-level grid map corresponding to each local subgraph, carrying out resolution reduction processing on each first-level grid map, wherein the first-level grid map is the highest-level grid map corresponding to the local subgraph, the method for carrying out resolution reduction processing in the embodiment of the invention is \8230that2 n centimeters are taken as the unit and 4n centimeters are taken as the unit, carrying out grid division on the local subgraphs, and carrying out grid division by adopting other preset resolutions in other embodiments to generate multilayer local grid subgraphs with different resolutions, wherein the larger the grid size is, the lower the resolution of the corresponding hierarchical grid map is, and the number of pixel points is reduced. For example, the area of the current local sub-graph is 1 square meter, and when the resolution is 5cm per pixel, each grid corresponds to 400 pixel points; if the compression is 10cm per pixel, 100 pixels are correspondingly arranged. For the constructed local subgraph with the same size, the number of pixel points is reduced, the calculation amount required during matching is greatly reduced, and the matching efficiency is improved. Note that each grid of the m-th layer map is shown adjacent to 2 ^m-1 The grid with the highest probability value in the grids with the resolution of n centimeters.

S102, selecting a current local grid subgraph from a plurality of local grid subgraphs; the current local grid subgraph is any unmatched local grid subgraph in the multiple local grid subgraphs;

s103, performing rotation operation and translation operation on the initial semantic point cloud to be matched for preset times to obtain a semantic point cloud set to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

specifically, considering that a large-angle deflection problem possibly exists between the semantic point cloud to be matched and the local grid subgraph in the matching process, the initial semantic point cloud to be matched is subjected to rotation operation and translation operation for preset times to obtain a semantic point cloud set to be matched, and the semantic point cloud set to be matched comprises a plurality of frames of semantic point clouds to be matched with different rotation angles and different translation amounts.

In one possible implementation, step S103 may include the steps of:

Specifically, a target rotation angle is determined based on the first rotation histogram of the initial semantic point cloud to be matched and the second rotation histogram of the current local grid sub-graph, for example, 30 degrees, a rotation range of the initial semantic point cloud to be matched for rotation operation is about 30 degrees, positive and negative adjustment is performed based on 30 degrees, as for the rotation frequency, the rotation frequency can be set according to an actual situation, for example, 20 times, and 1 degree of rotation is performed each time, so that the rotation angle of the semantic point cloud to be matched can be 30-10 degrees to 30+10 degrees, that is, 20 degrees to 40 degrees.

Further, the distance between the first geometric center of the initial semantic point cloud to be matched and the second geometric center of the current local grid subgraph is determined, that is, the translation range of the translation operation of the initial semantic point cloud to be matched is determined, and the translation times can also be set according to the actual conditions, for example, the distance is 5 meters, and the translation time is 0.5 meter per time and is 10 times in total. It should be noted that in the embodiment of the present invention, the process of the rotation operation and the translation operation is the translation operation performed for the preset number of times during each rotation operation, and the initial semantic point cloud to be matched is rotated to 20 degrees by taking the above example as a reference, and is translated for 10 times under the rotation condition, and each translation is performed for 0.5 meter, so as to obtain a frame of semantic point cloud to be matched; when the initial semantic point cloud to be matched rotates to 21 degrees, the initial semantic point cloud to be matched is translated for 10 times under the rotation condition, the initial semantic point cloud to be matched is translated for 0.5 meter each time, and another frame of semantic point cloud to be matched is obtained 8230, and the rest can be done in the same way until the initial semantic point cloud is rotated for 20 times, and multiple frames of semantic point clouds to be matched with different rotation angles and different translation amounts are obtained.

In one possible embodiment, the method for determining the target rotation angle of the rotation operation includes:

Specifically, referring to fig. 2, which shows an exemplary determination process of a first rotation histogram of an initial semantic point cloud to be matched, for the initial semantic point cloud to be matched, first, a mean value of coordinates of each point of the initial semantic point cloud to be matched is calculated to obtain a centroid c of the initial semantic point cloud to be matched; secondly, determining a connecting line of each point of the initial semantic point cloud to be matched and the centroid, and calculating an angle formed by each connecting line and the x axis; then, sequencing all points according to each angle, wherein the first point is the point with the smallest angle with the x axis, and the first point is set as a reference point a; the next point of the reference point a is a point b, a connection point b and a reference point a, an included angle between a connection line and an x axis is calculated, the included angle is mapped to a first rotation histogram, the initial first rotation histogram is divided into a plurality of parts according to a preset unit angle in a coordinate system where the semantic point cloud to be matched is located, for example, the unit angle is 2 degrees, the initial first rotation histogram is divided into 180 parts, and degrees corresponding to the included angle are mapped to an angle interval corresponding to the initial first rotation histogram; calculating the weight of the point b, that is, determining the vertical degree between ab and bc, may be understood that the closer the point a and the point b are, the more ab and bc are vertical, the greater the weight of the point b is, the same calculation process is performed on other points and the point b, and a histogram element in each angle interval in the first rotation histogram is an accumulated value of the weights of the points.

The current local grid sub-graph can contain multiple frames of images collected by historical frames and semantic point clouds obtained by semantic segmentation processing, the calculation process is similar to that of a first rotating histogram of the initial semantic point cloud to be matched, a rotating histogram of each frame of semantic point cloud is obtained, the rotating histograms of all frames of semantic point clouds are overlapped to obtain a second rotating histogram of the current local grid sub-graph, and the calculation process is carried out by adopting a grid map of the highest level corresponding to the current local grid sub-graph. And then, rotating the first rotating histogram of the initial semantic point cloud to be matched at different angles, wherein the angle and the rotating frequency of each rotation can be set differently in different embodiments, calculating the correlation degree of the first rotating histogram under each rotating angle and the second rotating histogram of the current local grid subgraph, determining a target first rotating histogram corresponding to the highest correlation degree score, and taking the rotating angle corresponding to the target first rotating histogram as the target rotating angle.

S104, carrying out occupation state matching on each frame of semantic point clouds to be matched in the semantic point cloud set to be matched based on a multi-resolution hierarchy grid map corresponding to the current local grid sub-graph to obtain a matching result;

in one possible implementation, step S104 includes:

carrying out occupation state matching on each frame of semantic point cloud to be matched in the semantic point cloud set to be matched and a lowest level grid map corresponding to the current local grid sub-graph, and determining a first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map;

determining a highest first matching degree score from the first matching degree scores corresponding to each frame of semantic point cloud to be matched and the lowest-level grid map;

when the highest first matching degree score is not smaller than the preset score threshold value, carrying out occupation state matching on the semantic point cloud to be matched corresponding to the highest first matching degree score from the lowest-level grid map corresponding to the current local grid subgraph layer by layer to obtain a second matching degree score of the semantic point cloud to be matched corresponding to the highest first matching degree score and each level grid map; wherein, the first and the second end of the pipe are connected with each other,

and when the second matching degree scores of the semantic point cloud to be matched and the grid maps of all levels are not less than the preset score threshold, determining that the matching result is a second matching result, wherein the second matching result indicates that the semantic point cloud to be matched corresponding to the highest first matching degree score is used as the target semantic point cloud to be matched.

Specifically, the occupation state matching refers to mapping each frame of semantic point clouds to be matched into the lowest-level grid map, accumulating all grids occupied by the point clouds according to probability values to calculate first matching degree scores, wherein the larger the probability value accumulation is, the higher the first matching degree score is, sorting the first matching degree scores corresponding to the semantic point clouds to be matched, determining the semantic point cloud to be matched with the highest first matching degree score, meanwhile, comparing the highest first matching degree score with a preset score threshold, wherein the preset score threshold represents the minimum matching degree score of the semantic point clouds to be matched and the current-level grid map of the current local sub-graph, and if the highest first matching degree score is smaller than the preset score threshold, the lowest-level grid map representing the semantic point clouds to be matched and the current local sub-graph fails to be matched, stopping occupation state matching, and selecting other local grid sub-graphs to perform occupation state matching again.

And if the highest first matching degree score is not less than the preset score threshold, representing that the semantic point cloud to be matched is successfully matched with the lowest-level grid map of the current local subgraph, determining the position with the highest score in the lowest-level grid map, and performing occupation state matching on the current local grid subgraph layer by adopting a recursive search strategy on the semantic point cloud to be matched corresponding to the highest first matching degree score at the position with the highest score, namely, performing progressive matching from the level with low resolution to the level with high resolution until the grid map of the highest level is matched.

In the matching process, if the second matching degree score of a certain layer is smaller than a preset score threshold value, the lowest-level grid map which represents the semantic point cloud to be matched and the current local sub-graph fails to be matched, the occupation state matching is stopped, other local grid sub-graphs are selected to perform occupation state matching again until a current local grid sub-graph is finally found, the first matching degree score of the lowest-level grid map and a certain frame of semantic point cloud to be matched in the semantic point cloud set to be matched is highest, the second matching degree obtained by the grid map of each level and the semantic point cloud to be matched with the highest first matching degree score is not smaller than the preset score threshold value, the semantic point cloud to be matched with the highest first matching degree score is the target semantic point cloud to be matched, and the current local grid sub-graph which is successfully matched is the local grid sub-graph which is successfully detected in a loop detection mode.

Dividing the local grid subgraph into grid maps of a plurality of resolution levels, starting matching from the grid map with the lowest resolution, stopping matching the semantic point cloud to be matched with the current local grid subgraph once the matching score is smaller than a preset score threshold value, accelerating the matching speed and enabling a loop matching algorithm to achieve real-time performance as the lower resolution is, the larger the grid is, the fewer pixel points are, the smaller the calculation amount is during the calculation of the matching score.

S105, under the condition that the matching result indicates that the semantic point cloud to be matched does not exist, the step of selecting the current local grid sub-graph from the plurality of local grid sub-graphs is executed until the matching result indicates that the semantic point cloud to be matched exists, and occupation state matching is finished;

and S106, updating the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph when the occupation state matching is finished.

In one possible implementation, step S106 includes:

Specifically, the pose of the target semantic point cloud to be matched is not necessarily the pose of the initial semantic point cloud to be matched, so that the rotation angle and the translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched and the initial pose of the initial semantic point cloud to be matched relative to the current local grid subgraph successfully matched are determined, the rotation angle and the translation distance and the initial pose are the optimized pose of the target semantic point cloud to be matched, the optimized pose is updated to the global semantic map, and loop detection is completed.

Corresponding to the above loop detection method, embodiments of the present invention further provide a loop detection apparatus, and since the loop detection apparatus provided in the embodiments of the present invention corresponds to the loop detection methods provided in the foregoing embodiments, the implementation manner of the loop detection method is also applicable to the loop detection apparatus provided in the embodiments of the present invention, and is not described in detail in the embodiments of the present invention.

Referring to fig. 3, it shows a loopback detection apparatus provided in an embodiment of the present invention, the apparatus has a function of implementing the loopback detection method in the foregoing method embodiment, where the function may be implemented by hardware, or by hardware executing corresponding software, and the apparatus may include:

an obtaining module 310, configured to obtain an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a grid map of a multi-resolution level;

a local mesh subgraph selecting module 320, configured to select a current local mesh subgraph from multiple local mesh subgraphs; the current local grid subgraph is any unmatched local grid subgraph in the multiple local grid subgraphs;

a semantic point cloud conversion module 330 to be matched, configured to perform rotation operation and translation operation on the initial semantic point cloud to be matched for a preset number of times to obtain a semantic point cloud set to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

the matching module 340 is configured to perform occupation state matching on each frame of semantic point clouds to be matched in the semantic point cloud set to be matched based on a multi-resolution hierarchical grid map corresponding to the current local grid sub-graph to obtain a matching result;

a map optimization module 350, configured to update the global semantic map based on the target semantic point cloud to be matched and the current local mesh sub-map when the occupancy state matching is finished.

Optionally, the loop detection module further includes:

the grid division module is used for carrying out grid division on each local subgraph according to a first preset resolution ratio to generate a first-level grid map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local sub-image;

Optionally, the loop detection module further includes:

the rotation histogram determination module is used for determining a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

and the target rotation angle determining module is used for determining a target first rotation histogram corresponding to the highest relevancy score and taking a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

Optionally, the semantic point cloud transforming module 330 to be matched includes:

a rotation range determination unit configured to determine a rotation range of the rotation operation based on the target rotation angle; determining preset rotation times based on the rotation range;

Optionally, the matching module 340 includes:

and the matching result generation module is used for obtaining a matching result based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map.

Optionally, the matching result generating module includes:

a second matching degree score determining unit, configured to, when the highest first matching degree score is not smaller than the preset score threshold, perform occupancy state matching on the to-be-matched semantic point clouds corresponding to the highest first matching degree score layer by layer from the lowest-level grid map corresponding to the current local grid sub-graph, to obtain second matching degree scores of the to-be-matched semantic point clouds corresponding to the highest first matching degree score and each level grid map; wherein the content of the first and second substances,

and the target semantic point cloud to be matched determining unit is used for determining that the matching result is a second matching result when the scores of the second matching degrees of the semantic point cloud to be matched and each level of grid map are not less than the preset score threshold, and the second matching result indicates that the semantic point cloud to be matched corresponding to the highest first matching degree score is taken as the target semantic point cloud to be matched.

Optionally, the map optimization module 350 includes:

the optimization pose information determination unit is used for determining the optimization pose information of the target semantic point cloud to be matched based on the rotation angle and the translation distance and the current local grid subgraph when the occupation state matching is finished;

and the map updating unit is used for updating the global semantic map based on the optimized pose information of the target semantic point cloud to be matched.

Optionally, the loop detection device includes:

It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the functional modules is illustrated, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure of the apparatus may be divided into different functional modules to implement all or part of the functions described above. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments for details, which are not described herein again.

The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, where the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the loop detection method.

The memory may be used to store software programs and modules, and the processor may execute various functional applications by executing the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory. The processor may be a central processing unit, but may also be other general purpose processors, such as a microprocessor or the processor may be any conventional processor, a digital signal processor, an application specific integrated circuit or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.

The method provided by the embodiment of the invention can be executed in a computer terminal, a server or a similar operation device. Taking the operation on a terminal as an example, fig. 4 is a schematic diagram of a hardware structure of a server for operating a loop detection method according to an embodiment of the present invention, and as shown in fig. 4, the server 400 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 410 (the processors 410 may include but are not limited to Processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory 430 for storing data, and one or more storage media 420 (e.g., one or more mass storage devices) for storing applications 423 or data 422. Memory 430 and storage medium 420 may be, among other things, transient or persistent storage. The program stored on the storage medium 420 may include one or more modules, each of which may include a series of instruction operations on a server. Further, the processor 410 may be configured to communicate with the storage medium 420 to execute a series of instruction operations in the storage medium 420 on the server 400. The server 400 may also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input-output interfaces 440, and/or one or more operating systems 421, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, and so forth.

The input/output interface 440 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 400. In one example, the input/output Interface 440 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the input/output interface 440 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

It will be understood by those skilled in the art that the structure shown in fig. 4 is only an illustration and is not intended to limit the structure of the electronic device. For example, server 400 may also include more or fewer components than shown in FIG. 4, or have a different configuration than shown in FIG. 4.

The embodiment of the present invention further provides a computer-readable storage medium, where at least one instruction or at least one program is stored in the computer-readable storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the steps of the loop detection method. In an embodiment of the invention, the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunications signal, software distribution medium, etc.

The present embodiments also provide a computer storage medium, where at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded by a processor and executed to implement the foregoing method. In an embodiment of the invention, the computer program comprises computer program code, which may be in source code form, object code form, an executable file or some intermediate form, etc. The computer-readable storage medium may include, but is not limited to: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunications signal, software distribution medium, etc.

Embodiments of the present invention also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the loop detection method provided in the various alternative implementations described above.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A loop detection method, comprising:

acquiring an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a grid map of a multi-resolution level;

2. The loop back detection method of claim 1, further comprising:

performing mesh division on each local sub-graph according to a first preset resolution ratio to generate a first-level mesh map corresponding to each local sub-graph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local subgraph;

3. The loop-back detection method of claim 1, wherein said selecting the current local mesh sub-graph from the plurality of local mesh sub-graphs further comprises:

and determining a target first rotation histogram corresponding to the highest correlation score, and taking a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

4. The loopback detection method as recited in claim 3, wherein the performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a semantic point cloud set to be matched comprises:

5. The loopback detection method as recited in claim 1, wherein the performing occupancy state matching on each frame of semantic point clouds to be matched in the semantic point cloud set to be matched based on the grid map of the multi-resolution hierarchy corresponding to the current local grid subgraph to obtain a matching result comprises:

6. The loopback detection method as recited in claim 5, wherein the obtaining a matching result based on the first matching degree score of each frame of semantic point cloud to be matched and the lowest level grid map comprises:

7. The loopback detection method as recited in claim 1, wherein said updating a global semantic map based on the target semantic point cloud to be matched and the current local network subgraph at the end of the occupancy state matching comprises:

8. The loop back detection method according to any one of claims 1 to 7, further comprising:

9. A loop detection apparatus, comprising:

10. An electronic device comprising a processor and a memory, wherein at least one instruction or at least one program is stored in the memory, and wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, in which at least one instruction or at least one program is stored, which is loaded and executed by a processor to implement the steps of a method according to any one of claims 1 to 8.