CN117570963A

CN117570963A - Pedestrian crosswalk-based map real-time updating method, device, equipment and medium

Info

Publication number: CN117570963A
Application number: CN202311531807.3A
Authority: CN
Inventors: 梁孝庆; 周尧; 万国伟; 朱振广
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-11-16
Filing date: 2023-11-16
Publication date: 2024-02-20

Abstract

The disclosure provides a map real-time updating method, device, equipment and medium based on crosswalk, relates to the field of artificial intelligence, and particularly relates to an environment sensing technology in the unmanned field. The specific implementation scheme is as follows: acquiring a current aerial view image of a road where a vehicle is located and a mask image corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road on which the vehicle is located comprises a crosswalk; performing image superposition processing on the current aerial view image based on the mask image to generate a superposition image; and updating the high-precision map in response to the change of the position of the crosswalk determined according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image. The position of the crosswalk is detected in real time in the driving process of the vehicle, and the updating efficiency of the map is improved.

Description

Pedestrian crosswalk-based map real-time updating method, device, equipment and medium

Technical Field

The disclosure relates to the field of artificial intelligence, in particular to a map real-time updating method, device, equipment and medium based on crosswalk, which can be used in the field of environmental perception of unmanned technology.

Background

When the vehicle is driven, the vehicle can automatically drive according to the high-precision map of the road. For example, it is desirable to slow down the walk as the vehicle passes through a crosswalk and alert the driver to the pedestrian.

If there is an error in the position of the crosswalk on the high-precision map, the driving safety of the vehicle is affected. Therefore, it is necessary to update the high-precision map accurately in time to improve the driving safety.

Disclosure of Invention

The disclosure provides a map real-time updating method, device, equipment and medium based on crosswalk.

According to a first aspect of the present disclosure, there is provided a crosswalk-based map real-time updating method, including:

acquiring a current aerial view image of a road where a vehicle is located and a mask image corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road on which the vehicle is located comprises a crosswalk

Performing image superposition processing on the current aerial view image based on the mask image to generate a superposition image;

and updating the high-precision map in response to the fact that the position of the crosswalk is determined to change according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image.

According to a second aspect of the present disclosure, there is provided a crosswalk-based map real-time updating apparatus, including:

the acquisition unit is used for acquiring a current aerial view image of a road where the vehicle is located and a mask image corresponding to the high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road on which the vehicle is located comprises a crosswalk

The generating unit is used for carrying out image superposition processing on the current aerial view image based on the mask image to generate a superposition image;

and the updating unit is used for updating the high-precision map in response to the fact that the position of the crosswalk is determined to change according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image.

According to a third aspect of the present disclosure, there is provided a method of vehicle-based autopilot comprising:

determining the current position of a vehicle, and acquiring the position point of a crosswalk corresponding to the current position based on the updated high-precision map in the method of the first aspect;

and controlling the vehicle to automatically drive according to the position points of the crosswalk in the updated high-precision map.

According to a fourth aspect of the present disclosure, there is provided a vehicle-based automatic driving apparatus comprising:

a position point determining unit, configured to determine a current position of a vehicle, and obtain a position point of a crosswalk corresponding to the current position based on the updated high-precision map in the device according to the second aspect;

and the control unit is used for controlling the vehicle to automatically drive according to the position points of the crosswalk in the updated high-precision map.

According to a fifth aspect of the present disclosure, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor;

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of the first aspect.

According to a seventh aspect of the present disclosure, there is provided a computer program product comprising: a computer program which, when executed by a processor, implements the method of the first aspect.

According to the technology disclosed by the invention, the updating efficiency of the high-precision map is improved.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a flow chart of a map real-time updating method based on crosswalk according to an embodiment of the present disclosure;

fig. 2 is a flow chart of a map real-time updating method based on crosswalk according to an embodiment of the present disclosure;

FIG. 3 is a schematic representation of the generation of superimposed images provided in accordance with an embodiment of the present disclosure;

Fig. 4 is a flow chart of a map real-time updating method based on crosswalk according to an embodiment of the present disclosure;

fig. 5 is a block diagram of a map real-time updating device based on crosswalk according to an embodiment of the present disclosure;

fig. 6 is a block diagram of a map real-time updating apparatus based on crosswalk according to an embodiment of the present disclosure;

FIG. 7 is a block diagram of an electronic device for implementing a travelator-based map real-time update method in accordance with an embodiment of the disclosure;

fig. 8 is a block diagram of an electronic device for implementing a travelator-based map real-time update method in accordance with an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

With the development of cities and the construction of infrastructure, road networks often change. These variations may include changes in crosswalk geometry, such as width, length, location, etc. If the changes cannot be reflected on the high-precision map in time, navigation errors can be caused, potential risks are brought to pedestrians or automatic driving vehicles, and driving safety is affected.

Currently, most map updating methods rely mainly on manual mapping or simple automated methods. For example, a worker periodically detects a network of roads, and updates a high-precision map based on the detection result. The method has higher manpower and time cost, can not timely and accurately reflect the position change of the crosswalk, and can not ensure the freshness of the map.

The shape of the crosswalk may include straight lines, curves, intersections, etc., and there may be different widths, heights and inclinations, and the crosswalk may have a large span, ranging from several tens meters to several hundreds meters, which makes it difficult to extract and identify characteristics of the crosswalk, affecting the updating accuracy of the map.

The disclosure provides a real-time map updating method, device, equipment and medium based on a crosswalk, which are applied to the field of artificial intelligence, in particular to the field of environment perception in unmanned driving, so as to improve the updating efficiency of a map.

Note that, the data in this embodiment is not specific to a specific user, and cannot reflect personal information of a specific user. It should be noted that, the data in this embodiment comes from the public data set.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

In order for the reader to more fully understand the principles of the implementations of the present disclosure, the embodiments are now further refined in conjunction with the following fig. 1-8.

Fig. 1 is a flowchart of a method for updating a map in real time based on a crosswalk according to an embodiment of the present disclosure, which may be performed by a device for updating a map in real time based on a crosswalk. As shown in fig. 1, the method comprises the steps of:

s101, acquiring a current aerial view image of a road where a vehicle is located and a mask image corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view angle corresponding to the high-precision map, the mask image indicates a crosswalk, and a road on which a vehicle is located comprises the crosswalk.

For example, during the process of the vehicle traveling on the road, an image of the environment where the vehicle is located may be acquired in real time, that is, an image of the environment where the vehicle is located may be acquired. The vehicle may have one or more image capture devices mounted thereon, for example, the image capture devices may be cameras. And (3) based on the image acquisition equipment installed on the vehicle, acquiring an image of the road where the vehicle is located. If a plurality of cameras are installed on the vehicle, a plurality of local images of the road can be acquired due to the fact that the cameras are provided with a certain acquisition range, and a global image of the road can be obtained according to the acquired local images. For example, a plurality of local images may be stitched into a global image.

And converting the acquired image into a BEV (Bird's Eye View) form, and obtaining an environment image of the Bird's Eye View as a current Bird's Eye image of the road where the vehicle is located. For example, the global image after stitching may be converted into a current bird's-eye view image, or the local images may be respectively converted into BEVs and stitched into the current bird's-eye view image. Thus, the current aerial view image of the road where the vehicle is can be determined in real time.

A high-precision map of each road is stored in advance, and can be acquired when a vehicle runs on the road. In this embodiment, the high-precision map is updated for the crosswalk, so that the high-precision map including the crosswalk can be obtained, that is, the crosswalk can be indicated in the high-precision map. The high-precision map is a map in a 3D form, and can be converted into a mask map of BEV as a mask image. The Mask image is a black-and-white image, and in the Mask image corresponding to the high-precision map, the crosswalk may be white, and the non-crosswalk may be black. The high-precision map of each road can be converted in advance, and mask images corresponding to the high-precision map are stored. When the vehicle runs on the road, the mask image of the road can be directly acquired. When the vehicle runs on the road, a high-precision map of the road can be acquired first, and then the high-precision map is converted into a mask image. In the present embodiment, the manner of converting the high-precision map into the mask image is not particularly limited. Different road lines are marked in advance in the high-precision map, and the position of a crosswalk of a road where a vehicle is located in the high-precision map can be determined according to preset marks. And cutting the high-precision map aiming at the crosswalk to obtain the high-precision map for indicating the crosswalk, thereby obtaining the mask image for indicating the crosswalk.

S102, performing image superposition processing on the current aerial view image based on the mask image to generate a superposition image.

For example, the current bird's eye image reflects the actual condition of the road, but there may be a problem in that photographing is incomplete or unclear. The mask image is derived from a high-definition map, which is a map stored in advance, having detailed road information, but may have erroneous road information. For example, the position of the crosswalk on the road changes, and the update is not performed in time on the high-precision map, so that the position of the crosswalk on the high-precision map is an erroneous position, that is, the position of the crosswalk on the mask image is an erroneous position.

Combining the current aerial view image with the mask image, and performing image superposition processing on the current aerial view image on the basis of the mask image to fuse the current aerial view image onto the mask image. And determining the image obtained after superposition as a superposition image.

When the image superposition processing is performed, whether a crosswalk exists in the current aerial view image can be judged first, and if the crosswalk does not exist, the execution of S102 and S103 in the embodiment is not required; if the crosswalk exists, the mask image and the current aerial view image can be subjected to image superposition processing. The crosswalk on the current bird's-eye view image can be covered on the black-and-white mask image, for example, the position and the shape of the crosswalk on the current bird's-eye view image can be determined, the crosswalk on the current bird's-eye view image is converted into a white area, and the white area is directly overlapped on the corresponding position on the mask image. That is, the superimposed image may be a mask image in black and white. In this embodiment, when the image superimposition processing is performed, the area of the crosswalk in the mask image may be first removed, and then the crosswalk may be added to the corresponding position of the mask image according to the position of the crosswalk in the current bird's-eye view image, so as to obtain the superimposed image.

And S103, determining that the position of the crosswalk changes according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image, and updating the high-precision map.

Illustratively, after the superimposed image is obtained, the position point of the crosswalk indicated in the superimposed image is determined, and the position point of the crosswalk indicated in the mask image is determined. The crosswalk of the intersection comprises a plurality of line segments, and the position point of the crosswalk can be the center point of a graph formed by the line segments. For example, a crosswalk is formed in a rectangular shape from a plurality of line segments, and the position point of the crosswalk can be represented by coordinates of the center point of the rectangle. That is, the center point coordinates of the crosswalk indicated in the superimposed image are determined, and the center point coordinates of the crosswalk indicated in the mask image are determined.

And judging whether the position of the crosswalk changes according to the position point of the crosswalk indicated in the superimposed image and the position point of the crosswalk indicated in the mask image. For example, it may be determined whether the coordinates of the center point of the white area in the superimposed image are identical to the coordinates of the center point of the white area in the mask image, and if so, it is determined that the position of the crosswalk is not changed; if the positions of the crosswalk are inconsistent, the positions of the crosswalk are determined to be changed. For example, a crosswalk on a road is newly added with two line segments, so that the coordinates of the center point of the crosswalk are changed, and the length of a rectangle represented by the crosswalk is increased.

The change of the position of the crosswalk means that the actual position of the crosswalk in the road is different from the position of the crosswalk in the high-precision map. If the position of the crosswalk is determined to be changed, updating a high-precision map according to the changed position of the crosswalk; if the position of the crosswalk is not changed, the high-precision map does not need to be updated. When the high-precision map is updated, information such as the position and the shape of the crosswalk in the superimposed image can be determined, and the crosswalk in the high-precision map is replaced according to the information such as the position and the shape of the crosswalk in the superimposed image. That is, the position and shape of the crosswalk are adjusted.

In this embodiment, a method for automatic driving based on a vehicle is further provided, where the vehicle may determine the current position in real time, and obtain the position point of the crosswalk corresponding to the current position based on the updated high-precision map; and controlling the vehicle to automatically drive according to the position points of the crosswalk in the updated high-precision map.

Specifically, the vehicle collects images of the front road in real time in the running process, and judges whether the pedestrian crossing on the front road changes in position or not, so that the position point of the pedestrian crossing in the high-precision map is updated in real time.

According to the updated high-precision map, an automatic driving decision can be made, so that the vehicle can automatically drive according to the latest high-precision map. For example, when the position of the pedestrian crossing in the high-precision map is changed, the pedestrian can be automatically decelerated and slowed down when the current position of the vehicle is driven to the position of the new pedestrian crossing, and the pedestrian is prevented from being touched.

The beneficial effect that sets up like this lies in, when the vehicle is traveling, the high-definition map in the place ahead is updated in advance to carry out the control of autopilot according to the high-definition map of latest, make the driving decision accord with actual road conditions, improve autopilot's control accuracy, prevent better that the vehicle from appearing passing through the problem that crosswalk did not slow down because of the change of crosswalk, can promote the security that autopilot vehicle passed effectively.

In the embodiment of the disclosure, a mask image corresponding to a current aerial view image and a high-precision map is obtained in real time in a vehicle driving process, and image superposition processing is performed on the mask image and the current aerial view image to generate a superposition image. And determining whether the position of the crosswalk changes according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image, and if so, updating the high-precision map. The position of the crosswalk is automatically detected in real time, the road is not required to be detected regularly, and the map updating efficiency is improved. Therefore, the vehicle is ensured to run according to the latest high-precision map, and the driving safety is improved.

Fig. 2 is a flow chart of a map real-time updating method based on crosswalk according to an embodiment of the disclosure.

In this embodiment, the current bird's eye image is subjected to image superimposition processing based on the mask image, and a superimposed image is generated, which may be thinned: performing feature extraction processing on the current aerial view image to obtain a feature vector of the current aerial view image; performing feature extraction processing on the mask image to obtain a feature vector of the mask image; and performing image superposition processing on the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image to obtain a superposition image.

As shown in fig. 2, the method comprises the steps of:

s201, acquiring a current aerial view image of a road where a vehicle is located and a mask image of the road corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road on which the vehicle is located includes a crosswalk.

For example, this step may refer to step S101, and will not be described in detail.

S202, performing feature extraction processing on the current aerial view image to obtain a feature vector of the current aerial view image; and performing feature extraction processing on the mask image to obtain feature vectors of the mask image.

For example, after the current bird's-eye view image and the mask image are obtained, feature extraction processing may be performed on the current bird's-eye view image and the mask image, respectively. For example, a model structure of the neural network may be preset, and feature extraction may be performed on the image through a convolutional layer, a pooling layer, and other network layers. In the present embodiment, the model structure of the neural network for extracting the feature vector is not particularly limited.

The feature extraction processing can be performed on the current aerial view image to obtain a feature vector of the current aerial view image, and the feature extraction processing can be performed on the mask image to obtain a feature vector of the mask image. In the present embodiment, the processing order of performing the feature extraction processing on the current bird's-eye view image and performing the feature extraction processing on the mask image is not particularly limited, and the current bird's-eye view image and the mask image may be processed simultaneously. The feature vector of the current aerial view image can represent information such as the position, width, height and inclination of the crosswalk in the current aerial view image; the feature vectors of the mask image may represent information such as the position, width, height, and inclination of the traversers in the mask image.

S203, performing image superposition processing on the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image to obtain a superposition image.

For example, the image superimposing process may be performed on the current bird's-eye view image and the mask image, and the feature vector of the current bird's-eye view image and the feature vector of the mask image may be fused. And decoding the calculated feature vector to obtain an image corresponding to the calculated feature vector, and taking the image as a superposition image. That is, the calculated feature vector is the feature vector of the superimposed image. By combining the current aerial view image and the mask image, the layout condition of the current crosswalk of the road can be determined, the accurate detection of the crosswalk is realized, and the updating precision of the crosswalk in the high-precision map is improved.

In this embodiment, according to the feature vector of the current aerial view image and the feature vector of the mask image, performing image superposition processing on the current aerial view image and the mask image to obtain a superimposed image, including: fusing the feature vector of the current aerial view image and the feature vector of the mask image to determine a target feature vector; the target feature vector is a feature vector for representing the superimposed image; and generating a superposition image according to the target feature vector.

Specifically, the feature vector obtained by fusing the feature vector of the current bird's eye view image and the feature vector of the mask image is determined as the target feature vector. For example, the weight of the feature vector of the current aerial view image and the weight of the feature vector of the mask image may be preset, and the feature vector of the current aerial view image and the feature vector of the mask image may be weighted and summed according to the preset weight to obtain the target feature vector.

The fusion process may be to add the feature vector of the current aerial image and the feature vector of the mask image, for example, the feature vector of the current aerial image is a 200×256 matrix, the feature vector of the mask image is also a 200×256 matrix, and when adding, two corresponding elements in the two matrices may be added, so that the obtained target feature vector is also a 200×256 matrix.

From the target feature vector, a superimposed image can be obtained. The process of obtaining the feature vector according to the image is a coding process, and the process of obtaining the image according to the feature vector is a decoding process. The encoding and decoding can be performed by adopting respective neural network models, namely, the target feature vector can be decoded by a preset neural network model, and the target feature vector is output to obtain a superimposed image.

The method has the advantages that the superimposed image can be obtained rapidly and accurately by fusing the characteristic vector of the current aerial view image and the characteristic vector of the mask image, errors in information of the crosswalk in the superimposed image are avoided, and the updating efficiency and the updating precision of the high-precision map are improved.

In this embodiment, the fusion processing is performed on the feature vector of the current aerial view image and the feature vector of the mask image, and the determination of the target feature vector includes: determining the similarity of the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image; and according to the similarity, carrying out fusion processing on the feature vector of the current aerial view image and the feature vector of the mask image, and determining a target feature vector.

Specifically, the similarity between the feature vector of the current bird's-eye view image and the feature vector of the mask image is calculated as the similarity between the current bird's-eye view image and the mask image. The similarity may represent a degree of coincidence of the crosswalk in the current bird's-eye image and the crosswalk in the mask image.

And according to the similarity, carrying out fusion processing on the feature vector of the current aerial view image and the feature vector of the mask image to obtain a target feature vector. For example, a similarity threshold may be preset, if the similarity is smaller than the preset similarity threshold, this indicates that the crosswalk is greatly changed, and the references of the crosswalk in the high-precision map are lower, that is, the references of the crosswalk in the mask image are lower, and at this time, the feature vector of the current bird's-eye image may be directly determined as the target feature vector. For example, the weight of the feature vector of the mask image may be set to 0 and the weight of the feature vector of the current bird's-eye image may be set to 1. If the similarity is equal to or greater than a preset similarity threshold, the fact that the crosswalk does not change greatly is indicated, and fusion processing can be carried out on the feature vector of the current aerial view image and the feature vector of the mask image to obtain a target feature vector. For example, if the current aerial view image is not clear of the far-end shooting of the road, the part of the crosswalk in the current aerial view image and the far-end part of the road in the mask image can be combined to obtain the complete target feature vector.

When the feature vector of the current aerial view image and the feature vector of the mask image are fused, the feature vector to be fused in the two images can be determined according to the pixel positions corresponding to the feature vector of the current aerial view image and the feature vector of the mask image, redundant feature vectors are screened out, and the determination accuracy of the target feature vector is improved. For example, the feature vector of the crosswalk part in the current bird's-eye image can be reserved, and the feature vector of the far end of the road can be screened out.

The method has the advantages that the similarity of the current aerial view image and the mask image is calculated, different fusion modes are executed according to the similarity, the determination accuracy of the target feature vector is improved, and then the updating accuracy of the high-precision map is improved.

In this embodiment, determining the similarity between the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image includes: determining the information of an included angle between the characteristic vector of the current aerial view image and the characteristic vector of the mask image; and determining the similarity of the current aerial view image and the mask image according to the included angle information.

Specifically, the included angle information between the feature vector of the current aerial view image and the feature vector of the mask image is calculated, and the included angle information can be expressed as a cosine included angle. And determining the similarity of the current aerial view image and the mask image according to the included angle information. For example, the smaller the cosine included angle is, the greater the similarity between the current aerial view image and the mask image is; the larger the cosine included angle is, the smaller the similarity between the current aerial view image and the mask image is. And if the included angle is 90 degrees, the similarity is 0.

The beneficial effects of setting up like this lie in, through calculating contained angle information, can confirm the similarity fast, the similarity of this embodiment can be cosine similarity, effectively improves the determination efficiency of target feature vector.

In this embodiment, according to the similarity, fusion processing is performed on the feature vector of the current aerial view image and the feature vector of the mask image, and the determining of the target feature vector includes: determining a first weight of a feature vector of the current aerial view image and a second weight of a feature vector of the mask image according to the similarity; wherein the first weight is greater than the second weight; and according to the first weight and the second weight, calculating weights of the feature vector of the current aerial view image and the feature vector of the mask image to obtain a target feature vector.

Specifically, the association relationship between different similarities and a first weight and a second weight may be preset, where the first weight is a weight of a feature vector of the current aerial view image, and the second weight is a weight of a feature vector of the mask image. And determining a first weight and a second weight corresponding to the similarity according to the determined similarity. The reality of the road condition represented by the current bird's-eye view image is higher, that is, the referential of the current bird's-eye view image is higher than the referential of the mask image, and therefore the preset first weight is greater than the second weight.

And according to the first weight and the second weight, calculating weights of the feature vector of the current aerial view image and the feature vector of the mask image to obtain a target feature vector. For example, the first weight may be multiplied by a feature vector of the current aerial image to obtain a first vector; and multiplying the second weight by the feature vector of the mask image to obtain a second vector. And adding the first vector and the second vector to obtain the target feature vector.

The method has the advantages that different weights are determined according to different similarities, the target feature vector can be determined according to the change degree of the crosswalk, the change of the crosswalk in the high-precision map can be accurately detected, and the updating precision of the high-precision map is improved.

In this embodiment, determining, according to the similarity, a first weight of a feature vector of the current bird's eye view image and a second weight of a feature vector of the mask image includes: determining the numerical range of the similarity; and determining a first weight of the feature vector of the current aerial view image and a second weight of the feature vector of the mask image according to the association relation between the preset numerical range and the weights.

Specifically, a numerical range of similarity is preset, and for different numerical ranges, different first weights and second weights are associated. After obtaining the similarity, determining the numerical range in which the similarity is located. And determining the weight corresponding to the numerical range where the similarity is located according to the association relation between the preset numerical range and the weight, namely determining the first weight of the feature vector of the current aerial view image and the second weight of the feature vector of the mask image.

For example, the preset numerical ranges include a range of 0 to 0.5 and a range of 0.5 to 1, and if the similarity is between 0 and 0.5, the first weight is determined to be 1, and the second weight is determined to be 0, that is, the target feature vector is determined only according to the feature vector of the current bird's-eye view image. If the similarity is between 0.5 and 1, determining that the first weight is 0.7, and the second weight is 0.3, namely, determining a target feature vector by taking the feature vector of the current aerial view image as a main component and taking the feature vector of the mask image as an auxiliary component.

The method has the advantages that the first weight and the second weight are determined pertinently according to the similarity, the determination accuracy of the target feature vector is improved, and the target feature vector accords with the actual condition of the road.

In this embodiment, generating a superimposed image according to the target feature vector includes: inputting the target feature vector into a preset feedforward neural network model, decoding the target feature vector based on the preset feedforward neural network model, and outputting to obtain a superimposed image; the preset feedforward neural network model is used for decoding the characteristic vector into an image.

Specifically, an FFN (Feed Forward Networks, feedforward neural network) model is set in advance, which can be used to decode the feature vector, i.e., convert the feature vector into an image. The input of the FFN model is a feature vector, and the output is an image.

And inputting the target feature vector into a preset FFN model, decoding the target feature vector through the FFN model, and outputting to obtain a superimposed image. The superimposed image is a mask image of a bird's eye view.

The method has the advantages that the superimposed image can be obtained rapidly through the FFN network, so that whether the pedestrian crosswalk is changed or not is judged according to the center point coordinates of the pedestrian crosswalk in the superimposed image, and the updating efficiency of the high-precision map is improved.

S204, in response to the fact that the position of the crosswalk is determined to be changed according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image, the high-precision map is updated.

For example, this step may refer to step S103, and will not be described in detail.

Fig. 3 is a schematic diagram of generation of a superimposed image according to the present embodiment. In fig. 3, BEV Map represents a current bird's eye view image, HD Map represents a mask image corresponding to a high-precision Map, and white area represents a crosswalk. And fusing the BEV Map and the HD Map, and decoding by a Decoder to obtain a superimposed image, wherein a white area in the superimposed image also represents the crosswalk.

Fig. 4 is a flow chart of a map real-time updating method based on crosswalk according to an embodiment of the present disclosure.

In this embodiment, the obtaining of the mask image corresponding to the high-precision map may be thinned as: acquiring a high-precision map of the road used during a historical period of time; and converting the high-precision map into a mask map under the aerial view angle, and taking the mask map as a mask image.

As shown in fig. 4, the method comprises the steps of:

s401, acquiring a current aerial view image of a road where a vehicle is located and a high-precision map of the road used in a historical time period, and converting the high-precision map into a mask map under the aerial view angle as a mask image.

Illustratively, a current bird's-eye image of a road on which the vehicle is located is acquired in real time, the current bird's-eye image being generated from an environmental image acquired by an image acquisition device on the vehicle. For example, three cameras are mounted on the vehicle, each camera can acquire an environmental image of a road, and the environmental images acquired by the cameras are projected to the BEV to obtain a current bird's-eye image.

The high-precision map of the road where the vehicle is located can be obtained in real time, and the high-precision map of the road can be obtained when the vehicle enters the road. The high-precision map is a pre-stored map, and the high-precision map may be different in different time periods, for example, the high-precision map may be updated once every month. The high-precision map of the road in the history period is acquired, for example, the high-precision map in the month immediately before the current time may be acquired, that is, the latest high-precision map may be acquired.

The high-precision map is a 3D map, and the visual angle of the high-precision map is the visual angle of human eyes. After the high-precision map is obtained, the high-precision map can be converted into a mask image under the bird's eye view angle, and the mask image can be used as a mask image corresponding to the high-precision map. The visual angle of the converted high-precision map is consistent with the visual angle of the current aerial view image, so that the image superposition processing of the converted high-precision map and the current aerial view image is facilitated, and the position of the crosswalk is prevented from being wrong after the images with different visual angles are superposed. And the pedestrian crosswalk in the high-precision map is represented in a mask mode, so that the geometric features of the pedestrian crosswalk can be conveniently identified, and the updating precision of the high-precision map is improved.

In this embodiment, converting a high-precision map into a mask map at a bird's eye view angle, as a mask image, includes: converting the high-precision map into an image in a mask form, and taking the image as an initial image; and converting the view angle of the initial image into a bird's eye view angle to obtain a mask image.

Specifically, the 3D high-precision map may be first converted into an image in the form of a 2D mask as an initial image. The white areas in the initial image may represent humanoid crossroads and the black areas represent non-crossroads. The 3D image may be changed into a 2D image by performing projection based on preset internal and external parameters. The internal parameters and the external parameters represent preset camera parameters, for example, the internal parameters may be a focal length, a pixel size, etc. of the image, and the external parameters may be a rotation direction, etc. of the image.

After the initial image is obtained, the initial image is projected to the view angle of the BEV to obtain a mask image with a bird's eye view angle.

The method has the beneficial effects that the high-precision map is converted into the mask-type image, and then the mask-type image is projected to the BEV, so that the mask image with the aerial view angle is obtained, the mask image and the current aerial view image are combined conveniently, and the accurate detection of the crosswalk is realized.

S402, performing image superposition processing on the current aerial view image based on the mask image to generate a superposition image.

For example, this step may refer to step S102, and will not be described in detail.

S403, updating the high-precision map in response to the fact that the position of the crosswalk is determined to change according to the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image.

Fig. 5 is a block diagram of a map real-time updating device based on crosswalk according to an embodiment of the present disclosure. For ease of illustration, only portions relevant to embodiments of the present disclosure are shown. Referring to fig. 5, the crosswalk-based map real-time updating apparatus 500 includes: an acquisition unit 501, a generation unit 502, and an update unit 503.

An obtaining unit 501, configured to obtain a current aerial view image of a road where a vehicle is located and a mask image corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road on which the vehicle is located comprises a crosswalk;

a generating unit 502, configured to perform image superimposition processing on the current aerial view image based on the mask image, and generate a superimposed image;

an updating unit 503 configured to update the high-precision map in response to a determination that the position of the crosswalk changes from the position point of the crosswalk indicated by the superimposed image and the position point of the crosswalk indicated by the mask image.

Fig. 6 is a block diagram of a map real-time updating device based on crosswalk according to an embodiment of the present disclosure, as shown in fig. 6, the map real-time updating device 600 based on crosswalk includes an obtaining unit 601, a generating unit 602 and an updating unit 603, wherein the generating unit 602 includes a feature extracting module 6021 and an overlapping processing module 6022.

The feature extraction module 6021 is configured to perform feature extraction processing on the current aerial view image to obtain a feature vector of the current aerial view image; extracting features of the mask image to obtain feature vectors of the mask image;

and a superposition processing module 6022, configured to perform image superposition processing on the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image, so as to obtain the superposition image.

In one example, the overlay processing module 6022 includes:

the fusion processing sub-module is used for carrying out fusion processing on the characteristic vector of the current aerial view image and the characteristic vector of the mask image to determine a target characteristic vector; wherein the target feature vector is a feature vector characterizing the superimposed image;

and the image generation sub-module is used for generating the superimposed image according to the target feature vector.

In one example, a fusion processing sub-module includes:

a first determining submodule, configured to determine a similarity between the current aerial view image and the mask image according to a feature vector of the current aerial view image and a feature vector of the mask image;

And the second determining submodule is used for carrying out fusion processing on the characteristic vector of the current aerial view image and the characteristic vector of the mask image according to the similarity and determining the target characteristic vector.

In one example, the first determining sub-module is specifically configured to:

determining the included angle information between the characteristic vector of the current aerial view image and the characteristic vector of the mask image;

and determining the similarity of the current aerial view image and the mask image according to the included angle information.

In one example, the second determining sub-module is specifically configured to:

determining a first weight of the feature vector of the current aerial view image and a second weight of the feature vector of the mask image according to the similarity; wherein the first weight is greater than the second weight;

and according to the first weight and the second weight, performing weight calculation on the feature vector of the current aerial view image and the feature vector of the mask image to obtain the target feature vector.

determining the numerical range of the similarity;

and determining a first weight of the feature vector of the current aerial view image and a second weight of the feature vector of the mask image according to the association relation between the preset numerical range and the weights.

In one example, the image generation sub-module is specifically configured to:

inputting the target feature vector into a preset feedforward neural network model, and decoding the target feature vector based on the preset feedforward neural network model to obtain the superimposed image.

In one example, the obtaining unit 601 includes:

a map acquisition module for acquiring a high-precision map of the road used in a history period;

and the map conversion module is used for converting the high-precision map into a mask image under the aerial view angle, and the mask image is obtained.

In one example, a map conversion module includes:

the first conversion sub-module is used for converting the high-precision map into an image in a mask form to serve as an initial image;

and the second conversion sub-module is used for converting the view angle of the initial image into a bird's eye view angle to obtain the mask image.

According to the embodiment of the disclosure, a map real-time updating device based on crosswalk is also provided. The map real-time updating device based on the crosswalk can comprise: a location point determination unit and a control unit.

A position point determining unit, configured to determine a current position of a vehicle, and obtain a position point of a crosswalk corresponding to the current position based on an updated high-precision map in the apparatus according to any one of claims 12 to 21;

According to an embodiment of the disclosure, the disclosure further provides an electronic device.

Fig. 7 is a block diagram of an electronic device according to an embodiment of the disclosure, and as shown in fig. 7, an electronic device 700 includes: at least one processor 702; and a memory 701 communicatively coupled to the at least one processor 702; wherein the memory stores instructions executable by the at least one processor 702 to enable the at least one processor 702 to perform the crosswalk-based map real-time updating method of the present disclosure.

The electronic device 700 further comprises a receiver 703 and a transmitter 704. The receiver 703 is configured to receive instructions and data transmitted from other devices, and the transmitter 704 is configured to transmit instructions and data to external devices.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

Fig. 8 illustrates a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 801 performs the respective methods and processes described above, for example, a real-time map updating method based on a crosswalk. For example, in some embodiments, the crosswalk-based map real-time updating method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the crosswalk-based map real-time updating method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the crosswalk-based map real-time update method in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A map real-time updating method based on a crosswalk comprises the following steps:

acquiring a current aerial view image of a road where a vehicle is located and a mask image corresponding to a high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road where the vehicle is located comprises a crosswalk;

2. The method of claim 1, wherein performing image superimposition processing on the current bird's-eye image based on the mask image, generating a superimposed image, comprises:

performing feature extraction processing on the current aerial view image to obtain a feature vector of the current aerial view image; extracting features of the mask image to obtain feature vectors of the mask image;

and performing image superposition processing on the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image to obtain the superposition image.

3. The method according to claim 2, wherein performing image superimposition processing on the current bird's-eye view image and the mask image according to the feature vector of the current bird's-eye view image and the feature vector of the mask image, to obtain the superimposed image, includes:

Performing fusion processing on the feature vector of the current aerial view image and the feature vector of the mask image to determine a target feature vector; wherein the target feature vector is a feature vector characterizing the superimposed image;

and generating the superimposed image according to the target feature vector.

4. A method according to claim 3, wherein the fusing of the feature vector of the current bird's-eye image and the feature vector of the mask image to determine a target feature vector comprises:

determining the similarity of the current aerial view image and the mask image according to the feature vector of the current aerial view image and the feature vector of the mask image;

and according to the similarity, carrying out fusion processing on the characteristic vector of the current aerial view image and the characteristic vector of the mask image, and determining the target characteristic vector.

5. The method of claim 4, wherein determining the similarity of the current bird's-eye view image and the mask image from the feature vector of the current bird's-eye view image and the feature vector of the mask image comprises:

6. The method according to claim 4, wherein the fusing the feature vector of the current bird's-eye view image and the feature vector of the mask image according to the similarity, determining the target feature vector, includes:

7. The method of claim 6, wherein determining a first weight of a feature vector of the current aerial image and a second weight of a feature vector of the mask image based on the similarity comprises:

determining the numerical range of the similarity;

8. The method of any of claims 3-7, wherein generating the superimposed image from the target feature vector comprises:

9. The method of any of claims 1-7, wherein obtaining a mask image corresponding to a high-precision map comprises:

acquiring a high-precision map of the road used during a historical period of time;

and converting the high-precision map into a mask map under the aerial view angle, and taking the mask map as the mask image.

10. The method of claim 9, wherein converting the high-precision map into a mask map at a bird's eye view angle, as the mask image, comprises:

converting the high-precision map into an image in a mask form to serve as an initial image;

and converting the view angle of the initial image into an aerial view angle to obtain the mask image.

11. A method of vehicle-based autopilot comprising:

determining the current position of a vehicle, and acquiring a position point of a crosswalk corresponding to the current position based on the updated high-precision map in the method of any one of claims 1-10;

12. A real-time map updating device based on a crosswalk, comprising:

the acquisition unit is used for acquiring a current aerial view image of a road where the vehicle is located and a mask image corresponding to the high-precision map; the current aerial view image is an environment image under an aerial view angle acquired in real time; the mask image is a mask image of an aerial view corresponding to the high-precision map, and indicates a crosswalk; the road where the vehicle is located comprises a crosswalk;

13. The apparatus of claim 12, wherein the generating unit comprises:

the feature extraction module is used for carrying out feature extraction processing on the current aerial view image to obtain a feature vector of the current aerial view image; extracting features of the mask image to obtain feature vectors of the mask image;

And the superposition processing module is used for carrying out image superposition processing on the current aerial view image and the mask image according to the characteristic vector of the current aerial view image and the characteristic vector of the mask image to obtain the superposition image.

14. The apparatus of claim 13, wherein the superposition processing module comprises:

15. The apparatus of claim 14, wherein the fusion processing sub-module comprises:

16. The apparatus of claim 15, wherein the first determining sub-module is specifically configured to:

17. The apparatus of claim 15, wherein the second determining sub-module is specifically configured to:

18. The apparatus of claim 17, wherein the second determining sub-module is specifically configured to:

determining the numerical range of the similarity;

19. The apparatus according to any of claims 14-18, wherein the image generation sub-module is specifically configured to:

20. The apparatus according to any one of claims 12-19, wherein the acquisition unit comprises:

and the map conversion module is used for converting the high-precision map into a mask image under the aerial view angle, and taking the mask image as the mask image.

21. The apparatus of claim 20, wherein the map transformation module comprises:

22. An apparatus for vehicle-based autopilot, comprising:

23. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-11.

25. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any of claims 1-11.