WO2021233165A1

WO2021233165A1 - Method and apparatus for obtaining road semantic information, and device, and medium

Info

Publication number: WO2021233165A1
Application number: PCT/CN2021/093154
Authority: WO
Inventors: 梁伯均; 林逸群; 王哲
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2020-05-22
Filing date: 2021-05-11
Publication date: 2021-11-25
Also published as: CN111707277A; CN111707277B

Abstract

Provided are a method and apparatus for obtaining road semantic information, and a device, a computer-readable storage medium, and a computer program product. The method comprises: obtaining a road image (S101); dividing the road image into at least two sections according to the change condition of the number of lane lines in the road image (S102); and generating the road semantic information corresponding to each section, the road semantic information at least comprising road semantic elements in this section and a connection relationship between this section and the adjacent section (S103). By adoption of the method, the road semantic information conforming to a semantic format can be automatically obtained according to the lane lines in the road image, thereby improving the efficiency of obtaining the road semantic information.

Description

获取道路语义信息的方法、装置、设备及介质Method, device, equipment and medium for obtaining road semantic information

交叉引用声明Cross-reference statement

本申请要求于2020年5月22日提交中国专利局的申请号为202010444146.0的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202010444146.0 filed with the Chinese Patent Office on May 22, 2020, the entire content of which is incorporated into this application by reference.

技术领域Technical field

本公开涉及图像处理技术领域，尤其涉及获取道路语义信息的方法、装置、设备、计算机可读存储介质及计算机程序产品。The present disclosure relates to the field of image processing technology, and in particular to methods, devices, equipment, computer-readable storage media, and computer program products for obtaining road semantic information.

背景技术Background technique

高精度地图包含丰富的道路元素。道路元素需要遵循特定的格式，例如OpenDrive，导航数据标准(navigation data standard，NDS)等语义格式，转化为相应的道路语义信息加以存储。The high-resolution map contains a wealth of road elements. Road elements need to follow a specific format, such as OpenDrive, navigation data standard (navigation data standard, NDS) and other semantic formats, which are converted into corresponding road semantic information for storage.

根据高精度地图中道路图像上的车道线，根据某种语义格式获取道路语义信息需要经过多个步骤。这一过程如果由人工标注完成，工作量大且容易出错。According to the lane lines on the road image in the high-precision map, obtaining road semantic information according to a certain semantic format requires multiple steps. If this process is completed by manual labeling, the workload is heavy and error-prone.

因此，如何提高获取道路语义信息的效率是本公开要解决的问题。Therefore, how to improve the efficiency of acquiring road semantic information is a problem to be solved by the present disclosure.

发明内容Summary of the invention

本公开提供获取道路语义信息的方案。The present disclosure provides a solution for obtaining road semantic information.

第一方面，提供了一种获取道路语义信息的方法，所述方法包括：获取道路图像；根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段；生成每一分段对应的道路语义信息，至少包括：所述分段中的道路语义元素，以及所述分段与相邻分段之间的连接关系。In a first aspect, a method for acquiring road semantic information is provided, the method comprising: acquiring a road image; dividing the road image into at least two segments according to a change in the number of lane lines in the road image; The generation of road semantic information corresponding to each segment includes at least: the road semantic elements in the segment and the connection relationship between the segment and adjacent segments.

第二方面，提供了一种获取道路语义信息的装置，所述装置包括：第一获取单元，用于获取道路图像；第一切分单元，用于根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段；第一生成单元，用于生成每一分段对应的道路语义信息，至少包括：所述分段中的道路语义元素，以及所述分段与相邻分段之间的连接关系。In a second aspect, a device for acquiring road semantic information is provided, the device comprising: a first acquiring unit for acquiring a road image; a first subdivision unit for acquiring a number of lane lines in the road image , The road image is divided into at least two segments; the first generating unit is used to generate the road semantic information corresponding to each segment, and at least includes: the road semantic elements in the segment, and the segment The connection relationship between the segment and the adjacent segment.

第三方面，提供了一种获取道路语义信息的设备，所述设备包括：存储器，用于存储计算机程序；处理器；其中，所述处理器用于调用所述存储器中存储的所述计算机程序，执行如第一方面的任一种所述的方法。In a third aspect, there is provided a device for obtaining road semantic information, the device comprising: a memory for storing a computer program; a processor; wherein the processor is used for calling the computer program stored in the memory, Perform the method as described in any one of the first aspect.

第四方面，提供了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如第一方面的任一种所述的方法。In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method as described in any one of the first aspect is implemented.

第五方面，提供了一种计算机程序产品，用于存储计算机可读指令，该计算机可读指令被执行时使得计算机执行如第一方面的任一种所述的方法。In a fifth aspect, a computer program product is provided for storing computer-readable instructions, which when executed, cause a computer to execute the method described in any one of the first aspects.

采用本公开的方案，具有如下有益效果：可以根据道路图像中的车道线，自动获取符合语义格式的道路语义信息，提高了获取道路语义信息的效率。Adopting the solution of the present disclosure has the following beneficial effects: the road semantic information conforming to the semantic format can be automatically obtained according to the lane lines in the road image, and the efficiency of obtaining road semantic information is improved.

附图说明Description of the drawings

为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

图1为本公开提供的获取道路语义信息的方法的一个流程示意图；FIG. 1 is a schematic flowchart of a method for obtaining road semantic information provided by the present disclosure;

图2为OpenDrive格式中的道路语义信息示意图；Figure 2 is a schematic diagram of road semantic information in OpenDrive format;

图3为本公开提供的获取道路语义信息的方法的又一个流程示意图；FIG. 3 is another schematic flowchart of the method for obtaining road semantic information provided by the present disclosure;

图4为道路图像中车道线的示意图；Figure 4 is a schematic diagram of lane lines in a road image;

图5为生成的多条车道骨骼线的示意图；Figure 5 is a schematic diagram of multiple lane skeleton lines generated;

图6为识别多条车道骨骼线的候选交叉位置的示意图；Fig. 6 is a schematic diagram of identifying candidate intersection positions of multiple lane skeleton lines;

图7为候选交叉位置聚类的示意图；Fig. 7 is a schematic diagram of candidate cross position clustering;

图8为识别出的多条车道骨骼线之间的交叉位置的示意图；Figure 8 is a schematic diagram of the identified crossing positions between multiple lane skeleton lines;

图9为确定与每个端点位置最近的交叉位置的示意图；Figure 9 is a schematic diagram of determining the nearest intersection position to each end point position;

图10为交叉位置与一个端点位置存在连线时的切分线的示意图；FIG. 10 is a schematic diagram of a cutting line when there is a connection between a crossing position and an end point position;

图11为交叉位置与多个端点位置存在连线时切分线的示意图；FIG. 11 is a schematic diagram of a cutting line when there is a connection between a crossing position and a plurality of end positions;

图12为合并交叉位置的示意图；Figure 12 is a schematic diagram of merging intersection positions;

图13为切分线切分后的分段示意图；Figure 13 is a schematic diagram of the segmentation after the slicing line is split;

图14为调整切分线的示意图；Figure 14 is a schematic diagram of adjusting the cutting line;

图15为切分后车道线的示意图；Figure 15 is a schematic diagram of lane lines after segmentation;

图16为又一切分后车道线的示意图；Figure 16 is a schematic diagram of the lane line after the division;

图17为本公开提供的获取道路语义信息的装置的一结构示意图；FIG. 17 is a schematic diagram of a structure of an apparatus for acquiring road semantic information provided by the present disclosure;

图18为本公开提供的获取道路语义信息的设备的一结构示意图。FIG. 18 is a schematic structural diagram of a device for acquiring road semantic information provided by the present disclosure.

具体实施方式Detailed ways

下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.

应理解，本公开中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。本公开中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。It should be understood that the term "and/or" in the present disclosure is only an association relationship describing associated objects, indicating that there can be three types of relationships, for example, A and/or B can mean that A alone exists, and A and A exist at the same time. B, there are three cases of B alone. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship. The terms "first", "second", etc. in the present disclosure are used to distinguish different objects, rather than to describe a specific order.

图1为本公开提供的获取道路语义信息的方法的一个流程示意图，该方法包括以下步骤。FIG. 1 is a schematic flowchart of a method for obtaining road semantic information provided by the present disclosure, and the method includes the following steps.

S101、获取道路图像，亦可称为目标道路图像。S101. Obtain a road image, which may also be referred to as a target road image.

首先可以从服务器或车载终端获取高精度地图，但不限于此，也可以是其他精度的地图等。高精度地图或其他精度的地图均包括完整的道路图像，该道路图像包括丰富的道路元素信息。然后，获取设定区域内的道路图像，以便后续对该设定区域内的道路图像获取道路语义信息，直至获取整个道路图像的道路语义信息。该设定区域可以是一个矩形区域，一个圆形区域，一条公路等。可以预先定义该设定区域的大小。First, a high-precision map can be obtained from a server or a vehicle-mounted terminal, but it is not limited to this, and it can also be a map of other precision. High-precision maps or maps with other accuracy include complete road images, which include rich road element information. Then, the road image in the set area is acquired, so as to subsequently obtain the road semantic information of the road image in the set area, until the road semantic information of the entire road image is acquired. The setting area can be a rectangular area, a circular area, a road, etc. The size of the setting area can be defined in advance.

S102、根据道路图像中车道线数目的变化，将道路图像切分为至少两个分段。为便于描述，该至少两个分段包括第一分段以及与第一分段相邻的第二分段。S102. Divide the road image into at least two segments according to the change in the number of lane lines in the road image. For ease of description, the at least two segments include a first segment and a second segment adjacent to the first segment.

在制作高精度地图时，需要根据某个语义格式，获取道路图像中的道路语义信息。该语义格式可以是OpenDrive，NDS等语义格式。可根据该语义格式生成道路语义信息，构建道路语义结构模型。When making a high-precision map, it is necessary to obtain the road semantic information in the road image according to a certain semantic format. The semantic format can be OpenDrive, NDS and other semantic formats. The road semantic information can be generated according to the semantic format, and the road semantic structure model can be constructed.

如图2所示，为OpenDrive格式中的道路语义信息示意图，其文件格式为XML(Extensible Markup Language，可扩展标记语言)。该XML文件中包含了丰富的道路语义信息，例如包括车道线(laneline)、车道(lane)、分段(section)、道路(road)以及车道间连接关系(lanelinks)等。在图2中，线条1左右两侧分别为分段1和分段2。横坐标为s，纵坐标为t，其中，s表示参考线(reference line)的方向，t表示相对于参考线的侧向位置。参考线将每个分段中的车道分为左车道(图中示为从参考线向t轴正向排列的车道)和/或右车道(图中示为从参考线向t轴负向排列的车道)。每个车道有相应的赋值，例如，左车道赋正值，右车道赋负值。在一个具体示例中，参考线赋值为0，则分段1中的左车道依次赋值为1、2、3，分段1的右车道依次赋值为-1、-2；分段2的左车道依次赋值为1、2、3，分段2的右车道依次赋值为-1、-2、-3。由此可见，分段1与分段2中的车道/车道线的数目不同。As shown in Figure 2, it is a schematic diagram of road semantic information in the OpenDrive format, and the file format is XML (Extensible Markup Language, Extensible Markup Language). The XML file contains a wealth of road semantic information, such as lanelines, lanes, sections, roads, lanelinks, and so on. In Figure 2, the left and right sides of line 1 are segment 1 and segment 2 respectively. The abscissa is s and the ordinate is t, where s represents the direction of the reference line, and t represents the lateral position relative to the reference line. The reference line divides the lanes in each segment into left lanes (shown in the figure as the lanes arranged from the reference line to the positive direction of the t-axis) and/or right lane (shown in the figure as being arranged from the reference line to the negative direction of the t-axis Lane). Each lane has a corresponding assignment, for example, the left lane is assigned a positive value, and the right lane is assigned a negative value. In a specific example, the reference line is assigned a value of 0, then the left lane in segment 1 is assigned the values 1, 2, and 3 in turn, the right lane of segment 1 is assigned the values -1 and -2 in turn; the left lane of segment 2 Assign the values 1, 2, and 3 in sequence, and assign the values of -1, -2, and -3 to the right lane of segment 2 in sequence. It can be seen that the number of lanes/lane lines in segment 1 and segment 2 is different.

具体地，在构建符合图2所示的语义格式的道路语义结构模型时，首先需要对道路图像进行分段。在每一分段内部，车道结构一致，即每一分段中的车道线数目相同，而不同分段包括的车道线数目不同。根据道路图像中识别的车道线，获得车道线之间的相应车道。其中，每相邻两条车道线之间形成一条车道。在车道线数目发生改变的位置，确定切分线，该切分线用于将道路图像切分为相邻的两个分段：第一分段和第二分段，两个分段中的车道线数量不同。例如，第二分段相比第一分段的车道线数量增多或减少。Specifically, when constructing a road semantic structure model that conforms to the semantic format shown in FIG. 2, it is first necessary to segment the road image. Within each segment, the lane structure is the same, that is, the number of lane lines in each segment is the same, but the number of lane lines included in different segments is different. According to the lane lines recognized in the road image, the corresponding lanes between the lane lines are obtained. Among them, a lane is formed between every two adjacent lane lines. At the position where the number of lane lines changes, determine the segmentation line. The segmentation line is used to divide the road image into two adjacent segments: the first segment and the second segment. The number of lane lines is different. For example, the number of lane lines in the second segment increases or decreases compared to the first segment.

S103、生成每一分段对应的道路语义信息，至少包括：该分段中的道路语义元素，以及该分段与相邻分段之间的连接关系。S103. Generate road semantic information corresponding to each segment, which includes at least: road semantic elements in the segment and the connection relationship between the segment and adjacent segments.

为构建道路语义结构模型，生成相应的道路语义信息，需要从道路图像中获取道路语义元素，以及分段间的连接关系。In order to build a road semantic structure model and generate corresponding road semantic information, it is necessary to obtain road semantic elements and the connection relationship between segments from the road image.

具体地，切分得到第一分段及相邻的第二分段后，生成第一和第二分段对应的道路语义信息。其中第一分段对应的道路语义信息至少包括：第一分段中的道路语义元素，以及第一分段与第二分段之间的连接关系。其中，第一分段中的道路语义元素可以包括以下元素中的一项或多项：车道线，车道线与车道中心线(如参考线)的相对位置关系，车道。其中，车道线与车道中心线的相对位置关系包括车道线位于车道中心线的物理左侧或物理右侧。其中，第一分段与第二分段之间的连接关系包括：第一分段与第二分段的前后继关系，以及第一分段与第二分段的车道连接关系。Specifically, after the first segment and the adjacent second segment are obtained by segmentation, the road semantic information corresponding to the first and second segments is generated. The road semantic information corresponding to the first segment includes at least: road semantic elements in the first segment, and the connection relationship between the first segment and the second segment. Wherein, the road semantic elements in the first segment may include one or more of the following elements: lane line, the relative position relationship between the lane line and the center line of the lane (such as the reference line), and the lane. Among them, the relative positional relationship between the lane line and the lane center line includes that the lane line is located on the physical left side or the physical right side of the lane center line. Wherein, the connection relationship between the first segment and the second segment includes: the successive relationship between the first segment and the second segment, and the lane connection relationship between the first segment and the second segment.

生成了各个分段对应的道路语义信息后，可以获取整个道路图像的道路语义信息。根据整个道路图像的道路语义信息，可以获得整个道路图像的道路语义结构。After the road semantic information corresponding to each segment is generated, the road semantic information of the entire road image can be obtained. According to the road semantic information of the entire road image, the road semantic structure of the entire road image can be obtained.

通过上述步骤获取道路图像的道路语义信息，无需人工标注，自动获得了各个分段的道路语义信息，提升了获取道路语义信息的效率。The road semantic information of the road image is acquired through the above steps, without manual labeling, the road semantic information of each segment is automatically obtained, and the efficiency of acquiring road semantic information is improved.

在实际操作过程中，可以将本方案作为一个插件集成到地图编辑标注工具中，自动获得道路图像的道路语义信息。In the actual operation process, this solution can be integrated into the map editing and labeling tool as a plug-in to automatically obtain the road semantic information of the road image.

根据本公开提供的一种获取道路语义信息的方法，可以根据道路图像中的车道线，自动获取道路语义信息，提高了获取道路语义信息的效率。According to a method for acquiring road semantic information provided by the present disclosure, road semantic information can be automatically acquired according to lane lines in a road image, which improves the efficiency of acquiring road semantic information.

图3为本公开提供的获取道路语义信息方法的又一个流程示意图，该方法包括以下步骤：Fig. 3 is another schematic flow chart of the method for obtaining road semantic information provided by the present disclosure. The method includes the following steps:

S201、获取道路图像。S201. Obtain a road image.

该步骤的具体实现可参考图1所示实施例的步骤S101。For the specific implementation of this step, refer to step S101 of the embodiment shown in FIG. 1.

在获取道路图像后，需要根据道路图像中车道线数目的变化情况，将道路图像切分为至少两个分段。具体地，可以根据每次道路图像中车道线数目发生改变的位置，得到道路图像的切分线，然后，利用切分线将道路图像切分为两个分段，例如第一分段以及与第一分段相邻的第二分段。After acquiring the road image, the road image needs to be divided into at least two segments according to the change in the number of lane lines in the road image. Specifically, the segmentation line of the road image can be obtained according to the position where the number of lane lines in the road image changes each time, and then the road image can be divided into two segments using the segmentation line, such as the first segment and the The first segment is adjacent to the second segment.

其中，根据每次道路图像中车道线数目发生改变的位置，得到道路图像的切分线，可以根据以下步骤S202和S203实现。Wherein, according to the position where the number of lane lines in the road image changes each time, the segmentation line of the road image is obtained, which can be implemented according to the following steps S202 and S203.

S202、获取道路图像中车道线的端点位置，生成多条车道骨骼线，以及识别多条车道骨骼线之间的一个或多个交叉位置。S202: Obtain the end positions of the lane lines in the road image, generate multiple lane skeleton lines, and identify one or more intersection positions between the multiple lane skeleton lines.

具体地，可以包括以下步骤A1～A4：Specifically, the following steps A1 to A4 may be included:

步骤A1，识别道路图像中的车道线。可以将道路图像中的车道线绘制出来。如图4所示的道路图像中的车道线的示意图，在该道路图像上显示出了道路图像中的多条车道线。可以采用本领域常见方法获取道路图像中的各条车道线，如通过曲线拟合，点云数据和道路图像融合辅助标识出车道线等方法，本公开对此不作限制。Step A1: Identify the lane line in the road image. The lane lines in the road image can be drawn. A schematic diagram of lane lines in a road image as shown in FIG. 4, on which multiple lane lines in the road image are displayed. Common methods in the art can be used to obtain each lane line in the road image, such as curve fitting, point cloud data and road image fusion to assist in identifying the lane line, etc. The present disclosure does not limit this.

步骤A2，生成多条车道骨骼线。Step A2, generate multiple lane skeleton lines.

由于车道线可能存在不连续的情况，位于车道线附近的、与车道线不连续的孤立的像素点，在生成车道骨骼线时，这些孤立的像素点极有可能生成冗余的车道骨骼线。因此，在生成多条车道骨骼线之前，需要对道路图像进行图像二值化处理(image binarization)。首先，获取道路图像上每个像素点到最近的车道线的距离，即计算道路图像上每个像素点到最近的车道线的距离。其中，像素点可能位于两条车道线之间，可以先大致估计像素点分别与两条车道线的距离，确定离像素点最近的车道线。然后，根据每个像素点到最近的车道线的距离与第一距离阈值的关系，对道路图像进行二值化处理。具体地，预先设置一个第一距离阈值，判断某一像素点到最近的车道线的距离值大于上述第一距离阈值时，则将该像素点的值置为0；判断该像素点到最近的车道线的距离值小于或等于上述第一距离阈值时，则将该像素点的值置为1。在另外的示例中，也可以是，当该像素点到最近的车道线的距离值大于上述第一距离阈值时，则将该像素点的值置为1；当该像素点到最近的车道线的距离值小于或等于上述第一距离阈值时，则将该像素点的值置为0。该第一距离阈值可以根据经验值设定。Since the lane line may be discontinuous, isolated pixels located near the lane line and discontinuous with the lane line are very likely to generate redundant lane bone lines when generating lane skeleton lines. Therefore, before generating multiple lane skeleton lines, the road image needs to be image binarized. First, obtain the distance from each pixel on the road image to the nearest lane line, that is, calculate the distance from each pixel on the road image to the nearest lane line. Among them, the pixel point may be located between two lane lines, and the distance between the pixel point and the two lane lines can be roughly estimated first, and the lane line closest to the pixel point can be determined. Then, according to the relationship between the distance from each pixel to the nearest lane line and the first distance threshold, the road image is binarized. Specifically, a first distance threshold is set in advance, and when it is determined that the distance from a certain pixel to the nearest lane line is greater than the first distance threshold, the value of the pixel is set to 0; the pixel is determined to be the nearest When the distance value of the lane line is less than or equal to the above-mentioned first distance threshold value, the value of the pixel point is set to 1. In another example, it may also be that when the distance from the pixel to the nearest lane line is greater than the first distance threshold, the value of the pixel is set to 1; when the pixel to the nearest lane When the distance value of is less than or equal to the above-mentioned first distance threshold, the value of the pixel is set to 0. The first distance threshold can be set according to empirical values.

进一步地，对像素点置为指定值的区域进行图像细化处理，得到多条车道骨骼线。每条车道骨骼线对应一个细化处理后的区域。在一个具体示例中，假设在上述图像二值化处理过程中，当像素点到最近的车道线的距离值大于上述第一距离阈值时，则将该像素点的值置为0，可以通过图像细化算法，对上述像素点为0的区域进行图像细化处理，得到多条车道骨骼线。其中，图像细化是指将图像的线条从多像素宽度减少到单位像素宽度。相邻两条车道线之间的区域为一条车道，通过图像细化处理，将车道细化为一条线，即得到一条车道骨骼线。如图5所示，为生成的多条车道骨骼线的示意图。其中，实线为识别出的车道线，虚线为生成的多条车道骨骼线。Further, the image thinning process is performed on the area where the pixel points are set to the specified value, and multiple lane skeleton lines are obtained. Each lane bone line corresponds to a refined area. In a specific example, suppose that in the image binarization process, when the distance value from the pixel to the nearest lane line is greater than the first distance threshold, then the value of the pixel is set to 0, which can be passed through the image The thinning algorithm performs image thinning processing on the above-mentioned area where the pixel point is 0, and obtains multiple lane skeleton lines. Among them, image thinning refers to the reduction of image lines from multi-pixel width to unit pixel width. The area between two adjacent lane lines is a lane. Through image thinning processing, the lane is thinned into a line, that is, a lane skeleton line is obtained. As shown in Figure 5, it is a schematic diagram of the generated multiple lane skeleton lines. Among them, the solid line is the identified lane line, and the dashed line is the generated multiple lane skeleton lines.

步骤A3，获取车道线的端点位置。Step A3, obtain the end position of the lane line.

每条车道线都有起点和终点，识别车道线的起点和终点，即得到车道线的端点位置。Each lane line has a start point and an end point. Identify the start point and end point of the lane line to get the end position of the lane line.

步骤A4，识别多条车道骨骼线之间的一个或多个交叉位置。Step A4: Identify one or more intersection positions between multiple lane bone lines.

若车道线数目发生变化，则生成的多条车道骨骼线可能会产生交叉，因此，需要识别多条车道骨骼线之间的交叉位置。反过来说，通过识别车道骨骼线之间的交叉位置，可以确定车道线数目发生改变。If the number of lane lines changes, the generated multiple lane bone lines may intersect. Therefore, it is necessary to identify the intersection position between multiple lane bone lines. Conversely, by identifying the intersection position between the lane bone lines, it can be determined that the number of lane lines has changed.

具体地，遍历多条车道骨骼线上的第一像素点。以每个第一像素点为中心，获取设定区域内的连通分支数目。连通分支是指从该第一像素点出发的不同的车道骨骼线，且这些车道骨骼线在该第一像素点是连通的；或者说该设定区域被车道骨骼线切分。在连通分支数目大于两个的情况下，将该第一像素点作为多条车道骨骼线的候选交叉位置。该设定区域可以根据经验值设定，其形状可以是圆形、矩形，本公开对此不作限定。例如，该设定区域的单位是像素x像素。Specifically, traverse the first pixel points on the bone lines of multiple lanes. With each first pixel as the center, the number of connected branches in the set area is obtained. Connected branches refer to different lane skeleton lines starting from the first pixel, and these lane skeleton lines are connected at the first pixel; in other words, the set area is divided by lane skeleton lines. In the case where the number of connected branches is greater than two, the first pixel is used as the candidate intersection position of multiple lane skeleton lines. The setting area can be set according to empirical values, and its shape can be a circle or a rectangle, which is not limited in the present disclosure. For example, the unit of the setting area is pixel x pixel.

具体实现中，遍历到某个第一像素点，以该第一像素点为中心，计算在一个设定区域(例如，10x10的像素区域)内的连通分支数目。如果连通分支数目大于2个，则把该第一像素点标记为一个候选交叉位置。如图6所示的识别多条车道骨骼线的候选交叉位置的示意图。遍历到第一像素点A，在以第一像素点A为中心的10x10的像素区域内，识别连通分支数目为2，则该第一像素点A不是骨骼线交叉位置。遍历到第一像素点B，在以第一像素点B为中心的10x10的像素区域内，识别连通分支数目为3，则该第一像素点B为候选交叉位置。In specific implementation, traverse to a certain first pixel point, and take the first pixel point as the center, and calculate the number of connected branches in a set area (for example, a 10×10 pixel area). If the number of connected branches is greater than 2, the first pixel is marked as a candidate intersection position. As shown in FIG. 6, a schematic diagram of identifying candidate intersection positions of multiple lane skeleton lines is shown. Traverse to the first pixel point A, in a 10×10 pixel area centered on the first pixel point A, and identify that the number of connected branches is 2, then the first pixel point A is not the position of the bone line intersection. Traverse to the first pixel point B, in a 10×10 pixel area centered on the first pixel point B, and identify that the number of connected branches is 3, then the first pixel point B is a candidate intersection position.

进一步地，在同一区域内，检测到的候选交叉位置会存在多个的情况。可以按照候选交叉位置之间的距离小于或等于第二距离阈值对多个候选交叉位置进行聚类，将聚类后的结果作为多条车道骨骼线之间的一个或多个交叉位置。具体地，对检测得到的多个候选交叉位置，可以使用聚类算法(比如meanshift算法)等对其进行聚类，每个簇保留中心的候选交叉位置作为车道骨骼线的交叉位置。Further, in the same area, there may be multiple detected candidate intersection positions. The multiple candidate intersection positions may be clustered according to the distance between the candidate intersection positions being less than or equal to the second distance threshold, and the result of the clustering may be used as one or more intersection positions between the multiple lane skeleton lines. Specifically, a clustering algorithm (such as a meanshift algorithm) can be used to cluster the multiple candidate intersection positions obtained by detection, and the candidate intersection position at the center of each cluster is reserved as the intersection position of the lane skeleton line.

如图7所示的候选交叉位置聚类的示意图，B1、B2、B3、B4都为候选交叉位置。通过meanshift聚类算法对邻近的候选交叉位置聚类，将聚类中心作为最终的交叉位置B′：As shown in the schematic diagram of the candidate intersection position clustering as shown in FIG. 7, B1, B2, B3, and B4 are all candidate intersection positions. Use the meanshift clustering algorithm to cluster adjacent candidate intersection positions, and use the cluster center as the final intersection position B′:

可以理解的是，通过聚类算法得到的最终的交叉位置可能位于车道骨骼线上，也可能不位于车道骨骼线上。It is understandable that the final intersection position obtained by the clustering algorithm may be located on the lane bone line, or may not be located on the lane bone line.

如图8所示，为采用上述方式，识别出的多条车道骨骼线之间的交叉位置的示意图。在两条或两条以上车道骨骼线的交叉点识别出一个车道骨骼线之间的交叉位置。该交叉位置可以直接是候选交叉位置，也可以是聚类交叉位置。As shown in FIG. 8, it is a schematic diagram of the intersection position between the multiple lane skeleton lines identified by the above method. At the intersection of two or more lane skeleton lines, the intersection position between one lane skeleton line is recognized. The intersection position can be directly a candidate intersection position or a cluster intersection position.

S203、根据端点位置和一个或多个交叉位置之间的相对位置关系，确定切分点。S203: Determine a segmentation point according to the relative position relationship between the end point position and one or more intersection positions.

在获得车道线的端点位置和多条车道骨骼线之间的一个或多个交叉位置后，可以根据端点位置和交叉位置之间的相对位置关系，确定切分点。其中，每条车道线都有两个端点。首先，可以分别获取每条车道线的端点位置与每个交叉位置之间的距离，选取距离小于或等于第三距离阈值的一个或多个端点位置作为候选端点位置。然后，构造每个候选端点位置和该交叉位置的连线。After obtaining the end position of the lane line and one or more intersection positions between the multiple lane skeleton lines, the cutting point can be determined according to the relative position relationship between the end position and the intersection position. Among them, each lane line has two end points. First, the distance between the end position of each lane line and each intersection position can be obtained separately, and one or more end positions whose distance is less than or equal to the third distance threshold can be selected as candidate end positions. Then, construct a line connecting each candidate endpoint position and the intersection position.

具体地，对每条车道线的端点位置，计算到其最近的交叉位置的距离。例如，如果某一端点位置到任一交叉位置的距离小于或等于设定阈值(如5m)，则能确定该交叉位置是与该候选端点位置距离最近的交叉位置。如图9所示，为确定与每个候选端点位置最近的交叉位置的示意图。其中，车道线的端点位置A1～A8到任一个车道骨骼线之间的交叉位置B1、B2的距离均大于上述设定阈值，则车道线的端点位置A1～A8不能作为候选端点位置；车道线的端点位置A9到车道骨骼线交叉位置B1之间的距离小于上述设定阈值，因此，构造候选端点位置A9与交叉位置B1的连线；车道线的端点位置A10到车道骨骼线交叉位置B2之间的距离小于上述设定阈值，因此，构造候选端点位置A10与交叉位置B2的连线。Specifically, for the end position of each lane line, the distance to the nearest intersection position is calculated. For example, if the distance from a certain endpoint position to any intersection position is less than or equal to a set threshold (such as 5m), it can be determined that the intersection position is the intersection position closest to the candidate endpoint position. As shown in FIG. 9, it is a schematic diagram of determining the intersection position closest to each candidate endpoint position. Among them, the distance between the end position A1～A8 of the lane line and the intersection position B1 and B2 between any lane skeleton line is greater than the above-mentioned set threshold, then the end position A1～A8 of the lane line cannot be regarded as the candidate end position; The distance between the end point position A9 of the lane line and the intersection position B1 of the lane skeleton line is less than the above-mentioned set threshold. Therefore, the connection line between the candidate end point position A9 and the intersection position B1 is constructed; The distance between the two is smaller than the above-mentioned set threshold, therefore, a line connecting the candidate end point position A10 and the intersection position B2 is constructed.

确定连线的中点位置作为切分点。由于可能存在多条连线，因此，该中点位置可能有一个或多个。Determine the position of the midpoint of the line as the cutting point. Since there may be multiple connections, there may be one or more midpoint positions.

S204、根据一个或多个候选端点位置对应的切分点，确定切分线。S204: Determine a segmentation line according to the segmentation point corresponding to one or more candidate endpoint positions.

可以根据切分点确定切分线。具体地，在存在一个中点位置的情况下，确定该切分线通过该中点位置且与上述连线垂直；在存在两个中点位置的情况下，确定连接两个中点位置的连线作为该切分线；在存在两个以上的中点位置的情况下，则可以根据该两个以上的中点位置拟合一条连线作为切分线。在一个示例中，可以采用最小二乘法进行拟合。The dividing line can be determined according to the dividing point. Specifically, in the case that there is a midpoint position, it is determined that the cutting line passes through the midpoint position and is perpendicular to the above-mentioned line; in the case that there are two midpoint positions, the connection connecting the two midpoint positions is determined The line is used as the cutting line; in the case where there are more than two midpoint positions, a line can be fitted as the cutting line according to the two or more midpoint positions. In one example, the least squares method can be used for fitting.

如图10所示，为一个交叉位置与一个候选端点位置存在连线时切分线的示意图。首先，将交叉位置C与候选端点位置D进行连线；然后，找到该连线的中点位置E；最后，在该中点位置E作出一条切分线将道路图像切分成两个分段，该切分线与上述连线DC垂直。As shown in FIG. 10, it is a schematic diagram of a cutting line when there is a connection between a cross position and a candidate end position. First, connect the intersection point C and the candidate end point position D; then, find the midpoint position E of the line; finally, make a segmentation line at the midpoint position E to divide the road image into two segments, The dividing line is perpendicular to the above-mentioned connecting line DC.

如图11所示，为一个交叉位置与多个候选端点位置存在连线时切分线的示意图。首先，将交叉位置C与每个候选端点位置D1、D2分别进行连线；然后，找到每条连线的中点位置E1、E2(图中用Δ标识的点)；最后，连线两个中点位置E1和E2，得到一条切分线，该切分线将道路图像切分成两个分段。As shown in FIG. 11, it is a schematic diagram of a cutting line when there is a connection between a crossing position and a plurality of candidate end positions. First, connect the intersection point C with each candidate endpoint position D1, D2 respectively; then, find the midpoint position E1, E2 (points marked by Δ in the figure) of each connection; finally, connect the two At midpoint positions E1 and E2, a segmentation line is obtained, and the segmentation line divides the road image into two segments.

进一步地，由于根据每个交叉位置可以确定一条切分线，若存在多个交叉位置，且该多个交叉位置的纵向距离较小，则切分出的分段较小，这样的分段对于构建道路语义结构是不合适的。具体地，首先确定与车道平行的一个向量，该向量为一个纵向向量，然后将多条车道骨骼线之间的交叉位置投影到该向量上，计算投影的交叉位置之间的距离，即为多个交叉位置的纵向距离。因此，根据获取的多个交叉位置之间的纵向距离，若至少两个交叉位置之间的纵向距离小于或等于第四距离阈值，将这两个交叉位置合并，得到合并后的交叉位置。如图12所示的合并交叉位置的示意图，车道骨骼线之间的交叉位置N1、N2的纵向距离Δy较小，因此取代根据中点位置M1、M2分别构造切分线，可以将通过该中点位置M1、M2的连线作为道路图像的切分线。Further, since a cutting line can be determined according to each intersection position, if there are multiple intersection positions, and the longitudinal distance of the multiple intersection positions is small, the divided segment is smaller. Such a segmentation It is inappropriate to construct the road semantic structure. Specifically, first determine a vector parallel to the lane, which is a longitudinal vector, and then project the intersection position between multiple lane bone lines onto the vector, and calculate the distance between the projected intersection positions, which is The longitudinal distance of each intersection. Therefore, according to the obtained longitudinal distances between the multiple intersection positions, if the longitudinal distance between at least two intersection positions is less than or equal to the fourth distance threshold, the two intersection positions are merged to obtain the merged intersection position. As shown in the schematic diagram of the merged intersection position as shown in Figure 12, the longitudinal distance Δy of the intersection position N1 and N2 between the lane bone lines is small. The line connecting the point positions M1 and M2 is used as the dividing line of the road image.

此外，若存在多个交叉位置，且该多个交叉位置两两之间的纵向距离较大，例如大于第四距离阈值，这样切分出的分段就不存在分段较小的问题，因此可以对于每个交叉位置，确定对应的连线的中点位置，并使切分线通过该中点位置且与该连线垂直。如图13所示，可以得到切分线L1以及切分线L2。In addition, if there are multiple crossing positions, and the longitudinal distance between the multiple crossing positions is relatively large, for example, greater than the fourth distance threshold, the divided segments will not have the problem of small segments. Therefore, For each crossing position, the midpoint position of the corresponding line can be determined, and the cutting line can pass through the midpoint position and be perpendicular to the line. As shown in FIG. 13, the cutting line L1 and the cutting line L2 can be obtained.

S205、利用切分线将道路图像切分为至少两个分段。S205: Use the dividing line to divide the road image into at least two segments.

确定切分线后，则可以利用切分线将道路图像切分为至少两个分段。如图13所示，切分线L1把道路图像切分为分段1和分段2，在分段1中包括3条车道、4条车道线，在分段2中包括2条车道、3条车道线，分段1和分段2的车道线数目在切分线L1处发生改变。切分线L2把道路图像切分为分段2和分段3，在分段2中包括2条车道、3条车道线，在分段3中包括3条车道、4条车道线，分段2和分段3的车道线数目在切分线L2处发生改变。After the segmentation line is determined, the segmentation line can be used to segment the road image into at least two segments. As shown in Figure 13, the segmentation line L1 divides the road image into segment 1 and segment 2. Segment 1 includes 3 lanes and 4 lane lines, and segment 2 includes 2 lanes, 3 For each lane line, the number of lane lines in section 1 and section 2 changes at the split line L1. The segmentation line L2 divides the road image into segment 2 and segment 3. In segment 2, there are 2 lanes and 3 lane lines, and in segment 3, there are 3 lanes and 4 lane lines. The number of lane lines in section 2 and section 3 changes at the split line L2.

进一步地，如果计算得到的切分线穿过中间某条车道线(非左右边界车道线)且该车道线对应于车道线数目的改变，则移动切分线，使其与车道线不相交。如图14所示，计算得到的切分线L1与车道线1相交，可以往车道线的正方向(从车道线的端点指向车道骨骼线之间的交叉位置的方向)移动切分线L1，直到与车道线1不相交，得到调整后的切分线L1’，作为道路图像的切分线。Further, if the calculated dividing line passes through a certain lane line in the middle (non-left and right boundary lane lines) and the lane line corresponds to a change in the number of lane lines, move the dividing line so that it does not intersect the lane line. As shown in Figure 14, the calculated segmentation line L1 intersects with the lane line 1, and the segmentation line L1 can be moved in the positive direction of the lane line (the direction from the end point of the lane line to the intersection position between the lane bone lines), Until it does not intersect with the lane line 1, the adjusted cutting line L1' is obtained as the cutting line of the road image.

S206、根据各切分后车道线与切分线的相对位置关系，确定属于各分段的切分后车道线。S206: Determine the divided lane line belonging to each segment according to the relative position relationship between each divided lane line and the divided line.

在将道路图像切分为第一分段和第二分段后，可以生成第一和第二分段对应的道路语义信息。第一分段对应的道路语义信息至少包括：第一分段中的道路语义元素，以及第一分段与第二分段之间的连接关系。其中，第一分段中的道路语义元素包括以下元素中的一项或多项：第一分段的车道线，车道线与车道中心线的相对位置关系，第一分段中的车道。After the road image is divided into the first segment and the second segment, road semantic information corresponding to the first and second segments can be generated. The road semantic information corresponding to the first segment includes at least: the road semantic elements in the first segment, and the connection relationship between the first segment and the second segment. Wherein, the road semantic elements in the first segment include one or more of the following elements: the lane line of the first segment, the relative position relationship between the lane line and the center line of the lane, and the lane in the first segment.

使用切分线对道路图像进行切分后，对于第一分段，可以根据各切分后车道线与切分线的相对位置关系，确定属于第一分段的切分后车道线。如图15所示的切分后车道线的示意图，切分线L把车道线L1切分为L1-1、L1-2两段，以及切分线L把车道线L3切分为L3-1、L3-2两段。L1-1、L2、L3-1位于切分线L一侧，它们属于第一分段；L1-2、L3-2位于切分线L的另一侧，它们属于第二分段。After the road image is segmented using the segmentation line, for the first segment, the segmented lane line belonging to the first segment can be determined according to the relative position relationship between each segmented lane line and the segmentation line. As shown in the schematic diagram of the divided lane line as shown in Figure 15, the dividing line L divides the lane line L1 into two segments L1-1 and L1-2, and the dividing line L divides the lane line L3 into L3-1 , L3-2 two paragraphs. L1-1, L2, L3-1 are located on one side of the cutting line L, and they belong to the first segment; L1-2, L3-2 are located on the other side of the cutting line L, and they belong to the second segment.

具体地，可以通过掩码运算，确定属于第一分段的切分后车道线。如图16所示的又一切分后车道线的示意图，切分线(section_line)1、切分线2把车道线切分为A0，A1，A2，A3，B0，B1，B2，B3，C0，C1，C2，C3多段。令切分线1、切分线2为带方向的线段，起点分别为S ₁、S ₂，方向分别为

首先，根据每条切分后车道线的中心点到切分线的起点的方向向量、以及切分线的方向向量，确定每条切分后车道线对应的方向标记。以切分后车道线A0为例，记其中心点为A _{0_c}。令

则

令

则

其中，

代表

在某一设定方向上(例如x轴或t轴)的分量，

代表

在另一设定方向上(例如y轴或s轴)的分量；

和

采相同方式拆解。进一步地，根据

和

各分量构成的2*2矩阵确定方向标记dirFlag1，根据

和

各分量构成的2*2矩阵确定方向标记dirFlag2。然后，根据每条切分后车道线对应的方向标记，获取多条切分后车道线的掩码向量。仍以切分后车道线A0为例，如果dirFlag1＞0，令掩码N1＝0，若dirFlag1＜0，令掩码N1＝1；如果dirFlag2＞0，令掩码N2＝0，若dirFlag2＜0，令掩码N2＝1。则N1、N2组成掩码向量<N1，N2>。对其它切分后车道线，计算掩码方式与A0相同。最后，对掩码向量相同的切分后车道线，标记它们属于同一分段，从而结合例如局部图像识别可以确定属于第一分段的切分后车道线。 Specifically, the segmented lane line belonging to the first segment can be determined through a mask operation. As shown in Figure 16, the schematic diagram of the lane line after dividing again, the section line (section_line) 1, the section line 2 divides the lane line into A0, A1, A2, A3, B0, B1, B2, B3, C0 , C1, C2, C3 multi-segment. Let dividing line 1 and dividing line 2 be line segments with directions, the starting points are S ₁ , S ₂ , and the directions are

First, according to the direction vector from the center point of each segmented lane line to the starting point of the segmentation line and the direction vector of the segmentation line, determine the direction mark corresponding to each segmented lane line. Take the lane line A0 after segmentation as an example, and record its center point as A _{0_c} . make

but

make

but

in,

represent

The component in a set direction (such as x-axis or t-axis),

represent

Component in another set direction (such as y-axis or s-axis);

with

Disassemble in the same way. Further, according to

with

The 2*2 matrix composed of each component determines the direction flag dirFlag1, according to

with

The 2*2 matrix formed by each component determines the direction flag dirFlag2. Then, according to the direction mark corresponding to each lane line after segmentation, the mask vectors of multiple lane lines after segmentation are obtained. Still taking the lane line A0 after segmentation as an example, if dirFlag1>0, set the mask N1=0, if dirFlag1<0, set the mask N1=1; if dirFlag2>0, set the mask N2=0, if dirFlag2< 0, let the mask N2=1. Then N1, N2 form a mask vector <N1, N2>. For other lane lines after segmentation, the calculation method of the mask is the same as that of A0. Finally, the segmented lane lines with the same mask vector are marked as belonging to the same segment, so that combined with, for example, partial image recognition, the segmented lane line belonging to the first segment can be determined.

下述步骤S207～S208为确定属于每一分段的切分后车道线与车道中心线的相对位置关系，以第一分段为例进行说明。The following steps S207 to S208 are to determine the relative positional relationship between the divided lane line and the center line of the lane belonging to each segment, and the first segment is taken as an example for description.

S207、对第一分段中的车道中心线(centerline)进行赋值。S207. Assign a value to the centerline of the lane in the first segment.

车道中心线可以是左右方向车道中间的绿化带，或位于两个不同方向的车道中间的分隔线，如黄实线等，其本身可能同时构成一条车道线。车道中心线可以是1条或2条。当第一分段中只包含1条车道中心线时，该车道中心线赋值为0。The lane centerline can be the green belt in the middle of the lanes in the left and right directions, or the dividing line between two lanes in different directions, such as a yellow solid line, which itself may constitute a lane line at the same time. There can be one or two lane centerlines. When the first segment contains only one lane centerline, the lane centerline is assigned a value of 0.

当第一分段中包含2条车道中心线时，左边的车道中心线赋值为1，右边的车道中心线赋值为-1。具体地，计算所有车道中心线起点均值centerStartPoint和方向均值centerDir。对某条车道中心线，记其起点为startPoint，从centerStartPoint到startPoint的方向向量为coVec。计算该条车道中心线的方向标记dirFlag为centerDir与coVec的点积，即cross(centerDir,coVec)。如果车道中心线centerline1的dirFlag大于0,则标记centerline1为左边，localID置1；如果车道中心线centerline2的dirFlag小于0，则标记centerline2为右边，localID置-1。When the first segment contains two lane centerlines, the left lane centerline is assigned a value of 1, and the right lane centerline is assigned a value of -1. Specifically, calculate the mean value centerStartPoint and the mean direction centerDir of the centerline start points of all lanes. For the centerline of a certain lane, record its starting point as startPoint, and the direction vector from centerStartPoint to startPoint as coVec. Calculate the direction mark dirFlag of the centerline of the lane as the dot product of centerDir and coVec, that is, cross(centerDir, coVec). If the dirFlag of the lane centerline centerline1 is greater than 0, the mark centerline1 is left and the localID is set to 1. If the dirFlag of the lane centerline centerline2 is less than 0, the mark centerline2 is the right, and the localID is set to -1.

S208、确定属于第一分段的切分后车道线与车道中心线的相对位置关系。其中，该相对位置关系包括：车道线位于车道中心线的物理左侧，车道线位于车道中心线的物理右侧。S208: Determine the relative position relationship between the divided lane line and the lane center line belonging to the first segment. Wherein, the relative position relationship includes: the lane line is located on the physical left side of the lane center line, and the lane line is located on the physical right side of the lane center line.

首先，获取车道中心线的方向均值。然后，获取车道中心线的起点与每条切分后车道线的起点之间的方向向量。然后，计算车道中心线的方向均值、以及方向向量之间的点积。根据点积的值，确定切分后车道线位于车道中心线的物理左侧或物理右侧。其中，点积大于0，则确定切分后车道线位于车道中心线的物理左侧；点积小于或等于0，则确定切分后车道线位于车道中心线的物理右侧。First, get the mean value of the direction of the centerline of the lane. Then, the direction vector between the starting point of the center line of the lane and the starting point of each lane line after segmentation is obtained. Then, the mean value of the direction of the center line of the lane and the dot product between the direction vectors are calculated. According to the value of the dot product, it is determined that the lane line after segmentation is located on the physical left or the physical right of the center line of the lane. Among them, if the dot product is greater than 0, it is determined that the lane line after segmentation is located on the physical left side of the lane centerline; if the dot product is less than or equal to 0, it is determined that the lane line after segmentation is located on the physical right side of the lane centerline.

具体实现中，对非车道中心线的每条切分后车道线，记其起点为startPoint，centerStartPoint到startPoint的方向向量为coVec。计算方向标记dirFlag为点积cross(centerDir,coVec)。如果dirFlag大于0，计算startPoint到centerStartPoint的距离dis2start，把该切分后车道线及距离记录至左列表(leftList)。如果dirFlag小于等于0，计算startPoint到centerStartPoint的距离dis2start，把该切分后车道线及距离记录至右列表(rightList)。In the specific implementation, for each segmented lane line that is not the center line of the lane, remember its starting point as startPoint, and the direction vector from centerStartPoint to startPoint as coVec. Calculate the direction flag dirFlag as the dot product cross(centerDir, coVec). If dirFlag is greater than 0, calculate the distance dis2start from startPoint to centerStartPoint, and record the segmented lane line and distance to the left list (leftList). If dirFlag is less than or equal to 0, calculate the distance dis2start from startPoint to centerStartPoint, and record the segmented lane line and distance to the right list (rightList).

对左列表中的车道线按dis2start进行从小至大排序。如果只有1条车道中心线,则分别按序赋值localID为1,2,3,…；如果是包含2条车道中心线的情况，则按序赋值localID为2,3,4,...。localID最大的车道线标记为左边界(left_roadside)。Sort the lane lines in the left list according to dis2start from small to large. If there is only one lane centerline, the localIDs are assigned sequentially as 1, 2, 3,...; if it contains two lane centerlines, the localIDs are assigned sequentially as 2,3,4,.... The lane line with the largest localID is marked as the left boundary (left_roadside).

对右列表中的车道线按dis2start进行从小至大排序。如果只有1条车道中心线,则分别按序赋值localID为-1,-2,-3,...；如果是包含2条车道中心线的情况，则按序赋值localID为-2,-3,-4,...。localID最小的车道线标记为右边界(right_roadside)。Sort the lane lines in the right list according to dis2start from small to large. If there is only one lane centerline, then assign the localIDs in sequence to -1, -2,-3,...; if it contains two lane centerlines, then assign the localIDs in sequence to -2,-3 ,-4,.... The lane line with the smallest localID is marked as the right boundary (right_roadside).

对上述左列表中的车道线，分别对相邻两车道线绑定生成一条车道。然后分别按顺序给车道赋值localID为1,2,3,…。For the lane lines in the above left list, bind two adjacent lane lines to generate one lane respectively. Then assign localIDs of 1, 2, 3,... to the lanes in order.

对上述右列表中的车道线，分别对相邻两车道线绑定生成一条车道。然后分别按顺序给车道赋值localID为-1,-2,-3,...。For the lane lines in the above right list, bind two adjacent lane lines to generate one lane respectively. Then assign localIDs of -1, -2, -3,... to the lanes in order.

S209、生成第一分段与相邻分段之间的前继关系或后继关系。S209: Generate a predecessor relationship or a successor relationship between the first segment and the adjacent segment.

具体地，计算第一分段的始端点与其他分段的末端点之间的距离，对应最近距离的分段为第一分段的前继；或者，计算第一分段的末端点与其他分段的始端点之间的距离，对应最近距离的分段为第一分段的后继。Specifically, calculate the distance between the start point of the first segment and the end points of other segments, and the segment corresponding to the closest distance is the predecessor of the first segment; or, calculate the end point of the first segment and other The distance between the start and end points of the segments, and the segment corresponding to the closest distance is the successor of the first segment.

具体实现中，在第一分段中，计算左右边界(left_roadside、right_roadside)两端点均值分别为始端点startpoint、末端点endpoint。In specific implementation, in the first segment, the average values of the two ends of the left and right boundaries (left_roadside, right_roadside) are calculated as the start point and the end point, respectively.

将第一分段分别与其它分段比较，计算该第一分段的startpoint到其它分段的endpoint的距离，距离最小者对应的分段为该分段的前继；计算该第一分段的endpoint到其它分段的startpoint的距离，距离最小者为该分段的后继。Compare the first segment with other segments respectively, calculate the distance from the startpoint of the first segment to the endpoints of other segments, the segment with the smallest distance is the predecessor of the segment; calculate the first segment The distance from the endpoint to the startpoint of other segments, the smallest distance is the successor of the segment.

S210、根据生成的前继关系或后继关系，获取第一分段中的多条车道与相邻分段中的多条车道之间的车道连接关系。S210: Acquire a lane connection relationship between multiple lanes in the first segment and multiple lanes in adjacent segments according to the generated predecessor relationship or successor relationship.

根据上述生成的第一分段和第二分段的前后继关系，获取相邻分段中多条车道的车道连接关系。具体地，第一分段和第二分段为相邻的两个分段。对第一分段中的每条车道，计算其到第二分段的各个车道之间的距离(车道之间的距离，是指车道之间左边界的横向距离)，将该第一分段中的车道与第二分段中的距离最小的车道配对，得到相邻分段的车道连接关系或配对(matchPair1)。同样，对第一分段中的其它车道，按照上述方式得到第一分段与第二分段的车道连接关系。需要说明的是，这里并不要求1比1的配对结果。考虑到车道线的变化，可能第一分段中的多条车道与第二分段中的一条车道配对，也可能第一分段中的一条车道与第二分段中的多条车道配对。According to the successive relationship between the first segment and the second segment generated above, the lane connection relationship of the multiple lanes in the adjacent segment is obtained. Specifically, the first segment and the second segment are two adjacent segments. For each lane in the first segment, calculate the distance between each lane of the second segment (the distance between lanes, which refers to the lateral distance of the left boundary between the lanes), and the first segment The lane in the middle section is paired with the lane with the smallest distance in the second section, and the lane connection relationship or pairing (matchPair1) of the adjacent section is obtained. Similarly, for other lanes in the first segment, the lane connection relationship between the first segment and the second segment is obtained in the above manner. It should be noted that a 1:1 pairing result is not required here. Considering the change of lane lines, multiple lanes in the first segment may be paired with one lane in the second segment, or one lane in the first segment may be paired with multiple lanes in the second segment.

计算上述车道连接关系或配对之间的并集，构成分段间的车道连接关系。Calculate the above-mentioned lane connection relationship or the union between the pairings to form the lane connection relationship between the segments.

对于多个分段中除第一分段之外的其他分段的道路语义信息，可以按照如S207-S210中的第一分段的处理方法进行提取，在此不再赘述。For the road semantic information of the other segments except the first segment in the multiple segments, the extraction can be performed according to the processing method of the first segment in S207-S210, which will not be repeated here.

通过上述道路图像的语义信息提取，无需人工标注，自动获得了各个分段的道路语义信息，从而获得了整个道路图像的语义信息，提升了提取道路语义信息的效率。Through the above-mentioned semantic information extraction of the road image, the road semantic information of each segment is automatically obtained without manual labeling, so that the semantic information of the entire road image is obtained, and the efficiency of extracting the road semantic information is improved.

根据本公开提供的一种获取道路语义信息的方法，可以根据道路图像中的车道线，自动获取道路图像中的道路语义信息，提高了获取道路语义信息的效率。According to the method for acquiring road semantic information provided by the present disclosure, the road semantic information in the road image can be automatically acquired according to the lane lines in the road image, which improves the efficiency of acquiring road semantic information.

基于上述获取道路语义信息的方法的同一构思，本公开还提供一种获取道路语义信息的装置。如图17所示，为本公开提供的获取道路语义信息的装置的一结构示意图，该装置1000包括：Based on the same concept of the above method for acquiring road semantic information, the present disclosure also provides a device for acquiring road semantic information. As shown in FIG. 17, it is a schematic structural diagram of the apparatus for obtaining road semantic information provided by the present disclosure. The apparatus 1000 includes:

第一获取单元11，用于获取道路图像；The first acquiring unit 11 is used to acquire road images;

第一切分单元12，用于根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段；The first segmentation unit 12 is configured to segment the road image into at least two segments according to the change in the number of lane lines in the road image;

第一生成单元13，用于生成每一分段对应的道路语义信息，至少包括：所述分段中的道路语义元素，以及所述分段与相邻分段之间的连接关系。The first generating unit 13 is configured to generate road semantic information corresponding to each segment, which includes at least: road semantic elements in the segment and the connection relationship between the segment and adjacent segments.

在一种可能的实现中(图中未示出)，所述第一切分单元12包括：In a possible implementation (not shown in the figure), the first splitting unit 12 includes:

第二获取单元，用于根据每次所述道路图像中车道线数目发生改变的位置，得到所述道路图像的切分线；The second acquiring unit is configured to obtain the dividing line of the road image according to the position where the number of lane lines in the road image changes each time;

第二切分单元，用于利用所述切分线将所述道路图像切分为两个分段。The second segmentation unit is configured to use the segmentation line to segment the road image into two segments.

在又一种可能的实现中(图中未示出)，所述第二获取单元包括：In another possible implementation (not shown in the figure), the second acquiring unit includes:

第一识别单元，用于识别所述道路图像中的所述车道线；A first recognition unit, configured to recognize the lane line in the road image;

第三获取单元，用于获取所述车道线的端点位置；The third acquiring unit is configured to acquire the position of the end point of the lane line;

第二生成单元，用于生成多条车道骨骼线；The second generating unit is used to generate multiple lane skeleton lines;

第二识别单元，用于识别所述多条车道骨骼线之间的一个或多个交叉位置；The second recognition unit is used to recognize one or more intersection positions between the multiple lane skeleton lines;

第一确定单元，用于根据所述端点位置和所述一个或多个交叉位置之间的相对位置关系，确定所述切分线。The first determining unit is configured to determine the cutting line according to the relative position relationship between the end point position and the one or more intersection positions.

在又一种可能的实现中(图中未示出)，所述第二生成单元包括：In yet another possible implementation (not shown in the figure), the second generating unit includes:

第四获取单元，用于获取每个像素点到最近的车道线的距离；The fourth acquiring unit is used to acquire the distance from each pixel to the nearest lane line;

二值化处理单元，用于根据每个像素点到最近的车道线的距离与第一距离阈值的关系，对所述道路图像进行二值化处理，其中，到最近的车道线的距离小于或等于所述第一距离阈值的像素点置为第一值，到最近的车道线的距离大于所述第一距离阈值的像素点置为第二值；The binarization processing unit is configured to perform binarization processing on the road image according to the relationship between the distance from each pixel to the nearest lane line and the first distance threshold, wherein the distance to the nearest lane line is less than or Pixels equal to the first distance threshold are set to a first value, and pixels whose distance to the nearest lane line is greater than the first distance threshold are set to a second value;

图像细化处理单元，用于对像素点置为第二值对应的区域进行图像细化处理，得到所述多条车道骨骼线。The image thinning processing unit is configured to perform image thinning processing on the area corresponding to the pixel points set to the second value to obtain the multiple lane skeleton lines.

在又一种可能的实现中(图中未示出)，所述第二识别单元用于遍历所述多条车道骨骼线上的第一像素点，以所述道路图像中的每个第一像素点为中心，获取设定区域内的连通分支数目；以及在所述连通分支数目大于两个的情况下，将所述第一像素点作为所述多条车道骨骼线的候选交叉位置。其中，候选交叉位置可直接作为所述多条车道骨骼线之间的一个或多个交叉位置。In yet another possible implementation (not shown in the figure), the second recognition unit is used to traverse the first pixel points on the bone lines of the multiple lanes, and use each first pixel point in the road image. The pixel point is the center, and the number of connected branches in the set area is acquired; and when the number of connected branches is greater than two, the first pixel point is used as the candidate intersection position of the multiple lane skeleton lines. Wherein, the candidate intersection position can be directly used as one or more intersection positions between the multiple lane skeleton lines.

在又一种可能的实现中，所述装置还包括：In yet another possible implementation, the device further includes:

聚类单元14，用于当包括多个候选交叉位置时，按照候选交叉位置之间的距离小于或等于第二距离阈值对所述多个候选交叉位置进行聚类，将聚类后的结果作为所述多条车道骨骼线之间的一个或多个交叉位置。The clustering unit 14 is configured to, when a plurality of candidate intersection positions are included, cluster the candidate intersection positions according to the distance between the candidate intersection positions being less than or equal to the second distance threshold, and use the clustering result as One or more intersection positions between the multiple lane bone lines.

在又一种可能的实现中(图中未示出)，所述第一确定单元用于获取与每个交叉位置之间的距离小于或等于第三距离阈值的一个或多个端点位置作为候选端点位置；对于每个候选端点位置，构造所述候选端点位置和所述交叉位置的连线；确定所述连线的中点位置作为切分点；根据所述一个或多个候选端点位置对应的切分点，确定所述切分线。In yet another possible implementation (not shown in the figure), the first determining unit is configured to obtain one or more endpoint positions whose distance from each intersection position is less than or equal to the third distance threshold as candidates End point position; for each candidate end point position, construct a line connecting the candidate end point position and the intersection position; determine the midpoint position of the line as a segmentation point; correspond to the one or more candidate end point positions To determine the cutting point, the cutting line.

在又一种可能的实现中(图中未示出)，所述第一确定单元用于：In another possible implementation (not shown in the figure), the first determining unit is configured to:

在存在一个中点位置的情况下，即所述切分点为所述中点位置，确定所述切分线穿过所述中点位置，并垂直于所述连线；或In the case that there is a midpoint position, that is, the segmentation point is the midpoint position, it is determined that the segmentation line passes through the midpoint position and is perpendicular to the line; or

在存在两个中点位置的情况下，即所述切分点包括所述两个中点位置，确定所述切分线穿过所述两个中点位置；或In the case that there are two midpoint positions, that is, the segmentation point includes the two midpoint positions, it is determined that the segmentation line passes through the two midpoint positions; or

在存在两个以上的中点位置的情况下，根据所述两个以上的中点位置，拟合得到一条切分线。In the case where there are more than two midpoint positions, a dicing line is obtained by fitting according to the two or more midpoint positions.

第五获取单元15，用于获取多个交叉位置之间的纵向距离；The fifth acquiring unit 15 is configured to acquire the longitudinal distance between multiple crossing positions;

合并单元16，用于对于纵向距离小于或等于第四距离阈值的至少两个交叉位置进行合并，根据其各自的中点位置确定所述切分线。The merging unit 16 is configured to merge at least two crossing positions whose longitudinal distance is less than or equal to the fourth distance threshold, and determine the cutting line according to their respective midpoint positions.

在又一种可能的实现中，所述第一生成单元13，包括：In another possible implementation, the first generating unit 13 includes:

第二确定单元131，用于根据各切分后车道线与所述切分线的相对位置关系，确定属于所述分段的切分后车道线；The second determining unit 131 is configured to determine the segmented lane line belonging to the segment according to the relative position relationship between each segmented lane line and the segmentation line;

赋值单元132，用于对所述分段中的车道中心线进行赋值；The assignment unit 132 is configured to assign a value to the center line of the lane in the segment;

第三确定单元133，用于确定属于所述分段的切分后车道线与所述车道中心线的相对位置关系。The third determining unit 133 is configured to determine the relative positional relationship between the divided lane line and the lane center line belonging to the segment.

在又一种可能的实现中(图中未示出)，所述第二确定单元131包括：In another possible implementation (not shown in the figure), the second determining unit 131 includes:

第四确定单元，用于根据每条切分后车道线的中心点到所述切分线起点的方向向量、以及所述切分线的方向向量，确定所述切分后车道线对应的方向标记；The fourth determining unit is configured to determine the direction corresponding to the lane line after segmentation according to the direction vector from the center point of each lane line after segmentation to the starting point of the segmentation line and the direction vector of the segmentation line mark;

第六获取单元，用于根据所述切分后车道线对应的方向标记，获取所述切分后车道线的掩码向量；The sixth acquiring unit is configured to acquire the mask vector of the lane line after segmentation according to the direction mark corresponding to the lane line after segmentation;

归属单元，用于将所述掩码向量相同的多条切分后车道线归属为同一分段。The attribution unit is used to attribute multiple segmented lane lines with the same mask vector to the same segment.

在又一种可能的实现中(图中未示出)，所述第三确定单元133包括：In another possible implementation (not shown in the figure), the third determining unit 133 includes:

第七获取单元，用于获取所述车道中心线的方向均值；The seventh acquiring unit is configured to acquire the direction average value of the center line of the lane;

第八获取单元，用于获取所述车道中心线的起点与每条切分后车道线的起点之间的方向向量；An eighth acquiring unit, configured to acquire a direction vector between the starting point of the center line of the lane and the starting point of each lane line after segmentation;

第九获取单元，用于获取所述车道中心线的方向均值、以及所述方向向量之间的点积；A ninth obtaining unit, configured to obtain the mean value of the direction of the center line of the lane and the dot product between the direction vectors;

第五确定单元，用于根据所述点积的值，确定所述切分后车道线位于所述车道中心线的物理左侧或物理右侧。The fifth determining unit is configured to determine, according to the value of the dot product, that the lane line after segmentation is located on the physical left side or the physical right side of the center line of the lane.

在又一种可能的实现中(图中未示出)，所述第五确定单元用于：In another possible implementation (not shown in the figure), the fifth determining unit is configured to:

在所述点积大于0的情况下，确定所述切分后车道线位于所述车道中心线的物理左侧；In a case where the dot product is greater than 0, it is determined that the lane line after segmentation is located on the physical left side of the lane centerline;

或是，在所述点积小于或等于0的情况下，确定所述切分后车道线位于所述车道中心线的物理右侧。Or, if the dot product is less than or equal to 0, it is determined that the lane line after the segmentation is located on the physical right side of the lane center line.

在又一种可能的实现中，所述第一生成单元13还包括：In another possible implementation, the first generating unit 13 further includes:

第三生成单元134，用于生成所述分段与相邻分段之间的前继关系或后继关系；The third generating unit 134 is configured to generate a predecessor relationship or a successor relationship between the segment and the adjacent segment;

第十获取单元135，用于根据生成的所述前继关系或所述后继关系，获取所述分段中的多条车道与所述相邻分段中的多条车道之间的车道连接关系。The tenth acquiring unit 135 is configured to acquire the lane connection relationship between the multiple lanes in the segment and the multiple lanes in the adjacent segment according to the generated predecessor relationship or the successor relationship .

在又一种可能的实现中(图中未示出)，所述第三生成单元134用于：In another possible implementation (not shown in the figure), the third generating unit 134 is configured to:

计算所述分段的始端点与其他分段的末端点之间的距离，确定对应最近距离的分段为所述第一分段的前继；Calculate the distance between the start point of the segment and the end points of other segments, and determine that the segment corresponding to the closest distance is the predecessor of the first segment;

或，计算所述分段的末端点与其他分段的始端点之间的距离，确定对应最近距离的分段为所述第一分段的后继。Or, calculating the distance between the end point of the segment and the beginning end point of other segments, and determining that the segment corresponding to the closest distance is the successor of the first segment.

有关上述各单元的具体实现可参考图1或图3所示实施例中的相关描述。For the specific implementation of the foregoing units, reference may be made to the related description in the embodiment shown in FIG. 1 or FIG. 3.

根据本公开提供的一种获取道路语义信息的装置，可以根据道路图像中的车道线，自动获取道路图像中的道路语义信息，提高了获取道路语义信息的效率。According to the device for acquiring road semantic information provided by the present disclosure, the road semantic information in the road image can be automatically acquired according to the lane lines in the road image, which improves the efficiency of acquiring road semantic information.

本公开还提供一种获取道路语义信息的设备，该设备用于执行上述获取道路语义信息的方法。上述方法中的部分或全部可以通过硬件来实现，也可以通过软件或固件来实现。The present disclosure also provides a device for acquiring road semantic information, and the device is used to execute the above-mentioned method for acquiring road semantic information. Part or all of the above methods can be implemented by hardware, and can also be implemented by software or firmware.

可选的，设备在具体实现时可以是芯片或者集成电路。Optionally, the device may be a chip or an integrated circuit during specific implementation.

可选的，当上述实施例的获取道路语义信息的方法中的部分或全部通过软件或固件来实现时，可以通过图18提供的一种获取道路语义信息的设备2000来实现。如图18所示，该设备2000可包括：Optionally, when part or all of the method for acquiring road semantic information in the foregoing embodiment is implemented by software or firmware, it may be implemented by a device 2000 for acquiring road semantic information provided in FIG. 18. As shown in FIG. 18, the device 2000 may include:

存储器21和处理器22(设备中的处理器22可以是一个或多个，图18中以一个处理器为例)。该设备2000还可以包括输入装置23、输出装置24。在本实施例中，输入装置23、输出装置24、存储器21和处理器22可通过总线或其它方式连接，其中，图18中以通过总线连接为例。The memory 21 and the processor 22 (the number of processors 22 in the device may be one or more, and one processor is taken as an example in FIG. 18). The device 2000 may also include an input device 23 and an output device 24. In this embodiment, the input device 23, the output device 24, the memory 21, and the processor 22 may be connected by a bus or other methods, wherein the connection by a bus is taken as an example in FIG. 18.

其中，处理器22用于执行图1、图3中所执行的方法步骤。The processor 22 is configured to execute the method steps executed in FIG. 1 and FIG. 3.

可选的，上述获取道路语义信息的方法的程序可以存储在存储器21中。该存储器21可以是物理上独立的单元，也可以与处理器22集成在一起。该存储器21也可以用于存储数据。Optionally, the program of the foregoing method for obtaining road semantic information may be stored in the memory 21. The memory 21 may be a physically independent unit, or may be integrated with the processor 22. The memory 21 can also be used to store data.

可选的，当上述实施例的获取道路语义信息的方法中的部分或全部通过软件实现时，该设备也可以只包括处理器。用于存储程序的存储器位于设备之外，处理器通过电路或电线与存储器连接，用于读取并执行存储器中存储的程序。Optionally, when part or all of the method for acquiring road semantic information in the foregoing embodiment is implemented by software, the device may also only include a processor. The memory used to store the program is located outside the device, and the processor is connected to the memory through a circuit or wire for reading and executing the program stored in the memory.

处理器可以是中央处理器(central processing unit，CPU)，网络处理器(network processor，NP)，或WLAN设备。The processor may be a central processing unit (CPU), a network processor (NP), or a WLAN device.

处理器还可以进一步包括硬件芯片。上述硬件芯片可以是专用集成电路(application-specific integrated circuit，ASIC)，可编程逻辑器件(programmable logic device，PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device，CPLD)，现场可编程逻辑门阵列(field-programmable gate array，FPGA)，通用阵列逻辑(generic array logic，GAL)或其任意组合。The processor may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

存储器可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储器也可以包括非易失性存储器(non-volatile memory)，例如快闪存储器(flash memory)，硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)；存储器还可以包括上述种类的存储器的组合。The memory may include volatile memory (volatile memory), such as random-access memory (RAM); the memory may also include non-volatile memory (non-volatile memory), such as flash memory (flash memory) , A hard disk drive (HDD) or a solid-state drive (solid-state drive, SSD); the memory may also include a combination of the foregoing types of memory.

根据本公开实施例提供的一种获取道路语义信息的设备，可以根据道路图像中的车道线，自动获取道路图像中的道路语义信息，提高了获取道路语义信息的效率。According to the device for acquiring road semantic information provided by the embodiments of the present disclosure, the road semantic information in the road image can be automatically acquired according to the lane lines in the road image, which improves the efficiency of acquiring road semantic information.

本领域技术人员应明白，本公开一个或多个实施例可提供为方法、***或计算机程序产品。因此，本公开一个或多个实施例可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且，本公开一个或多个实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that one or more embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of the present disclosure may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may adopt computer programs implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The form of the product.

本公开实施例还提供一种计算机可读存储介质，该存储介质上可以存储有计算机程序，所述程序被处理器执行时实现本公开任一实施例描述的获取道路语义信息的方法的步骤。其中，所述的“和/或”表示至少具有两者中的其中一个，例如，“A和/或B”包括三种方案：A、B、以及“A和B”。The embodiments of the present disclosure further provide a computer-readable storage medium, and the storage medium may store a computer program, which when executed by a processor implements the steps of the method for obtaining road semantic information described in any embodiment of the present disclosure. Wherein, the "and/or" means having at least one of the two, for example, "A and/or B" includes three schemes: A, B, and "A and B".

本公开中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于获取道路语义信息的装置实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。The various embodiments in the present disclosure are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the embodiment of the apparatus for obtaining road semantic information, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiment.

上述对本公开特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的行为或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。The specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

本公开中描述的主题及功能操作的实施例可以在以下中实现：数字电子电路、有形体现的计算机软件或固件、包括本公开中公开的结构及其结构性等同物的计算机硬件、或者它们中的一个或多个的组合。本公开中描述的主题的实施例可以实现为一个或多个计算机程序，即编码在有形非暂时性程序载体上以被数据处理装置执行或指示数据处理装置的操作的计算机程序指令中的一个或多个模块。可替代地或附加地，程序指令可以被编码在人工生成的传播信号上，例如机器生成的电、光或电磁信号，该信号被生成以将信息编码并传输到合适的接收机装置以由数据处理装置执行。计算机存储介质可以是机器可读存储设备、机器可读存储基板、随机或串行存取存储器设备、或它们中的一个或多个的组合。The embodiments of the subject and functional operations described in the present disclosure can be implemented in the following: digital electronic circuits, tangible computer software or firmware, computer hardware including the structures disclosed in the present disclosure and structural equivalents thereof, or among them A combination of one or more. The embodiments of the subject matter described in the present disclosure may be implemented as one or more computer programs, that is, one or one of the computer program instructions encoded on a tangible non-transitory program carrier to be executed by a data processing device or instruct the operation of the data processing device Multiple modules. Alternatively or in addition, the program instructions may be encoded on artificially generated propagated signals, such as machine-generated electrical, optical or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiver device for data transmission. The processing device executes. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

本公开中描述的处理及逻辑流程可以由执行一个或多个计算机程序的一个或多个可编程计算机执行，以通过根据输入数据进行操作并生成输出来执行相应的功能。所述处理及逻辑流程还可以由专用逻辑电路—例如现场可编程门阵列或专用集成电路来执行，并且装置也可以实现为专用逻辑电路。The processing and logic flow described in the present disclosure can be executed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating according to input data and generating output. The processing and logic flow can also be executed by a dedicated logic circuit, such as a field programmable gate array or an application specific integrated circuit, and the device can also be implemented as a dedicated logic circuit.

适合用于执行计算机程序的计算机包括，例如通用和/或专用微处理器，或任何其他类型的中央处理单元。通常，中央处理单元将从只读存储器和/或随机存取存储器接收指令和数据。计算机的基本组件包括用于实施或执行指令的中央处理单元以及用于存储指令和数据的一个或多个存储器设备。通常，计算机还将包括用于存储数据的一个或多个大容量存储设备，例如磁盘、磁光盘或光盘等，或者计算机将可操作地与此大容量存储设备耦接以从其接收数据或向其传送数据，抑或两种情况兼而有之。然而，计算机不是必须具有这样的设备。此外，计算机可以嵌入在另一设备中，例如移动电话、个人数字助理、移动音频或视频播放器、游戏操纵台、全球定位***接收机、或例如通用串行总线闪存驱动器的便携式存储设备，仅举几例。Computers suitable for executing computer programs include, for example, general-purpose and/or special-purpose microprocessors, or any other type of central processing unit. Generally, the central processing unit will receive instructions and data from a read-only memory and/or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Generally, the computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to this mass storage device to receive data from or send data to it. It transmits data, or both. However, the computer does not have to have such equipment. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant, a mobile audio or video player, a game console, a GPS receiver, or a portable storage device such as a universal serial bus flash drive. To give a few examples.

适合于存储计算机程序指令和数据的计算机可读介质包括所有形式的非易失性存储器、媒介和存储器设备，例如包括半导体存储器设备(例如EPROM、EEPROM和闪存设备)、磁盘(例如内部硬盘或可移动盘)、磁光盘以及CD ROM和DVD-ROM盘。处理器和存储器可由专用逻辑电路补充或并入专用逻辑电路中。Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or Removable disks), magneto-optical disks, CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by or incorporated into a dedicated logic circuit.

本公开还提供一种计算机程序产品，用于存储计算机可读指令，所述计算机可读指令被执行时使得计算机本公开任一实施例描述的获取道路语义信息的方法。The present disclosure also provides a computer program product for storing computer-readable instructions, which when executed, cause a computer to obtain the method for obtaining road semantic information described in any of the embodiments of the present disclosure.

虽然本公开包含许多具体实施细节，但是这些不应被解释为限制任何发明的范围或所要求保护的范围，而是主要用于描述特定发明的具体实施例的特征。本公开内在多个实施例中描述的某些特征也可以在单个实施例中被组合实施。另一方面，在单个实施例中描述的各种特征也可以在多个实施例中分开实施或以任何合适的子组合来实施。此外，虽然特征可以如上所述在某些组合中起作用并且甚至最初如此要求保护，但是来自所要求保护的组合中的一个或多个特征在一些情况下可以从该组合中去除，并且所要求保护的组合可以指向子组合或子组合的变型。Although the present disclosure contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claimed protection, but are mainly used to describe the features of specific embodiments of the specific invention. Certain features described in multiple embodiments within the present disclosure can also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. In addition, although features may function in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may in some cases be removed from the combination, and the claimed The combination of protection can be directed to a sub-combination or a variant of the sub-combination.

类似地，虽然在附图中以特定顺序描绘了操作，但是这不应被理解为要求这些操作以所示的特定顺序执行或顺次执行、或者要求所有例示的操作被执行，以实现期望的结果。在某些情况下，多任务和并行处理可能是有利的。此外，上述实施例中的各种***模块和组件的分离不应被理解为在所有实施例中均需要这样的分离，并且应当理解，所描述的程序组件和***通常可以一起集成在单个软件产品中，或者封装成多个软件产品。Similarly, although operations are depicted in a specific order in the drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can usually be integrated together in a single software product. In, or packaged into multiple software products.

由此，主题的特定实施例已被描述。其他实施例在所附权利要求书的范围以内。在某些情况下，权利要求书中记载的动作可以以不同的顺序执行并且仍实现期望的结果。此外，附图中描绘的处理并非必需所示的特定顺序或顺次顺序，以实现期望的结果。在某些实现中，多任务和并行处理可能是有利的。Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desired results. In addition, the processes depicted in the drawings are not necessarily in the specific order or sequential order shown in order to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

以上所述仅为本公开一个或多个实施例的较佳实施例而已，并不用以限制本公开一个或多个实施例，凡在本公开一个或多个实施例的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开一个或多个实施例保护的范围之内。The foregoing descriptions are only preferred embodiments of one or more embodiments of the present disclosure, and are not intended to limit one or more embodiments of the present disclosure. All within the spirit and principle of one or more embodiments of the present disclosure, Any modification, equivalent replacement, improvement, etc. made should be included in the protection scope of one or more embodiments of the present disclosure.

Claims

一种获取道路语义信息的方法，包括：A method for obtaining road semantic information, including:

获取道路图像；Obtain road images;

根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段；Dividing the road image into at least two segments according to the change in the number of lane lines in the road image;

生成每一分段对应的道路语义信息，至少包括：所述分段中的道路语义元素，以及所述分段与相邻分段之间的连接关系。The generation of road semantic information corresponding to each segment includes at least: the road semantic elements in the segment and the connection relationship between the segment and adjacent segments.
根据权利要求1所述的方法，其特征在于，所述根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段，包括：The method according to claim 1, wherein the dividing the road image into at least two segments according to a change in the number of lane lines in the road image comprises:

根据每次所述道路图像中车道线数目发生改变的位置，得到所述道路图像的切分线；Obtaining the segmentation line of the road image according to the position where the number of lane lines in the road image changes each time;

利用所述切分线将所述道路图像切分为两个分段。The road image is divided into two segments by using the segmentation line.
根据权利要求2所述的方法，其特征在于，所述根据每次所述道路图像中车道线数目发生改变的位置，得到所述道路图像的切分线，包括：The method according to claim 2, wherein the obtaining the segmentation line of the road image according to the position where the number of lane lines in the road image changes each time comprises:

识别所述道路图像中的所述车道线，并获取所述车道线的端点位置；Identifying the lane line in the road image, and obtaining the end position of the lane line;

生成多条车道骨骼线，并识别所述多条车道骨骼线之间的一个或多个交叉位置；Generating multiple lane skeleton lines, and identifying one or more intersection positions between the multiple lane skeleton lines;

根据所述端点位置和所述一个或多个交叉位置之间的相对位置关系，确定所述切分线。The cutting line is determined according to the relative position relationship between the end point position and the one or more intersection positions.
根据权利要求3所述的方法，其特征在于，所述生成多条车道骨骼线，包括：The method according to claim 3, wherein said generating multiple lane skeleton lines comprises:

对于所述道路图像上的每个像素点：For each pixel on the road image:

获取所述像素点到最近的车道线的距离；Acquiring the distance from the pixel to the nearest lane line;

根据所述像素点到所述最近的车道线的距离与第一距离阈值的关系，对所述道路图像进行二值化处理，其中，到所述最近的车道线的距离小于或等于所述第一距离阈值的所述像素点置为第一值，到所述最近的车道线的距离大于所述第一距离阈值的所述像素点置为第二值；According to the relationship between the distance from the pixel point to the nearest lane line and the first distance threshold, the road image is binarized, wherein the distance to the nearest lane line is less than or equal to the first distance threshold. The pixel point with a distance threshold is set to a first value, and the pixel point with a distance to the nearest lane line greater than the first distance threshold is set to a second value;

对所述道路图像中像素点置为第二值对应的区域进行图像细化处理，得到所述多条车道骨骼线。Perform image thinning processing on the area corresponding to the pixel points in the road image set to the second value to obtain the multiple lane skeleton lines.
根据权利要求3或4所述的方法，其特征在于，所述识别所述多条车道骨骼线之间的一个或多个交叉位置，包括：The method according to claim 3 or 4, wherein the identifying one or more intersection positions between the multiple lane bone lines comprises:

遍历所述多条车道骨骼线上的第一像素点；Traverse the first pixel on the bone line of the multiple lanes;

对于每个第一像素点：For each first pixel:

以所述第一像素点为中心，获取设定区域内的连通分支数目；Taking the first pixel as the center, acquiring the number of connected branches in the set area;

在所述连通分支数目大于两个的情况下，将所述第一像素点作为所述多条车道骨骼线的候选交叉位置。In the case where the number of connected branches is greater than two, the first pixel is used as a candidate intersection position of the multiple lane skeleton lines.
根据权利要求5所述的方法，其特征在于，所述识别所述多条车道骨骼线之间的一个或多个交叉位置，还包括：The method according to claim 5, wherein the identifying one or more intersection positions between the multiple lane bone lines further comprises:

将所述候选交叉位置作为所述多条车道骨骼线之间的一个或多个交叉位置。The candidate intersection position is taken as one or more intersection positions between the multiple lane skeleton lines.
根据权利要求5所述的方法，其特征在于，所述识别所述多条车道骨骼线之间的一个或多个交叉位置，还包括：The method according to claim 5, wherein the identifying one or more intersection positions between the multiple lane bone lines further comprises:

当包括多个候选交叉位置时，按照候选交叉位置之间的距离小于或等于第二距离阈值对所述多个候选交叉位置进行聚类，将聚类后的结果作为所述多条车道骨骼线之间的一个或多个交叉位置。When multiple candidate intersection positions are included, cluster the multiple candidate intersection positions according to the distance between the candidate intersection positions being less than or equal to the second distance threshold, and use the result of the clustering as the multiple lane skeleton lines One or more intersections between.
根据权利要求3-7中任一项所述的方法，其特征在于，所述根据所述端点位置和所述一个或多个交叉位置之间的相对位置关系，确定所述切分线，包括：The method according to any one of claims 3-7, wherein the determining the cutting line according to the relative position relationship between the end point position and the one or more intersection positions comprises :

获取与每个交叉位置之间的距离小于或等于第三距离阈值的一个或多个端点位置作为候选端点位置；Acquiring one or more end-point positions whose distance to each intersection position is less than or equal to the third distance threshold as candidate end-point positions;

对于每个所述候选端点位置：For each of the candidate endpoint positions:

构造所述候选端点位置和所述交叉位置的连线；Constructing a line connecting the candidate endpoint position and the intersection position;

确定所述连线的中点位置作为切分点；Determining the midpoint position of the connecting line as a cutting point;

根据所述一个或多个候选端点位置对应的切分点，确定所述切分线。The segmentation line is determined according to the segmentation point corresponding to the position of the one or more candidate end points.
根据权利要求8所述的方法，其特征在于，所述根据所述一个或多个候选端点位置对应的切分点，确定所述切分线，包括：The method according to claim 8, wherein the determining the segmentation line according to the segmentation point corresponding to the position of the one or more candidate end points comprises:

在存在一个中点位置的情况下，确定所述切分线穿过所述中点位置，并垂直于所述连线；或If there is a midpoint position, determine that the cutting line passes through the midpoint position and is perpendicular to the connecting line; or

在存在两个中点位置的情况下，确定所述切分线穿过所述两个中点位置；或In the case that there are two midpoint positions, determining that the cutting line passes through the two midpoint positions; or

在存在两个以上的中点位置的情况下，根据所述两个以上的中点位置，拟合得到一条切分线。In the case where there are more than two midpoint positions, a dicing line is obtained by fitting according to the two or more midpoint positions.
根据权利要求8或9所述的方法，还包括：The method according to claim 8 or 9, further comprising:

对于纵向距离小于或等于第四距离阈值的至少两个交叉位置进行合并，根据其各自的中点位置确定所述切分线。Combine at least two crossing positions whose longitudinal distance is less than or equal to the fourth distance threshold, and determine the cutting line according to their respective midpoint positions.
根据权利要求2-10中任一项所述的方法，其特征在于，所述生成每一分段对应的道路语义信息包括生成所述分段中的道路语义元素，包括：The method according to any one of claims 2-10, wherein said generating road semantic information corresponding to each segment comprises generating road semantic elements in said segment, comprising:

根据各切分后车道线与所述切分线的相对位置关系，确定属于所述分段的切分后车道线；Determine the segmented lane line belonging to the segment according to the relative position relationship between each segmented lane line and the segmented line;

对所述分段中的车道中心线进行赋值；Assign a value to the centerline of the lane in the segment;

确定属于所述分段的切分后车道线与所述车道中心线的相对位置关系。Determine the relative positional relationship between the segmented lane line and the lane center line belonging to the segment.
根据权利要求11所述的方法，其特征在于，所述根据各切分后车道线与所述切分线的相对位置关系，确定属于所述分段的切分后车道线，包括：The method according to claim 11, wherein the determining the segmented lane line belonging to the segment according to the relative position relationship between each segmented lane line and the segmentation line comprises:

针对每条切分后车道线，For each lane line after segmentation,

根据所述切分后车道线的中心点到所述切分线起点的方向向量、以及所述切分线的方向向量，确定所述切分后车道线对应的方向标记；Determine the direction mark corresponding to the lane line after segmentation according to the direction vector from the center point of the lane line after segmentation to the starting point of the segmentation line and the direction vector of the segmentation line;

根据所述切分后车道线对应的方向标记，获取所述切分后车道线的掩码向量；Acquiring the mask vector of the lane line after segmentation according to the direction mark corresponding to the lane line after segmentation;

将所述掩码向量相同的多条切分后车道线归属为同一分段。The multiple segmented lane lines with the same mask vector are assigned to the same segment.
根据权利要求11或12所述的方法，其特征在于，所述确定属于所述分段的切分后车道线与所述车道中心线的相对位置关系，包括：The method according to claim 11 or 12, wherein the determining the relative position relationship between the divided lane line and the lane center line belonging to the segment includes:

获取所述车道中心线的方向均值；Obtaining the mean value of the direction of the center line of the lane;

针对每条属于所述分段的切分后车道线：For each segmented lane line belonging to the segment:

获取所述车道中心线的起点与所述切分后车道线的起点之间的方向向量；Acquiring a direction vector between the starting point of the center line of the lane and the starting point of the lane line after segmentation;

获取所述车道中心线的方向均值、以及所述方向向量之间的点积；Obtaining the mean value of the direction of the center line of the lane and the dot product between the direction vectors;

根据所述点积的值，确定所述切分后车道线位于所述车道中心线的物理左侧或物理右侧。According to the value of the dot product, it is determined that the lane line after segmentation is located on the physical left side or the physical right side of the center line of the lane.
根据权利要求13所述的方法，其特征在于，所述根据所述点积的值，确定所述切分后车道线位于所述车道中心线的物理左侧或物理右侧，包括：The method according to claim 13, wherein the determining, according to the value of the dot product, that the lane line after segmentation is located on the physical left side or the physical right side of the lane centerline comprises:

在所述点积大于0的情况下，确定所述切分后车道线位于所述车道中心线的物理左侧；或In the case that the dot product is greater than 0, it is determined that the lane line after segmentation is located on the physical left side of the center line of the lane; or

在所述点积小于或等于0的情况下，确定所述切分后车道线位于所述车道中心线的物理右侧。In a case where the dot product is less than or equal to 0, it is determined that the lane line after segmentation is located on the physical right side of the lane center line.
根据权利要求1-14中任一项所述的方法，其特征在于，所述生成每一分段对应的道路语义信息包括生成所述分段与所述相邻分段之间的连接关系，包括：The method according to any one of claims 1-14, wherein the generating road semantic information corresponding to each segment comprises generating a connection relationship between the segment and the adjacent segment, include:

生成所述分段与所述相邻分段之间的前继关系或后继关系；Generating a predecessor relationship or a successor relationship between the segment and the adjacent segment;

根据生成的所述前继关系或所述后继关系，获取所述分段中的多条车道与所述相邻分段中的多条车道之间的车道连接关系。According to the generated predecessor relationship or the successor relationship, the lane connection relationship between the multiple lanes in the segment and the multiple lanes in the adjacent segment is acquired.
根据权利要求15所述的方法，其特征在于，所述生成所述分段与所述相邻分段之间的前继关系或后继关系，包括：The method according to claim 15, wherein the generating the predecessor relationship or the successor relationship between the segment and the adjacent segment comprises:

计算所述分段的始端点与其他分段的末端点之间的距离，确定对应最近距离的分段为所述第一分段的前继；Calculate the distance between the start point of the segment and the end points of other segments, and determine that the segment corresponding to the closest distance is the predecessor of the first segment;

或，计算确定所述分段的末端点与其他分段的始端点之间的距离，确定对应最近距离的分段为所述第一分段的后继。Or, calculating and determining the distance between the end point of the segment and the beginning end point of other segments, and determining that the segment corresponding to the closest distance is the successor of the first segment.
一种获取道路语义信息的装置，包括：A device for acquiring road semantic information, including:

第一获取单元，用于获取道路图像；The first acquiring unit is used to acquire road images;

第一切分单元，用于根据所述道路图像中车道线数目的变化，将所述道路图像切分为至少两个分段；The first segmentation unit is configured to segment the road image into at least two segments according to the change in the number of lane lines in the road image;

第一生成单元，用于生成每一分段对应的道路语义信息，至少包括：所述分段中的道路语义元素，以及所述分段与相邻分段之间的连接关系。The first generating unit is configured to generate road semantic information corresponding to each segment, which includes at least: road semantic elements in the segment and the connection relationship between the segment and adjacent segments.
一种获取道路语义信息的设备，包括：A device for obtaining road semantic information, including:

存储器，用于存储计算机程序；Memory, used to store computer programs;

处理器；processor;

其中，所述处理器用于调用所述存储器中存储的所述计算机程序，执行如权利要求1-16中任一项所述的方法。Wherein, the processor is configured to call the computer program stored in the memory to execute the method according to any one of claims 1-16.
一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1-16中任一项所述的方法。A computer-readable storage medium with a computer program stored thereon, wherein the computer program implements the method of any one of claims 1-16 when the computer program is executed by a processor.
一种计算机程序产品，用于存储计算机可读指令，所述计算机可读指令被执行时使得计算机执行如权利要求1-16中任一项所述的方法。A computer program product for storing computer readable instructions, which when executed, cause a computer to execute the method according to any one of claims 1-16.