CN114241444A - Lane line recognition method and apparatus, storage medium, and electronic apparatus - Google Patents

Lane line recognition method and apparatus, storage medium, and electronic apparatus Download PDF

Info

Publication number
CN114241444A
CN114241444A CN202111567487.8A CN202111567487A CN114241444A CN 114241444 A CN114241444 A CN 114241444A CN 202111567487 A CN202111567487 A CN 202111567487A CN 114241444 A CN114241444 A CN 114241444A
Authority
CN
China
Prior art keywords
target
lane line
pixel
point
pixel point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111567487.8A
Other languages
Chinese (zh)
Inventor
龙琛
巫立峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN202111567487.8A priority Critical patent/CN114241444A/en
Publication of CN114241444A publication Critical patent/CN114241444A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method and a device for identifying lane lines, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a target characteristic diagram of a target image; determining the target position of a lane line vanishing point of the target image according to the target feature map; determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines; and performing lane line fitting on at least part of the pixels in the target pixel point collection based on the position distribution of each pixel and the target position in the target pixel point collection to obtain a target lane line. The invention solves the problem of poor recognition effect of the far-end lane line in the current lane line segmentation method.

Description

Lane line recognition method and apparatus, storage medium, and electronic apparatus
Technical Field
The invention relates to the technical field of image processing, in particular to a lane line identification method and device, a storage medium and an electronic device.
Background
The lane line recognition technology is mainly applied to automatic driving, and after lane line recognition is completed, the position relation between a current vehicle and a lane can be determined, and the safety of the vehicle in the driving process can be improved. In the existing lane line identification technology, a narrow line at the far end of a lane line may disappear in the down-sampling process, so that the information of the lane line vanishing point is ignored. Therefore, the fitted lane line has poor effect and the recognition accuracy of the lane line is low.
In view of the above problems, no effective solution for the object exists.
Disclosure of Invention
The embodiment of the invention provides a method and a device for identifying a lane line, a storage medium and an electronic device, which are used for at least solving the problem of low accuracy rate of identifying the lane line.
According to an aspect of an embodiment of the present invention, there is provided a lane line identification method, including: acquiring a target characteristic diagram of a target image; determining the target position of a lane line vanishing point of the target image according to the target feature map; determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines; and performing lane line fitting on at least part of the pixels in the target pixel point collection based on the position distribution of each pixel and the target position in the target pixel point collection to obtain a target lane line.
In an exemplary embodiment, based on the position distribution of each pixel point and the target position in the target pixel point set, performing lane line fitting on at least part of the pixel points in the target pixel point set to obtain a target lane line, including: determining pixel points with the distance between the target pixel point congregation and the target position meeting the preset condition as sub-target pixel points; and determining the lane line fitted by the sub-target pixel points as a target lane line.
In one exemplary embodiment, obtaining a target feature map of a target image includes: coding pixel values of pixel points in a target image to obtain a basic characteristic diagram of the target image, wherein the target image comprises W multiplied by H pixel points; and performing fusion decoding on the basic feature map to obtain a target feature map of the target image, wherein the target feature map comprises W × H feature values.
In one exemplary embodiment, determining the target position of the lane line vanishing point of the target image according to the target feature map comprises: processing the target characteristic diagram to obtain a vanishing point characteristic diagram and a first offset characteristic diagram, wherein the vanishing point characteristic diagram comprises n × n grids and n × n characteristic values which are in one-to-one correspondence with the n × n grids, each characteristic value in the n × n characteristic values represents the probability of a lane line vanishing point existing in a corresponding grid in the n × n grids, the first offset characteristic diagram comprises 2 × n × n characteristic values, and each pair of characteristic values in the 2 × n × n characteristic values represents the horizontal coordinate offset and the vertical coordinate offset of the lane line vanishing point existing in the corresponding grid in the n × n grids compared with the reference point in the corresponding grid; acquiring a target grid with the maximum probability of the lane line vanishing point in the vanishing point characteristic diagram; and determining the target position of the lane line vanishing point according to the coordinate of the reference point in the target grid and the abscissa offset and the ordinate offset represented by a pair of corresponding feature values in the first offset feature map.
In one exemplary embodiment, determining a lane line example segmentation result of a target image according to a target feature map comprises: processing the target feature map to obtain a center point feature map, a second offset feature map and an edge threshold feature map, wherein the target image comprises W × H pixel points, the target feature map comprises W × H feature values, the center point feature map comprises C × W × H feature values, each feature value in the C × W × H feature values represents the probability that the corresponding pixel point is the center point in the corresponding lane line, C represents the number of channels, the number of channels is equal to the number of categories of the lane line in the target image, the second offset feature map comprises 2 × W × H feature values, each pair of feature values in the second offset feature map represents the horizontal coordinate offset and the vertical coordinate offset of the corresponding pixel point relative to a target reference point, the target reference point corresponds to a preset pixel point in the target image, and the edge threshold feature map comprises 2 × W × H feature values, each pair of feature values in the edge threshold feature map represents an abscissa distance threshold and an ordinate distance threshold between a corresponding pixel point and a corresponding lane line edge; and determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map and the edge threshold feature map.
In an exemplary embodiment, determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map and the edge threshold feature map includes: for W × H feature values on each of C × W × H feature values, performing the following operations, wherein the W × H feature values on each channel are the W × H feature values on a current channel corresponding to a current category of the lane line, when performing the following operations: searching a characteristic value larger than a preset threshold value from the W multiplied by H characteristic values on the current channel, and determining pixel points in a target image corresponding to the searched characteristic value as a first pixel point collection;
repeatedly executing the following operations until the pixel points in the first pixel point congregation are distributed completely: in the first pixel point collection, a pixel point with the highest probability of the center point in the unassigned pixel points is assigned as the center point of an example of the lane line of the current category; determining the distance between the center point of one example and each pixel point in a second pixel point aggregate according to the characteristic value corresponding to the center point of one example in the second offset characteristic diagram and the characteristic value corresponding to each pixel point in the second pixel point aggregate in the second offset characteristic diagram, wherein the second pixel point aggregate comprises the unallocated pixel points except the center point of one example in the first pixel point aggregate; and distributing the pixels with the distance smaller than the edge distance threshold value in the second pixel point collection as the pixels in one example of the lane line of the current category, wherein the edge distance threshold value is the distance threshold value represented by a pair of corresponding characteristic values in the edge threshold value characteristic diagram.
In one exemplary embodiment, each in the target pixel based aggregationThe position distribution of individual pixel and target location carries out lane line fitting to target pixel gathers at least part of pixel, obtains the target lane line, includes: when the target pixel point collection comprises N pixel point collections, lane line fitting is carried out on each pixel point collection in the N pixel point collections respectively to obtain N item marking lane lines meeting preset conditions, wherein N is S1+…+Si+…SC,SiAnd the number of the instances of the lane lines of the corresponding category identified on the ith channel in the C channels is represented, the pixel point in each pixel point aggregation in the N pixel point aggregations is the pixel point in one instance of the lane line of the corresponding category, and N is a positive integer.
In an exemplary embodiment, determining a pixel point, for which a distance between a target pixel point congregation and a target position satisfies a preset condition, as a sub-target pixel point includes: executing the following steps for each pixel point collection in the N pixel point collections, wherein each pixel point collection is the current pixel point collection when the following steps are executed:
selecting a pixel point in each row in the current pixel point congregation to form a third pixel point congregation;
repeatedly executing the following operations until the distance between the fitted lane line and the target position of the lane line vanishing point is less than or equal to a first preset distance threshold, and determining all pixel points in the fitted pixel point set as sub-target pixel points corresponding to the current pixel point set:
determining a fitting pixel point union set in a third pixel point union set, wherein the fitting pixel point union set comprises a first pixel point, a second pixel point and a target pixel point subset in the third pixel point union set, and the distance from the pixel point in the target pixel point subset to a straight line formed by the first pixel point and the second pixel point is smaller than or equal to a second preset distance threshold value; forming a fitted lane line by the pixels in the fitting pixel aggregation; and judging whether the distance between the fitted lane line and the target position of the vanishing point of the lane line is less than or equal to a first preset distance threshold value or not.
According to another aspect of the embodiments of the present invention, there is also provided an identification apparatus of a lane line, including: the acquisition module is used for acquiring a characteristic diagram of an input image; the first determining module is used for determining the target position of a lane line vanishing point of the target image; the second determination module is used for determining a lane line example segmentation result of the target image, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines; and the fitting module is used for performing lane line fitting on at least part of the pixels in the target pixel point collection based on the position distribution of each pixel and the target position in the target pixel point collection to obtain a target lane line.
According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned lane line identification method when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the lane line identification method through the computer program.
According to the invention, the position of the lane line vanishing point is determined in the target characteristic diagram of the target image, and when the lane line is fitted, the obtained positions of the target lane line and the lane line vanishing point meet the preset condition. The lane line is fitted by combining the vanishing point of the lane line, so that the purpose of more accurate fitted lane line is achieved, the problem of low recognition accuracy of the lane line is further solved, and the effect of improving the recognition accuracy of the lane line is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a block diagram of a hardware configuration of a mobile terminal of a lane line recognition method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a lane line identification method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an alternative confirmed vanishing point target location in accordance with an embodiment of the invention;
FIG. 4 is a schematic flow chart diagram illustrating an alternative lane line identification method according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a preferred lane line identification method according to an embodiment of the present invention;
fig. 6 is a block diagram illustrating an alternative lane line recognition apparatus according to an embodiment of the present invention.
Detailed Description
The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided in the embodiments of the present invention may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking the example of the operation on the mobile terminal, fig. 1 is a hardware structure block diagram of the mobile terminal of the lane line identification method according to the embodiment of the present invention. As shown in fig. 1, the mobile terminal may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal may further include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of an application software, such as a computer program corresponding to a lane line identification method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the above-mentioned method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a Network adapter (NIC), which can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In this embodiment, a method for identifying a lane line running on the mobile terminal is provided, and fig. 2 is a schematic flow chart of the method for identifying a lane line according to the embodiment of the present invention, as shown in fig. 2, the flow chart includes the following steps:
step S202, acquiring a target characteristic diagram of a target image;
the target image is an acquired image to be identified and containing a lane line, and feature extraction is performed on the target image through a network, so that a target feature map is generated, and the target feature map represents features of pixel points of the target image.
Step S204, determining the target position of the lane line vanishing point of the target image according to the target feature map;
after the target feature map representing the features of the target image is obtained, the target feature map is processed in a processing mode including but not limited to convolution and upsampling, and the target position of the lane line vanishing point of the target image can be determined according to the processed target feature map.
It should be noted that the vanishing point is an intersection point where parallel lines in the 3D space are transformed to the image in perspective. In practice, the lane lines are all parallel lines, a plurality of parallel lane lines disappear from the same point in the image, the point is the lane line vanishing point, and the distances from the lane line vanishing point to all the lane lines are limited to a small range because all the lane lines disappear from the lane line vanishing point. The purpose of considering the vanishing point position is to calculate the distance from the vanishing point position to the fitted lane line when fitting the example segmentation result subsequently, and further judge whether the fitted lane line is accurate.
Step S206, determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines;
after a target feature map representing the features of the target image is obtained, the target feature map is processed, and a lane line example segmentation result of the target image can be determined according to the processed target feature map, wherein the lane line example segmentation result is a corresponding target pixel point collection found for each example in the target image.
And S208, performing lane line fitting on at least part of pixel points in the target pixel point collection based on the position distribution of each pixel point and the target position in the target pixel point collection to obtain a target lane line.
When the target pixel point collection in the lane line example segmentation result is subjected to lane line fitting, partial pixel points in the target pixel point collection are selected to fit a lane line, one lane line is fitted to partial pixel points in the target pixel point collection corresponding to each example, different lane lines can be fitted due to different selected partial pixel points, and when the distance between the lane line fitted by the partial pixel points and the target position of a lane line vanishing point meets a preset condition, the lane line fitted by the partial pixel point collection is output as the target lane line.
Through the steps, the accurate position of the vanishing point is confirmed, the distance from the vanishing point to the lane line fitted through the example segmentation result is utilized to determine the target lane line, and when the distance is smaller than the preset threshold value, the fitting is stopped, and the target lane line is output. By adopting the technical scheme, the problem of low lane line identification accuracy is solved, and the effect of improving the lane line identification accuracy is achieved.
The main body of the above steps may be a terminal or an image processing device, and the main body of the above steps may also be other processing devices or processing units with similar processing capabilities, but is not limited thereto. The following description is given by taking an example in which the image processing apparatus performs the above operation (which is only an exemplary description, and may be performed by another apparatus or module in an actual operation).
As an optional implementation manner, based on the position distribution of each pixel point and the target position in the target pixel point set, performing lane line fitting on at least part of the pixel points in the target pixel point set to obtain a target lane line, including: determining pixel points with the distance between the target pixel point congregation and the target position meeting the preset condition as sub-target pixel points; and determining the lane line fitted by the sub-target pixel points as a target lane line.
It should be noted that the pixels of the target pixel point union are all the pixels on the lane line identified in the example segmentation, one lane line corresponds to one target pixel point union, sub-target pixel points meeting preset conditions are selected in the target pixel point union, and the lane line fitted according to the sub-target pixel points is output as the target lane line; and when the distance between the position of the vanishing point and the simulated lane line is smaller than a certain preset value, the partial pixel points can be determined as the sub-target pixel points.
As an optional implementation, acquiring a target feature map of a target image includes: coding pixel values of pixel points in a target image to obtain a basic characteristic diagram of the target image, wherein the target image comprises W multiplied by H pixel points; and performing fusion decoding on the basic feature map to obtain a target feature map of the target image, wherein the target feature map comprises W × H feature values.
Optionally, a target image containing W × H pixel points is input into the network, the target passes through an encoding layer of the network, the encoding layer encodes pixel values of the pixel points in the target image through convolution and pooling to obtain a basic feature map of the target image, and the target feature map is used for feature extraction of the target image. After the basic characteristic diagram is obtained through coding, the basic characteristic diagram passes through a decoding layer, and the decoding layer performs fusion decoding on the characteristics of different layers in the basic characteristic diagram through upsampling and convolution to obtain a target characteristic diagram comprising W multiplied by H characteristic values. In the decoding process, the feature maps at different positions of the coding layer are connected to the corresponding positions of the decoding layer through convolution, and the detail feature information of the lower layer is obtained.
As an optional implementation manner, determining a target position of a lane line vanishing point of a target image according to a target feature map includes: processing the target characteristic diagram to obtain a vanishing point characteristic diagram and a first offset characteristic diagram, wherein the vanishing point characteristic diagram comprises n × n grids and n × n characteristic values which are in one-to-one correspondence with the n × n grids, each characteristic value in the n × n characteristic values represents the probability of a lane line vanishing point existing in a corresponding grid in the n × n grids, the first offset characteristic diagram comprises 2 × n × n characteristic values, and each pair of characteristic values in the 2 × n × n characteristic values represents the horizontal coordinate offset and the vertical coordinate offset of the lane line vanishing point existing in the corresponding grid in the n × n grids compared with the reference point in the corresponding grid; acquiring a target grid with the maximum probability of the lane line vanishing point in the vanishing point characteristic diagram; and determining the target position of the lane line vanishing point according to the coordinate of the reference point in the target grid and the abscissa offset and the ordinate offset represented by a pair of corresponding feature values in the first offset feature map.
Optionally, the vanishing point feature map and the first offset feature map are obtained by processing the target feature map, the grid corresponding to the maximum feature value is the target grid with the maximum probability of the vanishing point of the lane line by comparing the size of n × n feature values in the vanishing point feature map, after the target grid is found, the abscissa offset and the ordinate offset corresponding to the target grid compared to the reference point in the grid are found in the first offset feature map, the abscissa offset and the ordinate offset are added to the coordinates of the reference point in the target grid, the coordinates of a point in the grid are determined, and the point is determined as the target position of the vanishing point of the lane line, where the reference point may be an upper left point in the grid or a lower right point in the grid, which is not limited in this embodiment.
Fig. 3 is a schematic diagram of an optional vanishing point target position confirmation according to an embodiment of the present invention, and as shown in fig. 3, a target grid with the highest vanishing point existence probability is found in the vanishing point feature map, and the target position of the lane line vanishing point is determined by adding the horizontal coordinate offset and the vertical coordinate offset corresponding to the target grid to the upper left point coordinate in the target grid.
It should be noted that, in the method for determining the vanishing point, the approximate position of the vanishing point in the image is obtained through coarse-grained division, and then the vanishing point position prediction of fine granularity is achieved through predicting the offset coordinate, so that the calculation amount is reduced, and a relatively accurate vanishing point position can be obtained.
As an optional implementation manner, determining a lane line example segmentation result of the target image according to the target feature map includes: processing the target feature map to obtain a center point feature map, a second offset feature map and an edge threshold feature map, wherein the target image comprises W × H pixel points, the target feature map comprises W × H feature values, the center point feature map comprises C × W × H feature values, each feature value in the C × W × H feature values represents the probability that the corresponding pixel point is the center point in the corresponding lane line, C represents the number of channels, the number of channels is equal to the number of categories of the lane line in the target image, the second offset feature map comprises 2 × W × H feature values, each pair of feature values in the second offset feature map represents the horizontal coordinate offset and the vertical coordinate offset of the corresponding pixel point relative to a target reference point, the target reference point corresponds to a preset pixel point in the target image, and the edge threshold feature map comprises 2 × W × H feature values, each pair of feature values in the edge threshold feature map represents an abscissa distance threshold and an ordinate distance threshold between a corresponding pixel point and a corresponding lane line edge; and determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map and the edge threshold feature map.
Optionally, assuming that there are two lane lines of a solid line and a dashed line in the target image, the width is 8, and the height is 4, then the number C of categories of the lane lines is 2, the target feature map is subjected to 4 times of 3 × 3 convolution operations and 5 times of upsampling, and a feature map with the number of channels of 6, the width of 8, and the height of 4 is obtained, where C is the number of categories of the identified lane lines, and the feature maps of 2 channels are used as feature maps of a central point, and in the feature maps of the other 4 channels, the feature maps of the first two channels are used as second offset feature maps, and the feature maps of the last two channels are used as edge threshold feature maps. And in the last up-sampling process, respectively outputting a second offset characteristic map, an edge threshold characteristic map and a central point characteristic map through different deconvolution. Each channel comprises 8 x 4 signatures. And according to the second offset characteristic diagram, the edge threshold characteristic diagram and the central point characteristic diagram, segmenting the result by using the example of each category.
As an optional implementation manner, determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map, and the edge threshold feature map includes: for W × H feature values on each of C × W × H feature values, performing the following operations, wherein the W × H feature values on each channel are the W × H feature values on a current channel corresponding to a current category of the lane line, when performing the following operations: searching a characteristic value larger than a preset threshold value from the W multiplied by H characteristic values on the current channel, and determining pixel points in a target image corresponding to the searched characteristic value as a first pixel point collection; repeatedly executing the following operations until the pixel points in the first pixel point congregation are distributed completely: in the first pixel point collection, a pixel point with the highest probability of the center point in the unassigned pixel points is assigned as the center point of an example of the lane line of the current category; determining the distance between the center point of one example and each pixel point in a second pixel point aggregate according to the characteristic value corresponding to the center point of one example in the second offset characteristic diagram and the characteristic value corresponding to each pixel point in the second pixel point aggregate in the second offset characteristic diagram, wherein the second pixel point aggregate comprises the unallocated pixel points except the center point of one example in the first pixel point aggregate; and distributing the pixels with the distance smaller than the edge distance threshold value in the second pixel point collection as the pixels in one example of the lane line of the current category, wherein the edge distance threshold value is the distance threshold value represented by a pair of corresponding characteristic values in the edge threshold value characteristic diagram.
Optionally, in the feature map of the center point of each channel, the feature values are compared, the pixel points greater than the preset threshold are used as masks (first pixel point collections) of the current category, and the pixel point with the maximum feature value is used as the center point of the current instance. For example, in a center point feature map of a solid line lane line category, a pixel point larger than a preset threshold is taken as a mask of the solid line lane line, wherein a pixel point with the largest feature value (probability of the center point) is taken as a center point of a first solid line lane line, the distance between a pixel point in a first pixel point set and the center point is determined according to a feature value in a second offset feature map, when the distance is smaller than a corresponding distance threshold in an edge threshold feature map, a corresponding pixel point is allocated as a pixel point on the first solid line lane line, the above process is repeated, a pixel point on the next solid line lane line is searched, and until the pixel point in the first pixel set is allocated, all lane lines have corresponding pixel point sets. And mapping all the determined pixel point collections in the channels of all the categories to the same graph, thereby generating a lane line example segmentation result.
As an optional implementation manner, based on the position distribution of each pixel point and the target position in the target pixel point set, performing lane line fitting on at least part of the pixel points in the target pixel point set to obtain a target lane line, including: when the target pixel point collection comprises N pixel point collections, lane line fitting is carried out on each pixel point collection in the N pixel point collections respectively to obtain N item marking lane lines meeting preset conditions, wherein N is S1+…+Si+…SC,SiAnd the number of the instances of the lane lines of the corresponding category identified on the ith channel in the C channels is represented, the pixel point in each pixel point aggregation in the N pixel point aggregations is the pixel point in one instance of the lane line of the corresponding category, and N is a positive integer.
Optionally, in the example segmentation, a mask of each lane line in each category of lane lines is found, that is, a corresponding pixel point aggregate, and it is assumed that there are two categories of lane lines, namely a solid line and a dotted line, in the target image, there are 2 lane lines in the solid line and three lane lines in the dotted line, so there are five lane lines in the target image, and there are five pixel point aggregates corresponding to each lane line in the example segmentation result.
As an optional implementation manner, determining, as a sub-target pixel, a pixel whose distance between the target pixel congregation and the target position satisfies a preset condition, includes: executing the following steps for each pixel point collection in the N pixel point collections, wherein each pixel point collection is the current pixel point collection when the following steps are executed: selecting a pixel point in each row in the current pixel point congregation to form a third pixel point congregation; repeatedly executing the following operations until the distance between the fitted lane line and the target position of the lane line vanishing point is less than or equal to a first preset distance threshold, and determining all pixel points in the fitted pixel point set as sub-target pixel points corresponding to the current pixel point set: determining a fitting pixel point union set in a third pixel point union set, wherein the fitting pixel point union set comprises a first pixel point, a second pixel point and a target pixel point subset in the third pixel point union set, and the distance from the pixel point in the target pixel point subset to a straight line formed by the first pixel point and the second pixel point is smaller than or equal to a second preset distance threshold value; forming a fitted lane line by the pixels in the fitting pixel aggregation; and judging whether the distance between the fitted lane line and the target position of the vanishing point of the lane line is less than or equal to a first preset distance threshold value or not.
Optionally, fitting a pixel point collection corresponding to each lane line, for example, when fitting a pixel point collection corresponding to a solid lane line, selecting a pixel point at each of the pixel point collections corresponding to the solid lane lines to form a third pixel point collection; and selecting two pixel points in the third pixel point congregation, confirming a straight line, determining the distance from other pixel points in the third pixel point congregation to the straight line, and taking the pixel points corresponding to the distance smaller than a second preset threshold value as interior points to form a fitting pixel point congregation. And forming a fitted lane line according to the fitting pixel point collection. And calculating the distance from the vanishing point to the fitted lane line, determining the pixel points in the current fitted pixel point set as sub-target pixel points of the current example pixel point set when the distance is less than a first preset threshold, and determining the lane line fitted by the sub-target pixel points as the example lane line and outputting. And when the distance does not meet the preset condition, randomly selecting two pixel points in the third pixel point congregation, and repeating the process until the distance is smaller than a first preset threshold value.
It should be noted that, in the existing fitting process, the number of pixels in the fitting pixel set, that is, the number of interior points, is determined to serve as a reference standard for whether to output the fitting lane line, in this example, the distance from the vanishing point to the fitting lane line is used as a reference standard for whether to output the fitting lane line, and when the distance from the vanishing point to the fitting lane line is smaller than a first preset threshold, the fitting lane line can be output as well even if the number of interior points is smaller than a preset value, so that the number of iterations in the fitting process is reduced to a certain extent.
Optionally, in this example, when the number of pixels in the fitting pixel aggregation, that is, the number of interior points is greater than the preset value, the fitting lane line is also output.
It is to be understood that the above-described embodiments are only a few, but not all, embodiments of the present invention.
The present invention will be described in detail with reference to the following examples:
fig. 4 is a schematic flow chart of an alternative lane line identification method according to an embodiment of the present invention, where the method includes the following steps:
step S401, inputting a target image, wherein the target image is an image obtained by shooting a lane line through image acquisition equipment;
step S402, after a target image is input, the target image is coded through a coding layer to obtain a basic feature map representing the features of the target image; the basic characteristic diagram is the shared characteristic of a vanishing point branch for determining a vanishing point and a segmentation branch for determining an example segmentation result, the characteristic extraction is mainly carried out on an input image, and the coding is mainly completed by convolution and pooling of 3 multiplied by 3;
step S403, performing fusion decoding on different layers in the basic feature map through a decoding layer to obtain a target feature map; the fused decoding is mainly done by upsampling and 1 x 1 convolution. In the decoding process, feature maps at different positions of the coding layer are also connected to corresponding positions of the decoding layer through convolution to obtain detail feature information, offset and edge threshold of a lower layer, which share the same decoding layer. After fusion decoding of a decoding layer, sending the obtained target feature graph to a vanishing point branch and a segmentation branch;
step S404, a vanishing point feature map and a first offset feature map are obtained by processing the target feature map in a vanishing point branch, wherein the vanishing point feature map is a grid feature map with the size of n multiplied by n, each grid corresponds to a feature value, each feature value represents the probability of vanishing points in the grid, the first offset feature map is a feature map with the size of 2 multiplied by n, the feature values represent the offset of possible vanishing points in each grid relative to the horizontal coordinate and the vertical coordinate of a reference in the grid, and n represents the size of the grid;
step S405, determining a target position of a lane line vanishing point according to the vanishing point feature map and the first offset feature map, finding a grid with the maximum vanishing point existence probability in the vanishing point feature map, and adding coordinates of reference points in the grid to the abscissa offset and the ordinate offset in the first offset feature map to obtain coordinates of the vanishing point, namely determining the accurate position of the vanishing point;
step S406, processing the target feature map in the division branch to obtain a feature map with the channel number of C +4 and the width and height equal to the size of the original map, wherein C represents the category number of the lane line, the first C channels are central point feature maps, the central point feature maps of different channels correspond to the central point feature maps of different categories, the feature maps of the first two channels of the other four channels are second offset feature maps in the learned abscissa direction and the ordinate direction, and the last two channels are edge threshold feature maps in the learned abscissa direction and the ordinate direction;
step S407, obtaining a lane line example segmentation result according to the central point feature map, the second offset feature map and the edge threshold feature map, screening out pixel points with the central point probability greater than a preset threshold value from the central point feature map of each channel, finding out pixel points with the maximum central point probability as the central points of the current example, calculating the distance from other pixel points with the central point probability greater than the preset value to the center of the current example according to the offset, distributing the pixel points with the distance less than the edge threshold value to the current example, searching the central point of the next example from other pixel points with the central point probability greater than the preset value, repeating the above processes until the distribution of the pixel points with the central point probability greater than the preset value is completed, and segmenting each lane line example; all the pixel points distributed by different examples on different channels are merged onto the same image to obtain a lane line example segmentation result;
it should be noted that, because the offset between the edge pixel of the larger example and the center point thereof is larger, it is not beneficial to learning the offset, and the coordinate of the pixel added with the offset may be closer to other examples in the post-processing process, thereby causing the misclassification of the pixel. Therefore, the pixel coordinates and the offset are added, the pixel coordinates and the offset are converted into the prediction pixel points corresponding to the center of the example through a Gaussian formula, the offset of the edge pixels is limited to a smaller range, and the prediction pixel points are supervised and learned to achieve a better segmentation effect;
it should be noted that, if the central points of all categories are filtered on the feature map of the same channel, it is difficult to distinguish the central points of all category examples by the same threshold, so the number of channels of the feature map of the central point is designed as the number of categories, which enables the filtering of the example central points of different categories on different channels to select the most accurate central point.
Step S408, finishing lane line fitting according to the lane line example segmentation result; the method comprises the steps of randomly selecting pixel points in a lane line example segmentation result in a line scanning mode for fitting, randomly selecting one pixel point in each line of the pixel points in each example, randomly selecting two pixel points in a selected pixel point congregation to calculate a straight line, calculating the distance from other pixel points in the pixel point congregation to the straight line, and taking the pixel points with the distance smaller than a preset value as inner points;
step S409, calculating the distance between the lane line and the vanishing point of the lane line obtained in the fitting process;
step S410, comparing the distance with a first preset threshold value, and judging whether the distance between the lane line and the lane line vanishing point obtained in the fitting process is smaller than the first preset value, if so, executing step S411, otherwise, repeatedly executing step S408;
it should be noted that the fitting times can be reduced to a certain extent through the steps, the time consumption of the algorithm is reduced, and meanwhile, a more accurate lane line model is obtained, so that the problem of poor recognition effect on the far-end lane line is solved.
And when the distance is still not less than the first preset value after the multiple fitting, outputting the fitted lane line when the number of the inner points is greater than a certain threshold value or the fitting times reaches a certain preset value.
And step S411, outputting the lane line obtained in the fitting process as a target lane line.
It should be noted that the execution sequence of the step S404 and the step S406 may be interchanged, that is, the step S406 may be executed first, and then the step S404 may be executed, or the steps may be executed simultaneously.
Fig. 5 is a schematic diagram of a preferred lane line identification method according to an embodiment of the present invention, and as shown in fig. 5, firstly, an image of a lane line to be identified is input into a neural network, and is encoded through an encoding layer of the neural network to obtain a basic feature map, and features are performed on the input image; after the basic characteristic diagram is obtained, fusion decoding is carried out through a decoding layer of the neural network, and characteristic diagrams of different layers are obtained;
respectively sending the feature maps of different layers output by the decoding layer into a vanishing point branch (VP) and a splitting branch for processing respectively:
in the vanishing point branch, the feature map output by the decoding layer passes through the vanishing point branch to obtain a grid feature map with the size of n × n and a vanishing point offset feature map with the size of 2 × n × n, wherein the grid feature map represents the probability of vanishing points existing in the grid, and the offset n of vanishing point coordinates possibly existing in each grid of the offset feature map relative to the coordinates of top left vertex of the grid can be 30 or 20. According to the method, the approximate position of a vanishing point in an image is obtained through coarse-grained division, and then the vanishing point position prediction of fine granularity is achieved through predicting offset coordinates, so that the calculation amount is reduced, and the more accurate vanishing point position can be obtained.
In the division branch, the feature map output by the decoding layer is subjected to 4 times of convolution with 3 × 3 times and 5 times of upsampling to respectively obtain a central point feature map, an offset and an edge threshold feature map, and in the last upsampling process, the offset feature map, the edge threshold feature map and the central point feature map are respectively output through different deconvolution.
The post-processing process is divided into two parts of lane line segmentation and lane line fitting. And dividing the lane line according to the offset characteristic diagram, the edge threshold characteristic diagram and the central point characteristic diagram. And for the central point feature graph, obtaining masks through threshold filtering in each channel, finding out a pixel point with the highest probability as the central point of the current class example, calculating the distance from a pixel vector in the mask to the center of the example, using the pixel smaller than the edge threshold as the mask of the current example, finding the central point of the next example, and calculating the same until the number of the masks of the current class is distributed. And merging all the example masks to the same graph to obtain a final lane line example segmentation result.
In the process of fitting the lane line, the proposal adopts a mode of scanning line by line to randomly select mask pixels for fitting. The method comprises the steps of selecting a mask pixel point in each row randomly, selecting two pixel points in the selected pixel points randomly to calculate a straight line, calculating the distance between the other selected mask pixel points and the straight line, taking the mask pixel points smaller than the distance as interior points, and selecting two pixel points in the interior points randomly to repeat the process.
And fitting a lane line in each repeated process, considering vanishing point information in each ransac fitting process, iteratively calculating the distance error between the vanishing point and the fitted lane line each time, stopping iteration when the error is less than a certain threshold value, and outputting the lane line, namely a lane line model. The method can reduce the ransac iteration times to a certain extent, and obtain a more accurate lane line model, thereby making up the problem of poor segmentation effect on the far-end lane line. When the number of the inner points is larger than the threshold value, the lane line is also output.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
In this embodiment, a lane line identification apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description of which has been already made is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 6 is a block diagram of an alternative lane line recognition apparatus according to an embodiment of the present invention, as shown in fig. 6, the apparatus including:
an obtaining module 602, configured to obtain a target feature map of an input image;
a first determining module 604, configured to determine a target position of a lane line vanishing point of the target image;
a second determining module 606, configured to determine a lane line instance segmentation result of the target image, where the lane line instance segmentation result includes N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane line;
the fitting module 608 performs lane line fitting on at least a part of the pixels in the target pixel set based on the position distribution of each pixel in the target pixel set and the target position, so as to obtain a target lane line.
Optionally, the device is further configured to determine, as sub-target pixel points, pixel points whose distance between the target pixel point congregation and the target position meets a preset condition; and determining the lane line fitted by the sub-target pixel points as a target lane line.
Optionally, the device is further configured to encode pixel values of pixels in the target image to obtain a basic feature map of the target image, where the target image includes W × H pixels; and performing fusion decoding on the basic feature map to obtain a target feature map of the target image, wherein the target feature map comprises W × H feature values.
Optionally, the apparatus is further configured to process the target feature map to obtain a vanishing point feature map and a first offset feature map, where the vanishing point feature map includes n × n grids and n × n feature values corresponding to the n × n grids in a one-to-one manner, each feature value in the n × n feature values represents a probability that a lane line vanishing point exists in a corresponding grid in the n × n grids, the first offset feature map includes 2 × n × n feature values, and each pair of feature values in the 2 × n × n feature values represents an abscissa offset and an ordinate offset of the lane line vanishing point existing in the corresponding grid in the n × n grids compared to a reference point in the corresponding grid; acquiring a target grid with the maximum probability of the lane line vanishing point in the vanishing point characteristic diagram; and determining the target position of the lane line vanishing point according to the coordinate of the reference point in the target grid and the abscissa offset and the ordinate offset represented by a pair of corresponding feature values in the first offset feature map.
Optionally, the apparatus is further configured to process the target feature map to obtain a center point feature map, a second offset feature map, and an edge threshold feature map, where the target image includes W × H pixel points, the target feature map includes W × H feature values, the center point feature map includes C × W × H feature values, each feature value in the C × W × H feature values indicates a probability that a corresponding pixel point is a center point in a corresponding lane line, C indicates a number of channels, the number of channels is equal to the number of categories of the lane line in the target image, the second offset feature map includes 2 × W × H feature values, each pair of feature values in the second offset feature map indicates an abscissa offset and an ordinate offset of the corresponding pixel point with respect to a target reference point, the target reference point corresponds to a preset pixel point in the target image, and the edge threshold feature map includes 2 × W × H feature values, each pair of feature values in the edge threshold feature map represents an abscissa distance threshold and an ordinate distance threshold between a corresponding pixel point and a corresponding lane line edge; and determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map and the edge threshold feature map.
Optionally, the apparatus is further configured to perform the following operations for W × H feature values on each channel of C × W × H feature values, where the W × H feature values on each channel are W × H feature values on a current channel, and the current channel corresponds to a current category of the lane line, when performing the following operations: searching a characteristic value larger than a preset threshold value from the W multiplied by H characteristic values on the current channel, and determining pixel points in a target image corresponding to the searched characteristic value as a first pixel point collection; repeatedly executing the following operations until the pixel points in the first pixel point congregation are distributed completely: in the first pixel point collection, a pixel point with the highest probability of the center point in the unassigned pixel points is assigned as the center point of an example of the lane line of the current category; determining the distance between the center point of one example and each pixel point in a second pixel point aggregate according to the characteristic value corresponding to the center point of one example in the second offset characteristic diagram and the characteristic value corresponding to each pixel point in the second pixel point aggregate in the second offset characteristic diagram, wherein the second pixel point aggregate comprises the unallocated pixel points except the center point of one example in the first pixel point aggregate; and distributing the pixels with the distance smaller than the edge distance threshold value in the second pixel point collection as the pixels in one example of the lane line of the current category, wherein the edge distance threshold value is the distance threshold value represented by a pair of corresponding characteristic values in the edge threshold value characteristic diagram.
Optionally, the device is further configured to, when the target pixel point set includes N pixel point sets, perform lane line fitting on each pixel point set in the N pixel point sets, respectively, to obtain N entry lane lines meeting preset conditions, where N is S1+…+Si+…SC,SiRepresenting the number of instances of the lane line of the corresponding category identified on the ith channel in the C channels, wherein the pixel points in each pixel point congregation in the N pixel point congregations are pairsN is a positive integer corresponding to a pixel point in an example of a lane line of a category.
Optionally, the apparatus is further configured to execute the following steps for each pixel point union in the N pixel point unions, where each pixel point union is a current pixel point union when executing the following steps: selecting a pixel point in each row in the current pixel point congregation to form a third pixel point congregation; repeatedly executing the following operations until the distance between the fitted lane line and the target position of the lane line vanishing point is less than or equal to a first preset distance threshold, and determining all pixel points in the fitted pixel point set as sub-target pixel points corresponding to the current pixel point set: determining a fitting pixel point union set in a third pixel point union set, wherein the fitting pixel point union set comprises a first pixel point, a second pixel point and a target pixel point subset in the third pixel point union set, and the distance from the pixel point in the target pixel point subset to a straight line formed by the first pixel point and the second pixel point is smaller than or equal to a second preset distance threshold value; forming a fitted lane line by the pixels in the fitting pixel aggregation; and judging whether the distance between the fitted lane line and the target position of the vanishing point of the lane line is less than or equal to a first preset distance threshold value or not.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above-mentioned method embodiments when executed.
In the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
s1, acquiring a target characteristic diagram of the target image;
s2, determining the target position of the lane line vanishing point of the target image according to the target feature map;
s3, determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines;
and S4, performing lane line fitting on at least part of the pixels in the target pixel collection based on the position distribution of each pixel and the target position in the target pixel collection to obtain a target lane line.
Optionally, in this embodiment, the computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
In an exemplary embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring a target characteristic diagram of the target image;
s2, determining the target position of the lane line vanishing point of the target image according to the target feature map;
s3, determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on the lane lines;
and S4, performing lane line fitting on at least part of the pixels in the target pixel collection based on the position distribution of each pixel and the target position in the target pixel collection to obtain a target lane line.
For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details of this embodiment are not repeated herein.
It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and they may be implemented using program code executable by the computing devices, such that they may be stored in a memory device and executed by the computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.

Claims (11)

1. A method for identifying a lane line, comprising:
acquiring a target characteristic diagram of a target image;
determining the target position of a lane line vanishing point of the target image according to the target feature map;
determining a lane line example segmentation result of the target image according to the target feature map, wherein the lane line example segmentation result comprises N target pixel point collections, and pixel points in the target pixel point collections are identified as points on lane lines;
and performing lane line fitting on at least part of the pixels in the target pixel point collection based on the position distribution of each pixel and the target position in the target pixel point collection to obtain a target lane line.
2. The method according to claim 1, wherein the performing lane line fitting on at least part of the pixels in the target pixel set based on the position distribution of each pixel and the target position in the target pixel set to obtain a target lane line comprises:
determining the pixel points with the distance between the target pixel point congregation and the target position meeting the preset condition as sub-target pixel points;
and determining the lane line fitted by the sub-target pixel points as the target lane line.
3. The method of claim 1, wherein the obtaining of the target feature map of the target image comprises:
coding pixel values of pixel points in the target image to obtain a basic feature map of the target image, wherein the target image comprises W × H pixel points;
and performing fusion decoding on the basic feature map to obtain the target feature map of the target image, wherein the target feature map comprises W × H feature values.
4. The method according to claim 1, wherein the determining the target position of the lane line vanishing point of the target image according to the target feature map comprises:
processing the target feature map to obtain a vanishing point feature map and a first offset feature map, wherein the vanishing point feature map comprises n × n grids and n × n feature values in one-to-one correspondence with the n × n grids, each feature value in the n × n feature values represents a probability that the lane line vanishing point exists in a corresponding grid in the n × n grids, the first offset feature map comprises 2 × n × n feature values, and each pair of feature values in the 2 × n × n feature values represents an abscissa offset and an ordinate offset of the lane line vanishing point existing in the corresponding grid in the n × n grids compared with a reference point in the corresponding grid;
acquiring a target grid with the highest probability of the lane line vanishing point existing in the vanishing point feature map;
and determining the target position of the lane line vanishing point according to the coordinates of the reference point in the target grid and the abscissa offset and the ordinate offset represented by a pair of corresponding feature values in the first offset feature map.
5. The method according to claim 1, wherein the determining a lane line instance segmentation result of the target image according to the target feature map comprises:
processing the target feature map to obtain a center point feature map, a second offset feature map and an edge threshold feature map, wherein the target image comprises W × H pixel points, the target feature map comprises W × H feature values, the center point feature map comprises C × W × H feature values, each feature value in the C × W × H feature values represents the probability that the corresponding pixel point is the center point in the corresponding lane line, C represents the number of channels, the number of channels is equal to the number of categories of the lane line in the target image, the second offset feature map comprises 2 × W × H feature values, each pair of feature values in the second offset feature map represents the horizontal coordinate offset and the vertical coordinate offset of the corresponding pixel point relative to a target reference point, and the target reference point corresponds to a preset pixel point in the target image, the edge threshold feature map comprises 2 xWxH feature values, and each pair of feature values in the edge threshold feature map represents an abscissa distance threshold and an ordinate distance threshold between a corresponding pixel point and a corresponding lane line edge;
and determining a lane line example segmentation result of the target image according to the central point feature map, the second offset feature map and the edge threshold feature map.
6. The method according to claim 5, wherein the determining a lane line instance segmentation result of the target image according to the center point feature map, the second offset feature map and the edge threshold feature map comprises:
for W × H feature values on each of C × W × H feature values, performing the following operations, wherein the W × H feature values on each channel are W × H feature values on a current channel corresponding to a current category of lane lines, when performing the following operations:
searching for a characteristic value larger than a preset threshold value from the W multiplied by H characteristic values on the current channel, and determining pixel points in the target image corresponding to the searched characteristic value as a first pixel point collection;
repeatedly executing the following operations until the pixel points in the first pixel point congregation are distributed completely: distributing the pixel point with the highest probability of the central point in the unassigned pixel points as the central point of an example of the lane line of the current category in the first pixel point set;
determining the distance between the center point of the example and each pixel point in a second pixel point congregation according to the characteristic value corresponding to the center point of the example in the second offset characteristic graph and the characteristic value corresponding to each pixel point in the second pixel point congregation in the second offset characteristic graph, wherein the second pixel point congregation comprises the unallocated pixel points except the center point of the example in the first pixel point congregation;
and allocating the pixel points of which the distance is smaller than an edge distance threshold in the second pixel point congregation as the pixel points in the one example of the lane line of the current category, wherein the edge distance threshold is a distance threshold represented by a corresponding pair of feature values in the edge threshold feature map.
7. The method according to claim 5, wherein the step of performing lane line fitting on at least part of the pixels in the target pixel set based on the position distribution of each pixel and the target position in the target pixel set to obtain a target lane line comprises:
when the target pixel point collection comprises N pixel point collections, lane line fitting is carried out on each pixel point collection in the N pixel point collections respectively to obtain N item lane lines meeting preset conditions, wherein N is S1+…+Si+…SC,SiAnd the number of the instances of the lane lines of the corresponding category identified on the ith channel in the C channels is represented, the pixel point in each pixel point aggregation in the N pixel point aggregations is the pixel point in one instance of the lane line of the corresponding category, and N is a positive integer.
8. The method of claim 2, wherein determining the pixels with the distance between the target pixel congregation and the target position satisfying a preset condition as sub-target pixels comprises:
executing the following steps for each pixel point collection in the N target pixel point collections, wherein each pixel point collection is a current pixel point collection when the following steps are executed:
selecting a pixel point in each row in the current pixel point congregation to form a third pixel point congregation;
repeatedly executing the following operations until the distance between the fitted lane line and the target position of the lane line vanishing point is less than or equal to a first preset distance threshold, and determining all the pixel points in the fitted pixel point set as sub-target pixel points corresponding to the current pixel point set:
determining a fitting pixel point union set in the third pixel point union set, wherein the fitting pixel point union set comprises a first pixel point and a second pixel point in the third pixel point union set and a target pixel point subset, and the distance from the pixel point in the target pixel point subset to a straight line formed by the first pixel point and the second pixel point is smaller than or equal to a second preset distance threshold value;
forming a fitted lane line by the pixels in the fitting pixel aggregation;
and judging whether the distance between the fitted lane line and the target position of the vanishing point of the lane line is smaller than or equal to the first preset distance threshold value or not.
9. A lane line identification apparatus, comprising:
the acquisition module is used for acquiring a target characteristic map of a target image;
the first determining module is used for determining the target position of a lane line vanishing point of the target image according to the target feature map;
a second determining module, configured to determine, according to the target feature map, a lane line instance segmentation result of the target image, where the lane line instance segmentation result includes N target pixel point collections, and pixel points in the target pixel point collections are identified as points on a lane line;
and the fitting module is used for performing lane line fitting on at least part of the pixels in the target pixel point collection based on the position distribution of each pixel and the target position in the target pixel point collection to obtain a target lane line.
10. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 8.
11. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 8 by means of the computer program.
CN202111567487.8A 2021-12-20 2021-12-20 Lane line recognition method and apparatus, storage medium, and electronic apparatus Pending CN114241444A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111567487.8A CN114241444A (en) 2021-12-20 2021-12-20 Lane line recognition method and apparatus, storage medium, and electronic apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111567487.8A CN114241444A (en) 2021-12-20 2021-12-20 Lane line recognition method and apparatus, storage medium, and electronic apparatus

Publications (1)

Publication Number Publication Date
CN114241444A true CN114241444A (en) 2022-03-25

Family

ID=80759904

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111567487.8A Pending CN114241444A (en) 2021-12-20 2021-12-20 Lane line recognition method and apparatus, storage medium, and electronic apparatus

Country Status (1)

Country Link
CN (1) CN114241444A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114445716A (en) * 2022-04-07 2022-05-06 腾讯科技（深圳）有限公司 Key point detection method, key point detection device, computer device, medium, and program product
CN114973180A (en) * 2022-07-18 2022-08-30 福思(杭州)智能科技有限公司 Lane line tracking method, device, equipment and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114445716A (en) * 2022-04-07 2022-05-06 腾讯科技（深圳）有限公司 Key point detection method, key point detection device, computer device, medium, and program product
CN114445716B (en) * 2022-04-07 2022-07-26 腾讯科技（深圳）有限公司 Key point detection method, key point detection device, computer device, medium, and program product
CN114973180A (en) * 2022-07-18 2022-08-30 福思(杭州)智能科技有限公司 Lane line tracking method, device, equipment and storage medium
CN114973180B (en) * 2022-07-18 2022-11-01 福思(杭州)智能科技有限公司 Lane line tracking method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109614935B (en) Vehicle damage assessment method and device, storage medium and electronic equipment
CN110163813B (en) Image rain removing method and device, readable storage medium and terminal equipment
CN114241444A (en) Lane line recognition method and apparatus, storage medium, and electronic apparatus
CN111210429A (en) Point cloud data partitioning method and device and obstacle detection method and device
CN111582054B (en) Point cloud data processing method and device and obstacle detection method and device
CN110135227B (en) Laser point cloud outdoor scene automatic segmentation method based on machine learning
CN114051628B (en) Method and device for determining target object point cloud set
CN111553946B (en) Method and device for removing ground point cloud and method and device for detecting obstacle
CN104899853A (en) Image region dividing method and device
CN104766275B (en) Sparse disparities figure denseization method and apparatus
CN106980851B (en) Method and device for positioning data matrix DM code
WO2022099528A1 (en) Method and apparatus for calculating normal vector of point cloud, computer device, and storage medium
CN104463183A (en) Cluster center selecting method and system
CN114777792A (en) Path planning method and device, computer readable medium and electronic equipment
CN115147333A (en) Target detection method and device
CN115082888A (en) Lane line detection method and device
CN113177941B (en) Steel coil edge crack identification method, system, medium and terminal
CN115100099A (en) Point cloud data processing method, device, equipment and medium
CN111899277B (en) Moving object detection method and device, storage medium and electronic device
CN111813882B (en) Robot map construction method, device and storage medium
CN109325455B (en) Iris positioning and feature extraction method and system
CN113838076A (en) Method and device for labeling object contour in target image and storage medium
CN114913175B (en) Speckle image quality evaluation method, electronic device, and storage medium
An et al. On the information coupling and propagation of visual 3D perception in vehicular networks with position uncertainty
CN106210691A (en) For the method and apparatus generating anaglyph

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination