WO2020258894A1 - 车道线属性检测 - Google Patents
车道线属性检测 Download PDFInfo
- Publication number
- WO2020258894A1 WO2020258894A1 PCT/CN2020/076036 CN2020076036W WO2020258894A1 WO 2020258894 A1 WO2020258894 A1 WO 2020258894A1 CN 2020076036 W CN2020076036 W CN 2020076036W WO 2020258894 A1 WO2020258894 A1 WO 2020258894A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- attribute
- value
- lane line
- probability
- point
- Prior art date
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 claims abstract description 90
- 230000009021 linear effect Effects 0.000 claims abstract description 74
- 238000013528 artificial neural network Methods 0.000 claims description 82
- 238000012549 training Methods 0.000 claims description 49
- 238000012545 processing Methods 0.000 claims description 18
- 239000003086 colorant Substances 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 12
- 239000007787 solid Substances 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000004044 response Effects 0.000 claims 12
- 238000010586 diagram Methods 0.000 description 29
- 230000008569 process Effects 0.000 description 29
- 238000013527 convolutional neural network Methods 0.000 description 9
- 238000005070 sampling Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 238000010606 normalization Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 101150064138 MAP1 gene Proteins 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 101100400452 Caenorhabditis elegans map-2 gene Proteins 0.000 description 2
- 238000005206 flow analysis Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N7/00—Computing arrangements based on specific mathematical models
- G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30256—Lane; Road marking
Definitions
- the embodiments of the present disclosure relate to computer technology, and in particular to a method, device, electronic device, and smart device for detecting lane line attributes.
- Assisted driving and automatic driving are two important technologies in the field of intelligent driving. Through assisted driving or automatic driving, it can reduce the distance between workshops, reduce the occurrence of traffic accidents, and reduce the burden on the driver. Therefore, it plays an important role in the field of intelligent driving.
- lane line attribute detection is required. Through the lane line attribute detection, the type of lane line on the road can be identified, such as white solid lines and white dashed lines. Based on the detection results of lane line attributes, path planning, path deviation warning, and traffic flow analysis can be performed, and it can also provide reference for precise navigation.
- lane line attribute detection is of great significance to assisted driving and automatic driving. How to perform accurate and efficient lane line attribute detection is an important topic worthy of research.
- the embodiments of the present disclosure provide a technical solution for lane line attribute detection.
- the first aspect of the embodiments of the present disclosure provides a lane line attribute detection method, including:
- each color attribute probability map represents Each point in the road image belongs to the probability of the color corresponding to the color attribute probability map
- each line attribute probability map represents the probability that each point in the road image belongs to the line type corresponding to the line attribute probability map.
- the edge attribute probability map represents the probability that each point in the road image belongs to the edge corresponding to the edge attribute probability map; according to the probability map, the lane line attribute in the road image is determined.
- a second aspect of the embodiments of the present disclosure provides a lane line attribute detection device, including:
- the first acquisition module is used to acquire the road surface image collected by the image acquisition device installed on the smart device; the first determination module is used to determine the probability map according to the road surface image, and the probability map includes: color attribute probability map, line There are at least two types of attribute probability maps and edge attribute probability maps.
- N1 color attribute probability maps there are N1 color attribute probability maps, N2 linear attribute probability maps, and N3 edge attribute probability maps, where N1, N2, and N3 are all Is an integer greater than 0; each color attribute probability map represents the probability that each point in the road image belongs to the corresponding color of the color attribute probability map, and each linear attribute probability map represents that each point in the road image belongs to the The linear attribute probability map corresponds to the linear probability, and each edge attribute probability map represents the probability that each point in the road image belongs to the corresponding edge of the edge attribute probability map; the second determining module is used to determine according to the probability map The lane line attributes in the road image.
- a third aspect of the embodiments of the present disclosure provides an electronic device, including:
- the memory is used to store program instructions; the processor is used to call and execute the program instructions in the memory to execute the method steps described in the first aspect above.
- a fourth aspect of the embodiments of the present disclosure provides an intelligent driving method for use in a smart device, including:
- a fifth aspect of the embodiments of the present disclosure provides a smart device, including:
- the image acquisition device is used to acquire road images; the memory is used to store program instructions, and the stored program instructions are executed to implement the lane line attribute detection method described in the first aspect; the processor is used to acquire images according to the The road surface image acquired by the device executes the program instructions stored in the memory to detect the lane line attributes in the road surface image, and output prompt information or perform driving control on the smart device according to the detected lane line attributes.
- a sixth aspect of the embodiments of the present disclosure provides a non-volatile readable storage medium in which a computer program is stored, and the computer program is configured to execute the method steps described in the first aspect.
- the lane line attribute detection method, device, electronic device, and smart device provided in the embodiments of the present disclosure divide the lane line attribute into three dimensions: color, line type, and edge, and then can use the obtained road image points in these three dimensions. Based on at least two of the three attribute probability maps in three dimensions, the lane line attributes in the road image can be determined. Since the three attribute probability maps obtained in the above process are respectively for the lane line attributes of one dimension, it can be considered that the determination of various probability maps based on the road image is a single task detection, which reduces the complexity of the detection task. degree. Then, according to the detection results of each task, the attributes of lane lines in the road image are determined, that is, the various detection results must be combined to obtain the attributes of lane lines.
- the lane line attribute detection method uses different attributes to detect the attributes of lane lines separately, and the detection results are then merged. The method improves the accuracy and robustness of predicting lane line attributes. Therefore, when the above method is applied to a scene with higher complexity, more accurate lane line attribute detection results can be obtained.
- FIG. 1 is a schematic diagram of a scene of a lane line attribute detection method provided by an embodiment of the disclosure.
- FIG. 2 is a schematic flowchart of a method for detecting lane line attributes according to an embodiment of the disclosure.
- FIG. 3 is a schematic flowchart of a method for detecting lane line attributes according to another embodiment of the present disclosure.
- FIG. 4 is a schematic flowchart of a method for detecting lane line attributes according to still another embodiment of the present disclosure.
- FIG. 5 is a schematic flowchart of a neural network training method for detecting lane line attributes according to an embodiment of the disclosure.
- FIG. 6 is a schematic structural diagram of a convolutional neural network provided by an embodiment of the disclosure.
- FIG. 7 is a schematic diagram of a flow of road image processing performed by a neural network for detecting lane line attributes according to an embodiment of the disclosure.
- FIG. 8 is a module structure diagram of a lane line attribute detection device provided by an embodiment of the disclosure.
- FIG. 9 is a block diagram of a module structure of a lane line attribute detection device provided by another embodiment of the disclosure.
- FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the disclosure.
- FIG. 11 is a schematic structural diagram of a smart device provided by an embodiment of the disclosure.
- FIG. 12 is a schematic flowchart of an intelligent driving method provided by an embodiment of the disclosure.
- FIG. 1 is a schematic diagram of a scene of a lane line attribute detection method provided by an embodiment of the disclosure.
- this method can be applied to a vehicle 120 equipped with an image acquisition device 110.
- the image acquisition device 110 may be a device with a shooting function installed on the vehicle 120, for example, a camera, a driving recorder and other devices.
- the image of the road is collected by the image acquisition device on the vehicle, and the attributes of the lane line on the road where the vehicle is located are detected based on the method provided in the present disclosure, so that the obtained detection results can be applied to assisted driving or autonomous driving in. For example, route planning, route deviation warning, and traffic flow analysis.
- the lane line attribute detection method provided in the present disclosure is also applicable to smart devices that require road recognition such as robots or blind guide devices.
- FIG. 2 is a schematic flowchart of a method for detecting lane line attributes according to an embodiment of the disclosure. As shown in FIG. 2, the method includes steps S201-S203.
- the image acquisition device installed on the vehicle can collect the road surface image of the vehicle on the road in real time. Furthermore, through subsequent steps, continuously updated lane attribute detection results can be obtained continuously based on the road image collected by the image collection device.
- the aforementioned probability graph includes at least two of a color attribute probability graph, a linear attribute probability graph, and an edge attribute probability graph.
- each color attribute probability map corresponds to one color
- N1 color attribute probability maps correspond to N1 colors.
- N2 linear attribute probability graphs each linear attribute probability graph corresponds to one line type
- the N2 linear attribute probability graphs correspond to N2 types of line types.
- N3 edge attribute probability graphs each edge attribute probability graph corresponds to one type of edge
- the N3 edge attribute probability graphs correspond to N3 types of edges.
- each color attribute probability map represents the probability that each point in the above-mentioned road image belongs to that kind of color
- each linear attribute probability map represents the probability that each point in the above-mentioned road image belongs to this kind of line type
- each edge attribute The probability map represents the probability that each point in the above road image belongs to this kind of edge.
- N1, N2 and N3 are all integers greater than zero.
- the above probability map can be determined based on a neural network.
- the above road image is input to a neural network, and the neural network outputs the above probability map.
- the aforementioned neural network may include, but is not limited to, a convolutional neural network.
- the attributes of the lane line are divided into three dimensions of color, line type, and edge, and the probability of each point of the road image in at least two of the three dimensions is predicted through the neural network.
- the aforementioned N1 colors may include at least one of the following: white, yellow, and blue.
- the color dimension it can also include two results, no lane line and other colors, that is, no lane line and other colors are also used as one color.
- the lane-free line indicates that the pixel point of the road image does not belong to the lane line, and the other colors indicate that the color of the point of the road image is colors other than white, yellow, and blue.
- Table 1 is an example of the above-mentioned color types in the color dimension. As shown in Table 1, the color dimension can include 5 color types, and the value of N1 is 5.
- the above-mentioned N2 line types may include at least one of the following: dashed line, solid line, double dashed line, double solid line, dashed solid line, solid dashed line, triple dashed line, and dashed dashed line.
- dashed line solid line
- double dashed line double solid line
- dashed solid line solid dashed line
- solid dashed line triple dashed line
- dashed dashed line there can also be two types of results: no lane line and other line types, that is, no lane line and other line types are also used as a line type.
- the no lane line indicates that the point of the road image does not belong to the lane line
- the line type of the point of the other line type indicates the road image is a line type other than the above-mentioned line type.
- the above dashed solid line can be in the left to right direction, the first line is a dashed line, and the second is a solid line; correspondingly, the above solid dashed line can be in the left to right direction, and the first line is a solid line , The second line is a dotted line.
- Table 2 is an example of the line type in the above-mentioned line type dimension. As shown in Table 2, the line type dimension can include 10 types of line types, and the value of N2 is 10.
- the aforementioned N3 types of edges may include at least one of the following: curb-shaped edges, fence-shaped edges, wall or flowerbed-shaped edges, virtual edges, and non-edges.
- non-edge indicates that the point of the road image does not belong to the edge, but belongs to the lane line.
- no lane line and other edges that is, no lane line and other edges are also regarded as a kind of edge respectively.
- no lane line indicates that the point of the road image does not belong to the lane line or the edge
- the points of other edges indicate the road image belong to the edge type other than the above-mentioned edge type.
- Table 3 is an example of the aforementioned edge in the edge dimension. As shown in Table 3, the edge dimension can include 7 edge types, and the value of N3 is 7.
- the neural network can output 5 color attribute probability maps and 10 line types. Attribute probability map and 7 edge attribute probability maps.
- each color attribute probability map in the 5 color attribute probability maps represents the probability that each point in the road image belongs to one of the colors in Table 1 above
- the probability of each linear attribute in the 10 linear attribute probability maps The graph represents the probability that each point in the road image belongs to one of the line types in Table 2 above
- the probability graph of each edge attribute in the 7 edge attribute probability maps represents that each point in the road image belongs to one of the types in Table 3 above. Probability of the edge.
- the probability map 2 can identify the probability that each point in the road image is white.
- the road image is represented by a 200*200 matrix.
- a 200*200 matrix can be output.
- the value of each element in the matrix is the corresponding position on the road image.
- Probability of white For example, in the 200*200 matrix output by the neural network, the value in the first row and the first column is 0.4, which means that the probability that the point in the first row and the first column in the road image belongs to the white dashed lane type is 0.4.
- the matrix output by the neural network can be expressed in the form of a color attribute probability map.
- the color attribute probability map, the line attribute probability map, and the edge attribute probability map belong to three kinds of probability maps.
- the probability map can be used at the same time. Multiple probability plots in. For example, using the color attribute probability map, N1 color attribute probability maps can be used to determine the color attributes of the road image.
- the aforementioned probability map may be two of the aforementioned color attribute probability map, the aforementioned linear attribute probability map, and the aforementioned edge attribute probability map, that is, the aforementioned color attribute probability map and the aforementioned linear attribute probability map can be used. And two of the above edge attribute probability maps determine the lane line attributes in the road image.
- the number of lane line attributes is the number of combinations of the two probability maps used, and each lane line attribute is the two probabilities used A collection of each attribute in the figure.
- a lane line type attribute is a collection of a color attribute and a line type attribute, that is, a lane line attribute includes a color attribute and a line type attribute.
- a certain lane line attribute is a white dashed line, which is a collection of white and dashed lines.
- the aforementioned probability map may be the aforementioned color attribute probability map, the aforementioned linear attribute probability map, and the aforementioned edge attribute probability map. That is, the aforementioned color attribute probability map and the aforementioned linear attribute probability map can be used at the same time.
- the graph and the above-mentioned edge attribute probability graph determine the lane line attributes in the road image.
- the number of lane line attributes is the number of combinations of the number of attributes corresponding to the three probability maps used, and each lane line attribute is the three probabilities used A combination of each attribute in the figure.
- a lane line attribute is a combination of a color attribute, a line type attribute and an edge attribute, that is, a lane line attribute includes a color attribute, a line type attribute and an edge attribute.
- a lane line whose attribute is a white dashed line is a combination of white, dashed and non-edge.
- N1*N2*N3 refers to all the combinations that can be supported by the embodiments of the present disclosure. In the specific implementation process, some combinations may not appear in the actual use process.
- the lane line attributes are divided into three dimensions: color, line type, and edge, and then three attribute probability maps of each point of the road image in these three dimensions can be obtained, based on these three attribute probability maps At least two of them can determine the attributes of lane lines in the road image. Since the three attribute probability maps obtained in the above process are respectively for the lane line attributes of one dimension, it can be considered that the determination of various probability maps based on the road image is a single task detection, which reduces the complexity of the detection task. degree. Then determine the lane line attributes in the road image based on the results of each task detection, that is, combine various detection results to get the lane line attributes.
- the lane line attribute detection method uses different attributes to detect lane line attributes separately, and then combines the detection results. The method improves the accuracy and robustness in predicting the attributes of lane lines.
- the edge is used as an attribute dimension in the present disclosure, so that the present disclosure can not only accurately detect the type of lane in a structured pavement scene with lane markings, but at the same time, when lane markings are missing or no lanes are marked. In the scene of marking lines, for example, in the scene of driving on a rural road, various edge types and the like can also be accurately detected by the method of this embodiment.
- this embodiment specifically describes the process of using the probability map to determine the lane line attributes in the road image.
- two of the color attribute probability map, the line attribute probability map, and the edge attribute probability map can be used to determine the lane line attributes in the road image.
- the probability map obtained in the above step S203 includes a first attribute probability map and a second attribute probability map.
- the first attribute probability map and the second attribute probability map are color attribute probability maps, linear attribute probability maps, and edges. There are two types of attribute probability maps, and the first attribute probability map is different from the second attribute probability map.
- FIG. 3 is a schematic flowchart of a method for detecting attributes of lane lines according to another embodiment of the present disclosure. As shown in FIG. 3, when the probability map includes a first attribute probability map and a second attribute probability map, the above step S203 is based on the probability The process of determining the attributes of lane lines in the road image includes the following steps.
- the road image can be preprocessed to obtain lane lines in the road image.
- a road image can be input to a trained neural network, and the neural network outputs the result of lane lines in the road image.
- a road image can be input to a trained semantic segmentation network, and the semantic segmentation network outputs the lane line segmentation result in the road image. Then use the method shown in Figure 3 to process the attributes of the lane lines and calculate the attributes of the lane lines. Thereby improving the accuracy of lane line recognition.
- the above steps S301-S303 can determine the value of the first attribute of a lane line in the road image.
- the first attribute is an attribute corresponding to the first attribute probability map.
- the first attribute probability map is a color attribute probability map
- the first attribute is a color attribute
- the value of the first attribute can be white, yellow, Blue, other colors, etc.
- the neural network can output L first attribute probability maps. For a point in a lane line in the road image, at each The first attribute probability map has a corresponding probability value. The greater the probability value, the greater the probability that the point belongs to the corresponding attribute of the probability map. Therefore, for this point, you can compare the L first attribute probability maps For the probability value of the corresponding position, the value of the first attribute corresponding to the first attribute probability map with the largest probability value is used as the value of the first attribute of the point.
- the first attribute probability map is a color attribute probability map
- the first attribute is a color attribute
- L is 5, that is, it includes 5 color attribute probability maps, which are respectively the probability map 0 and the probability map shown in Table 4 above.
- Probability graph 2 probability graph 3 and probability graph 4.
- Each probability graph corresponds to a color attribute. Assuming that a point in a lane line in a road image has the largest probability value in the probability map 1, it can be determined that the color attribute value of the point is the color attribute corresponding to the probability map 1.
- the value of the first attribute of each point at the position of a lane line in the road image can be obtained, and on this basis, the value of the first attribute of the lane line can be determined according to the value of the first attribute of each point.
- the value of the first attribute of each point at the position of the lane line may be , As the value of the first attribute of the lane line.
- the first attribute is a color attribute
- the number of points whose value of the first attribute is white accounts for 80% of the total number of points
- the value of the first attribute is yellow
- white can be used as the first attribute of the lane line Value, that is, the value of the color attribute.
- the value of the first attribute of each point at the position of the lane line may be used as the value of the first attribute of the lane line.
- the first attribute is a color attribute
- the value of the first attribute of all points on the lane line position is yellow
- yellow can be used as the value of the first attribute of the lane line, that is, the value of the color attribute value.
- S306 Determine the value of the second attribute of the lane line according to the value of the second attribute of each point at the position of the lane line in the road image.
- the above steps S304-S306 can determine the value of the second attribute of a lane line in the road image.
- the second attribute is the attribute corresponding to the second attribute probability map.
- the second attribute probability map is a linear attribute probability map
- the second attribute is a linear attribute
- the value of the second attribute can be a solid line , Dashed line, double solid line, double dashed line, etc.
- the neural network can output S second attribute probability maps. For a point in a lane line in the road image, at each The second attribute probability map has a corresponding probability value. The larger the probability value, the greater the probability that the point belongs to the corresponding attribute of the probability map. Therefore, for this point, you can compare the S second attribute probability maps For the probability value of the corresponding position, the value of the second attribute corresponding to the second attribute probability map with the largest probability value is used as the value of the second attribute of the point.
- the second attribute probability map is a linear attribute probability map
- the second attribute is a linear attribute
- S is 10, that is, 10 linear attribute probability maps are included, and each probability map corresponds to a linear attribute.
- a point in a lane line in the road image has the largest probability value in the first linear attribute probability map
- it can be determined that the line attribute value of the point is the line attribute corresponding to the first linear attribute probability map .
- the value of the second attribute of each point at the position of a lane line in the road image can be obtained.
- the value of the second attribute of the lane line can be determined according to the value of the second attribute of each point.
- the value of the second attribute of each point at the position of the lane line is different, the value of the second attribute of the point with the largest number of points with the same second attribute value at the position of the lane line can be , As the value of the second attribute of the lane line.
- the second attribute is a line attribute
- the number of points whose value of the second attribute is a solid line accounts for 81% of the total number of points
- the value of the second attribute The number of dots that are dotted lines account for 15% of the total number of points, and the number of points with the second attribute value of other line types account for 4% of the total number of points, then the solid line can be used as the lane line
- the value of the second attribute is the value of the line attribute.
- the value of the second attribute of each point at the position of the lane line may be used as the value of the second attribute of the lane line.
- the solid line can be used as the value of the second attribute of the lane line, namely The value of the line type attribute.
- the first attribute probability map is a color attribute probability map
- L is equal to N1
- the first attribute is a color attribute.
- L is equal to N2
- the first attribute is a linear attribute.
- the first attribute probability map is an edge attribute probability map
- L is equal to N3
- the first attribute is an edge attribute.
- S is equal to N1
- the second attribute is a color attribute probability map
- S is equal to N2
- the second attribute is a linear attribute probability map
- S is equal to N2
- the second attribute is a linear attribute probability map
- S is equal to N3
- the second attribute is an edge attribute.
- the second attribute probability map when the first attribute probability map is a color attribute probability map, can be a linear attribute probability map or an edge attribute probability map Figure; when the first attribute probability diagram is a linear attribute probability diagram, the second attribute probability diagram can be a color attribute probability diagram or an edge attribute probability diagram; when the first attribute probability diagram is an edge attribute probability diagram, the second attribute probability diagram
- the graph can be a color attribute probability graph or a linear attribute probability graph.
- the value of the first attribute and the value of the second attribute of a lane line can be combined, so that the value of the combined attribute can be used as the The value of the attribute of the lane line.
- the method of combination processing may be, for example, superimposing the value of the second attribute after the value of the first attribute, or superimposing the value of the first attribute after the value of the second attribute.
- the first attribute is a color attribute and the second attribute is a line attribute
- the value of the first attribute of a certain lane line in the road image is white
- the value of the second attribute is a solid line.
- a "white solid line” can be obtained, and the "white solid line” is the attribute value of the lane line.
- the color attribute probability map, the line attribute probability map, and the edge attribute probability map can be used simultaneously to determine the lane line attributes in the road image.
- the probability map obtained in step S203 above includes the third attribute probability map in addition to the aforementioned first attribute probability map and the second attribute probability map.
- the third attribute probability map is one of a color attribute probability map, a linear attribute probability map, and an edge attribute probability map, and the third attribute probability map, the second attribute probability map and the first attribute probability map are Probability graph with two different attributes.
- FIG. 4 is a schematic flow chart of a method for detecting lane line attributes according to another embodiment of the present disclosure.
- the above-mentioned probability map includes both a first attribute probability map and a second attribute probability map, as well as a third attribute probability map.
- the following steps may also be performed.
- S403 Determine the value of the third attribute of the lane line according to the value of the third attribute of each point at the position of the lane line in the road image.
- the above steps S401-S403 can determine the value of the third attribute of a lane line in the road image.
- the third attribute is the attribute corresponding to the third attribute probability map.
- the third attribute probability map is an edge attribute probability map
- the third attribute is an edge attribute
- the value of the third attribute can be a curb edge , Fence edge, virtual edge, etc.
- the neural network can output U third attribute probability maps. For a point in a lane line in the road image, at each The third attribute probability map has a corresponding probability value. The larger the probability value, the greater the probability that the point belongs to the corresponding attribute of the probability map. Therefore, for this point, you can compare U third attribute probability maps For the probability value of the corresponding position, the value of the third attribute corresponding to the third attribute probability map with the largest probability value is used as the value of the third attribute of the point.
- the third attribute probability graph is an edge attribute probability graph
- the third attribute is an edge attribute
- U is 7, that is, it includes 7 edge attribute probability graphs
- each probability graph corresponds to an edge attribute.
- a point in a lane line in the road image has the largest probability value in the 7th edge attribute probability map
- it can be determined that the edge attribute value of this point is the edge attribute corresponding to the 7th edge attribute probability map.
- the value of the third attribute of each point at the position of a lane line in the road image can be obtained.
- the value of the third attribute of the lane line can be determined according to the value of the third attribute of each point.
- the value of the third attribute of each point at the position of the lane line is different, the value of the third attribute of the point with the largest number of points with the same value of the third attribute at the position of the lane line can be , As the value of the third attribute of the lane line.
- the value of the third attribute is 82% of the total number of points on the curb edge.
- the number of points whose value is a virtual edge accounts for 14% of the total number of points, and the number of non-edge points whose value is the third attribute accounts for 4% of the total number of points, then the curb edge can be used as the The value of the third attribute of the lane line, that is, the value of the edge attribute.
- the value of the third attribute of each point at the position of the lane line may be used as the value of the third attribute of the lane line.
- the curb edge can be used as the third attribute of the lane line Value, that is, the value of the edge attribute.
- the third attribute probability map is a color attribute probability map
- U is equal to N1
- the third attribute is a color attribute.
- U is equal to N2
- the third attribute is a linear attribute.
- U is equal to N3
- the third attribute is an edge attribute.
- the value of the first attribute and the value of the second attribute of a lane line are performed in step S307.
- the value of the first attribute of the lane line, the value of the second attribute of the lane line, and the value of the third attribute of the lane line can be combined.
- the method of combined processing may be, for example, superimposing the value of the third attribute on the value of the second attribute and the value of the first attribute, or superimposing the value of the third attribute on the value of the second attribute and the value of the first attribute. Before the value of the attribute.
- the first attribute is a color attribute
- the second attribute is a line attribute
- the third attribute is an edge attribute.
- the value of the first attribute of a certain lane line in the road image is obtained through the aforementioned method, and the second attribute is white.
- the value of the attribute is a solid line
- the value of the third attribute is non-edge.
- the probability map can be obtained through a neural network.
- the road image can be input into the neural network, and the neural network can output the above-mentioned probability map.
- the following embodiments illustrate the training and use process of the neural network involved in the above embodiments.
- the above-mentioned neural network may preliminarily use a road surface training image set including color type, line type, and edge type label information for supervised training.
- the road training image set includes a large number of training images.
- Each training image is obtained through the process of collecting actual road images and labeling. In an example, you can first collect multiple actual road images in various scenes such as day, night, rain, tunnel, straight road, curve, strong light, etc., and then perform pixel-level labeling for each actual road image. That is, the category of each pixel in the actual road image is labeled as color type, line type, and edge type label information, thereby obtaining a training image set.
- the neural network after training can be used not only in some simple scenes, such as daytime scenes with good weather and lighting conditions. , Obtain accurate lane line attribute detection results. In scenes with high complexity, such as rainy weather, night, tunnel, curve, strong light, etc., accurate lane line attribute detection results can also be obtained.
- the training image set involved in the above process covers various scenes in practice. Therefore, the neural network trained using the training image set has good robustness for lane line attribute detection in various scenarios, and the detection The time is short and the accuracy of the detection result is high.
- the neural network can be trained according to the following process.
- FIG. 5 is a schematic flowchart of a method for training a neural network for lane line attribute detection according to an embodiment of the disclosure. As shown in FIG. 5, the training process of the above-mentioned neural network may include the following steps.
- the neural network processes the input training image, and outputs the predicted color attribute probability map, the predicted linear attribute probability map, and the predicted edge attribute probability map of the training image.
- the training image is included in the road surface training image set.
- the predicted color attribute probability map, the predicted line attribute probability map, and the predicted edge attribute probability map are the predicted color attribute probability map, the predicted line attribute probability map, and the predicted edge attribute probability map actually output by the neural network.
- S502 For each point at the position of a lane line in the training image, determine the value of the color attribute, the value of the line type, and the value of the edge attribute of the point.
- S503 Determine the predicted color type, predicted line type and value of the lane line according to the color attribute value, line type attribute value, and edge attribute value of each point at the lane line position in the above-mentioned training image. Predict the edge type.
- the predicted color type refers to the value of the color attribute of the lane line obtained from the probability map output by the neural network
- the predicted line type refers to the value of the line type attribute of the lane line obtained from the probability map output by the neural network.
- the predicted edge type It refers to the value of the edge attribute of the lane line obtained from the probability map output by the neural network.
- the color, line type, and edge dimensions may be processed separately to determine the predicted color type, predicted line type, and predicted edge type of a lane line in the training image.
- the neural network determines the line type attribute or edge attribute of each point at the position of a lane line, it judges whether the lane line is a dashed line or a solid line through the entire road image, and then gives a dotted line or a solid line on the lane line.
- the probability of the line this is because each pixel in the feature map extracted from the road image by the neural network aggregates the information of a large area in the road image, so it can determine the line type of the lane line or the type of the edge .
- the color type truth map expresses the color of the training image by means of logical algebra, and the color type truth map is obtained based on the label information of the color type of the training image.
- the linear truth map expresses the line type of the training image by means of logical algebra, and the linear truth map is obtained based on the label information of the line type of the training image.
- the edge type truth map expresses the edge type of the training image by means of logical algebra, and the edge type truth map is obtained based on the label information of the edge type of the training image.
- the loss function can be used to calculate the first loss value between the predicted color type and the color type of the color type truth map, and the second loss between the predicted line type and the line type of the line truth map Value, and the third loss value between the predicted edge type and the edge type in the edge type truth map.
- S505 Adjust parameter values of the neural network according to the first loss value, the second loss value, and the third loss value.
- the network parameters of the neural network may include the size of the convolution kernel, weight information, and so on.
- the above-mentioned loss value can be back-propagated in the neural network by way of gradient back propagation, so as to adjust the network parameter value of the neural network.
- the above steps S501-S504 can be continued until the first loss value is within the preset loss range, and the second loss value is within the preset loss range, and the first loss value is within the preset loss range.
- the third loss value is within the preset loss range.
- the parameter value of the neural network is obtained as the optimized parameter value, and the training of the neural network ends.
- the above steps S501-S504 can be continued until the sum of the first loss value, the second loss value, and the third loss value is within another preset loss range ,
- the parameter value of the neural network is obtained at this time the optimized parameter value, and the training of the neural network ends.
- one training image can be used to train the neural network at a time, or multiple training images can also be used to train the neural network at a time
- the aforementioned neural network may be a convolutional neural network
- the convolutional neural network may include a convolutional layer, a residual network unit, an up-sampling layer, and a normalization layer.
- the order of the convolutional layer and the residual network unit can be flexibly set as required, and the number of each layer can also be flexibly set as required.
- the aforementioned convolutional neural network may include connected 6-10 convolutional layers, connected 7-12 residual network units, and 1-4 upsampling layers.
- the convolutional neural network with this specific structure is used for lane line attribute detection, it can meet the requirements of multi-scene or complex scene lane line attribute detection, so that the detection result is more robust.
- the above convolutional neural network may include 8 connected convolutional layers, 9 connected residual network units, and 2 connected upsampling layers.
- Fig. 6 is a schematic structural diagram of a convolutional neural network provided by an embodiment of the present disclosure. As shown in Fig. 6, after a road image is input 610, it first passes through 8 consecutive convolutional layers 620 of the convolutional neural network. After the 8 convolutional layers 620, it includes 9 consecutive residual network units 630. After the consecutive 9 residual network units 630, it includes 2 consecutive up-sampling layers 640, and in the consecutive 2 up-sampling layers After 640, it is the normalization layer 650, and the normalization layer 650 finally outputs the probability map.
- each residual network unit 630 may include 256 filters, and each layer may include 128 3*3 and 128 1*1 filters.
- the output can be output according to the following process.
- FIG. 7 is a schematic diagram of a flow chart of road image processing performed by a neural network for detecting lane line attributes according to an embodiment of the disclosure. As shown in FIG. 7, the process of obtaining the above-mentioned probability map through the neural network is:
- S701 Extract low-level feature information of M channels of the road image through at least one convolutional layer of the neural network.
- M is the number of probability maps obtained in step S202.
- M is the sum of N1 and N2.
- M is the sum of N1, N2, and N3.
- the convolutional layer can reduce the resolution of the road image and retain the low-level features of the road image.
- the low-level feature information of the road image may include edge information, straight line information, and curve information in the image.
- each of the M channels of the above road image corresponds to a color attribute, or a linear attribute or a kind of Edge attributes.
- S702 Extract high-level feature information of the M channels of the road image based on the low-level feature information of the M channels by using at least one residual network unit of the neural network.
- the high-level feature information of the M channels of the road image extracted by the residual network unit includes semantic features, contours, and overall structure.
- S703 Perform up-sampling processing on the high-level feature information of the M channels through at least one upsampling layer of the neural network to obtain M probability maps that are as large as the road image.
- the image can be restored to the original size of the image input to the neural network.
- M probability maps that are as large as the road image input to the neural network can be obtained.
- the low-level feature information and the high-level feature information described in the embodiments of the present disclosure are relative concepts under a specific neural network.
- the features extracted by the shallower network layers are relative to the features extracted by the deeper network layers.
- the former extracts belong to low-level feature information, while the latter extracts belong to high-level feature information. .
- the neural network may further include a normalization layer after the above-mentioned up-sampling layer, and the above-mentioned M probability maps are output through the normalization layer.
- the feature map of the road image is obtained after upsampling, and the value of each pixel in the feature map is normalized so that the value of each pixel in the feature map is in the range of 0 to 1. , Thereby obtaining M probability maps.
- a normalization method is: first determine the maximum value of the pixel point in the feature map, and then divide the value of each pixel point by the maximum value, so that the value of each pixel point in the feature map is In the range of 0 to 1.
- the road surface image before the road surface image is input to the neural network in step S202, the road surface image may be subjected to distortion processing first to further improve the accuracy of the neural network output result.
- FIG. 8 is a module structure diagram of a lane line attribute detection device provided by an embodiment of the disclosure. As shown in FIG. 8, the device includes: a first acquisition module 801, a first determination module 802, and a second determination module 803.
- the first acquisition module 801 is used to acquire road images collected by an image acquisition device installed on a smart device.
- the first determining module 802 is configured to determine a probability map according to the road image, and the probability map includes at least two of: a color attribute probability map, a line attribute probability map, and an edge attribute probability map, wherein the color attribute probability map N1, linear attribute probability maps are N2, edge attribute probability maps are N3, where N1, N2, and N3 are integers greater than 0; each color attribute probability map represents each point in the road image The probability of belonging to this kind of color, each linear attribute probability map represents the probability that each point in the road image belongs to this kind of line type, and each edge attribute probability map represents that each point in the road image belongs to this kind of edge The probability.
- the second determining module 803 is configured to determine the lane line attributes in the road image according to the probability map.
- the colors corresponding to the N1 color attribute probability maps include at least one of the following: white, yellow, and blue.
- the line type corresponding to the N2 line type attribute probability maps includes at least one of the following: a dashed line, a solid line, a double dashed line, a double solid line, a dashed solid line, a solid dashed line, a triple dashed line, and a dashed line.
- edges corresponding to the N3 edge attribute probability graphs include at least one of the following: curb-shaped edges, fence-shaped edges, wall or flowerbed-shaped edges, virtual edges, and non-edges.
- the probability diagram includes a first attribute probability diagram and a second attribute probability diagram
- the first attribute probability diagram and the second attribute probability diagram are color attribute probability diagrams, linear attribute probability diagrams, and
- the second determining module 803 is specifically configured to: for each point at a lane line position in the road image, determine the respective probability value of the point in the corresponding position in the L first attribute probability maps; For this point, use the value of the first attribute corresponding to the first attribute probability map with the largest probability value as the value of the first attribute of the point; according to the first attribute of each point at the position of the lane line in the road image The value of the attribute determines the value of the first attribute of the lane line; for each point at the position of the lane line in the road image, determine the corresponding point in the S second attribute probability maps The respective probability value of the position; for this point, the value of the second attribute corresponding to the second attribute probability map with the largest probability value is used as the value of the second attribute of the point; according to the lane line in the road image The value of the second attribute of each point at the position determines the value of the second attribute of the lane line; the value of the first attribute of the lane line and the value of the second attribute of the lane line are
- the second determining module 803 determines the value of the first attribute of the lane line according to the first attribute of each point at the position of the lane line in the road image, including: responding to the line The value of the first attribute of each point at the position of the lane line is different, and the value of the first attribute of the point with the largest number of points with the same value of the first attribute at the position of the lane line is taken as the value of the first attribute of the lane line The value of the first attribute.
- the second determining module 803 determines the value of the first attribute of the lane line according to the first attribute of each point at the position of the lane line in the road image, including: responding to the line The value of the first attribute of each point at the lane line position is the same, and the value of the first attribute of the point at the lane line position is taken as the value of the first attribute of the lane line.
- the second determining module 803 determines the value of the second attribute of the lane line according to the value of the second attribute of each point at the position of the lane line in the road image, including: responding to The value of the second attribute of each point at the lane line position is different, and the second attribute value of the point with the largest number of points with the same second attribute value at the lane line position is taken as the lane line The value of the second attribute of the line.
- the second determining module 803 determines the value of the second attribute of the lane line according to the value of the second attribute of each point at the position of the lane line in the road image, including: responding to The value of the second attribute of each point at the position of the lane line is the same, and the value of the second attribute of the point at the position of the lane line is taken as the value of the second attribute of the lane line.
- the probability map further includes a third attribute probability map, the third attribute probability map being one of a color attribute probability map, a line attribute probability map, and an edge attribute probability map, and the first attribute probability map
- the third attribute probability map being one of a color attribute probability map, a line attribute probability map, and an edge attribute probability map
- the first attribute probability map The three-attribute probability map, the second attribute probability map, and the first attribute probability map are probability maps with two different attributes.
- the second determining module 803 is further configured to: before combining the value of the first attribute of the lane line with the value of the second attribute of the lane line, perform a reference to the lane in the road image For each point at the line position, determine the respective probability value of the point in the corresponding position in the U third attribute probability maps; for this point, the third attribute corresponding to the third attribute probability map with the largest probability value The value of is used as the value of the third attribute of the point; the value of the third attribute of the lane line is determined according to the value of the third attribute of each point at the position of the lane line in the road image; where, When the third attribute probability map is a color attribute probability map, U is equal to N1, and the third attribute is a color attribute; when the third attribute probability map is a linear attribute probability map, U is equal to N2, and the third attribute is a linear attribute; When the third attribute probability map is an edge attribute probability map, U is equal to N3, and the third attribute is an edge attribute; the second determination module 803 is the value of the
- the second determining module 803 determines the value of the third attribute of the lane line according to the value of the third attribute of each point at the position of the lane line in the road image, including: responding to The value of the third attribute of each point at the lane line position is different, and the third attribute value of the point with the largest number of points with the same third attribute value at the lane line position is taken as the lane line The value of the third attribute of the line.
- the second determining module 803 determines the value of the third attribute of the lane line according to the value of the third attribute of each point at the position of the lane line in the road image, including: responding to The value of the third attribute of each point at the position of the lane line is the same, and the value of the third attribute of the point at the position of the lane line is taken as the value of the third attribute of the lane line.
- the first determining module 802 is specifically configured to: input the road image into a neural network, and the neural network outputs the probability map; wherein, the neural network adopts a color type, a line type, and an edge.
- the road training image set of type label information is supervised and trained.
- FIG. 9 is a module structure diagram of a lane line attribute detection device provided by an embodiment of the present disclosure. As shown in FIG. 9, it further includes: a preprocessing module 804 for de-distorting the road image.
- the division of the various modules of the above device is only a division of logical functions, and may be fully or partially integrated into a physical entity in actual implementation, or may be physically separated.
- these modules can all be implemented in the form of software called by processing elements; they can also be implemented in the form of hardware; some modules can be implemented in the form of calling software by processing elements, and some of the modules can be implemented in the form of hardware.
- the determining module may be a separately established processing element, or it may be integrated into a certain chip of the above-mentioned device for implementation.
- each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or instructions in the form of software.
- the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuit (ASIC), or one or more microprocessors (digital signal processor, DSP), or, one or more field programmable gate arrays (FPGA), etc.
- ASIC application specific integrated circuit
- DSP digital signal processor
- FPGA field programmable gate arrays
- the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program codes.
- CPU central processing unit
- these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).
- SOC system-on-a-chip
- the computer program product includes one or more computer instructions.
- the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
- the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
- the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
- the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
- FIG. 10 is a schematic structural diagram of an electronic device provided by an embodiment of the disclosure.
- the electronic device 1000 may include a processor 1001, a memory 1002, a communication interface 1003, and a system bus 1004.
- the memory 1002 and the communication interface 1003 are connected to the processor 1001 through the system bus 1004. And complete mutual communication, the memory 1002 is used to store computer execution instructions, the communication interface 1003 is used to communicate with other devices, and the processor 1001 executes the computer program to implement the provision of the embodiment of the present disclosure The lane line attribute detection method.
- the system bus mentioned in FIG. 10 may be a peripheral component interconnect standard (PCI) bus or an extended industry standard architecture (EISA) bus, etc.
- PCI peripheral component interconnect standard
- EISA extended industry standard architecture
- the system bus can be divided into address bus, data bus, control bus, etc.
- the communication interface is used to realize the communication between the database access device and other devices (such as client, read-write library and read-only library).
- the memory may include random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
- the above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other Programming logic devices, discrete gates or transistor logic devices, discrete hardware components.
- a general-purpose processor including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other Programming logic devices, discrete gates or transistor logic devices, discrete hardware components.
- FIG. 11 is a schematic structural diagram of a smart device provided by an embodiment of the disclosure. As shown in FIG. 11, the smart device 1100 of this embodiment includes: an image acquisition device 1101, a processor 1102, and a memory 1103.
- the image acquisition device 1101 takes a road image and sends the road image to the processor 1102.
- the processor 1102 calls the memory 1103 and executes the program instructions in the memory 1103 to detect the acquired
- the attributes of the lane lines in the road image are outputted according to the detected attributes of the lane lines, and prompt information is output or driving control of the smart device is performed.
- the smart device in this embodiment is a smart device capable of driving on the road, such as a smart driving vehicle, a robot, a blind guide device, etc.
- the smart driving vehicle may be an autonomous driving vehicle or a vehicle with a driving assistance system.
- the prompt information may include lane line departure warning prompt, lane line keeping prompt, change of driving speed, change of driving direction, lane line maintenance, change of vehicle light state, etc.
- the above-mentioned driving control may include: braking, changing the driving speed, changing the driving direction, maintaining the lane line, changing the state of the lights, switching the driving mode, etc., wherein the driving mode switching may be the switching between assisted driving and automatic driving, for example, Switch assisted driving to automatic driving.
- FIG. 12 is a schematic flowchart of the intelligent driving method provided by the embodiments of the present disclosure.
- the embodiments of the present disclosure also provide an intelligent driving method, which is used in the smart device described in FIG. 11, as shown in FIG. As shown, the method includes:
- S1203 Output prompt information or perform driving control on the smart device according to the detected attributes of the lane line.
- the execution subject of this embodiment is a mobile smart device, such as a smart driving vehicle, a robot, a blind guide device, etc., where the smart driving vehicle may be an autonomous driving vehicle or a vehicle with an assisted driving system.
- the intelligent driving in this embodiment includes assisted driving, automatic driving, and/or driving mode switching between assisted driving and automatic driving.
- the lane line attribute detection result of the road image is obtained by the lane line attribute detection method of the foregoing embodiment, and the specific process is referred to the description of the foregoing embodiment, and will not be repeated here.
- the smart device executes the aforementioned lane line attribute detection method, obtains the lane line attribute detection result of the road image, and outputs prompt information and/or performs movement control according to the lane line attribute detection result of the road image.
- the prompt information may include lane line departure warning prompts, lane line maintenance prompts, change of driving speed, change of driving direction, lane line maintenance, change of vehicle light status, etc.
- the above-mentioned driving control may include: braking, changing the driving speed, changing the driving direction, and maintaining the lane line.
- the smart device obtains the lane line attribute detection result of the road image, and outputs prompt information or performs driving control of the smart device according to the lane line attribute detection result of the road image, thereby improving the safety of the smart device Sex and reliability.
- an embodiment of the present disclosure further provides a non-volatile storage medium that stores instructions in the storage medium, which when run on a computer, causes the computer to execute the lane line attribute detection method provided by the embodiment of the present disclosure .
- an embodiment of the present disclosure further provides a chip for executing instructions, and the chip is configured to execute the lane line attribute detection method provided by the embodiment of the present disclosure.
- the embodiments of the present disclosure further provide a program product, the program product includes a computer program, the computer program is stored in a storage medium, at least one processor can read the computer program from the storage medium, and the at least one When the processor executes the computer program, the lane line attribute detection method provided by the embodiment of the present disclosure can be implemented.
- At least one refers to one or more, and “multiple” refers to two or more.
- “And/or” describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, both A and B exist, and B exists alone, where A, B can be singular or plural.
- the character “/” generally indicates that the associated objects before and after are in an “or” relationship; in the formula, the character “/” indicates that the associated objects before and after are in a “division” relationship.
- “The following at least one item (a)” or similar expressions refers to any combination of these items, including any combination of a single item (a) or plural items (a).
- at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple A.
- the size of the sequence numbers of the foregoing processes does not mean the order of execution.
- the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present disclosure.
- the implementation process constitutes any limitation.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Mathematical Optimization (AREA)
- Algebra (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Probability & Statistics with Applications (AREA)
- Pure & Applied Mathematics (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
- Geometry (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
Abstract
Description
类型编号 | 0 | 1 | 2 | 3 | 4 |
类型名称 | 无车道线 | 其他颜色 | 白色 | 黄色 | 蓝色 |
颜色属性概率图 | 概率图0 | 概率图1 | 概率图2 | 概率图3 | 概率图4 |
颜色类型 | 无车道线 | 其他颜色 | 白色 | 黄色 | 蓝色 |
Claims (32)
- 一种车道线属性检测方法,其特征在于,包括:获取智能设备上安装的图像采集装置所采集的路面图像;根据所述路面图像,确定概率图,所述概率图包括:颜色属性概率图、线型属性概率图和边沿属性概率图中的至少两种,其中,所述颜色属性概率图为N1个,所述线型属性概率图为N2个,所述边沿属性概率图为N3个,其中,N1、N2和N3均为大于0的整数;每个所述颜色属性概率图表示所述路面图像中的各个点属于该颜色属性概率图对应颜色的概率,每个所述线型属性概率图表示所述路面图像中的各个点属于该线型属性概率图对应线型的概率,每个所述边沿属性概率图表示所述路面图像中的各个点属于该边沿属性概率图对应边沿的概率;根据所述概率图,确定所述路面图像中的车道线属性。
- 根据权利要求1所述的方法,其特征在于,所述N1个颜色属性概率图对应的颜色包括以下至少之一:白色、黄色、蓝色。
- 根据权利要求1或2所述的方法,其特征在于,所述N2个线型属性概率图对应的线型包括以下至少之一:虚线、实线、双虚线、双实线、虚实线、实虚线、三虚线、虚实虚线。
- 根据权利要求1-3任一项所述的方法,其特征在于,所述N3个边沿属性概率图对应的边沿包括以下至少之一:路牙型边沿、栅栏型边沿、墙或花坛型边沿、虚拟边沿、非边沿。
- 根据权利要求1-4任一所述的方法,其特征在于,所述概率图包括第一属性概率图和第二属性概率图,所述第一属性概率图和所述第二属性概率图为所述颜色属性概率图、所述线型属性概率图和所述边沿属性概率图中的两种,且所述第一属性概率图和所述第二属性概率图不同;根据所述概率图确定所述路面图像中的所述车道线属性,包括:针对所述路面图像中的一条车道线位置处的每个点,确定该点在L个所述第一属性概率图中的对应位置的各自的概率值;针对该点,将概率值最大的第一属性概率图对应的第一属性的值,作为该点的第一属性的值;根据所述路面图像中的该条车道线位置处的各个点的第一属性的值,确定该条车道线的第一属性的值;针对所述路面图像中的该条车道线位置处的每个点,确定该点在S个所述第二属性概率图中的对应位置的各自概率值;针对该点,将概率值最大的第二属性概率图对应的第二属性的值,作为该点的第二属性的值;根据所述路面图像中的该条车道线位置处的各个点的第二属性的值,确定该条车道线的第二属性的值;将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合;将组合后的属性的值作为该条车道线的属性的值;其中,当所述第一属性概率图为所述颜色属性概率图时,L等于N1,所述第一属性为颜色属性;当所述第一属性概率图为所述线型属性概率图时,L等于N2,所述第一属性为线型属性;当所述第一属性概率图为所述边沿属性概率图时,L等于N3,第一属性为边沿属性;当所述第二属性概率图为所述颜色属性概率图时,S等于N1,所述第二属性为所述颜色属性;当所述第二属性概率 图为所述线型属性概率图时,S等于N2,所述第二属性为所述线型属性;当所述第二属性概率图为所述边沿属性概率图时,S等于N3,所述第二属性为所述边沿属性。
- 根据权利要求5所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第一属性,确定该条车道线的所述第一属性的值,包括:响应于该条车道线位置处的各个点的所述第一属性的值不同,将该条车道线位置处的、第一属性的值相同的点的数量最多的点的所述第一属性的值,作为该条车道线的所述第一属性的值。
- 根据权利要求5或6所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第一属性,确定该条车道线的所述第一属性的值,包括:响应于该条车道线位置处的各个点的所述第一属性的值相同,将该条车道线位置处的点的所述第一属性的值作为该条车道线的所述第一属性的值。
- 根据权利要求5-7任一所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第二属性的值,确定该条车道线的所述第二属性的值,包括:响应于该条车道线位置处的各个点的所述第二属性的值不同,将该条车道线位置处的、第二属性的值相同的点的数量最多的点的所述第二属性的值,作为该条车道线的所述第二属性的值。
- 根据权利要求5-8任一所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第二属性的值,确定该条车道线的所述第二属性的值,包括:响应于该条车道线位置处的各个点的所述第二属性的值相同,将该条车道线位置处的点的所述第二属性的值作为该条车道线的所述第二属性的值。
- 根据权利要求5-9任一所述的方法,其特征在于,所述概率图还包括第三属性概率图,所述第三属性概率图为所述颜色属性概率图、所述线型属性概率图和所述边沿属性概率图中的一种,且所述第三属性概率图、所述第二属性概率图与所述第一属性概率图为属性两两不同的概率图;将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合之前,所述方法还包括:针对所述路面图像中的该条车道线位置处的每个点,确定该点在U个所述第三属性概率图中的对应位置的各自的概率值;针对该点,将概率值最大的第三属性概率图对应的第三属性的值,作为该点的第三属性的值;根据所述路面图像中的该条车道线位置处的各个点的第三属性的值,确定该条车道线的第三属性的值;其中,当所述第三属性概率图为所述颜色属性概率图时,U等于N1,所述第三属性为所述颜色属性;当所述第三属性概率图为所述线型属性概率图时,U等于N2,所述第三属性为所述线型属性;当所述第三属性概率图为所述边沿属性概率图时,U等于N3,所述第三属性为所述边沿属性;将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合,包括:将该条车道线的所述第一属性的值、该条车道线的所述第二属性的值以及该条车道线的所述第三属性的值进行组合。
- 根据权利要求10所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第三属性的值,确定该条车道线的所述第三属性的值,包括:响应于该条车道线位置处的各个点的所述第三属性的值不同,将该条车道线位置处的、第三属性的值相同的点的数量最大的点的所述第三属性的值,作为该条车道线的所述第三属性的值。
- 根据权利要求10或11所述的方法,其特征在于,根据所述路面图像中的该条车道线位置处的各个点的所述第三属性的值,确定该条车道线的所述第三属性的值,包括:响应于该条车道线位置处的各个点的所述第三属性的值相同,将该条车道线位置处的点的所述第三属性的值作为该条车道线的所述第三属性的值。
- 根据权利要求1-12任一项所述的方法,其特征在于,根据所述路面图像确定所述概率图,包括:将所述路面图像输入神经网络,所述神经网络输出所述概率图;其中,所述神经网络采用包括有颜色类型、线型以及边沿类型标注信息的路面训练图像集监督训练而得。
- 根据权利要求13所述的方法,其特征在于,所述将所述路面图像输入所述神经网络之前,所述方法还包括:对所述路面图像进行去畸变处理。
- 一种车道线属性检测装置,其特征在于,包括:第一获取模块,用于获取智能设备上安装的图像采集装置所采集的路面图像;第一确定模块,用于根据所述路面图像确定概率图,所述概率图包括:颜色属性概率图、线型属性概率图和边沿属性概率图中的至少两种,其中,所述颜色属性概率图为N1个,所述线型属性概率图为N2个,所述边沿属性概率图为N3个,其中,N1、N2和N3均为大于0的整数;每个所述颜色属性概率图表示所述路面图像中的各个点属于该颜色属性概率图对应颜色的概率,每个所述线型属性概率图表示所述路面图像中的各个点属于该线型属性概率图对应线型的概率,每个所述边沿属性概率图表示所述路面图像中的各个点属于该边沿属性概率图对应边沿的概率;第二确定模块,用于根据所述概率图确定所述路面图像中的车道线属性。
- 根据权利要求15所述的装置,其特征在于,所述N1个颜色属性概率图对应的颜色包括以下至少之一:白色、黄色、蓝色。
- 根据权利要求15或16所述的装置,其特征在于,所述N2个线型属性概率图对应的线型包括以下至少之一:虚线、实线、双虚线、双实线、虚实线、实虚线、三虚线、虚实虚线。
- 根据权利要求15-17任一项所述的装置,其特征在于,所述N3个边沿属性概率图对应的边沿包括以下至少之一:路牙型边沿、栅栏型边沿、墙或花坛型边沿、虚拟边沿、非边沿。
- 根据权利要求15-18任一所述的装置,其特征在于,所述概率图包括第一属性概率图和第二属性概率图,所述第一属性概率图和所述第二属性概率图为所述颜色属性概率图、所述线型属性概率图和所述边沿属性概率图中的两种,且所述第一属性概率图和所述第二属性概率图不同;所述第二确定模块具体用于:针对所述路面图像中的一条车道线位置处的每个点,确定该点在L个所述第一属性概率图中的对应位置的各自的概率值;针对该点,将概率值最大的第一属性概率图对应的第一属性的值,作为该点的第一属性的值;根据所述路面图像中的该条车道线位置处的各个点的第一属性的值,确定该条车道线的第一属性的值;针对所述路面图像中的该条车道线位置处的每个点,确定该点在S个所述第二属性概率图中的对应位置的各自的概率值;针对该点,将概率值最大的第二属性概率图对应的第二属性的值,作为该点的第二属性的值;根据所述路面图像中的该条车道线位置处的各个点的第二属性的值,确定该条车道线的第二属性的值;将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合;将组合后的属性的值作为该条车道线的属性的值;其中,当所述第一属性概率图为所述颜色属性概率图时,L等于N1,所述第一属性为颜色属性;当所述第一属性概率图为所述线型属性概率图时,L等于N2,所述第一属性为线型属性;当所述第一属性概率图为所述边沿属性概率图时,L等于N3,第一属性为边沿属性;当所述第二属性概率图为所述颜色属性概率图时,S等于N1,所述第二属性为所述颜色属性;当所述第二属性概率图为所述线型属性概率图时,S等于N2,所述第二属性为所述线型属性;当所述第二属性概率图为所述边沿属性概率图时,S等于N3,所述第二属性为所述边沿属性。
- 根据权利要求19所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的该条车道线位置处的各个点的所述第一属性,确定该条车道线的所述第一属性的值,包括:响应于该条车道线位置处的各个点的所述第一属性的值不同,将该条车道线位置处的、第一属性的值相同的点的数量最多的点的所述第一属性的值,作为该条车道线的所述第一属性的值。
- 根据权利要求19或20所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的该条车道线位置处的各个点的所述第一属性,确定该条车道线的所述第一属性的值,包括:响应于该条车道线位置处的各个点的所述第一属性的值相同,将该条车道线位置处的点的所述第一属性的值作为该条车道线的所述第一属性的值。
- 根据权利要求19-21任一所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的该条车道线位置处的各个点的所述第二属性的值,确定该条车道线的所述第二属性的值,包括:响应于该条车道线位置处的各个点的所述第二属性的值不同,将该条车道线位置处的、第二属性的值相同的点的数量最多的点的所述第二属性的值,作为该条车道线的所述第二属性的值。
- 根据权利要求19-22任一所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的该条车道线位置处的各个点的所述第二属性的值,确定该条车道线的所述第二属性的值,包括:响应于该条车道线位置处的各个点的所述第二属性的值相同,将该条车道线位置处的点的所述第二属性的值作为该条车道线的所述第二属性的值。
- 根据权利要求19-23任一所述的装置,其特征在于,所述概率图还包括第三属性概率图,所述第三属性概率图为所述颜色属性概率图、所述线型属性概率图和所述边沿属性概率图中的一种,且所述第三属性概率图、所述第二属性概率图与所述第一属性概率图为属性两两不同的概率图;所述第二确定模块,还用于:在将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合之前,针对所述路面图像中的该条车道线位置处的每个点,确定该点在U个所述第三属性概率图中的对应位置的各自的概率值;针对该点,将概率值最大的第三属性概率图对应的第三属性的值,作为该点的第三属性的值;根据所述路面图像中的该条车道线位置处的各个点的第三属性的值,确定该条车道线的第三属性的值;其中,当所述第三属性概率图为所述颜色属性概率图时,U等于N1,所述第三属性为所述颜色属性;当所述第三属性概率图为所述线型属性概率图时,U等于N2,所述第三属性为所述线型属性;当所述第三属性概率图为所述边沿属性概率图时,U等于N3,所述第三属性为所述边沿属性;所述第二确定模块将该条车道线的所述第一属性的值和该条车道线的所述第二属性的值进行组合,包括:将该条车道线的所述第一属性的值、该条车道线的所述第二属性的值以及该条车道线的所述第三属性的值进行组合。
- 根据权利要求24所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的该条车道线位置处的各个点的所述第三属性的值,确定该条车道线的所述第三属性的值,包括:响应于该条车道线位置处的各个点的所述第三属性的值不同,将该条车道线位置处的、第三属性的值相同的点的数量最大的点的所述第三属性的值,作为该条车道线的所述第三属性的值。
- 根据权利要求24或25所述的装置,其特征在于,所述第二确定模块根据所述路面图像中的一条车道线位置处的各个点的第三属性的值,确定该条车道线的第三属性的值,包括:响应于该条车道线位置处的各个点的所述第三属性的值相同,将该条车道线位置处的点的所述第三属性的值作为该条车道线的所述第三属性的值。
- 根据权利要求15-26任一项所述的装置,其特征在于,所述第一确定模块具体用于:将所述路面图像输入神经网络,所述神经网络输出所述概率图;其中,所述神经网络采用包括有颜色类型、线型以及边沿类型标注信息的路面训练图像集监督训练而得。
- 根据权利要求27所述的装置,其特征在于,所述装置还包括:预处理模块,用于对所述路面图像进行去畸变处理。
- 一种电子设备,其特征在于,包括:存储器,用于存储程序指令;处理器,用于调用并执行所述存储器中的所述程序指令,执行权利要求1-14任一项所述的方法步骤。
- 一种智能驾驶方法,用于智能设备,其特征在于,包括:获取路面图像;采用如权利要求1-14任一项所述的车道线属性检测方法,检测获取的所述路面图像中的车道线属性;根据检测得到的所述车道线属性输出提示信息或对所述智能设备进行行驶控制。
- 一种智能设备,其特征在于,包括:图像采集装置,用于获取路面图像;存储器,用于存储程序指令,存储的程序指令被执行时以实现如权利要求1-14任一项所述的车道线属性检测方法;处理器,用于根据所述图像采集装置获取的所述路面图像,执行所述存储器存储的所述程序指令以检测所述路面图像中的车道线属性,并根据检测得到的车道线属性输出提示信息或对智能设备进行行驶控制。
- 一种非易失性可读存储介质,其特征在于,所述可读存储介质中存储有计算机程序,所述计算机程序用于执行权利要求1-14任一项所述的方法步骤。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021500086A JP7119197B2 (ja) | 2019-06-25 | 2020-02-20 | 車線属性検出 |
SG11202013052UA SG11202013052UA (en) | 2019-06-25 | 2020-02-20 | Lane line attribute detection |
KR1020217000803A KR20210018493A (ko) | 2019-06-25 | 2020-02-20 | 차선 속성 검출 |
US17/137,030 US20210117700A1 (en) | 2019-06-25 | 2020-12-29 | Lane line attribute detection |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910556260.X | 2019-06-25 | ||
CN201910556260.XA CN112131914B (zh) | 2019-06-25 | 2019-06-25 | 车道线属性检测方法、装置、电子设备及智能设备 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/137,030 Continuation US20210117700A1 (en) | 2019-06-25 | 2020-12-29 | Lane line attribute detection |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020258894A1 true WO2020258894A1 (zh) | 2020-12-30 |
Family
ID=73849445
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/076036 WO2020258894A1 (zh) | 2019-06-25 | 2020-02-20 | 车道线属性检测 |
Country Status (6)
Country | Link |
---|---|
US (1) | US20210117700A1 (zh) |
JP (1) | JP7119197B2 (zh) |
KR (1) | KR20210018493A (zh) |
CN (1) | CN112131914B (zh) |
SG (1) | SG11202013052UA (zh) |
WO (1) | WO2020258894A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3955218A3 (en) * | 2021-01-25 | 2022-04-20 | Beijing Baidu Netcom Science Technology Co., Ltd. | Lane line detection method and apparatus, electronic device, computer storage medium, and computer program product |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112396044B (zh) * | 2021-01-21 | 2021-04-27 | 国汽智控(北京)科技有限公司 | 车道线属性信息检测模型训练、车道线属性信息检测方法 |
US11776282B2 (en) * | 2021-03-26 | 2023-10-03 | Here Global B.V. | Method, apparatus, and system for removing outliers from road lane marking data |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140185879A1 (en) * | 2011-09-09 | 2014-07-03 | Industry-Academic Cooperation Foundation, Yonsei University | Apparatus and method for detecting traffic lane in real time |
CN105260699A (zh) * | 2015-09-10 | 2016-01-20 | 百度在线网络技术(北京)有限公司 | 一种车道线数据的处理方法及装置 |
KR20170007596A (ko) * | 2015-07-09 | 2017-01-19 | 현대자동차주식회사 | 개선된 차선 인식 방법 |
CN108052904A (zh) * | 2017-12-13 | 2018-05-18 | 辽宁工业大学 | 车道线的获取方法及装置 |
CN109657632A (zh) * | 2018-12-25 | 2019-04-19 | 重庆邮电大学 | 一种车道线检测识别方法 |
CN109670376A (zh) * | 2017-10-13 | 2019-04-23 | 神州优车股份有限公司 | 车道线识别方法及*** |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11211659A (ja) * | 1998-01-23 | 1999-08-06 | Nagoya Denki Kogyo Kk | 路面状態判別方法およびその装置 |
JP2001263479A (ja) | 2000-03-17 | 2001-09-26 | Equos Research Co Ltd | 車両制御装置、車両制御方法及びそのプログラムを記録した記録媒体 |
JP5083658B2 (ja) | 2008-03-26 | 2012-11-28 | 本田技研工業株式会社 | 車両用車線認識装置、車両、及び車両用車線認識プログラム |
JP2010060371A (ja) * | 2008-09-02 | 2010-03-18 | Omron Corp | 物体検出装置 |
BR112012002884A2 (pt) * | 2009-08-12 | 2017-12-19 | Koninl Philips Electronics Nv | sistema de formação de imagens médicas para gerar dados de um objeto de características de uma região de interesse de um objeto, método para a geração de dados de um objeto de características de uma região de interesse de um objeto, elemento de programa de computador para controlar um aparelho e meio que pode ser lido por computador |
CN102862574B (zh) | 2012-09-21 | 2015-08-19 | 上海永畅信息科技有限公司 | 基于智能手机实现车辆主动安全的方法 |
JP5983238B2 (ja) * | 2012-09-25 | 2016-08-31 | 日産自動車株式会社 | 車線境界線検出装置及び車線境界線検出方法 |
CN104376297B (zh) * | 2013-08-12 | 2017-06-23 | 株式会社理光 | 道路上的线型指示标志的检测方法和装置 |
CN108216229B (zh) * | 2017-09-08 | 2020-01-10 | 北京市商汤科技开发有限公司 | 交通工具、道路线检测和驾驶控制方法及装置 |
US10628671B2 (en) * | 2017-11-01 | 2020-04-21 | Here Global B.V. | Road modeling from overhead imagery |
CN107945168B (zh) * | 2017-11-30 | 2021-12-10 | 上海联影医疗科技股份有限公司 | 一种医学图像的处理方法及医学图像处理*** |
CN108009524B (zh) * | 2017-12-25 | 2021-07-09 | 西北工业大学 | 一种基于全卷积网络的车道线检测方法 |
CN108875603B (zh) * | 2018-05-31 | 2021-06-04 | 上海商汤智能科技有限公司 | 基于车道线的智能驾驶控制方法和装置、电子设备 |
CN109147368A (zh) * | 2018-08-22 | 2019-01-04 | 北京市商汤科技开发有限公司 | 基于车道线的智能驾驶控制方法装置与电子设备 |
CN109635816B (zh) * | 2018-10-31 | 2021-04-06 | 百度在线网络技术(北京)有限公司 | 车道线生成方法、装置、设备以及存储介质 |
CN109740469B (zh) * | 2018-12-24 | 2021-01-22 | 百度在线网络技术(北京)有限公司 | 车道线检测方法、装置、计算机设备和存储介质 |
-
2019
- 2019-06-25 CN CN201910556260.XA patent/CN112131914B/zh active Active
-
2020
- 2020-02-20 SG SG11202013052UA patent/SG11202013052UA/en unknown
- 2020-02-20 KR KR1020217000803A patent/KR20210018493A/ko not_active Application Discontinuation
- 2020-02-20 WO PCT/CN2020/076036 patent/WO2020258894A1/zh active Application Filing
- 2020-02-20 JP JP2021500086A patent/JP7119197B2/ja active Active
- 2020-12-29 US US17/137,030 patent/US20210117700A1/en not_active Abandoned
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140185879A1 (en) * | 2011-09-09 | 2014-07-03 | Industry-Academic Cooperation Foundation, Yonsei University | Apparatus and method for detecting traffic lane in real time |
KR20170007596A (ko) * | 2015-07-09 | 2017-01-19 | 현대자동차주식회사 | 개선된 차선 인식 방법 |
CN105260699A (zh) * | 2015-09-10 | 2016-01-20 | 百度在线网络技术(北京)有限公司 | 一种车道线数据的处理方法及装置 |
CN109670376A (zh) * | 2017-10-13 | 2019-04-23 | 神州优车股份有限公司 | 车道线识别方法及*** |
CN108052904A (zh) * | 2017-12-13 | 2018-05-18 | 辽宁工业大学 | 车道线的获取方法及装置 |
CN109657632A (zh) * | 2018-12-25 | 2019-04-19 | 重庆邮电大学 | 一种车道线检测识别方法 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3955218A3 (en) * | 2021-01-25 | 2022-04-20 | Beijing Baidu Netcom Science Technology Co., Ltd. | Lane line detection method and apparatus, electronic device, computer storage medium, and computer program product |
US11741726B2 (en) | 2021-01-25 | 2023-08-29 | Beijing Baidu Netcom Science Technology Co., Ltd. | Lane line detection method, electronic device, and computer storage medium |
Also Published As
Publication number | Publication date |
---|---|
US20210117700A1 (en) | 2021-04-22 |
SG11202013052UA (en) | 2021-01-28 |
CN112131914B (zh) | 2022-10-21 |
JP7119197B2 (ja) | 2022-08-16 |
KR20210018493A (ko) | 2021-02-17 |
JP2021532449A (ja) | 2021-11-25 |
CN112131914A (zh) | 2020-12-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022126377A1 (zh) | 检测车道线的方法、装置、终端设备及可读存储介质 | |
WO2020103893A1 (zh) | 车道线属性检测方法、装置、电子设备及可读存储介质 | |
WO2021249071A1 (zh) | 一种车道线的检测方法及相关设备 | |
WO2020258894A1 (zh) | 车道线属性检测 | |
US11455805B2 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
US9171375B2 (en) | Multi-cue object detection and analysis | |
WO2020103892A1 (zh) | 车道线检测方法、装置、电子设备及可读存储介质 | |
CN112528878A (zh) | 检测车道线的方法、装置、终端设备及可读存储介质 | |
CN112287912B (zh) | 基于深度学习的车道线检测方法以及装置 | |
KR20210043516A (ko) | 궤적 계획 모델을 훈련하는 방법, 장치, 전자 기기, 저장 매체 및 프로그램 | |
WO2020258077A1 (zh) | 一种行人检测方法及装置 | |
WO2022217434A1 (zh) | 感知网络、感知网络的训练方法、物体识别方法及装置 | |
CN111931683B (zh) | 图像识别方法、装置及计算机可读存储介质 | |
CN110956119B (zh) | 一种图像中目标检测的方法 | |
CN113723377A (zh) | 一种基于ld-ssd网络的交通标志检测方法 | |
WO2021083126A1 (zh) | 目标检测、智能行驶方法、装置、设备及存储介质 | |
CN111079634B (zh) | 车辆行驶中检测障碍物的方法、装置、***及车辆 | |
CN110909656B (zh) | 一种雷达与摄像机融合的行人检测方法和*** | |
CN117218622A (zh) | 路况检测方法、电子设备及存储介质 | |
CN117975418A (zh) | 一种基于改进rt-detr的交通标识检测方法 | |
Al Mamun et al. | Efficient lane marking detection using deep learning technique with differential and cross-entropy loss. | |
CN111160206A (zh) | 一种交通环境元素视觉感知方法及装置 | |
CN114495049A (zh) | 识别车道线的方法和装置 | |
CN113449647A (zh) | 弯曲车道线的拟合方法、***、设备及计算机可读存储介质 | |
CN115082903B (zh) | 非机动车违停识别方法、装置、计算机设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2021500086 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 20217000803 Country of ref document: KR Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20830874 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20830874 Country of ref document: EP Kind code of ref document: A1 |