CN113343778B - Lane line detection method and system based on LaneSegNet - Google Patents
Lane line detection method and system based on LaneSegNet Download PDFInfo
- Publication number
- CN113343778B CN113343778B CN202110527595.6A CN202110527595A CN113343778B CN 113343778 B CN113343778 B CN 113343778B CN 202110527595 A CN202110527595 A CN 202110527595A CN 113343778 B CN113343778 B CN 113343778B
- Authority
- CN
- China
- Prior art keywords
- convolution
- module
- lane line
- branch
- lanesegnet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a Lane line detection method and system based on LaneSegNet. Firstly, carrying out polygon filling on an image to obtain an ROI (region of interest), then inputting the ROI image into a trained LaneSegNet network model to obtain a binary image containing lane lines, clustering the coordinates of the pixel points of the lane lines by using a DBSCAN (direct memory access controller) algorithm, carrying out polynomial fitting, and displaying the fitted lane lines on an original image. The constructed LaneSegNet network model comprises a network architecture of a coding module, a decoding module, an enhanced receptive field module and an enhanced feature module. The invention increases the network receptive field by using the parallel cavity convolution module, removes the characteristic information irrelevant to the current task by using the enhanced characteristic module, and adopts the asymmetric convolution to construct the characteristic extraction network, thereby reducing the network parameters. The method has the accuracy rate of 98.62 percent, can be used for detecting lane lines on the highway, and has better robustness and real-time property.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a lane line detection method and system based on a LaneSegNet (lane line segmentation network).
Background
With the improvement of living standards of people, automobiles play an increasingly important role in the lives of people, however, traffic accidents are more and more caused by the increase of the keeping quantity of the automobiles, in order to guarantee the driving safety, the automatic driving function receives more and more attention, and the lane line detection is an important component in the automatic driving function.
At present, in the traditional lane line detection method, if the lane line is detected by using Hough transformation, firstly, the edge characteristics of the lane line are extracted by using image processing, and then the line is detected and fitted by using the Hough transformation. The lane line detection method can only be used for detection with uniform illumination, single environment, no shielding and no blurring, and has poor robustness.
With the wide application of deep learning in various fields, the combination of lane line detection and deep learning becomes more and more compact, for example, in the end-to-end lane line detection proposed by Davy Neven and the like, the lane lines are detected through a deep neural network, then the lane lines are clustered by using a clustering algorithm, and finally the lane lines are fitted by using a polynomial. However, the detection process of the method is complex and time-consuming, and the real-time requirement is difficult to meet.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems of poor robustness, complex process and long detection time of the conventional lane line detection, the invention provides a lane line detection method and system based on LaneSegNet.
The technical scheme is as follows: in order to achieve the purpose, the invention adopts the following technical scheme:
a LaneSegNet-based lane line detection method comprises the following steps:
(1) carrying out polygon filling on the road image to obtain an ROI (region of interest) image containing a lane line;
(2) inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line;
the LaneSegNet network model comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced characteristic modules which are sequentially connected; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module;
(3) clustering the coordinates of the pixel points of the lane lines by using a DBSCAN algorithm for the binary image obtained in the step (2), marking out the lane lines of different types, and respectively fitting the lane lines of different types by using a quadratic polynomial;
(4) displaying the fitted lane line on the original road image to realize the visualization of the lane line detection.
Preferably, the initial module includes a convolutional layer with a convolutional kernel size of k × k and a step size of 1, a convolutional layer with a convolutional kernel size of k × k and a step size of 2, a maximum pooling layer, and a connection layer, where two convolutional layers are connected in sequence, the convolutional layers and the pooling layer are connected in parallel, and k is 3 or 5.
Preferably, the three convolution downsampling modules have the same unit structure, and each convolution downsampling module comprises a first branch and a second branch, wherein the first branch is formed by connecting a 1 × 1 convolution module, two kxk convolution modules and the 1 × 1 convolution modules in series, the second branch is formed by connecting three cavity convolution modules with expansion rates of 1, 2 and 5 in series, the first branch is connected with the second branch in parallel, the input of the module is added with two outputs connected in parallel, the module is divided into a third branch and a fourth branch, the third branch is formed by connecting a 1 × 1 convolution module, two kxk convolution modules and a 1 × 1 convolution module in series, the fourth branch is provided with a maximum pooling layer, the third branch is connected with the fourth branch in parallel, the outputs of the two branches are added, and k is 3 or 5.
Preferably, the enhanced receptive field module includes three parallel cavity convolution branches, a first cavity convolution branch includes k × k convolution with an expansion rate of 1, a second cavity convolution branch includes four consecutive k × k convolutions, the expansion rates are 2, 5, 9, and 13, the structure of a third cavity convolution branch is the same as that of the second cavity convolution branch, the output of the first cavity convolution branch is the input of the second cavity convolution branch, the output of the second cavity convolution branch is the input of the third cavity convolution branch, and finally the outputs of the three cavity convolution branches are added, where k is 3 or 5.
Preferably, the two enhanced feature modules have the same structure and comprise a first k × k asymmetric convolution, two parallel global average pooling and global maximum pooling, a second k × k asymmetric convolution, a 1 × 1 convolution, a sigmoid activation layer and a threshold layer, wherein the final obtained threshold is multiplied by the input, and k is 3 or 5; wherein the threshold function is:
preferably, the four convolution upsampling modules have the same structure and respectively comprise a 1 × 1 convolution, a k × k convolution, two parallel transposed convolution and upsampling and a 1 × 1 convolution which are sequentially connected; each convolution operation is followed by Batch Normalization and PReLU nonlinear activation function processing.
Preferably, the step of training the LaneSegNet network model in step (2) includes:
(2.1) inputting the ROI area image of the road image and the binary image of the marked lane line into a LaneSegNet network model as training sample data;
(2.2) calculating the loss of the LaneSegNet network, and continuously optimizing parameters in the network by taking the minimum loss as a target;
and (2.3) when the loss value is stabilized within a certain range, storing the network parameters to obtain a final lane line detection model.
Based on the same inventive concept, the lane line detection system based on the LaneSegNet provided by the invention comprises:
the preprocessing module is used for carrying out polygon filling on the road image and acquiring an ROI (region of interest) image containing a lane line;
the lane line recognition module is used for inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line; the LaneSegNet network model comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced characteristic modules which are sequentially connected; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module;
the lane line fitting module is used for clustering the coordinates of the pixel points of the lane lines by using a DBSCAN algorithm on the binary images obtained by the LaneSegNet network model, marking off the lane lines of different types, and respectively fitting the lane lines of different types by using a quadratic polynomial;
and the result output module is used for displaying the fitted lane line on the original road image to realize the visualization of the lane line detection.
Based on the same inventive concept, the lane line detection system based on the LaneSegNet provided by the invention comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is loaded to the processor, the lane line detection method based on the LaneSegNet is realized.
Has the advantages that: compared with the prior art, the invention has the following beneficial effects: 1. the LaneSegNet model architecture is utilized, the structure is simple, and the advantages of small parameter quantity are utilized to realize high-precision extraction of the lane lines in the road image. 2. The field-of-experience enhancing module increases the field of experience of the network model, and avoids the problems that the span of a lane line in an image is too large and the lane line is not easy to be segmented. 3. The enhanced feature module removes features irrelevant to tasks, so that the network is more concentrated on extracting lane line features, and the problem of difficult segmentation caused by small occupation ratio of lane line pixels in an image is solved.
Drawings
FIG. 1 is a flow chart of a method of an embodiment of the present invention;
FIG. 2 is a diagram of a LaneSegNet model network structure according to an embodiment of the present invention;
FIG. 3 is a network architecture diagram of an initial module in an embodiment of the invention;
FIG. 4 is a network structure diagram of an encoding module according to an embodiment of the present invention;
FIG. 5 is a diagram of an enhanced receptor field module network according to an embodiment of the present invention;
FIG. 6 is a diagram of an enhanced feature module network architecture in an embodiment of the present invention;
FIG. 7 is a diagram of a model network structure of a decoding module according to an embodiment of the present invention;
FIG. 8 is a schematic illustration of a portion of a data set used in an embodiment of the present invention;
FIG. 9 is a partial data set tag diagram used in an embodiment of the present invention;
FIG. 10 is a process for annotating images of a data set in accordance with an embodiment of the present invention;
FIG. 11 is a segmentation chart of an example of road segmentation line detection in an embodiment of the present invention;
FIG. 12 is a lane line fit graph according to an embodiment of the present invention;
FIG. 13 is a miou diagram during training in an embodiment of the present invention;
FIG. 14 is an acc diagram during training in an embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and specific embodiments.
As shown in fig. 1, in the lane line detection method based on LaneSegNet disclosed in the embodiment of the present invention, firstly, a road image is filled with polygons to obtain an ROI (region of interest) area containing a lane line; then, inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line; clustering the coordinates of the lane line pixel points in the binary image by using a DBSCAN algorithm, and respectively fitting different types of lane lines by using a quadratic polynomial; and finally, displaying the fitted lane line on the original road image to realize the visualization of the lane line detection.
The data set and the specific structure of the network model used in this embodiment will be first described in detail.
The method comprises the steps of preprocessing a road video captured by a camera, acquiring effective road video data, performing frame extraction on an acquired image, acquiring a road image to be marked, marking a lane line in the image to be marked by a marking tool, and acquiring a road image marked by a lane line target. Resulting in a constructed data set.
As shown in fig. 2, the LaneSegNet model constructed in the embodiment of the present invention mainly includes a coding module, a decoding module, an enhanced receptive field module, and an enhanced feature module. The coding module mainly comprises three identical convolution down-sampling modules, and the receptive field of the network is enlarged by using the hole convolution; the decoding module mainly comprises four same convolution up-sampling modules for recovering the characteristic information and the image size; the enhanced reception field module is used for further increasing the reception field of the network; and the enhanced feature module enhances the information related to the current task and discards the information unrelated to the current task. And inputting the training data into a LaneSegNet network for training to obtain a binary image containing the lane line. The model structure specifically comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced feature modules which are connected in sequence; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module.
Referring to fig. 3, the initial module includes a convolutional layer with a convolutional kernel size of 3 × 3 and a step size of 1, a convolutional layer with a convolutional kernel size of 3 × 3 and a step size of 2, a maximum pooling layer, and a connection layer, where two convolutional layers are connected in sequence, and the convolutional layer and the pooling layer are connected in parallel.
The three convolution downsampling modules have the same unit structure, as shown in fig. 4, each convolution downsampling module comprises a 1 × 1 convolution, a first branch circuit formed by connecting two 3 × 3 convolution products and the 1 × 1 convolution in series, and a second branch circuit formed by connecting three cavity convolution products with expansion rates of 1, 2 and 5 in series, the first branch circuit and the second branch circuit are connected in parallel, the input of the module and the output of the two parallel connection circuits are subjected to addition operation, the module is divided into a third branch circuit formed by connecting one 1 × 1 convolution product, two 3 × 3 convolution products and the 1 × 1 convolution in series and a fourth branch circuit provided with a maximum pooling layer, the third branch circuit and the fourth branch circuit are connected in parallel, and the output of the two branch circuits is subjected to addition operation.
As shown in fig. 5, the enhanced receptive field module includes three parallel cavity convolution branches, a first cavity convolution branch includes a 3 × 3 convolution with an expansion rate of 1, a second cavity convolution branch includes four consecutive 3 × 3 convolutions with expansion rates of 2, 5, 9, and 13, a third cavity convolution branch has the same structure as the second cavity convolution branch, an output of the first cavity convolution branch is an input of the second cavity convolution branch, an output of the second cavity convolution branch is an input of the third cavity convolution branch, and finally, outputs of the three cavity convolution branches are added.
The two enhanced feature modules have the same structure, as shown in fig. 6, and include a first 3 × 3 asymmetric convolution, two parallel global average pooling and global maximum pooling, a second 3 × 3 asymmetric convolution, a 1 × 1 convolution, a sigmoid activation layer, and a threshold layer, and a product operation is performed on the finally obtained threshold and the input; wherein the threshold function is:
the four convolution UpSampling modules have the same structure, and as shown in fig. 7, each convolution UpSampling module comprises a 1 × 1 convolution, a 3 × 3 convolution, two parallel Conv2 dtransposes and UpSampling2D, and a 1 × 1 convolution, which are connected in sequence; each convolution operation is followed by Batch Normalization and PReLU nonlinear activation function processing.
The training steps of the LaneSegNet network model are as follows: firstly, extracting an interested area from a road image, and setting six coordinate points: r1(0,270), r2(0, h), r3(w, h), r4(w,470), r5(670,150) and r6(570,150), wherein w is the width of the input image, h is the height of the input image, the outside of a region surrounded by coordinate points is set to be 0, a region of interest containing lane lines is obtained, then the region of interest image and a corresponding binary image are used as training sample data and input into a LaneSegNet network model, then the loss of the LaneSegNet network is calculated, and the parameters in the network are continuously optimized by taking the minimum loss as a target. And when the loss value is stabilized within a certain range, storing the network parameters to obtain a final lane line detection model. In this embodiment, the constructed data set is video data shot by a vehicle running on a highway, and the original video needs to be clipped and denoised because the original video data has information irrelevant to a training task. After processing, the video is decimated to obtain enough training data, which is shown in fig. 8.
Marking the lane lines in the acquired road image by using a Labelme tool, generating a json file under an original folder after marking one picture, and generating a corresponding binary image according to the json file, as shown in FIG. 9. In the present embodiment, 11807 road images are collectively labeled, and the labeling process is as shown in fig. 10.
In this embodiment, the data storage folder is Datasets, and includes two subfolders, which are Images and Labels, respectively, where Images store training Images, and Labels stores binary Images corresponding to the training Images, and a ratio of the training set to the verification set is 7: and 3, storing the training set image path in train.txt, and storing the verification set image path in val.txt, wherein the relative paths of the training image and the label are stored.
Training of the LaneSegNet network model: inputting the marked lane line data set into a LaneSegNet network for training, setting corresponding parameters, and then performing model training to obtain a trained LaneSegNet network model, wherein the method specifically comprises the following steps:
1) parameters are set including learning rate, Epochs size, batch size, etc. Where the initial learning rate is 1e-3 and eventually drops to 2.5 e-5. Wherein the batch-size is 2 and the Epochs are 100.
2) The data were trained, and the images were trained and predicted using the parameters set in 1), and the prediction results are shown in fig. 11.
After obtaining the binary image containing the lane lines, clustering the segmentation results by using DBSCAN and fitting the clustering results by using a quadratic polynomial, wherein the fitting effect is shown in fig. 12.
Obtaining the coordinates of the lane line pixel points from the output data of the LaneSegNet network, clustering the coordinates of the lane line pixel points by using a DBSCAN algorithm so as to mark off the lane lines of different types, respectively fitting the lane lines of different types by using a quadratic polynomial, drawing the lane lines in the original image by using the fitted quadratic polynomial, and realizing the visualization of lane line detection.
The experimental environment used in the examples of the present invention is as follows:
operating the system:
windows 1064 bit
Hardware environment:
inter Core i5-10400F @2.90GHZ hexanuclear
16GB DDR4 2600MHZ RAM
Nvidia GTX 2060 SUPER with 6GB DRAM
WDS 500G with SSD
Software environment:
deep learning frame Keras (2.2.5)
Operating environment Python 3.6
JetBrains PyCharm 2020.2x64
CUDA 10.2
miou is an evaluation index of semantic segmentation and is an important standard for measuring model performance. miou is the ratio of the intersection and union of the real value and the predicted value, and miou of the LaneSegNet network model provided by the invention is shown in FIG. 13, and as can be seen from the graph, the miou value of the lane line detection method based on LaneSegNet provided by the invention is 78.18%.
The method provided by the invention is used for detecting the lane line based on the network model of the LaneSegNet, the detection precision of the method achieves a better effect, and as shown in figure 14, the accuracy of the lane line detection based on the method provided by the invention reaches 98.62%.
Based on the same inventive concept, the lane line detection system based on the LaneSegNet disclosed by the embodiment of the invention comprises a preprocessing module, a detection module and a control module, wherein the preprocessing module is used for performing polygon filling on a road image to obtain an ROI (region of interest) image containing a lane line; the lane line recognition module is used for inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line; the LaneSegNet network model comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced characteristic modules which are sequentially connected; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module; the lane line fitting module is used for clustering the coordinates of the pixel points of the lane lines by using a DBSCAN algorithm on the binary images obtained by the LaneSegNet network model, marking off the lane lines of different types, and respectively fitting the lane lines of different types by using a quadratic polynomial; and the result output module is used for displaying the fitted lane line on the original road image to realize the visualization of the lane line detection. For specific details, the method is implemented by reference to the above method, and details are not described herein.
Based on the same inventive concept, the lane line detection system based on LaneSegNet provided by the embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the computer program is loaded into the processor, the lane line detection system based on LaneSegNet realizes the lane line detection method based on LaneSegNet.
Claims (7)
1. A LaneSegNet-based lane line detection method is characterized by comprising the following steps:
(1) carrying out polygon filling on the road image to obtain an ROI (region of interest) image containing a lane line;
(2) inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line; the LaneSegNet network model comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced characteristic modules which are sequentially connected; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module;
the three convolution downsampling modules have the same unit structure, and each convolution downsampling module comprises a 1 × 1 convolution, a first branch connected in series with two kxk convolution products and the 1 × 1 convolution, and a second branch connected in series with three cavity convolution with expansion rates of 1, 2 and 5 respectively, wherein the first branch is connected with the second branch in parallel, the input of the module is added with two outputs connected in parallel, and then the module is divided into a third branch connected in series with one 1 × 1 convolution product and two kxk convolution products and the 1 × 1 convolution and a fourth branch provided with a maximum pooling layer, the third branch is connected with the fourth branch in parallel, the outputs of the two branches are added, and k is 3 or 5;
the two enhanced feature modules have the same structure and comprise a first k multiplied by k asymmetric convolution, two parallel global average pooling and global maximum pooling, a second k multiplied by k asymmetric convolution, a 1 multiplied by 1 convolution, a sigmoid active layer and a threshold layer, and the final obtained threshold is multiplied by the input; wherein the threshold function is:
(3) clustering the coordinates of the pixel points of the lane lines by using a DBSCAN algorithm for the binary image obtained in the step (2), marking out the lane lines of different types, and respectively fitting the lane lines of different types by using a quadratic polynomial;
(4) displaying the fitted lane line on the original road image to realize the visualization of the lane line detection.
2. The LaneSegNet-based lane line detection method of claim 1, wherein the initialization module comprises a convolutional layer with a convolutional kernel size of k × k and a step size of 1, a convolutional layer with a convolutional kernel size of k × k and a step size of 2, a maximum pooling layer, and a connection layer, wherein two convolutional layers are connected in sequence, and the convolutional layers and the pooling layer are connected in parallel.
3. The LanesegNet-based lane line detection method as claimed in claim 1, wherein said enhanced field module comprises three parallel hole convolution branches, a first hole convolution branch comprises a kxk convolution with an expansion rate of 1, a second hole convolution branch comprises four consecutive kxk convolutions, the expansion rates are 2, 5, 9, and 13, respectively, a third hole convolution branch has the same structure as the second hole convolution branch, an output of the first hole convolution branch is an input of the second hole convolution branch, an output of the second hole convolution branch is an input of the third hole convolution branch, and finally, outputs of the three hole convolution branches are added.
4. The LaneSegNet-based lane line detection method according to claim 1, wherein the four convolution upsampling modules have the same structure, and each convolution upsampling module comprises a 1 x 1 convolution, a k x k convolution, two parallel transposed convolution products and upsampling, and a 1 x 1 convolution, which are connected in sequence; each convolution operation is followed by Batch Normalization and PReLU nonlinear activation function processing.
5. The LaneSegNet-based lane line detection method according to claim 1, wherein the step of training the LaneSegNet network model in step (2) comprises:
(2.1) inputting the ROI area image of the road image and the binary image of the marked lane line into a LaneSegNet network model as training sample data;
(2.2) calculating the loss of the LaneSegNet network, and continuously optimizing parameters in the network by taking the minimum loss as a target;
and (2.3) when the loss value is stabilized within a certain range, storing the network parameters to obtain a final lane line detection model.
6. A LaneSegNet-based lane line detection system, comprising:
the preprocessing module is used for carrying out polygon filling on the road image and acquiring an ROI (region of interest) image containing a lane line;
the lane line recognition module is used for inputting the ROI area image into a trained LaneSegNet network model to obtain a binary image containing a lane line; the LaneSegNet network model comprises an initial module, three convolution down-sampling modules, an enhanced receptive field module, four convolution up-sampling modules and two enhanced characteristic modules which are sequentially connected; the system comprises an initial module, a convolution down-sampling module, a receptive field enhancing module, a characteristic enhancing module and a convolution up-sampling module, wherein the initial module is used for reducing the size of an input image by half, the convolution down-sampling module is used for extracting the characteristic information of a lane line, the receptive field enhancing module is used for increasing the receptive field of a network, the characteristic enhancing module is used for enhancing the information of the lane line, and the convolution up-sampling module is used for recovering the size and the image characteristic of the image; the first enhancement feature module is connected with the output ends of the first convolution down-sampling module and the second convolution up-sampling module, the second enhancement feature module is connected with the output ends of the second convolution down-sampling module and the first convolution up-sampling module, the second convolution up-sampling module is connected with the output end of the second enhancement feature module, and the third convolution up-sampling module is connected with the output end of the first enhancement feature module; the three convolution downsampling modules have the same unit structure, and each convolution downsampling module comprises a 1 × 1 convolution, a first branch connected in series with two kxk convolution products and the 1 × 1 convolution, and a second branch connected in series with three cavity convolution with expansion rates of 1, 2 and 5 respectively, wherein the first branch is connected with the second branch in parallel, the input of the module is added with two outputs connected in parallel, and then the module is divided into a third branch connected in series with one 1 × 1 convolution product and two kxk convolution products and the 1 × 1 convolution and a fourth branch provided with a maximum pooling layer, the third branch is connected with the fourth branch in parallel, the outputs of the two branches are added, and k is 3 or 5; two enhanced feature modules are identical in structure, a bagThe method comprises the following steps of performing a first k x k asymmetric convolution, two parallel global average pooling and global maximum pooling, a second k x k asymmetric convolution, a 1 x 1 convolution, a sigmoid activation layer and a threshold layer, and performing product operation on a finally obtained threshold and input; wherein the threshold function is:
the lane line fitting module is used for clustering the coordinates of the pixel points of the lane lines by using a DBSCAN algorithm on the binary images obtained by the LaneSegNet network model, marking off the lane lines of different types, and respectively fitting the lane lines of different types by using a quadratic polynomial;
and the result output module is used for displaying the fitted lane line on the original road image to realize the visualization of the lane line detection.
7. LanesegNet-based lane line detection system comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements a LanesegNet-based lane line detection method according to any one of claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110527595.6A CN113343778B (en) | 2021-05-14 | 2021-05-14 | Lane line detection method and system based on LaneSegNet |
| PCT/CN2021/135474 WO2022237139A1 (en) | 2021-05-14 | 2021-12-03 | Lanesegnet-based lane line detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110527595.6A CN113343778B (en) | 2021-05-14 | 2021-05-14 | Lane line detection method and system based on LaneSegNet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113343778A CN113343778A (en) | 2021-09-03 |
| CN113343778B true CN113343778B (en) | 2022-02-11 |
Family
ID=77470031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110527595.6A Active CN113343778B (en) | 2021-05-14 | 2021-05-14 | Lane line detection method and system based on LaneSegNet |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113343778B (en) |
| WO (1) | WO2022237139A1 (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343778B (en) * | 2021-05-14 | 2022-02-11 | 淮阴工学院 | Lane line detection method and system based on LaneSegNet |
| CN113781607B (en) * | 2021-09-17 | 2023-09-19 | 平安科技（深圳）有限公司 | Processing method, device, equipment and storage medium for labeling data of OCR (optical character recognition) image |
| CN114241308B (en) * | 2021-12-17 | 2023-08-04 | 杭州电子科技大学 | Lightweight remote sensing image significance detection method based on compression module |
| CN116129379B (en) * | 2022-12-28 | 2023-11-07 | 国网安徽省电力有限公司芜湖供电公司 | Lane line detection method in foggy environment |
| CN115993365B (en) * | 2023-03-23 | 2023-06-13 | 山东省科学院激光研究所 | Belt defect detection method and system based on deep learning |
| CN116453121B (en) * | 2023-06-13 | 2023-12-22 | 合肥市正茂科技有限公司 | Training method and device for lane line recognition model |
| CN117764988B (en) * | 2024-02-22 | 2024-04-30 | 山东省计算中心（国家超级计算济南中心） | Road crack detection method and system based on heteronuclear convolution multi-receptive field network |
| CN118155105A (en) * | 2024-05-13 | 2024-06-07 | 齐鲁空天信息研究院 | Unmanned aerial vehicle mountain area rescue method, unmanned aerial vehicle mountain area rescue system, unmanned aerial vehicle mountain area rescue medium and electronic equipment |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107704866B (en) * | 2017-06-15 | 2021-03-23 | 清华大学 | Multitask scene semantic understanding model based on novel neural network and application thereof |
| US11068724B2 (en) * | 2018-10-11 | 2021-07-20 | Baidu Usa Llc | Deep learning continuous lane lines detection system for autonomous vehicles |
| US10853671B2 (en) * | 2019-03-06 | 2020-12-01 | Dura Operating, Llc | Convolutional neural network system for object detection and lane detection in a motor vehicle |
| CN111242037B (en) * | 2020-01-15 | 2023-03-21 | 华南理工大学 | Lane line detection method based on structural information |
| CN111460921B (en) * | 2020-03-13 | 2023-05-26 | 华南理工大学 | Lane line detection method based on multitasking semantic segmentation |
| CN111582083B (en) * | 2020-04-25 | 2023-05-23 | 华南理工大学 | Lane line detection method based on vanishing point estimation and semantic segmentation |
| CN113343778B (en) * | 2021-05-14 | 2022-02-11 | 淮阴工学院 | Lane line detection method and system based on LaneSegNet |
-
2021
- 2021-05-14 CN CN202110527595.6A patent/CN113343778B/en active Active
- 2021-12-03 WO PCT/CN2021/135474 patent/WO2022237139A1/en active Application Filing
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110276267A (en) * | 2019-05-28 | 2019-09-24 | 江苏金海星导航科技有限公司 | Method for detecting lane lines based on Spatial-LargeFOV deep learning network |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022237139A1 (en) | 2022-11-17 |
| CN113343778A (en) | 2021-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113343778B (en) | Lane line detection method and system based on LaneSegNet | |
| CN110322495B (en) | Scene text segmentation method based on weak supervised deep learning | |
| Zhang et al. | Deep learning in lane marking detection: A survey | |
| Xu et al. | Fast vehicle and pedestrian detection using improved Mask R‐CNN | |
| WO2021218786A1 (en) | Data processing system, object detection method and apparatus thereof | |
| CN112016467B (en) | Traffic sign recognition model training method, recognition method, system, device and medium | |
| CN111914698A (en) | Method and system for segmenting human body in image, electronic device and storage medium | |
| Cho et al. | Semantic segmentation with low light images by modified CycleGAN-based image enhancement | |
| CN111462050A (en) | Improved YO L Ov3 minimum remote sensing image target detection method, device and storage medium | |
| CN113095152A (en) | Lane line detection method and system based on regression | |
| CN114037674A (en) | Industrial defect image segmentation detection method and device based on semantic context | |
| CN112613387A (en) | Traffic sign detection method based on YOLOv3 | |
| CN115019274A (en) | Pavement disease identification method integrating tracking and retrieval algorithm | |
| CN114170231A (en) | Image semantic segmentation method and device based on convolutional neural network and electronic equipment | |
| Shao et al. | Semantic segmentation for free space and lane based on grid-based interest point detection | |
| CN117011635A (en) | Model training method, image data processing device and computer equipment | |
| Gao et al. | Robust lane line segmentation based on group feature enhancement | |
| CN113052071B (en) | Method and system for rapidly detecting distraction behavior of driver of hazardous chemical substance transport vehicle | |
| CN114359361A (en) | Depth estimation method, depth estimation device, electronic equipment and computer-readable storage medium | |
| CN113963272A (en) | Unmanned aerial vehicle image target detection method based on improved yolov3 | |
| Cho et al. | Modified perceptual cycle generative adversarial network-based image enhancement for improving accuracy of low light image segmentation | |
| CN117409358A (en) | BiFPN-fused light flame detection method | |
| CN115690787A (en) | Semantic segmentation method, image processing apparatus, and computer-readable storage medium | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| CN111797782B (en) | Vehicle detection method and system based on image features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20210903 Assignee: Jiangsu Kesheng Xuanyi Technology Co.,Ltd. Assignor: HUAIYIN INSTITUTE OF TECHNOLOGY Contract record no.: X2022320000363 Denomination of invention: A Lane Line Detection Method and System Based on LaneSegNet Granted publication date: 20220211 License type: Common License Record date: 20221210 |
|
| EE01 | Entry into force of recordation of patent licensing contract |