CN113989230A - Improved YOLOv 4-based road pavement disease detection method - Google Patents

Improved YOLOv 4-based road pavement disease detection method Download PDF

Info

Publication number
CN113989230A
CN113989230A CN202111261454.0A CN202111261454A CN113989230A CN 113989230 A CN113989230 A CN 113989230A CN 202111261454 A CN202111261454 A CN 202111261454A CN 113989230 A CN113989230 A CN 113989230A
Authority
CN
China
Prior art keywords
detection
road surface
pavement
model
improved
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111261454.0A
Other languages
Chinese (zh)
Inventor
罗晖
李佳敏
吴铭权
李琛彪
蔡联明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
East China Jiaotong University
Original Assignee
East China Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by East China Jiaotong University filed Critical East China Jiaotong University
Priority to CN202111261454.0A priority Critical patent/CN113989230A/en
Publication of CN113989230A publication Critical patent/CN113989230A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a highway pavement disease detection method based on improved YOLOv4, which can quickly detect various types of pavement diseases, trigger a camera to shoot through a mileage sensor, and acquire a highway pavement image in the driving process of a vehicle; the road surface damage detection model based on the improved YOLOv4 is used for detecting the road surface damage in the road surface image. In the CSPDarknet-53 backbone network of YOLOv4, the deep separable convolution is adopted to replace the common convolution, so that the calculation amount of network parameters is reduced. In a loss function calculation stage, a loss function of YOLOv4 is improved based on a Focal local function, and the problem of low detection precision caused by imbalance of positive and negative samples in a network training process is solved; and adding mileage position data to the detection result to form comprehensive information of the pavement diseases. The method provided by the invention can realize automatic and real-time detection of the road pavement diseases, greatly reduce the cost and improve the detection efficiency, and has better detection effect compared with the traditional target detection technology.

Description

Improved YOLOv 4-based road pavement disease detection method
Technical Field
The invention belongs to the field of pavement disease detection, and particularly relates to a highway pavement disease detection method based on improved YOLOv4 deep learning.
Background
As an important infrastructure, a large number of roads are constructed to provide an efficient, convenient and safe transportation mode for economic development. However, during the use of the road, various diseases may occur during the use of the road due to the influence of various natural factors such as the enlargement of vehicles, the overload, the repeated rolling of wheels, ice, rain, snow and the like. If the diseases are not treated in time in the initial stage, the attractiveness of the road is affected, the comfort of driving is reduced, the bearing capacity and the service life of the road are reduced, more serious damage is caused, more maintenance capital investment is caused slightly, and traffic accidents, serious economic loss and casualties are caused seriously. Therefore, the method has great significance for accurately and quickly detecting the road pavement diseases by adopting an effective visual detection method.
The traditional pavement disease detection method is based on manual vision field inspection, has high cost and long time consumption, and cannot meet the requirement of highway development. Meanwhile, the manual detection accuracy is poor, the interference of environmental factors is easy to occur, and the result has subjectivity; when the road surface is detected manually, in order to guarantee the safety of workers, the lanes or the road sections need to be closed frequently, and the use of the road is interfered.
Machine vision is used as a branch of emerging computer science, can effectively identify road surface diseases, and is more efficient and accurate compared with a manual detection method. Deep learning is an important branch of the field of machine vision, and has achieved unprecedented success in image recognition and the like.
The YOLO series is taken as a single-stage algorithm and plays an important role in the field of target detection based on deep learning, particularly real-time detection. Redmon et al in 2016 proposed the YOLOv1 algorithm, which combines the generated candidate box and the classification regression into one network, greatly reducing the complexity of network calculation, but having low target positioning accuracy; redmon et al successively propose the YOLOv2 and YOLOv3 algorithms, improve YOLOv1 from many aspects such as training data set, network structure and anchor point processing, and improve the target detection precision while guaranteeing the detection speed; the YOLOv4 algorithm adds modules such as space pyramid pooling and CSP on the basis of YOLOv3, and the average precision mean value and speed are respectively improved by 10% and 12%; compared with the target detection algorithm based on the regional nomination, the algorithm has slightly low detection accuracy on multi-class and multi-scale targets, but has better real-time performance.
Therefore, in order to meet the increasing detection requirements of road pavement diseases, realize the intelligent real-time detection of the road pavement diseases, ensure the real-time detection of the road pavement diseases and further improve the detection precision, the invention provides the improved YOLOv 4-based intelligent detection method for the road pavement diseases.
Disclosure of Invention
Based on this, it is necessary to provide a road pavement disease detection method based on improved YOLOv4, aiming at the problems that the traditional road pavement disease detection is time-consuming and labor-consuming and has poor result precision and the current situation that the road pavement disease detection method based on deep learning can well realize disease feature learning and feature classification. The highway pavement diseases are divided into three categories of cracks, surface diseases and deformation according to pavement damage types, wherein the crack diseases can be divided into transverse cracks, longitudinal cracks and reticular cracks; surface diseases mainly refer to pits and the like; the deformation diseases comprise ruts, bumps and the like. Common and difficult-to-detect diseases mainly include cracks and pit and groove diseases. On the basis of improving the YOLOv4 algorithm, the invention mainly detects and classifies 4 pavement diseases of transverse cracks, longitudinal cracks, reticular cracks and pits.
In order to achieve the purpose, the invention adopts the following technical scheme:
a highway pavement disease detection method based on improved YOLOv4 comprises the following steps:
(S1) acquiring a road surface image;
(S2) detecting a road surface disease in the road surface image using a road surface disease detection model based on the improved yollov 4;
(S3) outputting all road surface disease detection results, adding mileage position data, and counting the disease information of the whole road surface to form comprehensive road surface disease information.
Preferably, in the method for detecting the road surface damage based on the improved YOLOv4, the linear array or area array camera is vertically installed behind the detected vehicle on the ground (S1), an LED lamp or a laser can be used for supplementary lighting, and the camera is triggered by the mileage sensor to take an image, so as to obtain a road surface image corresponding to mileage; the pavement disease detection method further comprises the following steps:
and respectively executing a pavement disease detection method aiming at the pavement image acquired by each camera.
The preferable road pavement disease detection method based on the improved YOLOv4 provides two improvements to the CSPDarknet53 network framework and the loss function of the YOLOv4 algorithm: firstly, a method of replacing common convolution with deep separable convolution is adopted, so that the number of network parameters is reduced, the calculation complexity is reduced, and the detection speed is improved; and secondly, a Focal local function is introduced, the training speed of the network is improved, the problem of low detection precision caused by unbalance of positive and negative samples in the network training process is solved, and the detection precision of the road pavement diseases is improved while the detection speed is ensured.
(1) YOLOv4 algorithm principle
The YOLOv4 network consists of a CSPDarknet53 backbone network, a spatial pyramid pooling module, a PANET multi-scale feature extraction module and a YOLO Head output layer. The CSPDarknet53 combines jump connection in a residual error network ResNet as a feature extraction network, so that the learning capability of the convolutional neural network is enhanced, the network is lightened, and the detection accuracy is ensured; the Mish activation function is used after batch normalization, so that the characteristic information of the target can be better extracted, and the detection precision of the target is improved; SPP and PANet structural frameworks are used in the characteristic pyramid module; the SPP is used for performing maximum pooling processing of four different scales on the output of the last feature layer of the CSPDarknet53, and the sizes of convolution kernels are respectively 13 × 13, 9 × 9, 5 × 5 and 1 × 1, so that not only can the scale invariance of an image be improved, but also the overfitting phenomenon can be reduced; and after completing the feature extraction of the feature pyramid from bottom to top, the PANet also completes the feature extraction from top to bottom.
In the target detection process based on YOLOv4, firstly, dividing an image into S × S grids, each grid being responsible for predicting a target with a center falling into the grid, and calculating 3 prediction boxes, each prediction box corresponding to (5+ C) values; where C represents the total number of categories in the dataset and 5 represents the center point coordinates (x, y) of the predicted bounding box, the width and height dimensions (w, h) of the box, and confidence parameter information. Then, solving the confidence of the grid prediction class, the probability P (object) that the target falls into the grid, and the accuracy P (class) of the grid prediction i-th class targetiI object), complete intersection ratio is related, and the solving expression is as follows:
Figure BDA0003325891950000021
if the target center falls into the grid, p (object) is 1, otherwise 0;
Figure BDA0003325891950000022
two penalty factors alpha and v are introduced to the distance intersection and comparison between the predicted frame and the real frame. And finally, screening a prediction frame with a higher confidence value by using the DIoUNMS as a target detection frame, and outputting characteristic diagrams with the sizes of 13 × 13, 26 × 26 and 52 × 52 respectively, thereby realizing the positioning and classification of the target.
(2) Improvement of convolution mode
The depth separable convolution method proposed by MobileNet is added to a feature extraction network of CSPDarknet53, and standard convolution operation is decomposed into two processes of depth convolution and point-by-point convolution. Firstly, in the deep convolution process, performing single-channel convolution on an H multiplied by W multiplied by M input images by utilizing M convolution kernels with the size of K multiplied by 1 to obtain the output of H ' multipliedby W ' multipliedby M ' dimension, wherein the parameter calculation quantity C of the deep convolution in the processdThe formula of (b) is shown as:
Cd＝H×W×M×K×K
then, in the process of point-by-point convolution, N convolution kernels with the size of 1 × 1 × M are utilized to perform convolution operation again on the H ' × W ' × M ' dimensional output obtained in the previous step, the output characteristic diagram is H ' × W ' × N, and the parameter calculation quantity C of deep convolution in the process is H ' × W ' × NpThe formula of (b) is shown as:
CP＝H'×W'×M×N
in summary, the parameter computation C of the depth separable convolution is obtained2The formula of (b) is shown as:
C2＝Cd+CP＝H×W×M×K×K+H'×W'×M×N
the ratio of the parameter calculations for the depth separable convolution to the standard convolution is given by:
Figure BDA0003325891950000031
therefore, compared with the standard convolution operation, the parameter calculation amount of the depth separable convolution mode is reduced, and the target detection speed can be improved.
(3) Improvement of loss function
Aiming at the problem of unbalance between positive and negative sample numbers, a Focal loss function is introduced into a loss function of YOLOv 4. The Focal loss function is a loss function improved on the basis of standard cross entropy loss, and is shown as the following formula:
Figure BDA0003325891950000032
wherein y is a real tag; y 'is a predicted value of the label, and y' belongs to [0,1 ]; and gamma is an adjusting parameter and is used for adjusting the weight of the simple sample so as to improve the network training speed, so that the model can better realize the feature learning of the sample, and solve the problems that the larger the output probability of the positive sample is, the smaller the loss is, and the smaller the output probability of the negative sample is, in the standard cross entropy loss, the smaller the loss is, and further improve the target detection precision.
Therefore, the YOLOv4 loss function L' after introducing the Focal loss function is shown as:
Figure BDA0003325891950000033
in the above formula, the number of input image cells is s2(ii) a B is the predicted number of the bounding boxes of each cell;
Figure BDA0003325891950000034
the value of (1) or (0), namely whether a detection target exists in the jth boundary box of the ith cell or not is 1 if the detection target exists, or is 0 if the detection target exists;
Figure BDA0003325891950000035
respectively the center coordinates, height, width of the predicted boundary,
Figure BDA0003325891950000036
the center coordinates, height and width of the actual bounding box are referred to; lambda [ alpha ]objAnd λnoobjIs the weight of the cross-over ratio error;
Figure BDA0003325891950000037
in order to predict the degree of confidence,
Figure BDA0003325891950000038
the actual confidence is calculated in a specific manner as shown in the following formula; pi jIs the actual probability of the category to which the object in the cell belongs,
Figure BDA0003325891950000039
and predicting the probability.
Figure BDA0003325891950000041
Figure BDA0003325891950000042
Wherein alpha is a weight parameter; ν is a parameter for measuring the uniformity of the aspect ratio, and the specific calculation formula is shown as the following formula:
Figure BDA0003325891950000043
preferably, the method for detecting the road pavement diseases based on the improved YOLOv4, wherein the step (S2) specifically includes the following steps:
(S21) training the detection network, comprising the steps of:
(S211) constructing a data set, establishing a road pavement disease data set, and marking the pavement disease types in the data set;
(S212) performing data enhancement on the data set, and performing data set augmentation operations such as turning, cutting, brightness transformation, noise disturbance and the like on the marked road pavement disease sample;
(S213) setting initial training parameters of the detection network as follows: the input image size is 416 × 416, the initial learning rate is 0.001, the learning decay rate is set to 0.0001, the momentum is 0.9, the batch size is set to 8, and the iteration number is limited to 50 k;
(S214) generating an anchor frame required by detecting the network by using a K-means clustering method, and taking the anchor frame as an initial anchor frame;
(S215) predicting a score of each bounding box using logistic regression, thereby predicting a detection target score, each bounding box needing five basic parameters of coordinates (x, y), width and height (w, h) and confidence;
(S216) feature fusion is carried out by utilizing SPP, PANet, down-sampling and up-sampling, and feature maps with three different scales are output;
(S217) aiming at the problem of unbalance of positive and negative sample numbers, introducing a Focal loss function into the loss function of YOLOv 4;
(S218) training the network by adopting a random gradient descent method, and calculating a weight value and a bias value after the convolutional neural network is updated;
(S219) after continuous iteration, stopping training when the loss value of the loss function is not changed any more or is changed little, and storing the parameters of model learning;
(S22) detecting and identifying the road surface diseases, comprising the following steps:
(S221) taking the road surface image acquired by the vehicle-mounted camera as an input image, averagely dividing the image into 13 × 13, 26 × 26 and 52 × 52 grids according to the structure of the detection network model, wherein the sizes of the corresponding grids are 32 × 32, 16 × 16 and 8 × 8, and the grids are taken as down-sampling scales;
(S222) three prior boxes per scale, for a total of nine prior boxes for the three outputs. The down-sampling scale of 13 × 13 is suitable for detecting a large target, and the corresponding prior frames are the maximum three prior frames; 26 × 26 is suitable for detecting the medium-sized targets, and the corresponding prior frames are three prior frames with medium sizes; the down-sampling scale of 52 x 52 is suitable for detecting small targets, and the corresponding prior frames are the minimum three prior frames. Total 13 × 3+26 × 3+52 × 3 — 10647 prediction boxes;
(S223) the vector size dimension corresponding to each prediction frame is a (4+1+ C) -dimension vector, and the vector comprises 4 coordinates (x, y, w, h) of a frame, 1 frame confidence coefficient and the probability corresponding to C types of pavement disease object types;
(S224) screening the prediction frames with lower confidence coefficient according to a threshold value because of more number of the prediction frames and having redundancy prediction frames, and then removing the redundancy prediction frames by utilizing non-maximum value inhibition to respectively obtain the positions of the diseases; through the steps, the model automatically detects and identifies the position of the disease in the image.
Preferably, in the method for detecting road pavement diseases based on improved YOLOv4, the training expected effect criteria of the pavement disease detection model include: detection accuracy, detection speed, and PR curve.
A detection system using the improved YOLOv 4-based road pavement disease detection method is characterized in that: the road surface image acquisition system comprises a road surface image acquisition module (501), a model module (502), a model training module (503), a model reading module (504) and an output module (505);
the pavement image acquisition module (501) is a linear array or area array camera;
the model module (502) is internally loaded with a road surface disease detection model based on improved YOLOv4 and is used for detecting the road surface disease in the road surface image according to the road surface image acquired by the road surface image acquisition module;
the model training module (503) is used for training the pavement damage detection model by using the pavement damage image so as to obtain an optimal pavement damage detection model;
the model reading module (504) is used for receiving the optimal model obtained by training of the model training module, reading parameters in the model and allocating the memory space of the computer;
and the output module (505) is used for outputting all the disease detection results, adding mileage position data, and counting the disease information of the whole lane to form comprehensive road surface disease information.
Preferably, after the model training module (503) is trained, the module is trained and optimized according to preset parameters, and when the loss value of the loss function is not changed or is changed little, the training is stopped and the weight parameters of model learning are saved.
A pavement disease detection vehicle is provided with a detection system of the pavement disease detection method for detecting pavement diseases.
A computer readable medium having stored therein computer software which, when executed by a processor, is capable of implementing the improved YOLOv 4-based road pavement disease detection method.
The invention provides a highway pavement disease detection method based on improved YOLOv 4. Compared with the prior art, the method has the following beneficial effects:
(1) the pavement disease detection method provided by the invention can be used for processing the obtained pavement image by using the pavement disease detection model based on the improved YOLOv4 algorithm, and compared with the existing deep learning algorithm, the method has the advantages that the detection speed is ensured, the target detection precision is improved, the real-time performance is better, and the detection of the pavement disease can be quickly realized.
(2) The method can detect various pavement diseases and detect the multi-scale characteristics of four diseases such as transverse cracks, longitudinal cracks, reticular cracks and pits. The traditional road disease detection algorithm mainly focuses on the diseases of cracks, and in actual use, a large amount of manpower and material resources are needed to perform secondary detection on the rest types of diseases.
Drawings
FIG. 1 is a flow chart of a pavement damage detection method provided by the present invention;
FIG. 2 is a diagram illustrating a conventional standard convolution operation;
FIG. 3 is a schematic diagram of the improved depth separable convolution process of the present invention;
FIG. 4 is a diagram of a YOLOv4 network architecture;
fig. 5 is a block diagram of a detection system of the road surface defect detection method provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a highway pavement disease detection method based on improved YOLOv4, wherein the highway pavement diseases are divided into three categories of cracks, surface diseases and deformation according to pavement damage types, wherein the crack diseases can be divided into transverse cracks, longitudinal cracks and reticular cracks; surface diseases mainly refer to pits and the like; the deformation diseases comprise ruts, bumps and the like. Common and difficult-to-detect diseases mainly include cracks and pit and groove diseases. On the basis of improving the YOLOv4 algorithm, the invention mainly detects and classifies 4 pavement diseases of transverse cracks, longitudinal cracks, reticular cracks and pits. The method specifically comprises the following steps:
(S1) acquiring a road surface image;
specifically, the linear array or area array camera is arranged behind the detection vehicle perpendicular to the ground, an LED lamp or a laser can be used for supplementary lighting, the camera is triggered through the mileage sensor to carry out image shooting, and a road surface image corresponding to mileage is obtained. And respectively executing a pavement disease detection method aiming at the pavement image acquired by each camera.
(S2) detecting a road surface disease in the road surface image using a road surface disease detection model based on the improved yollov 4;
specifically, in the road surface detection method provided by the invention, basic equipment needs to be constructed in advance, and the specific process is as follows:
(1) acquiring a road pavement disease image, preprocessing and marking the image, and constructing a road pavement disease data set;
(2) constructing an improved YOLOv4 deep learning framework, and adopting a VOC2007 data set to pre-train a road pavement disease detection network by means of the idea of transfer learning to obtain a network pre-training weight parameter; then, selecting the established road pavement disease data set to perform parameter fine adjustment on the pre-trained network structure, debugging for multiple times, and obtaining an optimal model in a model training module (503);
(3) transplanting and constructing an optimal YOLOv4 framework model on a model module (502) by using a model reading module (504), and checking whether the model can normally run;
(4) mounting the finished detection system based on the improved YOLOv4 road surface disease detection method on a detection vehicle for actual detection of a road, and judging whether the equipment can achieve an expected detection effect;
(S3) outputting all road surface disease detection results, adding mileage position data, and counting the disease information of the whole road surface to form comprehensive road surface disease information.
As a preferable solution, in this embodiment, two improvements are proposed to the CSPDarknet53 network framework and the loss function of the YOLOv4 algorithm: firstly, a method of replacing common convolution with deep separable convolution is adopted, so that the number of network parameters is reduced, the calculation complexity is reduced, and the detection speed is improved; and secondly, a Focal local function is introduced, the training speed of the network is improved, the problem of low detection precision caused by unbalance of positive and negative samples in the network training process is solved, and the detection precision of the road pavement diseases is provided while the detection speed is ensured.
(1) Improvement of convolution mode
Adding a depth separable convolution method proposed by MobileNet into a feature extraction network of CSPDarknet53, wherein the method decomposes standard convolution operation into two processes of depth convolution and point-by-point convolution; firstly, in the deep convolution process, performing single-channel convolution on an H multiplied by W multiplied by M input images by utilizing M convolution kernels with the size of K multiplied by 1 to obtain the output of H ' multipliedby W ' multipliedby M ' dimension, wherein the parameter calculation quantity C of the deep convolution in the processdThe formula of (b) is shown as:
Cd＝H×W×M×K×K
then, in the process of point-by-point convolution, N convolution kernels with the size of 1 × 1 × M are utilized to perform convolution operation again on the H ' × W ' × M ' dimensional output obtained in the previous step, the output characteristic diagram is H ' × W ' × N, and the parameter calculation quantity C of deep convolution in the process is H ' × W ' × NpThe formula of (b) is shown as:
CP＝H'×W'×M×N
in summary, the parameter computation C of the depth separable convolution is obtained2The formula of (b) is shown as:
C2＝Cd+CP＝H×W×M×K×K+H'×W'×M×N
the ratio of the parameter calculations for the depth separable convolution to the standard convolution is given by:
Figure BDA0003325891950000061
therefore, compared with the standard convolution operation, the parameter calculation amount of the depth separable convolution mode is reduced, and the target detection speed can be improved.
(2) Improvement of loss function
Aiming at the problem of unbalance between positive and negative sample numbers, a Focal loss function is introduced into a loss function of YOLOv 4. The Focal loss function is a loss function improved on the basis of standard cross entropy loss, and is shown as the following formula:
Figure BDA0003325891950000071
wherein y is a real tag; y 'is a predicted value of the label, and y' belongs to [0,1 ]; and gamma is an adjusting parameter and is used for adjusting the weight of the simple sample so as to improve the network training speed, so that the model can better realize the feature learning of the sample, and solve the problems that the larger the output probability of the positive sample is, the smaller the loss is, and the smaller the output probability of the negative sample is, in the standard cross entropy loss, the smaller the loss is, and further improve the target detection precision.
Therefore, the YOLOv4 loss function L' after introducing the Focal loss function is shown as:
Figure BDA0003325891950000072
in the above formula, the number of input image cells is s2(ii) a B is the predicted number of the bounding boxes of each cell;
Figure BDA0003325891950000073
the value of (1) or (0), namely whether a detection target exists in the jth boundary box of the ith cell or not is 1 if the detection target exists, or is 0 if the detection target exists;
Figure BDA0003325891950000074
respectively the center coordinates, height, width of the predicted boundary,
Figure BDA0003325891950000075
the center coordinates, height and width of the actual bounding box are referred to; lambda [ alpha ]objAnd λnoobjIs the weight of the cross-over ratio error;
Figure BDA0003325891950000076
in order to predict the degree of confidence,
Figure BDA0003325891950000077
the actual confidence is calculated in a specific manner as shown in the following formula; pi jIs the actual probability of the category to which the object in the cell belongs,
Figure BDA0003325891950000078
and predicting the probability.
Figure BDA0003325891950000079
Figure BDA00033258919500000710
Wherein alpha is a weight parameter; ν is a parameter for measuring the uniformity of the aspect ratio, and the specific calculation formula is shown as the following formula:
Figure BDA00033258919500000711
preferably, in this embodiment, the step (S2) of the method for detecting a road pavement disease based on the improved YOLOv4 specifically includes the following steps:
(S21) training the detection network, comprising the steps of:
(S211) constructing a data set, establishing a road pavement disease data set, and marking the pavement disease types in the data set;
(S212) performing data enhancement on the data set, and performing data set augmentation operations such as turning, cutting, brightness transformation, noise disturbance and the like on the marked road pavement disease sample;
(S213) setting initial training parameters of the detection network as follows: the input image size is 416 × 416, the initial learning rate is 0.001, the learning decay rate is set to 0.0001, the momentum is 0.9, the batch size is set to 8, and the iteration number is limited to 50 k;
(S214) generating an anchor frame required by detecting the network by using a K-means clustering method, and taking the anchor frame as an initial anchor frame;
(S215) predicting a score of each bounding box using logistic regression, thereby predicting a detection target score, each bounding box needing five basic parameters of coordinates (x, y), width and height (w, h) and confidence;
(S216) feature fusion is carried out by utilizing SPP, PANet, down-sampling and up-sampling, and feature maps with three different scales are output;
(S217) aiming at the problem of unbalance of positive and negative sample numbers, introducing a Focal loss function into the loss function of YOLOv 4;
(S218) training the network by adopting a random gradient descent method, and calculating a weight value and a bias value after the convolutional neural network is updated;
(S219) after continuous iteration, stopping training when the loss value of the loss function is not changed any more or is changed little, and storing the parameters of model learning;
(S22) detecting and identifying the road surface diseases, comprising the following steps:
(S221) taking the road surface image acquired by the vehicle-mounted camera as an input image, averagely dividing the image into 13 × 13, 26 × 26 and 52 × 52 grids according to the structure of the detection network model, wherein the sizes of the corresponding grids are 32 × 32, 16 × 16 and 8 × 8, and the grids are taken as down-sampling scales;
(S222) three prior boxes per scale, for a total of nine prior boxes for the three outputs. The down-sampling scale of 13 × 13 is suitable for detecting a large target, and the corresponding prior frames are the maximum three prior frames; 26 × 26 is suitable for detecting the medium-sized targets, and the corresponding prior frames are three prior frames with medium sizes; the down-sampling scale of 52 x 52 is suitable for detecting small targets, and the corresponding prior frames are the minimum three prior frames. Total 13 × 3+26 × 3+52 × 3 — 10647 prediction boxes;
(S223) 4 types of road surface damage categories are included, so that the vector size dimension corresponding to each prediction frame is (4+1+4) ═ 9-dimensional vector, and the vector includes 4 coordinates (x, y, w, h) of the frame, 1 frame confidence level, and probabilities corresponding to 4 types of road surface damage object categories;
(S224) redundant prediction frames exist due to the fact that the number of the prediction frames is large, the prediction frames with low confidence coefficient are selected according to the threshold value, then the redundant prediction frames are restrained and removed by utilizing the non-maximum value, and the positions of the diseases can be obtained respectively; through the steps, the model automatically detects and identifies the position of the disease in the image.
As a preferable scheme, in this embodiment, the expected effect training standard of the pavement damage detection model includes: detection accuracy, detection speed, and PR curve.
Specifically, before training the improved YOLOv4 network, preprocessing the image in the pavement disease database should be performed, where the preprocessing includes labeling the pavement picture disease as a rectangular area box using a label program. And then the pavement disease database is divided into 6 parts: 2: the proportion of 2 is divided into a training set, a verification set and a test set which are not overlapped and crossed with each other.
Correspondingly, as shown in fig. 5, the invention further discloses a detection system of a road pavement disease detection method based on the improved YOLOv4, which comprises a pavement image acquisition module 501, a model module 502, a model training module 503, a model reading module 504 and an output module 505;
the road surface image acquisition module 501 is a linear array or area array camera, specifically, the camera is installed at the rear of the vehicle to be detected, an LED lamp or a laser is used for supplementary lighting, and the camera is triggered by a mileage sensor to take an image, so as to acquire a road surface image corresponding to mileage. Respectively executing a pavement disease detection method aiming at the pavement image acquired by each camera;
the model module 502 is internally loaded with a road surface disease detection model based on the improved YOLOv4, and is used for judging the road surface disease in the image according to the road surface image acquired by the road surface image acquisition module; specifically, the stage mainly comprises a forward data transmission stage, wherein a road surface image which is actually acquired is input into a network, three feature layers are obtained through a trunk feature extraction network and an enhanced feature extraction network, and feature extraction is effectively realized by using multiple convolution up-sampling and convolution down-sampling, so that three output of Yoloideal are finally obtained;
the model training module 503 is configured to train a pavement damage detection model by using a pavement damage image, so as to obtain an optimal pavement damage detection model; in particular, the method comprises the following steps of,
(1) setting initial training parameters of the detection network as follows: the input image size is 416 × 416, the initial learning rate is 0.001, the learning decay rate is set to 0.0001, the momentum is 0.9, the batch size is set to 8, and the iteration number is limited to 50 k;
(2) generating an anchor frame required by checking the network by using a K-means clustering method, and taking the anchor frame as an initial anchor frame;
(3) predicting the score of each bounding box by using logistic regression so as to predict the score of the detection target, wherein each bounding box needs five basic parameters of coordinates (x, y), width and height (w, h) and confidence coefficient;
(4) performing feature fusion by using SPP, PANET, down-sampling and up-sampling, and outputting feature maps with three different scales;
(5) aiming at the problem of unbalance of positive and negative sample numbers, a Focal loss function is introduced into a loss function of YOLOv 4;
(6) training the network by adopting a random gradient descent method, and calculating a weight value and a bias value after the convolutional neural network is updated;
(7) after 1000 iterations, the loss value begins to decrease greatly, the loss curve gradually tends to be stable after 6000 iterations, and the trained model is stored;
the model reading module 504 is configured to receive the optimal model obtained from the training of the model training module, read parameters in the model, and allocate a memory space of the computer;
and the output module 505 is configured to output all the disease detection results, attach mileage position data, and count the disease information of the whole lane to form road surface disease comprehensive information.
Correspondingly, the invention also provides a pavement disease detection vehicle, and a detection system loaded with the pavement disease detection method is used for detecting pavement diseases.
Correspondingly, the invention also provides a computer readable medium, in which computer software is stored, and when the software is executed by a processor, the method for detecting the road pavement disease based on the improved YOLOv4 can be realized.
And those not described in detail in this specification are well within the skill of those in the art.
Although embodiments of the present invention have been shown and described, it is not intended to limit the scope of the invention, and thus the invention is not to be limited to the specific embodiments disclosed herein, but to include all technical aspects which fall within the scope of the claims. It will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. A road pavement disease detection method based on improved YOLOv4 is characterized by comprising the following steps:
(S1) acquiring a road surface image;
(S2) detecting a road surface disease in the road surface image using a road surface disease detection model based on the improved yollov 4;
(S3) outputting all road surface disease detection results, adding mileage position data, and counting the disease information of the whole road surface to form comprehensive road surface disease information.
2. The improved YOLOv 4-based road surface disease detection method according to claim 1, wherein the step (S1) is implemented by vertically installing a linear or area camera behind the vehicle to be detected, triggering camera shooting through a mileage sensor, and acquiring a road surface image corresponding to mileage; the pavement disease detection method further comprises the following steps:
and respectively executing a pavement disease detection method aiming at the pavement image acquired by each camera.
3. The method for detecting the road pavement diseases based on the improved YOLOv4 as claimed in claim 1, wherein the step (S2) specifically comprises the following steps:
(S21) training the detection network, comprising the steps of:
(S211) constructing a data set, establishing a road pavement disease data set, and marking the pavement disease types in the data set;
(S212) performing data enhancement on the data set, and performing data set augmentation operations such as turning, cutting, brightness transformation, noise disturbance and the like on the marked road pavement disease sample;
(S213) setting initial training parameters of the detection network as follows: the input image size is 416 × 416, the initial learning rate is 0.001, the learning decay rate is set to 0.0001, the momentum is 0.9, the batch size is set to 8, and the iteration number is limited to 50 k;
(S214) generating an anchor frame required by detecting the network by using a K-means clustering method, and taking the anchor frame as an initial anchor frame;
(S215) predicting a score of each bounding box using logistic regression, thereby predicting a detection target score, each bounding box needing five basic parameters of coordinates (x, y), width and height (w, h) and confidence;
(S216) feature fusion is carried out by utilizing SPP, PANet, down-sampling and up-sampling, and feature maps with three different scales are output;
(S217) aiming at the problem of unbalance of positive and negative sample numbers, introducing a Focal loss function into the loss function of YOLOv 4;
(S218) training the network by adopting a random gradient descent method, and calculating a weight value and a bias value after the convolutional neural network is updated;
(S219) after continuous iteration, stopping training when the loss value of the loss function is not changed any more or is changed little, and storing the parameters of model learning;
(S22) detecting and identifying the road surface diseases, comprising the following steps:
(S221) taking the road surface image acquired by the vehicle-mounted camera as an input image, averagely dividing the image into 13 × 13, 26 × 26 and 52 × 52 grids according to the structure of the detection network model, wherein the sizes of the corresponding grids are 32 × 32, 16 × 16 and 8 × 8, and the grids are taken as down-sampling scales;
(S222) each scale corresponds to three prior frames, and the total of three outputs is nine prior frames; the down-sampling scale of 13 × 13 is suitable for detecting a large target, and the corresponding prior frames are the maximum three prior frames; 26 × 26 is suitable for detecting the medium-sized targets, and the corresponding prior frames are three prior frames with medium sizes; the down-sampling scale of 52 × 52 is suitable for detecting small targets, and the corresponding prior frames are the minimum three prior frames, and the total number of the prior frames is 13 × 3+26 × 3+52 × 3 — 10647 prediction frames;
(S223) the vector size dimension corresponding to each prediction frame is a (4+1+ C) -dimension vector, and the vector comprises 4 coordinates (x, y, w, h) of a frame, 1 frame confidence coefficient and the probability corresponding to C types of pavement disease object types;
(S224) screening the prediction frames with lower confidence coefficient according to a threshold value because of more number of the prediction frames and having redundancy prediction frames, and then removing the redundancy prediction frames by utilizing non-maximum value inhibition to respectively obtain the positions of the diseases; through the steps, the model automatically detects and identifies the position of the disease in the image.
4. The improved YOLOv 4-based road pavement disease detection method according to claim 3, wherein the standard convolution mode in the YOLOv4 detection network is improved, and specifically:
adding a depth separable convolution method into a feature extraction network of CSPDarknet53, wherein the method decomposes standard convolution operation into two processes of depth convolution and point-by-point convolution; firstly, in the deep convolution process, performing single-channel convolution on an H multiplied by W multiplied by M input images by utilizing M convolution kernels with the size of K multiplied by 1 to obtain the output of H ' multipliedby W ' multipliedby M ' dimension, wherein the parameter calculation quantity C of the deep convolution in the processdThe formula of (b) is shown as:
Cd＝H×W×M×K×K
then, in the process of point-by-point convolution, N convolution kernels with the size of 1 × 1 × M are utilized to perform convolution operation again on the H ' × W ' × M ' dimensional output obtained in the previous step, the output characteristic diagram is H ' × W ' × N, and the parameter calculation quantity C of deep convolution in the process is H ' × W ' × NpThe formula of (b) is shown as:
CP＝H'×W'×M×N
in summary, the parameter computation C of the depth separable convolution is obtained2The formula of (b) is shown as:
C2＝Cd+CP＝H×W×M×K×K+H'×W'×M×N
the ratio of the parameter calculations for the depth separable convolution to the standard convolution is given by:
Figure FDA0003325891940000021
therefore, compared with the standard convolution operation, the parameter calculation amount of the depth separable convolution mode is reduced, and the target detection speed can be improved.
5. The improved YOLOv 4-based road pavement disease detection method according to claim 3, wherein the loss function in the YOLOv4 detection network is improved, and a Focal loss function is introduced into the loss function, and specifically comprises the following steps:
the Focal loss function is a loss function improved on the basis of a standard cross entropy loss function, and is shown as the following formula:
Figure FDA0003325891940000022
wherein y is a real tag; y 'is a predicted value of the label, and y' belongs to [0,1 ]; gamma is an adjusting parameter and is used for adjusting the weight of a simple sample so as to improve the network training speed, so that the model can better realize the feature learning of the sample, and solve the problems that the larger the output probability of a positive sample is, the smaller the loss is, and the smaller the output probability of a negative sample is, in the standard cross entropy loss, the smaller the loss is, so that the target detection precision is improved;
therefore, the YOLOv4 loss function L' after introducing the Focal loss function is shown as the following formula.
Figure FDA0003325891940000023
6. The improved YOLOv 4-based road pavement disease detection method according to claim 3, wherein the training expected effect criteria of the pavement disease detection model comprise: detection accuracy, detection speed, and PR curve.
7. A detection system of the improved YOLOv 4-based road pavement disease detection method according to any one of claims 1-6, characterized in that: the road surface image acquisition system comprises a road surface image acquisition module (501), a model module (502), a model training module (503), a model reading module (504) and an output module (505);
the pavement image acquisition module (501) is a linear array or area array camera;
the model module (502) is internally loaded with a road surface disease detection model based on improved YOLOv4 and is used for detecting the road surface disease in the road surface image according to the road surface image acquired by the road surface image acquisition module;
the model training module (503) is used for training the pavement damage detection model by using the pavement damage image so as to obtain an optimal pavement damage detection model;
the model reading module (504) is used for receiving the optimal model obtained by training of the model training module, reading parameters in the model and allocating the memory space of the computer;
and the output module (505) is used for outputting all the disease detection results, adding mileage position data, and counting the disease information of the whole lane to form comprehensive road surface disease information.
8. The detection system of claim 7, wherein the detection system is based on the improved YOLOv4 road pavement disease detection method, and is characterized in that: after the model training module (503) trains, the module will train and optimize according to the preset parameters, and when the loss value of the loss function is not changed any more or is changed little, the training is stopped and the weight parameters of the model learning are saved.
9. A road surface disease detection vehicle, characterized in that a detection system of the road surface disease detection method according to any one of claims 7 to 8 is loaded for detecting road surface diseases.
10. A computer readable medium, wherein computer software is stored, and when executed by a processor, the software can implement the method for detecting road pavement diseases based on improved YOLOv4 as claimed in any one of claims 1-6.
CN202111261454.0A 2021-10-28 2021-10-28 Improved YOLOv 4-based road pavement disease detection method Pending CN113989230A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111261454.0A CN113989230A (en) 2021-10-28 2021-10-28 Improved YOLOv 4-based road pavement disease detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111261454.0A CN113989230A (en) 2021-10-28 2021-10-28 Improved YOLOv 4-based road pavement disease detection method

Publications (1)

Publication Number Publication Date
CN113989230A true CN113989230A (en) 2022-01-28

Family

ID=79743181

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111261454.0A Pending CN113989230A (en) 2021-10-28 2021-10-28 Improved YOLOv 4-based road pavement disease detection method

Country Status (1)

Country Link
CN (1) CN113989230A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114267003A (en) * 2022-03-02 2022-04-01 城云科技（中国）有限公司 Road damage detection method, device and application
CN114596335A (en) * 2022-03-01 2022-06-07 广东工业大学 Unmanned ship target detection tracking method and system
CN114998220A (en) * 2022-05-12 2022-09-02 湖南中医药大学 Tongue image detection and positioning method based on improved Tiny-YOLO v4 natural environment
CN115063679A (en) * 2022-06-15 2022-09-16 电子科技大学 Pavement quality assessment method based on deep learning
CN115424232A (en) * 2022-11-04 2022-12-02 深圳市城市交通规划设计研究中心股份有限公司 Method for identifying and evaluating pavement pit, electronic equipment and storage medium
CN115527109A (en) * 2022-08-29 2022-12-27 邯郸市亿润工程咨询有限公司 Underwater concrete disease monitoring method and device, underwater robot and medium
CN115619777A (en) * 2022-12-05 2023-01-17 中国民航大学 Method and system for detecting ice and snow state of road surface and computer readable storage medium
CN116310785A (en) * 2022-12-23 2023-06-23 兰州交通大学 Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN116343028A (en) * 2023-03-05 2023-06-27 理工雷科智途(北京)科技有限公司 Road surface detection method and device
CN116485843A (en) * 2023-05-24 2023-07-25 江苏全栈智能科技有限公司 Road disease identification tracking method
CN116681657A (en) * 2023-05-18 2023-09-01 中南大学 Asphalt pavement disease detection method based on improved YOLOv7 model

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596335B (en) * 2022-03-01 2023-10-31 广东工业大学 Unmanned ship target detection tracking method and system
CN114596335A (en) * 2022-03-01 2022-06-07 广东工业大学 Unmanned ship target detection tracking method and system
CN114267003A (en) * 2022-03-02 2022-04-01 城云科技（中国）有限公司 Road damage detection method, device and application
CN114998220A (en) * 2022-05-12 2022-09-02 湖南中医药大学 Tongue image detection and positioning method based on improved Tiny-YOLO v4 natural environment
CN115063679A (en) * 2022-06-15 2022-09-16 电子科技大学 Pavement quality assessment method based on deep learning
CN115063679B (en) * 2022-06-15 2024-04-26 电子科技大学 Pavement quality assessment method based on deep learning
CN115527109A (en) * 2022-08-29 2022-12-27 邯郸市亿润工程咨询有限公司 Underwater concrete disease monitoring method and device, underwater robot and medium
CN115424232A (en) * 2022-11-04 2022-12-02 深圳市城市交通规划设计研究中心股份有限公司 Method for identifying and evaluating pavement pit, electronic equipment and storage medium
CN115619777B (en) * 2022-12-05 2023-03-17 中国民航大学 Method and system for detecting ice and snow state of road surface and computer readable storage medium
CN115619777A (en) * 2022-12-05 2023-01-17 中国民航大学 Method and system for detecting ice and snow state of road surface and computer readable storage medium
CN116310785A (en) * 2022-12-23 2023-06-23 兰州交通大学 Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN116310785B (en) * 2022-12-23 2023-11-24 兰州交通大学 Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN116343028A (en) * 2023-03-05 2023-06-27 理工雷科智途(北京)科技有限公司 Road surface detection method and device
CN116681657A (en) * 2023-05-18 2023-09-01 中南大学 Asphalt pavement disease detection method based on improved YOLOv7 model
CN116681657B (en) * 2023-05-18 2024-03-22 中南大学 Asphalt pavement disease detection method based on improved YOLOv7 model
CN116485843A (en) * 2023-05-24 2023-07-25 江苏全栈智能科技有限公司 Road disease identification tracking method

Similar Documents

Publication Publication Date Title
CN113989230A (en) Improved YOLOv 4-based road pavement disease detection method
CN111444821B (en) Automatic identification method for urban road signs
US20220196459A1 (en) Real-time vehicle overload detection method based on convolutional neural network
CN111476756B (en) Method for identifying casting DR image loosening defect based on improved YOLOv network model
CN109977817B (en) Motor train unit bottom plate bolt fault detection method based on deep learning
CN110415233A (en) Pavement crack rapid extracting method based on two step convolutional neural networks
CN108509954A (en) A kind of more car plate dynamic identifying methods of real-time traffic scene
CN109508715A (en) A kind of License Plate and recognition methods based on deep learning
CN111553201A (en) Traffic light detection method based on YOLOv3 optimization algorithm
CN109034024B (en) Logistics vehicle type classification and identification method based on image target detection
CN112464911A (en) Improved YOLOv 3-tiny-based traffic sign detection and identification method
CN116310785B (en) Unmanned aerial vehicle image pavement disease detection method based on YOLO v4
CN104156734A (en) Fully-autonomous on-line study method based on random fern classifier
CN111274886B (en) Deep learning-based pedestrian red light running illegal behavior analysis method and system
CN114937033A (en) Rural highway pavement disease intelligent detection method based on deep convolutional neural network
CN111460919A (en) Monocular vision road target detection and distance estimation method based on improved YO L Ov3
CN113918538B (en) New road maintenance data migration system based on artificial neural network
CN110599459A (en) Underground pipe network risk assessment cloud system based on deep learning
CN110991523A (en) Interpretability evaluation method for unmanned vehicle detection algorithm performance
CN114202572A (en) Vehicle load measuring and transportation route planning method and system based on machine vision
CN115376101A (en) Incremental learning method and system for automatic driving environment perception
CN115170611A (en) Complex intersection vehicle driving track analysis method, system and application
CN111369053B (en) Method for predicting tracks of pedestrians and vehicles in human-vehicle interaction scene
CN116824543A (en) Automatic driving target detection method based on OD-YOLO
Yasir et al. Faster metallic surface defect detection using deep learning with channel shuffling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination