CN116229292A - Inspection system and method based on unmanned aerial vehicle road surface inspection disease - Google Patents
Inspection system and method based on unmanned aerial vehicle road surface inspection disease Download PDFInfo
- Publication number
- CN116229292A CN116229292A CN202310071309.9A CN202310071309A CN116229292A CN 116229292 A CN116229292 A CN 116229292A CN 202310071309 A CN202310071309 A CN 202310071309A CN 116229292 A CN116229292 A CN 116229292A
- Authority
- CN
- China
- Prior art keywords
- frame
- unmanned aerial
- feature map
- aerial vehicle
- inspection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 70
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000007689 inspection Methods 0.000 title claims description 57
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000000605 extraction Methods 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 230000004927 fusion Effects 0.000 claims description 55
- 230000007547 defect Effects 0.000 claims description 20
- 238000013138 pruning Methods 0.000 claims description 18
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000012549 training Methods 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 239000012633 leachable Substances 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000013139 quantization Methods 0.000 claims 3
- 238000010586 diagram Methods 0.000 description 20
- 230000006870 function Effects 0.000 description 20
- 238000004364 calculation method Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 5
- 238000011176 pooling Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004873 anchoring Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000002708 enhancing effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013434 data augmentation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 239000010426 asphalt Substances 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000004568 cement Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/176—Urban or other man-made structures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a system and a method for inspecting diseases based on unmanned aerial vehicle road surfaces, wherein the system at least comprises an image acquisition unit (10) and at least one processing unit (20), wherein the image acquisition unit (10) and the at least one processing unit (20) can be mounted on an unmanned aerial vehicle, the image acquisition unit (10) and the at least one processing unit (20) are connected in a wired and/or wireless mode, and the processing unit is configured to: -classifying pre-processing the first image dataset transmitted by the image acquisition unit (10) to form a second image dataset; calculating and determining an initial anchor frame based on the second image dataset; processing the second image dataset based on a feature extraction model to extract at least one standard feature map; and generating a target prediction frame of the standard feature map based on a prediction model. The method can realize the rapid detection of the unmanned aerial vehicle on the road surface diseases on the basis of improving the road disease identification accuracy.
Description
Technical Field
The invention relates to the technical field of road surface inspection, in particular to an inspection system and method based on unmanned aerial vehicle road surface inspection diseases.
Background
The highway construction development of China is rapid, carries important mission of the transportation center, and plays an important role in the national economic and social development. Along with the accumulation of the road service time, the number of vehicles and the trip rate rise year by year to damage the road surface, and the road surface diseases increase year by year. Road diseases shorten the service life of the road and greatly reduce the driving comfort and driving conditions of drivers, so that road detection technology is very necessary in the current society.
The road disease detection mode is implemented by manual actual measurement, manual video stream detection, road detection vehicle detection, unmanned aerial vehicle inspection and the like; the road disease detection technology is subjected to the technologies of manual identification, computer image processing, laser radar and the like. The unmanned aerial vehicle inspection mode can not only effectively acquire road surface information with lower cost, but also realize parallel detection; the angle of the unmanned aerial vehicle can be adjusted to expand the road surface detection visual angle, and the scene understanding capability of road surface diseases is expanded. The manual detection is naturally convenient through a large amount of data that unmanned aerial vehicle collected, but there is certain subjectivity, and computer image processing technique can process a large amount of image data convenient and fast.
Existing algorithms can be categorized into traditional image processing algorithms based on data analysis and machine vision algorithms based on deep learning. In the conventional image processing algorithm, document Image Based Automatic Road Surface Crack Detection for Achieving Smooth Driving on Deformed Roads calculates variance for each divided image sample, extracts road crack disease by adopting a discriminant analysis method, and obtains classification weight and threshold value for detection. The detection rate of the method reaches 94%, and the effect is good; but can only be used to detect a single road disease and is not suitable for road detection situations in complex environments. The document Assessing severity of road cracks using deep learning-based segmentation and detection discloses a pavement management system PMS for estimating the severity of pavement cracks, combines three deep learning algorithms of SquezeNet, U-Net and Mobilene-SSD, improves the accuracy of serious estimation of linear cracks to 94.39%, and also shows higher accuracy for crack type splitting. However, this system is too large, is not suitable for lightweight detection of unmanned aerial vehicles, and is largely affected by illumination.
The problems that the size and the position of a target are various, the background is complex, the target and the background are mutually fused and are not easy to detect and the like generally exist in the road surface image under unmanned aerial vehicle inspection, and the traditional computer image processing and machine learning classification cannot meet the detection speed and precision requirements of a large number of road disease images. Along with the development of deep learning, the target detection algorithm can realize rapid and efficient learning and fitting of a large amount of data, and provides a high-precision real-time monitoring technology for unmanned aerial vehicle inspection of pavement diseases.
For example, chinese patent publication No. CN115294040a discloses a road surface crack detection method based on DDRNets, which includes: determining a patrol area and acquiring barrier information of the patrol area; constructing a patrol environment map according to the obstacle information and a preset probability route map PRM algorithm; carrying out path planning according to the inspection environment map and a preset ant colony algorithm to obtain a target inspection path of the unmanned aerial vehicle; carrying out inspection according to the target inspection path and shooting an image to be detected; inputting the image to be detected into a pre-trained DDRNets model for image segmentation processing to obtain a segmented image; and determining a pavement crack detection result according to the segmentation image. The embodiment of the invention can be combined with the unmanned aerial vehicle and the DDRNets to detect the pavement cracks in the inspection area, and compared with the traditional manual inspection method, the technical scheme of the embodiment of the invention can effectively improve the pavement crack detection efficiency.
For example, chinese patent publication No. CN114415708A discloses a road self-inspection method, which is applied to an unmanned aerial vehicle, the unmanned aerial vehicle includes an unmanned aerial vehicle body and a pan-tilt camera, and the method includes: after the unmanned aerial vehicle body enters a stable flight state, controlling a cradle head camera to acquire road images of the inspected roads according to a set capturing period; carrying out flight route correction by carrying out recognition processing on each road image, and controlling the unmanned aerial vehicle body to fly according to the corrected flight route; when a vehicle inspection instruction is received, detecting driving behaviors of vehicles in inspected roads according to each road image; and returning to execute the collection operation of the road image again until the flight end point is reached. According to the invention, through the identification processing of each road image, the flight route correction is carried out, and the driving behavior detection is carried out on the vehicles in the inspected road according to each road image, so that the inspection efficiency is improved, the inspection visual field range is wide, and the limitation of road traffic conditions is avoided.
Currently, target detection algorithm research based on deep learning is roughly classified into two categories: first, a dual-stage target detection algorithm, such as Faster-RCNN; one is a single-stage object detection algorithm, such as SSD and the YOLO series. The double-stage target detection algorithm firstly acquires candidate frames, and secondly classifies the candidate frames, and although the accuracy is higher, the speed is slower, so that the requirement of real-time detection cannot be met; the reasoning speed of a single-stage target detection algorithm such as YOLOv5 is improved by 3 times compared with that of a fast-RCNN, mAP is improved by 0.34%, and the method has stronger application benefit.
Based on unmanned aerial vehicle equipment inspection road surface diseases, the current YOLOv5 algorithm has the following three defects: firstly, the pavement disease targets have large difference in size and aspect ratio, and the precision is reduced when an initial anchoring frame is generated; secondly, the unmanned aerial vehicle has low cruising duration, and the efficiency of single flight is improved by improving the reasoning speed; thirdly, the pavement disease target is similar to the background, and the false detection rate is high.
Therefore, the invention hopes to provide a system and a method for inspecting the pavement inspection disease based on the unmanned aerial vehicle, and the system and the method improve the characteristic extraction network based on the YOLOv5 algorithm, so that the identification precision of the pavement inspection disease is improved; and pruning is carried out on the model, the model is light, and the reasoning speed is improved.
Furthermore, there are differences in one aspect due to understanding to those skilled in the art; on the other hand, since the applicant has studied a lot of documents and patents while making the present invention, the text is not limited to details and contents of all but it is by no means the present invention does not have these prior art features, but the present invention has all the prior art features, and the applicant remains in the background art to which the right of the related prior art is added.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a patrol system based on unmanned aerial vehicle road surface patrol disease, which at least comprises an image acquisition unit and at least one processing unit, wherein the image acquisition unit and the at least one processing unit can be mounted on an unmanned aerial vehicle, the image acquisition unit and the at least one processing unit are connected in a wired and/or wireless mode, and the processing unit is configured to: performing classification preprocessing on the first image data set sent by the image acquisition unit to form a second image data set; calculating and determining an initial anchor frame based on the second image dataset; processing the second image dataset based on a feature extraction model to extract at least one standard feature map; and generating a target prediction frame of the standard feature map based on a prediction model.
Compared with the prior art, the invention has the following beneficial effects:
aiming at the problems that the target characteristics of the pavement diseases under the view angle of the unmanned aerial vehicle are disordered and are easy to be detected by omission, a DIoU-kmeans++ algorithm is used for calculating a 3-layer anchoring frame, and 1-DIoU is adopted as a distance, so that the condition that the pavement disease targets are relatively close can be considered; the problem that the initialization state of the clustering center is divided into a cluster due to large size difference of targets and finally the global minimum value cannot be converged is solved to a certain extent. Introducing an ELANB network into the small target feature layer, enhancing the fusion of features in different convolution layers, and improving the feature extraction of the small target; and a coordinated attention mechanism CA layer is introduced into the ELANB network, and the position information of the target diseases is embedded into the channel attention, so that the parameter quantity is reduced, and the feature extraction of the target is improved. Using SD-NMS algorithm, using DIoU as the detection omission caused by the closer distance of pavement disease targets (DIoU can find out a plurality of similar pavement disease target defects); the improved NMS algorithm improves the positioning accuracy of the detection frame through confidence estimation. Finally, the model is pruned, so that the model is light and suitable for industrial deployment.
Preferably, the method for processing the second image dataset to extract at least one standard feature map based on a feature extraction model at least comprises: processing the second image data set based on a first feature fusion module to extract a pavement disease feature map of at least one scale; and extracting a standard feature map from the pavement disease feature map of at least one scale based on a second feature fusion module.
Preferably, the method for establishing the feature extraction model at least comprises the following steps: to introduce a learnable reconstruction parameter and L 1 Pruning is carried out on the YOLOv5 algorithm model in a regular constraint mode, a back network module in the YOLOv5 algorithm model after pruning is updated to be a first feature fusion module, a Neck network module in the YOLOv5 algorithm model after pruning is updated to be a second feature fusion module, and the first feature fusion module transmits the extracted pavement disease feature map with at least one scale to the second feature fusion module for processing.
Preferably, the method of calculating and determining an initial anchor frame based on the second image dataset comprises at least: randomly selecting a wide and high array of rectangular frame sample points from a second image data set with category labels as a first initial clustering center c i The method comprises the steps of carrying out a first treatment on the surface of the Calculating the shortest distance between the rectangular frame sample points and the current clustering center; calculating the probability that the sample points of the real frame are selected as the next clustering center, and selecting the sample points of the rectangular frame corresponding to the maximum probability value as the next clustering center; repeatedly selecting cluster centers until k cluster centers are selected, wherein k is a positive integer not less than 1; calculating distances from the sample points of the real frames to the clustering centers of k anchor frames to obtain an error matrix; selecting an anchor frame corresponding to the minimum error, classifying the current real frame into the anchor frame, namely classifying the sample points of the current real frame into the nearest aggregateA class center; recording a real frame corresponding to each anchor frame; the accuracy of the anchor frame is calculated based on the distance-to-intersection ratio function.
Preferably to introduce a learnable reconstruction parameter and L 1 The step of pruning and quantifying the YOLOv5 algorithm model in a regular constraint mode at least comprises the following steps: calculating all sample elements x in the input feature map i Mean mu and variance sigma of (x) for all sample elements x i Standardization treatment; carrying out translation and scaling treatment on the standardized sample elements, and introducing the learnable reconstruction parameters gamma and beta; affine transformation is carried out by utilizing the leachable reconstruction parameters gamma and beta to obtain output characteristics y after batch regularization i The method comprises the steps of carrying out a first treatment on the surface of the Increasing L on the gradient of the learnable reconstruction parameters alpha and beta 1 And (3) regularly constraining the gradient to form a sparse learnable reconstruction parameter.
Preferably, the first feature fusion module comprises an efficient feature fusion network model; the second feature fusion module comprises a high-efficiency feature fusion network model; the structure of the efficient feature fusion network model at least comprises: a number of conventional convolution layers and a coordinated attention mechanism function; the conventional convolution layers include a CBS layer, a batch normalization function, and a loss function.
Preferably, the method for generating the target prediction frame of the standard feature map based on the prediction model at least comprises the following steps: minimizing a normalized distance between the predicted frame and the target frame based on the distance intersection ratio function as a regression loss function of the rectangular frame for labeling the defect, and generating at least one target detection frame; and detecting the target detection frame of the road surface diseases based on an SD-NMS algorithm to obtain the target detection frame with the highest confidence coefficient, and finally obtaining the target detection frame for the unmanned aerial vehicle inspection of the road surface diseases.
Preferably, the method for establishing the feature extraction model further comprises the following steps: and inputting the second image data set into the feature extraction model to carry out sparse training, and pruning the YOLOv5 algorithm model according to the parameter statistical result and the threshold value.
The invention also provides a patrol method based on the unmanned aerial vehicle road surface patrol disease, which at least comprises the following steps: performing classification preprocessing on the first image data set sent by the image acquisition unit to form a second image data set; calculating and determining an initial anchor frame based on the second image dataset; processing the second image dataset based on a feature extraction model to extract at least one standard feature map; and generating a target prediction frame of the standard feature map based on a prediction model.
Preferably, the method further comprises: processing the second image data set based on a first feature fusion module to extract a pavement disease feature map of at least one scale; and extracting a standard feature map from the pavement disease feature map of at least one scale based on a second feature fusion module.
Drawings
FIG. 1 is a simplified schematic diagram of a method for inspecting a disease based on an unmanned aerial vehicle road surface;
fig. 2 is a network structure schematic diagram of an inspection system based on unmanned aerial vehicle road surface inspection diseases, which is provided by the invention;
FIG. 3 is a schematic diagram of an ELANB network structure provided by the present invention;
FIG. 4 is a schematic diagram of an ELANA network architecture provided by the present invention;
FIG. 5 is a Neck network structure diagram of the improved Yolov5 algorithm provided by the invention;
fig. 6 is a schematic structural diagram of the inspection system based on unmanned aerial vehicle road inspection diseases.
List of reference numerals
11: crack class data; 12: pit class data; 13: subsidence class data; 14: rut class data.
Detailed Description
The following detailed description refers to the accompanying drawings.
The invention provides a system and a method for inspecting road surface inspection diseases based on an unmanned aerial vehicle. The invention also provides a road surface inspection device. The invention also provides an image recognition system of the intelligent inspection unmanned aerial vehicle.
The invention is explained in terms of some words.
True frame: the method is characterized in that the method comprises the step of manually marking a rectangular frame, wherein the center of the rectangular frame is positioned at the center of a defect or at least near the defect, the border of the rectangular frame is overlapped with the edge of the road surface defect in an image, namely, the real frame needs to frame and select all appearances of target defects according to the types of the defects.
True box sample points: the pixel point position of the real frame border in the image is indicated.
Prediction frame: the method is characterized in that the pavement defect in the picture to be detected is predicted through the AI after feature extraction and marked on a rectangular frame of the corresponding picture to be detected, namely, the rectangular frame is at least one rectangular frame with the pavement defect as the center for prediction and determination through the AI after feature extraction.
Detection frame: the method is characterized in that the prediction frames are subjected to algorithm processing and screening to obtain prediction frames with the maximum confidence coefficient, and the prediction frames are used as unique rectangular frames for marking each disease in the picture to be detected, namely, the rectangular frames are unique in number and determined by AI (analog input) after training and taking the pavement defect as a center for prediction.
Initial anchor frame: and calculating the marked picture to determine a picture range containing a defect area, namely an initial anchor frame, and calculating a reference object of a confidence level of a prediction frame, wherein the prediction frame is marked on the picture to be detected by AI after training.
Pavement damage: whether it is cement or asphalt pavement, cracks, pits, sinkers, ruts and other morphological defects appear successively after a period of use in traffic.
Road surface scene: unmanned aerial vehicle patrols and examines the background of road.
Image dataset: the image data set is an image set containing four diseases of cracks, pits, sinkers and ruts.
Gray scale map: the image subjected to the graying process is a gray scale image.
Equalization histogram: the histogram of the original image is transformed into a uniformly distributed form in a histogram mode, so that the effect of enhancing the overall contrast of the image is achieved, and the image with the image disease characteristics is enhanced, namely the equilibrium histogram.
The invention provides a patrol system based on unmanned aerial vehicle road surface patrol disease, which at least comprises an image acquisition unit 10 and at least one processing unit 20, wherein the image acquisition unit 10 and the at least one processing unit 20 can be carried on an unmanned aerial vehicle. The image acquisition unit 10 and the at least one processing unit 20 are connected in a wired and/or wireless manner. The image acquisition unit 10 is preferably an image acquisition device, for example a camera. The precision and the model of the image acquisition unit are not limited. Can meet shooting requirements. The processing unit 20 is preferably an application specific integrated chip, CPU, server and/or cloud server capable of running the running program of the inspection method of the present invention. The image acquisition unit 10 and the at least one processing unit 20 are capable of information transmission via signal lines, as well as capable of remote information transmission via wireless network signals or satellite signals. The processing unit 20 may be mounted on the unmanned aerial vehicle, or may be installed on the ground.
As shown in fig. 1, the processing unit 20 is configured to:
s1: performing classification preprocessing on the first image data set transmitted by the image acquisition unit 10 to form a second image data set;
s2: calculating and determining an initial anchor frame based on the second image dataset;
s3: processing the second image dataset based on the feature extraction model to extract at least one standard feature map;
s4: and generating a target prediction frame of the standard feature map based on the prediction model.
Aiming at the problems that the YOLOv5 target detector is applied to an unmanned aerial vehicle inspection complex road environment, the detection difficulty is high due to the fact that the size and the position of the detection target are uneven and are shielded and interfered by other vehicles and objects in the process of realizing target detection on road diseases, and the road disease characteristics are not obvious, so that the detection omission rate and the false detection rate are high, the target detection system and the target detection method for the unmanned aerial vehicle inspection road diseases can improve the network structure and the loss function of YOLOv5 on the premise of improving the original YOLOv5 precision, quantify the model pruning, improve the reasoning speed and realize the rapid and accurate target detection of the unmanned aerial vehicle inspection road diseases.
The present invention will be described in detail with reference to the respective steps.
The first image dataset comprises at least crack-class data 11, pit-class data 12, subsidence-class data 13 and rut-class data 14.
After the first image data set is received, data augmentation of different proportions is carried out on the first image data set, and 50% of image data is further set for Mosaic data augmentation.
Specifically, the collection background of the first image dataset, namely, the scene of the unmanned plane inspection road surface is divided into two types: class a: highway scenes containing a small or large number of vehicles. Class B: there are few common road scenes of vehicles.
Graying is performed on 10% of the images of the class a dataset and balanced histogram processing is performed on 15% of the images, the data including random flipping, panning, blurring and varying brightness and adding to the first image dataset to form a second image dataset.
Graying is performed on 5% of the images of the class B dataset, balanced histogram processing is performed on 10% of the images, and added to the first image dataset to form a second image dataset.
And classifying and marking the amplified first image data set by using a Labelimg tool to obtain a second image data set with a category label and a corresponding yaml file. The second image dataset was according to 8: and 2, dividing the training set and the testing set in proportion.
The method of calculating and determining the initial anchor frame based on the second image dataset is as follows. Preferably, the initial anchor box is calculated based on the DIoU-kmean++ algorithm.
S21: selecting an array of sample point widths b and heights h of a rectangular frame from an image data set χ comprising all marked pictures in a random and uniform distribution manner as a first initial clustering center c i 。
S22: the shortest distance between each rectangular box sample point of each image and the current cluster center is calculated and is represented by D (x).
S23: the probability P (x) that each real box sample point is selected as the next cluster center is calculated.
S24: and selecting a rectangular frame sample point corresponding to the maximum probability value as the next clustering center. The previous step is repeated until k cluster centers, i.e., k anchor boxes, are selected. k is a positive integer not less than 1.
S25: and calculating the distance from each group of real frame sample points to the clustering centers of k anchor frames to obtain an error matrix. And selecting an anchor frame corresponding to the minimum error, classifying the current real frame into the anchor frame, namely, classifying sample points of the current real frame into a cluster center closest to the current real frame.
S26: repeating the above operation for each real frame, and recording the real frame corresponding to each anchor frame. The above operations are repeated, and the width and height dimensions of the anchor frames are updated until the anchor frame class to which all the real frames belong is identical to the anchor frame class to which the real frames belong.
S27: the accuracy of the anchor frame is calculated based on the distance-to-intersection ratio function.
Specifically, the DIoU value is calculated according to the obtained anchor frame and each corresponding real frame, the highest DIoU value is selected for each real frame, and the average value, that is, the final accuracy, is obtained.
The specific calculation formula of the initial anchor frame is shown below.
x 1 ＝max(x A1 ，x B1 )；
y 1 ＝max(y A1 ，y B1 )；
x 2 ＝max(x A2 ，x B2 )；
y 2 ＝max(y A2 ，y B2 )；
S A ＝(x A2 -x A1 +1.0)·(y A2 -y A1 +1.0)；
S B ＝(x B2 -x B1 +1.0)·(y B2 -y B1 +1.0)；
A∩B＝max(x 2 -x 1 +1.0，0)·max(y 2 -y 1 +1.0，0)；
A∪B＝S A +S B -A∩B；
D(y)＝1-DIoU(y)；
In the formula, DIoU represents a distance intersection ratio, ioU represents an intersection ratio, and A represents a real frame, namely a manually marked rectangular frame; and B represents a prediction frame, namely, the AI predicts the pavement defect in the picture to be detected after training and marks the pavement defect on a rectangular frame of the corresponding picture to be detected. The rectangular frame is a rectangular frame predicted and determined by AI taking the pavement defect as the center after training. X is x A1 Representing the upper left corner coordinates of the A real frame; x is x B1 Representing the upper left corner coordinates of the B prediction frame; x is x A2 Representing the lower right corner coordinates of the A real frame; x is x B1 The lower right corner coordinates of the B prediction box are indicated. y represents the sample point array of the y-th prediction frame, b represents the center point of the prediction frame, b gt Representing the center point, ρ, of the real frame 2 Representing the euclidean distance between the two points, "the center point of the predicted frame" and "the center point of the real frame", c represents the diagonal distance of the minimum closure region that can contain both the predicted frame and the real frame, and P (y) represents the probability that the sample point of the rectangular frame is selected as the next cluster center.
Preferably, the method for establishing the feature extraction model at least comprises the following steps:
to introduce a learnable reconstruction parameter and L 1 Pruning is carried out on the YOLOv5 algorithm model in a regular constraint mode, a back network module in the YOLOv5 algorithm model after pruning is updated to be a first feature fusion module, a Neck network module in the YOLOv5 algorithm model after pruning is updated to be a second feature fusion module, and the first feature fusion module transmits the extracted pavement disease feature map with at least one scale to the second feature fusion moduleAnd the characteristic fusion module is used for processing.
To introduce a learnable reconstruction parameter and L 1 The step of pruning and quantifying the YOLOv5 algorithm model in a regular constraint mode at least comprises the following steps:
s31: calculating all sample elements x in the input feature map of each batch i Mean mu and variance sigma of (a), then for all sample elements x i And (5) standardization treatment.
The specific formula is shown below.
S32: the learnable reconstruction parameters gamma and beta are introduced.
Specifically, the method comprises the steps of introducing a learnable reconstruction parameter comprising a channel scaling factor alpha and a shift transformation parameter beta, and adding L to the channel scaling factor alpha 1 Regular constraints.
L 1 ＝∑ (x，y) l(f(x，W)，y)+λ∑ γ∈Γ g(α)；
Wherein in the first term, (x,) represents the training input and the target, W represents the trainable weight, L represents the Manhattan distance calculation formula, and the first term represents the normal training L 1 A loss function; in the second term, g (. Alpha.) represents L 1 Norms, λ represent regularized coefficients, adjusted according to the dataset.
Adding L on the gradient of the learnable reconstruction parameters alpha and beta 1 The sparse learnable reconstruction parameters can be obtained through the gradient of the regular constraint.
Addition of L 1 The formula for the gradient of the canonical constraint is, for example:
s33: carrying out translation and scaling treatment on the standardized sample elements by adopting leachable reconstruction parameters alpha and beta, carrying out affine transformation by using the sparse parameters, and obtaining output characteristics y after batch regularization i The specific calculation formula is as follows:
the step of updating the backhaul network module in the pruned quantized YOLOv5 algorithm model to the first feature fusion module is as follows.
Preferably, the first feature fusion module comprises an efficient feature fusion network model. The structure of the high-efficiency feature fusion network model at least comprises: a number of conventional convolution layers and a coordinated attention mechanism function; conventional convolution layers include a CBS layer, a batch normalization function, and a loss function.
And updating the Neck network module in the pruned quantized YOLOv5 algorithm model into a second feature fusion module. The second feature fusion module also includes an efficient feature fusion network model.
S34: the second image dataset is processed based on the first feature fusion module to extract a pavement slab feature map of at least one scale.
S341: the input 640 x 3 image is processed by a 2-layer conventional convolution layer CBS layer and a feature fusion C3 network, and then 160 x 128 feature images are output;
s342: inputting the obtained 160×160×128 feature map into a high-efficiency feature fusion ELANB network model composed of a conventional convolutional layer CBS and a coordinated attention mechanism CA layer, and outputting 80×80×256 feature maps;
s343: inputting the obtained 80 x 256 feature map into a conventional convolutional layer CBS layer and a feature fusion C3 network, and outputting a 40 x 512 feature map through a coordination attention mechanism CA layer after obtaining the feature map;
s344: and inputting the obtained 40 x 512 feature map into a conventional convolutional layer CBS layer and a feature fusion C3 network, obtaining the feature map, and outputting 20 x 1024 feature map through a rapid spatial pyramid pooling SPPF layer.
Preferably, specific processing steps of the efficient feature fusion network model (ELANB network model) of the present invention are as follows.
S351: after receiving the n×n×c feature map output from the previous layer, the n×n×2c feature map is output through the CBS layer of the conventional convolution layer.
S352: through 3 conventional convolution layers CBS layers, feature map a=n/2*n/2*c, feature map b=n/2*n/2*c/4, and feature map c=n/2*n/2*c/4, respectively, are obtained. After the feature map a passes through the conventional convolution layer CBS layer, a feature map c=n/2*n/2*c/4 is output, and a feature map d=n/2*n/2*c/4.
S353: the feature map C, D is subjected to a stitching operation to output a feature map e=n/2*n/2*c/2, and the feature map E, B, C is subjected to a stitching operation to output a feature map n/2*n/2*c.
S354: the generated feature map passes through the coordinated attention mechanism CA layer.
The processing steps of the coordinated attention mechanism CA layer are specifically shown below.
And converting global pooling into one-to-one dimensional feature coding operation, and carrying out global pooling operation on each channel along a horizontal coordinate and a vertical coordinate respectively by using pooling convolution kernels with the sizes of (h, 1) and (1, w), so as to finally obtain a feature map with the height of h output and a feature map with the width of w output. The formula for the coordinated attention mechanism CA layer is as follows:
wherein,,
a c-th channel output number with height h; />
Representing a first channel output number of width w; i represents all widths of the c-th channel; c represents the number of channels of the feature map, h represents the height of the feature map, and W represents the width of the feature map.
S355: splicing the feature images in the width and height directions of the obtained global receptive field together, sending the feature images into a convolution module with convolution kernel of 1 multiplied by 1, and then normalizing the feature images F in batches 1 And sending the feature map f to a Sigmoid function.
In particular, the method comprises the steps of,
where δ is the nonlinear activation function, f is the intermediate feature map that encodes spatial information in the horizontal and vertical directions, z h A characteristic diagram showing the output with height h, z w The width is w, and the characteristic diagram is output.
S356: the characteristic diagram F is transformed into a convolution with the convolution kernel of 1 multiplied by 1 according to the original height and width to respectively obtain a first characteristic diagram F which is the same as the original characteristic diagram h And a second characteristic diagram F w After Sigmoid function, the first attention weight g of the feature map in height and width is obtained h And a second attention weight g w 。
Wherein g h ＝σ(F h (f h ))；g w ＝σ(F w (f w ))。
S357: the feature map with the attention weight in the width and height directions finally is obtained through multiplication weighted calculation on the original feature map, namely the output y of the CA layer of the coordination attention mechanism.
y c (i, j) represents a feature map tensor with attention weights in width and height directions, x c (i, j) represents the original feature map tensor of the c-th channel with width i and height j,
representing the weight of the first channel of width i in the height direction, +.>
The weight of the c-th channel of height j in the width direction is shown.
And extracting a standard feature map from the pavement damage feature map of at least one scale based on the second feature fusion module. Preferably, the step of extracting the standard feature map by the second feature fusion module is as follows.
S361: and (3) the 20 x 1024 feature map output by the rapid space pyramid pooling is subjected to a conventional convolution layer CBS layer to output a feature map I with the size of 20 x 512.
S362: after up-sampling the feature map with the size of 20 x 512, splicing the feature map with the size of 40 x 512 generated by a second feature fusion network C3 layer in the backhaul network, and generating a feature map with the size of 40 x 512 by 3 feature fusion network C3 layers;
s363: the method comprises the steps of sending a 40 x 512 feature map to a conventional convolutional layer CBS layer, outputting a 40 x 256 feature map F, splicing the 40 x 256 feature map F with an 80 x 256 feature map output by an efficient feature fusion ELANA network in a backhaul network through upsampling, and outputting a feature map G with the size of 80 x 25 through an efficient feature fusion ELANB network;
s364: the characteristic diagram G with the size of 40 x 256 is sent to a CBS layer of a conventional convolution layer, splicing operation is carried out on the characteristic diagram G and the characteristic diagram F, and the characteristic diagram G with the size of 40 x 512 is output through a 3-layer characteristic fusion network C3 layer;
s365: sending the 80-51 feature map F into a conventional convolution layer CBS layer, splicing the output feature map and a feature map G, and outputting a feature map H with the size of 40-256 through 3 feature fusion network C3 layers;
s366: and (3) sending the 40-256 characteristic diagram H into a CBS layer of a conventional convolution layer, splicing the output characteristic diagram with the characteristic diagram I, and outputting a characteristic diagram J with the size of 20-1024 by a 3-characteristic fusion network C3 layer.
Preferably, the method for generating the target prediction frame of the standard feature map based on the prediction model at least comprises the following steps:
s41: minimizing a normalized distance between the predicted frame and the target frame based on the distance intersection ratio function as a regression loss function of the rectangular frame for labeling the defect, and generating at least one target predicted frame;
s42: and detecting the target prediction frame of the road surface diseases based on an SD-NMS algorithm to obtain the target detection frame with highest confidence, and finally obtaining the target detection frame for the unmanned aerial vehicle inspection road surface diseases.
The specific steps of the SD-NMS algorithm are shown below.
S421: dividing all the predicted frames according to categories, removing background categories, inputting a matrix B containing all the predicted frames B i The method comprises the steps of carrying out a first treatment on the surface of the Creating a null matrix D as a result matrix; setting a threshold N for non-maximum suppression calculation t 。
S422: the prediction frames in each object class are arranged in descending order according to the confidence scores of the classification, and a score matrix S corresponding to the prediction frames is output;
s423: the prediction frame M with the highest score, namely the detection frame M is stored in a result matrix D, the highest score prediction frame in the set B is removed, and the rest prediction frames are marked as B i ；
S424: performing a loop operation on the prediction box element in B, calculating DIoU (M, B i ) And calculating a Gaussian function, namely a weight weakening function, wherein the formula is as follows:
wherein sigma t Representing custom superparameters s i Representing the weight of the prediction frame for the final target position; the function attenuates the prediction scores of neighboring prediction frames overlapping the highest scoring prediction frame M, and the higher the degree of overlap of the prediction frames overlapping the M height, the more serious the score attenuation.
S425: taking the result of the DIoU value as the basis for reducing the score, and finallySetting a threshold N t Will score s i The prediction frames smaller than the threshold value are deleted from the matrix B;
s426: repeating the above operation, knowing that the prediction box matrix B becomes a null matrix, the prediction box score matrix S and the result matrix D are the return values of the SD-NMS algorithm.
Preferably, the method for establishing the feature extraction model further comprises the following steps: and inputting the second image data set into a feature extraction model to carry out sparse training, and pruning the Yolov5 algorithm model according to the parameter statistical result and the threshold value.
Specifically, the second image dataset is input into the feature extraction model for sparse training. And counting the learnable reconstruction parameters alpha of all modified batch regularized BN layers, and aligning and sequencing. And finding a threshold value corresponding to the pruning rate. And obtaining a mask of each batch of regularized BN layers according to the threshold value, and enabling the number of the pruned channels to be a multiple of 4. Because the pruned network cannot be aligned with the original network channel number, the network needs to be redefined and parsed, and the bottleck layer in the second feature extraction model is pruned.
Compared with the prior art, the invention has the following beneficial effects:
aiming at the problems that the target characteristics of the pavement diseases under the view angle of the unmanned aerial vehicle are disordered and are easy to be detected by omission, a DIoU-kmeans++ algorithm is used for calculating a 3-layer anchoring frame, and 1-DIoU is adopted as a distance, so that the condition that the pavement disease targets are relatively close can be considered; the problem that the initialization state of the clustering center is divided into a cluster due to large size difference of targets and finally the global minimum value cannot be converged is solved to a certain extent. Introducing an ELANB network into the small target feature layer, enhancing the fusion of features in different convolution layers, and improving the feature extraction of the small target; and a coordinated attention mechanism CA layer is introduced into the ELANB network, and the position information of the target diseases is embedded into the channel attention, so that the parameter quantity is reduced, and the feature extraction of the target is improved. Using SD-NMS algorithm, using DIoU as the detection omission caused by the closer distance of pavement disease targets (DIoU can find out a plurality of similar pavement disease target defects); the improved NMS algorithm improves the positioning accuracy of the detection frame through confidence estimation. Finally, the model is pruned, so that the model is light and suitable for industrial deployment.
It should be noted that the above-described embodiments are exemplary, and that a person skilled in the art, in light of the present disclosure, may devise various solutions that fall within the scope of the present disclosure and fall within the scope of the present disclosure. It should be understood by those skilled in the art that the present description and drawings are illustrative and not limiting to the claims. The scope of the invention is defined by the claims and their equivalents. The description of the invention encompasses multiple inventive concepts, such as "preferably," "according to a preferred embodiment," or "optionally," all means that the corresponding paragraph discloses a separate concept, and that the applicant reserves the right to filed a divisional application according to each inventive concept.
Claims (10)
1. An inspection system based on unmanned aerial vehicle road surface inspection diseases at least comprises an image acquisition unit (10) and at least one processing unit (20) which can be carried on an unmanned aerial vehicle, wherein the image acquisition unit (10) and the at least one processing unit (20) are connected in a wired and/or wireless mode,
it is characterized in that the method comprises the steps of,
the processing unit is configured to:
-classifying pre-processing the first image dataset transmitted by the image acquisition unit (10) to form a second image dataset;
calculating and determining an initial anchor frame based on the second image dataset;
processing the second image dataset based on a feature extraction model to extract at least one standard feature map;
and generating a target prediction frame of the standard feature map based on a prediction model.
2. The inspection system based on unmanned aerial vehicle road inspection diseases of claim 1, wherein the method for processing the second image dataset based on the feature extraction model to extract at least one standard feature map comprises at least:
processing the second image data set based on a first feature fusion module to extract a pavement disease feature map of at least one scale;
and extracting a standard feature map from the pavement disease feature map of at least one scale based on a second feature fusion module.
3. The inspection system based on unmanned aerial vehicle road surface inspection disease according to claim 1 or 2, wherein the method for establishing the feature extraction model at least comprises the following steps:
to introduce a learnable reconstruction parameter and L 1 Pruning quantization is carried out on the YOLOv5 algorithm model in a regular constraint mode,
updating a backhaul network module in the YOLOv5 algorithm model after pruning quantization into a first feature fusion module,
updating the Neck network module in the YOLOv5 algorithm model after pruning quantization into a second feature fusion module,
wherein,,
the first feature fusion module conveys the extracted pavement disease feature map with at least one scale to the second feature fusion module for processing.
4. A system according to any one of claims 1 to 3, wherein the method of calculating and determining an initial anchor frame based on the second image dataset comprises at least:
randomly selecting a wide and high array of rectangular frame sample points from a second image data set with category labels as a first initial clustering center c i ；
Calculating the shortest distance between the rectangular frame sample points and the current clustering center;
the probability that a true box sample point is selected as the next cluster center is calculated,
selecting the sample point of the rectangular frame corresponding to the maximum probability value as the next clustering center;
repeatedly selecting cluster centers until k cluster centers are selected, wherein k is a positive integer not less than 1;
calculating distances from the sample points of the real frames to the clustering centers of k anchor frames to obtain an error matrix;
selecting an anchor frame corresponding to the minimum error, classifying the current real frame into the anchor frame, namely, classifying sample points of the current real frame into a cluster center closest to the current real frame;
recording a real frame corresponding to each anchor frame;
the accuracy of the anchor frame is calculated based on the distance-to-intersection ratio function.
5. The inspection system based on unmanned aerial vehicle road surface inspection disease according to any one of claims 1 to 4, wherein the method comprises the steps of introducing a learnable reconstruction parameter and L 1 The step of pruning and quantifying the YOLOv5 algorithm model in a regular constraint mode at least comprises the following steps:
calculating all sample elements x in the input feature map i Mean mu and variance sigma of (x) for all sample elements x i Standardization treatment;
carrying out translation and scaling treatment on the standardized sample elements, and introducing the learnable reconstruction parameters gamma and beta;
affine transformation is carried out by utilizing the leachable reconstruction parameters gamma and beta to obtain output characteristics y after batch regularization i ；
Increasing L on the gradient of the learnable reconstruction parameters alpha and beta 1 And (3) regularly constraining the gradient to form a sparse learnable reconstruction parameter.
6. The inspection system based on unmanned aerial vehicle road surface inspection diseases according to any one of claims 1 to 5, wherein the first feature fusion module comprises a high-efficiency feature fusion network model;
the second feature fusion module comprises a high-efficiency feature fusion network model;
the structure of the efficient feature fusion network model at least comprises:
a number of conventional convolution layers and a coordinated attention mechanism function;
the conventional convolution layers include a CBS layer, a batch normalization function, and a loss function.
7. The inspection system based on unmanned aerial vehicle road surface inspection diseases according to any one of claims 1 to 6, wherein the method for generating the target prediction frame of the standard feature map based on the prediction model at least comprises:
minimizing a normalized distance between the predicted frame and the target frame based on the distance intersection ratio function as a regression loss function of the rectangular frame for labeling the defect, and generating at least one target detection frame;
and detecting the target detection frame of the road surface diseases based on an SD-NMS algorithm to obtain the target detection frame with the highest confidence coefficient, and finally obtaining the target detection frame for the unmanned aerial vehicle inspection of the road surface diseases.
8. The inspection system based on unmanned aerial vehicle road surface inspection disease according to any one of claims 1 to 7, wherein the method for establishing the feature extraction model further comprises:
inputting the second image dataset into the feature extraction model for sparse training,
pruning the YOLOv5 algorithm model according to the parameter statistical result and the threshold value.
9. The inspection method based on the unmanned aerial vehicle road surface inspection disease is characterized by at least comprising the following steps: -classifying pre-processing the first image dataset transmitted by the image acquisition unit (10) to form a second image dataset;
calculating and determining an initial anchor frame based on the second image dataset;
processing the second image dataset based on a feature extraction model to extract at least one standard feature map;
and generating a target prediction frame of the standard feature map based on a prediction model.
10. The inspection method based on unmanned aerial vehicle road surface inspection diseases of claim 9, wherein the method further comprises:
processing the second image data set based on a first feature fusion module to extract a pavement disease feature map of at least one scale;
and extracting a standard feature map from the pavement disease feature map of at least one scale based on a second feature fusion module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310071309.9A CN116229292A (en) | 2023-01-18 | 2023-01-18 | Inspection system and method based on unmanned aerial vehicle road surface inspection disease |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310071309.9A CN116229292A (en) | 2023-01-18 | 2023-01-18 | Inspection system and method based on unmanned aerial vehicle road surface inspection disease |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116229292A true CN116229292A (en) | 2023-06-06 |
Family
ID=86581845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310071309.9A Pending CN116229292A (en) | 2023-01-18 | 2023-01-18 | Inspection system and method based on unmanned aerial vehicle road surface inspection disease |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116229292A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116797436A (en) * | 2023-08-29 | 2023-09-22 | 北京道仪数慧科技有限公司 | Processing system for carrying out road disease inspection by utilizing bus |
| CN117351356A (en) * | 2023-10-20 | 2024-01-05 | 三亚中国农业科学院国家南繁研究院 | Field crop and near-edge seed disease detection method under unmanned aerial vehicle visual angle |
| CN117456325A (en) * | 2023-10-27 | 2024-01-26 | 绵阳职业技术学院 | Rice disease and pest detection method |
| CN118154604A (en) * | 2024-05-11 | 2024-06-07 | 南京信息工程大学 | Wall crack diagnosis method and device, computer equipment and storage medium |
-
2023
- 2023-01-18 CN CN202310071309.9A patent/CN116229292A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116797436A (en) * | 2023-08-29 | 2023-09-22 | 北京道仪数慧科技有限公司 | Processing system for carrying out road disease inspection by utilizing bus |
| CN116797436B (en) * | 2023-08-29 | 2023-10-31 | 北京道仪数慧科技有限公司 | Processing system for carrying out road disease inspection by utilizing bus |
| CN117351356A (en) * | 2023-10-20 | 2024-01-05 | 三亚中国农业科学院国家南繁研究院 | Field crop and near-edge seed disease detection method under unmanned aerial vehicle visual angle |
| CN117351356B (en) * | 2023-10-20 | 2024-05-24 | 三亚中国农业科学院国家南繁研究院 | Field crop and near-edge seed disease detection method under unmanned aerial vehicle visual angle |
| CN117456325A (en) * | 2023-10-27 | 2024-01-26 | 绵阳职业技术学院 | Rice disease and pest detection method |
| CN118154604A (en) * | 2024-05-11 | 2024-06-07 | 南京信息工程大学 | Wall crack diagnosis method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Spencer Jr et al. | Advances in computer vision-based civil infrastructure inspection and monitoring | |
| CN111259809B (en) | Unmanned aerial vehicle coastline floating garbage inspection system based on DANet | |
| US20210374965A1 (en) | Method for property feature segmentation | |
| CN116229292A (en) | Inspection system and method based on unmanned aerial vehicle road surface inspection disease | |
| Ma et al. | Automatic detection and counting system for pavement cracks based on PCGAN and YOLO-MF | |
| US9710714B2 (en) | Fusion of RGB images and LiDAR data for lane classification | |
| KR102507501B1 (en) | Artificial Intelligence-based Water Quality Contaminant Monitoring System and Method | |
| KR102346676B1 (en) | Method for creating damage figure using the deep learning-based damage image classification of facility | |
| CN110532937B (en) | Method for accurately identifying forward targets of train based on identification model and classification model | |
| CN113962960A (en) | Pavement disease detection method based on deep learning | |
| Al-Sheary et al. | Crowd monitoring system using unmanned aerial vehicle (UAV) | |
| CN110852164A (en) | YOLOv 3-based method and system for automatically detecting illegal building | |
| Guan et al. | A convolutional capsule network for traffic-sign recognition using mobile LiDAR data with digital images | |
| CN112801227A (en) | Typhoon identification model generation method, device, equipment and storage medium | |
| CN115719475A (en) | Three-stage trackside equipment fault automatic detection method based on deep learning | |
| CN112285733A (en) | Urban and rural planning verification and mapping data processing method | |
| CN115049948A (en) | Unmanned aerial vehicle inspection method and device based on neural network model and related equipment | |
| CN116824399A (en) | Pavement crack identification method based on improved YOLOv5 neural network | |
| CN113158954B (en) | Automatic detection method for zebra crossing region based on AI technology in traffic offsite | |
| CN112347967B (en) | Pedestrian detection method fusing motion information in complex scene | |
| CN115984646B (en) | Remote sensing cross-satellite observation oriented distributed target detection method and device and satellite | |
| CN116205879A (en) | Unmanned aerial vehicle image and deep learning-based wheat lodging area estimation method | |
| CN112233079B (en) | Method and system for fusing images of multiple sensors | |
| CN114913504A (en) | Vehicle target identification method of remote sensing image fused with self-attention mechanism | |
| CN114463628A (en) | Deep learning remote sensing image ship target identification method based on threshold value constraint |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |