CN114463597A - Bridge crack detection method, system and medium based on coding and decoding network - Google Patents
Bridge crack detection method, system and medium based on coding and decoding network Download PDFInfo
- Publication number
- CN114463597A CN114463597A CN202210083883.1A CN202210083883A CN114463597A CN 114463597 A CN114463597 A CN 114463597A CN 202210083883 A CN202210083883 A CN 202210083883A CN 114463597 A CN114463597 A CN 114463597A
- Authority
- CN
- China
- Prior art keywords
- layer
- decoding
- path
- bridge
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 51
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000013135 deep learning Methods 0.000 claims abstract description 38
- 230000003321 amplification Effects 0.000 claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 36
- 238000003199 nucleic acid amplification method Methods 0.000 claims abstract description 36
- 238000013526 transfer learning Methods 0.000 claims abstract description 8
- 238000011176 pooling Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 11
- 238000005070 sampling Methods 0.000 claims description 11
- 238000013508 migration Methods 0.000 claims description 8
- 230000005012 migration Effects 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 26
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 235000002566 Capsicum Nutrition 0.000 description 1
- 239000006002 Pepper Substances 0.000 description 1
- 235000016761 Piper aduncum Nutrition 0.000 description 1
- 235000017804 Piper guineense Nutrition 0.000 description 1
- 244000203593 Piper nigrum Species 0.000 description 1
- 235000008184 Piper nigrum Nutrition 0.000 description 1
- 241000277331 Salmonidae Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a bridge crack detection method, a system and a medium based on a coding and decoding network, which are applied to the fields of deep learning and crack detection, can effectively extract crack characteristics under a complex background, and realize automatic and high-precision bridge crack detection. The method comprises the following steps: acquiring an apparent crack image of the bridge; marking the apparent crack image of the bridge, and performing data amplification by an amplification data set method to obtain a data set; constructing a deep learning neural network of a coding and decoding structure; performing first training on the deep learning neural network according to a preset database to obtain a first model; performing second training on the first model according to the transfer learning mode and the data set to obtain a second model; and according to the second model, carrying out crack prediction on the input picture to be detected.
Description
Technical Field
The invention relates to the field of deep learning and crack detection, in particular to a bridge crack detection method, a bridge crack detection system and a bridge crack detection medium based on a coding and decoding network.
Background
The current bridge holding capacity of China is extremely large, so the safety of the bridge structure seriously affects the traffic operation and the economic development. Bridge cracks are used as a representative disease form of bridge structures, and how to effectively detect the bridge cracks is concerned greatly. In the related technology, the bridge crack detection mode is mainly manual visual detection, and the method is greatly interfered by the subjectiveness of detection personnel, and has low precision and low efficiency. In addition, some methods based on digital image processing are greatly interfered by image background, and the threshold value is manually adjusted to be a proper size so as to obtain an ideal detection effect.
Disclosure of Invention
In order to solve at least one of the above technical problems, the invention provides a bridge crack detection method, system and medium based on an encoding and decoding network, which can effectively extract crack characteristics under a complex background and realize automatic and high-precision bridge crack detection.
On one hand, the embodiment of the invention provides a bridge crack detection method based on an encoding and decoding network, which comprises the following steps:
acquiring an apparent crack image of the bridge;
marking the apparent crack image of the bridge, and performing data amplification by an amplification data set method to obtain a data set;
constructing a deep learning neural network of a coding and decoding structure;
performing first training on the deep learning neural network according to a preset database to obtain a first model;
performing second training on the first model according to the transfer learning mode and the data set to obtain a second model;
and according to the second model, carrying out crack prediction on the input picture to be detected.
The bridge crack detection method based on the coding and decoding network provided by the embodiment of the invention at least has the following beneficial effects: the method comprises the steps of obtaining a data set by carrying out data amplification by using an amplification data set method after marking the bridge apparent crack image, and obtaining the data set by amplifying the bridge apparent crack image by using the amplification data set method through a small number of bridge apparent crack images. Meanwhile, a deep learning neural network of a coding and decoding structure is constructed, and the bridge crack characteristics under the complex background can be well extracted by constructing the deep learning neural network of the coding and decoding structure, so that the accuracy of the bridge crack detection result is improved, the characteristics of the deep learning neural network can be utilized, the threshold value is not required to be manually adjusted to be in a proper size, and the bridge crack automatic detection under the complex background is realized. In addition, after the deep learning neural network is trained firstly according to the preset database to obtain a first model, the deep learning neural network is trained secondly through the migration learning in combination with the data set, so that a second model is obtained.
According to some embodiments of the invention, the data amplification by the amplification dataset method comprises:
data amplification is performed by at least one of scaling, adding noise, or adding shading to amplify the data set.
According to some embodiments of the invention, the deep learning neural network comprises an encoding path, a decoding path, and a lateral superposition path;
the encoding path comprises a first convolution layer and a pooling layer, wherein the first convolution layer is used for extracting a first feature map of an input image, and the pooling layer is used for compressing the first feature map;
the decoding path comprises a second convolutional layer and an up-sampling layer, the second convolutional layer is used for extracting a second feature map of the input image, and the up-sampling layer is used for amplifying the second feature map;
the transverse superposition path is used for superposing the first feature map extracted by the first convolution layer into the input features of the decoding path.
According to some embodiments of the present invention, the coding path is provided with 5 layers, the coding paths from layer 1 to layer 4 each include the first convolutional layer and the pooling layer, and the coding path from layer 5 includes the first convolutional layer, wherein the first convolutional layer includes two convolution kernels of 3 × 3, and the pooling layer includes a pooling kernel of 2 × 2.
According to some embodiments of the present invention, the decoding path is provided with 5 layers, each of the decoding paths of layer 2 to layer 4 includes the second convolutional layer and the upsampled layer, wherein the second convolutional layer in each of the decoding paths of layer 2 to layer 4 includes two convolution kernels of 3 × 3, the decoding path of layer 1 includes two convolution kernels of 3 × 3 and one convolution kernel of 1 × 1, and the decoding path of layer 5 includes the encoding path of layer 5 and the upsampled layer.
According to some embodiments of the present invention, the transverse superposition path is provided with 4 layers, and the 4 layers of transverse superposition paths are respectively provided between the corresponding levels of the encoding path and the decoding path from layer 1 to layer 4.
According to some embodiments of the invention, the upsampling layer comprises a bilinear interpolated upsampling layer.
On the other hand, the embodiment of the invention also provides a bridge crack detection system based on an encoding and decoding network, which comprises:
the acquisition module is used for acquiring an apparent crack image of the bridge;
the data processing module is used for marking the bridge apparent crack image and carrying out data amplification by an amplification data set method to obtain a data set;
the network construction module is used for constructing a deep learning neural network of a coding and decoding structure;
the first training module is used for carrying out first training on the deep learning neural network according to a preset database to obtain a first model;
the second training module is used for carrying out second training on the first model according to the migration learning and the data set to obtain a second model;
and the prediction module is used for performing crack prediction on the input picture to be detected according to the second model.
On the other hand, the embodiment of the invention also provides a bridge crack detection system based on an encoding and decoding network, which comprises:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor may implement the method for detecting a bridge crack based on an encoding and decoding network according to the above embodiments.
In another aspect, an embodiment of the present invention further provides a computer storage medium, in which a processor-executable program is stored, and when the processor-executable program is executed by the processor, the processor-executable program is configured to implement the method for detecting a bridge crack based on an encoding and decoding network according to the above embodiment.
Drawings
FIG. 1 is a flowchart of a bridge crack detection method based on an encoding and decoding network according to an embodiment of the present invention;
FIG. 2 is a block diagram of an encoding and decoding architecture network provided by an embodiment of the present invention;
FIG. 3 is a diagram of a blurred image before prediction by a second model according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a predicted result of the second model according to the embodiment of the present invention after predicting FIG. 3;
FIG. 5 is a diagram illustrating a shadow disturbance image before prediction by a second model according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a predicted result of the second model according to the embodiment of the present invention after predicting FIG. 5;
fig. 7 is a schematic block diagram of a bridge crack detection system based on an encoding and decoding network according to an embodiment of the present invention.
Detailed Description
The embodiments described in the embodiments of the present application should not be construed as limiting the present application, and all other embodiments that can be obtained by a person skilled in the art without making any inventive step shall fall within the scope of protection of the present application.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.
The safety of the bridge structure seriously influences traffic operation and economic development, so that the bridge structure safety detection method has important significance for the bridge structure safety detection. Bridge cracks are a representative disease form of bridge structures. Traditional bridge crack detection mode mainly looks through the manual work and detects, needs measurement personnel to be close to bridge structures's position to carry out artifical observation and discernment with the help of equipment such as crack detection appearance to the crack. However, the manual visual inspection method is greatly interfered by the subjectiveness of the inspection personnel, and has low precision and low efficiency. In the related art, methods based on digital image processing are applied to bridge crack detection, for example, a single-lens reflex camera or a smart phone is used to take an apparent photograph of a bridge, and then a Canny algorithm or a threshold segmentation algorithm is used to process an image, so as to identify cracks in the image. However, these methods are easily interfered by image background, so that the threshold value needs to be manually adjusted to a proper size to obtain a desirable detection effect.
Based on the above, the embodiment of the invention provides a bridge crack detection method based on an encoding and decoding network, which can effectively extract crack features under a complex background and realize automatic and high-precision bridge crack detection. Referring to fig. 1, the method of the embodiment of the present invention includes, but is not limited to, step S110, step S120, step S130, step S140, step S150, and step S160.
Specifically, the application process of the embodiment includes the following steps:
s110: and acquiring an apparent crack image of the bridge.
S120: and marking the apparent crack image of the bridge, and performing data amplification by using an amplification data set method to obtain a data set.
S130: and constructing a deep learning neural network of a coding and decoding structure.
S140: and carrying out first training on the deep learning neural network according to a preset database to obtain a first model.
S150: and carrying out second training on the first model according to the transfer learning mode and the data set to obtain a second model.
S160: and according to the second model, carrying out crack prediction on the input picture to be detected.
In the working process of the above embodiment, the bridge apparent crack image is obtained first. And correspondingly labeling the acquired bridge apparent crack image, and simultaneously performing data amplification on the labeled bridge apparent crack image by using an amplification data set method to obtain a data set. Further, a deep learning neural network of a coding and decoding structure is constructed, and preliminary training is carried out on the built deep learning neural network through a preset database, namely first training is carried out, so that a first model is obtained. The preset database comprises an ImageNet public database, and the constructed deep learning neural network of the coding and decoding structure is preliminarily trained through the ImageNet public database to obtain a preliminary model, namely the first model. And then, carrying out corresponding processing on the first model according to the migration learning mode, and carrying out optimization training, namely second training, on the first model after the migration learning processing through a data set to obtain a second model. And the finally obtained second model is obtained through data set optimization training, and the crack of the input picture to be detected is predicted according to the second model, so that automatic and high-precision bridge crack detection is realized.
In the above specific embodiment, after the bridge apparent crack image is obtained, pixel-level labeling of cracks is performed on the obtained bridge apparent crack image, so that each picture obtains a group trout label picture corresponding to each picture. Further, the marked picture is subjected to data amplification through a data set amplification method, so that original data with small data volume is amplified to a data set with large data volume. In addition, after a deep learning neural network of a coding and decoding structure is constructed, first training is carried out on the deep learning neural network through a preset database, and a first model is obtained. And carrying out second training on the first model according to the transfer learning mode and the data set to obtain a second model. Specifically, after the first model is correspondingly processed in a transfer learning mode, the first model is subjected to second training through a data set, so that a second model is obtained. The first training of the deep learning neural network through the preset database can effectively utilize a large amount of data of the preset database to carry out preliminary training on the deep learning neural network of the constructed coding and decoding structure, and a first model is obtained. Further, the first model is processed in a transfer learning mode, and then the first model after transfer learning is subjected to second training by using a data set, so that a second model is obtained. The deep learning neural network of the constructed coding and decoding structure can be preliminarily trained by utilizing the preset database through migration learning, and then, second training is carried out by combining a data set to obtain a second model, so that crack characteristics under a complex background are effectively extracted, automatic and high-precision bridge crack detection is realized, and training for a bridge crack detection model, namely training of the second model can be completed without excessive training pictures.
It should be noted that, in the process of performing the first training on the deep learning neural network through the preset database to obtain the first model, the preset database includes an ImageNet public database. Specifically, the constructed deep learning neural network of the coding and decoding structure is trained from an initialization state based on an ImageNet database which can be downloaded and used publicly. Illustratively, set the batch size to be 10, the learning rate to be 0.001, and the number of iterations to be 300, select Dice Loss as a Loss function, which is expressed as the following equation (1):
in the formula (I), the compound is shown in the specification,
and the value of each element in the output picture is represented, and y represents the corresponding group Truth tag value respectively.
And updating the model by using an SGD optimizer, checking the Dice Loss value in the training process, and if the die Loss value does not increase or decrease within 20 iterations, determining that the training is in the optimal state, and terminating the training in advance. And if the training is not finished after the 300 times of iteration is finished, increasing the iteration times until the model is trained to the optimal state. The best state training results are saved and used as preliminary model parameters, i.e. as first model parameters.
In some embodiments of the invention, data amplification is performed by an amplification dataset method, including, but not limited to, the steps of:
data amplification is performed by at least one of scaling, adding noise, or adding shading to amplify the data set.
In the working process of the above specific embodiment, when data amplification is performed by using the data set amplification method, the data amplification processing is performed on the marked bridge apparent crack image by using at least one of the data set amplification methods of scaling, adding noise, and adding shadow. Specifically, scaling is performed by synchronously enlarging or reducing the label picture corresponding to the image according to a certain proportion, and adding noise and shadow is to add salt and pepper noise and artificial shadow in the bridge apparent crack image. The scaling operation can obtain crack features with different sizes, the data set can have the crack features under noise by adding noise, and the data set can have the crack features under different illumination by adding shadows. The deep learning neural network is a data-driven model, and the influences of inconsistent crack sizes, excessive image noise and image shadow interference in detection can be effectively relieved through the operations, so that crack characteristics under a complex background are effectively extracted, and automatic and high-precision bridge crack detection is realized. Further, the amplified data set was as follows 7: 1: 2, randomly dividing in proportion to respectively obtain a training set, a verification set and a test set. Model parameters are updated through a training process, a training effect is checked through a verification process, and a testing process is used for testing the actual learning capacity of the deep learning neural network.
Illustratively, refer to fig. 3 to 6, where fig. 3 is a schematic diagram of a blurred image before prediction by the second model, and fig. 4 is a schematic diagram of a prediction result after prediction of fig. 3 by the second model. As can be seen from fig. 4, when the input picture to be detected is fuzzy, the second model can better predict the bridge crack in the picture to be detected, and extract the bridge crack characteristics in the fuzzy image. Meanwhile, fig. 5 is a schematic diagram of a shadow interference image before prediction by the second model, and fig. 6 is a schematic diagram of a prediction result after prediction of fig. 5 by the second model. As can be easily obtained from fig. 6, when there are many shadows in the input picture to be detected, the second model can also better predict the bridge crack in the picture to be detected. Data amplification is carried out through data set amplification methods such as zooming, noise adding or shadow adding to obtain a data set, and the data set is used for carrying out optimization training on the first model after the migration learning processing to obtain a second model, so that crack characteristics under a complex background can be effectively extracted, and high-precision bridge crack detection is realized.
In some embodiments of the present invention, the constructed deep learning neural network of coding and decoding structures comprises an encoding path, a decoding path, and a transverse superposition path. The encoding path comprises a first convolution layer and a pooling layer, wherein the first convolution layer is used for extracting a first feature map of an input image, and the pooling layer is used for compressing the first feature map; the decoding path comprises a second convolution layer and an up-sampling layer, the second convolution layer is used for extracting a second feature map of the input image, and the up-sampling layer is used for amplifying the second feature map; the transverse superposition path is used for superposing the first characteristic diagram extracted by the first convolution layer into the input characteristic of the decoding path. Specifically, the encoding path is used for extracting a first feature map of the input image by performing convolution operation on the first convolution layer, and extracting different semantic levels in the input image. The role of the pooling layer is to compress the size of the extracted first feature map, and the feature map is input into the next layer of coding path after passing through each pooling layer. With the depth of the network layer, the features capable of being extracted slowly enter the deep information from the shallow information. In addition, the decoding path extracts a second feature map of the input image by performing a convolution operation on the second convolution layer. Meanwhile, the decoding path is provided with an up-sampling layer, and the second characteristic diagram is amplified through up-sampling operation of the up-sampling layer and input to the next level. Correspondingly, a transverse superposition path is arranged between the coding path and the decoding path, and the first characteristic diagram extracted by the first convolution layer is superposed into the input characteristic of the decoding path, so that the detection and positioning capability of the deep learning neural network can be improved, and more accurate characteristic information can be acquired.
Referring to fig. 2, in some embodiments of the present invention, the coding path is provided with 5 layers, the layer 1 coding path to the layer 4 coding path each include a first convolutional layer and a pooling layer, and the layer 5 coding path includes the first convolutional layer. Wherein the first convolution layer comprises two convolution kernels of 3 x 3 and the pooling layer comprises pooling kernels of 2 x 2. Specifically, the coding path sets a total of 5 layers. And performing convolution twice on the input image passing through two 3 x 3 convolution kernels of the first convolution layer on the layer 1 coding path, and extracting to obtain a first feature map of the input image. Further, the first feature map is pooled through the pooling layer and then enters the layer 2 coding path, and then the first convolution layer in the layer 2 coding path is convolved twice to extract the first feature map of the input image. Pooling the first characteristic diagram, entering a 3 rd layer coding path, performing convolution twice on a first convolution layer in the 3 rd layer coding path, pooling the output first characteristic diagram, inputting the pooled first characteristic diagram into a 4 th layer coding path, performing convolution twice by 3 x 3, inputting the output first characteristic diagram into a pooling layer, compressing, and entering a 5 th layer coding path. And performing convolution twice on the first convolution layer in the layer 5 coding path. The convolution operation steps used from the layer 1 coding path to the layer 5 coding path are all 1, and the number of the filters adopted from the layer 1 coding path to the layer 5 coding path is 64, 128, 256, 512 and 1024 in sequence.
Referring to fig. 2, in some embodiments of the present invention, the decoding path is provided with 5 layers, each of the layer 2 to layer 4 decoding paths includes a second convolutional layer and an upsampled layer, wherein the second convolutional layer in each of the layer 2 to layer 4 decoding paths of the layer 2 decoding path includes two convolution kernels of 3 × 3, the layer 1 decoding path includes two convolution kernels of 3 × 3 and one convolution kernel of 1 × 1, and the layer 5 decoding path includes a layer 5 encoding path and an upsampled layer. Specifically, a first feature map output by a layer 5 decoding path (i.e., a layer 5 encoding path, where the layer 5 decoding path and the layer 5 encoding path are the same path) is amplified by an upsampling layer to obtain a second feature map. Inputting the second feature map into a layer 4 decoding path, convolving the second feature map by a second convolution layer in the layer 4 decoding path, extracting to obtain a corresponding second feature map, upsampling the output second feature map, and then entering a layer 3 decoding path. And performing convolution on a second convolution layer in the 3 rd layer decoding path, performing up-sampling on the output second characteristic diagram, entering the 2 nd layer decoding path, performing convolution on the second convolution layer, performing up-sampling on the output second characteristic diagram, entering the 1 st layer decoding path, performing convolution operation for three times, and finally obtaining an output result. In the convolution operation from the layer 4 decoding path to the layer 2 decoding path, the selected convolution kernels are all 3 × 3, and the step size is all 1. In the three convolution operations used in the layer 1 decoding path, the sizes of the convolution kernels selected in the first two times are all 3 × 3, the step sizes are all 1, and the size of the convolution kernel selected in the last time is 1 × 1. Meanwhile, the number of filters used from the layer 4 decoding path to the layer 2 decoding path is 512, 256, and 128 in sequence, and the number of filters used in the layer 1 is 64 and 2, respectively.
Referring to fig. 2, in some embodiments of the present invention, the transverse overlay path is provided with 4 layers, and the 4 layers of transverse overlay paths are respectively provided between corresponding levels of the layer 1 to layer 4 encoding paths and decoding paths. Specifically, the layer 1 transverse superposition path is arranged between the layer 1 coding path and the layer 1 decoding path, and the first characteristic diagram output by the first convolution layer of the layer 1 coding path is superposed with the input characteristic diagram of the layer 1 decoding path. The layer 2 transverse superposition path is arranged between the layer 2 coding path and the layer 2 decoding path, and a first characteristic diagram output by the first convolution layer of the layer 2 coding path is superposed with the input characteristic of the layer 2 decoding path. The layer 3 transverse superposition path is arranged between the layer 3 coding path and the layer 3 decoding path, and the first characteristic diagram output by the first convolution layer of the layer 3 coding path is superposed with the input characteristic diagram of the layer 3 decoding path. The 4 th layer transverse superposition path is arranged between the 4 th layer coding path and the 4 th layer decoding path, and the first characteristic diagram output by the first convolution layer of the 4 th layer coding path is superposed with the input characteristic diagram of the 4 th layer decoding path. The first feature maps extracted by the first convolution layer of the decoding paths from the 1 st layer to the 4 th layer are respectively superposed in the input features of the corresponding levels in the decoding paths from the 1 st layer to the 4 th layer, and the features of corresponding points are added when in superposition without increasing the number of channels. By arranging the corresponding transverse superposition path between the coding path and the decoding path of the corresponding hierarchy, the deep learning neural network of the coding and decoding structure has more accurate detection and positioning capability.
In some embodiments of the invention, the upsampled layers in the decoding path comprise bilinear interpolated upsampled layers. Specifically, in the decoding path, the convolution operation is performed on the input image through the second convolution layer, and after a corresponding second feature map is extracted, the second feature map is input to the bilinear interpolation upsampling layer. And performing bilinear interpolation upsampling on the input second feature map, so that the second feature map is amplified and is input into a decoding path of the next layer. The processing of the second feature map by the bilinear interpolation upsampling layer can enable the quality of the amplified image to be higher, and relieve the problem of discontinuous nearest neighbor interpolation.
An embodiment of the present invention further provides a bridge crack detection system based on an encoding and decoding network, including:
the acquisition module is used for acquiring an apparent crack image of the bridge;
the data processing module is used for marking the bridge apparent crack image and carrying out data amplification by an amplification data set method to obtain a data set;
the network construction module is used for constructing a deep learning neural network of a coding and decoding structure;
the first training module is used for carrying out first training on the deep learning neural network according to a preset database to obtain a first model;
the second training module is used for carrying out second training on the first model according to the migration learning combined data set to obtain a second model;
and the prediction module is used for predicting the crack of the input picture to be detected according to the second model.
Referring to fig. 7, an embodiment of the present invention further provides a bridge crack detection system based on an encoding and decoding network, including:
at least one process 210;
at least one memory 220 for storing at least one program;
when the at least one program is executed by the at least one processor 210, the at least one processor 210 implements the method for detecting a bridge crack based on an encoding and decoding network described in the above embodiments.
An embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions for execution by one or more control processors, e.g., to perform the steps described in the above embodiments.
One of ordinary skill in the art will appreciate that all or some of the steps, systems, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
While the preferred embodiments of the present invention have been described in detail, it will be understood by those skilled in the art that the foregoing and various other changes, omissions and deviations in the form and detail thereof may be made without departing from the scope of this invention.
Claims (10)
1. A bridge crack detection method based on an encoding and decoding network is characterized by comprising the following steps:
acquiring an apparent crack image of the bridge;
marking the apparent crack image of the bridge, and performing data amplification by an amplification data set method to obtain a data set;
constructing a deep learning neural network of a coding and decoding structure;
performing first training on the deep learning neural network according to a preset database to obtain a first model;
performing second training on the first model according to the transfer learning mode and the data set to obtain a second model;
and according to the second model, carrying out crack prediction on the input picture to be detected.
2. The bridge crack detection method based on the encoding and decoding network of claim 1, wherein the data amplification is performed by an amplification data set method, comprising:
data amplification is performed by at least one of scaling, adding noise, or adding shading to amplify the data set.
3. The bridge crack detection method based on the coding and decoding network of claim 1, wherein the deep learning neural network comprises a coding path, a decoding path and a transverse superposition path;
the encoding path comprises a first convolution layer and a pooling layer, wherein the first convolution layer is used for extracting a first feature map of an input image, and the pooling layer is used for compressing the first feature map;
the decoding path comprises a second convolutional layer and an up-sampling layer, the second convolutional layer is used for extracting a second feature map of the input image, and the up-sampling layer is used for amplifying the second feature map;
the transverse superposition path is used for superposing the first feature map extracted by the first convolution layer into the input features of the decoding path.
4. The bridge crack detection method based on the coding and decoding network of claim 3, wherein the coding path has 5 layers, the coding paths from layer 1 to layer 4 each include the first convolutional layer and the pooling layer, and the coding path at layer 5 includes the first convolutional layer, wherein the first convolutional layer includes two convolution kernels of 3 × 3, and the pooling layer includes a pooling kernel of 2 × 2.
5. The bridge crack detection method based on the encoding and decoding network of claim 4, wherein the decoding path is provided with 5 layers, each of the decoding paths from the 2 nd layer to the 4 th layer comprises the second convolutional layer and the upsampling layer, wherein the second convolutional layer in each of the decoding paths from the 2 nd layer to the 4 th layer comprises two convolution kernels of 3 x 3, the decoding path of the 1 st layer comprises two convolution kernels of 3 x 3 and one convolution kernel of 1 x 1, and the decoding path of the 5 th layer comprises the encoding path of the 5 th layer and the upsampling layer.
6. The bridge crack detection method based on the encoding and decoding network of claim 5, wherein the transverse superposition path is provided with 4 layers, and the 4 layers of transverse superposition paths are respectively provided between the corresponding levels of the encoding path and the decoding path from layer 1 to layer 4.
7. The encoding and decoding network-based bridge fracture detection method of claim 3, wherein the upsampling layer comprises a bilinear interpolation upsampling layer.
8. A bridge crack detection system based on coding and decoding network is characterized by comprising:
the acquisition module is used for acquiring an apparent crack image of the bridge;
the data processing module is used for marking the bridge apparent crack image and carrying out data amplification by an amplification data set method to obtain a data set;
the network construction module is used for constructing a deep learning neural network of a coding and decoding structure;
the first training module is used for carrying out first training on the deep learning neural network according to a preset database to obtain a first model;
the second training module is used for carrying out second training on the first model according to the migration learning and the data set to obtain a second model;
and the prediction module is used for performing crack prediction on the input picture to be detected according to the second model.
9. A bridge crack detection system based on an encoding and decoding network is characterized by comprising:
at least one processor;
at least one memory for storing at least one program;
when executed by the at least one processor, cause the at least one processor to implement the method of any one of claims 1 to 7.
10. A computer storage medium in which a processor-executable program is stored, wherein the processor-executable program, when executed by the processor, is configured to implement the encoding and decoding network-based bridge crack detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083883.1A CN114463597A (en) | 2022-01-20 | 2022-01-20 | Bridge crack detection method, system and medium based on coding and decoding network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083883.1A CN114463597A (en) | 2022-01-20 | 2022-01-20 | Bridge crack detection method, system and medium based on coding and decoding network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114463597A true CN114463597A (en) | 2022-05-10 |
Family
ID=81410817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210083883.1A Pending CN114463597A (en) | 2022-01-20 | 2022-01-20 | Bridge crack detection method, system and medium based on coding and decoding network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114463597A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110120041A (en) * | 2019-05-14 | 2019-08-13 | 郑州大学 | Pavement crack image detecting method |
| CN111986164A (en) * | 2020-07-31 | 2020-11-24 | 河海大学 | Road crack detection method based on multi-source Unet + Attention network migration |
| CN112949783A (en) * | 2021-04-29 | 2021-06-11 | 南京信息工程大学滨江学院 | Road crack detection method based on improved U-Net neural network |
| US20210295155A1 (en) * | 2020-03-07 | 2021-09-23 | Tata Consultancy Services Limited | Method and system for transfer learning based object detection |
| WO2021245563A1 (en) * | 2020-06-02 | 2021-12-09 | Matrix Jvco Ltd | Borehole image interpretation and analysis |
-
2022
- 2022-01-20 CN CN202210083883.1A patent/CN114463597A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110120041A (en) * | 2019-05-14 | 2019-08-13 | 郑州大学 | Pavement crack image detecting method |
| US20210295155A1 (en) * | 2020-03-07 | 2021-09-23 | Tata Consultancy Services Limited | Method and system for transfer learning based object detection |
| WO2021245563A1 (en) * | 2020-06-02 | 2021-12-09 | Matrix Jvco Ltd | Borehole image interpretation and analysis |
| CN111986164A (en) * | 2020-07-31 | 2020-11-24 | 河海大学 | Road crack detection method based on multi-source Unet + Attention network migration |
| CN112949783A (en) * | 2021-04-29 | 2021-06-11 | 南京信息工程大学滨江学院 | Road crack detection method based on improved U-Net neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232394B (en) | Multi-scale image semantic segmentation method | |
| US11151403B2 (en) | Method and apparatus for segmenting sky area, and convolutional neural network | |
| CN110276767B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN108764039B (en) | Neural network, building extraction method of remote sensing image, medium and computing equipment | |
| CN112232391B (en) | Dam crack detection method based on U-net network and SC-SAM attention mechanism | |
| CN108510451B (en) | Method for reconstructing license plate based on double-layer convolutional neural network | |
| CN111582093A (en) | Automatic small target detection method in high-resolution image based on computer vision and deep learning | |
| CN111160229B (en) | SSD network-based video target detection method and device | |
| CN113221925B (en) | Target detection method and device based on multi-scale image | |
| CN107506792B (en) | Semi-supervised salient object detection method | |
| CN112101195B (en) | Crowd density estimation method, crowd density estimation device, computer equipment and storage medium | |
| WO2021174940A1 (en) | Facial detection method and system | |
| CN112836625A (en) | Face living body detection method and device and electronic equipment | |
| CN112488046A (en) | Lane line extraction method based on high-resolution images of unmanned aerial vehicle | |
| US20240104900A1 (en) | Fish school detection method and system thereof, electronic device and storage medium | |
| WO2023065665A1 (en) | Image processing method and apparatus, device, storage medium and computer program product | |
| CN111209858A (en) | Real-time license plate detection method based on deep convolutional neural network | |
| CN114549555A (en) | Human ear image planning and division method based on semantic division network | |
| CN112749578A (en) | Remote sensing image automatic road extraction method based on deep convolutional neural network | |
| CN110119736B (en) | License plate position identification method and device and electronic equipment | |
| CN115578615A (en) | Night traffic sign image detection model establishing method based on deep learning | |
| CN116310095A (en) | Multi-view three-dimensional reconstruction method based on deep learning | |
| CN114639102A (en) | Cell segmentation method and device based on key point and size regression | |
| CN112132867B (en) | Remote sensing image change detection method and device | |
| CN113313162A (en) | Method and system for detecting multi-scale feature fusion target |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |