CN112700444A - Bridge bolt detection method based on self-attention and central point regression model - Google Patents

Bridge bolt detection method based on self-attention and central point regression model Download PDF

Info

Publication number
CN112700444A
CN112700444A CN202110188973.2A CN202110188973A CN112700444A CN 112700444 A CN112700444 A CN 112700444A CN 202110188973 A CN202110188973 A CN 202110188973A CN 112700444 A CN112700444 A CN 112700444A
Authority
CN
China
Prior art keywords
bolt
detection
image
attention
self
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110188973.2A
Other languages
Chinese (zh)
Other versions
CN112700444B (en
Inventor
鞠晓臣
赵欣欣
肖鑫
郭辉
左照坤
陈令康
王丽
刘晓光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Academy of Railway Sciences Corp Ltd CARS
Railway Engineering Research Institute of CARS
China State Railway Group Co Ltd
Original Assignee
China Academy of Railway Sciences Corp Ltd CARS
Railway Engineering Research Institute of CARS
China State Railway Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Academy of Railway Sciences Corp Ltd CARS, Railway Engineering Research Institute of CARS, China State Railway Group Co Ltd filed Critical China Academy of Railway Sciences Corp Ltd CARS
Priority to CN202110188973.2A priority Critical patent/CN112700444B/en
Publication of CN112700444A publication Critical patent/CN112700444A/en
Application granted granted Critical
Publication of CN112700444B publication Critical patent/CN112700444B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a bridge bolt detection method based on a self-attention and central point regression model, which comprises the following steps: acquiring a bridge bolt image to be detected; detecting the bridge bolt image through a self-attention and central point regression model obtained through pre-training; and determining whether the image bolt of the bridge bolt has a disease or not according to the detection result of the self-attention and central point regression model. The self-attention and central point regression-based model comprises a semantic segmentation network and a detection network. The two networks share the middle-layer features extracted by the convolutional neural network, the semantic segmentation network aims to perform semantic segmentation on the rectangular region where the bolt is located, and the features used for semantic segmentation are connected with the features of the detection module to position the bolt together. The method is applied to the field of bolt disease detection and identification, can overcome the defects of the traditional bridge disease image detection and identification technology, and can well solve the problems of efficiency, cost, safety and the like in bolt disease detection and identification.

Description

Bridge bolt detection method based on self-attention and central point regression model
Technical Field
The invention belongs to the technical field of vision, relates to application of a vision technology in bridge diseases, and particularly relates to a bridge bolt detection method based on a self-attention and central point regression model.
Background
High-strength bolts are one of the main connection modes of large steel structure facilities such as bridges, the steel for the high-strength bolts of bridges in China is developed from 40B to 20MnTiB and 35VB, and the popularization and the use to the present show that the high-strength bolts made of two materials can meet the use requirements through more than forty years of engineering practice tests. In recent years, the delayed fracture probability of high-strength bolts has increased due to various factors. In operation, the railway bridge is subjected to the effects of designing operation load, accelerating speed and overloading a train for a long time, is subjected to invasion of natural disasters such as earthquake, flood, debris flow and the like and invasion of harmful substances such as wind, rain, ice blocks, harmful ions and the like, bears accidental impact of the train or the ship and the like, so that the pier and the foundation can be damaged in different degrees, different types of diseases exist, and the bolt disease occupies a main position in bridge diseases. Therefore, timely detection of bridge bolt diseases is an important guarantee for avoiding bridge accidents and ensuring personal safety of people and property of the country from being damaged.
The existing methods mostly adopt a method of manually detecting and adding an auxiliary tool, regularly searching the bridge by bridge maintenance personnel, and assisting the bridge by visual inspection and equipment such as an ultrasonic detector, a cable force meter and the like. For example, ultrasonic detectors: the surface and internal quality of the bolt is checked by ultrasonic inspection. Compared with X-ray flaw detection, ultrasonic flaw detection has the advantages of higher flaw detection sensitivity, short period, low cost, flexibility, convenience, high efficiency, no harm to human bodies and the like. However, the use of the ultrasonic detector requires the bolt surface to be smooth, and the defect type can be identified only by experienced inspectors, so that the defect detection standard is not intuitive.
In the prior art, the bridge bolt diseases are mainly determined by the professional knowledge and the supervisor of the detection personnel, so that the time and labor waste efficiency is not high, the influence of subjective factors of the detection personnel is also received on the detection accuracy, and the potential safety hazard is often brought to the bridge maintenance staff. The requirements of light weight, automation and real-time property cannot be met. The difficulties are as follows:
(1) the bolt target is small and the density is high;
(2) when the precision of the auxiliary tool is reduced or the auxiliary tool is damaged, the auxiliary tool is inconvenient to replace and has higher cost;
(3) automation cannot be realized and a large amount of manual participation is required.
With the deep application of the deep convolutional neural network in the field of computer vision, the YOLO algorithm and the algorithm such as fast Rcnn based on Region pro posal have good detection effects in the industrial field and in practical application scenes, but the YOLO algorithm and the algorithm have the following challenges:
(1) both of the above methods are performed by enumerating all potential regions and then performing a classification operation on each region separately, which is wasteful, inefficient and requires additional post-processing (e.g., non-maxima suppression, etc.). This post-processing operation presents great difficulties in differentiation and training, resulting in the inability of most current detectors to truly achieve end-to-end training.
(2) The extracted candidate regions are considered to have the same contribution to final detection by algorithms such as the YOLO algorithm and the fast Rcnn algorithm, while in an actual life scene, the periphery of an object to be detected in a picture often has complex and rich semantic information, and the contribution of each region to final detection is different.
(3) When constructing a keypoint heat map (heatmap), if the center-point pixel is directly set to 1 and the remaining pixels are set to 0, errors are difficult to calculate due to discontinuity of an optimization target, and the whole model is difficult to optimize in a mode of error back propagation. Some use gaussian kernels to disperse the center point over the entire heat map, which has a certain effect, but because the calculated coordinate range is the entire heat map, the generated heat map foreground image cannot effectively reflect the shape of the bolt.
Therefore, practitioners of the same industry need to solve the problem of how to provide a bridge bolt detection method with high recognition rate and low cost.
Disclosure of Invention
The invention mainly aims to provide a bridge bolt detection method based on self-attention and central point regression models for at least partially solving the technical problems, combines a deep neural network technology with a traditional image detection and identification technology, is applied to the field of bolt disease detection and identification, can overcome the defects of the traditional bridge disease image detection and identification technology, and can well solve the problems of efficiency, cost, safety and the like in bolt disease detection and identification.
In order to achieve the purpose, the invention adopts the technical scheme that:
the embodiment of the invention provides a bridge bolt detection method based on a self-attention and central point regression model, which comprises the following steps:
acquiring a bridge bolt image to be detected;
detecting the bridge bolt image through a self-attention and central point regression model obtained through pre-training; the self-attention and central point regression model is obtained by training a plurality of groups of training data of a bolt detection scene data set; each group of data of the multiple groups of training data comprises multiple types of bridge bolt images and bolt marking frames in the bridge bolt images; the self-attention and center point based regression model comprises: a convolutional neural network, and a self-attention semantic segmentation network and a detection network based on object center point regression respectively connected with the convolutional neural network;
and determining whether the image bolt of the bridge bolt has a disease or not according to the detection result of the self-attention and central point regression model.
Further, the construction process of the bolt detection scene data set comprises the following steps:
selecting shot bolt scene images of a plurality of bridges; the scene images comprise a plurality of types of images, and each image is provided with a bolt marking frame;
zooming, cutting and turning the scene image to obtain a scene image data enhancement and increase sample set;
performing mask image processing on the image of the sample set, and taking all pixels in the bolt labeling frame as the foreground of the mask image;
and carrying out heat map construction on the bolt center point sensitive to the object size on the sample set data processed by the mask image.
Further, performing object size sensitive bolt center point heat map construction, comprising:
order to
Figure 276653DEST_PATH_IMAGE001
Indicates a width to be detected of
Figure 597389DEST_PATH_IMAGE002
Has a height ofHThe three-channel color image of (a),
Figure 580388DEST_PATH_IMAGE003
is a real number; marking frames according to bolts, generating and original drawingsISame size central point heatmap
Figure 382122DEST_PATH_IMAGE004
(ii) a The central point heat map
Figure 462074DEST_PATH_IMAGE004
Have only 0 and 1 pixel values;
since object detection based on convolutional networks leads to a reduction in the size of the generated feature map due to down-sampling, the purpose of the pre-processing is to generate a central point heat map
Figure 691061DEST_PATH_IMAGE005
Here, the
Figure 958094DEST_PATH_IMAGE006
Is due to the scaling factor formed by the convolution network downsampling;
original drawingIThe true heatmap is
Figure 360257DEST_PATH_IMAGE007
，
Figure 29135DEST_PATH_IMAGE007
The pixel value of each coordinate in the image is calculated according to a preset formula;
when the marked frame areas of a plurality of bolts in the same image are partially overlapped, the generated heat map of the central point
Figure 553658DEST_PATH_IMAGE008
The value of the middle overlapping pixel selects the largest gaussian spread result.
Further, the detection process based on the self-attention and central point regression model includes:
extracting middle-layer features of the bridge bolt image to be detected through a convolutional neural network;
inputting the extracted middle-layer features into a self-attention semantic segmentation network, and outputting a semantic segmentation feature map of the bridge bolt image;
inputting the extracted middle-layer features into a detection network based on object center point regression, and outputting a detection feature map of the bridge bolt image;
and connecting the semantic segmentation feature map with the detection feature map, and positioning the bolt through the detection network based on object center point regression.
Further, inputting the extracted middle-layer features into a self-attention semantic segmentation network, and outputting a semantic segmentation feature map of the bridge bolt image, wherein the semantic segmentation feature map comprises:
assuming that the extracted middle layer features are
Figure 448932DEST_PATH_IMAGE009
The output is the original drawingICorresponding annotated mask imageM；
Using a four-layer expanded convolutional network
Figure 389207DEST_PATH_IMAGE009
To label mask imageMSo that the mapping is used to predict the annotated mask imageMIs semantically segmented into feature dimensions of
Figure 443750DEST_PATH_IMAGE010
；
Using a convolution kernel size
Figure 873595DEST_PATH_IMAGE011
Is based on
Figure 256166DEST_PATH_IMAGE010
Predictive annotation mask imageM；
Assuming self-attention semantic segmentation of network pairsOn the original pictureIThe generated semantic segmentation result image is
Figure 62448DEST_PATH_IMAGE012
；
Then the optimization objective of the self-attention semantic segmentation network is to make the predicted segmentation result image
Figure 705918DEST_PATH_IMAGE012
Approximation label mask imageMDefining an error loss function;
and outputting a semantic segmentation feature map according to the error loss definition function.
Further, an error loss function is defined as follows:
Figure 241418DEST_PATH_IMAGE013
（2）
(2) in the formula (I), the compound is shown in the specification,
Figure 970339DEST_PATH_IMAGE014
for marking mask imagesMWhether the pixel with the x-axis and the y-axis as the abscissa is in the marked bolt rectangular area or not is 1 if the pixel with the x-axis and the y-axis is in the marked bolt rectangular area, and otherwise is 0;
Figure 455678DEST_PATH_IMAGE015
segmenting a resulting image for prediction
Figure 953656DEST_PATH_IMAGE012
X on the abscissa and y on the ordinate, 1 if yes, and 0 otherwise.
Further, connecting the semantic segmentation feature map with the detection feature map, and positioning the bolt through the detection network based on object center point regression, including:
connecting the semantic segmentation feature map with the detection feature map to obtain complete features
Figure 397406DEST_PATH_IMAGE016
；
Characterizing said integrity feature
Figure 551307DEST_PATH_IMAGE016
The predicted bolt size is compared to the actual bolt size using the L1 loss function as the optimization objective function for bolt size prediction:
Figure 371496DEST_PATH_IMAGE017
（3）
(3) in the formula, k is a bolt serial number; n is the total number of the bolts;
Figure 458400DEST_PATH_IMAGE018
is the predicted bolt size;
Figure 932107DEST_PATH_IMAGE019
the bolt size actually used as network supervision information;
the detection network based on object center point regression automatically learns the offset between the center point of the bolt marking frame and the real position, and the detection network is formulated to optimize the following formula:
Figure 573304DEST_PATH_IMAGE020
(4)
(4) in the formula (I), the compound is shown in the specification,
Figure 993921DEST_PATH_IMAGE006
due to the scaling factor formed by the convolution network downsampling,
Figure 873015DEST_PATH_IMAGE021
is the local displacement amount for each object center point;
predicting integrity features
Figure 252044DEST_PATH_IMAGE016
Whether each coordinate position of (a) is the center of the bolt; having the central point heat map of the detection network prediction represented as
Figure 177275DEST_PATH_IMAGE022
Taking a logistic regression loss function in the form of Focal loss as an optimization target:
Figure 808107DEST_PATH_IMAGE023
(5)
(5) in the formula (I), the compound is shown in the specification,
Figure 869604DEST_PATH_IMAGE024
is a hyper-parameter in the focal loss,
Figure 354288DEST_PATH_IMAGE025
the number of bolts in the current image;
detection network for complete features based on object center point regression
Figure 766815DEST_PATH_IMAGE016
Predicting 5 values at each position to realize bolt positioning; the 5 values are the probability of belonging to the center point, the offset of the two dimensions of the center point, and the width and height of the object, respectively.
Further, the overall optimization objective function of the self-attention and center point regression model is as follows:
Figure 794813DEST_PATH_IMAGE026
（6）
wherein the content of the first and second substances,
Figure 241975DEST_PATH_IMAGE027
and the hyperparameter is used for controlling the contribution degree of different loss functions to the integral model.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a bridge bolt detection method based on a self-attention and central point regression model, which comprises the following steps: acquiring a bridge bolt image to be detected; detecting the bridge bolt image through a self-attention and central point regression model obtained through pre-training; and determining whether the image bolt of the bridge bolt has a disease or not according to the detection result of the self-attention and central point regression model. The self-attention and central point regression-based model comprises a semantic segmentation network and a detection network. The two networks share the middle-layer features extracted by the convolutional neural network, the semantic segmentation network aims to perform semantic segmentation on the rectangular region where the bolt is located, and the features used for semantic segmentation are connected with the features of the detection module to position the bolt together. The two branch tasks are mutually promoted and mutually enhanced, the semantic segmentation network can provide object position prior guidance for the detection network, and the detection network can correct the semantic segmentation result.
In addition, the detection network adopts a detection method based on object center point regression, and reduces a large amount of detection time consumption brought by the traditional seed box (anchor) method while not influencing the detection precision.
Drawings
Fig. 1 is a flowchart of a bridge bolt detection method based on a self-attention and center point regression model according to an embodiment of the present invention;
FIG. 2 is a block diagram of an overall algorithm based on a self-attention and center point regression model according to an embodiment of the present invention;
fig. 3 is a flowchart of a bolt detection scene data set construction provided in the embodiment of the present invention;
FIG. 4 is an exemplary illustration of a constructed bolt detection dataset image sample;
FIG. 5 is a schematic diagram of a data preprocessing process according to an embodiment of the present invention;
FIG. 6 is a diagram of a detection process based on a self-attention and center point regression model according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of bolt position prediction according to an embodiment of the present invention;
FIG. 8 is a diagram illustrating the detection results of different models.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.
In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front", "rear", "both ends", "one end", "the other end", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "disposed," "connected," and the like are to be construed broadly, such as "connected," which may be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The invention provides a bridge bolt detection method based on a self-attention and central point regression model, as shown in figure 1, wherein the method comprises the following steps:
s10, acquiring a bridge bolt image to be detected;
s20, detecting the bridge bolt image through a self-attention and central point regression model obtained through pre-training; the self-attention and central point regression model is obtained by training a plurality of groups of training data of a bolt detection scene data set; each group of data of the multiple groups of training data comprises multiple types of bridge bolt images and bolt marking frames in the bridge bolt images; the self-attention and center point based regression model comprises: a convolutional neural network, and a self-attention semantic segmentation network and a detection network based on object center point regression respectively connected with the convolutional neural network;
and S30, determining whether the image bolt of the bridge bolt has a disease or not according to the detection result of the self-attention and central point regression model.
In the embodiment, the deep neural network technology is combined with the traditional image detection and identification technology, the method is applied to the field of bolt disease detection and identification, the defects of the traditional bridge disease image detection and identification technology can be overcome, the problems of efficiency, cost, safety and the like in bolt disease detection and identification can be well solved, and the development of the field of bridge safety monitoring in China is promoted.
The overall algorithm framework is shown in fig. 2, and for the convenience of subsequent expression, the self-attention semantic segmentation network is called a semantic segmentation module, and the detection network based on object center point regression is called a detection module.
The invention provides a self-attention mechanism and central point regression model (SACPR) algorithm based high-performance bolt detection, which comprises the following steps:
(1) different from a method based on region extraction, the method directly takes the target as a point to predict, and provides a new bolt center point heat map construction method, so that the target size of the bolt region is accurately predicted, and the operation has no non-maximum value to inhibit post-processing, and is simpler, faster and more accurate than a detection algorithm based on region extraction;
(2) in order to accurately detect the bolt area, the invention adopts a self-attention semantic segmentation algorithm, focuses attention on the target bolt area and inhibits a background area based on the illuminated target area in the semantic segmentation result, thereby improving the detection performance.
In order to realize the high-performance bridge bolt area detection, the invention constructs a high-quality bridge bolt missing scene data set based on a real scene. Then, a Keras deep learning framework is adopted to realize the SACPR algorithm, and a bolt detection experiment is carried out. In order to solve the problems in the prior art, the technical scheme provided by the invention is realized by adopting the following technical means and measures:
the construction of the bolt detection scene data set in step S20 described above, as shown in fig. 3, includes:
s21, selecting shooting bolt scene images of a plurality of bridges; the scene images comprise a plurality of types of images, and each image is provided with a bolt marking frame;
s22, zooming, cutting and turning the scene image to obtain a scene image data enhancement and increase sample set;
s23, performing mask image processing on the image of the sample set, and taking all pixels in the bolt labeling frame as the foreground of the mask image;
and S24, carrying out object size sensitive bolt center point heat map construction on the sample set data processed by the mask image.
When the data set is constructed in step S21, several bridges are selected as the shooting targets, for example, a bolt scene may be shot by using a mobile phone or other conventional mobile devices. In order to ensure the diversity of data, when shooting a target area of a specific scene, multiple images are required to be shot under the conditions of different angles, focal lengths, illumination and the like. And manually screening effective images and marking frames on the bolts in each image, wherein the marked frame images are shown in fig. 4, for example, 4205 pieces of information of the bolt detection data set preliminarily constructed through the steps are counted.
Step S22 data enhancement:
to further increase the diversity of the training data, data enhancement is used to increase the samples. The enhancement method comprises the following steps:
1) zooming: such as first scaling the short edge to 224; (the input image size for the classification detection task is often 224 x 224), the long edges are scaled equally;
2) cutting: then randomly cropping 224 x 244 sized regions from the post-zoom picture;
3) turning: and then, random horizontal overturning, random color change and random affine transformation operation are carried out on the cut image to increase the diversity of the training set pictures.
Step S23-S24 data preprocessing
(A) Masking image
As shown in fig. 2, the algorithm framework proposed by the present invention includes a detection module based on object center point regression and semantic segmentation. Semantic segmentation needs a mask image of an object, all pixels in a rectangular frame of the object marked in the construction process of a data set are directly used as the foreground of the mask image, as shown in (2) in fig. 5, a bolt area of the generated mask image is represented by 1, and other areas are represented by 0.
FIG. 5 is a schematic diagram of a data preprocessing process, in which (1) represents an original picture labeled in the data set construction; in the figure, (2) represents a mask picture for semantic segmentation; in the figure, (3) represents a heat map for central point regression for an existing method; in the figure, (4) shows a heat map for centroidal regression according to the invention.
(B) Object size sensitive bolt center point heat map construction method
In the detection module of the present invention, the object is represented by its center point, and the width and height of the object can be represented by the relative distance from the center point. Based on the representation of the central point, by using a relevant method in human body posture key point prediction literature for reference, the invention regards object detection as a key point prediction task, so a key point heat map (heatmap) needs to be constructed.
The invention provides a method for constructing a heat map of a bolt center point with sensitive object size, which is introduced in detail as follows:
order to
Figure 431648DEST_PATH_IMAGE001
Indicates a width to be detected of
Figure 737996DEST_PATH_IMAGE002
Has a height ofHThe three-channel color image of (a),
Figure 304106DEST_PATH_IMAGE003
is a real number; firstly, generating and original drawing according to the labeling informationICentral point heat map of the same size having only "0, 1" pixel values
Figure 809037DEST_PATH_IMAGE004
. Secondly, since object detection based on convolutional networks results in a smaller size of the generated feature map due to down-sampling, the goal of the pre-processing is to generate a center point heat map
Figure 169611DEST_PATH_IMAGE005
Where a is the scaling factor due to the convolutional network downsampling. Through the steps, the bolt is arranged on the original drawingICenter point of
Figure 87889DEST_PATH_IMAGE028
At generated central point heat map
Figure 445575DEST_PATH_IMAGE004
The coordinates in (A) are
Figure 319858DEST_PATH_IMAGE029
. Subsequently, the original imageIThe true heatmap is
Figure 799469DEST_PATH_IMAGE030
，
Figure 611568DEST_PATH_IMAGE030
The pixel value of each coordinate in (a) is calculated as follows:
Figure 50639DEST_PATH_IMAGE031
（1）
in formula (1)
Figure 467845DEST_PATH_IMAGE032
Is the gaussian kernel radius calculated from the dimensions of the bolt;
Figure 639063DEST_PATH_IMAGE033
and
Figure 735195DEST_PATH_IMAGE034
respectively center point is located at
Figure 977958DEST_PATH_IMAGE035
Bolt at generated center point heat map
Figure 984091DEST_PATH_IMAGE004
Medium corresponding width and height, then
Figure 654107DEST_PATH_IMAGE036
And
Figure 237535DEST_PATH_IMAGE037
respectively is a central point heat map generated by a bolt rectangular frame marked during data set construction
Figure 687583DEST_PATH_IMAGE004
Coordinates of the upper left corner point;
Figure 813802DEST_PATH_IMAGE038
representing true heatmaps
Figure 123561DEST_PATH_IMAGE030
The middle abscissa is
Figure 725444DEST_PATH_IMAGE039
The ordinate is
Figure 778850DEST_PATH_IMAGE040
Processing the corresponding pixel value; if the rectangular areas of a plurality of bolts are partially overlapped, a central point heat map is generated
Figure 821893DEST_PATH_IMAGE008
Selecting the maximum Gaussian propagation result from the values of the middle overlapped pixels; the heat map generated using the algorithm of the present invention for centroidal regression is shown in fig. 5 (4).
Comparing fig. 5 (3) with fig. 5 (4), it can be seen that this method has the following two advantages:
(1) in the heat map for center point regression, the method provided by the invention is 0 only outside the bolt marked rectangular region, while the existing method has non-0 value outside the bolt marked rectangular region;
(2) the shape of the foreground image in the heat map obtained by the method changes along with the shape change of the bolt, and the existing method generates a round foreground image for the bolt with any shape.
The training process of the self-attention and central point regression model by using the above data set is the prior art, and the details thereof are not described in this embodiment.
In one embodiment, referring to fig. 6, the detection process based on the self-attention and center point regression model includes:
s61, extracting middle-layer features of the bridge bolt image to be detected through a convolutional neural network;
s62, inputting the extracted middle-layer features into a self-attention semantic segmentation network, and outputting a semantic segmentation feature map of the bridge bolt image;
s63, inputting the extracted middle-layer features into a detection network based on object center point regression, and outputting a detection feature map of the bridge bolt image;
and S64, connecting the semantic segmentation feature map with the detection feature map, and positioning the bolt through the detection network based on object center point regression.
The following description will be made from two aspects of self-attention semantic segmentation and detection based on object center point regression.
1. Self-attention semantic segmentation
As shown in FIG. 2, the input to the semantic segmentation module is
Figure 302553DEST_PATH_IMAGE009
The output is the original drawingICorresponding annotated mask imageM. In order to better capture the context semantic information of the image, the invention uses a four-layer expansion convolution network to complete
Figure 1518DEST_PATH_IMAGE009
To
Figure 858616DEST_PATH_IMAGE041
The number of output channels of the four expansion convolutional layers is (48, 64, 96, 128), respectively, so thatIn predicting
Figure 818482DEST_PATH_IMAGE041
Is semantically segmented into feature dimensions of
Figure 470043DEST_PATH_IMAGE042
(is described as
Figure 921884DEST_PATH_IMAGE010
) Then using a convolution kernel of size
Figure 317093DEST_PATH_IMAGE011
Is based on
Figure 865886DEST_PATH_IMAGE010
PredictionM. Hypothesis semantic segmentation module for imagesIThe generated semantic segmentation result image is
Figure 157190DEST_PATH_IMAGE012
Then the optimization goal of the semantic segmentation network is to make the prediction
Figure 424223DEST_PATH_IMAGE012
AndMas equal as possible, the error loss is defined as follows:
Figure 888703DEST_PATH_IMAGE013
（2）
in the above formula, the first and second carbon atoms are,
Figure 292002DEST_PATH_IMAGE014
representing annotated mask imagesMOn the abscissa of
Figure 957470DEST_PATH_IMAGE039
The ordinate is
Figure 711799DEST_PATH_IMAGE040
Is in the marked bolt rectangular area, if yes, is 1, otherwise is 0.
Based on the illuminated target area in the semantic segmentation result, the attention is focused on the target bolt area and the background area is suppressed, so that the detection performance can be improved.
2. Object center point regression-based detection
As shown in FIG. 2, the input to the bolt detection module is still
Figure 652074DEST_PATH_IMAGE009
In keeping with the semantic segmentation module, the same four-layer expanded convolution network is used to obtain the dimensionality of
Figure 175459DEST_PATH_IMAGE042
And (5) feature diagrams. It should be noted that these feature maps are not directly used for the prediction of the bolt position, and the complete feature for detection is the connection between the feature map of the detection module and the feature map of the semantic segmentation module, i.e. the complete feature for the prediction of the bolt position is
Figure 136462DEST_PATH_IMAGE016
(dimension of
Figure 112508DEST_PATH_IMAGE043
）。
Order to
Figure 791226DEST_PATH_IMAGE044
Bolt with indication mark
Figure 434697DEST_PATH_IMAGE045
The rectangular area of (a) is,
Figure 301022DEST_PATH_IMAGE046
respectively represent the coordinates of the upper left corner and the lower right corner, so that the bolt
Figure 29944DEST_PATH_IMAGE045
The coordinate of the central point is
Figure 312021DEST_PATH_IMAGE047
Its width and height dimensions can be expressed as
Figure 809998DEST_PATH_IMAGE048
. And because of the characteristic diagram obtained by the detection module
Figure 722590DEST_PATH_IMAGE016
Compared with the original image size, the width and the height which are actually used as network supervision information are reduced
Figure 938808DEST_PATH_IMAGE049
. So that the detection module detects the secondary characteristics
Figure 86893DEST_PATH_IMAGE016
Predicted bolt size
Figure 439377DEST_PATH_IMAGE018
To the extent possible and practical
Figure 585187DEST_PATH_IMAGE019
Equally, embodiments of the present invention use the L1 loss function as the optimization objective function for bolt size (width and height) prediction:
Figure 23122DEST_PATH_IMAGE017
（3）
although the above formula is optimized, the predicted object center point can be close to the position of the real center point of the object, but the feature map is reduced by a times, so that a pixel areas of the original picture belong to the same pixel on the feature map. Therefore, if the center point obtained by the optimization formula (3) is directly offset from the real position, a certain offset error will occur. To solve this problem, the present embodiment allows the detection network to automatically learn the offset, and the formula is expressed as the following formula:
Figure 850263DEST_PATH_IMAGE020
(4)
where a is the scale factor due to the convolution network downsampling,
Figure 322833DEST_PATH_IMAGE021
is the local displacement amount for each object center point.
And whether each coordinate position of the feature map is the center of the bolt needs to be predicted. The heat map of the center point predicted by the detection module is shown as
Figure 436283DEST_PATH_IMAGE022
Then one optimization objective of the present embodiment is to make
Figure 361513DEST_PATH_IMAGE022
Obtained by preprocessing dataYConsidering that the central point is generally sparse as equal as possible, in order to alleviate the problem of imbalance between positive and negative samples, the present embodiment adopts a logistic regression loss function in the form of Focal loss as an optimization target:
Figure 992346DEST_PATH_IMAGE023
(5)
in the above formula
Figure 319422DEST_PATH_IMAGE024
Is a hyper-parameter in the focal loss,Nis the number of bolts in the current image.
From the above analysis, the detection module needs to predict 5 values for each position of the feature map, which are the probability of belonging to the central point, the offset of the two dimensions of the central point, and the width and height of the object. This embodiment achieves 5-value prediction simultaneously with one convolutional layer, as shown in FIG. 7, with five groups
Figure 807035DEST_PATH_IMAGE050
The convolution kernel of
Figure 219562DEST_PATH_IMAGE016
Of five channels of outputFive classes of values for prediction are required.
The overall optimization objective function of the model provided by the invention is as follows:
Figure 981982DEST_PATH_IMAGE026
（6）
wherein the content of the first and second substances,
Figure 694723DEST_PATH_IMAGE027
and the hyperparameter is used for controlling the contribution degree of different loss functions to the integral model. In the experiment, the
Figure 149975DEST_PATH_IMAGE051
The value is set to 0.1,
Figure 190743DEST_PATH_IMAGE052
the number of bits is set to 1,
Figure 756854DEST_PATH_IMAGE053
is set to 0.5. The hyper-parameter may be chosen within several real numbers.
Finally, the detection result of the bridge bolt detection method based on the self-attention and center point regression model provided by the embodiment of the present invention is shown in fig. 8, where the rightmost side is the detection result of the model of the present invention.
Therefore, compared with the detection results of YOLO, false-RCNN and RetinaNet, the detection result of the model disclosed by the invention has higher identification accuracy.
The bridge bolt detection method based on the self-attention and center point regression model provided by the embodiment of the invention is mainly characterized by the following points:
(1) the patent provides a method for constructing a bolt center point heat map with sensitive object size, a bolt target is directly used as a point to predict, and then the target size of a bolt area is accurately predicted, and the operation has no non-maximum value to inhibit post-processing, so that the method is simpler, faster and more accurate than a detection algorithm based on area extraction.
(2) The self-attention semantic segmentation algorithm adopted by the invention focuses attention on the target bolt area and suppresses the background area based on the illuminated target area in the semantic segmentation result, thereby improving the detection performance.
(3) The invention provides a self-attention mechanism and central point regression model (SACPR) algorithm based on which the semantic segmentation features and the detection module features are connected to locate the bolt. The two branch tasks are mutually promoted and mutually enhanced, the semantic segmentation characteristics can provide object position prior guidance for the detection module, and the detection module can correct the semantic segmentation result.
Compared with the prior art, the method has the following advantages:
(1) automation: the high-performance bridge bolt detection technology based on the self-attention mechanism and the central point regression model simultaneously comprises a semantic segmentation and detection module, realizes automatic bolt detection and liberates manual labor;
(2) light weight: the material only needs to take pictures and does not need people to carry a plurality of tools to the site.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (8)

1. The bridge bolt detection method based on the self-attention and center point regression model is characterized by comprising the following steps: the method comprises the following steps:
acquiring a bridge bolt image to be detected;
detecting the bridge bolt image through a self-attention and central point regression model obtained through pre-training; the self-attention and central point regression model is obtained by training a plurality of groups of training data of a bolt detection scene data set; each group of data of the multiple groups of training data comprises multiple types of bridge bolt images and bolt marking frames in the bridge bolt images; the self-attention and center point based regression model comprises: a convolutional neural network, and a self-attention semantic segmentation network and a detection network based on object center point regression respectively connected with the convolutional neural network;
and determining whether the image bolt of the bridge bolt has a disease or not according to the detection result of the self-attention and central point regression model.
2. The bridge bolt detection method based on the self-attention and center point regression model according to claim 1, characterized in that: the construction process of the bolt detection scene data set comprises the following steps:
selecting shot bolt scene images of a plurality of bridges; the scene images comprise a plurality of types of images, and each image is provided with a bolt marking frame;
zooming, cutting and turning the scene image to obtain a scene image data enhancement and increase sample set;
performing mask image processing on the image of the sample set, and taking all pixels in the bolt labeling frame as the foreground of the mask image;
and carrying out heat map construction on the bolt center point sensitive to the object size on the sample set data processed by the mask image.
3. The bridge bolt detection method based on the self-attention and center point regression model according to claim 2, characterized in that: performing object size sensitive bolt center point thermographic construction, comprising:
order to
Figure 654591DEST_PATH_IMAGE001
Indicates a width to be detected of
Figure 280745DEST_PATH_IMAGE002
Has a height ofHThe three-channel color image of (a),
Figure 136705DEST_PATH_IMAGE003
is a real number; marking frames according to bolts, generating and original drawingsISame size central point heatmap
Figure 240927DEST_PATH_IMAGE004
(ii) a The central point heat map
Figure 69206DEST_PATH_IMAGE005
Have only 0 and 1 pixel values;
since object detection based on convolutional networks leads to a reduction in the size of the generated feature map due to down-sampling, the purpose of the pre-processing is to generate a central point heat map
Figure 866261DEST_PATH_IMAGE006
Where a is the scaling factor due to the convolutional network downsampling;
original drawingIThe true heatmap is
Figure 943938DEST_PATH_IMAGE007
，
Figure 117431DEST_PATH_IMAGE007
The pixel value of each coordinate in the image is calculated according to a preset formula;
when the marked frame areas of a plurality of bolts in the same image are partially overlapped, the generated heat map of the central point
Figure 862533DEST_PATH_IMAGE004
The value of the middle overlapping pixel selects the largest gaussian spread result.
4. The bridge bolt detection method based on the self-attention and center point regression model according to claim 3, characterized in that: the detection process based on the self-attention and central point regression model comprises the following steps:
extracting middle-layer features of the bridge bolt image to be detected through a convolutional neural network;
inputting the extracted middle-layer features into a self-attention semantic segmentation network, and outputting a semantic segmentation feature map of the bridge bolt image;
inputting the extracted middle-layer features into a detection network based on object center point regression, and outputting a detection feature map of the bridge bolt image;
and connecting the semantic segmentation feature map with the detection feature map, and positioning the bolt through the detection network based on object center point regression.
5. The bridge bolt detection method based on the self-attention and center point regression model according to claim 4, characterized in that: inputting the extracted middle-layer features into a self-attention semantic segmentation network, and outputting a semantic segmentation feature map of the bridge bolt image, wherein the semantic segmentation feature map comprises the following steps:
assuming that the extracted middle layer features are
Figure 830489DEST_PATH_IMAGE008
The output is the original drawingICorresponding annotated mask imageM；
Using a four-layer expanded convolutional network
Figure 395462DEST_PATH_IMAGE008
To label mask imageMSo that the mapping is used to predict the annotated mask imageMIs semantically segmented into feature dimensions of
Figure 841487DEST_PATH_IMAGE009
；
Convolutional layer basis using a convolutional kernel size of 1X 1
Figure 644358DEST_PATH_IMAGE010
Predictive annotation mask image
Figure 783215DEST_PATH_IMAGE011
；
Assuming self-attention semantic segmentation of network pairsOn the original pictureIThe generated semantic segmentation result image is
Figure 101064DEST_PATH_IMAGE012
；
Then the optimization objective of the self-attention semantic segmentation network is to make the predicted segmentation result image
Figure 350780DEST_PATH_IMAGE012
Approximation label mask imageMDefining an error loss function;
and outputting a semantic segmentation feature map according to the error loss definition function.
6. The bridge bolt detection method based on the self-attention and center point regression model according to claim 5, characterized in that: the error loss function is defined as follows:
Figure 804895DEST_PATH_IMAGE013
（2）
(2) in the formula (I), the compound is shown in the specification,
Figure 114654DEST_PATH_IMAGE014
for marking mask imagesMWhether the pixel with the x-axis and the y-axis as the abscissa is in the marked bolt rectangular area or not is 1 if the pixel with the x-axis and the y-axis is in the marked bolt rectangular area, and otherwise is 0;
Figure 857482DEST_PATH_IMAGE015
segmenting a resulting image for prediction
Figure 910888DEST_PATH_IMAGE012
X on the abscissa and y on the ordinate, 1 if yes, and 0 otherwise.
7. The bridge bolt detection method based on the self-attention and center point regression model according to claim 6, characterized in that: connecting the semantic segmentation feature map with the detection feature map, and positioning the bolt through the detection network based on object center point regression, wherein the method comprises the following steps:
connecting the semantic segmentation feature map with the detection feature map to obtain complete features
Figure 216580DEST_PATH_IMAGE016
；
Characterizing said integrity feature
Figure 697240DEST_PATH_IMAGE016
The predicted bolt size is compared to the actual bolt size using the L1 loss function as the optimization objective function for bolt size prediction:
Figure 724102DEST_PATH_IMAGE017
（3）
(3) in the formula, k is a bolt serial number; n is the total number of the bolts;
Figure 581200DEST_PATH_IMAGE018
is the predicted bolt size;
Figure 744328DEST_PATH_IMAGE019
the bolt size actually used as network supervision information;
the detection network based on object center point regression automatically learns the offset between the center point of the bolt marking frame and the real position, and the detection network is formulated to optimize the following formula:
Figure 395889DEST_PATH_IMAGE020
(4)
(4) where a is the scale factor due to the convolution network downsampling,
Figure 113309DEST_PATH_IMAGE021
is the local displacement amount for each object center point;
predicting integrity features
Figure 508518DEST_PATH_IMAGE016
Whether each coordinate position of (a) is the center of the bolt; having the central point heat map of the detection network prediction represented as
Figure 526153DEST_PATH_IMAGE022
Taking a logistic regression loss function in the form of Focal loss as an optimization target:
Figure 614195DEST_PATH_IMAGE023
(5) in the formula (I), the compound is shown in the specification,
Figure 615649DEST_PATH_IMAGE024
is a hyper-parameter in the focal loss,
Figure 548970DEST_PATH_IMAGE025
the number of bolts in the current image;
detection network for complete features based on object center point regression
Figure 889952DEST_PATH_IMAGE016
Predicting 5 values at each position to realize bolt positioning; the 5 values are the probability of belonging to the center point, the offset of the two dimensions of the center point, and the width and height of the object, respectively.
8. The bridge bolt detection method based on the self-attention and center point regression model according to claim 7, characterized in that: the total optimization objective function of the self-attention and central point regression model is as follows:
Figure 148895DEST_PATH_IMAGE026
（6）
wherein the content of the first and second substances,
Figure 106487DEST_PATH_IMAGE027
and the hyperparameter is used for controlling the contribution degree of different loss functions to the integral model.
CN202110188973.2A 2021-02-19 2021-02-19 Bridge bolt detection method based on self-attention and central point regression model Active CN112700444B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110188973.2A CN112700444B (en) 2021-02-19 2021-02-19 Bridge bolt detection method based on self-attention and central point regression model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110188973.2A CN112700444B (en) 2021-02-19 2021-02-19 Bridge bolt detection method based on self-attention and central point regression model

Publications (2)

Publication Number Publication Date
CN112700444A true CN112700444A (en) 2021-04-23
CN112700444B CN112700444B (en) 2023-06-23

Family

ID=75516718

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110188973.2A Active CN112700444B (en) 2021-02-19 2021-02-19 Bridge bolt detection method based on self-attention and central point regression model

Country Status (1)

Country Link
CN (1) CN112700444B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113553938A (en) * 2021-07-19 2021-10-26 黑芝麻智能科技(上海)有限公司 Safety belt detection method and device, computer equipment and storage medium
CN113642558A (en) * 2021-08-16 2021-11-12 云南电网有限责任公司电力科学研究院 X-ray image identification method and device for strain clamp crimping defects
CN113658273A (en) * 2021-08-19 2021-11-16 上海新氦类脑智能科技有限公司 Scene self-adaptive target positioning method and system based on spatial perception
CN114092542A (en) * 2021-11-22 2022-02-25 华侨大学 Bolt measuring method and system based on two-dimensional vision
CN114219976A (en) * 2021-11-04 2022-03-22 腾讯科技（深圳）有限公司 Image processing method, image processing device, electronic equipment, storage medium and computer product
CN115330808A (en) * 2022-07-18 2022-11-11 广州医科大学 Segmentation-guided automatic measurement method for key parameters of spine of magnetic resonance image
CN115690704A (en) * 2022-09-27 2023-02-03 淮阴工学院 LG-CenterNet model-based complex road scene target detection method and device
CN116777871A (en) * 2023-06-20 2023-09-19 无锡日联科技股份有限公司 Defect detection method, device, equipment and medium based on X-rays
CN116777895A (en) * 2023-07-05 2023-09-19 重庆大学 Concrete bridge Liang Biaoguan disease intelligent detection method based on interpretable deep learning

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108549893A (en) * 2018-04-04 2018-09-18 华中科技大学 A kind of end-to-end recognition methods of the scene text of arbitrary shape
CN109543754A (en) * 2018-11-23 2019-03-29 中山大学 The parallel method of target detection and semantic segmentation based on end-to-end deep learning
CN109583517A (en) * 2018-12-26 2019-04-05 华东交通大学 A kind of full convolution example semantic partitioning algorithm of the enhancing suitable for small target deteection
WO2019144575A1 (en) * 2018-01-24 2019-08-01 中山大学 Fast pedestrian detection method and device
CN110442882A (en) * 2018-05-02 2019-11-12 中国铁道科学研究院铁道建筑研究所 A kind of LONG-SPAN RAILWAY bridge cruising inspection system and method based on BIM technology
CN111275688A (en) * 2020-01-19 2020-06-12 合肥工业大学 Small target detection method based on context feature fusion screening of attention mechanism
US20200210761A1 (en) * 2018-12-28 2020-07-02 Shanghai United Imaging Intelligence Co., Ltd. System and method for classification determination
CN111640089A (en) * 2020-05-09 2020-09-08 武汉精立电子技术有限公司 Defect detection method and device based on feature map center point
CN111898439A (en) * 2020-06-29 2020-11-06 西安交通大学 Deep learning-based traffic scene joint target detection and semantic segmentation method
CN112070768A (en) * 2020-09-16 2020-12-11 福州大学 Anchor-Free based real-time instance segmentation method
CN112101277A (en) * 2020-09-24 2020-12-18 湖南大学 Remote sensing target detection method based on image semantic feature constraint
CN112183395A (en) * 2020-09-30 2021-01-05 深兰人工智能(深圳)有限公司 Road scene recognition method and system based on multitask learning neural network
CN112257609A (en) * 2020-10-23 2021-01-22 重庆邮电大学 Vehicle detection method and device based on self-adaptive key point heat map

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019144575A1 (en) * 2018-01-24 2019-08-01 中山大学 Fast pedestrian detection method and device
WO2019192397A1 (en) * 2018-04-04 2019-10-10 华中科技大学 End-to-end recognition method for scene text in any shape
CN108549893A (en) * 2018-04-04 2018-09-18 华中科技大学 A kind of end-to-end recognition methods of the scene text of arbitrary shape
CN110442882A (en) * 2018-05-02 2019-11-12 中国铁道科学研究院铁道建筑研究所 A kind of LONG-SPAN RAILWAY bridge cruising inspection system and method based on BIM technology
CN109543754A (en) * 2018-11-23 2019-03-29 中山大学 The parallel method of target detection and semantic segmentation based on end-to-end deep learning
CN109583517A (en) * 2018-12-26 2019-04-05 华东交通大学 A kind of full convolution example semantic partitioning algorithm of the enhancing suitable for small target deteection
US20200210761A1 (en) * 2018-12-28 2020-07-02 Shanghai United Imaging Intelligence Co., Ltd. System and method for classification determination
CN111275688A (en) * 2020-01-19 2020-06-12 合肥工业大学 Small target detection method based on context feature fusion screening of attention mechanism
CN111640089A (en) * 2020-05-09 2020-09-08 武汉精立电子技术有限公司 Defect detection method and device based on feature map center point
CN111898439A (en) * 2020-06-29 2020-11-06 西安交通大学 Deep learning-based traffic scene joint target detection and semantic segmentation method
CN112070768A (en) * 2020-09-16 2020-12-11 福州大学 Anchor-Free based real-time instance segmentation method
CN112101277A (en) * 2020-09-24 2020-12-18 湖南大学 Remote sensing target detection method based on image semantic feature constraint
CN112183395A (en) * 2020-09-30 2021-01-05 深兰人工智能(深圳)有限公司 Road scene recognition method and system based on multitask learning neural network
CN112257609A (en) * 2020-10-23 2021-01-22 重庆邮电大学 Vehicle detection method and device based on self-adaptive key point heat map

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
BAOHUA QIANG 等: "Convolutional Neural Networks-Based Object Detection Algorithm by Jointing Semantic Segmentation for Images", 《SENSORS (BASEL)》 *
JIE JIANG 等: "Point Set Attention Network For Semantic Segmentation", 《2020 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)》 *
SANDLER M 等: "Mobilenetv2: invert residuals and linear bottlenecks", 《PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 *
张婷婷;章坚武;郭春生;陈华华;周迪;王延松;徐爱华;: "基于深度学习的图像目标检测算法综述", 《电信科学》 *
徐扬: "基于深度学习的行人遮挡检测研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
赵欣欣 等: "基于卷积神经网络的铁路桥梁高强螺栓缺失图像识别方法", 《中国铁道科学》 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113553938A (en) * 2021-07-19 2021-10-26 黑芝麻智能科技(上海)有限公司 Safety belt detection method and device, computer equipment and storage medium
CN113553938B (en) * 2021-07-19 2024-05-14 黑芝麻智能科技(上海)有限公司 Seat belt detection method, apparatus, computer device, and storage medium
CN113642558A (en) * 2021-08-16 2021-11-12 云南电网有限责任公司电力科学研究院 X-ray image identification method and device for strain clamp crimping defects
CN113658273B (en) * 2021-08-19 2024-04-26 上海新氦类脑智能科技有限公司 Scene self-adaptive target positioning method and system based on space perception
CN113658273A (en) * 2021-08-19 2021-11-16 上海新氦类脑智能科技有限公司 Scene self-adaptive target positioning method and system based on spatial perception
CN114219976A (en) * 2021-11-04 2022-03-22 腾讯科技（深圳）有限公司 Image processing method, image processing device, electronic equipment, storage medium and computer product
CN114092542A (en) * 2021-11-22 2022-02-25 华侨大学 Bolt measuring method and system based on two-dimensional vision
CN115330808A (en) * 2022-07-18 2022-11-11 广州医科大学 Segmentation-guided automatic measurement method for key parameters of spine of magnetic resonance image
CN115330808B (en) * 2022-07-18 2023-06-20 广州医科大学 Segmentation-guided magnetic resonance image spine key parameter automatic measurement method
CN115690704A (en) * 2022-09-27 2023-02-03 淮阴工学院 LG-CenterNet model-based complex road scene target detection method and device
CN115690704B (en) * 2022-09-27 2023-08-22 淮阴工学院 LG-CenterNet model-based complex road scene target detection method and device
CN116777871A (en) * 2023-06-20 2023-09-19 无锡日联科技股份有限公司 Defect detection method, device, equipment and medium based on X-rays
CN116777895A (en) * 2023-07-05 2023-09-19 重庆大学 Concrete bridge Liang Biaoguan disease intelligent detection method based on interpretable deep learning
CN116777895B (en) * 2023-07-05 2024-05-31 重庆大学 Concrete bridge Liang Biaoguan disease intelligent detection method based on interpretable deep learning

Also Published As

Publication number Publication date
CN112700444B (en) 2023-06-23

Similar Documents

Publication Publication Date Title
CN112700444A (en) Bridge bolt detection method based on self-attention and central point regression model
Li et al. Automatic defect detection of metro tunnel surfaces using a vision-based inspection system
WO2022193420A1 (en) Intelligent detection method for multiple types of diseases of bridge near water, and unmanned surface vessel device
CN109460753B (en) Method for detecting floating object on water
CN110222701B (en) Automatic bridge disease identification method
CN109858367B (en) Visual automatic detection method and system for worker through supporting unsafe behaviors
CN110826514A (en) Construction site violation intelligent identification method based on deep learning
Yang et al. Deep learning‐based bolt loosening detection for wind turbine towers
CN108257114A (en) A kind of transmission facility defect inspection method based on deep learning
Kumar et al. A deep learning based automated structural defect detection system for sewer pipelines
CN113643228A (en) Nuclear power station equipment surface defect detection method based on improved CenterNet network
Dong et al. Intelligent segmentation and measurement model for asphalt road cracks based on modified mask R-CNN algorithm
Bai et al. Detecting cracks and spalling automatically in extreme events by end-to-end deep learning frameworks
CN113420619A (en) Remote sensing image building extraction method
Naddaf-Sh et al. Real‐Time Explainable Multiclass Object Detection for Quality Assessment in 2‐Dimensional Radiography Images
Xue et al. A high efficiency deep learning method for the x-ray image defect detection of casting parts
Zhang et al. Research on pipeline defect detection based on optimized faster r-cnn algorithm
Bush et al. Deep Neural Networks for visual bridge inspections and defect visualisation in Civil Engineering
Chen et al. Deep learning based underground sewer defect classification using a modified RegNet
CN115082650A (en) Implementation method of automatic pipeline defect labeling tool based on convolutional neural network
Cho et al. Application of deep learning-based crack assessment technique to civil structures
Yasuno et al. One-class steel detector using patch GAN discriminator for visualising anomalous feature map
Neeli Use of photogrammetry aided damage detection for residual strength estimation of corrosion damaged prestressed concrete bridge girders
Xu et al. The steel surface multiple defect detection and size measurement system based on improved yolov5
Sinha et al. Corrosion estimation of underwater structures employing bag of features (BoF)

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant