CN112699736A - Bridge bearing fault identification method based on space attention - Google Patents

Bridge bearing fault identification method based on space attention Download PDF

Info

Publication number
CN112699736A
CN112699736A CN202011442501.7A CN202011442501A CN112699736A CN 112699736 A CN112699736 A CN 112699736A CN 202011442501 A CN202011442501 A CN 202011442501A CN 112699736 A CN112699736 A CN 112699736A
Authority
CN
China
Prior art keywords
image
bridge
attention
sample image
spatial
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011442501.7A
Other languages
Chinese (zh)
Other versions
CN112699736B (en
Inventor
艾志勇
崔弥达
荣耀
张恺
吴刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
JIANGXI TRAFFIC SCIENCE RESEARCH INSTITUTE
Southeast University
Original Assignee
JIANGXI TRAFFIC SCIENCE RESEARCH INSTITUTE
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by JIANGXI TRAFFIC SCIENCE RESEARCH INSTITUTE, Southeast University filed Critical JIANGXI TRAFFIC SCIENCE RESEARCH INSTITUTE
Priority to CN202011442501.7A priority Critical patent/CN112699736B/en
Publication of CN112699736A publication Critical patent/CN112699736A/en
Application granted granted Critical
Publication of CN112699736B publication Critical patent/CN112699736B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a bridge bearing disease identification method based on space attention, which comprises the following steps: acquiring image data of the bridge support, and endowing labels with various support diseases possibly occurring in the process of normal support and bridge service by a manual labeling method. Constructing a neural network model with a space attention mechanism, wherein the space attention mechanism generates 4 attention coordinate values through a small neural network, screens out valuable areas in the image according to the 4 coordinate values, and scales to a specified size through a lattice point generating function and a bilinear interpolation method; and training the output of the spatial attention mechanism as the input of a convolutional neural network to obtain a neural network model with the function of predicting the support diseases. The attention model can enable the network model to automatically extract the valuable area in the bridge support image for learning, and compared with the traditional convolutional neural network model, the recognition accuracy of the support diseases can be effectively improved.

Description

Bridge bearing fault identification method based on space attention
Technical Field
The invention relates to a bridge bearing fault identification method based on space attention, and belongs to the technical field of civil engineering and artificial intelligence interaction.
Background
With the rapid development of infrastructure construction, the civil engineering industry develops rapidly, and after a large number of bridges are constructed, the bridge support is an important stressed component of the bridge, and diseases such as aging, cracking and the like can occur in the long-term service process, so that the normal use function of the bridge support is influenced. People always rely on means such as daily and periodic inspection, sampling and temporary inspection and the like to acquire relevant information of the structure. However, the existing bridge appearance detection mainly depends on manual detection, the method is low in efficiency, long in time consumption and high in cost, the manual detection method is greatly influenced by factors such as environment, professional technical literacy of detection personnel and the like, and the detection result is uncertain.
The image processing technology based on deep learning is rapidly developed and widely applied to various industries, however, a deep learning model usually needs a large amount of labeled data to be trained so as to achieve higher precision. In actual engineering, some disease image data are difficult to obtain, and the support is used as a connecting member of a bridge, so that other members and some background information are included in the image acquisition engineering, which is difficult to avoid, and the identification result of the model is influenced. Therefore, a method for improving the identification precision of the model under the conditions of limited data set and complex scene is urgently needed.
Disclosure of Invention
The invention aims to solve the technical problem of providing a bridge support fault identification method based on space attention, which is used for improving the training efficiency and the precision of a neural network for identifying bridge support faults.
The invention adopts the following technical scheme for solving the technical problems: the invention designs a bridge support fault identification method based on space attention, which is characterized in that a target neural network based on a space attention system for identifying bridge support faults is trained and obtained by applying the following steps A to E based on a plurality of sample images respectively only containing one bridge support; the target neural network comprises a space coordinate generation network and a classification network; then, sequentially applying the space coordinates to generate a network and a classification network aiming at a target image to be detected, so as to obtain a classification result of the bridge bearing contained in the target image;
a, respectively associating each sample image with a preset classification label type of a bridge support contained in the sample image to construct a bridge support sample image database;
b, defining a minimum rectangular area where a bridge support is located in a sample image as a spatial attention area, and constructing a spatial coordinate generation network by taking the sample image as input and four spatial coordinates of the spatial attention area in the sample image as output based on a bridge support sample image database, wherein the four spatial coordinates of the spatial attention area are xmin,ymin,xmax,ymaxWherein x ismin,yminThe abscissa and ordinate, x, of the upper left corner of the spatial attention areamax,ymaxThe horizontal coordinate and the vertical coordinate of the right lower corner point of the spatial attention area are shown;
step C, aiming at each sample image, according to four space coordinates of a space attention area in the sample image, applying a lattice point generating function to obtain coordinates of each lattice point under the corresponding preset proportion grid division in the space attention area in the sample image, and then entering the step D;
step D, aiming at each sample image, obtaining the pixel value of each lattice point in a space attention area in the sample image through a preset interpolation method, and obtaining a processed image which corresponds to each sample image and is based on an attention mechanism;
and E, constructing a classification network which takes the pictures of the sample images based on the attention mechanism as input and the classes of the support images in the corresponding sample images as output based on the labels in the bridge support sample image database and the pictures of the sample images based on the attention mechanism, taking the difference between the output classes of the classification network and the label classes in the step A as a target of network optimization, and performing end-to-end iterative training on the spatial coordinates to generate a network and the classification network to obtain a target neural network.
As a preferred technical scheme of the invention, in the step a, the different states of the bridge support included in the constructed bridge support sample image database include both a normal support state and states of various support diseases preset in the service process of the bridge.
In a preferred embodiment of the present invention, in step a, the resolution of the image in the constructed bridge support sample image database is above 800 × 600.
As a preferred technical solution of the present invention, the spatial coordinate generation network in step B includes a convolutional layer, a pooling layer, and a full connection layer.
As a preferred embodiment of the present invention, in step C, the lattice point generating function generates image lattice points according to a proportional relationship between 4 image coordinates generated by the attention mechanism and the original image, and the lattice point generating function is described as follows:
Figure BDA0002822901610000021
wherein, x'ijIs the corresponding abscissa, y 'of the grid point i, j in the original image'ijIs the corresponding ordinate of the grid point i, j in the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, xijIs the abscissa, y, of the grid point i, j in the preset grid point imageijIs the ordinate of the grid point i, j in the preset grid point image.
As a preferred embodiment of the present invention, in step D, the preset interpolation method is a differentiable interpolation method to meet the requirement of error back propagation.
As a preferred technical solution of the present invention, in step E, the classification network is a neural network that satisfies the preset specified feature characterization capability.
As a preferred technical solution of the present invention, the neural network with the preset designated feature characterization capability is VGG or ResNet.
As a preferred technical solution of the present invention, in step E, a gradient descent method is selected for iterative training of the target neural network.
Compared with the prior art, the bridge bearing disease identification method based on space attention has the following technical effects:
according to the method, under a limited data set, a target neural network consisting of a spatial attention mechanism is obtained through a combined training spatial coordinate generation network and a classification network, aiming at a bridge support image to be detected, an interested area in the bridge support image is firstly identified, and then whether the bridge support is damaged or not and the type of the damage are obtained through the classification network.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of a spatial coordinate generation network in accordance with the present invention;
FIG. 3 is a schematic flow chart of the method for obtaining attention area according to the present invention;
fig. 4 is a schematic diagram of a VGG-16 classification network used in the present invention.
Detailed Description
The following description will explain embodiments of the present invention in further detail with reference to the accompanying drawings.
The hardware conditions for training the convolutional neural network of the present embodiment are: the amazon AWS cloud computing service is enabled to configure an amazon EC2P2.xlarge instance, the instance is configured with 1 GPU, 4 vCPUs, 61GB random access memories, the system adopts an ubuntu system, the programming language adopts python, and the deep learning platform uses pytorch.
As shown in the flowchart of fig. 1, in the method for identifying a bridge bearer fault based on spatial attention implemented in this embodiment, 5000 sample images with a resolution of more than 800 × 600 are selected, each sample image respectively includes different states of the bridge bearer, including a normal bearer state and states of various bearer faults preset in the bridge service process, where the bridge bearer states in the sample images in this embodiment include three types of a normal bearer, a cracked bearer, and a shear deformation bearer.
Based on the sample image, the following steps A to E are applied, and a target neural network which is used for identifying the bridge support diseases and is based on a space attention mechanism is trained and obtained; the target neural network comprises a space coordinate generation network and a classification network.
And A, respectively associating each sample image with a preset classification label type of the bridge support contained in the sample image to construct a bridge support sample image database, wherein the classification label in the embodiment contains three types of bridge supports, namely a normal support, a cracking support and a shearing deformation support.
Step B, as shown in FIG. 2, defining a minimum rectangular area where a bridge support is located in a sample image as a spatial attention area, constructing a spatial coordinate generation network based on a bridge support sample image database, wherein the spatial coordinate generation network takes the sample image as input and four spatial coordinates of the spatial attention area in the sample image as output, and comprises a convolution layer, a pooling layer and a full-connection layer, wherein the four spatial coordinates of the spatial attention area are xmin,ymin,xmax,ymaxWherein x ismin,yminThe abscissa and ordinate, x, of the upper left corner of the spatial attention areamax,ymaxThe abscissa and the ordinate of the lower right corner of the spatial attention area.
And step C, as shown in FIG. 3, applying a grid point generating function according to the four spatial coordinates of the spatial attention area in the sample image respectively for each sample image, and obtaining the coordinates of each grid point under the corresponding preset proportion grid division in the spatial attention area in the sample image.
The lattice point generating function generates image lattice points according to the proportional relation between 4 image coordinates generated by the attention mechanism and the original image, and the lattice point generating function is described as follows:
Figure BDA0002822901610000041
wherein, x'ijIs the corresponding abscissa, y 'of the grid point i, j in the original image'ijIs the corresponding ordinate of the grid point i, j in the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, xijIs the abscissa, y, of the grid point i, j in the preset grid point imageijIs the ordinate of the grid point i, j in the preset grid point image.
And D, aiming at each sample image, obtaining the pixel value of each lattice point in a spatial attention area in the sample image by a differentiable interpolation method meeting the requirement of error back propagation, wherein a bilinear interpolation method is specifically selected in the embodiment, and obtaining a processed image corresponding to each sample image based on an attention mechanism.
Step e, as shown in fig. 4, based on the tags in the bridge support sample image database and the attention-based pictures of each sample image, constructing a classification network that uses the attention-based pictures of each sample image as input and uses the category of the support image in the corresponding sample image as output and satisfies the preset specified feature characterization capability, specifically selecting a VGG network in a classical convolutional neural network in the present embodiment, then using the difference between the output category of the classification network and the category marked in step a as a target of network optimization, and performing joint training on the space coordinate generation network and the classification network by using a gradient descent method to obtain a target neural network.
And then, sequentially applying the space coordinates to generate a network and a classification network aiming at the target image to be detected, so as to obtain a classification result of the bridge bearing included in the target image.
The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.

Claims (9)

1. A bridge support fault identification method based on space attention is characterized in that a target neural network based on a space attention system for identifying bridge support faults is trained and obtained by applying the following steps A to E based on a plurality of sample images respectively only containing one bridge support; the target neural network comprises a space coordinate generation network and a classification network; then, sequentially applying the space coordinates to generate a network and a classification network aiming at a target image to be detected, so as to obtain a classification result of the bridge bearing contained in the target image;
a, respectively associating each sample image with a preset classification label type of a bridge support contained in the sample image to construct a bridge support sample image database;
b, defining a minimum rectangular area where a bridge support is located in a sample image as a spatial attention area, and constructing a spatial coordinate generation network by taking the sample image as input and four spatial coordinates of the spatial attention area in the sample image as output based on a bridge support sample image database, wherein the four spatial coordinates of the spatial attention area are xmin,ymin,xmax,ymaxWherein x ismin,yminThe abscissa and ordinate, x, of the upper left corner of the spatial attention areamax,ymaxThe horizontal coordinate and the vertical coordinate of the right lower corner point of the spatial attention area are shown;
step C, aiming at each sample image, according to four space coordinates of a space attention area in the sample image, applying a lattice point generating function to obtain coordinates of each lattice point under the corresponding preset proportion grid division in the space attention area in the sample image, and then entering the step D;
step D, aiming at each sample image, obtaining the pixel value of each lattice point in a space attention area in the sample image through a preset interpolation method, and obtaining a processed image which corresponds to each sample image and is based on an attention mechanism;
and E, constructing a classification network which takes the pictures of the sample images based on the attention mechanism as input and the classes of the support images in the corresponding sample images as output based on the labels in the bridge support sample image database and the pictures of the sample images based on the attention mechanism, taking the difference between the output classes of the classification network and the label classes in the step A as a target of network optimization, and performing end-to-end iterative training on the spatial coordinates to generate a network and the classification network to obtain a target neural network.
2. The method for identifying the bridge bearing diseases based on the spatial attention of claim 1, wherein in the step A, the different states of the bridge bearing contained in the constructed bridge bearing sample image database comprise a normal bearing state and states of various bearing diseases preset in the bridge service process.
3. The method for identifying bridge bearing diseases based on spatial attention according to claim 1, wherein in the step A, the resolution of the image in the bridge bearing sample image database is above 800 x 600.
4. The method for identifying the bridge bearing diseases based on the spatial attention of claim 1, wherein the spatial coordinate generation network in the step B comprises a convolutional layer, a pooling layer and a full-link layer.
5. The method for identifying the bridge bearing diseases based on the spatial attention of claim 1, wherein in the step C, a lattice point generating function generates image lattice points according to a proportional relation between 4 image coordinates generated by an attention mechanism and an original image, and the lattice point generating function is described as follows:
Figure FDA0002822901600000021
wherein, x'ijIs the corresponding abscissa, y 'of the grid point i, j in the original image'ijIs the corresponding ordinate of the grid point i, j in the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, xijIs the abscissa, y, of the grid point i, j in the preset grid point imageijIs the ordinate of the grid point i, j in the preset grid point image.
6. The method for identifying the bridge bearing diseases based on the spatial attention of claim 1, wherein in the step D, a preset interpolation method is a differentiable interpolation method so as to meet the requirement of error back propagation.
7. The method for identifying diseases of bridge supports based on spatial attention according to claim 1, wherein in the step E, the classification network is a neural network satisfying the characterization capability of preset designated features.
8. The method for identifying bridge bearing diseases based on spatial attention of claim 7, wherein the preset neural network with designated characteristic characterization capability is VGG or ResNet.
9. The method for identifying diseases of bridge supports based on spatial attention according to claim 1, wherein in step E, a gradient descent method is adopted for iterative training of the target neural network.
CN202011442501.7A 2020-12-08 2020-12-08 Bridge bearing disease identification method based on spatial attention Active CN112699736B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011442501.7A CN112699736B (en) 2020-12-08 2020-12-08 Bridge bearing disease identification method based on spatial attention

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011442501.7A CN112699736B (en) 2020-12-08 2020-12-08 Bridge bearing disease identification method based on spatial attention

Publications (2)

Publication Number Publication Date
CN112699736A true CN112699736A (en) 2021-04-23
CN112699736B CN112699736B (en) 2024-06-07

Family

ID=75507773

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011442501.7A Active CN112699736B (en) 2020-12-08 2020-12-08 Bridge bearing disease identification method based on spatial attention

Country Status (1)

Country Link
CN (1) CN112699736B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113297959A (en) * 2021-05-24 2021-08-24 南京邮电大学 Target tracking method and system based on corner attention twin network

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108288269A (en) * 2018-01-24 2018-07-17 东南大学 Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks
CN108363961A (en) * 2018-01-24 2018-08-03 东南大学 Bridge pad disease recognition method based on transfer learning between convolutional neural networks
WO2018214195A1 (en) * 2017-05-25 2018-11-29 中国矿业大学 Remote sensing imaging bridge detection method based on convolutional neural network
CN109784347A (en) * 2018-12-17 2019-05-21 西北工业大学 Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism
CN110070008A (en) * 2019-04-04 2019-07-30 中设设计集团股份有限公司 Bridge disease identification method adopting unmanned aerial vehicle image
CN110222701A (en) * 2019-06-11 2019-09-10 北京新桥技术发展有限公司 A kind of bridge defect automatic identifying method
CN111562220A (en) * 2020-06-02 2020-08-21 吉林大学 Rapid and intelligent detection method for bridge diseases
WO2020199538A1 (en) * 2019-04-04 2020-10-08 中设设计集团股份有限公司 Bridge key component disease early-warning system and method based on image monitoring data

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018214195A1 (en) * 2017-05-25 2018-11-29 中国矿业大学 Remote sensing imaging bridge detection method based on convolutional neural network
CN108288269A (en) * 2018-01-24 2018-07-17 东南大学 Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks
CN108363961A (en) * 2018-01-24 2018-08-03 东南大学 Bridge pad disease recognition method based on transfer learning between convolutional neural networks
CN109784347A (en) * 2018-12-17 2019-05-21 西北工业大学 Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism
CN110070008A (en) * 2019-04-04 2019-07-30 中设设计集团股份有限公司 Bridge disease identification method adopting unmanned aerial vehicle image
WO2020199538A1 (en) * 2019-04-04 2020-10-08 中设设计集团股份有限公司 Bridge key component disease early-warning system and method based on image monitoring data
CN110222701A (en) * 2019-06-11 2019-09-10 北京新桥技术发展有限公司 A kind of bridge defect automatic identifying method
CN111562220A (en) * 2020-06-02 2020-08-21 吉林大学 Rapid and intelligent detection method for bridge diseases

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113297959A (en) * 2021-05-24 2021-08-24 南京邮电大学 Target tracking method and system based on corner attention twin network

Also Published As

Publication number Publication date
CN112699736B (en) 2024-06-07

Similar Documents

Publication Publication Date Title
CN110059694B (en) Intelligent identification method for character data in complex scene of power industry
Vega Díaz et al. Solar panel detection within complex backgrounds using thermal images acquired by UAVs
Turker et al. Building‐based damage detection due to earthquake using the watershed segmentation of the post‐event aerial images
WO2020238256A1 (en) Weak segmentation-based damage detection method and device
CN112989452B (en) Identification method for labeling text on component lead in CAD water supply and drainage professional drawing
CN113033516A (en) Object identification statistical method and device, electronic equipment and storage medium
CN116128839A (en) Wafer defect identification method, device, electronic equipment and storage medium
CN113569852A (en) Training method and device of semantic segmentation model, electronic equipment and storage medium
CN110705400A (en) Method for automatically splitting examination paper layout questions
CN112488990B (en) Bridge bearing disease identification method based on attention regularization mechanism
CN114519819A (en) Remote sensing image target detection method based on global context awareness
CN110660068B (en) Semi-automatic brain region segmentation method for three-dimensional cell construction image
CN112767351A (en) Transformer equipment defect detection method based on sensitive position dependence analysis
CN112699736B (en) Bridge bearing disease identification method based on spatial attention
CN116758477A (en) Kitchen personnel dressing detection method based on improved YOLOv7 model
Sanders Neural networks, AI, phone-based VR, machine learning, computer vision and the CUNAT automated translation app–not your father’s archaeological toolkit
Augusto Costa et al. A convolutional neural network for detecting faults in power distribution networks along a railway: a case study using yolo
CN110334651B (en) Substation coordinate verification method based on transfer learning
CN115272956A (en) Chicken health degree monitoring method based on improved YOLOv5
CN114863274A (en) Surface green net thatch cover extraction method based on deep learning
Shi et al. Development of large-scale synthetic 3D point cloud datasets for vision-based bridge structural condition assessment
Bahreini et al. Dynamic graph CNN based semantic segmentation of concrete defects and as-inspected modeling
Fatty et al. Instance segmentation based building extraction in a dense urban area using multispectral aerial imagery data
CN113822148B (en) Intelligent identification method for trace tiny carryover based on convolutional neural network
Mykolaichuk et al. Image Segmentation in Loss Detection for Conservation and Restoration of Old Paintings

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province

Applicant after: Jiangxi Academy of Transportation Sciences Co.,Ltd.

Applicant after: SOUTHEAST University

Address before: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province

Applicant before: Jiangxi Transportation Research Institute

Applicant before: SOUTHEAST University

CB02 Change of applicant information
GR01 Patent grant