CN114758206A - Steel truss structure abnormity detection method and device - Google Patents
Steel truss structure abnormity detection method and device Download PDFInfo
- Publication number
- CN114758206A CN114758206A CN202210662669.1A CN202210662669A CN114758206A CN 114758206 A CN114758206 A CN 114758206A CN 202210662669 A CN202210662669 A CN 202210662669A CN 114758206 A CN114758206 A CN 114758206A
- Authority
- CN
- China
- Prior art keywords
- attention
- channel
- spatial
- remodeling
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 211
- 239000010959 steel Substances 0.000 title claims abstract description 211
- 238000001514 detection method Methods 0.000 title claims abstract description 91
- 230000004927 fusion Effects 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000007634 remodeling Methods 0.000 claims description 135
- 238000010586 diagram Methods 0.000 claims description 40
- 230000005856 abnormality Effects 0.000 claims description 38
- 238000010606 normalization Methods 0.000 claims description 18
- 230000003190 augmentative effect Effects 0.000 claims description 16
- 238000012549 training Methods 0.000 claims description 14
- 150000001875 compounds Chemical class 0.000 claims description 12
- 230000003416 augmentation Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims description 3
- 230000001788 irregular Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 17
- 238000010276 construction Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000011179 visual inspection Methods 0.000 description 2
- 235000021167 banquet Nutrition 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a steel truss structure abnormity detection method and a steel truss structure abnormity detection device, wherein the method comprises the following steps: acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module; acquiring a structural image of the steel truss to be detected; performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map; and determining a steel grid structure abnormity detection result based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map. According to the invention, the attention module is guided to improve the feature extraction and learning capacity of the complex irregular grid structure, and the accuracy of the steel grid structure abnormity detection result can be improved.
Description
Technical Field
The invention relates to the technical field of steel mesh frame structure detection, in particular to a steel mesh frame structure abnormity detection method and device.
Background
With the rapid soaring of society and economy, large and medium-sized public buildings such as natatoriums, stadiums, airport terminals, industrial factory buildings, exhibition halls, banquet halls and the like are highly valued by all countries in the world. The steel grid structure has the advantages of light weight, large space rigidity, good earthquake resistance, economic material, convenient construction, beautiful appearance and the like, and is widely applied. However, in the development of a large-span building, the scale of the steel net rack becomes larger and larger, and the structural form of the net rack becomes more complicated, so that the construction difficulty of the net rack is increased. Meanwhile, in order to pursue rapid and low-cost construction, the probability of quality safety problems of the steel truss structure in the construction process is increased. In addition, after the steel truss structure is built, the steel truss structure is corroded by the external environment for a long time, the effective section of the steel truss structure is reduced, the stress of the rod piece is larger, the strength and the rigidity of the steel frame are reduced, the steel frame enters the yield limit of the material too early, the structure is deformed more, the internal force is redistributed, and the safe use of the steel frame structure is influenced, and even the steel frame structure collapses suddenly. The problems of durability and safety of steel grid structures are increasingly prominent. The steel truss structure abnormity detection and identification are very necessary, and the steel truss structure abnormity detection and identification method has important significance for safety accident protection.
The traditional method is mainly based on visual inspection of the steel framework structure condition on a manual site, and the mode belongs to a labor-intensive task. Because the steel grid structure has large span and high height, the common visual inspection is difficult to completely relate to the whole structural details, and extremely heavy manual labor is inevitably brought. And because of individual difference, subjective judgment errors are inevitably introduced by manual inspection. More seriously, potential safety accidents can be caused by the high-altitude climbing operation. This makes it necessary to develop an intelligent anomaly detection method for a steel grid structure that is safe, convenient, and accurate. Therefore, the intelligent detection of the steel mesh frame structure abnormity through the deep learning neural network is provided in the prior art, but the steel mesh frame structure is complex and is often irregular, and the accuracy of the deep learning neural network in the prior art on the steel mesh frame structure abnormity detection is low.
Disclosure of Invention
In view of the above, it is necessary to provide a method and an apparatus for detecting an abnormality of a steel grid structure, so as to solve the technical problem in the prior art that the accuracy of detecting an abnormality of a steel grid structure is low.
In one aspect, the invention provides a steel truss structure abnormality detection method, which comprises the following steps:
Acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of a steel truss to be detected;
performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and determining the detection result of the abnormality of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
In some possible implementations, the attention directing module includes a first tensor stitching sub-module, a space attention sub-module, a channel attention sub-module, a first tensor summing sub-module, a first codec, a first dot product operation sub-module, a second tensor stitching sub-module, a second tensor summing sub-module, a second codec, and a third dot product operation sub-module;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
The spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel features in the first spliced feature map to obtain a first channel attention feature map;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion feature map to obtain a second spliced feature map;
The spatial attention submodule is further used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention sub-module is further used for extracting channel features in the second spliced feature map to obtain a second channel attention feature map;
the second tensor summation submodule is used for carrying out tensor summation on the second spatial attention feature map and the second channel attention feature map to obtain a second attention feature map;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
and the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result.
In some possible implementations, the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total guide loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs;
is the guidance loss in the nth characteristic diagram; 
A reconstruction constraint penalty for the nth signature;
is a first coding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; | | non-woven hair2Is a two-norm.
In some possible implementations, the spatial attention submodule includes a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer, and a spatial attention tensor concatenation layer;
the first spatial attention convolutional layer, the second spatial attention convolutional layer and the third spatial attention convolutional layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
the first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively performing spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map to obtain a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map correspondingly;
The first space attention dot product operation layer is used for performing dot product operation on the first space remodeling graph and the second space remodeling graph to obtain space correlation of each pixel position in the first space remodeling graph and the second space remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on the spatial correlation to obtain spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product map to obtain a fourth spatial remodeling map;
the space attention tensor splicing layer is used for carrying out tensor splicing on the fourth space remodeling graph and the first splicing feature graph to obtain the first space attention feature graph.
In some possible implementation manners, the channel attention submodule includes a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer, and a channel attention tensor splicing layer;
The first channel attention convolution layer, the second channel attention convolution layer and the third channel attention convolution layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map are correspondingly obtained;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature diagram, the second channel sub-feature diagram and the third channel sub-feature diagram to correspondingly obtain a first channel remodeling diagram, a second channel remodeling diagram and a third channel remodeling diagram;
the first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
The fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product map to obtain a fourth channel remodeling map;
the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling image and the first splicing characteristic image to obtain the first channel attention characteristic image.
In some possible implementation manners, the obtaining a well-trained target steel mesh structure abnormality detection model includes:
constructing an initial steel truss structure abnormity detection model;
constructing a steel mesh frame structure sample set;
and training the initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model with complete training.
In some possible implementations, the total loss function is:
in the formula (I), the compound is shown in the specification,
as a function of total loss;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focus factor.
In some possible implementations, the constructing a steel lattice structure sample set includes:
acquiring a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
performing first type augmentation treatment on the initial steel grid structure image sample set to obtain the steel grid structure sample set;
and/or the presence of a gas in the atmosphere,
and carrying out second type augmentation treatment on the initial steel grid structure image sample set to obtain the steel grid structure sample set.
In some possible implementations, the performing a second type of augmentation process on the initial set of steel grid structure image samples to obtain the set of steel grid structure samples includes:
randomly selecting a first initial steel grid structure image and a second initial steel grid structure image from the initial steel grid structure image sample set;
determining a first label image and a second label image of the first initial steel lattice structure image and the second initial steel lattice structure image, respectively;
randomly zooming the first initial steel grid structure image and the first label image to obtain a first zoomed steel grid structure image and a first random label image;
Randomly determining a target type, and obtaining a binary mask file according to the target type and the first random label image;
mixing the first scaled steel grid structure image, the second initial steel grid structure image, the first random label image and the second label image according to the binaryzation mask file to obtain an augmented steel grid structure image and an augmented label image;
and obtaining the steel grid structure sample set according to the initial steel grid structure image sample set, the augmented steel grid structure image and the augmented label image.
On the other hand, the invention also provides a steel truss structure abnormity detection device, which comprises:
the target detection model acquisition unit is used for acquiring a target steel grid structure abnormity detection model which is trained completely, and the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
the to-be-detected image acquisition unit is used for acquiring a to-be-detected steel truss structure image;
the multi-scale feature fusion unit is used for performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map;
And the abnormality detection unit is used for determining the detection result of the abnormality of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
The beneficial effects of adopting the embodiment are as follows: according to the steel grid structure abnormity detection method provided by the invention, the target steel grid structure abnormity detection model comprises the multi-scale fusion module and the attention guide module, and the attention guide module is used for improving the feature extraction and learning capacity of a complex irregular grid structure, so that the accuracy of the steel grid structure abnormity detection result can be improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of an embodiment of a steel framework structure anomaly detection method provided by the present invention;
FIG. 2 is a schematic structural diagram of an embodiment of a multi-scale fusion module provided in the present invention;
FIG. 3 is a schematic structural diagram of an attention module according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an embodiment of a spatial attention submodule provided in the present invention;
FIG. 5 is a schematic structural diagram of an embodiment of a channel attention submodule provided in the present invention;
FIG. 6 is a schematic flow chart of one embodiment of S101 of FIG. 1;
FIG. 7 is a flowchart illustrating an embodiment of S602 in FIG. 6 according to the present invention;
FIG. 8 is a flowchart illustrating an embodiment of S703 of FIG. 7 according to the present invention;
FIG. 9 is a schematic structural view of an embodiment of the steel grid structure abnormality detection apparatus provided in the present invention;
fig. 10 is a schematic structural diagram of an embodiment of an electronic device provided in the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be understood that the schematic drawings are not necessarily to scale. The flowcharts used in this disclosure illustrate operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flow diagrams may be performed out of order, and that steps without logical context may be performed in reverse order or concurrently. One skilled in the art, under the direction of this summary, may add one or more other operations to, or remove one or more operations from, the flowchart.
In the description of the embodiment of the present invention, "and/or" describes an association relationship of an association object, which means that three relationships may exist, for example: a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone.
Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor systems and/or microcontroller systems.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein may be combined with other embodiments.
The embodiment of the invention provides a steel truss structure abnormity detection method and a steel truss structure abnormity detection device, which are respectively explained below.
Fig. 1 is a schematic flow diagram of an embodiment of a steel grid structure abnormality detection method provided by the present invention, and as shown in fig. 1, the steel grid structure abnormality detection method includes:
s101, acquiring a target steel grid structure abnormity detection model with complete training, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
s102, acquiring a structural image of the steel truss to be detected;
s103, performing multi-scale feature extraction on the steel space truss structure image to be detected based on a multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and S104, determining a steel grid structure abnormity detection result based on the attention guiding module, the plurality of characteristic diagrams and the multi-scale fusion characteristic diagram.
Compared with the prior art, the steel grid structure abnormity detection method provided by the embodiment of the invention has the advantages that the target steel grid structure abnormity detection model comprises the multi-scale fusion module and the attention guiding module, and the attention guiding module is used for improving the feature extraction and learning capacity of the complex irregular grid structure, so that the accuracy of the steel grid structure abnormity detection result can be improved.
In a specific embodiment of the present invention, the multi-scale fusion module is a network structure left after resinet 50 goes out of a connection layer, specifically: as shown in fig. 2, the multi-scale fusion module includes 4 residual error units, 4 upsampling layers corresponding to the 4 residual error units one by one, a multi-scale tensor splicing layer, and a multi-scale convolutional layer, which are connected in sequenceEach residual unit is used for extracting features of one scale, extracting features of 4 scales, and obtaining four feature maps which are F respectively after passing through an upsampling layer1’、F2’、F3' and F4’。F1’、F2’、F3' and F4After passing through a multi-scale tensor splicing layer and a multi-scale convolution layer, a multi-scale fusion eigen map F is generatedMS。
In some embodiments of the present invention, as shown in fig. 3, the attention directing module includes a first tensor splicing sub-module, a spatial attention sub-module, a channel attention sub-module, a first tensor summing sub-module, a first codec, a first dot product operation sub-module, a second tensor splicing sub-module, a second tensor summing sub-module, a second codec, and a third dot product operation sub-module;
the first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
The spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel characteristics in the first spliced characteristic diagram to obtain a first channel attention characteristic diagram;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coder-decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion characteristic graph to obtain a second spliced characteristic graph;
the spatial attention submodule is also used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
The channel attention sub-module is also used for extracting channel characteristics in the second spliced characteristic diagram to obtain a second channel attention characteristic diagram;
the second tensor summation submodule is used for carrying out tensor summation on the second space attention characteristic diagram and the second channel attention characteristic diagram to obtain a second attention characteristic diagram;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
and the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result.
It should be noted that: the first codec and the second codec in the embodiment of the present invention are configured as UNet network structures, and are not described in detail herein.
In a specific embodiment of the present invention, the attention loss function of the direct attention module is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total guide loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs;
is the guidance loss in the nth characteristic diagram;
a reconstruction constraint penalty for the nth signature;
is a first coding feature; 
Is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; | | non-woven hair2Is a two-norm.
According to the embodiment of the invention, the results of the attention loss function of the attention guiding module for constraining the space attention and the channel attention are consistent as much as possible before and after passing through different encoders and decoders, so that the accuracy of detecting the abnormal steel truss structure can be further improved.
In some embodiments of the present invention, as shown in fig. 4, the spatial attention submodule includes a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation (softmax) layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer, and a spatial attention tensor concatenation layer;
the first spatial attention convolution layer, the second spatial attention convolution layer and the third spatial attention convolution layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
The first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively carrying out spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map, and correspondingly obtaining a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map;
the first space attention dot product operation layer is used for performing dot product operation on the first space remodeling graph and the second space remodeling graph to obtain the space correlation of each pixel position in the first space remodeling graph and the second space remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on spatial correlation to obtain spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product diagram to obtain a fourth spatial remodeling diagram;
and the spatial attention tensor splicing layer is used for carrying out tensor splicing on the fourth spatial remodeling image and the first splicing characteristic image to obtain a first spatial attention characteristic image.
In a specific embodiment of the present invention, the spatial correlation of each pixel position in the first spatial remodeling map and the second spatial remodeling map is:
In the formula (I), the compound is shown in the specification,
a spatial correlation for each pixel position in the first and second spatial remodelling maps;
reshaping an ith pixel position in the first spatial reshape map;
reshaping a jth pixel position in the second spatial reshape map; w, H are the width and height of the first and second spatial remodeling maps, respectively.
First spatial attention profile
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
is a spatial attention weight;
a third spatial remodeling map;
a spatial correlation matrix for the first and second spatial remodelling maps;
is a first stitching signature.
It should be understood that: the spatial attention module is further configured to obtain a second spatial attention feature map according to the second stitching feature map, and a process of the spatial attention module is the same as the process of obtaining the first spatial attention feature map according to the first stitching feature map, which is not described in detail herein.
In some embodiments of the present invention, as shown in fig. 5, the channel attention submodule includes a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation (softmax) layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer, and a channel attention tensor splicing layer;
The first channel attention convolution layer, the second channel attention convolution layer and the third channel attention convolution layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map are correspondingly obtained;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature map, the second channel sub-feature map and the third channel sub-feature map so as to correspondingly obtain a first channel remodeling map, a second channel remodeling map and a third channel remodeling map;
the first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
the fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product map to obtain a fourth channel remodeling map;
And the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling graph and the first splicing characteristic graph to obtain the first channel attention characteristic graph.
In a specific embodiment of the present invention, the channel correlation for each pixel position in the first spatial remodeling map and the second spatial remodeling map is:
in the formula (I), the compound is shown in the specification,
correlating the channels in the first channel remodeling map and the second channel remodeling map;
remodeling the ith channel in the map for the first channel;
remodeling a jth channel in the graph for the second channel; and c is the total number of channels in the first channel remodeling map and the second channel remodeling map.
First channel attention feature map
Comprises the following steps:
in the formula (I), the compound is shown in the specification,
is the channel attention weight;
remodeling the third channel with a map;
a channel correlation matrix for the first channel remodeling map and the second channel remodeling map;
is a first stitching signature.
It should be understood that: the channel attention module is further configured to obtain a second channel attention feature map according to the second stitched feature map, and a process of the channel attention module is the same as the process of obtaining the first channel attention feature map according to the first stitched feature map, which is not described in detail herein.
In some embodiments of the present invention, as shown in fig. 6, step S101 includes:
S601, constructing an initial steel framework structure abnormity detection model;
s602, constructing a steel framework structure sample set;
s603, training an initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function, and obtaining a target steel grid structure abnormity detection model with complete training.
Because in the actual scene, for background and normal steel framework structure, the condition that appears unusual steel framework structure is rare, promptly: in order to improve the detection accuracy of the unbalanced samples, in some embodiments of the present invention, the total loss function is:
in the formula (I), the compound is shown in the specification,
is a total loss function;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focus factor.
Initial weight
Is determined by a weighting coefficient
The control is carried out by controlling the temperature of the air conditioner,
the larger the initial weight of the class with the smaller frequency of occurrence is. The dynamic correction quantity of the weight is controlled by the focusing coefficient, and the larger the dynamic correction quantity of the weight is, the stronger the function of the model for relieving the category unbalance phenomenon by adjusting the weight is. Therefore, the class training loss with less sample size and the sample training loss with larger sample size reach a balanced state, and the problem caused by unbalanced class of the training set is solved. Therefore, the detection accuracy of the steel truss structure abnormity detection can be improved.
Further, in order to improve the training speed of the initial steel grid structure abnormality detection model, find the optimal solution to obtain the target steel grid structure abnormality detection model which is completely trained, in some embodiments of the present invention, the learning rate dynamic adjustment strategy in the training process is a Poly strategy, specifically, the Poly strategy is:
in the formula (I), the compound is shown in the specification,
is a new learning rate;
a reference learning rate;
the current iteration number is;
is the maximum iteration number;
to control the form factor of the shape of the curve.
According to the embodiment of the invention, the learning rate dynamic adjustment strategy in the training process is set to the Poly strategy, so that the training speed of the initial steel mesh frame structure abnormity detection model can be increased.
In some embodiments of the present invention, as shown in fig. 7, step S602 includes:
s701, obtaining a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
s702, performing first-class augmentation treatment on the initial steel mesh frame structure image sample set to obtain a steel mesh frame structure sample set;
and/or the presence of a gas in the gas,
and S703, performing second type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set.
According to the embodiment of the invention, the first type of augmentation treatment and/or the second type of augmentation treatment are/is carried out on the initial steel grid structure image sample set, so that the image quantity of the formed steel grid structure sample set can be increased, and the generalization capability of the formed target steel grid structure abnormality detection model can be improved.
In a specific embodiment of the present invention, the slicing process in step S701 specifically includes: the steel lattice structure image was sliced, each slice having a size of 512 × 3. The labels in step S701 are specifically: and (3) digitally expressing the object types in each slice, wherein the background type, the normal steel net rack type and the abnormal steel net rack type are respectively expressed as 0, 1 and 2.
In an embodiment of the present invention, the first type of augmentation process in step S702 is: and (4) turning, rotating, zooming, cutting, adjusting the color and the like on each slice in the initial steel mesh frame structure image sample set.
In some embodiments of the present invention, as shown in fig. 8, step S703 includes:
s801, randomly selecting a first initial steel grid structure image from the initial steel grid structure image sample set
And a second initial steel lattice structure image
；
S802, respectively determining a first initial steel grid structure image I AAnd a second initial steel lattice structure image IBFirst label image of
And a second label image
；
S803, forming the first initial steel mesh frame structure image
And a first label image
Carrying out random zooming to obtain a first zoomed steel grid structure image
And a first random label image
;
S804, randomly determining a target class c, and according to the target class c and the first random label image
Obtaining a binary mask file
；
S805, according to the binary mask file
The first scaling steel mesh frame structure image
Second initial steel grid structure image
First random label image
And a second label image
Mixing to obtain an augmented steel grid structure image X and an augmented label image Y;
s806, obtaining a steel framework structure sample set according to the initial steel framework structure image sample set, the augmented steel framework structure image X and the augmented label image Y.
Specifically, a binary mask file
Comprises the following steps:
the image X of the augmented steel grid structure and the image Y of the augmented label are respectively as follows:
in the formula (I), the compound is shown in the specification,
is Hadamard (Hadamar)d) And (4) performing product operation.
According to the invention, the augmented steel mesh structure image and the augmented label image are obtained by the category scaling and mixing method, so that the sample number of the steel mesh structure sample set can be further increased, and the reliability of the target steel mesh structure abnormity detection model is ensured.
In order to better implement the steel truss structure abnormality detection method in the embodiment of the present invention, on the basis of the steel truss structure abnormality detection method, correspondingly, the embodiment of the present invention further provides a steel truss structure abnormality detection apparatus, as shown in fig. 9, the steel truss structure abnormality detection apparatus 900 includes:
a target detection model obtaining unit 901, configured to obtain a target steel grid structure abnormality detection model which is completely trained, where the target steel grid structure abnormality detection model includes a multi-scale fusion module and a guidance attention module;
the to-be-detected image acquisition unit 902 is used for acquiring a to-be-detected steel truss structure image;
a multi-scale feature fusion unit 903, configured to perform multi-scale feature extraction on the steel space truss structure image to be detected based on a multi-scale fusion module to obtain a plurality of feature maps, and fuse the plurality of feature maps to obtain a multi-scale fusion feature map;
and an anomaly detection unit 904, configured to determine a steel grid structure anomaly detection result based on the guidance attention module and the plurality of feature maps and the multi-scale fusion feature map.
The steel truss structure abnormality detection apparatus 900 provided in the above-mentioned embodiment can implement the technical solutions described in the above-mentioned steel truss structure abnormality detection method embodiments, and the specific implementation principles of the above-mentioned modules or units can refer to the corresponding contents in the above-mentioned steel truss structure abnormality detection method embodiments, and are not described here again.
As shown in fig. 10, the present invention further provides an electronic device 1000 accordingly. The electronic device 1000 includes a processor 1001, a memory 1002, and a display 1003. Fig. 10 shows only some of the components of the electronic device 1000, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The processor 1001 may be a Central Processing Unit (CPU), a microprocessor or other data Processing chip in some embodiments, and is used to run program codes stored in the memory 1002 or process data, such as the steel grid structure abnormality detection method in the present invention.
In some embodiments, processor 1001 may be a single server or a group of servers. The server groups may be centralized or distributed. In some embodiments, the processor 1001 may be local or remote. In some embodiments, the processor 1001 may be implemented in a cloud platform. In an embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an intra-house, a multi-cloud, and the like, or any combination thereof.
The storage 1002 may be an internal storage unit of the electronic device 1000 in some embodiments, such as a hard disk or a memory of the electronic device 1000. The memory 1002 may also be an external storage device of the electronic device 1000 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, provided on the electronic device 1000.
Further, the memory 1002 may also include both internal storage units and external storage devices for the electronic device 1000. The memory 1002 is used for storing application software and various data for installing the electronic apparatus 1000.
The display 1003 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like in some embodiments. The display 1003 is used to display information at the electronic device 1000 and to display a visual user interface. The components 1001 and 1003 of the electronic device 1000 communicate with each other through a system bus.
In one embodiment, when the processor 1001 executes the steel grid structure abnormality detection program in the memory 1002, the following steps may be implemented:
acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of a steel truss to be detected;
performing multi-scale feature extraction on a steel space truss structure image to be detected based on a multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
And determining the abnormal detection result of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
It should be understood that: when the processor 1001 executes the steel grid structure abnormality detection program in the memory 1002, other functions may be implemented in addition to the above functions, which may be specifically referred to the description of the corresponding method embodiment above.
Further, the type of the mentioned electronic device 1000 is not specifically limited in the embodiment of the present invention, and the electronic device 1000 may be a portable electronic device such as a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a wearable device, and a laptop computer (laptop). Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices that carry an IOS, android, microsoft, or other operating system. The portable electronic device may also be other portable electronic devices such as laptop computers (laptop) with touch sensitive surfaces (e.g., touch panels) and the like. It should also be understood that in other embodiments of the present invention, the electronic device 1000 may not be a portable electronic device, but may be a desktop computer having a touch-sensitive surface (e.g., a touch pad).
Accordingly, the embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium is used to store a computer-readable program or instruction, and when the program or instruction is executed by a processor, the step or the function in the steel framework structure abnormality detection method provided in the foregoing method embodiments can be implemented.
Those skilled in the art will appreciate that all or part of the processes of the methods of the above embodiments may be implemented by instructing relevant hardware (such as a processor, a controller, etc.) by a computer program, and the computer program may be stored in a computer readable storage medium. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The steel truss structure abnormality detection method and device provided by the invention are described in detail above, specific examples are applied in the text to explain the principle and the implementation of the invention, and the description of the above embodiments is only used to help understanding the method and the core idea of the invention; meanwhile, for those skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed, and in summary, the content of the present specification should not be construed as limiting the present invention.
Claims (10)
1. A steel truss structure abnormality detection method is characterized by comprising the following steps:
acquiring a target steel grid structure abnormity detection model which is trained completely, wherein the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
acquiring a structural image of a steel truss to be detected;
performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the plurality of feature maps to obtain a multi-scale fusion feature map;
and determining the detection result of the abnormality of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
2. The steel truss structure abnormality detection method according to claim 1, wherein the attention-directing module includes a first tensor stitching sub-module, a space attention sub-module, a channel attention sub-module, a first tensor summing sub-module, a first codec, a first dot product operation sub-module, a second tensor stitching sub-module, a second tensor summing sub-module, a second codec, and a third dot product operation sub-module;
The first vector splicing submodule is used for splicing the multiple feature maps and the multi-scale fusion feature map to obtain a first spliced feature map;
the spatial attention submodule is used for extracting spatial features in the first spliced feature map to obtain a first spatial attention feature map;
the channel attention submodule is used for extracting channel features in the first spliced feature map to obtain a first channel attention feature map;
the first tensor summation submodule is used for carrying out tensor summation on the first spatial attention feature map and the first channel attention feature map to obtain a first attention feature map;
the first coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a first coding feature and a first decoding result;
the first dot product operation sub-module is used for performing dot product operation on the first attention feature map and the first decoding result to obtain a first dot product result;
the second dot product operation sub-module is used for performing dot product operation on the first dot product result and the plurality of feature maps to obtain a second dot product result;
the second tensor splicing submodule is used for splicing the second dot product result and the multi-scale fusion characteristic diagram to obtain a second spliced characteristic diagram;
The spatial attention submodule is further used for extracting spatial features in the second spliced feature map to obtain a second spatial attention feature map;
the channel attention submodule is also used for extracting channel features in the second spliced feature map to obtain a second channel attention feature map;
the second tensor summation submodule is used for carrying out tensor summation on the second spatial attention feature map and the second channel attention feature map to obtain a second attention feature map;
the second coding decoder is used for coding and decoding the multiple feature maps and the multi-scale fusion feature map to obtain a second coding feature and a second decoding result;
and the third dot product operation sub-module is used for performing dot product operation on the second attention feature map and the second decoding result to obtain a steel grid structure abnormity detection result.
3. The method for detecting abnormality of steel lattice structure according to claim 2, wherein the attention loss function of the module for directing attention is:
in the formula (I), the compound is shown in the specification,
as a function of attention loss;
is the total guide loss;
loss of total structural constraint; n is the total number of the plurality of characteristic graphs; 
Is the guidance loss in the nth characteristic diagram;
a reconstruction constraint penalty for the nth signature;
is a first coding feature;
is a second coding feature;
a first splicing characteristic diagram;
is a first decoding result;
a second mosaic characteristic diagram;
is a second decoding result;
is a first weight;
is a second weight; | | non-woven hair2Is a two-norm.
4. The method for detecting the anomaly in the steel truss structure according to the claim 2, wherein the spatial attention submodule comprises a first spatial attention convolution layer, a second spatial attention convolution layer, a third spatial attention convolution layer, a first spatial attention remodeling layer, a second spatial attention remodeling layer, a third spatial attention remodeling layer, a first spatial attention dot product operation layer, a spatial attention normalization operation layer, a second spatial attention dot product operation layer, a fourth spatial attention remodeling layer and a spatial attention tensor splicing layer;
the first spatial attention convolutional layer, the second spatial attention convolutional layer and the third spatial attention convolutional layer are used for respectively carrying out spatial feature extraction on the first splicing feature map to correspondingly obtain a first spatial sub-feature map, a second spatial sub-feature map and a third spatial sub-feature map;
The first spatial attention remodeling layer, the second spatial attention remodeling layer and the third spatial attention remodeling layer are used for respectively performing spatial dimension remodeling on the first spatial sub-feature map, the second spatial sub-feature map and the third spatial sub-feature map to obtain a first spatial remodeling map, a second spatial remodeling map and a third spatial remodeling map correspondingly;
the first spatial attention dot product operation layer is used for performing dot product operation on the first spatial remodeling graph and the second spatial remodeling graph to obtain spatial correlation of each pixel position in the first spatial remodeling graph and the second spatial remodeling graph;
the spatial attention normalization operation layer is used for performing normalization operation on the spatial correlation to obtain spatial attention weight;
the second spatial attention dot product operation layer is used for obtaining a second spatial dot product graph based on the spatial attention weight and the third spatial remodeling graph;
the fourth spatial attention remodeling layer is used for performing spatial dimension remodeling on the second spatial dot product diagram to obtain a fourth spatial remodeling diagram;
the space attention tensor splicing layer is used for carrying out tensor splicing on the fourth space remodeling image and the first splicing characteristic image to obtain the first space attention characteristic image.
5. The method for detecting the abnormality of the steel truss structure according to claim 2, wherein the channel attention submodule comprises a first channel attention convolution layer, a second channel attention convolution layer, a third channel attention convolution layer, a first channel attention remodeling layer, a second channel attention remodeling layer, a third channel attention remodeling layer, a first channel attention dot product operation layer, a channel attention normalization operation layer, a second channel attention dot product operation layer, a fourth channel attention remodeling layer and a channel attention tensor splicing layer;
the first channel attention convolution layer, the second channel attention convolution layer and the third channel attention convolution layer are used for respectively carrying out channel feature extraction on the first splicing feature map, and a first channel sub-feature map, a second channel sub-feature map and a third channel sub-feature map are correspondingly obtained;
the first channel attention remodeling layer, the second channel attention remodeling layer and the third channel attention remodeling layer are used for respectively carrying out channel dimension remodeling on the first channel sub-feature map, the second channel sub-feature map and the third channel sub-feature map, and correspondingly obtaining a first channel remodeling map, a second channel remodeling map and a third channel remodeling map;
The first channel attention dot product operation layer is used for performing dot product operation on the first channel remodeling graph and the second channel remodeling graph to obtain channel correlation of each channel in the first channel remodeling graph and the second channel remodeling graph;
the channel attention normalization operation layer is used for performing normalization operation on the channel correlation to obtain a channel attention weight;
the second channel attention dot product operation layer is used for performing dot product operation on the basis of the channel attention weight and the third channel remodeling graph to obtain a second channel dot product graph;
the fourth channel attention remodeling layer is used for performing channel dimension remodeling on the second channel dot product diagram to obtain a fourth channel remodeling diagram;
the channel attention tensor splicing layer is used for carrying out tensor splicing on the fourth channel remodeling graph and the first splicing feature graph to obtain the first channel attention feature graph.
6. The method for detecting the abnormality of the steel lattice structure according to claim 3, wherein the obtaining of the target steel lattice structure abnormality detection model which is trained completely comprises:
constructing an initial steel truss structure abnormity detection model;
constructing a steel mesh frame structure sample set;
And training the initial steel grid structure abnormity detection model according to the steel grid structure sample set and a preset total loss function to obtain a target steel grid structure abnormity detection model with complete training.
7. The method for detecting abnormality of a steel truss structure according to claim 6, wherein said total loss function is:
in the formula (I), the compound is shown in the specification,
as a function of total loss;
is a third weight;
to improve the focus loss function;
the true category of the pixel point is;
the confidence that the pixel point is the c-th class is given;
the occurrence frequency of the c-type pixel points in the steel mesh frame structure sample set is shown;
is a negative class gating coefficient;
is the initial weight value;
is a weighting coefficient;
is the focus factor.
8. The method for detecting the abnormality of the steel lattice structure according to claim 6, wherein the constructing a steel lattice structure sample set includes:
acquiring a steel grid structure image, and carrying out slicing processing and labeling on the steel grid structure image to obtain an initial steel grid structure image sample set;
performing first type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set;
and/or the presence of a gas in the gas,
and carrying out second type augmentation treatment on the initial steel mesh frame structure image sample set to obtain the steel mesh frame structure sample set.
9. The method for detecting the abnormality of the steel lattice structure according to claim 8, wherein the performing of the second kind of augmentation process on the initial steel lattice structure image sample set to obtain the steel lattice structure sample set includes:
randomly selecting a first initial steel grid structure image and a second initial steel grid structure image from the initial steel grid structure image sample set;
determining a first label image and a second label image of the first initial steel lattice structure image and the second initial steel lattice structure image, respectively;
randomly zooming the first initial steel grid structure image and the first label image to obtain a first zoomed steel grid structure image and a first random label image;
randomly determining a target type, and obtaining a binary mask file according to the target type and the first random label image;
mixing the first scaled steel grid structure image, the second initial steel grid structure image, the first random label image and the second label image according to the binaryzation mask file to obtain an augmented steel grid structure image and an augmented label image;
and obtaining the steel grid structure sample set according to the initial steel grid structure image sample set, the augmented steel grid structure image and the augmented label image.
10. A steel truss structure abnormality detection apparatus, comprising:
the target detection model acquisition unit is used for acquiring a target steel grid structure abnormity detection model which is trained completely, and the target steel grid structure abnormity detection model comprises a multi-scale fusion module and an attention guide module;
the device comprises an image acquisition unit to be detected, a data acquisition unit and a data processing unit, wherein the image acquisition unit to be detected is used for acquiring a steel truss structure image to be detected;
the multi-scale feature fusion unit is used for performing multi-scale feature extraction on the steel truss structure image to be detected based on the multi-scale fusion module to obtain a plurality of feature maps, and fusing the feature maps to obtain a multi-scale fusion feature map;
and the abnormity detection unit is used for determining the abnormity detection result of the steel grid structure based on the attention guiding module, the plurality of feature maps and the multi-scale fusion feature map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662669.1A CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210662669.1A CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114758206A true CN114758206A (en) | 2022-07-15 |
| CN114758206B CN114758206B (en) | 2022-10-28 |
Family
ID=82336676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210662669.1A Active CN114758206B (en) | 2022-06-13 | 2022-06-13 | Steel truss structure abnormity detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114758206B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861306A (en) * | 2023-02-20 | 2023-03-28 | 武汉纺织大学 | Industrial product abnormity detection method based on self-supervision jigsaw module |
| CN117409327A (en) * | 2023-12-13 | 2024-01-16 | 中国地质调查局武汉地质调查中心(中南地质科技创新中心) | Dynamic quantitative remote sensing monitoring method for surface mine |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN110209713A (en) * | 2019-06-26 | 2019-09-06 | 国家电网有限公司 | Abnormal grid structure recognition methods and device |
| CN112784685A (en) * | 2020-12-28 | 2021-05-11 | 山东师范大学 | Crowd counting method and system based on multi-scale guiding attention mechanism network |
| CN113435269A (en) * | 2021-06-10 | 2021-09-24 | 华东师范大学 | Improved water surface floating object detection and identification method and system based on YOLOv3 |
| CN114332007A (en) * | 2021-12-28 | 2022-04-12 | 福州大学 | Transformer-based industrial defect detection and identification method |
| WO2022090483A1 (en) * | 2020-11-02 | 2022-05-05 | Tomtom Global Content B.V. | Neural network models for semantic image segmentation |
| CN114529839A (en) * | 2022-02-09 | 2022-05-24 | 国网河南省电力公司电力科学研究院 | Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system |
-
2022
- 2022-06-13 CN CN202210662669.1A patent/CN114758206B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019153908A1 (en) * | 2018-02-11 | 2019-08-15 | 北京达佳互联信息技术有限公司 | Image recognition method and system based on attention model |
| CN110209713A (en) * | 2019-06-26 | 2019-09-06 | 国家电网有限公司 | Abnormal grid structure recognition methods and device |
| WO2022090483A1 (en) * | 2020-11-02 | 2022-05-05 | Tomtom Global Content B.V. | Neural network models for semantic image segmentation |
| CN112784685A (en) * | 2020-12-28 | 2021-05-11 | 山东师范大学 | Crowd counting method and system based on multi-scale guiding attention mechanism network |
| CN113435269A (en) * | 2021-06-10 | 2021-09-24 | 华东师范大学 | Improved water surface floating object detection and identification method and system based on YOLOv3 |
| CN114332007A (en) * | 2021-12-28 | 2022-04-12 | 福州大学 | Transformer-based industrial defect detection and identification method |
| CN114529839A (en) * | 2022-02-09 | 2022-05-24 | 国网河南省电力公司电力科学研究院 | Unmanned aerial vehicle routing inspection-oriented power transmission line hardware anomaly detection method and system |
Non-Patent Citations (3)
| Title |
|---|
| ASHISH SINHA 等: "Multi-Scale Self-Guided Attention for Medical Image Segmentation", 《IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS》 * |
| 王俊等: "结合深度学习去噪和超分辨的SAR检测识别", 《空天防御》 * |
| 赵绪 等: "结合注意力机制的乳腺双模态超声分类网络", 《中国图象图形学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861306A (en) * | 2023-02-20 | 2023-03-28 | 武汉纺织大学 | Industrial product abnormity detection method based on self-supervision jigsaw module |
| CN117409327A (en) * | 2023-12-13 | 2024-01-16 | 中国地质调查局武汉地质调查中心(中南地质科技创新中心) | Dynamic quantitative remote sensing monitoring method for surface mine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114758206B (en) | 2022-10-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114758206B (en) | Steel truss structure abnormity detection method and device | |
| CN107328420A (en) | Localization method and device | |
| US11861919B2 (en) | Text recognition method and device, and electronic device | |
| CN112163447A (en) | Multi-task real-time gesture detection and recognition method based on Attention and Squeezenet | |
| Guo et al. | Adaln: a vision transformer for multidomain learning and predisaster building information extraction from images | |
| CN112132037B (en) | Pavement detection method, device, equipment and medium based on artificial intelligence | |
| CN115937993A (en) | Living body detection model training method, living body detection device and electronic equipment | |
| CN113591881B (en) | Intention recognition method and device based on model fusion, electronic equipment and medium | |
| CN115578358A (en) | RecTransformer-based surface defect detection model construction method | |
| Wang et al. | Pyramid self-attention mechanism-based change detection in hyperspectral imagery | |
| CN115187660A (en) | Knowledge distillation-based multi-person human body posture estimation method and system | |
| Ding et al. | A novel deep learning framework for detecting seafarer’s unsafe behavior | |
| CN112749707A (en) | Method, apparatus, and medium for object segmentation using neural networks | |
| CN106647411A (en) | Agent construction method and Agent construction device of information physical fusion system CPS | |
| Wang et al. | CAW: A Remote‐Sensing Scene Classification Network Aided by Local Window Attention | |
| Bi et al. | Refined regression detector for multiclass-oriented target in optical remote sensing images | |
| Wu et al. | A deep learning-based attribute adaptive infrared maritime ship target detection method | |
| CN117542085B (en) | Park scene pedestrian detection method, device and equipment based on knowledge distillation | |
| Wang et al. | PSR-Net: A Dual-Branch Pyramid Semantic Reasoning Network for Segmentation of Remote Sensing Images | |
| De Luca et al. | Analytic solution of the lifeguard problem | |
| CN117456530B (en) | Building contour segmentation method, system, medium and equipment based on remote sensing image | |
| Zhao et al. | Lightweight Smoke Recognition Based on Deep Convolution and Self‐Attention | |
| Zhang | Attention-based U-Net for building extraction from remote sensing images | |
| CN116092090A (en) | Reading sequence prediction method, training method and device of reading sequence prediction model | |
| Zhang et al. | Design of signage guidance system for tourist attractions based on computer vision technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Mingpeng Inventor after: Gao Jian Inventor before: Li Mingpeng Inventor before: Gao Jian Inventor before: Hu Xudong |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |