CN118052811B - NAM-DSSD model-based aircraft skin defect detection method - Google Patents
NAM-DSSD model-based aircraft skin defect detection method Download PDFInfo
- Publication number
- CN118052811B CN118052811B CN202410425461.7A CN202410425461A CN118052811B CN 118052811 B CN118052811 B CN 118052811B CN 202410425461 A CN202410425461 A CN 202410425461A CN 118052811 B CN118052811 B CN 118052811B
- Authority
- CN
- China
- Prior art keywords
- resolution image
- image set
- resolution
- aircraft skin
- nam
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 41
- 230000007547 defect Effects 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 63
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 45
- 238000012549 training Methods 0.000 claims abstract description 20
- 239000013598 vector Substances 0.000 claims abstract description 20
- 230000008569 process Effects 0.000 claims description 22
- 230000000903 blocking effect Effects 0.000 claims description 13
- 230000007246 mechanism Effects 0.000 claims description 13
- 238000000638 solvent extraction Methods 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 3
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 5
- 230000015556 catabolic process Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000006731 degradation reaction Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000005728 strengthening Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000005808 skin problem Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to an aircraft skin defect detection method based on a NAM-DSSD model, which comprises the following steps: acquiring a high-resolution image set, and performing downsampling treatment to obtain a low-resolution image set; extracting high-frequency features of the high-resolution image set and edge texture detail features of the medium-resolution image set respectively, and converting the obtained paired feature blocks into high-resolution image vectors and low-resolution image vectors; reconstructing the high-resolution image set by using a parallel dictionary super-resolution reconstruction algorithm to finally obtain the high-resolution image set for recovering more image details; and inputting the data set obtained based on data enhancement into an improved model for network training, and identifying and classifying the defect types of the aircraft skin. According to the invention, the data set after aircraft skin reinforcement is input into the algorithm with strong detection capability for small targets to perform network training, so that the speed and accuracy of aircraft skin detection and classification can be improved, and the method has a wide application prospect.
Description
Technical Field
The invention relates to the technical field of aircraft skin detection and measurement, in particular to an aircraft skin defect detection method based on a NAM-DSSD model.
Background
The aircraft skin has the effects of protection, aerodynamic contouring, drag reduction, safety performance improvement, and corrosion and air tightness performance, with reliability and maintainability being two key factors in determining its probability of survival and life cycle costs. In the running process of the aircraft, the pressure change in the lifting process of the aircraft causes the skin to expand and contract periodically, so that the defects of tiny cracks, holes or scratches and the like are formed on the materials around the rivet of the aircraft, and the flight safety and the performance of the aircraft are seriously affected.
Therefore, it is extremely important to detect defects of the aircraft skin by taking necessary industrial measures, operation control means and the like during the production process and the service period of the aircraft skin. The defect detection method of the aircraft skin comprises three stages: firstly, preprocessing an image, secondly, strengthening an image data set, such as a Mosaic data enhancement method, a GAN data enhancement method and the like, and finally training based on a deep learning algorithm, and identifying and classifying the type of the aircraft skin defects, such as a Fast-R CNN algorithm, an SSD algorithm, a DSSD algorithm and the like. However, there is relatively little research on how to balance the detection speed and the detection accuracy in the defect detection algorithm. The Fast-R CNN algorithm has a good target detection effect, but the detection speed is slower due to the large calculation amount of each target classification; SSD algorithms have faster detection speeds, but have poorer detection results for small objects. DSSD, while having good detection rates and speeds, cannot suppress some of the unwanted characteristic information. How to find the most suitable algorithm for detecting the skin defects of the aircraft is still worth intensive research in the academic field.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an aircraft skin defect detection method based on a NAM-DSSD model, which solves the problem of poor detection effect of the traditional method on small targets.
In order to solve the technical problems, the invention provides the following technical scheme: an aircraft skin defect detection method based on NAM-DSSD model comprises the following steps:
S1, shooting and sampling an aircraft skin image through an industrial CMOS camera to obtain a high-resolution image set HR, and performing downsampling treatment on the high-resolution image set HR to obtain a corresponding low-resolution image set LR;
S2, preprocessing a low-resolution image set LR to obtain a medium-resolution image set MR, extracting gradient characteristics of the medium-resolution image set MR and partitioning the gradient characteristics to obtain a first-layer low-resolution image set LR sample Y 1;
S3, respectively extracting the high-frequency characteristic H y1 of the high-resolution image set HR and the edge texture detail characteristic M y1 of the medium-resolution image set MR, and performing blocking operation on M y1 and H y1 to obtain paired characteristic blocks which are converted into a high-resolution image vector X and a low-resolution image vector Y;
S4, reconstructing an aircraft skin high-resolution image set HR acquired by an industrial CMOS camera by using a parallel dictionary super-resolution reconstruction algorithm to finally obtain a high-resolution image set H r for recovering more image details;
S5, synthesizing a plurality of images by using the H r based on a Mosaic data enhancement method to obtain a data set of the aircraft skin after the aircraft skin is enhanced;
s6, inputting the data set with the reinforced aircraft skin into an improved NAM-DSSD model for network training, and identifying and classifying the defect types of the aircraft skin by adding NAM attention mechanisms before each deconvolution of the network and combining a multi-scale feature map and a convolution decoder.
Further, the step S2 specifically includes: the bilinear interpolation method is used for amplifying the low-resolution image set LR, semantic information of the image is effectively reserved, the size of the image is the same as that of the original high-resolution image set HR, the medium-resolution image set MR is obtained, gradient characteristics of the medium-resolution image set MR are extracted, and the medium-resolution image set MR is segmented to obtain a first-layer low-resolution image set LR sample Y 1.
Further, the specific process of the step S3 includes the following steps:
s31, converting the images of the high-resolution image set HR and the medium-resolution image set MR into gray-scale images, and respectively enabling the brightness components of the gray-scale images to be H y and M y;
S32, subtracting M y from H y to serve as high-frequency information H y1 of an HR image of a high-resolution image set, filtering M y through a gradient operator, capturing edge and texture detail information in the image, and marking the extracted feature as M y1;
S33, performing partitioning operation on H y1 and M y1 according to the same partitioning size and the same overlapping number; and converts the resulting paired feature blocks into high resolution image vectors X and low resolution image vectors Y.
Further, the specific process of the step S4 includes the following steps:
S41, solving a low-resolution dictionary D L and a sparse coefficient alpha through a K-singular value decomposition K-SVD algorithm, and introducing KL divergence loss, wherein the formula is expressed as follows:
；
Wherein Y is a low resolution image vector, For the trade-off factor of sparse representation and reconstruction error, D KL represents KL divergence, the distribution of P M coding coefficient M and P desired are expected distribution;
S42, fitting the high-resolution dictionary D H by adopting a Cholesky Qiao Lie Stokes square root method decomposition mode:
；
wherein X is a high-resolution image vector, and U is an upper triangular matrix;
s43, minimizing the error of the fitting result by using a least square method:
；
Wherein, X 1 is the set of high-frequency information reconstructed by the low-resolution image set LR, X 2 is the high-resolution H R sample set for training the second-layer dictionary, and epsilon is the error threshold;
s44, solving a first-layer low-resolution dictionary D L1 and a sparse coefficient alpha by a K-SVD algorithm introducing a KL divergence loss function through M y1 and H y1, adopting a Cholesky Qiao Lie Stokes square root method to decompose and fit to solve a first-layer high-resolution dictionary D H1, and then reducing errors by a least square method to solve a first-layer dictionary D 1;
S45, calculating a sparse coefficient alpha 1 of LR on a first layer dictionary D 1 by adopting an orthogonal matching pursuit algorithm, solving a residual error between a first layer low-resolution image set LR sample Y 1 and the first layer dictionary D 1 by adopting a column updating method, and reconstructing to obtain a first layer high-resolution image X r1 by iteratively repeating the process until the residual error is smaller than epsilon 2; repeating the steps to obtain a second-layer dictionary D 2 and a second-layer high-resolution image X r2;
S46, splicing and integrating the image blocks of X r1 and X r2 again, and performing inverse blocking; in the process of inverse blocking, for the pixels in the overlapping area, adopting the average value of the corresponding overlapping pixels as the value of the overlapping pixels; and adding the X r1 and the X r2 after the inverse blocking to obtain reconstructed high-frequency image information H R1; then, H R1 is combined with the color components of the low-resolution image, resulting in a high-resolution image set H r that recovers more image detail.
Further, the specific process of the step S5 includes the following steps:
S51, firstly randomly selecting four images in a high-resolution image set H r which is generated by reconstruction and recovers more image details, randomly selecting an initial target frame in one image, and recording the coordinates of the boundary frame and the corresponding class label;
S52, randomly selecting three target frames from the other three images, adjusting the coordinates of the original target frames, placing four images in a large image through an image processing library for splicing, splicing the four images into one image, enabling the four images to adapt to the coordinate space of the large image, and recording the target frames and class labels of the large image;
And S53, carrying out random translation and scaling data enhancement operation on the large image, enhancing data diversity, repeating the operation, expanding the data set, and obtaining the data set after aircraft skin enhancement.
Further, the specific process of step S6 includes the following steps:
s61, firstly, replacing the residual error-101 with a base network VGG16 of the SSD, wherein the base network VGG has a deeper convolution layer, and the accuracy of an algorithm is improved;
s62, secondly, carrying out batch normalization processing on each convolution layer on the original SSD, replacing bilinear interpolation up-sampling by a deconvolution layer, fusing high-level semantic information with bottom-level semantic information, adding NAM attention mechanism before each deconvolution, namely, adjusting the attention weight by adjusting the variances of channel dimension and space dimension, and extracting a fused feature map; wherein:
For the channel attention module, the BN representation is normalized in batches:
；
Wherein, For input feature map,/>The characteristic diagram is output; /(I)And/>Affine transformation parameters trainable for scale and displacement respectively; /(I)And/>Respectively, small batches/>Average and standard deviation of (a); /(I)A number infinitely close to and other than 0;
in a channel attention module For the scale factor of each channel, the weight is/>Input feature map is/>I.e./>Output characteristics are/>I.e./>; The weight of the spatial attention module is/>Input feature map is/>Output characteristics are/>Said/>And/>The formula of (2) is as follows:
；
；
；
；
Wherein, Input feature graphs for channel attention modules, i.e./>，/>Input feature graphs of the spatial attention module, H, W, C are the height, width and number of channel dimensions respectively,/>Scaling factors for spatial dimensions;
To suppress less important weights in the model, NAM introduced regularization terms to adjust the loss function, whose formula is shown below:
；
Wherein, As a loss function,/>For/>Norm penalty factor,/>To balance/>And/>Penalty factor of/>、/>Respectively input and output, Q is a network weight;
And S63, finally, introducing a prediction module into the feature layer of the original SSD model by the NAM-DSSD as an output layer, and optimizing the feature map input of the bounding box regression and classification tasks by adding a residual block structure.
By means of the technical scheme, the invention provides an aircraft skin defect detection method based on a NAM-DSSD model, which has at least the following beneficial effects:
According to the invention, firstly, the image is preprocessed through a parallel dictionary super-resolution reconstruction algorithm, the resolution of the image is improved, NAM-DSSD algorithm training is facilitated, the training speed can be accelerated, secondly, the accuracy and robustness of the NAM-DSSD algorithm are improved through the Mosaic method, and finally, NAM attention mechanism is added into the DSSD algorithm to strengthen key characteristics, and the data set after the aircraft skin reinforcement is input into the NAM-DSSD algorithm with stronger detection capability for small targets for network training, so that the speed and accuracy of aircraft skin detection and classification can be improved. The method has strong detection and classification capability on small targets such as aircraft skin defect images, gives consideration to detection speed and precision, and has wide application prospect.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 shows an aircraft skin defect detection method based on NAM-DSSD model;
FIG. 2 is a schematic diagram of the NAM-DSSD model framework structure of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. Therefore, the realization process of how to apply the technical means to solve the technical problems and achieve the technical effects can be fully understood and implemented.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in a method of implementing an embodiment described above may be implemented by a program to instruct related hardware, and thus, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Referring to fig. 1-2, a specific implementation manner of the present embodiment is shown, the present embodiment is an aircraft skin defect detection method based on a NAM-DSSD (Normalization-based Attention Module-Deconvolutional Single Shot Detector) model, firstly, an industrial camera is used to shoot a defect image represented by an aircraft skin, and an image reconstruction effect is improved through a super resolution reconstruction algorithm of a parallel dictionary to perform preprocessing, so as to improve the resolution of the image, secondly, a mosoic is used to strengthen a data set, finally, a deep learning algorithm of the NAM-DSSD is used to perform learning training on the reinforced aircraft skin data set, a deconvolution layer is added, high-level semantic information is fused into bottom semantic information, and a NAM attention mechanism is added into the DSSD to strengthen key features, improve the detection precision of small targets, so as to achieve the purpose of automatically identifying the aircraft skin defect types; the high-resolution image can be obtained through the parallel dictionary super-resolution reconstruction algorithm, the training speed of the NAM-DSSD algorithm can be increased, the NAM-DSSD algorithm has strong detection and classification capability on small targets such as aircraft skin defect images, and the detection speed and precision are both considered.
Referring to fig. 1, the present embodiment provides an aircraft skin defect detection method based on a NAM-DSSD model, which includes the following steps:
S1, shooting and sampling an aircraft skin image through an industrial CMOS camera to obtain a high-resolution image set HR, and performing downsampling treatment on the high-resolution image set HR to obtain a corresponding low-resolution image set LR;
S2, preprocessing a low-resolution image set LR to obtain a medium-resolution image set MR, extracting gradient characteristics of the medium-resolution image set MR and partitioning the gradient characteristics to obtain a first-layer low-resolution image set LR sample Y 1;
As a preferred embodiment of step S2, step S2 specifically includes: in order to simplify the calculation process, a bilinear interpolation method is used for amplifying the low-resolution image set LR, semantic information of the image is effectively reserved, the size of the image is the same as that of an original high-resolution image set HR, a medium-resolution image set MR is obtained, gradient characteristics of the medium-resolution image set MR are extracted, and a first-layer low-resolution image set LR sample Y 1 is obtained through blocking.
S3, respectively extracting the high-frequency characteristic H y1 of the high-resolution image set HR and the edge texture detail characteristic M y1 of the medium-resolution image set MR, and performing blocking operation on M y1 and H y1 to obtain paired characteristic blocks which are converted into a high-resolution image vector X and a low-resolution image vector Y;
As a preferred embodiment of step S3, the specific process of step S3 includes the following steps:
s31, converting the images of the high-resolution image set HR and the medium-resolution image set MR into gray-scale images, and respectively enabling the brightness components of the gray-scale images to be H y and M y;
S32, subtracting M y from H y to serve as high-frequency information H y1 of an HR image of a high-resolution image set, filtering M y through a gradient operator, capturing edge and texture detail information in the image, and marking the extracted feature as M y1;
S33, performing partitioning operation on H y1 and M y1 according to the same partitioning size and the same overlapping number; and converts the resulting paired feature blocks into high resolution image vectors X and low resolution image vectors Y.
S4, reconstructing an aircraft skin high-resolution image set HR acquired by an industrial CMOS camera by using a parallel dictionary super-resolution reconstruction algorithm to finally obtain a high-resolution image set H r for recovering more image details;
As a preferred embodiment of step S4, the specific process of step S4 includes the following steps:
S41, solving a low-resolution dictionary D L and a sparse coefficient alpha through a K-singular value decomposition K-SVD algorithm, and introducing KL divergence loss, wherein the formula is expressed as follows:
；
Wherein Y is a low resolution image vector, For the trade-off factor of sparse representation and reconstruction error, D KL represents KL divergence, the distribution of P M coding coefficient M and P desired are expected distribution;
S42, fitting the high-resolution dictionary D H by adopting a Cholesky Qiao Lie Stokes square root method decomposition mode:
；
wherein X is a high-resolution image vector, and U is an upper triangular matrix;
s43, minimizing the error of the fitting result by using a least square method:
；
Wherein, X 1 is the set of high-frequency information reconstructed by the low-resolution image set LR, X 2 is the high-resolution H R sample set for training the second-layer dictionary, and epsilon is the error threshold;
s44, solving a first-layer low-resolution dictionary D L1 and a sparse coefficient alpha by a K-SVD algorithm introducing a KL divergence loss function through M y1 and H y1, adopting a Cholesky Qiao Lie Stokes square root method to decompose and fit to solve a first-layer high-resolution dictionary D H1, and then reducing errors by a least square method to solve a first-layer dictionary D 1;
S45, calculating a sparse coefficient alpha 1 of LR on a first layer dictionary D 1 by adopting an orthogonal matching pursuit algorithm, solving a residual error between a first layer low-resolution image set LR sample Y 1 and the first layer dictionary D 1 by adopting a column updating method, and reconstructing to obtain a first layer high-resolution image X r1 by iteratively repeating the process until the residual error is smaller than epsilon 2; repeating the steps to obtain a second-layer dictionary D 2 and a second-layer high-resolution image X r2;
S46, splicing and integrating the image blocks of X r1 and X r2 again, and performing inverse blocking; in the process of inverse blocking, for the pixels in the overlapping area, adopting the average value of the corresponding overlapping pixels as the value of the overlapping pixels; and adding the X r1 and the X r2 after the inverse blocking to obtain reconstructed high-frequency image information H R1; then, H R1 is combined with the color components of the low-resolution image, resulting in a high-resolution image set H r that recovers more image detail.
In the embodiment, the image degradation and the up-sampling are carried out on the high-resolution image through the parallel dictionary super-resolution reconstruction algorithm, so that a high-resolution image set for recovering more image details is obtained; the method breaks through the limitation of hardware, improves the resolution of the aircraft skin image by using a software method, improves the definition of the image by enhancing more image information with high resolution, and can obtain the high-resolution image, thereby improving the image quality and the feature extraction capability, providing richer detail information, providing better input data for the subsequent data enhancement and network training steps, making a high-quality basis for accelerating the training speed of the subsequent NAM-DSSD algorithm, and better learning and identifying the skin, thereby being beneficial to more accurately detecting the skin problem.
Image degradation refers to the degradation of a high resolution image to a low resolution image, and the mathematical model is:
；
Wherein S is a sampling operator, F is a constructed fuzzy matrix operator, and n is additive noise.
S5, synthesizing a plurality of images by using the H r based on a Mosaic data enhancement method to obtain a data set of the aircraft skin after the aircraft skin is enhanced;
as a preferred embodiment of step S5, the specific process of step S5 includes the following steps:
S51, firstly randomly selecting four images in a high-resolution image set H r which is generated by reconstruction and recovers more image details, randomly selecting an initial target frame in one image, and recording the coordinates of the boundary frame and the corresponding class label;
S52, randomly selecting three target frames from the other three images, adjusting the coordinates of the original target frames, placing four images in a large image through an image processing library for splicing, splicing the four images into one image, enabling the four images to adapt to the coordinate space of the large image, and recording the target frames and class labels of the large image;
And S53, carrying out random translation and scaling data enhancement operation on the large image, enhancing data diversity, repeating the operation, expanding the data set, and obtaining the data set after aircraft skin enhancement.
In the embodiment, a stitching datum point is randomly selected, four preprocessed aircraft skin images are selected, the sizes and the scaling ratios of the images are adjusted according to the datum point, the four images are stitched into a picture, batch size is repeated for times to generate batch size images, and finally the reinforced aircraft skin image dataset is trained through a subsequent NAM-DSSD model; technology for simulating complexity and diversity in the real world by synthesizing a plurality of images based on a Mosaic data enhancement method enables a model to learn characteristics with robustness and generalization capability, and strengthens a data set.
S6, inputting the data set with the reinforced aircraft skin into an improved NAM-DSSD model for network training, and identifying and classifying the defect types of the aircraft skin by adding NAM attention mechanisms before each deconvolution of the network and combining a multi-scale feature map and a convolution decoder.
As a preferred embodiment of step S6, the specific process of step S6 includes the following steps:
s61, firstly, replacing a residual error-101 with a base network VGG16 of the SSD, and forming a characteristic layer of NAM-DSSD by Conv3_x and five subsequent convolution layers; the method has a deeper convolution layer, and improves the accuracy of the algorithm;
S62, secondly, since the defects of the aircraft skin are very tiny, accurate identification and detection are difficult to perform, a deconvolution layer is added behind a feature layer to jointly form a deconvolution module, the advanced feature representation capability of a small target model is enhanced, batch normalization processing is performed on each convolution layer on the original SSD, the deconvolution layer is used for replacing bilinear interpolation up-sampling, high-level semantic information and bottom-layer semantic information are fused, the resolution of an image is improved, the algorithm speed is influenced by considering some unimportant weights, NAM attention mechanisms are added before each deconvolution, the test precision and speed are improved, NAM is an attention mechanism for adjusting the weight of attention by adjusting the variances of channel dimensions and space dimensions on the basis of CBAM, and therefore a fused feature map is extracted; wherein:
For the channel attention module, the BN representation is normalized in batches:
；
Wherein, For input feature map,/>The characteristic diagram is output; /(I)And/>Affine transformation parameters trainable for scale and displacement respectively; /(I)And/>Respectively, small batches/>Average and standard deviation of (a); /(I)A number infinitely close to and other than 0;
in a channel attention module For the scale factor of each channel, the weight is/>Input feature map is/>I.e./>Output characteristics are/>I.e./>; The weight of the spatial attention module is/>Input feature map is/>Output characteristics are/>Said/>And/>The formula of (2) is as follows:
；
；
；
；
Wherein, Input feature graphs for channel attention modules, i.e./>，/>Input feature graphs of the spatial attention module, H, W, C are the height, width and number of channel dimensions respectively,/>Scaling factors for spatial dimensions;
To suppress less important weights in the model, NAM introduced regularization terms to adjust the loss function, whose formula is shown below:
；
Wherein, As a loss function,/>For/>Norm penalty factor,/>To balance/>And/>Penalty factor of/>、/>Respectively input and output, Q is a network weight;
S63, finally, introducing a prediction module into a feature layer of an original SSD model by NAM-DSSD as an output layer, optimizing feature map input of a bounding box regression and classification task by adding a residual block structure, and improving the detection precision of the model, so that not only can feature reuse be enhanced, but also the detection task performance of the model on an aircraft skin dataset is improved, as shown in figure 2;
The NAM-DSSD specifically comprises the following steps:
feature extraction: and pre-training the data set after the aircraft skin is reinforced by taking the data set as a basic network, and extracting a feature map.
Feature fusion: after Resnet-101 the last layer, a deconvolution layer is introduced to gradually restore the feature map size to the same size as the input image, to preserve more image detail and provide more accurate location information, enhancing the detection capability of small-size targets.
Adding NAM attention mechanism: and adding NAM mechanism before all deconvolutions, screening out some non-critical features, strengthening critical features and improving the accuracy and speed of the algorithm.
Feature map processing: on the high-resolution feature map after feature fusion, the NAM-DSSD further extracts feature information of the small target object through convolution operation.
Boundary box generation: the NAM-DSSD generates a set of prediction boxes on the feature map, including an anchor box and additional prediction boxes, by a prediction module to cover targets of different scales and aspect ratios, each prediction box corresponding to a location on the feature map.
Category prediction and bounding box regression: for each prediction box, performing category prediction and bounding box regression through convolution and full connection layers, wherein the category prediction outputs a confidence score of each category by using a softmax activation function, and the bounding box regression predicts the offset of the position and the shape of the small target object.
Outputting a result: and filtering the highly overlapped prediction frames by using non-maximum suppression NMS, wherein the rest prediction frames are detection results, namely the position of the prediction frames and the category confidence level.
In the embodiment, the deep learning algorithm of NAM-DSSD carries out learning training on the reinforced aircraft skin data set, a deconvolution layer is added, high-level semantic information is fused into bottom-level semantic information, multi-scale feature map extraction and feature fusion are carried out, NAM attention mechanism is added into DSSD, key features are strengthened, and finally the defect types of the aircraft skin are identified and classified. The invention has strong detection and classification capability on small targets such as aircraft skin defect images, improves the detection precision of the small targets, achieves the purpose of automatically identifying the types of aircraft skin defects, and takes the detection speed and precision into account.
Based on the above example, the invention firstly carries out preprocessing on the image through the parallel dictionary super-resolution reconstruction algorithm, improves the resolution of the image, is convenient for NAM-DSSD algorithm training, secondly enhances the data set of the aircraft skin through the Mosaic method, improves the accuracy and the robustness of the NAM-DSSD algorithm, finally adds a NAM attention mechanism into the DSSD algorithm, strengthens key characteristics, and can improve the speed and the accuracy of aircraft skin detection and classification by carrying out network training on the NAM-DSSD algorithm with stronger detection capability on small targets.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.
The foregoing embodiments have been presented in a detail description of the invention, and are presented herein with a particular application to the understanding of the principles and embodiments of the invention, the foregoing embodiments being merely intended to facilitate an understanding of the method of the invention and its core concepts; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (5)
1. An aircraft skin defect detection method based on a NAM-DSSD model is characterized by comprising the following steps of:
S1, shooting and sampling an aircraft skin image through an industrial CMOS camera to obtain a high-resolution image set HR, and performing downsampling treatment on the high-resolution image set HR to obtain a corresponding low-resolution image set LR;
S2, preprocessing a low-resolution image set LR to obtain a medium-resolution image set MR, extracting gradient characteristics of the medium-resolution image set MR and partitioning the gradient characteristics to obtain a first-layer low-resolution image set LR sample Y 1;
S3, respectively extracting the high-frequency characteristic H y1 of the high-resolution image set HR and the edge texture detail characteristic M y1 of the medium-resolution image set MR, and performing blocking operation on M y1 and H y1 to obtain paired characteristic blocks which are converted into a high-resolution image vector X and a low-resolution image vector Y;
S4, reconstructing an aircraft skin high-resolution image set HR acquired by an industrial CMOS camera by using a parallel dictionary super-resolution reconstruction algorithm to finally obtain a high-resolution image set H r for recovering more image details;
The specific process comprises the following steps:
S41, solving a low-resolution dictionary D L and a sparse coefficient alpha through a K-singular value decomposition K-SVD algorithm, and introducing KL divergence loss, wherein the formula is expressed as follows:
；
Wherein Y is a low resolution image vector, For the trade-off factor of sparse representation and reconstruction error, D KL represents KL divergence, the distribution of P M coding coefficient M and P desired are expected distribution;
S42, fitting the high-resolution dictionary D H by adopting a Cholesky Qiao Lie Stokes square root method decomposition mode:
；
wherein X is a high-resolution image vector, and U is an upper triangular matrix;
s43, minimizing the error of the fitting result by using a least square method:
；
Wherein, X 1 is the set of high-frequency information reconstructed by the low-resolution image set LR, X 2 is the high-resolution H R sample set for training the second-layer dictionary, and epsilon is the error threshold;
s44, solving a first-layer low-resolution dictionary D L1 and a sparse coefficient alpha by a K-SVD algorithm introducing a KL divergence loss function through M y1 and H y1, adopting a Cholesky Qiao Lie Stokes square root method to decompose and fit to solve a first-layer high-resolution dictionary D H1, and then reducing errors by a least square method to solve a first-layer dictionary D 1;
S45, calculating a sparse coefficient alpha 1 of LR on a first layer dictionary D 1 by adopting an orthogonal matching pursuit algorithm, solving a residual error between a first layer low-resolution image set LR sample Y 1 and the first layer dictionary D 1 by adopting a column updating method, and reconstructing to obtain a first layer high-resolution image X r1 by iteratively repeating the process until the residual error is smaller than epsilon 2; repeating the steps to obtain a second-layer dictionary D 2 and a second-layer high-resolution image X r2;
S46, splicing and integrating the image blocks of X r1 and X r2 again, and performing inverse blocking; in the process of inverse blocking, for the pixels in the overlapping area, adopting the average value of the corresponding overlapping pixels as the value of the overlapping pixels; and adding the X r1 and the X r2 after the inverse blocking to obtain reconstructed high-frequency image information H R1; then, combining H R1 with color components of the low-resolution image to finally obtain a high-resolution image set H r for recovering more image details;
S5, synthesizing a plurality of images by using the H r based on a Mosaic data enhancement method to obtain a data set of the aircraft skin after the aircraft skin is enhanced;
s6, inputting the data set with the reinforced aircraft skin into an improved NAM-DSSD model for network training, and identifying and classifying the defect types of the aircraft skin by adding NAM attention mechanisms before each deconvolution of the network and combining a multi-scale feature map and a convolution decoder.
2. The method for detecting the defects of the aircraft skin based on the NAM-DSSD model according to claim 1, wherein the method comprises the following steps: the step S2 specifically includes: the bilinear interpolation method is used for amplifying the low-resolution image set LR, semantic information of the image is effectively reserved, the size of the image is the same as that of the original high-resolution image set HR, the medium-resolution image set MR is obtained, gradient characteristics of the medium-resolution image set MR are extracted, and the medium-resolution image set MR is segmented to obtain a first-layer low-resolution image set LR sample Y 1.
3. The method for detecting the defects of the aircraft skin based on the NAM-DSSD model according to claim 1, wherein the method comprises the following steps: the specific process of the step S3 comprises the following steps:
s31, converting the images of the high-resolution image set HR and the medium-resolution image set MR into gray-scale images, and respectively enabling the brightness components of the gray-scale images to be H y and M y;
S32, subtracting M y from H y to serve as high-frequency information H y1 of an HR image of a high-resolution image set, filtering M y through a gradient operator, capturing edge and texture detail information in the image, and marking the extracted feature as M y1;
S33, performing partitioning operation on H y1 and M y1 according to the same partitioning size and the same overlapping number; and converts the resulting paired feature blocks into high resolution image vectors X and low resolution image vectors Y.
4. The method for detecting the defects of the aircraft skin based on the NAM-DSSD model according to claim 1, wherein the method comprises the following steps: the specific process of the step S5 comprises the following steps:
S51, firstly randomly selecting four images in a high-resolution image set H r which is generated by reconstruction and recovers more image details, randomly selecting an initial target frame in one image, and recording the coordinates of the boundary frame and the corresponding class label;
S52, randomly selecting three target frames from the other three images, adjusting the coordinates of the original target frames, placing four images in a large image through an image processing library for splicing, splicing the four images into one image, enabling the four images to adapt to the coordinate space of the large image, and recording the target frames and class labels of the large image;
And S53, carrying out random translation and scaling data enhancement operation on the large image, enhancing data diversity, repeating the operation, expanding the data set, and obtaining the data set after aircraft skin enhancement.
5. The method for detecting the defects of the aircraft skin based on the NAM-DSSD model according to claim 1, wherein the method comprises the following steps: the specific process of the step S6 comprises the following steps:
s61, firstly, replacing the residual error-101 with a base network VGG16 of the SSD, wherein the base network VGG has a deeper convolution layer, and the accuracy of an algorithm is improved;
s62, secondly, carrying out batch normalization processing on each convolution layer on the original SSD, replacing bilinear interpolation up-sampling by a deconvolution layer, fusing high-level semantic information with bottom-level semantic information, adding NAM attention mechanism before each deconvolution, namely, adjusting the attention weight by adjusting the variances of channel dimension and space dimension, and extracting a fused feature map; wherein:
For the channel attention module, the BN representation is normalized in batches:
；
Wherein, For input feature map,/>The characteristic diagram is output; /(I)And/>Affine transformation parameters trainable for scale and displacement respectively; /(I)And/>Respectively, small batches/>Average and standard deviation of (a); /(I)A number infinitely close to and other than 0;
in a channel attention module For the scale factor of each channel, the weight is/>Input feature map is/>I.e./>Output characteristics are/>I.e./>; The weight of the spatial attention module is/>Input feature map is/>Output characteristics are/>The saidAnd/>The formula of (2) is as follows:
；
；
；
；
Wherein, Input feature graphs for channel attention modules, i.e./>，/>Input feature graphs of the spatial attention module, H, W, C are the height, width and number of channel dimensions respectively,/>Scaling factors for spatial dimensions;
To suppress less important weights in the model, NAM introduced regularization terms to adjust the loss function, whose formula is shown below:
；
Wherein, As a loss function,/>For/>Norm penalty factor,/>To balance/>And/>Is used to determine the penalty factor of (1),、/>Respectively input and output, Q is a network weight;
And S63, finally, introducing a prediction module into the feature layer of the original SSD model by the NAM-DSSD as an output layer, and optimizing the feature map input of the bounding box regression and classification tasks by adding a residual block structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410425461.7A CN118052811B (en) | 2024-04-10 | 2024-04-10 | NAM-DSSD model-based aircraft skin defect detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410425461.7A CN118052811B (en) | 2024-04-10 | 2024-04-10 | NAM-DSSD model-based aircraft skin defect detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118052811A CN118052811A (en) | 2024-05-17 |
| CN118052811B true CN118052811B (en) | 2024-06-11 |
Family
ID=91052109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410425461.7A Active CN118052811B (en) | 2024-04-10 | 2024-04-10 | NAM-DSSD model-based aircraft skin defect detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118052811B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741256A (en) * | 2018-12-13 | 2019-05-10 | 西安电子科技大学 | Image super-resolution rebuilding method based on rarefaction representation and deep learning |
| CN110992270A (en) * | 2019-12-19 | 2020-04-10 | 西南石油大学 | Multi-scale residual attention network image super-resolution reconstruction method based on attention |
| CN117788471A (en) * | 2024-02-27 | 2024-03-29 | 南京航空航天大学 | Method for detecting and classifying aircraft skin defects based on YOLOv5 |
-
2024
- 2024-04-10 CN CN202410425461.7A patent/CN118052811B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109741256A (en) * | 2018-12-13 | 2019-05-10 | 西安电子科技大学 | Image super-resolution rebuilding method based on rarefaction representation and deep learning |
| CN110992270A (en) * | 2019-12-19 | 2020-04-10 | 西南石油大学 | Multi-scale residual attention network image super-resolution reconstruction method based on attention |
| CN117788471A (en) * | 2024-02-27 | 2024-03-29 | 南京航空航天大学 | Method for detecting and classifying aircraft skin defects based on YOLOv5 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118052811A (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112966684B (en) | Cooperative learning character recognition method under attention mechanism | |
| CN111914698B (en) | Human body segmentation method, segmentation system, electronic equipment and storage medium in image | |
| CN113159120A (en) | Contraband detection method based on multi-scale cross-image weak supervision learning | |
| CN109671071A (en) | A kind of underground piping defect location and grade determination method based on deep learning | |
| CN113920107A (en) | Insulator damage detection method based on improved yolov5 algorithm | |
| CN117253154B (en) | Container weak and small serial number target detection and identification method based on deep learning | |
| CN111680705B (en) | MB-SSD method and MB-SSD feature extraction network suitable for target detection | |
| CN113449784B (en) | Image multi-classification method, device, equipment and medium based on priori attribute map | |
| CN111401293A (en) | Gesture recognition method based on Head lightweight Mask scanning R-CNN | |
| CN112800955A (en) | Remote sensing image rotating target detection method and system based on weighted bidirectional feature pyramid | |
| CN116645592A (en) | Crack detection method based on image processing and storage medium | |
| CN114565594A (en) | Image anomaly detection method based on soft mask contrast loss | |
| CN116883933A (en) | Security inspection contraband detection method based on multi-scale attention and data enhancement | |
| CN116342536A (en) | Aluminum strip surface defect detection method, system and equipment based on lightweight model | |
| WO2022219402A1 (en) | Semantically accurate super-resolution generative adversarial networks | |
| CN114550134A (en) | Deep learning-based traffic sign detection and identification method | |
| CN111401209B (en) | Action recognition method based on deep learning | |
| CN113313678A (en) | Automatic sperm morphology analysis method based on multi-scale feature fusion | |
| CN113326734A (en) | Rotary target detection method based on YOLOv5 | |
| CN118052811B (en) | NAM-DSSD model-based aircraft skin defect detection method | |
| Chen et al. | A novel multi-sample data augmentation method for oriented object detection in remote sensing images | |
| Wang et al. | Detection of plane in remote sensing images using super-resolution | |
| Yang et al. | Multi-category decomposition editing network for the accurate visual inspection of texture defects | |
| CN115205616A (en) | Cloth flaw data enhancement method based on generation countermeasure network | |
| CN114782322A (en) | YOLOv5 model arc additive manufacturing molten pool defect detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |