CN117058396B - Fan blade defect area rapid segmentation method and system based on artificial intelligence - Google Patents
Fan blade defect area rapid segmentation method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN117058396B CN117058396B CN202311313238.5A CN202311313238A CN117058396B CN 117058396 B CN117058396 B CN 117058396B CN 202311313238 A CN202311313238 A CN 202311313238A CN 117058396 B CN117058396 B CN 117058396B
- Authority
- CN
- China
- Prior art keywords
- fan blade
- fan
- feature
- scale
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 42
- 230000011218 segmentation Effects 0.000 title claims abstract description 32
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 239000011159 matrix material Substances 0.000 claims description 25
- 238000009826 distribution Methods 0.000 claims description 18
- 238000001914 filtration Methods 0.000 claims description 17
- 230000004927 fusion Effects 0.000 claims description 15
- 238000010586 diagram Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 11
- 230000002457 bidirectional effect Effects 0.000 claims description 10
- 230000002950 deficient Effects 0.000 claims description 9
- 230000004807 localization Effects 0.000 claims description 8
- 238000012546 transfer Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 24
- 238000012545 processing Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 description 11
- 230000003044 adaptive effect Effects 0.000 description 7
- 230000006378 damage Effects 0.000 description 7
- 239000000463 material Substances 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000007797 corrosion Effects 0.000 description 3
- 238000005260 corrosion Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000007774 longterm Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 229920000049 Carbon (fiber) Polymers 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 239000004917 carbon fiber Substances 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000003365 glass fiber Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 239000003562 lightweight material Substances 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000003595 mist Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000013535 sea water Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a fan blade defect area rapid segmentation method and system based on artificial intelligence. Firstly, acquiring a fan blade image of a detected fan blade acquired by a camera, then, carrying out multi-scale image feature extraction on the fan blade image to obtain a multi-scale fan feature map, and then, generating a fan blade image containing a fan blade defect area positioning anchor window based on the multi-scale fan feature map. In this way, the image processing can be performed on the fan blade image of the detected fan blade before the image recognition and defect detection are performed, and the defect area in the fan blade image can be automatically positioned.
Description
Technical Field
The application relates to the field of intelligent detection, and more particularly, to a fan blade defect area rapid segmentation method and system based on artificial intelligence.
Background
During long-term operation of the wind driven generator, the surface of the blade may exhibit various damages, such as damage to the blade protective film, paint falling from the blade, icing on the blade, cracks on the blade, and greasy dirt on the blade, and therefore, it is necessary to periodically detect defects of the blade of the wind driven generator.
Currently, most fan blade defect detection is generally divided into two cases, and one traditional method is to use manpower to make inspection, namely, manually checking the visible light pictures of the fan blades, which are shot by an unmanned aerial vehicle or are collected by other means; another more advanced approach is to use computer vision techniques to select the defective picture, then locate the exact location on the fan blade where the defect is located based on the captured information, and finally dispatch personnel for repair or replacement.
The manual detection method is a rough blade defect detection method lacking informatization, and has the problems of extremely relying on manual work, low effective utilization rate, poor instantaneity and the like. The method using computer vision technology generally directly sends the collected picture to the detection network, and then gives the position and the type of the defect in the picture, and the method may detect the target of the non-blade area as the defect, which increases the workload of manually eliminating the false detection.
Therefore, an optimized fan blade defect region fast segmentation scheme is desired.
Disclosure of Invention
The present application has been made in order to solve the above technical problems. The embodiment of the application provides a fan blade defect area rapid segmentation method and system based on artificial intelligence. Before image recognition and defect detection, the method can perform image processing on the fan blade image of the detected fan blade, and automatically locate the defect area in the fan blade image.
According to one aspect of the present application, there is provided a fan blade defect area rapid segmentation method based on artificial intelligence, including:
acquiring a fan blade image of a detected fan blade acquired by a camera;
carrying out multi-scale image feature extraction on the fan blade image to obtain a multi-scale fan feature map;
and generating a fan blade image comprising a fan blade defect area positioning anchor window based on the multi-scale fan characteristic diagram.
According to another aspect of the present application, there is provided an artificial intelligence based fan blade defect area fast segmentation system, comprising:
the image acquisition module is used for acquiring fan blade images of detected fan blades acquired by the camera;
the multi-scale image feature extraction module is used for carrying out multi-scale image feature extraction on the fan blade image so as to obtain a multi-scale fan feature map;
and the positioning anchor window image generation module is used for generating a fan blade image containing the positioning anchor window of the fan blade defect area based on the multi-scale fan characteristic diagram.
Compared with the prior art, the method and the system for rapidly dividing the defect area of the fan blade based on the artificial intelligence are characterized in that firstly, the fan blade image of the detected fan blade collected by the camera is obtained, then, the multi-scale image feature extraction is carried out on the fan blade image to obtain a multi-scale fan feature map, and then, the fan blade image containing the fan blade defect area positioning anchor window is generated based on the multi-scale fan feature map. In this way, the image processing can be performed on the fan blade image of the detected fan blade before the image recognition and defect detection are performed, and the defect area in the fan blade image can be automatically positioned.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below, which are not intended to be drawn to scale in terms of actual dimensions, with emphasis on illustrating the gist of the present application.
FIG. 1 is a flow chart of a method for rapidly segmenting a defective area of a fan blade based on artificial intelligence according to an embodiment of the present application.
FIG. 2 is a schematic architecture diagram of an artificial intelligence based method for rapidly segmenting defective areas of fan blades according to an embodiment of the present application.
FIG. 3 is a flowchart of sub-step S120 of an artificial intelligence based fan blade defect region rapid segmentation method according to an embodiment of the present application.
FIG. 4 is a flowchart of sub-step S121 of an artificial intelligence based fan blade defect area fast segmentation method according to an embodiment of the present application.
FIG. 5 is a flowchart of sub-step S130 of an artificial intelligence based fan blade defect region rapid segmentation method according to an embodiment of the present application.
FIG. 6 is a flowchart of sub-step S131 of an artificial intelligence based fan blade defect region rapid segmentation method according to an embodiment of the present application.
FIG. 7 is a block diagram of an artificial intelligence based fan blade defect area fast segmentation system according to an embodiment of the present application.
FIG. 8 is an application scenario diagram of an artificial intelligence based fan blade defect area fast segmentation method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present application without making any inventive effort, are also within the scope of the present application.
As used in this application and in the claims, the terms "a," "an," "the," and/or "the" are not specific to the singular, but may include the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server. The modules are merely illustrative, and different aspects of the systems and methods may use different modules.
Flowcharts are used in this application to describe the operations performed by systems according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Wind blades of wind generators are key components for capturing wind energy and converting it into mechanical energy, and are typically made of lightweight materials such as glass fibers, carbon fibers or composite materials, which are designed to maximize the capture of wind energy and the conversion thereof into rotational force. The fan blades are generally flat, elongated in shape, similar to the wings of an aircraft, and are designed using aerodynamic principles to provide optimal wind energy capture efficiency. The shape and size of the blades may vary depending on the size and design requirements of the wind turbine. The blade surfaces are typically smooth to reduce wind drag and wind losses, the blade leading edges are typically thicker and the blade trailing edges are thinner to provide optimum aerodynamic performance, and the blade shape may also be curved to accommodate different wind directions and speeds. Modern wind generators typically employ a three-bladed design in order to balance wind energy capture efficiency and structural stability. The blades are rotated by a rotor connected to the main shaft, which in turn drives a generator to produce electricity.
And the fan blade of the wind driven generator can generate defects due to material fatigue, external damage, manufacturing defects, environmental corrosion, operation overload and the like. Specifically, material fatigue: the fan blade is subjected to the continuous action of wind power in long-term operation, so that the fatigue of materials can be caused, the long-term wind power action can cause the damage and deformation of blade materials, and the crack, fracture or deformation of the blade can be caused; external injury: the fan blade may be damaged by external factors, such as hail, bird strike, storm, etc., which may cause scoring, cracking or deformation of the blade surface, thereby affecting the performance and stability of the blade; manufacturing defects: in the manufacturing process of the fan blade, manufacturing defects such as uneven materials, weak joints, improper process and the like may exist, and the manufacturing defects may cause problems such as cracks, breakage or deformation of the blade in the use process; environmental corrosion: wind power generators are usually installed in areas of ocean or windstorm, and blades are exposed to severe environments and can be corroded by seawater, salt mist and the like, and the environmental factors can cause corrosion and damage of blade materials, so that the performance and service life of the blades are affected; operation overload: if the wind turbine is operated for a long period of time under conditions exceeding its design load, the blades may be subjected to excessive pressures and stresses, resulting in damage and deformation of the blades. Therefore, it is necessary to detect defects of the fan blades of the wind turbine at regular time intervals.
In view of the foregoing technical problems, the technical idea of the present application is to perform image processing on a fan blade image of a detected fan blade in a source domain before performing image recognition and defect detection, and automatically locate a defect area in the fan blade image.
FIG. 1 is a flow chart of a method for rapidly segmenting a defective area of a fan blade based on artificial intelligence according to an embodiment of the present application. FIG. 2 is a schematic architecture diagram of an artificial intelligence based method for rapidly segmenting defective areas of fan blades according to an embodiment of the present application. As shown in fig. 1 and 2, a method for quickly dividing a defective area of a fan blade based on artificial intelligence according to an embodiment of the present application includes the steps of: s110, acquiring a fan blade image of a detected fan blade acquired by a camera; s120, carrying out multi-scale image feature extraction on the fan blade image to obtain a multi-scale fan feature map; and S130, generating a fan blade image containing a fan blade defect area positioning anchor window based on the multi-scale fan characteristic diagram.
Specifically, in the technical scheme of the application, firstly, a fan blade image of a detected fan blade acquired by a camera is acquired, and the fan blade image is subjected to bidirectional filtering to obtain an enhanced fan blade image. That is, by means of bidirectional filtering, the advantages of spatial filtering and frequency domain filtering are combined, noise in the fan blade image is removed, and meanwhile detailed information of the image is kept. It should be appreciated that the camera used to acquire the image of the detected fan blade may be selected from a variety of different types of cameras, with the selection of the appropriate camera depending on the particular application scenario and requirements. Factors to be considered include image quality, resolution, frame rate, cost, and environmental suitability. For example, a high speed camera may be used, which has a very high frame rate and can capture fast moving objects. For rotational movement of the fan blades, the high speed camera may provide a clearer image for defect detection and segmentation.
Then, the enhanced fan blade image passes through an encoder model comprising a shallow layer feature extractor, a middle layer feature extractor and a deep layer feature extractor to obtain a fan blade shallow layer feature map, a fan blade middle layer feature map and a fan blade deep layer feature map; and then, fusing the shallow layer feature map of the fan blade, the middle layer feature map of the fan blade and the deep layer feature map of the fan blade through a self-adaptive fusion module to obtain a multi-scale fan feature map.
The shallow layer feature map, the middle layer feature map and the deep layer feature map are fused, so that multi-level information can be reserved at the same time. In the technical scheme of the application, in order to excessively increase the parameter number of the model and keep the channel number unchanged, an adaptive fusion module (adaptive fusion) is utilized to carry out fusion operation, so that the original channel number can be kept unchanged under the condition of not increasing excessive parameters, multi-level characteristic fusion can be carried out, and multi-level information is fully utilized.
Accordingly, as shown in fig. 3, the multi-scale image feature extraction is performed on the fan blade image to obtain a multi-scale fan feature map, which includes: s121, extracting shallow layer features, middle layer features and deep layer features of the fan blade image; and S122, fusing the shallow layer features, the middle layer features and the deep layer features to obtain the multi-scale fan feature map.
In step S121, as shown in fig. 4, the extracting the shallow, middle and deep features of the image of the fan blade includes: s1211, performing bidirectional filtering on the fan blade image to obtain an enhanced fan blade image; and S1212, passing the enhanced fan blade image through an encoder model comprising a shallow feature extractor, a middle layer feature extractor and a deep feature extractor to obtain a fan blade shallow feature map, a fan blade middle layer feature map and a fan blade deep feature map, wherein the fan blade shallow feature map is used as the shallow feature, the fan blade middle layer feature map is used as the middle layer feature, and the fan blade deep feature map is used as the deep feature.
It should be appreciated that in step S121, bi-directional filtering of the fan blade image to obtain an enhanced fan blade image is mentioned, which is an image processing technique that combines spatial filtering and gray value filtering for smoothing the image and reducing noise. The bidirectional filtering is used for smoothing the image while maintaining the image edge information. Conventional mean or gaussian filtering may blur the edges of the image because they only consider neighborhood information around the pixel. Bi-directional filtering, which filters by taking into account the spatial distance of the pixels and the gray value similarity, uses two filters: a spatial distance-based filter and a gray value similarity-based filter, which ensures that edge details of the image are preserved while the image is smoothed. In fan blade image processing, bidirectional filtering can help to remove noise in images, so that the edges of fan blades are clearer, and the accuracy of subsequent feature extraction is improved. By enhancing the fan blade image, shallow layer, middle layer and deep layer features can be better extracted for subsequent fan blade defect detection and analysis.
It should be appreciated that the encoder model, which includes shallow, middle, and deep feature extractors, is a neural network model for extracting different levels of feature representations from an input fan blade image. Encoder models in deep learning are typically composed of multiple levels of neural networks, each level being responsible for extracting different levels of features. In this case, the shallow feature extractor is responsible for extracting some low-level image features, such as edges, textures, etc., the middle feature extractor may extract more abstract and semantic features, such as shapes, contours, etc., while the deep feature extractor may extract higher-level features, such as structures, combinations, etc., of objects. The purpose of such encoder models is to capture different levels of information in an image by extracting features layer by layer. The different levels of feature representation may provide more rich and diversified information, facilitating better understanding and analysis of the fan blade images. In fan blade defect detection, the fan blade image can be converted into a feature map with different hierarchical feature representations using an encoder model. These feature maps may be used for subsequent defect detection, classification, or other tasks, and by utilizing the multi-layer features extracted by the encoder model, the accuracy and robustness to fan blade defects may be improved.
In step S122, fusing the shallow feature, the middle layer feature and the deep feature to obtain the multi-scale fan feature map includes: and the self-adaptive fusion module is used for fusing the shallow layer feature map of the fan blade, the middle layer feature map of the fan blade and the deep layer feature map of the fan blade to obtain the multi-scale fan feature map. It should be appreciated that the adaptive fusion module is a technique used in deep learning models to fuse together different levels of feature maps to obtain a multi-scale feature representation, and that the use of multi-scale features in fan blade defect detection and analysis may provide more comprehensive, accurate information that facilitates a better understanding and analysis of the state of the fan blade. The adaptive fusion module is used for learning how to adaptively fuse the shallow feature map, the middle feature map and the deep feature map according to the input shallow feature map, the middle feature map and the deep feature map. Such fusion may be achieved by different methods, such as weighted summation, cascade connection or parallel connection, etc. The adaptive fusion module can determine how to fuse the feature graphs according to the importance and the relevance of the input features by learning adaptive weights or parameters. The self-adaptive fusion module can fully utilize the characteristic information of different layers, and improve the performance of fan blade defect detection and analysis. By fusing the multi-scale features, the details and structural information of the fan blade can be better captured.
Further, the multi-scale fan profile is passed through a decoder to obtain a fan blade decoded image, wherein the decoder includes a plurality of deconvolution layers. And then, the fan blade decoding image is passed through a Yolo network to obtain a fan blade image containing a fan blade defect area positioning anchor window.
Accordingly, in one example, as shown in fig. 5, generating a fan blade image including a fan blade defect region localization anchor window based on the multi-scale fan feature map includes: s131, performing feature distribution optimization on the multi-scale fan feature map to obtain an optimized multi-scale fan feature map; s132, enabling the optimized multi-scale fan characteristic diagram to pass through a decoder to obtain a fan blade decoding image, wherein the decoder comprises a plurality of deconvolution layers; and S133, passing the fan blade decoded image through a Yolo network to obtain a fan blade image containing a fan blade defect area positioning anchor window.
It should be appreciated that deconvolution layer (Deconvolutional layer) is one type of layer commonly used in deep learning models for the process of mapping low-dimensional feature maps back to high-dimensional feature maps. Among image processing tasks, deconvolution is commonly used for tasks such as image reconstruction, image segmentation, and object localization. The deconvolution layer is used for expanding the spatial dimension of the input feature map through learning inverse operation, so that the spatial resolution of the original input is restored. The use of deconvolution layers in the decoder can gradually restore the low resolution feature map to a high resolution image, which is very useful for image reconstruction and object localization. The operation of the deconvolution layer is similar to that of the convolution layer, but the deconvolution layer uses a transposed convolution (transposed convolution) operation. Transpose convolution can spread and fill each pixel value of the input feature map with a convolution kernel to achieve upsampling of the feature map. Deconvolution layers typically contain multiple transpose convolution operations, each of which can adjust the way in which the expansion and filling are performed by learning parameters. In the process of generating the fan blade image based on the multi-scale fan feature map, the deconvolution lamination is used for decoding the optimized multi-scale fan feature map into the fan blade image. By step up sampling and feature reconstruction, the deconvolution layer can recover an image similar to the original fan blade for subsequent fan blade defect region localization and analysis.
It should be appreciated that Yolo (You Only Look Once) is a popular target detection algorithm, which is a real-time target detection method based on deep learning. Yolo has a faster speed and higher accuracy than conventional target detection algorithms. The main idea of the Yolo network is to translate the object detection task into a regression problem by predicting the bounding box and class labels of the object directly in the image. The Yolo network divides the input image into a fixed-size grid and predicts a number of bounding boxes in each grid cell. Each bounding box contains location information of the object and a corresponding class probability. The Yolo network has the advantage that its end-to-end design can accomplish both target detection and classification in a single forward propagation. This makes Yolo have very fast detection speed, suitable for real-time application scenario. Furthermore, the Yolo network is also capable of handling the detection of multiple targets and capturing targets of different scales and different categories. Yolo networks have good advantages in terms of real-time and accuracy.
In the technical scheme of the application, when the enhanced fan blade image is obtained through an encoder model comprising a shallow feature extractor, a middle feature extractor and a deep feature extractor, and the fan blade shallow feature image, the fan blade middle feature image and the fan blade deep feature image are respectively expressed, so that the shallow, middle and deep coding image semantic features of the enhanced fan blade image are respectively expressed, the fan blade shallow feature image, the fan blade middle feature image and the fan blade deep feature image are fused through an adaptive fusion module to obtain a multi-scale fan feature image, and then the multi-scale fan feature image is subjected to decoding regression through a decoder, so that probability distribution corresponding to a label can be obtained based on space domain mapping from a feature space as a feature regression process to a probability distribution space, and a decoding result is obtained. Here, considering the synchronous association expression of the multi-scale fan feature map relative to the image coding semantic features of different depths, probability distribution enrichment corresponding to feature distribution diversification of different feature distribution dimensions in a probability distribution domain of a decoding result may be caused in a spatial domain mapping process, so that a mapping convergence effect to a probability distribution space in the decoding process is affected.
Based on the method, in the training process of the multi-scale fan feature map through the decoder, the weight space exploration constraint based on class matrix regularization is carried out on the weight matrix for carrying out domain mapping on the multi-scale fan feature vector obtained after the multi-scale fan feature map is unfolded, so that the improved multi-scale fan feature vector is obtained.
Accordingly, as shown in fig. 6, performing feature distribution optimization on the multi-scale fan feature map to obtain an optimized multi-scale fan feature map, including: s1311, expanding the multi-scale fan feature map to obtain a multi-scale fan feature vector; s1312, performing weight space exploration constraint on the multi-scale fan feature vector based on class matrix regularization to obtain an optimized multi-scale fan feature vector; and S1313, performing dimension reconstruction on the optimized multi-scale fan feature vector to obtain the optimized multi-scale fan feature map. In step S1312, performing a weight space exploration constraint on the multi-scale fan feature vector based on the class matrix regularization to obtain an optimized multi-scale fan feature vector, including: carrying out weight space exploration constraint on the multi-scale fan feature vector based on class matrix regularization by using the following optimization formula to obtain the optimized multi-scale fan feature vector;
wherein, the optimization formula is:
wherein,is the multi-scale fan characteristic vector, which is specifically expressed as a column vector, +>Is the optimized multi-scale fan characteristic vector which is a row vector,>is a domain transfer matrix which can be learned, +.>The real number domain is represented by the number,/>representing the length of the multi-scale fan feature vector, and +.>Is a weight matrix which can be learned, +.>For the weight matrix after the domain mapping, +.>Representing the transpose of the vector>Representing a matrix multiplication.
Here, consider the weight spatial domain of the weight matrix and the multi-scale fan feature vectorDomain differences (domain gap) between probability distribution domains of the domain mapping result of (2) by weight matrix +.>Relative to the multiscale blower feature vector +.>The regularized class matrix representation of (2) is used as an inter-domain migration agent (inter-domain transferring agent) to transfer the probability distribution of the valuable weight constraint into the weight space, so that excessive exploration (over-explloit) of the weight distribution in the weight space by a probability distribution domain with rich probability distribution (rich propability distributed) in the domain mapping process based on the weight space is avoided, the mapping convergence effect to the probability distribution space in the decoding process is improved, and the training effect of the decoder is also improved.
In summary, the method for rapidly dividing the defect area of the fan blade based on the artificial intelligence according to the embodiment of the application is explained, which can perform image processing on the fan blade image of the detected fan blade before performing image recognition and defect detection, and automatically position the defect area in the fan blade image.
FIG. 7 is a block diagram of an artificial intelligence based fan blade defect area fast segmentation system 100 according to an embodiment of the present application. As shown in fig. 7, an artificial intelligence based fan blade defect area fast segmentation system 100 according to an embodiment of the present application includes: an image acquisition module 110, configured to acquire a fan blade image of the detected fan blade acquired by the camera; a multi-scale image feature extraction module 120, configured to perform multi-scale image feature extraction on the fan blade image to obtain a multi-scale fan feature map; and a localization anchor window image generation module 130, configured to generate a fan blade image including a localization anchor window of a fan blade defect area based on the multi-scale fan feature map.
In one example, in the above-described artificial intelligence based fan blade defect region rapid segmentation system 100, the multi-scale image feature extraction module 120 includes: the feature extraction unit is used for extracting shallow layer features, middle layer features and deep layer features of the fan blade image; and the feature fusion unit is used for fusing the shallow layer features, the middle layer features and the deep layer features to obtain the multi-scale fan feature map.
In one example, in the above-described fan blade defect area rapid segmentation system 100 based on artificial intelligence, the feature extraction unit includes: the bidirectional filtering subunit is used for carrying out bidirectional filtering on the fan blade image to obtain an enhanced fan blade image; and an encoding subunit, configured to pass the enhanced fan blade image through an encoder model that includes a shallow feature extractor, a middle layer feature extractor, and a deep feature extractor to obtain a fan blade shallow feature map, a fan blade middle layer feature map, and a fan blade deep feature map, where the fan blade shallow feature map is used as the shallow feature, the fan blade middle layer feature map is used as the middle layer feature, and the fan blade deep feature map is used as the deep feature.
Here, it will be appreciated by those skilled in the art that the specific functions and operations of the respective modules in the above-described artificial intelligence-based fan blade defect area rapid dividing system 100 have been described in detail in the above description of the artificial intelligence-based fan blade defect area rapid dividing method with reference to fig. 1 to 6, and thus, repetitive descriptions thereof will be omitted.
As described above, the fan blade defect area rapid segmentation system 100 based on artificial intelligence according to the embodiment of the present application may be implemented in various wireless terminals, such as a server or the like having an fan blade defect area rapid segmentation algorithm based on artificial intelligence. In one example, the artificial intelligence based fan blade defect region rapid segmentation system 100 according to embodiments of the present application may be integrated into a wireless terminal as one software module and/or hardware module. For example, the artificial intelligence based fan blade defect region rapid segmentation system 100 may be a software module in the operating system of the wireless terminal, or may be an application developed for the wireless terminal; of course, the artificial intelligence based fan blade defect area fast segmentation system 100 may also be one of many hardware modules of the wireless terminal.
Alternatively, in another example, the artificial intelligence based fan blade defect area fast segmentation system 100 may be a separate device from the wireless terminal, and the artificial intelligence based fan blade defect area fast segmentation system 100 may be connected to the wireless terminal through a wired and/or wireless network and communicate interaction information in accordance with a agreed data format.
FIG. 8 is an application scenario diagram of an artificial intelligence based fan blade defect area fast segmentation method according to an embodiment of the present application. As shown in fig. 8, in this application scenario, first, a fan blade image (e.g., D illustrated in fig. 8) of a detected fan blade (e.g., N illustrated in fig. 8) acquired by a camera (e.g., C illustrated in fig. 8) is acquired, and then, the fan blade image is input to a server (e.g., S illustrated in fig. 8) deployed with an artificial intelligence-based fan blade defect area fast segmentation algorithm, wherein the server is capable of processing the fan blade image using the artificial intelligence-based fan blade defect area fast segmentation algorithm to obtain a fan blade image including a fan blade defect area localization anchor window.
Furthermore, those skilled in the art will appreciate that the various aspects of the invention are illustrated and described in the context of a number of patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.) or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the present application may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the following claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.
Claims (7)
1. A fan blade defect area rapid segmentation method based on artificial intelligence is characterized by comprising the following steps:
acquiring a fan blade image of a detected fan blade acquired by a camera;
carrying out multi-scale image feature extraction on the fan blade image to obtain a multi-scale fan feature map;
generating a fan blade image containing a fan blade defect area positioning anchor window based on the multi-scale fan characteristic diagram;
based on the multi-scale fan feature map, generating a fan blade image including a fan blade defect region localization anchor window, comprising:
performing feature distribution optimization on the multi-scale fan feature map to obtain an optimized multi-scale fan feature map;
passing the optimized multi-scale fan feature map through a decoder to obtain a fan blade decoded image, wherein the decoder comprises a plurality of deconvolution layers;
and passing the fan blade decoded image through a Yolo network to obtain a fan blade image comprising a fan blade defect area positioning anchor window;
performing feature distribution optimization on the multi-scale fan feature map to obtain an optimized multi-scale fan feature map, including:
expanding the multi-scale fan characteristic map to obtain a multi-scale fan characteristic vector;
performing weight space exploration constraint based on class matrix regularization on the multi-scale fan feature vector to obtain an optimized multi-scale fan feature vector;
performing dimension reconstruction on the optimized multi-scale fan feature vector to obtain an optimized multi-scale fan feature map;
performing a weight space exploration constraint based on class matrix regularization on the multi-scale fan feature vector to obtain an optimized multi-scale fan feature vector, comprising:
carrying out weight space exploration constraint on the multi-scale fan feature vector based on class matrix regularization by using the following optimization formula to obtain the optimized multi-scale fan feature vector;
wherein, the optimization formula is:
；
；
wherein,is the multi-scale fan characteristic vector, +.>Is the optimized multi-scale fan feature vector, < >>Is a domain transfer matrix which can be learned, +.>，/>Representing the real number field, ++>Representing the length of the multi-scale fan feature vector, and +.>Is a weight matrix which can be learned, +.>，/>For the weight matrix after the domain mapping, +.>Representing the transpose of the vector>Representing a matrix multiplication.
2. The method for quickly segmenting the defective area of the fan blade based on artificial intelligence according to claim 1, wherein the step of extracting the multi-scale image features from the fan blade image to obtain a multi-scale fan feature map comprises the following steps:
extracting shallow layer features, middle layer features and deep layer features of the fan blade image;
and fusing the shallow layer feature, the middle layer feature and the deep layer feature to obtain the multi-scale fan feature map.
3. The method for quickly segmenting the defective area of the fan blade based on artificial intelligence according to claim 2, wherein extracting the shallow layer feature, the middle layer feature and the deep layer feature of the image of the fan blade comprises:
performing bidirectional filtering on the fan blade image to obtain an enhanced fan blade image;
and passing the enhanced fan blade image through an encoder model comprising a shallow feature extractor, a middle layer feature extractor and a deep feature extractor to obtain a fan blade shallow feature map, a fan blade middle layer feature map and a fan blade deep feature map, wherein the fan blade shallow feature map is used as the shallow feature, the fan blade middle layer feature map is used as the middle layer feature, and the fan blade deep feature map is used as the deep feature.
4. The method of claim 3, wherein fusing the shallow features, the middle features, and the deep features to obtain the multi-scale fan feature map comprises:
and the self-adaptive fusion module is used for fusing the shallow layer feature map of the fan blade, the middle layer feature map of the fan blade and the deep layer feature map of the fan blade to obtain the multi-scale fan feature map.
5. An artificial intelligence based fan blade defect area rapid segmentation system, which is characterized by comprising:
the image acquisition module is used for acquiring fan blade images of detected fan blades acquired by the camera;
the multi-scale image feature extraction module is used for carrying out multi-scale image feature extraction on the fan blade image so as to obtain a multi-scale fan feature map;
the positioning anchor window image generation module is used for generating a fan blade image containing a positioning anchor window of a fan blade defect area based on the multi-scale fan characteristic diagram;
the positioning anchor window image generation module is specifically used for:
performing feature distribution optimization on the multi-scale fan feature map to obtain an optimized multi-scale fan feature map;
passing the optimized multi-scale fan feature map through a decoder to obtain a fan blade decoded image, wherein the decoder comprises a plurality of deconvolution layers;
and passing the fan blade decoded image through a Yolo network to obtain a fan blade image comprising a fan blade defect area positioning anchor window;
the positioning anchor window image generation module is specifically used for:
expanding the multi-scale fan characteristic map to obtain a multi-scale fan characteristic vector;
performing weight space exploration constraint based on class matrix regularization on the multi-scale fan feature vector to obtain an optimized multi-scale fan feature vector;
performing dimension reconstruction on the optimized multi-scale fan feature vector to obtain an optimized multi-scale fan feature map;
the positioning anchor window image generation module is specifically used for:
carrying out weight space exploration constraint on the multi-scale fan feature vector based on class matrix regularization by using the following optimization formula to obtain the optimized multi-scale fan feature vector;
wherein, the optimization formula is:
；
；
wherein,is the multi-scale fan characteristic vector, +.>Is the optimized multi-scale fan feature vector, < >>Is a domain transfer matrix which can be learned, +.>，/>Representing the real number field, ++>Representing the length of the multi-scale fan feature vector, and +.>Is a weight matrix which can be learned, +.>，/>For the weight matrix after the domain mapping, +.>Representing the transpose of the vector>Representing a matrix multiplication.
6. The system for rapidly segmenting a defective area of a fan blade based on artificial intelligence of claim 5, wherein the multi-scale image feature extraction module comprises:
the feature extraction unit is used for extracting shallow layer features, middle layer features and deep layer features of the fan blade image;
and the characteristic fusion unit is used for fusing the shallow layer characteristic, the middle layer characteristic and the deep layer characteristic to obtain the multi-scale fan characteristic diagram.
7. The fan blade defect area fast segmentation system based on artificial intelligence according to claim 6, wherein the feature extraction unit comprises:
the bidirectional filtering subunit is used for carrying out bidirectional filtering on the fan blade image to obtain an enhanced fan blade image;
and an encoding subunit, configured to pass the enhanced fan blade image through an encoder model that includes a shallow feature extractor, a middle layer feature extractor, and a deep feature extractor to obtain a fan blade shallow feature map, a fan blade middle layer feature map, and a fan blade deep feature map, where the fan blade shallow feature map is used as the shallow feature, the fan blade middle layer feature map is used as the middle layer feature, and the fan blade deep feature map is used as the deep feature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311313238.5A CN117058396B (en) | 2023-10-11 | 2023-10-11 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311313238.5A CN117058396B (en) | 2023-10-11 | 2023-10-11 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117058396A CN117058396A (en) | 2023-11-14 |
| CN117058396B true CN117058396B (en) | 2023-12-26 |
Family
ID=88664770
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311313238.5A Active CN117058396B (en) | 2023-10-11 | 2023-10-11 | Fan blade defect area rapid segmentation method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117058396B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117351062B (en) * | 2023-12-04 | 2024-02-23 | 尚特杰电力科技有限公司 | Fan blade defect diagnosis method, device and system and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565571A (en) * | 2022-02-21 | 2022-05-31 | 华能华家岭风力发电有限公司 | Fan blade defect detection method and device based on computer vision |
| CN114926400A (en) * | 2022-04-20 | 2022-08-19 | 上海电机学院 | Fan blade defect detection method based on improved YOLOv5 |
| CN115049856A (en) * | 2022-06-27 | 2022-09-13 | 中广核(北京)新能源科技有限公司 | Fan blade fault detection method and system based on improved YOLOv5 |
| CN115187878A (en) * | 2022-06-22 | 2022-10-14 | 河南理工大学 | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device |
| CN115359239A (en) * | 2022-08-25 | 2022-11-18 | 中能电力科技开发有限公司 | Wind power blade defect detection and positioning method and device, storage medium and electronic equipment |
| CN116310272A (en) * | 2022-12-30 | 2023-06-23 | 西安交通大学 | Fan blade defect detection method based on improved YOLOv4 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986071B (en) * | 2018-05-31 | 2020-12-01 | 上海扩博智能技术有限公司 | Automatic blade detection and tracking method, system, equipment and storage medium |
| KR20230134846A (en) * | 2022-03-15 | 2023-09-22 | 연세대학교 산학협력단 | Multiscale object detection device and method |
-
2023
- 2023-10-11 CN CN202311313238.5A patent/CN117058396B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565571A (en) * | 2022-02-21 | 2022-05-31 | 华能华家岭风力发电有限公司 | Fan blade defect detection method and device based on computer vision |
| CN114926400A (en) * | 2022-04-20 | 2022-08-19 | 上海电机学院 | Fan blade defect detection method based on improved YOLOv5 |
| CN115187878A (en) * | 2022-06-22 | 2022-10-14 | 河南理工大学 | Unmanned aerial vehicle image analysis-based blade defect detection method for wind power generation device |
| CN115049856A (en) * | 2022-06-27 | 2022-09-13 | 中广核(北京)新能源科技有限公司 | Fan blade fault detection method and system based on improved YOLOv5 |
| CN115359239A (en) * | 2022-08-25 | 2022-11-18 | 中能电力科技开发有限公司 | Wind power blade defect detection and positioning method and device, storage medium and electronic equipment |
| CN116310272A (en) * | 2022-12-30 | 2023-06-23 | 西安交通大学 | Fan blade defect detection method based on improved YOLOv4 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117058396A (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110070530B (en) | Transmission line icing detection method based on deep neural network | |
| CN111696075A (en) | Intelligent fan blade defect detection method based on double-spectrum image | |
| CN108416294B (en) | Fan blade fault intelligent identification method based on deep learning | |
| CN113486865B (en) | Power transmission line suspended foreign object target detection method based on deep learning | |
| CN117058396B (en) | Fan blade defect area rapid segmentation method and system based on artificial intelligence | |
| CN112837315B (en) | Deep learning-based transmission line insulator defect detection method | |
| CN109741320A (en) | A kind of wind electricity blade fault detection method based on Aerial Images | |
| CN112365468A (en) | AA-gate-Unet-based offshore wind power tower coating defect detection method | |
| CN115861263A (en) | Insulator defect image detection method based on improved YOLOv5 network | |
| CN113297915A (en) | Insulator recognition target detection method based on unmanned aerial vehicle inspection | |
| Hu et al. | A method for measuring ice thickness of wind turbine blades based on edge detection | |
| CN112330592A (en) | Fan blade defect detection method and device and storage medium | |
| CN112465777A (en) | Fan blade surface defect identification technology based on video stream | |
| CN111767826A (en) | Timing fixed-point scene abnormity detection method | |
| CN112465776B (en) | Crack intelligent detection method based on wind turbine surface blurred image | |
| CN110136059A (en) | The wind electricity blade image split-joint method of linear feature | |
| Zhu | Detection and recognition of concrete cracks on building surface based on machine vision | |
| CN115423810B (en) | Blade icing form analysis method for wind generating set | |
| CN116342693A (en) | Bridge cable surface damage rapid positioning method based on point cloud and convolutional neural network | |
| CN115953312A (en) | Joint defogging detection method and device based on single image and storage medium | |
| CN115410154A (en) | Method for identifying thermal fault of electrical equipment of wind power engine room | |
| CN115049856A (en) | Fan blade fault detection method and system based on improved YOLOv5 | |
| KR20190062834A (en) | Apparatus for inspecting blade of wind power generator and method thereof | |
| CN113095181A (en) | Traffic sign identification method based on Defense-GAN | |
| CN111709887A (en) | Image rain removing method based on sparse blind detection and image multiple feature restoration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |