CN112907614A - Yoov 5-segnet insulator string contour extraction method based on depth feature fusion - Google Patents

Abstract

The invention discloses a yolov5-segnet insulator string contour extraction method based on depth feature fusion, which comprises the following steps of 1, preprocessing an aerial insulator image set to obtain a preprocessed insulator image set T; 2. randomly selecting m pieces of the preprocessed aerial insulator image set for expansion processing to obtain expanded aerial insulator imagesCollection T₁And taking the rest images in the T as a test set T₂(ii) a 3. Collecting T the extended aerial insulator image₁The n aerial-photographing insulator images are subjected to mask marking, and an aerial-photographing insulator image set marked by the mask is used as a training set T for insulator detection₁₁Taking the rest aerial image insulator image as an insulator verification set T₂₂(ii) a 4. And constructing a yolov5-segnet deep learning network with depth feature fusion. The invention overcomes the defect of a large amount of environmental interference caused by the detection of a large-scale long insulator string by the traditional network, and extracts the outline of the aerial photographing insulator, thereby effectively eliminating the interference for the defect detection of the insulator.

Description

Yoov 5-segnet insulator string contour extraction method based on depth feature fusion

Technical Field

The invention relates to the field of deep learning image processing, in particular to a yolov5-segnet insulator string contour extraction method based on depth feature fusion.

Background

The insulator is an important component for fixing a current-carrying conductor and preventing current from flowing back to the ground in the power transmission line, but the insulator is corroded by the environment for a long time, so that the potential safety hazard of breakage or falling is easily caused, and huge economic loss and casualties are caused due to serious consequences; therefore, the method is particularly important for the regular detection and timely maintenance of the insulator. With the continuous development of economy and technology and the proposal of a smart power grid concept, the power grid inspection is also developed from the traditional manual inspection into the power inspection of helicopters and unmanned planes; under the technical background, the unmanned aerial vehicle or the helicopter acquires visible, infrared or ultraviolet images of an insulator in a power transmission line by carrying a multi-mode camera, and the image processing technology based on deep learning analyzes the characteristics of the images, so that the intelligent detection of a power transmission system and equipment is realized, and the inspection efficiency and accuracy are greatly improved.

At present, an insulator self-explosion fault detection method based on deep learning, chen, Yan, and the like, which are 'aerial photography insulator convolution neural network detection and self-explosion identification research' (electronic measurement and instrument study, 2017), provides an improved Fast-RCNN algorithm based on aerial photography insulator detection and self-explosion fault identification; an effective insulator self-explosion defect positioning method (electronics and informatics, 2020) of old literature, li-color forest, yun-bin and the like provides an insulator self-explosion fault detection method based on deep learning. In the above insulator target detection based on deep learning, the detection result is that the insulator region is marked by the rectangular frame, and there are problems that the background and the insulators cannot be accurately distinguished, and the insulators in dense distribution are difficult to separate. It is difficult to further improve the fault detection accuracy.

Therefore, a related learner proposes a deep learning directional recognition algorithm in a specific field, angle rotation parameters are introduced into an axis alignment rectangular frame to form a directional detection frame to detect a glass insulator at any angle in an aerial image, the output form of the rectangular frame is not changed at all, the method has no universality for large-scale long-string and radian insulator strings and still brings huge environmental interference, and therefore, the method has great significance for researching a high-precision contour extraction algorithm for detecting the insulator strings under a complex background.

Disclosure of Invention

The invention aims to provide a yolov5-segnet insulator string contour extraction method based on depth feature fusion to solve the problems in the background technology.

In order to achieve the purpose, the invention provides the following technical scheme:

the yolov5-segnet insulator string contour extraction method based on depth feature fusion comprises the following steps:

s1, preprocessing the aerial insulator image set to obtain a preprocessed insulator image set;

s2, randomly selecting m pieces of the preprocessed aerial photographing insulator image set seeds for expansion processing to obtain an expanded aerial photographing insulator image set T₁And taking the rest images in the T as a test set T₂；

S3, carrying out mask marking on the n aerial photographing insulator images in the extended aerial photographing insulator image set T, and taking the aerial photographing insulator image set marked by the mask as a training set T for insulator detection₁₁And taking the rest aerial image insulator images as a verification set T for insulator detection₂₂；

S4, constructing a yolov5-segnet deep learning network with depth feature fusion;

s5 construction test T₂The reference network selects the parameter model of the μ best detection precision in the μ iteration results in the step 4.4.3 as the test set T₂The reference network of (1). Test set T₂Inputting the data into a reference network for testing to obtain a test result data set T₃To makeUsing contour extraction function in openCV to pair T₃Detecting the outline of the picture in the data set and drawing the outline to obtain an insulator outline picture,

the method for constructing the yolov5-segnet deep learning network with the depth feature fusion comprises the following steps:

s4.1, improving a pooling layer of a yolov5 target detection network, and firstly replacing original max pooling with dynamic k max pooling to obtain an improved yolov5 target detection network;

s4.2, building a semantic segmentation network, extracting target features on the basis of yolov5 output feature graphs, performing convolution and upsampling operations, performing 3 × 3 convolution and 2 times upsampling twice after extracting feature targets on the 13 × 13 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on the 26 × 26 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on the 52 × 52 output feature graphs, performing 3 × 3 convolution twice after extracting feature targets on the 52 × 52 output feature graphs, performing concat (tension splicing) on the feature graphs with unified sizes for unifying the feature graph sizes, inputting the feature graphs into a spatial attention module to refine features, and then performing 3 × 3 convolution twice to form the semantic segmentation module;

the improved yolov5 target detection module is combined with a semantic segmentation module to form the yolov5-segnet network with the depth feature fusion;

s4.3, starting to train the network, defining the current iteration number as mu, and initializing mu to be 1; the maximum iteration number is mu max; carrying out mu-th random initialization on parameters of each layer in the depth feature fused yolov5-segnet network, thereby obtaining a mu-th iterative depth feature fused yolov5-segnet network;

s4.4, training set T₁₁Inputting the yolov5-segnet network of the depth feature fusion after the mu-th iteration for training.

As a further aspect of the invention, the training set T₁₁Inputting the yolov5-segnet network of the depth feature fusion after the mu-th iteration for training, and comprising the following steps:

s4.4.1 YO after first stage traininglov5 object detection Module for T₁₁The training set is operated in parallel, on one hand, T is used₁₁Carrying out contour detection on all pictures in a training set to obtain a marked contour point (xi, yi), wherein i is 1,2,3,4.. n, then scratching the contour map, calculating a minimum bounding box of all the contour maps, wherein the coordinates of the upper left corner and the lower right corner of the minimum bounding box are (xmin, ymax and ymin), then carrying out loss calculation and precision solving on the target detection iteration output result, and on the other hand, carrying out T on the T by a front 52 lamination layer of a trunk CSPDarkNet-53 of improved yolov5₁₁Carrying out convolution operation on the training set, wherein each layer of convolution operation is followed by a batch regularization layer and a LeakyReLU activation layer;

s4.4.2, training semantic segmentation module in the second stage, screening by confidence coefficient on the basis of the training result in the first stage, and retaining the block diagram with confidence coefficient more than 0.5 for matting to form N₁Training data set due to N₁The sizes of the pictures in the method are different, and N is clustered by using a k-means clustering method₁All the pictures in the picture are clustered to obtain the optimal mask size, and then N is used₁Image size resize to optimal mask size forming N₂Training set, will N₂Inputting the training set into a semantic segmentation module for training;

the clustering process is as follows:

A. setting central points of 1 feature graph, wherein each central point corresponds to 1 clustering center;

B. the distance d between each sample in n1 and the center point is found using equation (1):

in formula (1), b (xi), b (yi) are the abscissa and ordinate of the ith sample, and c (xi), c (yi) are the abscissa and ordinate of the ith central point;

C. calculating the mean value of the distance from each sample to the central point by using the formula (2)

D. Iteratively updating the clustering center until the clustering center is not changed any more;

s4.4.3, performing mu iterations, using the validation set T after each iteration₂₂The network is authenticated and the authentication results and network parameters are retained for each pass.

Compared with the prior art, the invention has the beneficial effects that: the built semantic segmentation module is combined with the yolov5 target detection module to form a new yolov5-segnet deep learning network with depth feature fusion, so that the defect of a large amount of environmental interference caused by detection of a large long insulator string in the traditional network is overcome, the outline of the insulator is extracted by aerial photography, and the interference is effectively eliminated for insulator defect detection.

Drawings

FIG. 1 is a training flow chart of the yolov5-segnet network algorithm for depth feature fusion in the invention.

FIG. 2 is a flow chart of semantic segmentation according to the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without any inventive step, are within the scope of the present invention.

Referring to fig. 1 and 2, the method for extracting the outline of the yolov5-segnet insulator string based on depth feature fusion comprises the following steps:

s1, preprocessing the aerial insulator image set, detecting whether the calculation capability of a computer meets the calculation requirement of the size of the original image and whether the problems of shaking blur and the like exist, and performing size conversion and shaking and noise removal processing to obtain 1000 preprocessed insulator images;

s2, randomly selecting 200 aerial insulated sub-images in the preprocessed 1000 aerial insulated sub-images to be used as a test set T₁Expanding the remaining 800 aerial photography insulator images, and performing operations such as mirror image overturning, rotation and the like on the preprocessed aerial photography insulator on the basis of processing the images in the step 1 to simulate the conditions that an unmanned aerial vehicle or a helicopter suffers turbulence jitter and is at different shooting visual angles in the aerial photography process, expanding the data, and obtaining an expanded 1600 aerial photography insulator image set T₂；

S3, at T₂1280 aerial photography insulator images are selected in the image set, artificial mask marking is carried out on the 1280 aerial photography insulator images by adopting a Labelling tool to generate a json file, and therefore the marked aerial photography insulator image set is obtained and serves as a training set T for insulator detection₂₁；T₂Taking the remaining 320 aerial photography insulator image sets of the image set as a verification set T₂₂；

S4, constructing a yolov5-segnet deep learning network with depth feature fusion;

s4.1, improving a pooling layer of a yolov5 target detection network, and firstly replacing original max pooling with dynamic k max pooling to obtain an improved yolov5 target detection network;

s4.2, building a semantic segmentation network, extracting target features on the basis of yolov5 output feature graphs, performing convolution and upsampling operations, performing 3 × 3 convolution and 2 times upsampling twice after extracting feature targets on the 13 × 13 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on the 26 × 26 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on the 52 × 52 output feature graphs, performing 3 × 3 convolution twice after extracting feature targets, performing concat (tension splicing) on the feature graphs with unified sizes for the purpose of unifying the feature graph sizes, inputting the feature graphs into a spatial attention module to refine features, and then performing 3 × 3 convolution twice, thus forming the semantic segmentation module.

The modified yolov5 target detection module is combined with a semantic segmentation module to form the yolov5-segnet network with the depth feature fusion.

S4.3, starting to train the network, defining the current iteration number as mu, initializing the current iteration number as mu being 1, setting the maximum iteration number as mu max, and carrying out the mu-th random initialization on the parameters of each layer in the depth feature fused yolov5-segnet network so as to obtain the depth feature fused yolov5-segnet network of the mu-th iteration;

s4.4, training set T₁₁Inputting the yolov5-segnet network with the depth feature fused after the mu-th iteration, and training in two stages;

s4.4.1, a yolov5 target detection module after the first stage training improvement, for T₁₁The training set is operated in parallel, on one hand, T is used₁₁Carrying out contour detection on all pictures in a training set to obtain a marked contour point (xi, yi), wherein i is 1,2,3,4.. n, then scratching the contour map, calculating a minimum bounding box of all the contour maps, wherein the coordinates of the upper left corner and the lower right corner of the minimum bounding box are (xmin, ymax and ymin), then carrying out loss calculation and precision solution on the target detection iteration output result, and on the other hand, carrying out T on the T by a front 52 convolution layer of a trunk CSPDarkNet-53 of improved YOLOv5₁₁Performing convolution operation on the training set, wherein the sizes of convolution kernels are 3 × 3 and 1 × 1 respectively, and each layer of convolution operation is followed by a batch regularization layer and a LeakyReLU activation layer for accelerating the training speed;

s4.4.2, training semantic segmentation module in the second stage, screening by confidence coefficient on the basis of the training result in the first stage, and retaining the block diagram with confidence coefficient more than 0.5 for matting to form N₁Training data set due to N₁The sizes of the pictures in the N1 are different, all the pictures in the N1 are clustered by using a k-means clustering method to obtain the optimal mask size, and then the N is used₁Image size resize to optimal mask size forming N₂Training set, will N₂Inputting the training set into a semantic segmentation module for training;

the clustering process is as follows:

A. setting central points of 1 feature graph, wherein each central point corresponds to 1 clustering center;

B. the distance d between each sample in N1 and the center point is found using equation (1):

in formula (1), b (xi), b (yi) are the abscissa and ordinate of the ith sample, and c (xi), c (yi) are the abscissa and ordinate of the ith central point;

C. calculating the mean value of the distance from each sample to the central point by using the formula (2)

D. Iteratively updating the clustering center until the clustering center is not changed any more;

s4.4.3, performing mu iterations, using the validation set T after each iteration₂₂The network is authenticated and the authentication results and network parameters are retained for each pass.

S5 construction test T₂The Mu best parameter in S4.4.3 is selected as the test set T₂The reference network inputs the test set into the reference network for testing to obtain a test result.

While the preferred embodiments of this patent have been described in detail, this patent is not limited to the embodiments described above, and variations can be made within the knowledge of those skilled in the art without departing from the spirit of the patent.

Claims (2)

1. The yolov5-segnet insulator string contour extraction method based on depth feature fusion comprises the following steps:

s1, preprocessing the aerial insulator image set to obtain a preprocessed insulator image set;

s2, randomly selecting m pieces of the preprocessed aerial photographing insulator image set seeds for expansion processing to obtain an expanded aerial photographing insulator image set T1, wherein the rest images in the T are used as a test set T2;

s3, carrying out mask marking on the n aerial photographing insulator images in the extended aerial photographing insulator image set T, taking the aerial photographing insulator image set marked by the mask as a training set T11 for insulator detection, and taking the rest aerial photographing insulator images as a verification set T22 for insulator detection;

s4, constructing a yolov5-segnet deep learning network with depth feature fusion;

s5, constructing a reference network for testing T2, and selecting the μ best parameter model in the μ iteration result in the step 4.4.3 as the reference network of the test set T2. Inputting the test set T2 into a reference network for testing to obtain a test result data set T3, carrying out contour detection on pictures in the T3 data set by using a contour extraction function in openCV and drawing a contour to obtain an insulator contour diagram,

the method is characterized in that the method for constructing the yolov5-segnet deep learning network with the depth feature fusion comprises the following steps: s4.1, improving a pooling layer of a yolov5 target detection network, and firstly replacing original max pooling with dynamic k max pooling to obtain an improved yolov5 target detection network;

s4.2, building a semantic segmentation network, extracting target features on the basis of yolov5 output feature graphs, performing convolution and upsampling operations, performing 3 × 3 convolution and 2 times upsampling twice after respectively extracting feature targets on 13 × 13 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on 26 × 26 output feature graphs, performing 3 × 3 convolution and 2 times upsampling once after extracting feature targets on 52 × 52 output feature graphs, performing concat (tensor splicing) on the feature graphs with unified sizes for the purpose of unifying the feature graph sizes, inputting a spatial attention module to refine features, and performing 3 × 3 convolution twice, wherein the semantic segmentation module is formed by the operations;

the improved yolov5 target detection module is combined with a semantic segmentation module to form the yolov5-segnet network with the depth feature fusion;

s4.3, starting to train the network, defining the current iteration number as mu, and initializing mu to be 1; the maximum iteration number is mu max; carrying out mu-th random initialization on parameters of each layer in the depth feature fused yolov5-segnet network, thereby obtaining a mu-th iterative depth feature fused yolov5-segnet network;

s4.4, training set T₁₁Inputting the yolov5-segnet network of the depth feature fusion after the mu-th iteration for training.

2. The method for extracting the outline of the yolov5-segnet insulator string based on depth feature fusion as claimed in claim 1, wherein the training set T11 is input into the yolov5-segnet network of depth feature fusion after the μ iteration for training, and comprises the following steps:

s4.4.1, a yolov5 target detection module after the first stage training improvement, for T₁₁The training set is operated in parallel, on one hand, T is used₁₁Carrying out contour detection on all pictures in a training set to obtain a marked contour point (xi, yi), wherein i is 1,2,3,4.. n, then scratching the contour map, calculating a minimum bounding box of all the contour maps, wherein the coordinates of the upper left corner and the lower right corner of the minimum bounding box are (xmin, ymax and ymin), then carrying out loss calculation and precision solving on the target detection iteration output result, and on the other hand, carrying out T-layer pair on the T-layer by the first 52 volume layers of the trunk CSPDarkNet-53 of improved yolov5₁₁Carrying out convolution operation on the training set, wherein each layer of convolution operation is followed by a batch regularization layer and a LeakyReLU activation layer;

s4.4.2, training semantic segmentation module in the second stage, screening by confidence coefficient on the basis of the training result in the first stage, and retaining the block diagram with confidence coefficient more than 0.5 for matting to form N₁Training data set due to N₁The sizes of the pictures in the method are different, and N is clustered by using a k-means clustering method₁All the pictures in the picture are clustered to obtain the optimal mask size, and then N is used₁Image size resize to optimal mask size forming N₂Training set, will N₂Inputting the training set into a semantic segmentation module for training;

the clustering process is as follows:

A. setting central points of 1 feature graph, wherein each central point corresponds to 1 clustering center;

B. calculating N by the formula (1)₁Distance d between each sample and the center point:

in formula (1), b (xi), b (yi) are the abscissa and ordinate of the ith sample, and c (xi), c (yi) are the abscissa and ordinate of the ith central point;

C. calculating the mean value of the distance from each sample to the central point by using the formula (2)

D. Iteratively updating the clustering center until the clustering center is not changed any more;

s4.4.3, performing mu iterations, using the validation set T after each iteration₂₂The network is authenticated and the authentication results and network parameters are retained for each pass.

Images