CN115761537B - Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism - Google Patents
Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism Download PDFInfo
- Publication number
- CN115761537B CN115761537B CN202211418023.5A CN202211418023A CN115761537B CN 115761537 B CN115761537 B CN 115761537B CN 202211418023 A CN202211418023 A CN 202211418023A CN 115761537 B CN115761537 B CN 115761537B
- Authority
- CN
- China
- Prior art keywords
- foreign matter
- foreign
- image
- transmission line
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000007246 mechanism Effects 0.000 title claims abstract description 45
- 230000001502 supplementing effect Effects 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 78
- 238000012549 training Methods 0.000 claims abstract description 40
- 239000002775 capsule Substances 0.000 claims abstract description 21
- 230000004927 fusion Effects 0.000 claims abstract description 19
- 239000011159 matrix material Substances 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims description 12
- 230000006399 behavior Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 230000002457 bidirectional effect Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 238000004891 communication Methods 0.000 claims description 7
- 238000012795 verification Methods 0.000 claims description 7
- 230000009545 invasion Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 6
- 230000033001 locomotion Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000005764 inhibitory process Effects 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract description 18
- 230000006870 function Effects 0.000 description 11
- 239000013598 vector Substances 0.000 description 10
- 230000000694 effects Effects 0.000 description 8
- 230000002159 abnormal effect Effects 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 6
- 238000007689 inspection Methods 0.000 description 6
- 238000009499 grossing Methods 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 230000003068 static effect Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000012212 insulator Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 101100441251 Arabidopsis thaliana CSP2 gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000005770 birds nest Nutrition 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000010924 continuous production Methods 0.000 description 1
- 238000013434 data augmentation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000005684 electric field Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000002440 industrial waste Substances 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 239000002985 plastic film Substances 0.000 description 1
- 229920006255 plastic film Polymers 0.000 description 1
- APTZNLHMIGJTEW-UHFFFAOYSA-N pyraflufen-ethyl Chemical compound C1=C(Cl)C(OCC(=O)OCC)=CC(C=2C(=C(OC(F)F)N(C)N=2)Cl)=C1F APTZNLHMIGJTEW-UHFFFAOYSA-N 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 229920003169 water-soluble polymer Polymers 0.000 description 1
- 235000005765 wild carrot Nutrition 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of foreign matter identification of transmission lines, and discloses a transmission line foreign matter intrusion identification method for a dynamic feature supplementing mechanism, which comprises the following steps: acquiring video images around a real-time power transmission line; preprocessing by adopting a Gaussian filter algorithm; inputting the video frame sequence into an image detection model with multi-layer feature fusion, and identifying and positioning the foreign matters; extracting foreign matter type information by a matrix capsule network classifier; expanding the training data set by a method for generating an countermeasure network; judging and determining the foreign matter type information by using a pre-trained foreign matter detection model facing the dynamic feature supplementing mechanism, and determining the early warning level; after the early warning level is output, the system prompts the influence of the foreign matters on the transmission line on the line, and the foreign matter intrusion recognition and early warning method can better avoid the problem of missing recognition, reduces the labor cost and improves the foreign matter monitoring efficiency.
Description
Technical Field
The invention relates to the technical field of foreign matter identification of transmission lines, in particular to a method for identifying foreign matter invasion of a transmission line facing a dynamic characteristic supplementing mechanism.
Background
The transmission line is a main carrier for power transmission, is also an important component of a power system, and plays a vital role in safe and stable operation of the system. In recent years, with the increasing of electricity consumption, the voltage level of the power transmission lines in China is continuously improved, the number of the power transmission lines is greatly increased, and the power transmission lines are widely distributed in various areas such as towns and rural areas in China. In addition, the positions of the transmission lines are complex and changeable, such as places of highland areas, hills, basins, mountains and the like, and foreign matters of the transmission lines are easy to attach. And living goods such as knitwear, balloons, kites, plastic films and the like are easy to attach in places such as densely populated residential areas, commercial areas and the like. Transmission lines are also subject to bird attack in natural environments, such as bird nest in transmission lines. If the foreign matters are not found and cleaned in time, the normal operation of the transmission line can be affected, and personal safety can be endangered under severe conditions, so that unexpected electric shock accidents and the like are caused. When there is dense forest near transmission line, meetting the circumstances that weather environment is abominable, like storm, thunderbolt etc. probably can make the branch adhere to transmission line, and the branch can have the effect that electric field weakens to transmission line at this moment, probably leads to the power failure accident when serious. If the foreign matters cannot be found and cleaned in time, the normal work of the power transmission line can be influenced, and even power failure or safety accidents are caused.
At present, unmanned aerial vehicle inspection technology starts to be used in daily transmission line inspection, can effectively save inspection time, also can ensure inspection personnel's safety simultaneously. Generally, an unmanned aerial vehicle is adopted to monitor a power transmission line, a video is shot by an onboard camera of the unmanned aerial vehicle, and then an operator checks a video image shot by the unmanned aerial vehicle, so that the foreign matter condition on a power transmission cable is checked. The method relies on manual identification of foreign matters, and cannot meet the requirement of monitoring the foreign matters on the transmission line in real time under dangerous conditions. Therefore, a more accurate and convenient algorithm identification method is urgently needed to liberate manual labor force and accurately identify the type of foreign matters and the influence of the foreign matters on a power transmission line.
Disclosure of Invention
The invention provides a transmission line foreign matter intrusion identification method oriented to a dynamic feature supplementing mechanism, which trains data oriented to the dynamic feature supplementing mechanism, further improves the accuracy of system monitoring data and the processing capacity of the data, and solves the problems of low accuracy of a traditional sample and inaccurate identifiable features.
The invention provides a transmission line foreign matter intrusion identification method facing a dynamic characteristic supplementing mechanism, which comprises the following steps:
transmitting video images around the power transmission line acquired by the unmanned aerial vehicle to a power supply management platform through a wireless communication interface, and preprocessing the received video images by the power supply management platform through a Gaussian filtering algorithm to obtain processed video images;
Inputting the processed video image into an image detection model with multi-layer feature fusion, and identifying and positioning the foreign matters to obtain a video frame sequence with a foreign matter mark frame;
establishing data sets of different foreign body types of the power transmission line, wherein the data sets comprise a training data set, a testing data set and a verification data set;
Matrix capsule network classifier for extracting foreign matter type information
Performing foreign object classification on the video frame sequence with the foreign object marking frame through a matrix capsule network classifier to obtain a classified foreign object image set, and performing transformation processing to generate a new foreign object image;
Expanding a training data set by a method for generating an countermeasure network, and training a foreign matter detection model facing a dynamic feature supplementing mechanism by using the expanded training data set;
judging and determining the foreign matter type information by using a pre-trained foreign matter detection model facing the dynamic feature supplementing mechanism, and judging and determining the early warning level according to the foreign matter type;
After the early warning level is output, the system prompts the influence of the foreign matters on the transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
The above preprocessing by gaussian filtering algorithm, and the weighted average of the whole image, wherein the value of each pixel point is obtained by weighted average of itself and other pixel values in the neighborhood, and the specific method is as follows:
Each pixel in the acquired video frame image is scanned with a gaussian mask template of 3 x 3 size, and the value of the center pixel point of the gaussian mask template is replaced with the weighted average gray value of the pixels in the neighborhood determined by the gaussian mask template.
The specific method for identifying and positioning the foreign matters by inputting the video frame sequence into the image detection model with the multi-layer feature fusion comprises the following steps:
Adopting a feature extraction method based on target detection and tracking, and performing behavior time sequence feature extraction on the foreign matters in the complex scene by using an image detection model;
performing targeted training on the image detection model fused by the multi-layer features by using a self-built data set;
Processing and analyzing the extracted behavior time sequence characteristics of the foreign matters to obtain a video frame sequence with foreign matter mark frames;
The image detection model of the multi-layer feature fusion is used for dynamically optimizing the expression capacity of feature graphs by fusing different-layer feature graphs to aggregate context information and adaptively generating the output weights of the feature graphs of each layer according to the size of a training target sample in consideration of different contributions of information contained in the feature graphs of different layers to a small target detection task.
The specific method for establishing the data sets of different foreign matter types of the power transmission line comprises the following steps:
The different foreign matter type data sets comprise data sets of a plurality of different types of foreign matter samples, and the data sets are divided into a training data set, a test data set and a verification data set according to a certain proportion; the training data set is used for training the dynamic feature-oriented supplementary model, the test data set is used for checking the generalization capability of the model after model training is completed, and the verification data set is used for checking whether the model after training is completed has the fitting phenomenon or not.
The specific method for expanding the training data set by the method for generating the countermeasure network comprises the following steps:
generating an countermeasure network, and transforming the image with the foreign object mark frame through horizontal overturning, zooming and translation to generate a new image;
and splicing the foreign object part into the foreign object-free image by using an image splicing mode.
The foreign matter type information is judged by using a pre-trained foreign matter detection model facing the dynamic characteristic supplementing mechanism, and the specific method for determining the foreign matter type is as follows:
because the transmission line belongs to a small probability event under the daily condition, a large amount of resources are consumed if the tracking data of each foreign object in a complex scene are detected, on the basis of ensuring the precision, the video frame sequence with the foreign object marking frame is firstly analyzed;
And setting a plurality of detection thresholds, and classifying all foreign object types in the video according to the detection thresholds, wherein the foreign object types comprise foreign objects, foreign-like objects and foreign object-free objects.
The foreign matter detection model facing the dynamic feature supplementing mechanism is input at the current moment not only depends on the previous video frame, but also depends on the next video frame.
Compared with the prior art, the invention has the beneficial effects that:
(1) At present, the collection of foreign matter samples of a power transmission line is difficult, so that the number of samples is small, and the problems of insufficient sample diversity and few samples can be well solved by adopting a generated countermeasure network technology.
(2) The traditional algorithm network only singly adopts the traditional picture as the analyzed data, the definition of the traditional visible light picture is limited, the training network has no universality, all the identifiable foreign matter conditions cannot be completely included, and the problem can be well optimized by utilizing the preprocessed video image.
(3) The foreign matter on the transmission line is a continuous process in time, and specific foreign matter types are difficult to judge only by means of single picture information, so that misjudgment and poor detection effect are easier to occur in complex scenes. For monitoring of foreign matters, the most important is to extract robust behavior characteristics to adapt to environmental changes in complex scenes, and the second is to detect the foreign matters on the power transmission line by using the space-time characteristics in continuous video frames, context information and various judging methods. Based on the characteristics extracted and obtained by the image detection network, an improved foreign matter identification algorithm based on the fusion attention mechanism pre-judgment of the dynamic characteristic supplementing mechanism is used for integrating the time sequence information of the video so as to monitor the foreign matters. In order to reduce model parameters and improve detection efficiency, a pre-judging method based on attention mechanism behavior characteristics is used, and a foreign matter monitoring algorithm is improved.
(4) The dynamic feature supplementing mechanism oriented network well solves the problem that the traditional long-short-time memory network can only rely on the previous information to predict and the result after neglecting has response influence on the current information.
Drawings
Fig. 1 is a flow chart of a method for identifying foreign matter invasion of a power transmission line facing a dynamic feature supplementing mechanism.
Fig. 2 is a block diagram of the foreign matter monitoring feature extraction of a transmission line in a complex environment.
Fig. 3 is a schematic diagram of a matrix capsule classifier.
Fig. 4 is a general framework of a transmission line foreign matter monitoring model in a complex environment.
Fig. 5 is a network architecture diagram for a dynamic feature replenishment mechanism.
FIG. 6 is a flow chart of an early warning system.
Detailed Description
One embodiment of the present invention will be described in detail below with reference to fig. 1-6, but it should be understood that the scope of the present invention is not limited by the embodiment.
As shown in fig. 1, the method for identifying foreign matter intrusion of a power transmission line facing a dynamic feature supplementing mechanism provided by the embodiment of the invention comprises the following steps:
1. The monitoring platform is arranged on an unmanned aerial vehicle, flies close to the power transmission line through the unmanned aerial vehicle, and uses a system camera to carry out real-time video recording and monitoring to acquire video images on the real-time power transmission line; preprocessing the acquired video image data to obtain corresponding processing data; inputting the preprocessed video stream data into a multi-layer feature fused image detection model, identifying and positioning the foreign matters in the complex environment by using the multi-layer feature fused image detection model, marking the foreign matters in the video, and identifying the foreign matters; establishing different foreign matter type data sets of the power transmission lines, wherein the data sets comprise sampling sequences of the different power transmission lines and corresponding foreign matter type label values, extracting foreign matter type information from a matrix capsule network classifier, arranging the foreign matter type information into time sequence data, expanding a training data set by a method for generating an countermeasure network, and then sending the training data set into a foreign matter detection model of the next step; judging the time sequence data and the foreign matter type information by using a pre-trained network foreign matter detection model facing the dynamic feature supplementing mechanism, judging according to output, and determining an early warning level; after the early warning level is output, the system prompts the influence of the foreign matters on the transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
2. The specific method for identifying and early warning the foreign body invasion comprises the following steps:
(1) Real-time video recording and wireless communication interface for transmission:
The real-time video recording and transmission of the unmanned aerial vehicle needs to be used for a wireless communication interface, a bidirectional wireless communication link is established with a remote power supply management platform, the wireless communication interface is used for sending a control instruction by the power supply management platform, and the wireless communication interface is also used for forwarding foreign matter position information, foreign matter types and foreign matter early warning information sent by a digital signal processor to the power supply management platform, wherein the control instruction comprises power transmission line measurement height and power transmission line measurement positioning data.
(2) The collected image data is preprocessed by a Gaussian filter algorithm:
And (3) carrying out weighted average on the whole video frame image by adopting a Gaussian filtering algorithm, wherein the value of each pixel point is obtained by carrying out weighted average on the pixel point and other pixel values in the neighborhood. The specific operation is that each pixel in the obtained video frame image is scanned by a Gaussian mask template (or convolution, mask) with the size of 3 multiplied by 3, and the weighted average gray value of the pixels in the neighborhood determined by the template is used for replacing the value of the central pixel point of the template.
(3) Identifying and positioning foreign matters in a complex environment by using a multi-layer feature fused image detection model structure:
The image detection model with the multi-layer feature fusion is obtained by adding a real-time target tracking method module in the existing YOLOv s algorithm model, a foreign object marking frame detected in the detection method of the image detection model with the multi-layer feature fusion can well track a foreign object to be marked, and the image detection model with the multi-layer feature fusion has strong robustness in a complex environment.
The image detection model structure with the multi-layer feature fusion is used for identifying and positioning the foreign matters in the complex environment, the feature extraction method based on target detection and tracking is adopted, and the image detection model is used for extracting behavior time sequence features of the foreign matters in the complex scene. And performing targeted training on the image detection model fused with the multi-layer characteristics by using the self-built data set to improve the foreign object detection rate on the power transmission line, and then processing and analyzing the extracted foreign object characteristics. The method comprises the steps of designing a multi-level feature fusion layer to integrate different feeling information by considering different contributions of information contained in feature images of different levels to a small target detection task, aggregating context information by fusing the feature images of different levels, and adaptively generating output weights of the feature images of each level according to the size of a training target sample to dynamically optimize the expression capability of the feature images.
The YOLOv s network structure is mainly divided into four parts of an input end and Backbone, neck, prediction.
The Focus structure and CSP structure are used in the Backbone portion of the body. Two CSP structures were designed in YOLOv, csp1_x structure in the Backbone network, csp2_x structure in Neck. In part Neck, YOLOv s uses the same fpn+pan structure as in the prior art YOLOv s, but the difference is that the Neck structure of YOLOv s uses CSP2 structure referenced CSPnet to enhance the network feature fusion capability. GIOU Loss was used as a penalty for Bounding Box in the output section. Class probability and target score penalty are calculated using binary cross entropy and Logits penalty functions.
The real-time object tracking method uses a standard Kalman filter with constant motion and a linear observation model to describe the states (u, v, gamma, And.) as a direct observation model of the dynamic foreign object state, predicting an updated track state (u, v, gamma), wherein (u, v) represents the position of the central point of the boundary box, gamma represents the aspect ratio of the boundary box, h represents the height, and the last four parameters represent the relative speeds of the first four parameters on the corresponding image coordinates. And each track calculates the difference value between the current frame and the last successfully matched frame. And judging the change of the life cycle of the dynamic foreign matter in the track by using the max_age threshold, if the dynamic change is not generated, deleting the track, if the tracking and monitoring target fails to match with the dynamic track of the existing foreign matter, initializing the tracking and monitoring target as a new track. The state of the new path when being initialized is an undetermined state, and the undetermined state can be converted into the determined state only if the condition that three continuous frames all conform to the preset running track of the dynamic object is met. If the tracking and monitoring target in an undetermined state does not match with foreign object detection in the initial track set, the tracking and monitoring target is changed to a deleted state, namely, is deleted from the track set.
The specific implementation flow of the image detection model with the multi-layer feature fusion is to add the least filling to the original image by using a self-adaptive method to the video image preprocessed by the Gaussian filter algorithm, uniformly scale the video image to a standard size and then input the video image into a detection network. And detecting and positioning foreign matters around the transmission line, which are objects to be detected, in the video image by using YOLOv s algorithm, and after the foreign matters are positioned, carrying out target tracking on the dynamic foreign matters by using a real-time target tracking method in order to grasp continuous behavior characteristics of the dynamic foreign matters, thereby obtaining a motion track of the target dynamic foreign matters, and being used for subsequent feature extraction and foreign matter type discrimination.
(4) Extracting foreign matter type information by using a matrix capsule network classifier:
The matrix capsule network is used as a classifier of the target detection frame to judge the relation among the different types of foreign matter constituent features, so as to obtain a more accurate foreign matter type classification result. . In order to accurately identify the same type of foreign object image under different conditions such as different gestures, angles, illumination, etc., a large amount of training data is required to encompass all these differences. (5) Augmenting training data sets by generating a countermeasure network
For transmission line faults, data collection is very difficult, sample diversity is stronger, and the number is smaller, so in order to prevent the fault identification model from being over fitted and enhance the generalization capability of the fault identification model, a training data set needs to be expanded by using a method for generating an countermeasure network.
The method comprises the steps of generating an countermeasure network, transforming an image with a foreign object mark frame through a horizontal overturning method, a zooming method, a translation method and the like to generate a new image, splicing a foreign object part into a foreign object part-free image by using an image splicing method, and aiming at obtaining a more accurate recognition result under the training of a small number of samples. The practical line erection mode and the flying angle of the unmanned aerial vehicle adjacent to the power transmission line are various, so that the image operation is significant for data expansion of power transmission components such as insulators, wires and the like.
The generation of the antagonism network consists of two important parts: a Generator (abbreviated as G) and a discriminator (Discriminator, abbreviated as D). In a specific training process, the generated model simulates generated data based on a hidden random vector z to be G (z) by inputting image data with a foreign object marking frame, x is real foreign object image data collected by the unmanned aerial vehicle, and the relation between the two is written into a form of a formula, namely x=G (z). The input parameters of the judgment model are x, the probability that x is the image data of the real foreign matters is output by D (x), the image data of x and G (z) are respectively input into the judgment model to carry out two-class prediction, and finally, the two-class cross entropy loss is used for updating the parameters of the judgment model; and then fixing the judgment model to optimize the generated model, and for the generated model, in order to deceive the judgment model as much as possible, namely, to make the judgment model judge the generated 'fake' foreign object image as real foreign object image data as much as possible, optimizing the generated model by taking the judgment probability of the generated model as the target as the maximum. If the output D (x) probability is 1, it represents that 100% of the output foreign object image data is real data, and the output is 0, it represents data that is not possible to be real.
In the training optimization stage, on one hand, the objective function is maximized so that the prediction probability D (x) of the true data sampling sample x approaches 1 as much as possible and the prediction probability D (G (z)) of the generated sample G (z) approaches 0; on the other hand, the object function is to be minimized for the generated model, and log D (x) is independent of the generated model, so that the latter term is primarily minimized at this time, so that the generated model generation sample makes the judgment model prediction probability D (G (z)) approach 1.
(6) Foreign object detection model using dynamic feature-oriented replenishment mechanism:
The improved foreign matter identification algorithm for the pre-judgment of the dynamic characteristic supplementing mechanism processes time sequence information between continuous frames through the bidirectional long-short-time memory algorithm, so that the foreign matter on the power transmission line is identified and monitored. Firstly analyzing behavior feature time sequence information obtained by the image detection network model structure with the multi-layer feature fusion in the step (3), then setting a coarse detection threshold value, classifying all foreign object types in a video, directly distinguishing easily distinguished non-foreign object images, and sending foreign object images and foreign object-like images which are difficult to distinguish into a foreign object detection model of a bidirectional long and short time memory network based on an attention mechanism for detection and judgment, thereby reducing the computational complexity and improving the detection efficiency.
The dynamic feature supplementing mechanism well solves the problem that the traditional long and short time memory network can only rely on the previous information to predict the subsequent result, and the ignored result also has response influence on the current information.
The network facing the dynamic feature supplementing mechanism comprises an input layer, a forward propagation layer, a reverse propagation layer and an output layer, wherein the forward propagation layer and the reverse propagation layer are connected with the output layer together, forward calculation is carried out on the forward propagation layer from time 1 to time t, and the output of the forward propagation layer at each time is obtained and stored. And (3) reversely calculating one time along the time t to the time 1 at the back propagation layer to obtain and store the output of the back propagation layer at each time, and combining the results of the corresponding outputs of the forward propagation layer and the back propagation layer at each time to obtain a final output. And taking the average value of the two vectors corresponding to the time as an output characteristic vector, and inputting the vector into an attention mechanism to learn the network weight.
(7) The early warning system is mainly used for sending abnormal early warning information and recovering operation and maintenance feedback after being judged by the expert analysis system. Different prompts are made according to different danger levels by abnormal early warning, and if a popup window prompt occurs in dangerous situations and an alarm sound occurs, only the prompt is flashed in other situations; abnormal early warning information can appear on the click-in interface, and the abnormal early warning information comprises abnormal information (such as geographical area and tower pole information, terminal equipment data, abnormal terminal equipment number, abnormal time and the like), foreign matter information (such as probability of occurrence of foreign matters and foreign matter type), early warning level (classified into dangerous, general and temporary no influence), dispatch maintenance information (such as dispatch maintenance personnel number and maintenance condition) and operation and maintenance feedback (such as whether foreign matters exist, foreign matter type, whether maintenance and maintenance description and the like).
The static memory is used for pre-storing a reference image template of various foreign matters and a pre-storing a foreign matter type alarm comparison table, wherein the reference image template of each foreign matter is an image obtained by shooting each reference foreign matter in advance, the foreign matter type alarm comparison table takes the foreign matter type as an index, the alarm grade of each foreign matter type is stored, the higher the alarm grade is, the greater the damage of the foreign matters of the corresponding type to a power transmission line is, the static memory is also used for pre-storing an alarm grade threshold value, and the various foreign matter types in the foreign matter type alarm comparison table pre-stored in the static memory comprise plastic bags, balloons, kites, branches and the like;
S100: the unmanned aerial vehicle flies close to the power transmission line, and a system camera is used for carrying out real-time video recording and monitoring to obtain video images on the real-time power transmission line;
S200: inputting the preprocessed video stream data into an image detection algorithm with multi-layer feature fusion, identifying and positioning foreign matters in a complex environment by using a target detection algorithm, marking the foreign matters in the video, and identifying the foreign matters;
S300: establishing different foreign matter type data sets of the power transmission lines, wherein the data sets comprise sampling sequences of the different power transmission lines and corresponding foreign matter type label values, extracting foreign matter type information from a matrix capsule network classifier, and sorting the foreign matter type information into time sequence data;
s400: expanding the training data set by a method for generating an countermeasure network, and then sending the training data set into a foreign matter detection model of the next step;
S500: and judging the time sequence data and the foreign matter type information by using a pre-trained bidirectional long-short-time memory network foreign matter detection model based on an attention mechanism, judging according to output, and determining an early warning level.
S600: after the early warning level is output, the system prompts the influence of the foreign matters on the transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
2. The most important parameter of the gaussian filter template is the standard deviation delta of the gaussian distribution. It represents the degree of dispersion of the data, if delta is smaller, the larger the center coefficient of the generated template is, and the smaller the surrounding coefficient is, the less obvious the smoothing effect of the image is; when delta is larger, the difference of the coefficients of the generated templates is not large, the coefficients are close to the average template, and the smoothing effect on the image is obvious. What the gaussian filtering first does is to find the gaussian kernel of the image: assuming that the center point coordinates are (0, 0), taking the 8 points nearest to it, and setting δ=1.5, a gaussian kernel with a blur radius of 1 can be obtained. At this time, it is also ensured that these nine points are added up to 1 (characteristic of gaussian template), so that the final gaussian template is obtained by dividing all 9 values by the weight of these 9 points.
The two-dimensional Gaussian function is in the form of a formula (1), discrete points are formed on an image, the function is discretized to obtain a formula (2), the smoothing effect of Gaussian filtering depends on delta, the larger the delta is, the closer the generated template center coefficient is to the surrounding neighborhood coefficients, the better the smoothing effect is, the smaller the delta is, the larger the center coefficient is, the smaller the surrounding neighborhood coefficients are, and the worse the smoothing effect is.
Gaussian convolution kernel H (2k+1) × (2k+1), k being the filter kernel and δ being the variance.
For a3×3 image, let δ=1.5 and blur radius be 1, a weight matrix can be obtained, which is multiplied by 1 pair of image points in steps, and the entire image is traversed, whereby an image after gaussian filtering can be obtained.
3. The image detection method of the multi-layer feature fusion firstly inputs an original 608 x 3 image into a Focus structure, adopts slicing operation, firstly changes the original 608 x 3 image into a 304 x 12 feature map, and finally changes the original 608 x 3 image into a 304 x 32 feature map through convolution operation of 32 convolution kernels. And detecting and positioning foreign matters around the power transmission line in YOLOv s network, and finally performing target tracking detection on the dynamic foreign matters by using a real-time target tracking method.
The main flow of the real-time target tracking method is as follows: 1) Reading the position of a foreign matter detection frame of the current frame and the depth characteristics of each detection frame image block; 2) Filtering the foreign matter detection frame according to the confidence coefficient, and deleting the foreign matter detection frame and the characteristics with insufficient confidence coefficient; 3) Performing non-great inhibition on the detection frame, and eliminating a plurality of frames of one foreign object image; 4) And (3) predicting: the position of the foreign object in the current frame is predicted using kalman filtering. 4. In the matrix capsule network, let the gesture matrix output by the capsule i of the first layer be V i, require the output of the capsule j of the first layer +1st layer, multiply V i with the transformation matrix W ij with view angle invariance to obtain voting (vot) V j|i＝WijVi; wherein W ij is trained by Back-Propagation (BP), a local global relationship is represented. Voting needs to be weighted by protocol coefficients, which are clustered by Expectation-Maximization (EM) based on a gaussian mixture model (Gaussian Mixture Model, GMM). Finally, if the routing protocol (Routing Agreement) of the capsule i of the first layer converges to the capsule j of the first +1 layer, it means that the feature extracted by the capsule i is one of the components of the capsule j and is clustered into the same cluster together with the other capsules of the first layer that are part of the feature of the capsule j. The matrix capsule classifier is schematically shown in fig. 3.
5. In the specific training process of generating the countermeasure network, the generator and the discriminator train alternately: firstly, fixing a generator, using the generator to simulate G (z) as a negative sample based on a hidden random vector z, sampling to obtain a positive sample x from real data, inputting the positive and negative samples to a discriminator for performing classification prediction, and updating parameters of the discriminator by using the classification cross entropy loss; then, the arbiter optimizing generator is fixed, and in order to cheat the arbiter as much as possible, i.e. make the arbiter judge the generated false sample as a positive sample as much as possible, the generator is generally considered to optimize with the aim of maximizing the discrimination probability of the generated sample.
Generating a cost function in the countermeasure network, for a arbiter D, half of the input samples are from real data, half are from a generator, the cost function is written as J (D) J { (D) } J (D), then the two classes of cross entropy loss of D can be expressed as:
Wherein E represents a desired probability, x-Pdatax represents that x satisfies the P_ { data } distribution,
The sum of cross entropy losses of the real data and the generated data is respectively corresponding to, so that an optimized objective function V is obtained as follows:
Then in the training optimization stage, on one hand, the arbiter maximizes the objective function so that the prediction probability D (x) of the true data sampling sample x approaches 1 and the prediction probability D (G (z)) of the generated sample G (z) approaches 0; on the other hand, to minimize the generator, the generator is to minimize the objective function, and log D (x) is one term independent of the generator, so that the latter term is primarily minimized at this time, so that the generator generates samples to make the discriminant prediction probability D (G (z)) approach 1.
6. The algorithm for the dynamic feature supplementing mechanism well solves the problem that the traditional long-short-time memory network can only rely on the previous information to predict the subsequent result, and the neglected subsequent result also has response influence on the current information, and is realized through three structures: the input door, the forget door and the output door, wherein the forget door comprises a forward propagation layer and a backward propagation layer. The forward propagation layer and the backward propagation layer are commonly connected to the output layer, and include 6 shared weights g 1-g6. And forward calculation is carried out from time 1 to time t on the forward propagation layer, and the output of the forward hidden layer at each time is obtained and stored. And reversely calculating one time along the time t to the time 1 at the back propagation layer to obtain and store the output of the backward hidden layer at each time. The final output is obtained by combining the results of the corresponding outputs at each time instant of the forward propagation layer and the backward propagation layer, and the mathematical expression is as follows:
In the above formula, the water content of the water-soluble polymer, Is the bias of the bidirectional long and short time memory network, o 't and o' t are the results of processing the feature vectors output by the VGG layer at the corresponding time by two memory units, and tanh is the activation function. As shown in formula (9), the average value of two vectors corresponding to time is taken as an output feature vector o t, and the vector is input into an attention mechanism to learn the network weight.
The attention mechanism facing dynamic feature supplement is similar to the brain signal processing mechanism special for human vision, wherein important features are highlighted, and the whole network model can show better performance by calculating the weight of the output feature vector of the bidirectional long-short-time memory network in different time steps. The calculation formula (10) is as follows:
Wherein the α t parameter in formula (10) is the softmax value at the x t position, calculation formula (11) is as follows
In order to verify the effect of different parameters input to a foreign matter detection model facing a dynamic feature supplementing mechanism, verification is carried out by two methods.
The first method is to send only the video of easily identified static foreign matters (such as kites or industrial wastes, which are easy to distinguish) into an improved bidirectional long-short-term memory network for detecting the types of the foreign matters, and directly judge the residual foreign matters which are difficult to identify (the foreign matters with smaller volumes and difficult to judge or are dynamic) and the images without the foreign matters as non-foreign matters.
The second method is to send foreign matters in various forms (easily identifiable static foreign matters and difficult-to-identify foreign matters) into an improved two-way long short-time memory network for foreign matter type detection, and only the image without foreign matters is directly judged as non-foreign matters. By comparing the two methods, a method which is more suitable in terms of accuracy and speed is selected as the final result.
The invention provides a transmission line foreign matter invasion recognition method oriented to a dynamic characteristic supplementing mechanism, which belongs to the technical field of transmission line recognition and comprises the steps of utilizing a monitoring model to monitor foreign matters in video images shot and processed by an unmanned aerial vehicle, extracting a characteristic model of key parts of the transmission line and foreign matter phenomena of the key parts and using a classifier to judge the key parts, so as to realize intelligent recognition and diagnosis early warning of phenomena such as irregular icing of a transmission line insulator or a transmission line, external damage, growth and contact line of trees under the line, dancing and adhesion of the transmission line, defect of an insulator string, strand breakage and strand breakage of the transmission line, surface defects of the transmission line and the like under a complex environment background. The main research content comprises a region candidate network design combined with priori scale information of the power transmission component, a classifier design aiming at a multi-posture small sample, a fine-granularity foreign matter identification network design, data augmentation and data set construction. The power transmission line image inspection can effectively detect foreign matters, eliminate hidden danger in time, avoid the damage of the foreign matters to the normal operation of the power transmission line, and well solve the problems of high cost, easy omission, low efficiency and the like of the traditional manual inspection method.
According to the invention, the monitoring model is adopted to replace manual identification of foreign matters, so that the problem of missing identification can be well avoided, and the labor cost is reduced. In addition, due to the processing speed of the computer, a large number of power transmission line images can be rapidly and accurately identified by using the monitoring model, and the foreign matter monitoring efficiency is improved.
The described embodiments of the invention are only some, but not all, embodiments of the invention. All other embodiments, which can be made by a person of ordinary skill in the art based on the examples of the invention without any inventive effort, are within the scope of the invention.
Claims (6)
1. A transmission line foreign matter intrusion identification method facing a dynamic characteristic supplementing mechanism is characterized by comprising the following steps:
transmitting video images around the power transmission line acquired by the unmanned aerial vehicle to a power supply management platform through a wireless communication interface, and preprocessing the received video images by the power supply management platform through a Gaussian filtering algorithm to obtain processed video images;
Inputting the processed video image into an image detection model with multi-layer feature fusion, and identifying and positioning the foreign matters to obtain a video frame sequence with a foreign matter mark frame; the specific method for identifying and positioning the foreign matters comprises the following steps: adopting a feature extraction method based on target detection and tracking, and performing behavior time sequence feature extraction on the foreign matters in the complex scene by using an image detection model; performing targeted training on the image detection model fused by the multi-layer features by using a self-built data set; processing and analyzing the extracted behavior time sequence characteristics of the foreign matters to obtain a video frame sequence with foreign matter mark frames; the image detection model of the multi-layer feature fusion considers that the contribution of information contained in feature images of different layers to a small target detection task is different, and dynamically optimizes the expression capacity of the feature images by fusing the feature images of different layers to aggregate context information, adaptively generating the output weights of the feature images of each layer according to the size of a training target sample;
the image detection model with the multi-layer feature fusion is obtained by adding a real-time target tracking method module to the existing YOLOv s algorithm model; the specific implementation flow of the image detection model with the multi-layer feature fusion is as follows: adding the least filling to the original image by using a self-adaptive method to the video image preprocessed by the Gaussian filter algorithm, uniformly scaling to a standard size, inputting the video image into a detection network, detecting and positioning foreign matters around an object to be detected in the video image, namely a transmission line by using a YOLOv s algorithm, and after the foreign matters are positioned, carrying out target tracking on the dynamic foreign matters by using a real-time target tracking method in order to grasp continuous behavior characteristics of the dynamic foreign matters, thereby obtaining a motion track of the target dynamic foreign matters, and being used for later characteristic extraction and foreign matter type discrimination;
The main flow of the real-time target tracking method is as follows: reading the position of a foreign matter detection frame of the current frame and the depth characteristics of each detection frame image block; filtering the foreign matter detection frame according to the confidence coefficient, and deleting the foreign matter detection frame and the characteristics with insufficient confidence coefficient; performing non-great inhibition on the detection frame, and eliminating a plurality of frames of one foreign object image; and (3) predicting: predicting the position of the foreign object in the current frame by using Kalman filtering;
establishing data sets of different foreign body types of the power transmission line, wherein the data sets comprise a training data set, a testing data set and a verification data set;
Extracting foreign matter type information by a matrix capsule network classifier;
Performing foreign object classification on the video frame sequence with the foreign object marking frame through a matrix capsule network classifier to obtain a classified foreign object image set, and performing transformation processing to generate a new foreign object image;
Expanding a training data set by a method for generating an countermeasure network, and training a foreign matter detection model facing a dynamic feature supplementing mechanism by using the expanded training data set;
Judging and determining the foreign matter type by using a pre-trained foreign matter detection model facing the dynamic feature supplementing mechanism, wherein the foreign matter type information is extracted by the matrix capsule network classifier, and judging and determining the early warning level according to the foreign matter type; the foreign matter detection model facing the dynamic feature supplementing mechanism is a bidirectional long-short-term memory network foreign matter detection model based on an attention mechanism;
After the early warning level is output, the system prompts the influence of the foreign matters on the transmission line on the line, and meanwhile, the state of the foreign matters in the video is marked.
2. The transmission line foreign matter intrusion recognition method for the dynamic feature supplementing mechanism according to claim 1, wherein the preprocessing is performed by a gaussian filtering algorithm, the weighted average is performed on the whole image, and the value of each pixel point is obtained by weighted average of the value of each pixel point and the values of other pixels in a neighborhood, and the specific method is as follows:
Each pixel in the acquired video frame image is scanned with a gaussian mask template of 3 x 3 size, and the value of the center pixel point of the gaussian mask template is replaced with the weighted average gray value of the pixels in the neighborhood determined by the gaussian mask template.
3. The method for identifying foreign matter invasion of power transmission line facing dynamic feature supplementing mechanism according to claim 1, wherein the specific method for establishing data sets of different foreign matter types of power transmission line is as follows:
The different foreign matter type data sets comprise data sets of a plurality of different types of foreign matter samples, and the data sets are divided into a training data set, a test data set and a verification data set according to a certain proportion; the training data set is used for training the dynamic feature-oriented supplementary model, the test data set is used for checking the generalization capability of the model after model training is completed, and the verification data set is used for checking whether the model after training is completed has the fitting phenomenon or not.
4. The transmission line foreign matter intrusion identification method for dynamic feature replenishment mechanism according to claim 1, wherein the specific method for expanding the training data set by the method for generating the countermeasure network is:
generating an countermeasure network, and transforming the image with the foreign object mark frame through horizontal overturning, zooming and translation to generate a new image;
and splicing the foreign object part into the foreign object-free image by using an image splicing mode.
5. The method for identifying foreign matter intrusion into a power transmission line by using a dynamic feature supplementing mechanism according to claim 1, wherein the foreign matter type information is judged by using a pre-trained foreign matter detection model by using the dynamic feature supplementing mechanism, and the specific method for determining the foreign matter type is as follows:
because the transmission line belongs to a small probability event under the daily condition, a large amount of resources are consumed if the tracking data of each foreign object in a complex scene are detected, on the basis of ensuring the precision, the video frame sequence with the foreign object marking frame is firstly analyzed;
And setting a plurality of detection thresholds, and classifying all foreign object types in the video according to the detection thresholds, wherein the foreign object types comprise foreign objects, foreign-like objects and foreign object-free objects.
6. The method for identifying foreign object invasion of power transmission line facing dynamic feature supplementing mechanism according to claim 5, wherein the foreign object detection model facing dynamic feature supplementing mechanism is input at present time depending on not only the previous video frame but also the next video frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211418023.5A CN115761537B (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211418023.5A CN115761537B (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115761537A CN115761537A (en) | 2023-03-07 |
| CN115761537B true CN115761537B (en) | 2024-06-21 |
Family
ID=85370680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211418023.5A Active CN115761537B (en) | 2022-11-14 | 2022-11-14 | Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115761537B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116563680B (en) * | 2023-05-05 | 2024-02-06 | 北京卫星信息工程研究所 | Remote sensing image feature fusion method based on Gaussian mixture model and electronic equipment |
| CN116486313B (en) * | 2023-06-25 | 2023-08-29 | 安元科技股份有限公司 | Video analysis system and method suitable for scenes |
| CN117275156B (en) * | 2023-09-13 | 2024-06-28 | 武汉卓讯互动信息科技有限公司 | Unattended chess and card room reservation sharing system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906620A (en) * | 2021-03-09 | 2021-06-04 | 唐山职业技术学院 | Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment |
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN114332744A (en) * | 2022-03-10 | 2022-04-12 | 成都诺比侃科技有限公司 | Transformer substation self-adaptive security method and system based on machine vision |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210073692A1 (en) * | 2016-06-12 | 2021-03-11 | Green Grid Inc. | Method and system for utility infrastructure condition monitoring, detection and response |
| CN112464851A (en) * | 2020-12-08 | 2021-03-09 | 国网陕西省电力公司电力科学研究院 | Smart power grid foreign matter intrusion detection method and system based on visual perception |
| CN114914843A (en) * | 2021-02-09 | 2022-08-16 | 江苏三变科技有限公司 | Automatic identification and aiming algorithm for floating foreign matters in power transmission line and laser remote removal system |
-
2022
- 2022-11-14 CN CN202211418023.5A patent/CN115761537B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906620A (en) * | 2021-03-09 | 2021-06-04 | 唐山职业技术学院 | Unmanned aerial vehicle-assisted insulator fault detection method and device and electronic equipment |
| CN113076899A (en) * | 2021-04-12 | 2021-07-06 | 华南理工大学 | High-voltage transmission line foreign matter detection method based on target tracking algorithm |
| CN114332744A (en) * | 2022-03-10 | 2022-04-12 | 成都诺比侃科技有限公司 | Transformer substation self-adaptive security method and system based on machine vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115761537A (en) | 2023-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115761537B (en) | Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism | |
| CN109977921B (en) | Method for detecting hidden danger of power transmission line | |
| CN107818571B (en) | Ship automatic tracking method and system based on deep learning network and average drifting | |
| CN106816038A (en) | A kind of inland waters abnormal behaviour ship automatic identification system and method | |
| CN106951889A (en) | Underground high risk zone moving target monitoring and management system | |
| EP2798578A2 (en) | Clustering-based object classification | |
| CN110569843B (en) | Intelligent detection and identification method for mine target | |
| CN112966589A (en) | Behavior identification method in dangerous area | |
| CN111539325A (en) | Forest fire detection method based on deep learning | |
| CN108664875A (en) | Underground belt-conveying monitoring method based on image recognition | |
| CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
| CN111860117A (en) | Human behavior recognition method based on deep learning | |
| CN115546742A (en) | Rail foreign matter identification method and system based on monocular thermal infrared camera | |
| CN115620090A (en) | Model training method, low-illumination target re-recognition method and device and terminal equipment | |
| CN114998783A (en) | Front-end equipment for video analysis of smoke, fire and personnel behaviors | |
| CN118038021A (en) | Transformer substation operation site foreign matter intrusion detection method based on improvement yolov4 | |
| CN116310922A (en) | Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium | |
| CN117423157A (en) | Mine abnormal video action understanding method combining migration learning and regional invasion | |
| CN111325708B (en) | Transmission line detection method and server | |
| CN115083229B (en) | Intelligent recognition and warning system of flight training equipment based on AI visual recognition | |
| CN116610991A (en) | Multi-mode data fusion tunnel disaster risk intelligent identification method and application thereof | |
| CN115865988A (en) | Passenger ship passenger treading event monitoring system and method utilizing mobile phone sensor network | |
| CN115205786A (en) | On-line automatic identification and alarm method for mobile phone pirate behavior | |
| CN113989930A (en) | Human body abnormal behavior detection system based on small target and motion recognition | |
| CN103177237A (en) | Video monitoring method and device based on on-line lasers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |