CN111209864A - Target identification method for power equipment - Google Patents

Target identification method for power equipment Download PDF

Info

Publication number
CN111209864A
CN111209864A CN202010012244.7A CN202010012244A CN111209864A CN 111209864 A CN111209864 A CN 111209864A CN 202010012244 A CN202010012244 A CN 202010012244A CN 111209864 A CN111209864 A CN 111209864A
Authority
CN
China
Prior art keywords
power equipment
equipment
nodes
information
probability
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010012244.7A
Other languages
Chinese (zh)
Other versions
CN111209864B (en
Inventor
刘亚东
严英杰
李喆
钱勇
罗林根
汪可友
宋辉
盛戈皞
江秀臣
熊思衡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN202010012244.7A priority Critical patent/CN111209864B/en
Publication of CN111209864A publication Critical patent/CN111209864A/en
Application granted granted Critical
Publication of CN111209864B publication Critical patent/CN111209864B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/29—Graphical models, e.g. Bayesian networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the technical field of image recognition, and discloses a power equipment target recognition method which comprises the steps of establishing a data set containing a plurality of power equipment images, marking power equipment contained in each power equipment image, and training and learning a neural network and a Bayesian network by taking the power equipment as input; recognizing the images of the electric power equipment to be recognized by utilizing the trained neural network, and outputting a plurality of recognition results; and screening the recognition result of the neural network by using the trained Bayesian network, and screening out the most accurate recognition result corresponding to the image of the electrical equipment to be recognized. The whole process is simple in structure, fast in calculation and high in accuracy.

Description

Target identification method for power equipment
Technical Field
The invention relates to the technical field of image recognition, in particular to a power equipment target recognition method.
Background
The inspection of high-voltage equipment is an effective means for ensuring reliable power supply of the equipment, and along with the implementation of the policy of setting posts and the development of a power grid, the inspection work of the equipment is increasingly prominent as long as the inspection work is shown in the following steps: the inspection robot has the advantages of few and many people, difficult implementation of an inspection system, low inspection quality and urgent need to improve the usability of the inspection robot.
If advanced detection and diagnosis technologies can be adopted, unmanned inspection of key equipment of the transformer substation and automatic judgment of potential danger abnormal conditions are realized, inspection burdens of staff of the front-line team can be greatly relieved, the construction of the front-line team is facilitated to be changed from a working type to a management type, and the operation and maintenance level of the current transformer substation is greatly improved.
The operation state and the operation environment condition of the power transmission and transformation equipment are mastered, and the key problem of operation and maintenance management of the power transformation equipment is to find the hidden trouble of the operation of the power grid equipment in time. Because the number of equipment is many, the operational environment is complicated, the limitation of state monitoring technology, how in time, accurate grasp equipment running state still needs to solve urgently, the main problem of detection means commonly used at present shows:
(1) the existing on-line monitoring system has low recognition and low use value. The traditional inspection and live detection method cannot monitor the whole operation process of equipment due to a fixed detection test period, and equipment hidden dangers cannot be found timely.
(2) The currently applied transformer substation robot polling mainly collects field images and infrared thermal image data, but the robot polling cannot carry out all-dimensional real-time monitoring for 24 hours, cannot achieve full coverage in time and space, mainly stays at a field data collection and simple threshold judgment stage at present, and is lack of an analysis technology which can only automatically and accurately process and diagnose detection data.
In the field of power equipment target identification, scholars at home and abroad have already conducted certain research and obtained certain achievements.
In foreign countries, learners use infrared and visible light images to achieve the detection target of the electric wire, firstly obtain video stream images from infrared and visible light sensors, then use image processing to judge whether the electric wire is in fault, and display the fault in real time by synthesizing the image stream. Besides the transmission line, the direction of the insulator is tried by scholars, and the scholars propose a video-based online detection method which uses information such as templates, histograms and edges, considers the situation of different brightness and is used for detecting the inclination degree and the snow cover of the insulator.
In China, six years ago, people indicate that the development of an image recognition technology provides technical guarantee for realizing a smart grid, can solve the problem of huge calculation amount faced by on-line monitoring of power equipment, and has very important significance for improving a power system. Some documents extract invariant moment of a target shape of power equipment and use the invariant moment as a feature vector after preprocessing a transformer substation picture acquired by a camera, identify the type of the power equipment by using a support vector machine, and judge whether a fault occurs by comparing a device operation picture with a picture in a database. There is also a new way to locate and identify the electric equipment label, so as to read out the kind and parameters of the electric equipment. For example, in order to reduce the running time of an image recognition algorithm, a template matching algorithm is used, a certain part of the power equipment is taken as a template, the whole picture is subjected to traversal matching, if a transformer is recognized, only an insulator needs to be found in the picture, and the operation efficiency is greatly improved.
Disclosure of Invention
The invention provides a target identification method for electric equipment, which solves the problems that the shooting angle of pictures or videos of the electric equipment needs to be known in advance, otherwise the expected effect cannot be achieved, the calculation efficiency is low and the like in the conventional method.
The invention can be realized by the following technical scheme:
a target identification method for power equipment comprises the following steps:
step one, establishing a data set containing a plurality of power equipment images, marking the power equipment contained in each power equipment image, and training and learning a neural network and a Bayesian network by taking the power equipment as input;
secondly, recognizing the images of the electric power equipment to be recognized by using the trained neural network, and outputting a plurality of recognition results;
and step three, screening the recognition result of the neural network by using the trained Bayesian network, and screening out the most accurate recognition result corresponding to the image of the electrical equipment to be recognized.
Further, the power equipment contained in each power equipment image is labeled by using a labeling tool, the node and directed edge information of the corresponding Bayesian network are calculated according to the labeling information, the corresponding directed acyclic graph is constructed, the Bayesian network is trained by using a plurality of power equipment images of the data set, and the corresponding conditional probability table is obtained.
Further, setting a threshold value, screening all recognition results of the neural network, recalculating the node and directed edge information of the corresponding Bayesian network according to a plurality of screened recognition results, constructing a corresponding directed acyclic graph, calculating a corresponding joint probability by combining a conditional probability table obtained by training, and selecting a recognition result corresponding to the highest joint probability as the most accurate recognition result of the image of the electric power equipment to be recognized.
Further, each node is set as a device, each directed edge is set as the relation between the devices, the direction of the directed edge is pointed to the node with lower priority by the node with higher priority,
the probability distribution of the whole directed acyclic graph comprises three parts, wherein the first part is set as the probability Ps of another device on the premise that some devices exist, and the conditional probability expression of the probability Ps is as follows:
Ps＝P(p2|p1，p4)P(p3|p2，p4)...
the second part is set to set the self probability PaThat is, the relative area s of each device is related to the kind of the device itself, and the conditional probability expression is as follows:
Pa＝P(s1|p1)P(s2|p2)...
the third part is set as the spatial relation probability PrThat is, the spatial relationship between the devices is related to the types of the two devices, and the conditional probability expression is as follows:
Pr＝P(R1，2|p1，p2)...P(R9，10|p9，p10)
the joint probability P distribution expression is as follows:
P＝PsPaPr＝P(p2|p1，p4)P(p3|p2，p4)...P(s1|p1)P(s2|p2)...P(R1，2|p1，p2)...P(R9，10|p9，p10)。
further, the labeling information comprises the outline, the type and the image quality information of the power equipment, and is stored in a json format file, and the outline information is set to be a polygon formed by connecting more points on the outer edge of each equipment;
the nodes comprise 10 parameters which are respectively set as a device type, a device relative area, a horizontal coordinate centrioidx of a device centroid, a vertical coordinate centrioidy of the device centroid, a horizontal coordinate xs of each marking point on the device outline, a vertical coordinate ys of each marking point on the device outline, a device serial number, a child node child of the device, all pixel points mask contained in the device and a priority privilledge of the device,
the equipment type, the abscissa xs of each marking point on the equipment outline, the ordinate ys of each marking point on the equipment outline, the equipment serial number and the priority priviledge of the equipment are directly obtained from the marking information, and the rest parameters are obtained by calculation according to the outline information of the equipment;
the directed edges comprise weight parameters of edges, whether the edges exist between two nodes is judged firstly, then the weight parameters of the edges are calculated, and the directed edges are obtained by calculation according to the contour information of the equipment.
Further, the relative area of the device is set as the area of a polygon corresponding to the node, and is obtained by calculation by adopting a vector cross product method;
setting the abscissa centrioidx and the ordinate centrioidy of the centroid of the device as the average value of the abscissas and the average value of the ordinates of all the vertexes of the polygon;
all pixel point masks contained in the equipment are obtained by painting RGB (red, green, blue) in an area where a polygon is located, so that a three-dimensional pixel matrix in an RGB format is obtained, and any one of the three-dimensional pixel matrix can be obtained;
judging whether mask information corresponding to the two nodes is contained or not by adopting an interaction over Union index, namely when the area of the shared part of the mask information corresponding to the two nodes is divided by the area of the combined two mask information and is larger than a threshold value, judging that the two nodes have inclusion relation;
whether an edge exists between two nodes is judged by judging whether an intersection exists between the external rectangles of the five pixel points which are outwards expanded by the polygon corresponding to the two nodes, the weight of the edge is obtained by contrasting the position matrix with the relative position of the outline of each node corresponding to the centroid of the polygon, and the centroid is set as the average value of all vertex coordinates of the polygon.
Further, the recognition result comprises a rectangular frame parameter rois, a category parameter class _ ids, a score parameter scores and a masks parameter, the threshold is set as a score threshold, only recognition results with score parameters larger than the score threshold are screened out,
recalculating nodes corresponding to the Bayesian network, wherein the areas of the nodes are obtained by calculating mask parameters; setting the abscissa centrioidx and the ordinate centrioidy of the equipment centroid as the average value of the abscissas and the average value of the ordinates of all the vertexes of the rectangular frame parameter rois;
recalculating the directed edges corresponding to the Bayesian network, judging whether the rectangular frame parameters rois corresponding to the two nodes have intersection, and acquiring the weight of the edges by comparing the relative position of the centroid of the rectangular frame parameters rois corresponding to each node with the position matrix, wherein the centroid is set as the average value of all vertex coordinates of the rectangular frame parameters rois.
Further, the types of the devices are set to tower, line, insulator, nest, pole.
The beneficial technical effects of the invention are as follows:
according to the method, the data set containing a plurality of images of the electrical equipment is established, the electrical equipment contained in the data set is labeled, on the basis, the neural network and the Bayesian network are trained, a plurality of recognition results of the neural network are screened by using the trained Bayesian network, and the most accurate recognition result corresponding to the image of the electrical equipment to be recognized is screened out. During the period, the Bayesian network is required to be constructed twice, during training, construction is carried out on the basis of the labeling information of each power equipment image, and during screening, construction is carried out on the basis of each recognition result of the neural network. The method successfully improves the mAP value of the MaskR-CNN output result from 0.699 to 0.819, improves the mAP value by 12 percent and has obvious effect; as the Bayesian network and the Mask R-CNN are trained simultaneously, the training time of the Bayesian network is shorter than that of the Mask R-CNN, the verification of the Mask R-CNN output result by the Bayesian network is hardly time-consuming, and the total program running time is not increased.
Drawings
FIG. 1 is a schematic overall flow diagram of the present invention;
FIG. 2 is a schematic diagram of an example of the labeling of an image using a labeling tool according to the present invention;
FIG. 3 is a diagram illustrating four cases of calculating the polygon area by using the vector product according to the present invention;
fig. 4 is three schematic diagrams illustrating screening of recognition results of a neural network by using a bayesian network in the present invention, where a denotes a first example, ①② denotes a recognition result processed by using the neural network, ② denotes a recognition result screened by using the bayesian network, b denotes a second example, ③④ denotes a recognition result processed by using the neural network, ④ denotes a recognition result screened by using the bayesian network, which denotes a first example, C denotes a third example, ⑤⑥ denotes a recognition result processed by using the neural network, and ⑥ denotes a recognition result screened by using the bayesian network.
Detailed Description
The following detailed description of the preferred embodiments will be made with reference to the accompanying drawings.
On one hand, although a certain achievement exists in the field of power equipment target identification, the limitation is large, and for this reason, the R-CNN area convolution neural network is considered to be used, and the shooting angle of the picture is also taken as a parameter for learning. On the other hand, although R-CNN is developed rapidly, there is a big problem in application in the field of power equipment, and for this problem, we consider to introduce a concept of human-like learning, and let a machine gradually understand things from recognizing basic components of things to recognizing the relationship between these basic components. The essence of humanoid learning is the PGM probabilistic graphical model, which is a model that uses probability to characterize performance. The probabilistic graphical model combines the knowledge of the capability and the graph to represent the joint probability distribution of variables associated with a model using the graph.
In summary, the invention combines Mask R-CNN with a probabilistic graph model, and provides a power equipment target identification method, which is mainly to establish a data set containing a plurality of power equipment images, label the power equipment contained in the data set, train a neural network and a bayesian network on the basis of the data set, screen a plurality of identification results of the neural network by using the trained bayesian network, and screen out the most accurate identification result corresponding to the power equipment image to be identified. During the period, the Bayesian network is required to be constructed twice, during training, construction is carried out on the basis of the labeling information of each power equipment image, and during screening, construction is carried out on the basis of each recognition result of the neural network. As shown in fig. 1, the method specifically comprises the following steps:
step one, establishing a data set containing a plurality of power equipment images, labeling the power equipment contained in each power equipment image, and training and learning the neural network and the Bayesian network by taking the labeled power equipment as input.
With respect to neural networks:
Mask-RCNN as the latest R-CNN network, draws the advantages of all ancestors and makes further improvement on their basis. The Mask R-CNN uses RoI Align to replace RoI firing, specifically, the original rounding operation is removed, the floating point number obtained by calculation is reserved, and bilinear interpolation is used to complete the operation of the pixel. Therefore, accurate alignment at the pixel level is achieved, and the method runs in the example segmentation field.
Since we use a bayesian network at the top layer, which means we need to construct a network structure by the position relationship between devices, the pixel-level segmentation is particularly important. If the output result only contains a frame containing the object, the frame generally contains other devices besides the identified device, so that a large error is introduced when the Bayesian network is constructed, and the problem can be solved well by using the mask.
Because the field of the electric power equipment lacks an open source data set available for machine learning, the data set containing 330 pictures of the electric power equipment is established by using a VIA VGG Image indicator tag tool before applying MaskR-CNN. The label of each picture is shown in fig. 2 and is obtained by shooting by the substation inspection robot, and the label information comprises the outline, the type and the picture quality of the equipment.
After the pictures are marked, a json-format file is generated, the file contains all the marking information, the outline information of the equipment in the marking information is represented by polygons, namely more points are taken on the outer edge of each equipment to be connected into polygons to approximate the appearance of the equipment. If the parts belong to different parts of the same equipment, the parts are respectively marked when being marked, but the types of the equipment are the same.
According to the information of the data set, the type of the equipment is set as tower, line, insulator, bird nest, pole, so as to establish a Bayesian network in the following.
Each deep convolutional neural network is flexible and variable within a certain range, and the variation is caused by different network parameters. After debugging, we finally select the network parameters as follows:
learning rate of 0.02
Iteration number epochs is 30
Divide all samples into how many steps per epoch 100
The threshold value detection min confidence of RoI confidence coefficient is 0.9
Each GPU processes picture number images per GPU 2
As the labeling tool VIA is used, boundary-crossing labeling points are inevitably generated in the labeling process, and two parts which belong to the same equipment and are separated due to occlusion are also labeled separately, so that data preprocessing is required.
The data preprocessing is divided into three parts, one part is used for solving the problem of boundary crossing of the marked points, and all points exceeding four boundaries of the picture are pulled back to the boundaries, so that the problem is solved well; the second part is to merge the parts which belong to the same device but are separately labeled, and the devices with the same label name are merged into a complete device to generate a mask; and the other part is to randomly divide the data set into two parts, namely a training set and a test set, wherein the training set is used for training a Mask R-CNN network, and the test set is used for verifying the actual effect of the network.
Regarding bayesian networks:
the probability map model PGM is a model for describing the actual situation, the core of the model is conditional probability, the prior knowledge is essentially utilized to establish the association constraint relation between random variables, and the purpose of conveniently obtaining the conditional probability is finally achieved.
The specific process comprises the following steps: the method comprises the steps of marking the power equipment contained in each power equipment image by using a labeling tool, calculating nodes and directed edge information of a corresponding Bayesian network according to marking information, constructing a corresponding directed acyclic graph, training the Bayesian network by using a plurality of power equipment images of a data set, and obtaining a corresponding conditional probability table.
Each node is set as a device, each directed edge is set as the relation between the devices, and the direction of the directed edge is pointed to the node with lower priority by the node with higher priority.
The probability distribution of the whole directed acyclic graph comprises three parts, wherein the first part is set as the probability Ps of another device on the premise that some devices exist, and the conditional probability expression of the probability Ps is as follows:
Ps＝P(p2|p1，p4)P(p3|p2，p4)...
the second part is set to set the self probability PaThat is, the relative area s of each device is related to the kind of the device itself, and the conditional probability expression is as follows:
Pa＝P(s1|p1)P(s2|p2)...
the third part is set as the spatial relation probability PrThat is, the spatial relationship between the devices is related to the types of the two devices, and the conditional probability expression is as follows:
Pr＝P(R1，2|p1，p2)...P(R9，10|p9，p10)
the joint probability P distribution expression is as follows:
P＝PsPaPr＝P(p2|p1，p4)P(p3|p2，p4)...P(s1|p1)P(s2|p2)...P(R1，2|p1，p2)...P(R9，10|p9，p10)。
each node of the bayesian network contains 10 parameters, respectively: type, area, centroidx, centroidy, xs, ys, number, children, mask, priviledge. The corresponding Chinese meaning is: the device type, the relative area of the device, the abscissa of the device centroid, the ordinate of the device centroid, the abscissa of each labeled point on the device profile, the ordinate of each labeled point on the device profile, the device serial number, the child nodes of the device, all the pixel points contained in the device, and the priority of the device.
Each edge of the bayesian network contains 2 parameters, which are the direction of the edge and the weight of the edge. The weight value represents the relative position between the devices, and the corresponding relationship between the weight value and the relative position is shown in the following table:
Figure RE-GDA0002417228640000101
in order to simplify the representation, I make point modification on the spatial position matrix, and change the point modification from top left to bottom right in sequence into: 1,2,3,4,5,6,7,8,9.
The construction of the Bayesian network is divided into two parts: importing a data set and constructing a probability graph, which comprises the following steps:
import of data set
From the above explanation, we can know that the format of the data set is json. The json file contains the label information of all pictures.
Since we recruit several volunteer annotations, we finally get several json documents, which we need to merge; in addition, the VIA labeling tool also exports the unlabelled picture information to the json file, so we also need to delete the unlabelled picture information. Then, the merged json file can be read in, and the construction of the nodes of the Bayesian network is completed.
Each node contains 10 parameters: type, area, centroidx, centroidy, xs, ys, number, children, mask, priviledge. The values of type, xs, ys, number, and priviledge can be directly obtained from a json file, and the following describes the method for acquiring the remaining parameters.
First is the area calculation. The outline of each marking device is a polygon, and the area of the polygon is obtained through the coordinates of the vertices of the polygon. I use the method of vector product, that is, the area of each triangle is solved by the vector product, and then the final area is obtained by summation. Meaning of the vector product:
Figure RE-GDA0002417228640000111
when using the vector product, the coordinates of three points need to be used, except that two vertexes of the polygon are selected each time, another reference point is needed, for the convenience of calculation, i choose to use the origin as the reference point, consider the concave polygon and the convex polygon, and need to consider four cases as shown in fig. 3 during calculation:
in the first diagram, the origin is inside the polygon, so that when calculating the cross product vector, the directions of the four cross products are all clockwise, and the area is the sum of the absolute values of the four triangles, in the second diagram, the origin is outside the polygon, and when calculating the cross product vector, the direction of △ OAB is opposite to the directions of the other three triangles, so that the area of the whole triangle is equal to the sum of the areas of the other three triangles minus the area of △ OAB, and the other two triangles have the same reason as the first triangle and the second triangle.
From this, we can see that we can get the area of the polygon by using the vector cross product method, no matter the position of the reference point is inside or outside the polygon.
Second, the centroid is acquired. Since we know the coordinates of each vertex of the polygon, the coordinates of the centroid is equal to the average of the coordinates of all vertices, the corresponding abscissa is the abscissa centroidx of the device centroid, and the ordinate is the ordinate centroidy of the device centroid.
And then, acquiring a child node child of the equipment, firstly converting the labeled polygon of the node into a matrix with the same output format as that of the Mask R-CNN, and then operating the Mask. Because the mask is a boolean second-order matrix, it is mainly determined whether mask information corresponding to two nodes is included by using an Intersection over Union index, that is, when the area of the common part of the mask information corresponding to the two nodes divided by the area of the merged mask information is greater than a threshold, the two nodes, that is, the device, have an inclusion relationship.
Besides, we need to transform the node labeled polygon into the same matrix as the Mask R-CNN output format. Here, the area where the polygon is located is colored (1,1,1) using the fillPoly () function in the cv2 library, so that a three-dimensional pixel matrix in the RGB format is obtained, and mask information can be obtained by taking any one of the three-dimensional pixel matrix.
Each edge contains two parameters, which are the direction of the edge and the weight of the edge.
First, we need to determine whether there is an edge, i.e., whether two nodes are adjacent.
In order to eliminate errors caused by labeling, whether the adjacent rectangles are adjacent or not is judged by using the circumscribed rectangles of the five pixel points which are outwards expanded from each node polygon, and the judgment of the adjacent problems is converted into the judgment of whether the two rectangles have intersection or not.
After the information of whether the edge exists is obtained, the weight value of the edge and the direction of the edge are obtained.
The direction of the edge is directed by the higher priority node, i.e. the device, to the lower priority node by the previously defined priority information. The weight of the edge is obtained by comparing the relative position of the polygon centroid of each node outline with the position matrix.
And step two, recognizing the images of the electric power equipment to be recognized by utilizing the trained neural network, and outputting a plurality of recognition results.
Firstly, the output results of the neural network are not screened in the neural network, but all the identification results directly output by the neural network are reserved, and then the neural network is screened by setting a threshold according to the actual situation of the power equipment.
The identification result comprises a rectangular frame parameter rois, a category parameter class _ ids, a score parameter scores and a masks parameter, the threshold is set as a score threshold, and only the identification result of which the score parameter is greater than the score threshold is screened out for secondary screening of a subsequent Bayesian network.
And step three, screening the recognition result of the neural network by using the trained Bayesian network, and screening out the most accurate recognition result corresponding to the image of the electrical equipment to be recognized.
The specific process comprises the following steps: and recalculating the nodes and the directed side information of the corresponding Bayesian network according to the screened multiple identification results, constructing a corresponding directed acyclic graph, calculating corresponding joint probabilities by combining with a conditional probability table obtained by training, and selecting the identification result corresponding to the highest joint probability as the most accurate identification result of the image of the electrical equipment to be identified.
Recalculating nodes corresponding to the bayesian network:
of the 10 parameters of the node, the values of type, number, and priviledge can be directly obtained, and the remaining parameters are calculated as follows:
firstly, the area is calculated, the Mask information in the recognition result is a three-dimensional Boolean matrix with specification of (1024,1024, instance _ numbers), the last parameter represents the number of instances recognized by the Mask R-CNN, namely the number of masks, therefore, the Mask for each instance can be represented as masks [: i ], the specification is a two-dimensional Boolean matrix, and the calculation of the area can be directly realized by sum (Mask).
Second, the centroid is acquired. Since the Mask R-CNN output result includes rectangular box information rois, that is, the rectangular box including the instance, the coordinates of the centroid are equal to the average of the coordinates of the four vertices of the rectangle, and the abscissa and the ordinate thereof correspond to the abscissa centrioidx and the ordinate centrioidy of the centroid of the device, respectively.
Next, an acquisition for the device child node children follows, similar to the method described above.
Each edge contains 2 parameters, which are the direction of the edge and the weight of the edge.
First, we need to determine whether there is an edge, i.e., whether two nodes are adjacent. Different from the Bayesian network established during training, the rectangular frame parameter rois in the Mask R-CNN recognition result is directly used as an external rectangle, and other steps are similar.
The experiment is based on a GTX 1080-TI video card, and the Mask R-CNN part is carried out under a tensoflow framework. As mentioned above, there are a total of 330 pictures in our dataset, with the training data comprising 264 pictures and the validation data comprising 66 pictures.
We use the deep convolutional neural network weight import obtained by training the pictures in the 264 training sets, then detect the pictures in the test set, and select three types of comparison representatives from the final output results, as shown in fig. 4:
and after screening through the Bayesian network, selecting the identification result on the right side as the identification result most prepared corresponding to the image of the power equipment to be identified.
Although many people in the field of electric power equipment target identification propose a set of own methods, the methods are not universal and have large limitations, and the shooting angles of known pictures are required; although R-CNN is developed rapidly, due to the lack of mature data sets in the field of power equipment and the unique characteristics of the large class of power equipment, the application of the current deep convolutional neural network in the field of power equipment has a great problem.
Aiming at the problems, the invention mainly provides a brand-new power equipment target identification method, wherein the bottom layer design is a Mask R-CNN deep convolution neural network, the top layer design is a Bayesian network, and the main idea is to introduce the results output by the Mask R-CNN into the Bayesian network for optimization, so that better results are obtained.
Experiments show that the method successfully improves the mAP value of the Mask R-CNN output result from 0.699 to 0.819, improves the mAP value by 12 percent and has obvious effect; as the Bayesian network and the Mask R-CNN are trained simultaneously, the training time of the Bayesian network is shorter than that of the Mask R-CNN, the verification of the Mask R-CNN output result by the Bayesian network is hardly time-consuming, and the total program running time is not increased.
In addition, a large amount of data is needed for support in general deep convolutional neural networks, and only 330 labeled pictures are used as data sets in the experiments, so that better results are obtained.
Generally, the method for identifying the target of the power equipment provided by the invention can well optimize the output result of the bottom layer without increasing the time complexity, and is more satisfactory.
Although specific embodiments of the present invention have been described above, it will be appreciated by those skilled in the art that these are merely examples and that many variations or modifications may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is therefore defined by the appended claims.

Claims (8)

1. A target identification method for electric power equipment is characterized by comprising the following steps:
step one, establishing a data set containing a plurality of power equipment images, marking the power equipment contained in each power equipment image, and training and learning a neural network and a Bayesian network by taking the power equipment as input;
secondly, recognizing the images of the electric power equipment to be recognized by using the trained neural network, and outputting a plurality of recognition results;
and step three, screening the recognition result of the neural network by using the trained Bayesian network, and screening out the most accurate recognition result corresponding to the image of the electrical equipment to be recognized.
2. The machine learning-based power equipment target identification method according to claim 1, characterized in that: the method comprises the steps of marking the power equipment contained in each power equipment image by using a labeling tool, calculating nodes and directed edge information of a corresponding Bayesian network according to marking information, constructing a corresponding directed acyclic graph, training the Bayesian network by using a plurality of power equipment images of a data set, and obtaining a corresponding conditional probability table.
3. The power equipment target identification method according to claim 2, characterized in that: setting a threshold value, screening all recognition results of the neural network, recalculating the node and directed edge information of the corresponding Bayesian network according to a plurality of screened recognition results, constructing a corresponding directed acyclic graph, calculating the corresponding joint probability by combining with a conditional probability table obtained by training, and selecting the recognition result corresponding to the highest joint probability as the most accurate recognition result of the image of the electrical equipment to be recognized.
4. The power equipment target identification method according to claim 3, characterized in that: each of the nodes is set as a device, each of the directed edges is set as a relation between the devices, the direction of the directed edges is pointed to a node with lower priority by a node with higher priority,
the probability distribution of the whole directed acyclic graph comprises three parts, wherein the first part is set as the probability Ps of another device on the premise that some devices exist, and the conditional probability expression of the probability Ps is as follows:
Ps＝P(p2|p1，p4)P(p3|p2，p4)...
the second part is set to set the self probability PaThat is, the relative area s of each device is related to the kind of the device itself, and the conditional probability expression is as follows:
Pa＝P(s1|p1)P(s2|p2)...
the third part is set as the spatial relation probability PrThat is, the spatial relationship between the devices is related to the types of the two devices, and the conditional probability expression is as follows：
Pr＝P(R1，2|p1，p2)...P(R9，10|p9，p10)
The joint probability P distribution expression is as follows:
P＝PsPaPr＝P(p2|p1，p4)P(p3|p2，p4)...P(s1|p1)P(s2|p2)...P(R1，2|p1，p2)...P(R9，10|p9，p10)。
5. the power equipment target identification method according to claim 4, characterized in that: the labeling information comprises the outline, the type and the image quality information of the power equipment and is stored in a json format file, and the outline information is set to be a polygon formed by connecting more points on the outer edge of each equipment;
the nodes comprise 10 parameters which are respectively set as a device type, a device relative area, a horizontal coordinate centrioidx of a device centroid, a vertical coordinate centrioidy of the device centroid, a horizontal coordinate xs of each marking point on the device outline, a vertical coordinate ys of each marking point on the device outline, a device serial number, a child node child of the device, all pixel points mask contained in the device and a priority privilledge of the device,
the equipment type, the abscissa xs of each marking point on the equipment outline, the ordinate ys of each marking point on the equipment outline, the equipment serial number and the priority priviledge of the equipment are directly obtained from the marking information, and the rest parameters are obtained by calculation according to the outline information of the equipment;
the directed edges comprise weight parameters of edges, whether the edges exist between two nodes is judged firstly, then the weight parameters of the edges are calculated, and the directed edges are obtained by calculation according to the contour information of the equipment.
6. The power equipment target identification method according to claim 5, characterized in that: the relative area of the equipment is set as the area of a polygon corresponding to the node and is obtained by calculation by adopting a vector cross product method;
setting the abscissa centrioidx and the ordinate centrioidy of the centroid of the device as the average value of the abscissas and the average value of the ordinates of all the vertexes of the polygon;
all pixel point masks contained in the equipment are obtained by painting RGB (red, green, blue) in an area where a polygon is located, so that a three-dimensional pixel matrix in an RGB format is obtained, and any one of the three-dimensional pixel matrix can be obtained;
judging whether mask information corresponding to the two nodes is contained or not by adopting an interaction over Union index, namely when the area of the shared part of the mask information corresponding to the two nodes is divided by the area of the combined two mask information and is larger than a threshold value, judging that the two nodes have inclusion relation;
whether an edge exists between two nodes is judged by judging whether an intersection exists between the external rectangles of the five pixel points which are outwards expanded by the polygon corresponding to the two nodes, the weight of the edge is obtained by contrasting the position matrix with the relative position of the outline of each node corresponding to the centroid of the polygon, and the centroid is set as the average value of all vertex coordinates of the polygon.
7. The power equipment target identification method according to claim 5, characterized in that: the recognition results comprise a rectangular frame parameter rois, a class parameter class _ ids, a score parameter scores and a masks parameter, the threshold is set as a score threshold, only recognition results with the score parameters larger than the score threshold are screened out,
recalculating nodes corresponding to the Bayesian network, wherein the areas of the nodes are obtained by calculating mask parameters; setting the abscissa centrioidx and the ordinate centrioidy of the equipment centroid as the average value of the abscissas and the average value of the ordinates of all the vertexes of the rectangular frame parameter rois;
recalculating the directed edges corresponding to the Bayesian network, judging whether the rectangular frame parameters rois corresponding to the two nodes have intersection, and acquiring the weight of the edges by comparing the relative position of the centroid of the rectangular frame parameters rois corresponding to each node with the position matrix, wherein the centroid is set as the average value of all vertex coordinates of the rectangular frame parameters rois.
8. The power equipment target identification method according to claim 4, characterized in that: the types of the equipment are set as tower, line, insulator, bird nest and pole.
CN202010012244.7A 2020-01-07 2020-01-07 Power equipment target identification method Active CN111209864B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010012244.7A CN111209864B (en) 2020-01-07 2020-01-07 Power equipment target identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010012244.7A CN111209864B (en) 2020-01-07 2020-01-07 Power equipment target identification method

Publications (2)

Publication Number Publication Date
CN111209864A true CN111209864A (en) 2020-05-29
CN111209864B CN111209864B (en) 2023-05-26

Family

ID=70786624

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010012244.7A Active CN111209864B (en) 2020-01-07 2020-01-07 Power equipment target identification method

Country Status (1)

Country Link
CN (1) CN111209864B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102296A (en) * 2020-09-17 2020-12-18 上海交通大学 Power equipment target identification method based on human concept
CN112418202A (en) * 2021-01-11 2021-02-26 上海恒能泰企业管理有限公司 MaskRCNN-based power transformation equipment abnormity identification and positioning method and system
CN112884705A (en) * 2021-01-06 2021-06-01 西北工业大学 Two-dimensional material sample position visualization method
CN114325219A (en) * 2021-12-03 2022-04-12 陕西省地方电力(集团)有限公司延安供电分公司 Method for detecting and identifying running state of distribution network equipment
CN114485694A (en) * 2020-11-13 2022-05-13 元平台公司 System and method for automatically detecting building coverage area

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106408015A (en) * 2016-09-13 2017-02-15 电子科技大学成都研究院 Road fork identification and depth estimation method based on convolutional neural network
CN108197569A (en) * 2017-12-29 2018-06-22 驭势科技（北京）有限公司 Obstacle recognition method, device, computer storage media and electronic equipment
WO2019000653A1 (en) * 2017-06-30 2019-01-03 清华大学深圳研究生院 Image target identification method and apparatus
CN109598241A (en) * 2018-12-05 2019-04-09 武汉大学 Satellite image marine vessel recognition methods based on Faster R-CNN
CN110245587A (en) * 2019-05-29 2019-09-17 西安交通大学 A kind of remote sensing image object detection method based on Bayes's transfer learning
CN110378408A (en) * 2019-07-12 2019-10-25 台州宏创电力集团有限公司 Power equipment image-recognizing method and device based on transfer learning and neural network

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106408015A (en) * 2016-09-13 2017-02-15 电子科技大学成都研究院 Road fork identification and depth estimation method based on convolutional neural network
WO2019000653A1 (en) * 2017-06-30 2019-01-03 清华大学深圳研究生院 Image target identification method and apparatus
CN108197569A (en) * 2017-12-29 2018-06-22 驭势科技（北京）有限公司 Obstacle recognition method, device, computer storage media and electronic equipment
CN109598241A (en) * 2018-12-05 2019-04-09 武汉大学 Satellite image marine vessel recognition methods based on Faster R-CNN
CN110245587A (en) * 2019-05-29 2019-09-17 西安交通大学 A kind of remote sensing image object detection method based on Bayes's transfer learning
CN110378408A (en) * 2019-07-12 2019-10-25 台州宏创电力集团有限公司 Power equipment image-recognizing method and device based on transfer learning and neural network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李迎春;范有臣: "一种创新性激光图像三维目标识别算法", 《激光与红外》 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102296A (en) * 2020-09-17 2020-12-18 上海交通大学 Power equipment target identification method based on human concept
CN114485694A (en) * 2020-11-13 2022-05-13 元平台公司 System and method for automatically detecting building coverage area
CN112884705A (en) * 2021-01-06 2021-06-01 西北工业大学 Two-dimensional material sample position visualization method
CN112884705B (en) * 2021-01-06 2024-05-14 西北工业大学 Two-dimensional material sample position visualization method
CN112418202A (en) * 2021-01-11 2021-02-26 上海恒能泰企业管理有限公司 MaskRCNN-based power transformation equipment abnormity identification and positioning method and system
CN114325219A (en) * 2021-12-03 2022-04-12 陕西省地方电力(集团)有限公司延安供电分公司 Method for detecting and identifying running state of distribution network equipment

Also Published As

Publication number Publication date
CN111209864B (en) 2023-05-26

Similar Documents

Publication Publication Date Title
CN111209864B (en) Power equipment target identification method
CN108428229B (en) Lung texture recognition method based on appearance and geometric features extracted by deep neural network
CN111402224B (en) Target identification method for power equipment
CN111160440B (en) Deep learning-based safety helmet wearing detection method and device
US11810348B2 (en) Method for identifying power equipment targets based on human-level concept learning
CN112633535A (en) Photovoltaic power station intelligent inspection method and system based on unmanned aerial vehicle image
CN109614488A (en) Distribution network live line work condition distinguishing method based on text classification and image recognition
CN110133443B (en) Power transmission line component detection method, system and device based on parallel vision
CN110675421B (en) Depth image collaborative segmentation method based on few labeling frames
CN113989944B (en) Operation action recognition method, device and storage medium
CN110838105A (en) Business process model image identification and reconstruction method
CN113553977A (en) Improved YOLO V5-based safety helmet detection method and system
CN111553176B (en) Wireless transmission checking method and system suitable for wiring of substation screen cabinet
CN116052222A (en) Cattle face recognition method for naturally collecting cattle face image
CN114120141A (en) All-weather remote sensing monitoring automatic analysis method and system thereof
Jiao et al. A deep learning model for small-size defective components detection in power transmission tower
CN117456358A (en) Method for detecting plant diseases and insect pests based on YOLOv5 neural network
CN116452850A (en) Road ponding area identification method based on data mining and deep learning
CN114299394A (en) Intelligent interpretation method for remote sensing image
CN114037873A (en) Transformer substation secondary circuit protection pressing plate detection system and method based on artificial intelligence
CN114219753A (en) Power equipment surface defect detection method based on deep learning and terminal
CN113191320A (en) Food safety index monitoring method based on deep learning
KR20210037199A (en) Apparatus for dividing, tagging an image and for detecting defect of facilities using the same
CN113837178A (en) Deep learning-based automatic positioning and unified segmentation method for meter of transformer substation
CN113869122A (en) Distribution network engineering reinforced control method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant