CN116502381A

CN116502381A - Power grid key node identification method

Info

Publication number: CN116502381A
Application number: CN202310703654.XA
Authority: CN
Inventors: 孙泽军; 张燕燕; 常新峰; 王飞飞; 王启明; 闫奥琪
Original assignee: Pingdingshan University
Current assignee: Pingdingshan University
Priority date: 2023-06-14
Filing date: 2023-06-14
Publication date: 2023-07-28

Abstract

The invention discloses a method for identifying key nodes of a power grid, which relates to the technical field of power big data processing, and comprises the following steps: the structural matrix of the network is used as input, the structural characteristics of the node and the neighbor are aggregated through the multi-layer graph neural network, meanwhile, the network node is characterized according to the service attribute of the node, and the virtual node is added to complete the sub-graphRepresentation is performed to obtain representation of actions and states, and corresponding actions are output through a full connection layer and softmax _Q Executing the optimal action and obtaining corresponding rewards and the next state, and optimizing the loss function by not iterating to finally obtain the optimal model parameters; according to the invention, the grid structure is perceived by adopting the graph neural network, the nodes are represented, the motion and the state are represented by vectors, the decision is made by a deep learning method, and the optimal motion is selected to obtain the optimal model, so that the accuracy of identifying the key nodes of the grid is improved.

Description

Power grid key node identification method

Technical Field

The invention belongs to the technical field of power big data processing, and particularly relates to a power grid key node identification method.

Background

The electric power safety is a world common problem of social stability and economic development, and a certain key node in a power grid is failed, so that cascading failure can be caused, and large-area power failure of the whole area is caused.

In order to evaluate the robustness of the power network, complex network theory is gradually introduced into the field of grid security and rapidly developed. Complex networks provide a natural and powerful way to model systems in different fields, and in the field of studying grid security with complex networks, an important topic is how to identify key nodes or lines from the grid structure. A key node in a power grid refers to a node that can affect the structure and function of the power grid to a greater extent than other nodes. For example, removing a node in the grid, causing the network to be divided into multiple fragmented networks and causing multiple network power outages, may be considered a critical node. In a network, the number of key nodes is not large, but the faults of the key nodes can rapidly spread across the whole network. The key nodes of the power system are identified, potential safety hazards existing in the structural characteristics of the power grid are analyzed, corresponding measures are taken in a targeted mode, and the method has important significance in avoiding occurrence of blackout accidents.

At present, the main methods for identifying key nodes of a power grid comprise the following methods: based on graph theory and complex network theory analysis method, the method mainly takes network topology as a research object, evaluates the importance of nodes through indexes such as betweenness, degree, feature vector and the like based on complex network theory, adopts feature vector centrality measure based on network structure to detect key nodes in the network, and only identifies the key nodes from single power grid structure index, thus having certain unilateral performance. The PageRank algorithm is applied to the detection of key nodes of the power grid, and the power flow tracking method is adopted to improve the power flow tracking method. Based on graph theory, complex network theory and information theory, an evaluation matrix is established to identify key nodes by considering node degree values, betweenness and entropy. The method is based on network structure characteristics of the power grid, and a complex network method is adopted to identify key nodes in the power grid according to a certain index or a plurality of indexes of the network structure. The method only judges the importance of the node from a single network structure of the power grid, but ignores the internet of things characteristic of the power grid, so that the detection accuracy is not high; based on the detection method of the power grid business attribute, the method analyzes and detects key nodes based on the operation parameters of the power grid, such as power flow distribution, reactance and power change of the power grid. Through researching the system trend and voltage change characteristics, a key node identification method based on a Taer entropy theory is provided. By considering different roles of power business in a power communication system and a power system and adopting an entropy weight method and a weight degradation method, the identification method of the key nodes of the power communication network is provided. The key node identification is mainly considered from the aspects of the characteristics and the service of the power grid, and the structural consideration of the network is insufficient; the key nodes of the power system are limited only by considering the network topology or the physical characteristics of the power grid, and a learner combines the two methods to study, provides the node tide transfer degree and the node position importance index, and adopts a multi-attribute decision method to detect the key nodes in the power system. A dielectric-based directed network model of an electric power system is combined with a structure controllability theory and electric characteristics of the electric power system, so that an electric cactus structure is established, and a theoretical basis is provided for evaluation indexes. The method is mainly aimed at directed weight networks. And combining the topological structure and the electrical characteristics of the power network, and providing a plurality of comprehensive multi-attribute power grid key node identification methods.

Although the traditional method achieves a certain effect on certain power networks, with the gradual expansion of large-scale and wide interconnection of the power networks, the gradual improvement of the new energy permeability of randomness and volatility leads to the increasingly complex power systems and the continuous increase of fault risks. This presents a new challenge for grid key node identification:

(1) Modern power grids are larger and larger in scale and more complex in structure, and higher requirements are put on the running speed and accuracy of the existing algorithm.

(2) The influence of various random and uncertain factors is more complicated, such as diversified, intelligent and plug-and-play load random changes, so that the network structure dynamic changes, key nodes also change, and the existing algorithm is difficult to adapt to the requirements.

(3) Most of the existing fusion methods only integrate the power grid structure and the service attribute hard, but do not mine the hidden link between the structure and the attribute, so that the identification effect is general.

Disclosure of Invention

Aiming at the problem that the importance of nodes is judged only from a single network structure of a power grid in the prior art and the characteristics of the internet of things of the power grid are ignored, the invention provides a method for identifying key nodes of the power grid, which is characterized in that the network structure is perceived through a graph neural network GCN, the motion and the state are represented by vectors, the optimal motion is selected through the decision of a DQN architecture, the gradient return is carried out, and the optimal model is obtained by continuously iterating the optimization parameters, so that the problem of low detection accuracy in the prior art is solved.

A method for identifying key nodes of a power grid comprises the following steps:

constructing a power grid key node identification model based on a graph neural network and deep reinforcement learning;

inputting a network structure matrix of a power grid and an attribute matrix of the power grid into a power grid key node identification model, and detecting and identifying key nodes in the power grid;

the construction process of the power grid key node identification model comprises the following steps:

the method comprises the steps of aggregating structural features and attribute features of nodes and neighbors in a power grid through a multi-layer graph neural network GCN to obtain low-dimensional vector representation of the nodes, and representing motion vectors; the action is removing the selected key node;

adding a virtual node in a network structure of a power grid, and representing a state through an embedded vector of the virtual node; the state is the rest subnet structure of the network after removing a certain node;

outputting the value corresponding to each action through a full connection layer and a softmax output layer;

selecting actions to be executed by adopting a greedy strategy according to the value corresponding to each action;

the environment gives out corresponding rewards and the next state according to the selected actions; the environment is an analyzed network;

and adopting a decision based on the DQN deep reinforcement learning architecture, selecting an optimal action, carrying out gradient feedback, and continuously iterating optimization parameters to obtain an optimal power grid key node identification model.

Further, the definition of the key node is as follows:

given a network g= (V, E), where V is a set of nodes, i.e. v= { V ₁ ,v ₂ …v _n }，v _i Representing a node in the network; e is a collection of edges, E being denoted as E= { E ₁ ,e ₂ …e _n }，e _k Represents an edge in the network and has e _k ＝(v _i ,v _j ) The method comprises the steps of carrying out a first treatment on the surface of the The total number of nodes and edges in the network is represented as n= |v|, m= |e|, a represents the adjacency matrix, and X represents the attribute matrix, respectively.

Further, the importance degree of the nodes is measured by removing the influence of the nodes on the network connectivity, and the key node detection aims at searching a sequenceThe following is satisfied:

wherein,,representing connectivity metrics->Representing a removed node v _i The size of the maximum connected subgraph of the post-surplus network G' is formalized as follows:

wherein G' = { g\v _i }，δ _gcc () The function is used to calculate the size of the extremely large connected subgraph for a given network.

Further, the connectivity phi adopting accumulated normalization represents that the connectivity metric index value reaches the minimum:

where N represents the total number of nodes, delta, in the network G _gcc (G) Indicating a connectivity index value, delta, of the network without removing any nodes _gcc (G\{v ₁ ,v ₂ ,…,v _i }) represents removing the node set { v } ₁ ,v ₂ ,…,v _i After } the connectivity index value of the network;

the above formula is adjusted to obtain weighted cumulative normalized connectivity, represented by Φ':

wherein σ (v) _i ) Representing node v _i The removal cost of (2); normalizing the removal cost of all the nodes to obtain

Further, the graph neural network is a graph rolling network GCN, and the working process includes:

taking the power grid adjacent matrix A and the attribute matrix X as the input of the GCN, and then obtaining states and characterization of different actions through multi-layer convolution and ReLU activation;

outputting a Q (S, A) value through the full connection layer and the softmax output layer;

and selecting Action corresponding to the Q value, and giving corresponding rewards and the next state according to the Action by the environment.

Further, the states include a subnet structure and business attributes of the power grid.

Further, the vector of Q values is expressed as:

wherein the method comprises the steps ofVector matrix transpose representing motion, Z _s And (3) representing a state vector matrix, wherein W is a weight, adding a ReLU activation function, and finally adding a softmax layer to output Q values selected by different actions.

Further, the selecting an action to be performed using a greedy strategy includes the steps of:

selecting the action corresponding to the highest Q value with the probability of 1-epsilon, wherein epsilon is expressed as:

where N is the total training number and k is the current training number. After training is completed, the testing stage directly uses the action corresponding to the highest Q value.

Further, the DQN includes an empirical playback pool, an estimation network, a target network, and an environment; wherein the empirical return visit pool is for each learned quadruple (s, a, r, s '), s representing the current state, a being the action taken, r being the reward earned, s' representing the next state.

Further, the loss function expression of the DQN is:

L(θ)＝Ε[(r+γmax _a' Q(s',a'；θ)-Q(s,a；θ)) ² ]

wherein r represents the prize in the current state, r+γmax _a' Q (s ', a'; θ) represents the return obtained by the target network, and Q (s, a; θ) represents the return obtained by the estimated network;

since the key nodes are highly correlated to the network topology of the grid, the original grid network structure remains in the embedded space, whose loss function is expressed as:

wherein, E [ (r+γmax) _a' Q(s',a'；θ)-Q(s,a；θ)) ² ]Representing the reinforcement learning loss function, i.e. L (θ), βRepresenting a loss function of the reconstruction, N is the total number of nodes of the power grid, i and j respectively represent two nodes in the power grid, and s _i,j Indicating whether two nodes have connected edges, y _i ，y _j The token vectors for nodes i and j, respectively.

The invention provides a method for identifying key nodes of a power grid, which has the following beneficial effects:

the invention constructs a power grid key node identification model based on a graph neural network and deep reinforcement learning, firstly takes a structural matrix of the network as input and an attribute matrix as input; and then aggregating the structural characteristics of the node and the neighbor through the multi-layer graph neural network GCN, and simultaneously considering the service attribute of the node to jointly characterize the network node. The method comprises the steps of representing the whole sub-graph by adding virtual nodes so as to obtain characterization of actions and states, and outputting a Q value corresponding to each action through a full connection layer and softmax; and then executing the optimal action, obtaining corresponding rewards and the next state, and finally learning the optimal model parameters by continuously iterating and optimizing the loss function. Aiming at the problems that the power grid is larger and larger in scale and the structure is more and more complex, the deep learning method is adopted to characterize and extract the characteristics of the power grid structure, the network topology structure of the power grid is mapped to a low-dimensional vector space, the constraint of edges in an adjacent matrix is eliminated, and the dimension reduction treatment is carried out on the large-scale interconnected power grid, so that the treatment efficiency is improved. And the deep neural network is utilized to fuse the power grid structure and the service attribute, so that the internal connection of the power grid structure and the service is naturally excavated. The intelligent power grid key node identification method based on deep reinforcement learning is provided, and a good power grid key node identification effect is achieved.

Drawings

FIG. 1 is a schematic diagram of a reinforcement learning interaction process in an embodiment of the invention;

FIG. 2 is a diagram of a DQN architecture in an embodiment of the invention;

FIG. 3 is a graph of cumulative connectivity metrics versus polylines for algorithms on a Powergrid network in accordance with an embodiment of the present invention;

FIG. 4 is a schematic diagram of an IEEE118 node power system topology visualization in an embodiment of the invention;

FIG. 5 is a graph of cumulative connectivity metric effects versus polyline for various algorithms over an IEEE118 network in accordance with an embodiment of the present invention;

FIG. 6 is a flow chart of key node identification for deep reinforcement learning in an embodiment of the invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.

The invention provides a method for identifying key nodes of a power grid based on deep reinforcement learning.

(1) Aiming at the problems that the power grid is larger and larger in scale and is more complex in structure, the deep learning method is adopted to characterize and extract the characteristics of the power grid structure, the network topology structure of the power grid is mapped to a low-dimensional vector space, the constraint of edges in an adjacent matrix is eliminated, and the dimension reduction treatment is carried out on the large-scale interconnected power grid, so that the treatment efficiency is improved.

(2) And the deep neural network is utilized to fuse the power grid structure and the service attribute, so that the internal connection of the power grid structure and the service is naturally excavated.

(3) The intelligent power grid key node identification method based on the graph neural network and the deep reinforcement learning is provided, and a good power grid key node identification effect is achieved.

The embodiment of the invention provides a method for identifying key nodes of a power grid. The invention constructs a power grid key node identification model based on a graph neural network and deep reinforcement learning, wherein the model mainly comprises a GCN neural network and a DQN framework. Firstly, taking a structural matrix and an attribute matrix of a network as input; and then, through the multi-layer graph neural network GCN, the structural features and the attribute features of the node and the neighbor are aggregated to obtain a low-dimensional vector representation of the node, and the low-dimensional vector representation is used for representing the motion vector. Representing the whole sub-graph by adding virtual nodes so as to obtain the representation of actions and states, and outputting the Q value (Q value, namely the Q value in a depth Q network is the 'quality' of an action in a given state) corresponding to each action through a full connection layer and softmax; and then executing the optimal action and obtaining corresponding rewards and the next state, and finally learning the optimal model parameters by optimizing the loss function without iteration. In addition, a key node recognition algorithm KNDDRL based on deep reinforcement learning is designed on the basis of a model, and a large number of experiments are carried out on different data sets, and the effectiveness of the method is proved by experimental results.

At present, artificial intelligence technology represented by deep learning is widely applied to different fields of natural language processing, image recognition, video processing and the like, wherein deep reinforcement learning has become one of hot spots of artificial intelligence research, and the deep reinforcement learning has been widely applied to various fields of end-to-end control, robot control, recommendation systems, natural language dialogue systems and the like. Deep reinforcement learning is a combination of deep learning and reinforcement learning, and uses deep learning to sense and reinforcement learningAnd making a decision. Reinforcement learning is a process of learning in the middle of interaction with an environment in order to achieve one goal. The basic elements of reinforcement learning mainly comprise Agent, state, action, reward, environment, and represent Agent, state, action, rewards and Environment respectively. The basic principle is that the intelligent agent makes an action according to the current state, the environment gives corresponding rewards according to the action, and simultaneously gives the next state, so that iteration is continued until the optimization target is reached. The reinforcement learning interaction process is a Markov decision process and may be performed using four tuples (S _t ,A _t ,R _t ,S _t+1 ) For, S _t Representing the current state, A _t Representing the action performed, R _t Representing the obtained returns, S _t+1 Indicating the next state obtained.

Reinforcement learning simulates animal thinking and decision processes, and reinforcement learning requires learning a strategy function or action cost function to select the best action, wherein one classical approach is the Q-learning algorithm, which creates a mapping relationship between states and actions in a tabular form, and selects the best action by looking up the Q-value table. This method is simple, but the states and actions in real life tend to be high-dimensional, and even infinite-dimensional situations may occur, which can lead to explosion of the Q-value table, and failure to maintain and find. With the deep development of deep learning, students proposed to replace action dielectric functions by deep learning techniques. The input is the state, and the output is the score of the action after training by the neural network. The deep reinforcement learning utilizes the perception capability of the deep learning to solve the modeling problem of strategies and cost functions, and then uses an error back propagation algorithm to optimize an objective function; and meanwhile, the decision capability of reinforcement learning is utilized to define problems and optimization targets. Deep reinforcement learning has been applied to industrial production as an important technology of artificial intelligence, and has achieved remarkable effects. However, the existing work is mostly to process images or languages, and less complex networks, and the invention applies deep reinforcement learning to the power network, and identifies key nodes by combining structural information and business attributes of the power network.

Relevant definition in a power grid key node identification method based on deep reinforcement learning:

(1) Key nodes:

given a network g= (V, E), where V is a set of nodes, i.e. v= { V ₁ ,v ₂ …v _n }，v _i Representing a node in the network; e is a collection of edges, E being denoted as E= { E ₁ ,e ₂ …e _n }，e _k Represents an edge in the network and has e _k ＝(v _i ,v _j ). The total number of nodes and edges in the network is represented as n= |v|, m= |e|, a represents the adjacency matrix, and X represents the attribute matrix, respectively. The key nodes generally refer to some nodes which can influence the network structure and the function to a greater extent, and different judging methods are provided for the key nodes according to different application scenes. Given the structural characteristics of the power grid, the method adopts the influence of the removed nodes on the network connectivity to measure the importance degree of the nodes. Formally defined as the number of bits in a data stream,representing connectivity metrics, the goal of key node detection is to find a sequence +.>The following is satisfied:

wherein,,representing a removed node v _i The size of the extremely large connected subgraph of the post-remaining network G'. Formalizing as follows:

wherein G' = { g\v _i }，δ _gcc () Function for calculating a given netThe size of the maximum connected subgraph of the collaterals.

(2) Cumulative connectivity:

the objective of deep reinforcement learning is to optimize the overall connectivity metric index value to be minimum, and the method adopts cumulative normalized connectivity phi representation, which is formally defined as:

where N represents the total number of nodes, delta, in the network G _gcc (G) Indicating a connectivity index value, delta, of the network without removing any nodes _gcc (G\{v ₁ ,v ₂ ,…,v _i }) represents removing the node set { v } ₁ ,v ₂ ,…,v _i After } the connectivity index value of the network. The weight of each section in different networks may be different, as may the cost of its removal. For example, in an electrical power network, each generator node and load node may differ in their power generation capacity and load, and therefore in their removal costs. Adjusting the formula (3) to obtain weighted cumulative normalized connectivity, denoted by Φ', as follows:

wherein σ (v) _i ) Representing node v _i Is not limited by the cost of removal. The invention normalizes the removal cost of all nodes so as to ensure that

The construction process of the power grid key node identification model based on deep reinforcement learning comprises the following steps:

the invention constructs a power grid deep reinforcement learning model shown in figure 1, wherein the model consists of input, an intelligent agent and an environment. The input of the model is a network structure matrix of the power grid and an attribute matrix of the power grid; the intelligent agent mainly comprises a neural network, wherein the graph neural network adopted by the invention is a graph rolling network (GCN), the State is used as the input of the GCN, then the states and the representation of different actions are obtained through multi-layer convolution and ReLU activation, and then the Q (S, A) value is output through a full-connection layer and a softmax output layer; and selecting an Action (Action) corresponding to the Q value, and giving corresponding rewards and the next state according to the Action by the environment. In this model, an Action (Action) refers to removing a selected key node, and a State (State) refers to the remaining subnet structure of the network after removing a node, which State includes the subnet structure and the business attributes of the grid. The environment refers to the network being analyzed, minimizing cumulative connectivity as in equation (4), optimizing the loss function to detect key nodes, by iterating constantly.

In the process of identifying key nodes of the model, firstly, a power grid adjacent matrix A and an attribute matrix X of a power grid are used as inputs; then aggregating the structural attributes of the node and the neighbor through the multi-layer graph neural network GCN, and simultaneously considering the service attribute of the node to characterize the action Z _a . The invention adds a virtual node, and the node and all nodes have a connecting edge, so that the node and all nodes are connected, and the embedded vector of the node is used for representing the state Z _s . The two matrices are then multiplied to represent a vector representation of Q (s, a) in the form:

wherein the method comprises the steps ofVector matrix transpose representing motion, W being a weight, Z _s And (3) representing a state vector matrix, adding a ReLU activation function, and finally adding a softmax layer to output Q values selected by different actions. And selecting actions to be executed by adopting an epsilon greedy strategy according to the Q value, randomly selecting actions with epsilon probability, and selecting actions corresponding to the highest Q value with 1-epsilon probability. The value of epsilon in the training phase is dynamically changed, and the initial value is smaller, so that the probability of exploring more action space is higher, and the learned model is learned along with continuous trainingThe model effect is better and better, a larger epsilon value is set, the action corresponding to the highest Q value is selected, and epsilon is set as follows:

The invention adopts a DQN-based deep reinforcement learning architecture, and the structure of the DQN-based deep reinforcement learning architecture is shown in figure 2. DQN corresponds to a traditional Q learning (Q-learning) +neural network, which is mainly composed of an empirical playback pool, an estimation network, a target network, and an environment. The experience playback pool is used for four groups (s, a, r, s ') learned each time, s represents the current state, a is the action taken, r is the obtained reward, s' represents the next state, after the experience playback pool is full, the old record can be deleted, new records are added, the circulation is continued, and the update is kept continuously. The training can randomly sample data from the experience playback pool in batches, so that the correlation among training samples can be disturbed, and the training is more efficient. Because no real sample label exists, the Q value calculated by the Q target network is used as a real value, the Q estimation network continuously optimizes the parameters through gradient descent to reduce the error between the parameters and the real value, and the learned parameters are updated to the Q target network after the iteration of C rounds is carried out. The loss function formalization of DQN is expressed as follows:

L(θ)＝Ε[(r+γmax _a' Q(s',a'；θ)-Q(s,a；θ)) ² ] (7)

wherein r represents the prize in the current state, and r+γmax is calculated by the formulas (2) - (4) _a' Q (s ', a'; θ) represents the return obtained by the target network, Q (s, a; θ) represents the return obtained by the estimated network, with the goal of minimizing the return from both. In the grid network, not only the minimum Q value between the estimated network and the target network, but also the original grid network structure needs to be considered, because the key nodes are highly correlated with the network topology of the grid, and the loss function is defined as follows:

wherein the first part on the right side of the equal sign represents the loss function of reinforcement learning, namely L (theta), the second part represents the loss function of graph reconstruction, N is the total number of nodes of the power grid, i and j respectively represent two nodes in the power grid, and s _i,j Indicating whether two nodes have connected edges, y _i ，y _j The token vectors for nodes i and j, respectively.

According to the constructed model, the basic flow of the network key node identification based on the graph neural network and the deep reinforcement learning is as follows:

s1, initializing a model, wherein the model comprises iteration times, experience playback pool size and various weights.

S2, a Q target network and a Q estimation network are established according to fig. 1 and 2, and the two networks have the same structure and only have different parameter updates.

And S3, generating a network by using the scaleless network BA model.

S4, training is carried out by using the generated network or the real network as the input of the model, and Q (S, a) is output.

S5, selecting an action a to be executed according to a greedy strategy of a formula (6) _t Obtain return r _t At the same time get the next state s _t+1 。

S6, sample (S) _t ,a _t ,r _t ,t,s _t-1 ) And storing the old samples into the experience pool M, and if the experience pool is full, deleting the old samples and storing the new samples.

S7, randomly extracting a group of samples of the mini-batch from the experience pool M, and performing parameter optimization by using a gradient descent method according to the figure 2 and the formula (8).

And S8, updating the target network parameters after each C round of iteration.

S9, until the maximum iteration times are reached, storing the optimal model parameters.

S10, testing by using the optimal model parameters.

The experimental verification process is as follows:

according to experimental data, two power networks (Powergird, IEEE, 118) are selected for experiments, and as shown in table 1, symbols |V|, |E|, < k >, kmax and < C > respectively represent node numbers, edge numbers, average degrees, maximum degrees and clustering coefficients.

Table 1 statistical properties of network data

The power grid (Powergird) contains information about the power grid in the western states of the united states, where the nodes represent generators, transformers or substations and the edge representatives are power lines. IEEE118 is a power network with attributes that include 118 nodes, 186 edges, and 7 dimensions of data including voltage, current, load, line type, node type, and electrical distance.

In order to widely study the performance effect of the KNDDRL method, the invention compares the KNDDRL method with a plurality of representative key node detection algorithms such as centrality, proximity centrality, feature vector centrality, webpage ranking, improved K-shell method and the like, and the comparison algorithm is simply introduced below.

Closeness Centrality (CC), also known as proximity centrality algorithm. The method considers that important nodes are closer to other nodes in the network, namely, the smaller the average value of the distances between the important nodes and other nodes is, the faster the information propagates in the network, so that the more important the nodes are. The disadvantage of this algorithm is the high time complexity.

Eigenvector Centrality (EC), feature vector centrality algorithm. The method considers that the influence of the nodes in the network is not only related to the number of neighbors connected with the nodes, but also related to the influence of the neighbors. Specifically, the feature vector centrality method determines the influence of nodes by calculating feature vectors and feature values of a network adjacency matrix. The eigenvector centrality approach works very well in most networks.

PageRank (PR), which is the most important algorithm in *** early search engines, has the advantage of having a high quality *** search effect mainly due to the algorithm. The basic idea is similar to the feature vector centrality method, namely, the influence of the WEB pages in the WEB network is determined by the quantity and quality of the WEB pages pointing to the WEB pages. The PageRank method was originally proposed for use in a directed network, and later modified versions could also be used in an undirected network.

The improved K-shell (KBKNR) comprehensively considers different influences of one-hop neighbor nodes and two-hop neighbor nodes, obtains the comprehensive degree of each node, and judges the importance of the node by combining the K-shell.

Evaluation index:

(1) SIR estimation model the present invention uses the most widely used node impact estimation method SIR model to estimate the propagation impact of nodes in the network. SIR is a disease propagation model consisting of three components, the susceptible state (S), the infected state (I) and the recovered state (R), and the propagation process of the SIR model comprises the following steps:

first, one or more seed nodes (nodes in the I state) start. The disease then propagates in the network, and the seed node infects neighboring nodes with a probability of λ. Next, the infected node recovers with a probability of β, and the node in the infected state still infects its neighbor node with a probability of λ.

Finally, the infected and recovered nodes in the network are used to calculate the propagation impact of the seed nodes.

The more important nodes are considered by the researchers, the more nodes are infected, and therefore the importance degree of the nodes is judged by detecting the propagation influence of the nodes.

(2) And (5) a connectivity index.

The key nodes refer to nodes with larger influence on the structure and the function of the network, the key nodes are removed to possibly cause the collapse of the network structure, and the connectivity index is the connectivity of the rest of the network after the removed nodes are calculated. The invention adopts the method that after removing the nodes, the maximum connected subgraph of the residual network is obtained. See formula (2) and formula (3).

Experimental effect: all experiments of the invention are run on a desktop computer, and the hardware configuration of the invention is that the CPU is Intel Core i5,3.3GHz and the memory is 16.0GB. Without loss of generality, each comparison algorithm adopts default parameters recommended by its author, the propagation probability lambda of the SIR evaluation model is 0.01, and the value of the recovery probability beta is 1.

(1) The first 10 important nodes detected by different algorithms:

as shown in table 2, on the Powergrid network, the DC method is closest to the SIR detection result, the PR and KNDDRL method detection results are consistent, and the nodes detected by the data also appear in the first 10 node sets detected by the SIR.

Table 2 different algorithms top 10 node sequences in Powergrid networks

It can be observed from the above comparison that the KNDDRL method is not completely consistent with the SIR evaluation model on all networks, because SIR is a disease propagation model, and is mainly used for evaluating information propagation, while the KNDDRL method focuses on the influence of node removal on the network, which better meets the actual requirements of the power grid network, such as that a node fails or is attacked, and what influence is caused on the power grid.

(2) Performance of different algorithms on connectivity indicators.

In order to further compare the key node detection effect of each algorithm, the invention researches the connectivity index performance obtained by different algorithms on the Powergrid data set. The accumulated connectivity metric is calculated by adopting formulas (2) - (3), no node is removed at the beginning, the connectivity index value is maximum, and the value of the connectivity index value gradually drops along with the removal of the key node. And finally, forming a broken line, wherein the area under the broken line is shown by the accumulated connectivity metric value, and the smaller the area is, the more accurate the detected key node is. As shown in fig. 3, the abscissa represents the proportion of removed nodes, wherein the nodes are arranged in descending order of importance, and the ordinate represents the connectivity metric value of the remaining network after removing the proportional nodes. It can be observed that on the Powergrid network, the method KNDDRL provided by the invention achieves the best effect, the area under the folding line is the smallest, so that the key nodes identified by the KNDDRL method are more accurate, if the nodes can be removed to break down the network more quickly, the nodes need to be more concerned and protected in the actual power grid. PR and DC methods also achieved good detection results. The key node identification effect of the EC method is worst on the network, and the identification effect of the EC method is also not ideal.

(3) Detection effect on the property network.

The above is directed to non-attribute networks, and the actual grid may have related business attributes, so the present invention selects the power system network IEEE118 with attributes, and its topology visualization is shown in fig. 4. The key node detection effect of each comparison algorithm is shown in fig. 5, and it can be observed that KNDDRL still obtains the best performance, and the CC method has better effect of a few nodes, but the overall performance is worse than that of the KNDDRL method.

Through a large number of experimental comparisons on different data sets and different indexes, the key node identification accuracy of the KND DRL method provided by the invention can be found to be higher than that of a compared representative algorithm. Because the KNDDRL method utilizes the deep reinforcement learning technology to perform characterization learning on the network, the characteristics of the node are captured, the structural characteristics of the multi-layer neighbors are gathered, and meanwhile, the attribute information can be learned.

The invention constructs a power grid key node identification model based on deep reinforcement learning, which mainly comprises a GCN neural network and a DQN framework. Firstly, taking a structural matrix and an attribute matrix of a network as input; and then aggregating the structural characteristics of the node and the neighbor through the multi-layer graph neural network GCN, and simultaneously considering the service attribute of the node to jointly characterize the network node. The method comprises the steps of representing the whole sub-graph by adding virtual nodes so as to obtain characterization of actions and states, and outputting a Q value corresponding to each action through a full connection layer and softmax; and then executing the optimal action and obtaining corresponding rewards and the next state, and finally learning the optimal model parameters by optimizing the loss function without iteration. In addition, a key node recognition algorithm KNDDRL based on deep reinforcement learning is designed on the basis of a model, and a large number of experiments are carried out on different data sets, and the effectiveness of the method is proved by experimental results.

The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims

1. The method for identifying the key nodes of the power grid is characterized by comprising the following steps of:

acquiring a network structure matrix and an attribute matrix of a power grid;

inputting a network structure matrix and an attribute matrix of the power grid into a key node identification model of the power grid, and detecting and identifying key nodes in the power grid;

the construction of the power grid key node identification model specifically comprises the following steps:

the method comprises the steps of aggregating structural features and attribute features of nodes and neighbors in a power grid through a multi-layer graph neural network GCN to obtain low-dimensional vector representation of the nodes, and representing action vectors by the low-dimensional vectors of the nodes; the action is removing the selected key node;

2. A method of identifying key nodes of a power grid according to claim 1, wherein the key nodes in the power grid are defined as:

given a network g= (V, E), where V is a set of nodes, i.e. v= { V ₁ ,v ₂ …v _n }，v _i Representing a node in the network; e is a collection of edges, E being denoted as E= { E ₁ ,e ₂ …e _n }，e _k Represents an edge in the network and has e _k ＝(v _i ,v _j ) The method comprises the steps of carrying out a first treatment on the surface of the The total number of nodes and edges in the network is represented as n= |n|, m= |e|, a represents the adjacency matrix, and X represents the attribute matrix, respectively.

3. The method for identifying key nodes of a power grid according to claim 2, further comprising: measuring the importance of a node by removing the influence of the node on network connectivity, and searching a sequence by the key node detectionThe following is satisfied:

wherein,,representing connectivity metrics->Representing a removed node v _i The size of the maximum connected subgraph of the post-remaining network G' is formally expressed as:

4. A method of grid key node identification as defined in claim 3, further comprising: the connectivity phi adopting accumulated normalization indicates that the connectivity metric index value reaches the minimum:

5. The method for identifying key nodes of a power grid according to claim 1, wherein the graph neural network is a graph roll-up network GCN, and the working steps include:

6. A method of grid critical node identification according to claim 1, characterized in that the status comprises the sub-network structure and the service properties of the grid.

7. The method for identifying key nodes of a power grid according to claim 1, wherein the vector of Q values is expressed as:

8. The method for identifying key nodes of a power grid according to claim 1, wherein the greedy strategy epsilon is adopted to select the action to be executed, and the action corresponding to the highest Q value is selected with the probability of 1-epsilon, and epsilon is expressed as:

where N is the total training number and k is the current training number.

9. The method of claim 1, wherein the DQN includes an empirical playback pool, an estimated network, a target network, and an environment; wherein the empirical return visit pool is for each learned quadruple (s, a, r, s '), s representing the current state, a being the action taken, r being the reward earned, s' representing the next state.

10. The method for identifying key nodes of a power grid according to claim 9, wherein the loss function expression of the DQN is:

L(θ)＝Ε[(r+γmax _a' Q(s',a'；θ)-Q(s,a；θ)) ² ]

since the key nodes are highly correlated to the network topology of the grid, the original grid network structure needs to be preserved in the embedded space, and its loss function is expressed as:

wherein, E [ (r+γmax) _a' Q(s',a'；θ)-Q(s,a；θ)) ² ]Representing the loss function of reinforcement learning, i.e., L (theta), representing the reconstructed loss function of the graph, wherein N is the total number of nodes of the power grid, i and j respectively represent two nodes in the power grid, and s _i,j Indicating whether two nodes have connected edges, y _i ，y _j The token vectors for nodes i and j, respectively. />