CN117040809B - Method for generating defense strategy of industrial information physical system based on Bayesian random game - Google Patents
Method for generating defense strategy of industrial information physical system based on Bayesian random game Download PDFInfo
- Publication number
- CN117040809B CN117040809B CN202310894809.2A CN202310894809A CN117040809B CN 117040809 B CN117040809 B CN 117040809B CN 202310894809 A CN202310894809 A CN 202310894809A CN 117040809 B CN117040809 B CN 117040809B
- Authority
- CN
- China
- Prior art keywords
- attacker
- game
- state
- bayesian
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 84
- 230000007123 defense Effects 0.000 title claims abstract description 59
- 238000006243 chemical reaction Methods 0.000 claims abstract description 8
- 238000005259 measurement Methods 0.000 claims abstract description 7
- 238000010276 construction Methods 0.000 claims abstract description 3
- 230000009471 action Effects 0.000 claims description 57
- 230000006870 function Effects 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 33
- 238000009826 distribution Methods 0.000 claims description 17
- 238000011084 recovery Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000013139 quantization Methods 0.000 claims description 12
- 230000003993 interaction Effects 0.000 claims description 11
- 230000007704 transition Effects 0.000 claims description 11
- 230000008901 benefit Effects 0.000 claims description 5
- 230000001010 compromised effect Effects 0.000 claims description 3
- 238000010924 continuous production Methods 0.000 claims description 3
- 230000008439 repair process Effects 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 206010001488 Aggression Diseases 0.000 description 1
- 208000001613 Gambling Diseases 0.000 description 1
- 230000016571 aggressive behavior Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008260 defense mechanism Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008846 dynamic interplay Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000009533 lab test Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1433—Vulnerability analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general
- H04L63/205—Network architectures or network communication protocols for network security for managing network security; network security policies in general involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a method for generating an industrial information physical system defense strategy based on Bayesian random game. The invention establishes a Bayesian random game model to describe the characteristics of attack and defense competition under the environment of an industrial information physical system, and generates a defensive reaction strategy under the condition of no complete network attack information; adopting a unified measurement and balance framework to standardize the configuration of game parameters, and ensuring the consistency and compatibility of the information physical model construction; adopting a Harsanyi conversion method to convert incomplete information about an attacker utility function into imperfect information about the attacker type, and ensuring the existence of Bayesian Nash equilibrium strategy; the invention also provides a multi-agent Bayesian Q-learning algorithm to learn under the dynamic network environment and unknown game parameters to obtain a Nash equilibrium defense strategy, so as to help a decision maker to formulate a robust and effective security defense strategy.
Description
Technical Field
The invention relates to a network security defense method of an industrial information physical system, belongs to the field of industrial control network space security, and particularly relates to a method for generating a defense strategy of the industrial information physical system based on Bayesian random game.
Background
The rapid development and application of information and communication technology has drastically changed the traditional industrial control system into a networked industrial information physical system. This digital revolution has greatly improved the production efficiency and manufacturing capacity of the industry. However, advanced information and communication technologies also expose industrial network physical systems to more and more attacks from network space (unauthorized access, data leakage, system destruction, and other malicious activities), which can compromise the confidentiality, integrity, and availability of critical infrastructure and industrial processes. Therefore, making effective network security countermeasures is of great importance.
Security measures in the field of industrial information physical systems today rely mainly on passive defense mechanisms (such as intrusion detection systems and firewalls). In view of the durability and complexity of the network, it is critical to develop active decision methods that can effectively predict and deal with emerging threats in real time. Game theory, as a formal theoretical framework for research strategic interactions, provides a systematic approach to help system administrators formulate timely and proactive defense strategies. By analyzing the interactive interactions between the attacker and defender, the game theory allows the administrator to learn the incentives, motivations, and potential strategies of the adversaries, thereby developing a defensive strategy that maximizes the security outcome while taking into account the associated costs and benefits.
Disclosure of Invention
The invention aims to solve the problem of active decision response in the network security defense of the industrial information physical system. Aiming at the defect that the existing game theory is applied to the research of the field of safety protection of the industrial information physical system, the method for generating the defense strategy of the industrial information physical system based on Bayesian random game is provided. The method for generating the defense strategy of the industrial information physical system based on the Bayesian random game has guiding significance on network security protection of key industrial infrastructure.
The aim of the invention can be achieved by the following technical scheme:
the invention firstly provides a method for generating an industrial information physical system defense strategy based on Bayesian random game, which comprises the following steps:
1) Constructing a Bayesian random game model, wherein the Bayesian random game model describes an attack-defense interaction process under the condition that network attack information is incompletely known in an industrial information physical system;
2) The industrial information physical system comprises an information layer and a physical layer, and a unified quantization method based on time is utilized to construct game utility functions of the information layer and the physical layer, which are targeted by both the attack and the defense in the Bayesian random game model, by taking time as a quantization index;
3) Converting incomplete information of an attacker-to-attacker game utility function into imperfect information about attacker types by using a Harsanyi conversion method to obtain an imperfect information Bayesian random game model participated by various attacker types, and ensuring the existence of Bayesian Nash equilibrium defense strategies;
4) Based on the Bayesian random game model obtained in the step 3), learning by utilizing a multi-agent Bayesian Q-learning algorithm (MABQL) under the conditions of a dynamic network environment and unknown game parameters to obtain a Nash equilibrium defense strategy, and obtaining the optimal defense strategy of the industrial information physical system.
According to a preferred scheme of the invention, the construction process of the Bayesian random game model in the step 1) specifically comprises the following steps:
abstracting attack and defense interaction in an industrial information physical system into a non-zero Bayesian random game model, under the model, dispersing a continuous process of the attack and defense interaction into a plurality of game states by utilizing a random game theory, wherein each game state corresponds to a specific security state of an information layer or a physical layer in the system; the attacker and the defender have corresponding game utility functions in the information layer or physical layer game state; the current state may transition to the next gaming state based on a probability distribution determined jointly by the actions of the defender and the attacker;
considering that an defender does not have complete information about an attacker in an actual attack and defense scene, converting each game state into a static Bayesian game by utilizing a Bayesian game theory, wherein a utility function is unilateral information;
finally, the bayesian stochastic game model is defined as an 11-tuple:
G=<N,S,Θ,P A ,A,D,T,O,π A ,π D ,U A ,U D >
wherein the elements are defined as: n= { Attacker, defender } is the set of game participants; s= { S 1 ,s 2 ,...,s K -a set of game states, wherein each state represents a security state of the network; Θ= { θ 1 ,θ 2 ,...,θ I -representing a set of attacker types;a set of probability distributions representing defenders versus aggressor types, whereRepresentative state s k Distribution of the lower part; a= { a 1 ,A 2 ,...,A K The action set of an attackerWherein A is k ={a 1 ,a 2 ,...,a n Represents state s k A lower action set; d= { D 1 ,D 2 ,...,D K The actions set of the defender, where D k ={d 1 ,d 2 ,...,d n Represents state s k A lower action set; />Is the policy set of the attacker; />Is a policy set of defenders; u (U) A And U D Utility functions of the offender and defender, respectively.
According to a preferred scheme of the invention, in step 2), the game utility functions of the attack and defense parties aiming at the information layer and the physical layer in the Bayesian random game model are constructed by taking time as a quantization index, and specifically:
2.1 Defining attack time T a The method comprises the steps of carrying out a first treatment on the surface of the It represents the time required for an attacker to perform an attack action on a particular target, which quantifies the time from the vulnerability scanning to the attacker's successful exploitation of the vulnerability;
2.2 Defining a defence time T d The method comprises the steps of carrying out a first treatment on the surface of the Which represents the time required for the defender to perform the defending action, which quantifies the time from recognition of the attack to completion of the defending action;
2.3 Defining recovery time T r The method comprises the steps of carrying out a first treatment on the surface of the Representing the time required for the attacked device to resume its normal operating state after the attack; t for unified quantization of recovery time of information layer and physical layer r Is defined as:
T r =T r,c +T r,p
T r,c representing the time required to restore the compromised device to a normal state by bug fixes or switches to the standby device; t (T) r,p Representing the time required to restore the physical layer control process to a normal state; when the attacked device is an information layer host device, T r =T r,c The method comprises the steps of carrying out a first treatment on the surface of the When the attacked device is a physical layer sensorOr actuator device, then T r =T r,c +T r,p The method comprises the steps of carrying out a first treatment on the surface of the This marks that the defender needs not only to repair the damaged equipment, but also to resume the control process;
2.4 For calculating T r,p The physical layer control process is modeled as:
x k is the system state at time k, y k Is the measurement state at time k, u' k Represents the attack signal at the moment k, A represents the transfer matrix, B represents the input control matrix, C represents the output observation matrix, B a Represents an attack matrix, w k Representing process noise, v k Representing measurement noise; t (T) r,p Indicating the overall recovery time for the state variable to deviate from the normal range to recover to the normal range when the physical layer control process is attacked.
According to a preferred embodiment of the present invention, the Harsanyi conversion method in step 3) is:
3.1 Introducing a third party participant, and converting incomplete information of the defender on the utility function of the attacker into imperfect information on the action of the third party participant;
3.2 After the attack and defense parties select respective actions, the third party participant selects the next game state according to the transition probability, and the new state can be observed by the attacker and the defender;
3.3 Third party participants select the type of attacker based on the probability distribution of the defender to the type of attacker, and can only be observed by the attacker;
3.4 After the third party participant completes the selection, the attacker and defender both select actions according to the respective strategies and then obtain the respective rewards;
3.5 Repeating the processes from 3.1) to 3.4) until the game is finished, and obtaining the imperfect information Bayesian random game model with participation of various attacker types. According to a preferred embodiment of the present invention, the utility function of the attacker and defender in step 3) is defined as:
R A (s t ,a,d,θ i ) And R is D (s t ,a,d,θ i ) Instant rewards representing aggressors and defenders,and->Representing a future reward;
the immediate rewards in the utility function are defined as:
R A (s t ,a,d,θ i )=ε(s t ,a,d)T r (a,θ i )-T a (θ i )
R D (s t ,a,d,θ i )=-ε(s t ,a,d)T r (a,θ i )-T d
T r (a,θ i ) Is attacker theta i Recovery time required by the system after successful attack; t (T) a (θ i ) Representing attacker theta i The time required to perform the attack action; epsilon(s) t A, d) represents the probability of success of the attack under action (a, d); t (T) d Representing the time required to implement a defensive action;
for future awards in the utility function, O (s k |s t A, d) represents the state s in the case of an attacker and defender taking action (a, d), respectively t Transition to state s k And O(s) k |s t ,a,d)=ε k (a,d);Andrepresenting the desired utility value in a state, also referred to as a state value, can be found by the following formula:
bayesian nash equalization for each game state is:
namely the Bayesian Nash equilibrium strategy. Because the attacker type set Θ, the state set S, the attacker action set A and the defender action set D are all finite sets, bayesian Nash equilibrium exists in the game, namely a Bayesian Nash equilibrium defending strategy exists.
According to the preferred scheme of the invention, in the step 4), a multiple agent Bayesian Q-learning algorithm (MABQL) is utilized to learn and obtain a Nash equilibrium defense strategy under the conditions of a dynamic network environment and unknown game parameters, specifically:
4.1 Q-function of Q-learning algorithm is defined as:
alpha represents the learning rate, gamma represents the fit factor, and the expected benefit of the Q function can be defined as:
4.2 The flow of learning to obtain Nash equilibrium defense strategy is as follows: (1) initializing parameters: r is equal to or is equal to 0,k=1, 2..k; (2) third party participant selection of current gaming state s t The method comprises the steps of carrying out a first treatment on the surface of the (3) Third party participants select attacker theta based on theta distribution i The method comprises the steps of carrying out a first treatment on the surface of the (4) Selecting actions by the two game parties based on the epsilon-greedy strategy; (5) the third party participant selects the next state; (6) updating the Q-function based on the Chinese formula of 4.1); (7) nash equalization strategy for calculating current state +.>(8) r is ≡r+1 and jumps to ≡ 2) repeating the above process; (9) when meeting the requirementsReturning to a Nash equilibrium strategy when the delta is a threshold parameter; obtaining the best strategy for all types of attackers and the best defense strategy for cooperatively defending all the attackers.
The invention has the beneficial effects of solving the problem that the industrial information physical system lacks active decision capability when facing high-persistence and high-concealment network attack. The invention provides a safety decision method based on a Bayesian random game model, which accurately captures the dynamic interaction process of attack and defense and can generate cooperative attack aiming at different types of attackers to perform optimal defense. In order to overcome the challenges brought by different objective utility functions between information and physical layers in a traditional game model, the invention introduces a unified quantization method based on time, integrates various utility functions into one frame, and bridges the gap between the information and physical security fields. Unlike traditional methods that require complete information to solve the game problem, the present invention proposes a data-driven reinforcement learning method, i.e., MABQL, that can obtain an optimal defense strategy in unknown game parameters or uncertain network environments. According to the invention, the network attack process in the actual industrial control scene is accurately modeled, and the Bayesian Nash equilibrium theory and the MABQL algorithm are utilized to generate the optimal defense strategy in the dynamic network security state, so that the network penetration attack on the industrial information physical system is actively and efficiently prevented.
Drawings
FIG. 1 is a flow chart of an implementation of the present invention;
FIG. 2 is a Harsanyi converted game tree;
FIG. 3 is a diagram of a test platform system architecture;
FIG. 4 is a diagram of a gambling state Markov transition;
FIG. 5 is a MABQL algorithm defense strategy convergence diagram;
FIG. 6 is a graph of expected benefit convergence for the MABQL algorithm;
figure 7 is a defensive strategy evolution process under varying circumstances.
Detailed Description
A clearer explanation of the objects and effects of the present invention will be provided with reference to the attached drawings.
As shown in fig. 1, the implementation flow chart of the method for generating the defense strategy of the industrial information physical system based on the bayesian random game mainly comprises 4 sub-steps, and in a specific embodiment, the method specifically comprises the following steps:
1) Constructing a Bayesian random game model, wherein the Bayesian random game model describes an attack-defense interaction process under the condition that network attack information is incompletely known in an industrial information physical system;
specifically, the attack and defense interaction in the industrial information physical system is abstracted into a non-zero Bayesian random game model, under the model, the continuous process of the attack and defense interaction is discretized into a plurality of game states by utilizing a random game theory, and each game state corresponds to a specific security state of an information layer or a physical layer in the system; the attacker and the defender have corresponding game utility functions in the information layer or physical layer game state; the current state may transition to the next gaming state based on a probability distribution determined jointly by the actions of the defender and the attacker;
considering that an defender does not have complete information about an attacker in an actual attack and defense scene, converting each game state into a static Bayesian game by utilizing a Bayesian game theory, wherein a utility function is unilateral information;
finally, the bayesian stochastic game model is defined as an 11-tuple:
G=<N,S,Θ,P A ,A,D,T,O,π A ,π D ,U A ,U D >
wherein the elements are defined as: n= { Attacker, defender } is the set of game participants; s= { S 1 ,s 2 ,...,s K -a set of game states, wherein each state represents a security state of the network; Θ= { θ 1 ,θ 2 ,...,θ I -representing a set of attacker types;a set of probability distributions representing defenders versus aggressor types, whereRepresentative state s k Distribution of the lower part; a= { a 1 ,A 2 ,...,A K An attacker action set, where A k ={a 1 ,a 2 ,...,a n Represents state s k A lower action set; d= { D 1 ,D 2 ,...,D K The actions set of the defender, where D k ={d 1 ,d 2 ,...,d n Represents state s k A lower action set; />Is the policy set of the attacker; />Is a policy set of defenders; u (U) A And U D Utility functions of the offender and defender, respectively.
2) The industrial information physical system comprises an information layer and a physical layer, and a unified quantization method based on time is utilized to construct game utility functions of the information layer and the physical layer, which are targeted by both the attack and the defense in the Bayesian random game model, by taking time as a quantization index; the method comprises the following steps:
2.1 Defining attack time T a The method comprises the steps of carrying out a first treatment on the surface of the It represents the time required for an attacker to perform an attack action on a particular target, which quantifies the time from the vulnerability scanning to the attacker's successful exploitation of the vulnerability;
2.2 Defining a defence time T d The method comprises the steps of carrying out a first treatment on the surface of the Which represents the time required for the defender to perform the defending action, which quantifies the time from recognition of the attack to completion of the defending action;
2.3 Defining recovery time T r The method comprises the steps of carrying out a first treatment on the surface of the Representing the time required for the attacked device to resume its normal operating state after the attack; t for unified quantization of recovery time of information layer and physical layer r Is defined as:
T r =T r,c +T r,p
T r,c representing the time required to restore the compromised device to a normal state by bug fixes or switches to the standby device; t (T) r,p Representing the time required to restore the physical layer control process to a normal state; when the attacked device is an information layer host device, T r =T r,c The method comprises the steps of carrying out a first treatment on the surface of the When the attacked device is a physical layer sensor or actuator device then T r =T r,c +T r,p The method comprises the steps of carrying out a first treatment on the surface of the This marks that the defender needs not only to repair the damaged equipment, but also to resume the control process;
2.4 For calculating T r,p The physical layer control process is modeled as:
x k is the system state at time k, y k Is the measurement state at time k, u' k Represents the attack signal at the moment k, A represents the transfer matrix, B represents the input control matrix, C represents the output observation matrix, B a Represents an attack matrix, w k Representing process noise, v k Representing measurement noise; t (T) r,p Indicating the overall recovery time for the state variable to deviate from the normal range to recover to the normal range when the physical layer control process is attacked.
3) Converting incomplete information of an attacker-to-attacker game utility function into imperfect information about attacker types by using a Harsanyi conversion method to obtain an imperfect information Bayesian random game model participated by various attacker types, and ensuring the existence of Bayesian Nash equilibrium defense strategies;
as shown in fig. 2, the Harsanyi conversion method is as follows:
3.1 Introducing a third party participant, and converting incomplete information of the defender on the utility function of the attacker into imperfect information on the action of the third party participant;
3.2 After the attack and defense parties select respective actions, the third party participant selects the next game state according to the transition probability, and the new state can be observed by the attacker and the defender;
3.3 Third party participants select the type of attacker based on the probability distribution of the defender to the type of attacker, and can only be observed by the attacker;
3.4 After the third party participant completes the selection, the attacker and defender both select actions according to the respective strategies and then obtain the respective rewards;
3.5 Repeating the processes from 3.1) to 3.4) until the game is finished, and obtaining the imperfect information Bayesian random game model with participation of various attacker types.
The utility function of the attacker and defender in step 3) is defined as:
R A (s t ,a,d,θ i ) And R is D (s t ,a,d,θ i ) Representative attackInstant rewards of the batters and defenders,and->Representing a future reward;
the immediate rewards in the utility function are defined as:
R A (s t ,a,d,θ i )=ε(s t ,a,d)T r (a,θ i )-T a (θ i )
R D (s t ,a,d,θ i )=-ε(s t ,a,d)T r (a,θ i )-T d
T r (a,θ i ) Is attacker theta i Recovery time required by the system after successful attack; t (T) a (θ i ) Representing attacker theta i The time required to perform the attack action; epsilon(s) t A, d) represents the probability of success of the attack under action (a, d); t (T) d Representing the time required to implement a defensive action;
for future awards in the utility function, O (s k |s t A, d) represents the state s in the case of an attacker and defender taking action (a, d), respectively t Transition to state s k And O(s) k |s t ,a,d)=ε k (a,d);Andrepresenting the desired utility value in a state, also referred to as a state value, can be found by the following formula:
bayesian nash equalization for each game state is:
namely the Bayesian Nash equilibrium strategy. Because the attacker type set Θ, the state set S, the attacker action set A and the defender action set D are all finite sets, bayesian Nash equilibrium exists in the game, namely a Bayesian Nash equilibrium defending strategy exists.
4) Based on the Bayesian random game model obtained in the step 3), learning by utilizing a multi-agent Bayesian Q-learning algorithm (MABQL) under the conditions of a dynamic network environment and unknown game parameters to obtain a Nash equilibrium defense strategy, and obtaining the optimal defense strategy of the industrial information physical system.
In the step 4), a multiple-agent bayesian Q-learning algorithm (MABQL) is used to learn and obtain a nash equilibrium defense strategy under the conditions of a dynamic network environment and unknown game parameters, specifically:
4.1 Q-function of Q-learning algorithm is defined as:
alpha represents the learning rate, gamma represents the fit factor, and the expected benefit of the Q function can be defined as:
4.2 The flow of learning to obtain Nash equilibrium defense strategy is as follows: (1) initializing parameters: r is equal to or is equal to 0,k=1, 2..k; (2) third party participant selection of current gaming state s t The method comprises the steps of carrying out a first treatment on the surface of the (3) Third party participants select attacker theta based on theta distribution i The method comprises the steps of carrying out a first treatment on the surface of the (4) Selecting actions by the two game parties based on the epsilon-greedy strategy; (5) the third party participant selects the next state; (6) updating the Q-function based on the Chinese formula of 4.1); (7) nash equalization strategy for calculating current state +.>(8) r is ≡r+1 and jumps to ≡ 2) repeating the above process; (9) when meeting the requirementsReturning to a Nash equilibrium strategy when the delta is a threshold parameter; obtaining the best strategy for all types of attackers and the best defense strategy for cooperatively defending all the attackers.
The present invention utilizes a laboratory test platform at university of Zhejiang to simulate the power generation process to evaluate the proposed method. Fig. 3 shows the system architecture of the test platform used. In enterprise networks, management host 1 (MH 1) plays the role of an external network service, while management host 2 (MH 2) acts as an SQL server for data storage. They all have the ability to communicate with hosts within the control network. In a control network, human-machine interfaces (HMI) are used to monitor the control process, while Engineer Stations (ES) and Operator Stations (OS) are used to issue control commands to Programmable Logic Controllers (PLCs). S7-1200 series PLCs were selected to effectively control the underlying actuators and sensors.
There are several security vulnerabilities in the test platform environment, as shown in table 1.
TABLE 1
Each vulnerability is assigned a unique ID based on a common vulnerability and a public vulnerability database (CVE). For example, CVE-2017-0144 represents a vulnerability to Windows Server Message Block (SMB) allowing remote arbitrary code execution over the SMB interface. The present invention determines by evaluating the probability of a successful exploit vulnerability using the availability score in the universal vulnerability scoring system (CVSS).
We assume that the main goal of an attacker is to destroy the sensors and actuators. In view of the tight access control in industrial networks, an attacker should first exploit vulnerabilities to elevate his rights on enterprise network hosts. Then, the attacker moves transversely to destroy the control network host and establishes hidden communication with the controller by adopting disguise of legal hosts. Eventually, the attacker will continue to launch targeted attacks on the executor or sensor, aiming at disturbing the physical process. Table 2 shows security states accessible to an attacker and fig. 4 illustrates the corresponding transition relationships between gaming states.
TABLE 2
For an attacker, the invention considers the attack type theta= { theta 1 ,θ 2 And }, wherein θ 1 Representing a low-level attacker, θ 2 Representing a high-level attacker. Table 3 lists a list of all potential aggressions and the corresponding action times required for each type of attacker.
TABLE 3 Table 3
The set of attack actions available in each state can be represented as A 1 ={a 1 ,a 2 },A 2 ={a 3 ,a 4 ,a 5 },A 3 ={a 3 ,a 4 ,a 5 },A 4 ={a 6 ,a 7 ,a 8 },A 5 ={a 6 ,a 7 ,a 8 },A 6 ={a 3 ,a 4 },In addition, table 4 shows the recovery time T required after a successful attack r (a,θ i )。
TABLE 4 Table 4
For defenders, in each state, the prior probability distribution of the attacker type is P 1 A ={0.7,0.3},P 2 A ={0.5,0.5},P 3 A ={0.5,0.5},P 4 A ={0.26,0.74},P 5 A ={0.26,0.74},P 6 A = {0.3,0.7}. To prevent potential intrusion by an attacker, defensive actions are recorded in table 5.
TABLE 5
The set of defensive actions available in each state can then be described as D 1 ={d 1 ,d 2 ,d 3 },D 2 ={d 4 ,d 5 ,d 6 },D 3 ={d 4 ,d 5 ,d 6 },D 4 ={d 7 ,d 8 ,d 9 ,d 10 },D 5 ={d 7 ,d 8 ,d 9 ,d 10 },D 6 ={d 4 ,d 5 },
In order to verify the feasibility and convergence of the proposed MABQL algorithm, a conventional nonlinear programming method is first used to obtain an optimal attack defense strategy with complete information of all parameters. The results are presented in table 6, which provides a comprehensive overview of the policies employed by the attacker and defender at the nash equilibrium point and their corresponding expected utility values.
TABLE 6
The results show that attacker θ 1 And theta 2 Different optimal strategies are selected. For example, in state S 1 Under attacker θ 1 A mixing strategy will be employedAnd attacker theta 2 Will employ a pure strategy->To combat attacker theta 1 And theta 2 Is +.>These experimental results illustrate the subtle and varied decision processes employed by different players in gaming. In addition, table 6 provides an overview of the player's expected utility values when the Nash equilibrium point is reached, indicating that the utility function in the proposed gaming model is non-zero and.
Subsequently, the invention applies the proposed MABQL algorithm to solve the gaming model under incomplete information. The learning rate α is set to 0.5, and the discount factor γ is set to 1. Fig. 5 depicts the process of defender strategy learning, converging after about 100 learning steps. Fig. 6 illustrates the evolution of the expected utility value of the defender in each state. Experimental results show that the proposed algorithm can converge to a nash equalization strategy.
The present invention defines the following network environment to evaluate the suitability of the proposed algorithm: (1) in state t=t 1 Under the condition, the state transition probability in the network environment changes, O (s k |s t ,a,d)=ε k (a, d)/2; (2) in state t=t 2 Next, the defender updates the probability beliefs for the type of attacker in the network. In state S 2 In the case of an example of this,from {0.5,0.5} to {0.3,0.7}.
The results are shown in fig. 7, indicating that the algorithm initially converged to nash equalization. When the network environment is at t=t 1 When the position changes, the algorithm is quickly adapted and converges again after 100 learning steps, and the defense strategy is changed into {0.543,0.309,0.148}. Also, at t=t 2 When the probability distribution of the attacker type changes, the algorithm quickly converges to a new nash equalization strategy {0.402,0.386,0.212}. The experimental result verifies that the provided algorithm can quickly learn the optimal defense strategy, and shows the effectiveness of the provided invention in the aspect of formulating the optimal defense strategy.
The foregoing list is only illustrative of specific embodiments of the invention. Obviously, the invention is not limited to the above embodiments, but many variations are possible. All modifications directly derived or suggested to one skilled in the art from the present disclosure should be considered as being within the scope of the present invention.
Claims (1)
1. A method for generating an industrial information physical system defense strategy based on Bayesian random game is characterized by comprising the following steps:
1) Constructing a Bayesian random game model, wherein the Bayesian random game model describes an attack-defense interaction process under the condition that network attack information is incompletely known in an industrial information physical system;
the construction process of the Bayesian random game model in the step 1) specifically comprises the following steps:
abstracting attack and defense interaction in an industrial information physical system into a non-zero Bayesian random game model, under the model, dispersing a continuous process of the attack and defense interaction into a plurality of game states by utilizing a random game theory, wherein each game state corresponds to a specific security state of an information layer or a physical layer in the system; the attacker and the defender have corresponding game utility functions in the information layer or physical layer game state; the current state transits to the next game state according to the probability distribution determined by the actions of the defender and the attacker together;
considering that an defender does not have complete information about an attacker in an actual attack and defense scene, converting each game state into a static Bayesian game by utilizing a Bayesian game theory, wherein a utility function is unilateral information;
finally, the bayesian stochastic game model is defined as an 11-tuple:
G=<N,S,Θ,P A ,A,D,T,O,π A ,π D ,U A ,U D >
wherein the elements are defined as: n= { Attacker, defender } is the set of game participants; s= { S 1 ,s 2 ,...,s K -a set of game states, wherein each state represents a security state of the network; Θ= { θ 1 ,θ 2 ,...,θ I -representing a set of attacker types;a set of probability distributions representing defenders versus aggressor types, whereRepresentative state s k Distribution of the lower part; a= { a 1 ,A 2 ,...,A K An attacker action set, where A k ={a 1 ,a 2 ,...,a n -represents the set of actions in state sk; d= { D 1 ,D 2 ,...,D K The actions set of the defender, where D k ={d 1 ,d 2 ,...,d n Represents state s k A lower action set; />Is the policy set of the attacker; />Is a policy set of defenders; u (U) A And U D Utility functions of the attacker and defender, respectively;
2) The industrial information physical system comprises an information layer and a physical layer, and a unified quantization method based on time is utilized to construct game utility functions of the information layer and the physical layer, which are targeted by both the attack and the defense in the Bayesian random game model, by taking time as a quantization index;
in the step 2), game utility functions of the attack and defense parties aiming at the information layer and the physical layer in the Bayesian random game model are constructed by taking time as a quantization index, and the method specifically comprises the following steps:
2.1 Defining attack time T a The method comprises the steps of carrying out a first treatment on the surface of the It represents the time required for an attacker to perform an attack action on a particular target, which quantifies the time from the vulnerability scanning to the attacker's successful exploitation of the vulnerability;
2.2 Defining a defence time T d The method comprises the steps of carrying out a first treatment on the surface of the Which represents the time required for the defender to perform the defending action, which quantifies the time from recognition of the attack to completion of the defending action;
2.3 Defining recovery time T r The method comprises the steps of carrying out a first treatment on the surface of the Representing the time required for the attacked device to resume its normal operating state after the attack; t for unified quantization of recovery time of information layer and physical layer r Is defined as:
T r =T r,c +T r,p
T r,c representing the time required to restore the compromised device to a normal state by bug fixes or switches to the standby device; t (T) r,p An overall recovery time indicating that the state variable deviates from the normal range to be recovered to the normal range when the physical layer control process is attacked; when the attacked device is an information layer host device, T r =T r,c The method comprises the steps of carrying out a first treatment on the surface of the When the attacked device is a physical layer sensor or actuator device then T r =T r,c +T r,p The method comprises the steps of carrying out a first treatment on the surface of the This marks that the defender needs not only to repair the damaged equipment, but also to resume the control process;
2.4 For calculating T r,p The physical layer control process is modeled as:
x k is the system state at time k, x k+1 Is the system state at time k+1, y k Is the measurement state at time k, u' k Represents the attack signal at the moment k, A represents the transfer matrix, B represents the input control matrix, C represents the output observation matrix, B a Represents an attack matrix, w k Representing process noise, v k Representing measurement noise;
3) Converting incomplete information of an attacker-to-attacker game utility function into imperfect information about attacker types by using a Harsanyi conversion method to obtain an imperfect information Bayesian random game model participated by various attacker types, and ensuring the existence of Bayesian Nash equilibrium defense strategies;
the Harsanyi conversion method in the step 3) comprises the following steps:
3.1 Introducing a third party participant, and converting incomplete information of the defender on the utility function of the attacker into imperfect information on the action of the third party participant;
3.2 After the attack and defense parties select respective actions, the third party participant selects the next game state according to the transition probability, and the new state is observed by the attacker and the defender;
3.3 Third party participants select the type of attacker based on the probability distribution of the defender to the type of attacker, and can only be observed by the attacker;
3.4 After the third party participant completes the selection, the attacker and defender both select actions according to the respective strategies and then obtain the respective rewards;
3.5 Repeating the processes from 3.1) to 3.4) until the game is finished, and obtaining an imperfect information Bayesian random game model with participation of various attacker types;
the utility function of the attacker and defender in step 3) is defined as:
R A (s t ,a,d,θ i ) And R is D (s t ,a,d,θ i ) Instant rewards representing aggressors and defenders,and->Representing a future reward;
the immediate rewards in the utility function are defined as:
R A (s t ,a,d,θ i )=ε(s t ,a,d)T r (a,θ i )-T a (θ i )
R D (s t ,a,d,θ i )=-ε(s t ,a,d)T r (a,θ i )-T d
T r (a,θ i ) Is the recovery time required by the system after an attacker θi successfully attacks; t (T) a (θ i ) The time required to perform an attack action on behalf of an attacker θi; epsilon(s) t A, d) represents the probability of success of the attack under action (a, d); t (T) d Representing the time required to implement a defensive action;
for future awards in the utility function, O (s k |s t A, d) represents the state s in the case of an attacker and defender taking action (a, d), respectively t Transition to state s k And O(s) k |s t ,a,d)=ε k (a,d);And->Representing the desired utility value in a state, also referred to as a state value, is obtained by the following formula:
bayesian nash equalization for each game state is:
namely a Bayesian Nash equilibrium strategy; because the attacker type set Θ, the state set S, the attacker action set A and the defender action set D are all finite sets, bayesian Nash equilibrium exists in the game, namely a Bayesian Nash equilibrium defending strategy exists;
4) Based on the Bayesian random game model obtained in the step 3), learning by utilizing a multi-agent Bayesian Q-learning algorithm (MABQL) under the conditions of a dynamic network environment and unknown game parameters to obtain a Nash equilibrium defense strategy, and obtaining an optimal defense strategy of the industrial information physical system;
in the step 4), a multiple-agent bayesian Q-learning algorithm (MABQL) is used to learn and obtain a nash equilibrium defense strategy under the conditions of a dynamic network environment and unknown game parameters, specifically:
4.1 Q-function of Q-learning algorithm is defined as:
alpha represents the learning rate, gamma represents the fit factor, and the expected benefit of the Q function is defined as:
4.2 The flow of learning to obtain Nash equilibrium defense strategy is as follows: (1) initializing parameters: r is equal to or is equal to 0,(2) third party participant selection of current gaming state s t The method comprises the steps of carrying out a first treatment on the surface of the (3) Third party participants select attacker theta based on theta distribution i The method comprises the steps of carrying out a first treatment on the surface of the (4) Selecting actions by the two game parties based on the epsilon-greedy strategy; (5) the third party participant selects the next state; (6) updating the Q-function based on the Chinese formula of 4.1); (7) nash equalization strategy for calculating current state +.>(8) r is ≡r+1 and jumps to ≡ 2) repeating the above process; (9) when meeting the requirementsReturning to a Nash equilibrium strategy when the delta is a threshold parameter; obtaining the best strategy for all types of attackers and the best defense strategy for cooperatively defending all the attackers.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310894809.2A CN117040809B (en) | 2023-07-20 | 2023-07-20 | Method for generating defense strategy of industrial information physical system based on Bayesian random game |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310894809.2A CN117040809B (en) | 2023-07-20 | 2023-07-20 | Method for generating defense strategy of industrial information physical system based on Bayesian random game |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117040809A CN117040809A (en) | 2023-11-10 |
CN117040809B true CN117040809B (en) | 2024-04-05 |
Family
ID=88638137
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310894809.2A Active CN117040809B (en) | 2023-07-20 | 2023-07-20 | Method for generating defense strategy of industrial information physical system based on Bayesian random game |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117040809B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105307175A (en) * | 2015-09-22 | 2016-02-03 | 绍兴文理学院 | Method for selecting IDA (intrusion detection agent) start strategies of wireless sensor network |
CN107070956A (en) * | 2017-06-16 | 2017-08-18 | 福建中信网安信息科技有限公司 | APT Attack Prediction methods based on dynamic bayesian game |
CN112115469A (en) * | 2020-09-15 | 2020-12-22 | 浙江科技学院 | Edge intelligent moving target defense method based on Bayes-Stackelberg game |
CN115102166A (en) * | 2022-07-27 | 2022-09-23 | 南京邮电大学 | Active power distribution network dynamic defense performance optimization method based on game theory |
CN115348064A (en) * | 2022-07-28 | 2022-11-15 | 南京邮电大学 | Power distribution network defense strategy design method based on dynamic game under network attack |
CN115932752A (en) * | 2023-01-06 | 2023-04-07 | 中国人民解放军海军大连舰艇学院 | Radar cognitive interference decision method based on incomplete information game |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR3063554B1 (en) * | 2017-03-03 | 2021-04-02 | Mbda France | METHOD AND DEVICE FOR PREDICTING OPTIMAL ATTACK AND DEFENSE SOLUTIONS IN A MILITARY CONFLICT SCENARIO |
US20230185912A1 (en) * | 2021-12-13 | 2023-06-15 | International Business Machines Corporation | Defending deep generative models against adversarial attacks |
-
2023
- 2023-07-20 CN CN202310894809.2A patent/CN117040809B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105307175A (en) * | 2015-09-22 | 2016-02-03 | 绍兴文理学院 | Method for selecting IDA (intrusion detection agent) start strategies of wireless sensor network |
CN107070956A (en) * | 2017-06-16 | 2017-08-18 | 福建中信网安信息科技有限公司 | APT Attack Prediction methods based on dynamic bayesian game |
CN112115469A (en) * | 2020-09-15 | 2020-12-22 | 浙江科技学院 | Edge intelligent moving target defense method based on Bayes-Stackelberg game |
CN115102166A (en) * | 2022-07-27 | 2022-09-23 | 南京邮电大学 | Active power distribution network dynamic defense performance optimization method based on game theory |
CN115348064A (en) * | 2022-07-28 | 2022-11-15 | 南京邮电大学 | Power distribution network defense strategy design method based on dynamic game under network attack |
CN115932752A (en) * | 2023-01-06 | 2023-04-07 | 中国人民解放军海军大连舰艇学院 | Radar cognitive interference decision method based on incomplete information game |
Also Published As
Publication number | Publication date |
---|---|
CN117040809A (en) | 2023-11-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Falco et al. | A master attack methodology for an AI-based automated attack planner for smart cities | |
Maeda et al. | Automating post-exploitation with deep reinforcement learning | |
Huang et al. | A game-theoretic approach to cross-layer security decision-making in industrial cyber-physical systems | |
Kaloudi et al. | The ai-based cyber threat landscape: A survey | |
Tuptuk et al. | Security of smart manufacturing systems | |
Kalech | Cyber-attack detection in SCADA systems using temporal pattern recognition techniques | |
Macas et al. | A survey on deep learning for cybersecurity: Progress, challenges, and opportunities | |
Zhu et al. | On multi-phase and multi-stage game-theoretic modeling of advanced persistent threats | |
Hu et al. | Optimal network defense strategy selection based on incomplete information evolutionary game | |
Durkota et al. | Case studies of network defense with attack graph games | |
Liu et al. | On deep reinforcement learning security for Industrial Internet of Things | |
Hu et al. | Adaptive cyber defense against multi-stage attacks using learning-based POMDP | |
Khoury et al. | A hybrid game theory and reinforcement learning approach for cyber-physical systems security | |
Ridley | Machine learning for autonomous cyber defense | |
Liu et al. | FlipIt game model-based defense strategy against cyberattacks on SCADA systems considering insider assistance | |
Zhong et al. | An efficient parallel reinforcement learning approach to cross-layer defense mechanism in industrial control systems | |
Chen et al. | Optimal defense strategy selection for spear-phishing attack based on a multistage signaling game | |
Xu et al. | Autoattacker: A large language model guided system to implement automatic cyber-attacks | |
Hammar et al. | Learning near-optimal intrusion responses against dynamic attackers | |
CN117040809B (en) | Method for generating defense strategy of industrial information physical system based on Bayesian random game | |
Figetakis et al. | Evolved prevention strategies for 6g networks through stochastic games and reinforcement learning | |
Li et al. | Robust moving target defense against unknown attacks: A meta-reinforcement learning approach | |
Mesadieu et al. | Leveraging Deep Reinforcement Learning Technique for Intrusion Detection in SCADA Infrastructure | |
CN116582330A (en) | Industrial control network automatic defense decision-making method oriented to part of unknown security states | |
Luo et al. | A fictitious play‐based response strategy for multistage intrusion defense systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |