CN117991647A - Performance monitoring and self-healing control method and device for industrial control system - Google Patents
Performance monitoring and self-healing control method and device for industrial control system Download PDFInfo
- Publication number
- CN117991647A CN117991647A CN202410396935.XA CN202410396935A CN117991647A CN 117991647 A CN117991647 A CN 117991647A CN 202410396935 A CN202410396935 A CN 202410396935A CN 117991647 A CN117991647 A CN 117991647A
- Authority
- CN
- China
- Prior art keywords
- self
- performance
- control system
- industrial control
- healing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 165
- 238000000034 method Methods 0.000 title claims abstract description 123
- 238000011156 evaluation Methods 0.000 claims abstract description 79
- 230000015556 catabolic process Effects 0.000 claims abstract description 58
- 238000006731 degradation reaction Methods 0.000 claims abstract description 58
- 230000008569 process Effects 0.000 claims abstract description 36
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 23
- 230000002787 reinforcement Effects 0.000 claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims description 142
- 230000006870 function Effects 0.000 claims description 51
- 230000003416 augmentation Effects 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 15
- 238000005457 optimization Methods 0.000 claims description 7
- 230000001502 supplementing effect Effects 0.000 claims description 7
- 230000003190 augmentative effect Effects 0.000 claims description 6
- 238000013499 data model Methods 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 238000013178 mathematical model Methods 0.000 abstract description 8
- 239000013589 supplement Substances 0.000 abstract description 4
- 238000004519 manufacturing process Methods 0.000 description 16
- 238000011084 recovery Methods 0.000 description 14
- 230000002159 abnormal effect Effects 0.000 description 13
- 238000013461 design Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 8
- 239000000047 product Substances 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 238000009776 industrial production Methods 0.000 description 6
- 230000007246 mechanism Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 4
- 238000005859 coupling reaction Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 235000019800 disodium phosphate Nutrition 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000005299 abrasion Methods 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000005562 fading Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000004540 process dynamic Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention relates to the field of operation monitoring and self-healing control, in particular to a method and a device for monitoring and self-healing performance of an industrial control system, wherein the method comprises the following steps: acquiring real-time operation data of an industrial control system to be controlled; obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and a performance degradation monitoring module based on a Bellman equation, and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result; and according to the performance degradation monitoring result and the self-optimizing feedback supplement self-healing control module based on the reinforcement learning algorithm, obtaining the self-healing control result of the industrial control system. The invention realizes real-time performance monitoring by utilizing the data of the industrial operation process under the condition of not depending on the controlled object and the mathematical model of the control system; the original controller structural parameters are not changed, and only the data and reinforcement learning method is used for designing feedback compensation to recover performance degradation, so that self-healing control is realized.
Description
Technical Field
The invention relates to the field of operation monitoring and self-healing control, in particular to a method and a device for monitoring and self-healing performance of an industrial control system.
Background
In recent years, the rapid development of social economy promotes the rapid increase of the demand of various industrial products, and simultaneously, higher requirements are also put forward on the quality of the products, and the modern industrial production gradually develops towards refinement and intensification. The degree of automation of industrial production process is continuously improved, and new challenges are presented to the design of safety and reliability of the industrial control system, so that the performance monitoring and self-healing control technology of the industrial control system is increasingly paid attention to.
The production environment of the industrial process is bad, the operation condition is complex, the limitation of materials and equipment is caused, the controlled object, the actuating mechanism and the sensor of the industrial control system can be failed, and the system performance is reduced; meanwhile, due to factors such as frequent switching of certain production conditions, abrasion and aging of equipment, improper operation and the like, abnormal operation conditions of a control system are easy to cause, performance degradation of the control system is further caused, normal operation of the control system is affected, product quality and yield are reduced, safety accidents are also caused due to serious conditions, and irrecoverable loss is caused. Therefore, it is necessary to establish a set of abnormal operation monitoring scheme and a self-healing recovery mechanism for coping with performance degradation of the control system for the industrial control system, so as to ensure safe and stable operation of the industrial process.
However, the actual industrial production process has complex structure, high process continuity, serious coupling condition between variables and unclear mechanism, and it is difficult to obtain an accurate mathematical model for designing a performance monitoring scheme and a self-healing control method. However, with the development of sensor technology and network communication technology, massive real-time data about complex industrial production operation processes are stored, and the data contains abundant industrial control system operation states and information. Therefore, how to utilize industrial operation data to effectively monitor the performance of the industrial production process control system and design a self-healing control mechanism with performance fading gradually becomes a technical problem worthy of research in the industry and academia.
The current popular performance evaluation method of the industrial control system is based on minimum variance performance evaluation, the method focuses on product quality, variance fluctuation of system output is utilized to measure the performance of the control system, but in the actual production process, the product quality is ensured, cost consumption is also required to be focused, and control cost also influences the evaluation of the performance of the control system, so that control input and process output of the industrial process need to be considered simultaneously in the process of designing the performance evaluation standard of the industrial control system.
In order to solve the problem, the LQG (Linear Quadratic Gaussian ) standard is put forward, so that not only is the variance of the output considered, but also the control input is added into the standard of system performance measurement, the information used in the evaluation process is enriched, and the performance evaluation standard is further more reasonable. However, the LQG-based reference has a high dependency on the model, and is constructed only after obtaining a relatively accurate mathematical model, so that in order to overcome the disadvantage, an LQG performance evaluation method based on subspace identification technology is proposed, but the process still needs to identify a parameter matrix of a controlled system, the required calculation cost is high, and performance monitoring is performed after the identification process, wherein errors also affect the performance evaluation effect.
In order to ensure that the industrial production process can complete corresponding control tasks under abnormal operation conditions and ensure the safety and reliability of system operation, fault-tolerant control technology is developed in industrial control systems. General fault-tolerant control techniques are classified into passive fault-tolerant and active fault-tolerant techniques according to whether they rely on fault diagnosis units. Among them, the more advantageous active fault-tolerant control approaches are roughly classified into control law reconstruction and control parameter rescheduling. Most industrial control systems, however, write parameters or programs in the control unit and are not easily modified. Meanwhile, most fault-oriented fault-tolerant control technologies are designed aiming at control units, performance-oriented fault-tolerant control system designs are in a development stage, and fault tolerance focusing on performance recovery of the control system gradually becomes a hot spot. With the appearance of self-healing control ideas, the research of self-healing control of the operation process from abnormal working conditions to normal working conditions is realized through control means, a flight control system, a power grid and an internet system are taken as research objects in the early stage, however, the research of the performance recovery self-healing means of the industrial process control system with environmental complexity and process dynamics is still in a starting stage. For industrial processes, the self-healing control means that the input and output of a control loop are regulated by an active control means under abnormal working conditions of the industrial process deviating from safe operation. At present, most of self-healing control methods aiming at abnormal working conditions are self-healing control methods taking an optimal control loop set value as a means, and lack of intelligent self-healing decisions aiming at complexity and dynamics of industrial processes, fusion of data and knowledge and real-time feedback of an execution process.
Disclosure of Invention
In order to solve the technical problems of performance monitoring and self-healing recovery of an industrial control system in a complex operating environment, the embodiment of the invention provides a method and a device for performance monitoring and self-healing control of the industrial control system. The technical scheme is as follows:
In one aspect, there is provided an industrial control system performance monitoring and self-healing control method implemented by an industrial control system performance monitoring and self-healing control device, the method comprising:
s1, acquiring real-time operation data of an industrial control system to be controlled.
S2, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the performance degradation monitoring module based on the Bellman equation, and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result.
And S3, supplementing a self-healing control module according to the performance degradation monitoring result and the self-optimization feedback based on the reinforcement learning algorithm to obtain a self-healing control result of the industrial control system.
Optionally, the performance degradation monitoring module in S2, which is based on the real-time operation data and Bellman equation, obtains a real-time performance evaluation result of the industrial control system, and obtains a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result, where the performance degradation monitoring module includes:
S21, acquiring input and output data in the operation process of the industrial control system.
S22, building performance indexes of the industrial control system according to the input and output data.
S23, obtaining a performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation and the performance index.
S24, designing an identification method of a performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix.
S25, obtaining the performance monitoring reference matrix of the industrial control system according to the input and output data and the identification method of the performance monitoring matrix.
S26, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the identification method of the performance monitoring matrix.
And S27, obtaining performance degradation monitoring indexes of the industrial control system according to the performance monitoring reference matrix and the real-time performance evaluation result, and obtaining the performance degradation monitoring result of the industrial control system according to the performance degradation monitoring indexes and the designed monitoring logic.
Optionally, the performance index of the industrial control system in S22 is as shown in the following formula (1):
(1)
In the method, in the process of the invention, Representing performance index of industrial control system,/>Representing discount factors,/>Representing the sampling instant of the data,/>,/>Tracking error matrix representing system output,/>Representing the transpose of the matrix,/>Representing system output data,/>Representing reference input,/>Representing a tracking error weight matrix,/>Representing a matrix of control input weights,/>Representing system input data.
Optionally, obtaining the performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation, and the performance index in S23 includes:
S231, constructing an augmentation system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller and the mathematical description of the reference input generator.
S232, rewriting the performance index into a recursive form of a Bellman equation.
S233, obtaining a parameterized form of the performance index of the industrial control system according to the augmentation system, the recursion form and the Bellman optimality principle.
S234, obtaining a performance evaluation matrix of the industrial control system according to the parameterized form, wherein the performance evaluation matrix is shown in the following formula (2):
(2)
In the method, in the process of the invention, Performance evaluation matrix representing an industrial control system,/>Representing a matrix of weights of the tracking error,Representing an output-related parameter matrix of an augmentation system,/>,Representing the reference input state quantity,/>Representing the state quantity of the system,/>Representing a matrix of control input weights,Representing an augmented system control input related parameter matrix,/>Output parameter matrix representing reference input,/>Representing feedback control gain,/>Representing the output parameter matrix of the system,/>Representing the discount factor(s),,/>,/>State parameter matrix representing reference input,/>Representing a state parameter matrix of the system,/>,/>Representing a matrix of input parameters of the system.
Optionally, the identifying method of the performance monitoring matrix of the industrial control system is designed according to the performance index and the performance evaluation matrix in S24, including:
S241, obtaining the rewritten performance index according to the performance evaluation matrix and the performance index.
S242, obtaining the replaced performance index according to the rewritten performance index and the Kalman filter type data model.
S243, vectorizing the replaced performance index to obtain the vectorized performance index.
S244, obtaining a performance monitoring matrix of the industrial control system according to the input and output data, the vectorized performance index and the recursive least square identification method.
Optionally, the self-optimizing feedback supplementing self-healing control module based on the performance degradation monitoring result and the reinforcement learning algorithm in S3 obtains a self-healing control result of the industrial control system, which includes:
s31, acquiring input and output data in the operation process of the industrial control system.
S32, designing a representation form of a self-healing control law.
S33, defining a Q value function according to the performance index of the industrial control system and the parameterized form of the performance index, and rewriting the Q value function according to the representation form of the self-healing control law to obtain the Q value function of the self-healing control law.
S34, minimizing the Q value function of the self-healing control law to obtain the self-healing control law parameter representation form.
S35, obtaining the self-healing control law according to the input and output data, the Q value function of the self-healing control law, the self-healing control law parameter representation form and the Q-learning algorithm.
S36, obtaining a self-healing control result of the industrial control system according to the performance degradation monitoring result and the self-healing control law.
Optionally, obtaining the self-healing control law according to the input/output data, the Q value function of the self-healing control law, the parameter representation form of the self-healing control law and the Q-learning algorithm in S35 includes:
s351, rewriting the Q value function of the self-healing control law to obtain the Q value function of the self-healing control law after rewriting.
S352, obtaining the self-healing control law representation according to the representation form, the parameter representation form and the Q value function of the rewritten self-healing control law.
S353, constructing a time sequence difference TD error representation form, and vectorizing the TD error representation form to obtain the vectorized TD error representation form.
S354, obtaining the self-healing control law according to the input and output data, the vectorized TD error representation form, the self-healing control law representation and the Q-learning algorithm.
In another aspect, there is provided an apparatus for monitoring and self-healing performance of an industrial control system, the apparatus being applied to a method for monitoring and self-healing performance of an industrial control system, the apparatus comprising:
And the acquisition module is used for acquiring real-time operation data of the industrial control system to be controlled.
And the performance monitoring module is used for obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the performance degradation monitoring module based on the Bellman equation and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result.
And the self-healing control module is used for supplementing the self-healing control module according to the performance degradation monitoring result and the self-optimization feedback based on the reinforcement learning algorithm to obtain the self-healing control result of the industrial control system.
Optionally, the performance monitoring module is further configured to:
S21, acquiring input and output data in the operation process of the industrial control system.
S22, building performance indexes of the industrial control system according to the input and output data.
S23, obtaining a performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation and the performance index.
S24, designing an identification method of a performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix.
S25, obtaining the performance monitoring reference matrix of the industrial control system according to the input and output data and the identification method of the performance monitoring matrix.
S26, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the identification method of the performance monitoring matrix.
And S27, obtaining performance degradation monitoring indexes of the industrial control system according to the performance monitoring reference matrix and the real-time performance evaluation result, and obtaining the performance degradation monitoring result of the industrial control system according to the performance degradation monitoring indexes and the designed monitoring logic.
Optionally, the performance index of the industrial control system is as shown in the following formula (1):
(1)
In the method, in the process of the invention, Representing performance index of industrial control system,/>Representing discount factors,/>Representing the sampling instant of the data,/>,/>Tracking error matrix representing system output,/>Representing the transpose of the matrix,/>Representing system output data,/>Representing reference input,/>Representing a tracking error weight matrix,/>Representing a matrix of control input weights,/>Representing system input data.
Optionally, the performance monitoring module is further configured to:
S231, constructing an augmentation system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller and the mathematical description of the reference input generator.
S232, rewriting the performance index into a recursive form of a Bellman equation.
S233, obtaining a parameterized form of the performance index of the industrial control system according to the augmentation system, the recursion form and the Bellman optimality principle.
S234, obtaining a performance evaluation matrix of the industrial control system according to the parameterized form, wherein the performance evaluation matrix is shown in the following formula (2):
(2)
In the method, in the process of the invention, Performance evaluation matrix representing an industrial control system,/>Representing a matrix of weights of the tracking error,Representing an output-related parameter matrix of an augmentation system,/>,Representing the reference input state quantity,/>Representing the state quantity of the system,/>Representing a matrix of control input weights,Representing an augmented system control input related parameter matrix,/>Output parameter matrix representing reference input,/>Representing feedback control gain,/>Representing the output parameter matrix of the system,/>Representing the discount factor(s),,/>,/>State parameter matrix representing reference input,/>Representing a state parameter matrix of the system,/>,/>Representing a matrix of input parameters of the system.
Optionally, the performance monitoring module is further configured to:
S241, obtaining the rewritten performance index according to the performance evaluation matrix and the performance index.
S242, obtaining the replaced performance index according to the rewritten performance index and the Kalman filter type data model.
S243, vectorizing the replaced performance index to obtain the vectorized performance index.
S244, obtaining a performance monitoring matrix of the industrial control system according to the input and output data, the vectorized performance index and the recursive least square identification method.
Optionally, the self-healing control module is further configured to:
s31, acquiring input and output data in the operation process of the industrial control system.
S32, designing a representation form of a self-healing control law.
S33, defining a Q value function according to the performance index of the industrial control system and the parameterized form of the performance index, and rewriting the Q value function according to the representation form of the self-healing control law to obtain the Q value function of the self-healing control law.
S34, minimizing the Q value function of the self-healing control law to obtain the self-healing control law parameter representation form.
S35, obtaining the self-healing control law according to the input and output data, the Q value function of the self-healing control law, the self-healing control law parameter representation form and the Q-learning algorithm.
S36, obtaining a self-healing control result of the industrial control system according to the performance degradation monitoring result and the self-healing control law.
Optionally, the self-healing control module is further configured to:
s351, rewriting the Q value function of the self-healing control law to obtain the Q value function of the self-healing control law after rewriting.
S352, obtaining the self-healing control law representation according to the representation form, the parameter representation form and the Q value function of the rewritten self-healing control law.
S353, constructing a time sequence difference TD error representation form, and vectorizing the TD error representation form to obtain the vectorized TD error representation form.
S354, obtaining the self-healing control law according to the input and output data, the vectorized TD error representation form, the self-healing control law representation and the Q-learning algorithm.
In another aspect, there is provided an industrial control system performance monitoring and self-healing control apparatus, the industrial control system performance monitoring and self-healing control apparatus comprising: a processor; and a memory having stored thereon computer readable instructions which, when executed by the processor, implement any one of the industrial control system performance monitoring and self-healing control methods described above.
In another aspect, a computer readable storage medium having stored therein at least one instruction that is loaded and executed by a processor to implement any of the above-described industrial control system performance monitoring and self-healing control methods is provided.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
In the embodiment of the invention, only input and output data are utilized, the performance of the control system in the industrial process is monitored under the condition of not depending on the controlled object and the accurate mathematical model of the control system, and meanwhile, the performance of the control system in an abnormal running state is recovered to a certain extent by a data driving feedback mode based on a reinforcement learning method under the premise of not changing a preset controller, so that self-healing control is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for monitoring and self-healing control of the performance of an industrial control system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of monitoring abnormal performance and self-healing control of performance degradation of an industrial control system according to an embodiment of the present invention;
FIG. 3 is a block diagram of an industrial control system performance monitoring and self-healing control device according to an embodiment of the present invention;
Fig. 4 is a schematic structural diagram of an industrial control system performance monitoring and self-healing control device according to an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is described below with reference to the accompanying drawings.
In embodiments of the invention, words such as "exemplary," "such as" and the like are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term use of an example is intended to present concepts in a concrete fashion. Furthermore, in embodiments of the present invention, the meaning of "and/or" may be that of both, or may be that of either, optionally one of both.
In the embodiments of the present invention, "image" and "picture" may be sometimes used in combination, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized. "of", "corresponding (corresponding, relevant)" and "corresponding (corresponding)" are sometimes used in combination, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized.
In embodiments of the present invention, sometimes a subscript such as W 1 may be expressed in a non-subscript form such as W1, and the meaning of the expression is consistent when de-emphasizing the distinction.
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides an industrial control system performance monitoring and self-healing control method, which can be realized by industrial control system performance monitoring and self-healing control equipment, wherein the industrial control system performance monitoring and self-healing control equipment can be a terminal or a server. The flow chart of the industrial control system performance monitoring and self-healing control method shown in fig. 1, the process flow of the method can comprise the following steps:
s1, acquiring real-time operation data of an industrial control system to be controlled.
In a feasible implementation manner, in order to solve the problems of abnormal monitoring and self-healing control of performance degradation of an industrial control system in a complex operation environment, the invention designs a performance degradation monitoring module based on a Bellman equation and a self-optimizing feedback supplementing self-healing control module based on a reinforcement learning algorithm, which is triggered according to the monitoring result of the performance degradation monitoring module, as shown in fig. 2.
In the present invention, control performance monitoring (Control Performance Monitoring): the method is used for monitoring the trend and the state of the statistical value which can reflect the control performance of the system and change along with the time.
Self-healing Control (Self-healing Control): the system can timely detect abnormal states and take reasonable measures to inhibit or eliminate the influence of the abnormality on the system performance so as to improve the running reliability of the system.
S2, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the performance degradation monitoring module based on the Bellman equation, and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result.
Optionally, the step S2 may include the following steps S21 to S27:
s21, acquiring, transmitting and storing input/output data (I/O data) in the operation process through an industrial sensor and a data storage device, wherein the input/output data (I/O data) comprises a control input And system output/>。
S22, building performance indexes of the industrial control system according to the input and output data.
In a possible embodiment, the performance index form in step S22 is defined as follows, taking into account both the system output and the control input:
(1)
In the method, in the process of the invention, Representing performance index of industrial control system,/>Representing discount factors, in order to make the cost function bounded, the value range is/>,/>Representing the sampling instant of the data, the single step cost is represented as,/>Tracking error matrix representing system output,/>Representing the transpose of the matrix,/>Representing system output data,/>Representing reference input,/>Representing a tracking error weight matrix,/>Representing a matrix of control input weights,/>Representing system input data.
S23, obtaining a representation form of a closed-loop control system performance evaluation matrix by using a Bellman equation according to the controlled object, the controller and the general mathematical description of the reference input generator.
Optionally, the step S23 may include the following steps S231 to S2314:
S231, constructing an augmentation system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller and the mathematical description of the reference input generator.
In a possible embodiment, a general mathematical description of the controlled object, the controller and the reference input generator is given.
Specifically, assume that the state space of an industrial control closed loop system is described as:
(2)
In the method, in the process of the invention, 、/>And/>Is a system parameter matrix,/>、/>And/>Respectively expressed at/>Control input, state input and state output of moment; /(I)Representing feedback control gain,/>Representing a reference input.
Giving reference inputThe form of (2): /(I)。
S232, rewriting the performance index into a recursive form of a Bellman equation:
(3)
Wherein, 。
S233, obtaining a parameterized form of the performance index of the industrial control system according to the augmentation system, the recursion form and the Bellman optimality principle.
In a possible embodiment, the performance index is parameterized according to the principle of optimality in Bellman and the recursive representation (3), wherein an augmentation system is constructed consisting of the controlled object and the reference input, wherein the state of the augmentation system is represented asI.e. the closed loop control system is described as:
(4)
Wherein the method comprises the steps of ,/>,/>,/>。
Thereafter, the parameterized form of the performance index can be expressed as an augmented state quadratic form as follows:
(5)
Bringing form (5) into recursive form (3), there are;
(6)
a parameterized performance evaluation matrix can be obtained In the above, the ratio of/>Performance evaluation matrix representing an industrial control system,/>Representing a tracking error weight matrix,/>Representing a matrix of parameters related to the output of the augmentation system,,/>Representing the reference input state quantity,/>Representing the state quantity of the system,/>Representing a matrix of control input weights,/>Representing an augmented system control input related parameter matrix,/>Output parameter matrix representing reference input,/>Representing feedback control gain,/>Representing the output parameter matrix of the system,/>Representing discount factors,/>,/>,/>State parameter matrix representing reference input,/>Representing a state parameter matrix of the system,/>,/>Representing a matrix of input parameters of the system.
S24, designing an identification method of a performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix.
In one possible implementation, the system performance monitoring matrix is directly identified using data identification techniques.
Optionally, the step S24 may include the following steps S241 to S244:
S241, obtaining the rewritten performance index according to the performance evaluation matrix and the performance index.
In one possible implementation, the performance evaluation matrix may be equivalently represented asFurther, the performance index can be rewritten as follows:
(7)
s242, in order to identify a performance evaluation matrix by using input/output data, a Kalman filter type data model (I/O model) is introduced, specifically:
(8)
Wherein the method comprises the steps of ,/>If step size/>Large enough (larger than the system order), there is/>I.e. can be approximated by:
(9)
Parameter matrix:
Data matrix:
,/>,。
s243 using the formula (9) Substitution/>The method comprises the following steps: /(I)
(10)
Wherein a performance monitoring matrix is defined。
S244 to enable performance monitoring matrix by direct recognitionAfter vectorization of the above formula, it is possible to obtain:
(11)
And (3) making:
;
;
;
;
;
The method comprises the following steps:
(12)
the data collected in step S21 can be further utilized to obtain the data through a recursive least squares identification method Thereby being capable of directly obtaining the performance monitoring matrix/>。
S25, collecting stored historical operation data based on the step S21, and obtaining a performance evaluation benchmark based on a Bellman equation through the step S24 by utilizing the data of normal operation of the industrial process in a stable state.
S26, acquiring real-time operation data on line, and acquiring real-time performance evaluation results of the industrial control system on line by utilizing the data according to the identification method in the fourth step.
And S27, comparing the performance evaluation standard obtained in the step S25 with the real-time performance evaluation result obtained in the step S26 to obtain a performance degradation monitoring index of the industrial control system, and giving a performance degradation monitoring result according to the designed monitoring logic.
In a possible embodiment, according to step S25, a performance monitoring reference matrix is obtainedAccording to S26, a real-time performance monitoring matrix/>, is obtained。
Further, defining performance monitoring indexAnd the index threshold value of normal operation is/>。
Further, an industrial control system performance degradation monitoring logic is designed:
(13)
And S3, supplementing a self-healing control module according to the performance degradation monitoring result and the self-optimization feedback based on the reinforcement learning algorithm to obtain a self-healing control result of the industrial control system.
Optionally, the step S3 may include the following steps S31 to S36:
s31, acquiring input and output data in the operation process of the industrial control system.
In a possible implementation, step S31 is the same as step S21, and input/output data during operation is collected and stored.
S32, designing a feedback supplement type self-healing control law which does not change the performance recovery of the industrial control system of the original controller.
In one possible implementation, the design performance recovery self-healing control law is in the form of:
(14)
Original controller Unchanged and define self-healing control law/>Parameterized form.
S33, further obtaining a Q value function related to a self-healing control law in a performance recovery mechanism of the industrial control system according to the performance index of the industrial control system defined in the step S22 of the first functional module and the state space representation form of the closed-loop control system described in the step S23.
In a possible embodiment, the following Q-value function is defined:
(15)
further, it is rewritten as a result of And/>The form represented: /(I)
(16)
Wherein,,/>,,/>,/>。
To this end, a control law about self-healing recovery is obtainedExpressed as a Q function of (c).
S34, obtaining a self-healing control law parameter representation form according to the Q value function obtained in the step three to minimize the value of the Q value function.
In a possible embodiment, the control law is recovered with respect to self-healing obtained according to step S33In order to minimize its value, the Q-function expression is required for/>Is zero, i.e。
Further, according to the correlation matrix in step S33The self-healing control law expression form is obtained as follows:
(17)
S35, in a model-free mode, based on the historical operation data collected and stored in the first step, the self-healing control law related parameters are solved through a Q-learning algorithm.
Optionally, the step S35 may include the following steps S351-S355:
s351, rewriting the Q value function of the self-healing control law to obtain the Q value function of the self-healing control law after rewriting.
In a possible implementation, to implement model-free solution, according to equation (9) of step S24 in the functional module one, the Q-value function in step three is expressed as follows:
(18)
s352, obtaining the self-healing control law representation according to the representation form, the parameter representation form and the Q value function of the rewritten self-healing control law.
In a possible implementation manner, according to the self-healing control law parameterized form designed in step S32 and the self-healing control law expression in step S34, the parameters of the self-healing control law are obtained according to the description form of (18):,/>。
Namely, the self-healing control law is expressed as:
(19)
s353, constructing a representation form of the time sequence difference TD error:
(20)
vectorizing the obtained product to obtain the product:
(21)
Wherein,
,。
S354, minimizing TD error according to Q-learning algorithm, and performing recursive least square identification according to the operation data collected in step S31 and data with a certain step length to obtainThe expression is as follows:
(22)
when TD (Temporal Difference) errors meet the minimum, output Further can obtain the self-healing recovery control law/>。
S36, triggering a performance degradation recovery mechanism in time according to the monitoring result of the first functional module, and calling the self-healing control law parameter to realize performance recovery.
The invention constructs the performance index of the closed-loop control system based on the Belman equation, designs the control performance evaluation matrix, and constructs reasonable performance degradation monitoring logic; the performance evaluation is performed after the recognition system model is replaced by a direct recognition evaluation matrix, so that errors caused by redundant recognition processes are avoided; and further designing a control performance degradation recovery method based on the reinforcement learning theory by utilizing the performance index in the form of the Bellman equation.
Aiming at the situation that the common industrial control system performance monitoring method is conservative in designing and evaluating the benchmark and the situation that part of the evaluating benchmark depends on an accurate mathematical model of the industrial process, the control input is added to the design process of the benchmark, and meanwhile, the industrial control system performance evaluating benchmark is built by using only measured data. Furthermore, the performance evaluation matrix of the closed loop control system is directly identified, so that identification errors caused by a performance evaluation mode of identifying system parameters and then performing LQG reference design are reduced, and performance evaluation accuracy is improved.
Aiming at the problem of control performance decline caused by abnormal operation conditions of an industrial process in a complex environment, the design of a feedback supplement based on a reinforcement learning method is realized by considering that a pre-designed controller or an algorithm program is solidified in hardware equipment and is not easy to modify and adjust, and the performance recovery of a control system is realized by using data only on the premise of not changing the original controller.
The invention aims to solve the problems of industrial control system performance monitoring and self-healing recovery in a complex operating environment. Comprising the following steps: a control system performance evaluation matrix and monitoring scheme based on a Bellman equation and a control system performance self-healing control method based on a reinforcement learning method are designed. Real-time performance monitoring is realized by utilizing the data of the industrial operation process under the condition of not depending on the controlled object and the mathematical model of the control system; the original controller structural parameters are not changed, and only the data and reinforcement learning method is used for designing feedback compensation to recover performance degradation, so that self-healing control is realized.
In the embodiment of the invention, only input and output data are utilized, the performance of the control system in the industrial process is monitored under the condition of not depending on the controlled object and the accurate mathematical model of the control system, and meanwhile, the performance of the control system in an abnormal running state is recovered to a certain extent by a data driving feedback mode based on a reinforcement learning method under the premise of not changing a preset controller, so that self-healing control is realized.
FIG. 3 is a block diagram of an industrial control system performance monitoring and self-healing control device for an industrial control system performance monitoring and self-healing control method, according to an exemplary embodiment. Referring to fig. 3, the apparatus includes an acquisition module 310, a performance monitoring module 320, and a self-healing control module 330. Wherein:
An acquisition module 310 is configured to acquire real-time operation data of an industrial control system to be controlled.
The performance monitoring module 320 is configured to obtain a real-time performance evaluation result of the industrial control system according to the real-time operation data and the performance degradation monitoring module based on the Bellman equation, and obtain a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result.
The self-healing control module 330 is configured to supplement the self-healing control module with self-optimization feedback based on the reinforcement learning algorithm according to the performance degradation monitoring result, and obtain a self-healing control result of the industrial control system.
Optionally, the performance monitoring module 320 is further configured to:
S21, acquiring input and output data in the operation process of the industrial control system.
S22, building performance indexes of the industrial control system according to the input and output data.
S23, obtaining a performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation and the performance index.
S24, designing an identification method of a performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix.
S25, obtaining the performance monitoring reference matrix of the industrial control system according to the input and output data and the identification method of the performance monitoring matrix.
S26, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the identification method of the performance monitoring matrix.
And S27, obtaining performance degradation monitoring indexes of the industrial control system according to the performance monitoring reference matrix and the real-time performance evaluation result, and obtaining the performance degradation monitoring result of the industrial control system according to the performance degradation monitoring indexes and the designed monitoring logic.
Optionally, the performance index of the industrial control system is as shown in the following formula (1):
(1)
In the method, in the process of the invention, Representing performance index of industrial control system,/>Representing discount factors,/>Representing the sampling instant of the data,/>,/>Tracking error matrix representing system output,/>Representing the transpose of the matrix,/>Representing system output data,/>Representing reference input,/>Representing a tracking error weight matrix,/>Representing a matrix of control input weights,/>Representing system input data.
Optionally, the performance monitoring module 320 is further configured to:
S231, constructing an augmentation system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller and the mathematical description of the reference input generator.
S232, rewriting the performance index into a recursive form of a Bellman equation.
S233, obtaining a parameterized form of the performance index of the industrial control system according to the augmentation system, the recursion form and the Bellman optimality principle.
S234, obtaining a performance evaluation matrix of the industrial control system according to the parameterized form, wherein the performance evaluation matrix is shown in the following formula (2):
(2)
In the method, in the process of the invention, Performance evaluation matrix representing an industrial control system,/>Representing a matrix of weights of the tracking error,Representing an output-related parameter matrix of an augmentation system,/>,Representing the reference input state quantity,/>Representing the state quantity of the system,/>Representing a matrix of control input weights,Representing an augmented system control input related parameter matrix,/>Output parameter matrix representing reference input,/>Representing feedback control gain,/>Representing the output parameter matrix of the system,/>Representing the discount factor(s),,/>,/>State parameter matrix representing reference input,/>Representing a state parameter matrix of the system,/>,/>Representing a matrix of input parameters of the system.
Optionally, the performance monitoring module 320 is further configured to:
S241, obtaining the rewritten performance index according to the performance evaluation matrix and the performance index.
S242, obtaining the replaced performance index according to the rewritten performance index and the Kalman filter type data model.
S243, vectorizing the replaced performance index to obtain the vectorized performance index.
S244, obtaining a performance monitoring matrix of the industrial control system according to the input and output data, the vectorized performance index and the recursive least square identification method.
Optionally, the self-healing control module 330 is further configured to:
s31, acquiring input and output data in the operation process of the industrial control system.
S32, designing a representation form of a self-healing control law.
S33, defining a Q value function according to the performance index of the industrial control system and the parameterized form of the performance index, and rewriting the Q value function according to the representation form of the self-healing control law to obtain the Q value function of the self-healing control law.
S34, minimizing the Q value function of the self-healing control law to obtain the self-healing control law parameter representation form.
S35, obtaining the self-healing control law according to the input and output data, the Q value function of the self-healing control law, the self-healing control law parameter representation form and the Q-learning algorithm.
S36, obtaining a self-healing control result of the industrial control system according to the performance degradation monitoring result and the self-healing control law.
Optionally, the self-healing control module 330 is further configured to:
s351, rewriting the Q value function of the self-healing control law to obtain the Q value function of the self-healing control law after rewriting.
S352, obtaining the self-healing control law representation according to the representation form, the parameter representation form and the Q value function of the rewritten self-healing control law.
S353, constructing a time sequence difference TD error representation form, and vectorizing the TD error representation form to obtain the vectorized TD error representation form.
S354, obtaining the self-healing control law according to the input and output data, the vectorized TD error representation form, the self-healing control law representation and the Q-learning algorithm.
In the embodiment of the invention, only input and output data are utilized, the performance of the control system in the industrial process is monitored under the condition of not depending on the controlled object and the accurate mathematical model of the control system, and meanwhile, the performance of the control system in an abnormal running state is recovered to a certain extent by a data driving feedback mode based on a reinforcement learning method under the premise of not changing a preset controller, so that self-healing control is realized.
Fig. 4 is a schematic structural diagram of an industrial control system performance monitoring and self-healing control device according to an embodiment of the present invention, where as shown in fig. 4, the industrial control system performance monitoring and self-healing control device may include the industrial control system performance monitoring and self-healing control device shown in fig. 3. Alternatively, the industrial control system performance monitoring and self-healing control device 410 may include the first processor 2001.
Optionally, the industrial control system performance monitoring and self-healing control device 410 may also include a memory 2002 and a transceiver 2003.
The first processor 2001 may be connected to the memory 2002 and the transceiver 2003, for example, via a communication bus.
The various constituent elements of the industrial control system performance monitoring and self-healing control device 410 are described in detail below in conjunction with FIG. 4:
The first processor 2001 is a control center of the industrial control system performance monitoring and self-healing control device 410, and may be one processor or a generic name of a plurality of processing elements. For example, the first processor 2001 is one or more central processing units (central processing unit, CPU), may be an Application SPECIFIC INTEGRATED Circuit (ASIC), or may be one or more integrated circuits configured to implement embodiments of the present invention, such as: one or more microprocessors (DIGITAL SIGNAL processors, DSPs), or one or more field programmable gate arrays (field programmable GATE ARRAY, FPGAs).
Alternatively, the first processor 2001 may perform various functions of the industrial control system performance monitoring and self-healing control device 410 by running or executing a software program stored in the memory 2002 and invoking data stored in the memory 2002.
In a specific implementation, first processor 2001 may include one or more CPUs, such as CPU0 and CPU1 shown in fig. 4, as an example.
In a particular implementation, as one example, the industrial control system performance monitoring and self-healing control device 410 may also include a plurality of processors, such as the first processor 2001 and the second processor 2004 shown in FIG. 4. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
The memory 2002 is used for storing a software program for executing the solution of the present invention, and is controlled by the first processor 2001 to execute the solution, and the specific implementation may refer to the above method embodiment, which is not described herein.
Alternatively, memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ-only memory, EEPROM), compact disc read-only memory (compact disc read-only memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, without limitation. The memory 2002 may be integrated with the first processor 2001, may be present independently, and may be coupled to the first processor 2001 through an interface circuit (not shown in fig. 4) of the industrial control system performance monitoring and self-healing control device 410, as embodiments of the present invention are not specifically limited in this regard.
A transceiver 2003 for communicating with a network device or with a terminal device.
Alternatively, transceiver 2003 may include a receiver and a transmitter (not separately shown in fig. 4). The receiver is used for realizing the receiving function, and the transmitter is used for realizing the transmitting function.
Alternatively, the transceiver 2003 may be integrated with the first processor 2001, or may exist separately, and be coupled to the first processor 2001 through an interface circuit (not shown in fig. 4) of the industrial control system performance monitoring and self-healing control device 410, as embodiments of the present invention are not limited in this regard.
It should be noted that the structure of the industrial control system performance monitoring and self-healing control device 410 shown in fig. 4 is not limited to this router, and an actual knowledge structure recognition device may include more or less components than illustrated, or may combine certain components, or may have a different arrangement of components.
In addition, the technical effects of the industrial control system performance monitoring and self-healing control device 410 may refer to the technical effects of the industrial control system performance monitoring and self-healing control method described in the above method embodiments, and are not described herein again.
It is to be appreciated that the first processor 2001 in embodiments of the invention may be a central processing unit (central processing unit, CPU) which may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, DSPs), application Specific Integrated Circuits (ASICs), off-the-shelf programmable gate arrays (field programmable GATE ARRAY, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It should also be appreciated that the memory in embodiments of the present invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an erasable programmable ROM (erasable PROM), an electrically erasable programmable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of random access memory (random access memory, RAM) are available, such as static random access memory (STATIC RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (double DATA RATE SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and direct memory bus random access memory (direct rambus RAM, DR RAM).
The above embodiments may be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. In addition, the character "/" herein generally indicates that the associated object is an "or" relationship, but may also indicate an "and/or" relationship, and may be understood by referring to the context.
In the present invention, "at least one" means one or more, and "a plurality" means two or more. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.
It should be understood that, in various embodiments of the present invention, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working procedures of the apparatus, device and unit described above may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another device, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. An industrial control system performance monitoring and self-healing control method, which is characterized by comprising the following steps:
s1, acquiring real-time operation data of an industrial control system to be controlled;
S2, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and a performance degradation monitoring module based on a Bellman equation, and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result;
and S3, supplementing a self-healing control module according to the performance degradation monitoring result and the self-optimization feedback based on the reinforcement learning algorithm to obtain a self-healing control result of the industrial control system.
2. The method for monitoring and self-healing control of performance of an industrial control system according to claim 1, wherein the performance degradation monitoring module in S2 obtains a real-time performance evaluation result of the industrial control system according to the real-time operation data and Bellman equation, and obtains a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result, comprising:
S21, acquiring input and output data in the operation process of the industrial control system;
s22, building performance indexes of the industrial control system according to the input and output data;
s23, obtaining a performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation and the performance index;
s24, designing an identification method of a performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix;
s25, obtaining a performance monitoring reference matrix of the industrial control system according to the input and output data and the identification method of the performance monitoring matrix;
s26, obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the identification method of the performance monitoring matrix;
And S27, obtaining a performance degradation monitoring index of the industrial control system according to the performance monitoring reference matrix and the real-time performance evaluation result, and obtaining a performance degradation monitoring result of the industrial control system according to the performance degradation monitoring index and the designed monitoring logic.
3. The method for monitoring and self-healing control of performance of an industrial control system according to claim 2, wherein the performance index of the industrial control system in S22 is represented by the following formula (1):
(1)
In the method, in the process of the invention, Representing performance index of industrial control system,/>Representing discount factors,/>Representing the sampling instant of the data,/>,/>Tracking error matrix representing system output,/>Representing the transpose of the matrix,/>Representing system output data,/>Representing reference input,/>Representing a tracking error weight matrix,/>Representing a matrix of control input weights,/>Representing system input data.
4. The method for monitoring and self-healing control of performance of an industrial control system according to claim 2, wherein the obtaining the performance evaluation matrix of the industrial control system according to the mathematical description of the controlled object of the industrial control system, the mathematical description of the controller, the mathematical description of the reference input generator, the Bellman equation, and the performance index in S23 includes:
s231, constructing an augmentation system according to mathematical description of a controlled object of the industrial control system, mathematical description of a controller and mathematical description of a reference input generator;
s232, rewriting the performance index into a recursive form of a Bellman equation;
s233, obtaining a parameterized form of the performance index of the industrial control system according to the augmentation system, the recursion form and the Bellman optimality principle;
S234, obtaining a performance evaluation matrix of the industrial control system according to the parameterized form, wherein the performance evaluation matrix is shown in the following formula (2):
(2)
In the method, in the process of the invention, Performance evaluation matrix representing an industrial control system,/>Representing a matrix of weights of the tracking error,Representing an output-related parameter matrix of an augmentation system,/>,Representing the reference input state quantity,/>Representing the state quantity of the system,/>Representing a matrix of control input weights,Representing an augmented system control input related parameter matrix,/>Output parameter matrix representing reference input,/>Representing feedback control gain,/>Representing the output parameter matrix of the system,/>Representing the discount factor(s),,/>,/>State parameter matrix representing reference input,/>Representing a state parameter matrix of the system,/>,/>Representing a matrix of input parameters of the system.
5. The method for monitoring and self-healing control of performance of an industrial control system according to claim 2, wherein the identifying method for designing the performance monitoring matrix of the industrial control system according to the performance index and the performance evaluation matrix in S24 comprises:
s241, obtaining rewritten performance indexes according to the performance evaluation matrix and the performance indexes;
s242, obtaining a replaced performance index according to the rewritten performance index and a Kalman filter form data model;
S243, vectorizing the replaced performance index to obtain a vectorized performance index;
S244, obtaining a performance monitoring matrix of the industrial control system according to the input and output data, the vectorized performance index and the recursive least square identification method.
6. The method for monitoring and self-healing control of industrial control system according to claim 1, wherein the step of obtaining the self-healing control result of the industrial control system according to the performance degradation monitoring result and the self-optimization feedback supplementary self-healing control module based on the reinforcement learning algorithm in step S3 comprises the following steps:
s31, acquiring input and output data in the operation process of the industrial control system;
s32, designing a representation form of a self-healing control law;
S33, defining a Q value function according to the performance index of the industrial control system and the parameterized form of the performance index, and rewriting the Q value function according to the representation form of the self-healing control law to obtain the Q value function of the self-healing control law;
s34, minimizing the Q value function of the self-healing control law to obtain a self-healing control law parameter representation form;
S35, obtaining a self-healing control law according to the input and output data, the Q value function of the self-healing control law, the self-healing control law parameter representation form and the Q-learning algorithm;
s36, obtaining a self-healing control result of the industrial control system according to the performance degradation monitoring result and the self-healing control law.
7. The method for monitoring and self-healing control of industrial control system according to claim 6, wherein the step of obtaining the self-healing control law according to the input/output data, the Q-value function of the self-healing control law, the parameter representation of the self-healing control law and the Q-learning algorithm in S35 comprises:
S351, rewriting the Q value function of the self-healing control law to obtain the Q value function of the rewritten self-healing control law;
S352, obtaining a self-healing control law representation according to the representation form of the self-healing control law, the representation form of the self-healing control law parameter and the Q value function of the self-healing control law after rewriting;
s353, constructing a time sequence difference TD error representation form, and vectorizing the TD error representation form to obtain a vectorized TD error representation form;
S354, obtaining the self-healing control law according to the input and output data, the vectorized TD error expression form, the self-healing control law expression and the Q-learning algorithm.
8. An industrial control system performance monitoring and self-healing control device for implementing the industrial control system performance monitoring and self-healing control method according to any one of claims 1 to 7, characterized in that the device comprises:
the acquisition module is used for acquiring real-time operation data of the industrial control system to be controlled;
The performance monitoring module is used for obtaining a real-time performance evaluation result of the industrial control system according to the real-time operation data and the performance degradation monitoring module based on the Bellman equation, and obtaining a performance degradation monitoring result of the industrial control system according to the real-time performance evaluation result;
and the self-healing control module is used for supplementing the self-healing control module according to the performance degradation monitoring result and the self-optimization feedback based on the reinforcement learning algorithm to obtain the self-healing control result of the industrial control system.
9. An industrial control system performance monitoring and self-healing control device, characterized in that the industrial control system performance monitoring and self-healing control device comprises:
A processor;
a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 7.
10. A computer readable storage medium having stored therein program code which is callable by a processor to perform the method of any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410396935.XA CN117991647B (en) | 2024-04-03 | 2024-04-03 | Performance monitoring and self-healing control method and device for industrial control system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202410396935.XA CN117991647B (en) | 2024-04-03 | 2024-04-03 | Performance monitoring and self-healing control method and device for industrial control system |
Publications (2)
Publication Number | Publication Date |
---|---|
CN117991647A true CN117991647A (en) | 2024-05-07 |
CN117991647B CN117991647B (en) | 2024-06-21 |
Family
ID=90900879
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202410396935.XA Active CN117991647B (en) | 2024-04-03 | 2024-04-03 | Performance monitoring and self-healing control method and device for industrial control system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117991647B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20110067418A (en) * | 2009-12-14 | 2011-06-22 | 한국전자통신연구원 | System and method for monitoring and evaluating a self-healing system |
CN109460890A (en) * | 2018-09-21 | 2019-03-12 | 浙江大学 | A kind of intelligent self-healing method based on intensified learning and control performance monitoring |
CN110109355A (en) * | 2019-04-29 | 2019-08-09 | 山东科技大学 | A kind of unmanned boat unusual service condition self-healing control method based on intensified learning |
CN113721469A (en) * | 2021-09-03 | 2021-11-30 | 华东理工大学 | Self-healing control method and system for additive faults of sensor of sewage treatment system |
CN113721468A (en) * | 2021-09-03 | 2021-11-30 | 华东理工大学 | Self-healing control method and system for additive fault of sewage treatment system actuator |
CN117170236A (en) * | 2023-09-12 | 2023-12-05 | 浙江大学 | Industrial speed regulation system actuator fault intelligent self-healing method based on inverse reinforcement learning |
-
2024
- 2024-04-03 CN CN202410396935.XA patent/CN117991647B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20110067418A (en) * | 2009-12-14 | 2011-06-22 | 한국전자통신연구원 | System and method for monitoring and evaluating a self-healing system |
CN109460890A (en) * | 2018-09-21 | 2019-03-12 | 浙江大学 | A kind of intelligent self-healing method based on intensified learning and control performance monitoring |
CN110109355A (en) * | 2019-04-29 | 2019-08-09 | 山东科技大学 | A kind of unmanned boat unusual service condition self-healing control method based on intensified learning |
CN113721469A (en) * | 2021-09-03 | 2021-11-30 | 华东理工大学 | Self-healing control method and system for additive faults of sensor of sewage treatment system |
CN113721468A (en) * | 2021-09-03 | 2021-11-30 | 华东理工大学 | Self-healing control method and system for additive fault of sewage treatment system actuator |
CN117170236A (en) * | 2023-09-12 | 2023-12-05 | 浙江大学 | Industrial speed regulation system actuator fault intelligent self-healing method based on inverse reinforcement learning |
Also Published As
Publication number | Publication date |
---|---|
CN117991647B (en) | 2024-06-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Wu et al. | Data-driven remaining useful life prediction via multiple sensor signals and deep long short-term memory neural network | |
Yin et al. | An improved incremental learning approach for KPI prognosis of dynamic fuel cell system | |
CN117991647B (en) | Performance monitoring and self-healing control method and device for industrial control system | |
CN115390459A (en) | Model prediction control method and device | |
Chen et al. | Multi-scale memory-enhanced method for predicting the remaining useful life of aircraft engines | |
CN112911533B (en) | Temperature detection system based on remove end App | |
CN112270454B (en) | Method and device for predicting short-term load of power system under influence of extreme factors | |
Mu et al. | Catalyst optimization design based on artificial neural network | |
CN117289668A (en) | Distributed speed reducer network cooperative control method, device, equipment and storage medium | |
CN117352762A (en) | Control system of battery manufacturing equipment | |
Zhou et al. | Identification-based sensor and actuator fault diagnosis for industrial control systems and its application to HTR-PM | |
Lu et al. | A dual path hybrid neural network framework for remaining useful life prediction of aero‐engine | |
CN113221064B (en) | Model processing method, device, computer equipment and storage medium | |
CN103631991A (en) | Parameter identification system and method of prime mover speed regulating system | |
Suskis et al. | Converter state-space model estimation using dynamic mode decomposition | |
Yi et al. | Process Quality Prediction Algorithm of Multi output Workshop Based on ATT-CNN-TCN | |
Gu et al. | Improved similarity-based residual life prediction method based on grey Markov model | |
CN115630563A (en) | Photovoltaic array fault detection method and device and electronic equipment | |
Agnihotri et al. | Optimal tuning of PID controller with time delay system using CS and SRMR technique | |
CN118035832A (en) | GSO-LSTM-based power grid harmonic source identification method and device | |
CN118099347A (en) | Battery pole piece rolling device and running state prediction system | |
CN116938613B (en) | Dormancy method of monitoring sensor based on nonlinear information age and electronic equipment | |
CN115598973A (en) | Nonlinear sliding mode prediction control method, device, equipment and storage medium | |
Zhang et al. | Transformer Fault Diagnosis Model and Method Based on DBNI in Photoelectric Sensors Diagnosis System | |
CN117892463B (en) | Low-voltage distribution network design system based on load prediction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |