WO2021251906A1 - Method and system for detecting anomaly in a physical process associated with a networked control system - Google Patents

Method and system for detecting anomaly in a physical process associated with a networked control system Download PDF

Info

Publication number
WO2021251906A1
WO2021251906A1 PCT/SG2021/050340 SG2021050340W WO2021251906A1 WO 2021251906 A1 WO2021251906 A1 WO 2021251906A1 SG 2021050340 W SG2021050340 W SG 2021050340W WO 2021251906 A1 WO2021251906 A1 WO 2021251906A1
Authority
WO
WIPO (PCT)
Prior art keywords
state
physical
skew
run
runs
Prior art date
Application number
PCT/SG2021/050340
Other languages
French (fr)
Inventor
Chuadhry Mujeeb AHMED
Jianying Zhou
Original Assignee
Singapore University Of Technology And Design
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Singapore University Of Technology And Design filed Critical Singapore University Of Technology And Design
Publication of WO2021251906A1 publication Critical patent/WO2021251906A1/en

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/05Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
    • G05B19/058Safety, monitoring
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/20Pc systems
    • G05B2219/24Pc safety
    • G05B2219/24215Scada supervisory control and data acquisition
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the present invention generally relates to a method and a system for detecting anomaly in a physical process associated with a networked control system, and more particularly, for detecting an attack on the networked control system.
  • An Industrial Control System is a networked control system comprising sensors, actuators, controllers and communication networks configured to control one or more physical processes in an industry, such as water treatment, water distribution, smart grid, autonomous transportation, and so on.
  • connectivity in an ICS provides improved monitoring and operation of a physical process.
  • Such advancements are helpful but also bring about challenges of secure operation of the connected devices for the ICS to operate securely.
  • an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data, and manipulate the actuators to cause anomaly of operations, which may eventually lead to physical damages to the ICS.
  • traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic.
  • sensor data may be transmitted to a Programmable Logic Controller (PLC) to perform an appropriate action based on the sensor measurement.
  • PLC Programmable Logic Controller
  • an adversary can spoof sensor data in the digital or physical domain, the adversary can cause the ICS to go to an unsafe state.
  • the focus here is not on the confidentiality of the sensor data as in legacy computer security but the integrity and trustworthiness of the sensor data.
  • a method of detecting anomaly in a physical process associated with a networked control system using at least one processor comprising: obtaining, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detecting anomaly in the physical process based on the process ske
  • a system for detecting anomaly in a physical process associated with a networked control system comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; produce, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detect
  • a computer program product embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform the method of detecting anomaly in a physical process according to the above-mentioned first aspect of the present invention.
  • FIG. 1 depicts a schematic flow diagram of a method of detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention
  • FIG. 2 depicts a schematic block diagram of a system for detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention, such as corresponding to the method of detecting anomaly in a physical process as described with reference to FIG. 1;
  • FIG. 3 depicts a schematic block diagram of an exemplary computer system which may be used to realize or implement the system as described with reference to FIG. 2;
  • FIG. 4 depicts a schematic drawing of a Secure Water Treatment Testbed (SWaT) Network Architecture, based on which the method of detecting anomaly in a physical process may be applied as an illustrative example, according to various example embodiments of the present invention for illustration purpose;
  • SWaT Secure Water Treatment Testbed
  • FIG. 5 depicts a schematic flow diagram illustrating an overview of a process skew based detection method according to various example embodiments of the present invention, such as corresponding to the method of detecting anomaly in a physical process as described with reference to FIG. 1;
  • FIG. 6 depicts a schematic drawing showing an abstraction of an industrial control system (ICS) in relation to a thread model;
  • ICS industrial control system
  • FIGs. 7A and 7B depict a table (Table 1) providing a list of attacks on the SWaT testbed that may be carried out in experiments conducted, according to various example embodiments of the present invention
  • FIG. 8 depicts a plot of level sensor (LIT-101) measurements in the SWaT testbed in stage 1 for a duration of a normal process (i.e., under normal operation);
  • FIG. 9 shows a table (Table 2) providing a list of example design parameters for each type of state process or control action, according to various example embodiments of the present invention
  • FIGs. 10A to 10D depict four plots showing four possible state processes of a physical process, respectively, along with measured sensor data obtained with respect to the water level in a water tank (tank 1) as described with reference to Table 2, according to various example embodiments of the present invention
  • FIG. 11 depicts a plot illustrating the concept of process offsets (or process skews) associated with a physical process, according to various example embodiments of the present invention
  • FIG. 12 depicts a schematic diagram of an example physical system for stage 1 of the SWaT testbed, according to various example embodiments of the present invention
  • FIGs. 13A to 13C depict process offset data obtained based on measured sensor data for different state processes at stage 1 of the SWaT testbed, according to various example embodiments of the present invention
  • FIG. 14 depicts a plot of a plurality of process offset data obtained based on measured sensor data for the water filing process in stage 1 of the SWaT testbed, with a linear regression model fitted thereon, according to various example embodiments of the present invention
  • FIGs. 15A to 15C depict process skew fingerprint distributions for eight state processes at three stages of the SWaT testbed, according to various example embodiments of the present invention
  • FIG. 16 illustrates mutual information across the eight process skew fingerprints discussed with reference to FIGs. 15A to 15C, according to various example embodiments of the present invention
  • FIG. 17 depicts a table (Table 3) showing the model accuracies for three stages of the SWaT and corresponding state processes used in experiments conducted (from the SWaT testbed), according to various example embodiments of the present invention
  • FIG. 18 depicts a table (Table 4) showing example design and performance of a cumulative sum (CUSUM) detector on the normal operation data, according to various example embodiments of the present invention
  • FIG. 19 depicts a plot of process offsets obtained based on measured sensor data on process runs in relation to tank 4 in stage 4 of the SWaT testbed, including a mixture of normal process runs and a few attacks, according to various example embodiments of the present invention
  • FIGs. 20A and 20B depict plots in relation to an attack detection example for an outflow process in relation to level sensor LIT-401, according to various example embodiments of the present invention
  • FIG. 21 depicts a table (Table 5) showing evaluation results of the process skew based detection method, according to various example embodiments of the present invention, on attack data from the SWaT testbed;
  • FIGs. 22 A and 22B show an example execution of a stealthy attack on stage 1 of the SWaT testbed, according to various example embodiments of the present invention.
  • Various embodiments of the present invention provide a method and a system for detecting anomaly in a physical process associated with a networked control system, and more particularly, for detecting an attack on the networked control system.
  • the networked control system may be implemented in any industries (industrial applications) as an Industrial Control System (ICS) as desired or as appropriate that requires an industrial process control, such but not limited to, water treatment, chemical processing, power generation, oil and gas processing, and so on.
  • the networked control system may comprise a sensor network comprising a plurality of sensors.
  • a sensor network may comprise a plurality of sensors that are spatially positioned or installed in the networked control system, each being arranged or positioned for monitoring and collecting sensor data (e.g., measurements or readings, which may be referred to herein as measured sensor data) relating to one or more process states of one or more physical processes associated with the networked control system, such as relating to a physical condition or a physical property of a surrounding environment (e.g., in relation to a medium or an object), such as but not limited to, temperature, sound, pressure, fluid flow rate, fluid level and so on.
  • sensor data e.g., measurements or readings, which may be referred to herein as measured sensor data
  • process states of one or more physical processes associated with the networked control system such as relating to a physical condition or a physical property of a surrounding environment (e.g., in relation to a medium or an object), such as but not limited to, temperature, sound, pressure, fluid flow rate, fluid level and so on.
  • the plurality of sensors may be communicatively coupled to a processor (e.g., a central processor or a sensor data processor, such as a Programmable Logic Controller (PLC)) based on any communications technologies known in the art, such as wired communications technologies or wireless communications technologies, and need not be described herein.
  • a processor e.g., a central processor or a sensor data processor, such as a Programmable Logic Controller (PLC)
  • PLC Programmable Logic Controller
  • each sensor may be any type of sensor known in the art configured for capturing a physical condition or physical property of a surrounding environment and outputting corresponding measured sensor data in relation to the surrounding environment.
  • an attack on the networked control system may refer to any type of security or malicious attack on the networked control system known in the art and need not be described herein, such as an attack on one or more sensors in the sensor network.
  • Various possible types of attack on a networked control system are known in the art, and for illustration purpose, example types of attack will be described later below according to various example embodiments of the present invention.
  • an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data and may manipulate the actuators, which may result in anomaly of operations and eventually lead to physical damages to the ICS.
  • traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic.
  • sensor data may be transmitted to a PLC to perform an appropriate action based on the sensor measurement.
  • the adversary can cause the ICS to go to an unsafe state. Therefore, the integrity and trustworthiness of the sensor data are important for the ICS to operate securely.
  • various embodiments of the present invention provide a method and a system for detecting anomaly in a physical process associated with a networked control system, that seek to overcome, or at least ameliorate, one or more problems relating to conventional method and a system for detecting anomaly in a physical process, and more particularly, enabling or improving anomaly detection (in particular, attack detection) in a physical process associated with a networked control system in an effective manner.
  • anomaly detection in particular, attack detection
  • FIG. 1 depicts a schematic flow diagram of a method 100 of detecting anomaly in a physical process associated with a networked control system using at least one processor, according to various embodiments of the present invention.
  • the method 100 comprises: obtaining (at 102), for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing (at 104), for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining (at 106) process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information
  • the measured sensor data associated with a process run may be obtained from one or more sensors of the sensor network of the networked control system.
  • a physical process associated with a network control system may comprise one or more process states, and for each process state, there may be one or more state processes (i.e., one or more types of state processes) that may be performed (or may occur) at the process state.
  • state processes i.e., one or more types of state processes
  • a plurality of process runs (or process instances) for the state process may be performed (or may occur).
  • a water treatment plant there may be a plurality of process stages, such as a raw water storage stage to hold the raw water for treatment, a chemical dosing stage to treat the water, an ultra-filtrating stage, and so on.
  • a raw water storage stage to hold the raw water for treatment
  • a chemical dosing stage to treat the water
  • an ultra-filtrating stage and so on.
  • the raw water storage stage there may be an associated physical process comprising a plurality of process states, such as a water emptying state, a water filling state and a water filling and emptying state.
  • the one or more types of processes performed at each process state may thus be one or more state processes (i.e., one or more types of state processes) at the process state.
  • a state process e.g., water emptying process
  • a process state e.g., water emptying state
  • a plurality of process runs e.g., hundreds of process runs (or process instances) of the water emptying process over time
  • measured sensor data associated with each of the plurality of process runs in relation to a physical state e.g., water level in a water tank
  • the networked control system may be configured to perform or control a physical process, and each process stage of the networked control system may thus be configured to perform or control an associated physical sub-process.
  • the modelled sensor data in relation to the physical state associated with the state process may be sensor data in relation to the physical state that is expected or intended (e.g., configured or designed to be) based on design parameters associated with the state process.
  • the modelled sensor data in relation to the water level during the water emptying process may be sensor data in relation to the water level that is expected over time during the water emptying process based on design parameters associated with water emptying process in relation to the water tank.
  • sensor data in relation to a physical state that is expected may be determined by estimating the sensor data in relation to the physical state based on a system model.
  • characteristic information associated with the plurality of process offset data associated with the plurality of process runs for the state process may be determined (e.g., extracted) so as to produce the process skew fingerprint information associated with the plurality of process runs.
  • the characteristic information may be a slope parameter associated with a linear regression model applied to the plurality of process offset data.
  • the method 100 of detecting anomaly in the physical process for additional one or more other state processes at the process state of the physical process, or at one or more other process states of the physical process may be performed in the same or similar manner as described herein in relation to the above- mentioned state process at the above-mentioned process state of the physical process. It will also be understood by a person skilled in the art that the method 100 of detecting anomaly in additional one or more physical processes associated with the networked control system may also be performed in the same or similar manner as described herein in relation to the above- mentioned state process at the above-mentioned process state of the physical process.
  • the method 100 of detecting anomaly in a physical process is based on a plurality of process offset data (e.g., which may be referred to herein as a plurality of process skew data) associated with a plurality of process runs for a state process at a process state of the physical process. Therefore, the method 100 of detecting anomaly in a physical process is advantageously based on inaccuracies (e.g., deviations due to an attack) in the physical process itself, and thus is based on physical process dynamics.
  • a plurality of process offset data e.g., which may be referred to herein as a plurality of process skew data
  • inaccuracies e.g., deviations due to an attack
  • a plurality of process skew data associated with a plurality of process runs for a state process of a physical process is produced and used to fingerprint the plurality of process runs for the state process, so as to produce process skew fingerprint information for detecting anomaly in the state process of the physical process.
  • Such a technical approach in detecting anomaly in the physical process has been found to enable or improve anomaly detection (in particular, attack detection) in a physical process associated with a networked control system in an effective manner.
  • the method 100 further comprises determining the modelled sensor data in relation to the physical state associated with the state process, comprising estimating sensor data in relation to the physical state associated with the state process based on a system model representing the state process and one or more design parameters associated with the state process.
  • process offset data associated with the process run comprises: determining, for each time step of a plurality of time steps, a process offset between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain a plurality of process offset information associated with the process run at the plurality of time steps, respectively.
  • the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
  • the above-mentioned determining, for the above- mentioned each time step of the plurality of time steps, the process offset comprises determining, for the above-mentioned each time step of the plurality of time steps, a difference between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain the plurality of process offset information associated with the process run at the plurality of time steps, respectively.
  • the above-mentioned determining (at 106) the process skew fingerprint information associated with the plurality of process runs comprises determining a slope parameter associated with the plurality of process offset data associated with the plurality of process runs in relation to the physical state associated with the state process.
  • the process skew fingerprint information comprises the slope parameter.
  • the above-mentioned determining the slope parameter associated with the plurality of process offset data comprising applying a linear regression model to the plurality of process offset data to obtain a regression coefficient.
  • the slope parameter comprises the regression coefficient.
  • the above-mentioned detecting (at 108) anomaly in the physical process comprises detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process and a reference process skew fingerprint information associated with the state process at the process state of the physical process.
  • the method 100 further comprises determining the reference process skew fingerprint information associated with the state process at the process state of the physical process, comprising: obtaining, for each reference process run of a plurality of reference process runs for the state process at the process state of the physical process, measured sensor data associated with the reference process run in relation to the physical state associated with the state process at the process state; producing, for said each reference process run of the plurality of reference process runs, reference process offset data associated with the reference process run based on the measured sensor data associated with the reference process run and the modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of reference process offset data associated with the plurality of reference process runs, respectively, in relation to the physical state associated with the state process; and determining the reference process skew fingerprint information associated with the plurality of reference process runs based on the plurality of reference process offset data, the reference process skew fingerprint information comprising characteristic information associated with the plurality of reference process offset data.
  • the reference process skew comprising characteristic information associated with the plurality of reference
  • the above-mentioned reference process skew fingerprint information associated with the state process may be determined in the same or similar manner as the above-mentioned process skew fingerprint information associated with the state process determined at 106, except that the above-mentioned reference process skew fingerprint information is determined based on measured sensor data associated with the plurality of reference process runs for the state process.
  • the above-mentioned detecting anomaly in the physical process comprises detecting an attack on the networked control system based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
  • FIG. 2 depicts a schematic block diagram of a system 200 for detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention, such as corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system as described hereinbefore with reference to FIG. 1 according to various embodiments of the present invention.
  • the system 200 comprises: a memory 202; and at least one processor 204 communicatively coupled to the memory 202 and configured to: obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; produce, for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detect anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process
  • the at least one processor 204 may be configured to perform various functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 204 to perform various functions or operations. Accordingly, as shown in FIG.
  • the system 200 may comprise a measured sensor data module (or a measured sensor data circuit) 206 configured to obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; a process offset data producing module (or a process offset data producing circuit) 208 configured to produce, for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; a process skew fingerprint information determining module (a process skew fingerprint information determining circuit) 210 configured to determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information
  • modules are not necessarily separate modules, and one or more modules may be realized by or implemented as one functional module (e.g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention.
  • two or more of the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and the anomaly detection module 212 may be realized (e.g., compiled together) as one executable software program (e.g., software application or simply referred to as an “app”), which for example may be stored in the memory 202 and executable by the at least one processor 204 to perform various functions/operations as described herein according to various embodiments of the present invention.
  • one executable software program e.g., software application or simply referred to as an “app”
  • the system 200 corresponds to the method 100 of detecting anomaly as described hereinbefore with reference to FIG. 1, therefore, various functions or operations configured to be performed by the least one processor 204 may correspond to various steps or operations of the method 100 of detecting anomaly as described herein according to various embodiments, and thus need not be repeated with respect to the system 200 for detecting anomaly for clarity and conciseness.
  • various embodiments described herein in context of the methods are analogously valid for the corresponding systems, and vice versa.
  • the memory 202 may have stored therein the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212, which respectively correspond to various steps (or operations or functions) of the method 100 of detecting anomaly as described herein according to various embodiments, which are executable by the at least one processor 204 to perform the corresponding functions/operations as described herein.
  • a computing system, a controller, a microcontroller or any other system providing a processing capability may be provided according to various embodiments in the present disclosure.
  • Such a system may be taken to include one or more processors and one or more computer-readable storage mediums.
  • the system 200 described hereinbefore may include a processor (or controller) 204 and a computer-readable storage medium (or memory) 202 which are for example used in various processing carried out therein as described herein.
  • a memory or computer-readable storage medium used in various embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).
  • DRAM Dynamic Random Access Memory
  • PROM Programmable Read Only Memory
  • EPROM Erasable PROM
  • EEPROM Electrical Erasable PROM
  • flash memory e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).
  • a “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof.
  • a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g., a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor).
  • a “circuit” may also be a processor executing software, e.g., any kind of computer program, e.g., a computer program using a virtual machine code, e.g., Java.
  • a “module” may be a portion of a system according to various embodiments in the present invention and may encompass a “circuit” as above, or may be understood to be any kind of a logic-implementing entity therefrom.
  • the present specification also discloses a system (e.g., which may also be embodied as a device or an apparatus), such as the system 200, for performing various operations/functions of various methods described herein.
  • a system e.g., which may also be embodied as a device or an apparatus
  • Such a system may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer.
  • the algorithms presented herein are not inherently related to any particular computer or other apparatus.
  • Various general-purpose machines may be used with computer programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform various method steps may be appropriate.
  • the present specification also at least implicitly discloses a computer program or software/functional module, in that it would be apparent to the person skilled in the art that individual steps of various methods described herein may be put into effect by computer code.
  • the computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein.
  • the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the scope of the invention.
  • modules described herein may be software module(s) realized by computer program(s) or set(s) of instructions executable by a computer processor to perform the required functions, or may be hardware module(s) being functional hardware unit(s) designed to perform the required functions. It will also be appreciated that a combination of hardware and software modules may be implemented.
  • a computer program/module or method described herein may be performed in parallel rather than sequentially.
  • Such a computer program may be stored on any computer readable medium.
  • the computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer.
  • the computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the methods described herein.
  • a computer program product embodied in one or more computer-readable storage mediums (non-transitory computer-readable storage medium(s)), comprising instructions (e.g., the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212) executable by one or more computer processors to perform the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein with reference to FIG. 1 according to various embodiments.
  • various computer programs or modules described herein may be stored in a computer program product receivable by a system therein, such as the system 200 as shown in FIG. 2, for execution by at least one processor 204 of the system 200 to perform various functions.
  • a module is a functional hardware unit designed for use with other components or modules.
  • a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC). Numerous other possibilities exist.
  • ASIC Application Specific Integrated Circuit
  • the system 200 may be realized by any computer system (e.g., desktop or portable computer system) including at least one processor and a memory, such as a computer system 300 as schematically shown in FIG. 3 as an example only and without limitation.
  • Various methods/steps or functional modules e.g., the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212 may be implemented as software, such as a computer program being executed within the computer system 300, and instructing the computer system 300 (in particular, one or more processors therein) to conduct various functions or operations as described herein according to various embodiments.
  • the computer system 300 may comprise a computer module 302, input modules, such as a keyboard and/or a touchscreen 304 and a mouse 306, and a plurality of output devices such as a display 308, and a printer 310.
  • the computer module 302 may be connected to a computer network 312 via a suitable transceiver device 314, to enable access to e.g., the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN).
  • the computer module 302 in the example may include a processor 318 for executing various instructions, a Random Access Memory (RAM) 320 and a Read Only Memory (ROM) 322.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • the computer module 302 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 324 to the display 308, and I/O interface 326 to the keyboard 304.
  • I/O Input/Output
  • the components of the computer module 302 typically communicate via an interconnected bus 328 and in a manner known to the person skilled in the relevant art.
  • any reference to an element or a feature herein using a designation such as “first”, “second” and so forth does not limit the quantity or order of such elements or features, unless stated or the context requires otherwise.
  • such designations may be used herein as a convenient way of distinguishing between two or more elements or instances of an element.
  • a reference to first and second elements does not necessarily mean that only two elements can be employed, or that the first element must precede the second element.
  • a phrase referring to “at least one of’ a list of items refers to any single item therein or any combination of two or more items therein.
  • the present invention is not limited to being applied to or implemented in a networked control system for a water treatment plant, and may be applied to or implemented in any industry as desired or as appropriate that may require an industrial process control, such but not limited to, chemical processing, power generation, oil and gas processing, and so on.
  • various example embodiments provide a method (or technique), which may be referred to herein as a process skew based detection method, that uses process offsets (which may also be referred to as process skews, e.g., small deviations) in the ICS process for anomaly detection (e.g., attack detection).
  • process offsets which may also be referred to as process skews, e.g., small deviations
  • the process skew based detection method may be based on fingerprinting a physical process for anomaly detection (i.e., determining process skew fingerprint information (i.e., process skew based fingerprint information), which may be simply referred to herein as a process skew fingerprint).
  • process skew fingerprint information i.e., process skew based fingerprint information
  • various example embodiments determine the process skew fingerprint based on noise in sensor measurements due to the process fluctuations (or process offsets).
  • a process skew fingerprint determined such a manner is unique to a physical process due to the intrinsic operational constraints of the physical process, and is hard to be forged even for a powerful attacker knowing the process operation.
  • the process skew based detection method was validated using data from a real-world water treatment testbed.
  • experimental results obtained demonstrate that the process skew based detection method effectively detected anomaly in a physical process associated with a networked control system (e.g., detected process anomaly with a very low false-positive rate), as well as demonstrate that the process skew based detection method can effectively identify a process (e.g., a state process at a process state of the physical process) based on the process skew fingerprint.
  • a process e.g., a state process at a process state of the physical process
  • An ICS is a networked control system comprising sensors, actuators, controllers and communication networks configured to control one or more physical processes in an industry.
  • connectivity in an ICS provides improved monitoring and operation of a physical process. Such advancements are helpful but also bring about challenges of secure operation of the connected devices for the ICS to operate securely.
  • an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data, and manipulate the actuators to cause anomaly of operations, which may eventually lead to physical damages to the ICS.
  • Traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic.
  • sensor data may be transmitted to a PLC to perform an appropriate action based on the sensor measurement.
  • the adversary can spoof sensor data in the digital or physical domain, the adversary can cause the ICS to go to an unsafe state. Therefore, according to various example embodiments, the focus is not on the confidentiality of the sensor data as in legacy computer security but the integrity and trustworthiness of the sensor data.
  • an open problem with conventional attack detection methods is that it is not possible to localize the source of attacks. This is especially an issue associated with conventional machine learning-based methods that use raw data from a process and feed them to well-known machine learning models.
  • various example embodiments note that, in contrast to the process skew based detection method, these conventional attack detection methods are not aware of or do not take into account process dynamics, and thus, they are unable to determine or locate the physical state where the attack occurred.
  • a process skew based detection method (e.g., corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein according to various embodiments) configured to identify a physical process (e.g., a state process of the physical process) and detect data integrity attacks in an ICS in an effective manner.
  • a physical process e.g., a state process of the physical process
  • the process skew based detection method uses deviations (e.g., small deviations) in a physical process (in particular, deviations in measured sensor data associated with the physical process) with respect to a modelled physical process (in particular, modelled sensor data of the physical process) modelled based on design parameters (e.g., which may be referred to as process offsets or process skews) for fingerprinting the physical process so as to obtain a process skew fingerprint associated with the physical process (in particular, a process fingerprint associated with the measured sensor data associated with the physical process).
  • design parameters e.g., which may be referred to as process offsets or process skews
  • these deviations in the physical process may be noise that appears in sensor measurements due to the process fluctuations (or process offsets).
  • uniqueness in the process skew fingerprint obtained can be achieved due to the specified operational constraints of the physical process.
  • various example embodiments create a process skew fingerprint by extracting process offset information (or process skew information) from the sensor measurements.
  • process offset information or process skew information
  • various example embodiments note that for a physical process, due to inaccuracies in the physical process, it would have a skew from what it is designed for.
  • An example is that of a water pipe delivering water to fill a tank. Pipes and tanks of two different sizes would take/store a different amount of water. Even if the pipes are of the same size, two different amounts of pumping force would result in a different amount of water flowing or being stored.
  • the flow of water in a pipe and water storage in a tank are examples of state processes of a physical process.
  • these state processes may be designed to meet certain operational requirements (i.e., based on design parameters).
  • certain operational requirements i.e., based on design parameters.
  • various example embodiments note that when these state processes are running, they show small deviations or offsets from the designed parameters due to the physical inaccuracies in the state processes, for example, no two water pipes can have the same diameter at a micro-scale due to manufacturing imperfections. Therefore, the process skew based detection method according to various example embodiments advantageously makes use of inaccuracies in the physical process itself.
  • the process skew based detection method does not depend on the specific state of the system (e.g., does not need to wait for the process to be static) and uses the dynamics of the physical process (or a state process thereof) to create a system model, and uses process skews (or process offsets) for fingerprinting the physical process (or a state process thereof) so as to obtain a process skew fingerprint associated with the physical process (or a state process thereof).
  • the process skew based detection method advantageously provides a distinctive way of passively fingerprinting processes, by using process skews to determine a process skew fingerprint to detect attacks.
  • a method of utilizing process skews to fingerprint a physical process a method of analyzing the effects of stealthy attacks on a process skew based detection method, and a method of detecting sensor attacks under a multitude of adversarial scenarios.
  • SWaT Secure Water Treatment Testbed
  • ICS is a broad domain of connected industrial systems.
  • a particular example of a water treatment industrial process, and more particularly, the SWaT is considered.
  • the SWaT is a fully functional testbed and is open for researchers to use.
  • a brief introduction of the SWaT is provided below, but is described in detail in A. P. Mathur and N. O. Tippenhauer, "SWaT: a water treatment testbed for research and training on ICS security," 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), 2016, pp. 31-36, doi: 10.1109/CySWater.2016.7469060.
  • the SWaT testbed produces purified water and it is a scaled-down version of a real water treatment process.
  • FIG. 4 depicts a schematic drawing of the SWaT Test-bed Network Architecture 400.
  • the testbed is distributed and there are different stages, where each stage is labeled as P n where n is the n th stage.
  • Each stage is equipped with a set of sensors and actuators. Sensors include water quantity measures, such as level, flow, and pressure and water quality measures, such as pH, ORP and conductivity.
  • Actuators are different motorized valves and electric pumps.
  • Stage 1 is a raw water stage configured to hold the raw water for the treatment and stage 2 is a chemical dosing stage configured to treat the water depending on the measurements from the water quality sensors.
  • Stage 3 is an ultrafiltration stage.
  • Stage 4 includes a de-chlorinator and stage 5 is equipped with reverse osmosis filters. Stage 6 is configured to hold the treated water for distribution.
  • data from the sensors and actuators are communicated to the PLCs using a level 0 network and PLC communicates to each other over a level 1 network as shown in FIG. 4.
  • FIG. 5 depicts a schematic flow diagram illustrating an overview of a process skew based detection method 500 according to various example embodiments of the present invention (e.g., corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein according to various embodiments).
  • the method 500 comprises extracting (at 504) measurements (i.e., measured sensor data) for a specific state of the physical process (i.e., a process state of the physical process). For example, measured sensor data associated with each process state of the physical process may be obtained.
  • the process state of the physical process For example, if the inlet pump is switched on, then the water is being filled in a tank, and thus, by knowing the state of the inlet pump, it is possible to know the process state of the physical process (e.g., in this case, the process state is that of the tank being filled with water, and the state process is a process at that process state, such as, the process of the water filling the tank).
  • the process state of the physical process e.g., in this case, the process state is that of the tank being filled with water, and the state process is a process at that process state, such as, the process of the water filling the tank.
  • state information from the sensors and actuators may be spoofed by an attacker.
  • a system model along with the design parameters associated with the state process at the process state of the physical process, is used to model (e.g., estimate based on modelling) the physical state associated with the state process to obtain estimated sensor data (which may be referred to as modelled sensor data since the estimated sensor data is obtained based on a system model) in relation to the physical state associated with the state process at the process state (e.g., in the above-mentioned case, the physical state may be the water level in a tank).
  • model e.g., estimate based on modelling
  • the difference between the estimate (modelled sensor data) and real sensor measurement (measured sensor data) establishes an offset value (or a process offset), which corresponds to an amount by which the physical process (or state process) is offset from what it should be, as designed.
  • This process offset data may be obtained for each process run of a plurality of process runs for a state process at the process state of the physical process, thereby obtaining a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process.
  • the difference between the modelled sensor data and the measured sensor data associated with the process run may be obtained or determined at each time step of a plurality of time steps, thereby obtaining a plurality of process offsets (or a plurality of process offset information) associated with the process run at the plurality of time steps, respectively.
  • the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
  • the plurality process offset data associated with the plurality of process runs for a state process when accumulated over time, reveal the process skews associated with the plurality of process runs.
  • the plurality of process offset data associated with the plurality of process runs for a state process obtained may have fluctuations, for example, due to sensor noise.
  • various example embodiments apply (at 516) a linear regression model to the plurality of process offset data to obtain the best fit for the state process, so as to determine process skew fingerprint information (which may simply be referred to herein as a process skew fingerprint) associated with the plurality of process runs.
  • the process skew fingerprint may be obtained by calculating the rate of change of linear regression on the plurality of process offset data with respect to time.
  • a theoretical proof based on the calculated entropy of the process skew fingerprint is used to establish the uniqueness of the process skew fingerprint obtained.
  • an anomaly detector e.g., a CUSUM detector
  • the process skew based detection method 500 will be described in further details later below according to various example embodiments of the present invention.
  • sensors play an important role by sending physical measurements (measured sensor data) to a controller to execute an appropriate control action.
  • an adversary can spoof sensor measurements either through cyber domain or physical domain intrusions. Accordingly, it is important to detect or authenticate whether the data is originating from the real physical process or is being modified in some manner.
  • various example embodiments note that due to computational limitations and legacy compliant equipment, it is not feasible to rely on cryptographic methods. Accordingly, various example embodiments provide a process skew based detection method 500 to achieve a process skew based authentication of a physical process. In this regard, various example embodiments seek to identify a physical process (or a state process of the physical process) based on its physical dynamics.
  • FIG. 6 depicts a schematic drawing showing an abstraction of an ICS 600 in relation to a thread model, whereby may or may not be attacked sensor measurement.
  • an attacker can modify a rightful sensor measurement by an attack value .
  • the attacker model encompasses the attacker’ s intentions and capabilities.
  • the attacker may choose its goals from a set of intentions, including performance degradation, disturbing a physical property of the system, or damaging a component.
  • a range of attacks are considered from already published attack scenarios in the literature.
  • An example attacker model will now be described.
  • the attacker has access to the sensor’s measurements.
  • a powerful attacker can arbitrarily change sensor measurements to the desired sensor value.
  • a malicious insider may be an attacker with physical access to the plant and thus to its devices, such as level sensors.
  • an attacker who can physically replace or tamper sensors may not necessarily be an insider, because critical infrastructures, e.g., for water and power, are generally distributed across large areas.
  • An outsider for example, an end user, may also carry out a physical attack on sensors, such as smart energy monitors.
  • a type of data injection attack scenario is a generic sensor spoofing attack.
  • the process skew based detection method 500 was evaluated for a range of network attack scenarios from benchmark attacks on the SWaT testbed. These benchmark attacks cover a wide range of 36 attacks on both sensors and actuators. Since the process skew based detection method 500 extracts process skew information for various physical properties, attacks on chemical sensors are thus excluded, resulting in a total of 25 attacks remaining as summarized in Table 1 shown in FIGs. 7A and 7B.
  • Table 1 provides a list of attacks on the SWaT testbed carried out in experiments conducted according to various example embodiments of the present invention.
  • an attack vector may be defined as:
  • Equation (1) where y k are the real sensor measurement (measured sensor data), is sensor measurement with a possible attack and is the data injected by an attacker at time step k. Details about each (attack vector) is described in Table 1 shown in FIGs. 7 A and 7B, where it can be seen that it ranges from an abrupt injection of data to a more slow/stealthy change in sensor measurements.
  • Another type of data injection attack scenario is stealthy attacks for model based techniques. These attacks are designed to be stealthy by changing sensor measurements such that the system model based detection mechanism would fail. Since model based detectors use a system model, a Kalman filter and a statistical detector, an attacker who wants to remain stealthy may try to choose injected readings so as not to exceed the detector threshold. To do this, an attacker may learn the system model and the detector parameters. In this regard, various example embodiments assume that an attacker has the ability to do so, but does not possess the process skew knowledge.
  • FIG. 8 depicts a plot of level sensor (LIT- 101) measurements in the SWaT testbed in stage 1 labeled as LIT- 101 for a duration of a normal process (i.e., under normal operation).
  • LIT- 101 level sensor
  • the design parameters in Table 2 indicate the design for the inflow and outflow processes and which state process is present in a particular process state.
  • the tank 1 in stage 1 of the SWaT testbed has one inlet valve labeled as MV- 101 and one outlet pump labeled as P-101. There is also a secondary backup pump at the outlet labeled as P-102. Accordingly, there can be four possible state processes for the water level (physical state) in tank 1 based on input and output flow processes, namely, output flow process is present but no input flow process (SI), neither input nor output flow processes (S2), input flow process is present but no output flow process (S3), both input and output flow processes are present (S4).
  • SI input flow process
  • S2 neither input nor output flow processes
  • S3 input flow process
  • both input and output flow processes are present
  • FIGs. 10A to 10D depict four plots showing four possible state processes of a physical process, respectively, along with measured sensor data obtained with respect to the water level in a water tank (tank 1) as described with reference to Table 2.
  • the level sensor in the SWaT testbed in stage 1 is labeled as LIT- 101 under normal operations. Accordingly, the measured sensor data shown in FIGs. 10A to 10D present the particular state processes extracted from the seven days of normal operations. As shown, for each state process, there may be a plurality of occurrences (or process runs), for example, hundreds of occurrences.
  • FIG. 10A For example, for the water emptying process (SI) shown in FIG. 10A, there may be hundreds of occurrences (hundreds of process runs) of the water empty process during the observation period. From FIGs. 10A to 10D, the effects of noise (e.g., deviations or fluctuations) on the measured sensor data obtained in the plurality of process runs noted according to various example embodiments of the present invention can be seen.
  • noise e.g., deviations or fluctuations
  • Each physical process (or state process of the physical process) is expected to behave according to design parameters, such as shown in Table 2 in FIG. 9.
  • design parameters such as shown in Table 2 in FIG. 9.
  • FIGs. 10A to 10D there are deviations due to the process noise.
  • a first state process S 1 shows different process runs of the water emptying process from the tank 1.
  • the variations in each process run due to the sensor noise can be observed in FIG. 10A. This can also be observed from the static (S2) and water filling (S3, S4) state processes.
  • FIG. 11 depicts a plot illustrating the concept of process skews associated with a physical process, according to various example embodiments of the present invention.
  • the sensor measurements (measured sensor data) with respect to the water level for the water filling process and estimated sensor value (modelled sensor data) based on design parameters are shown.
  • the difference between the measured sensor data and the modelled sensor data at each time step may be referred to as a process offset.
  • the accumulated process offsets over time are also labelled for illustration purpose and better understanding.
  • FIG. 12 depicts a schematic diagram of an example physical system 1200 for stage 1 according to various example embodiments of the present invention, for modelling the physical process with respect to the level sensor in a tank (Tank 1) 1204.
  • the tank 1204 in stage 1 of the SWaT testbed is being used as a running example to demonstrate the process skew based detection method 500 according to various example embodiments of the present invention.
  • the water level in the tank 1204 is measured using a level sensor 1208 and the inflow and outflow of the water are controlled by the motorized valve (MV- 101) 1212 at the input and pump (P-101) 1216 at the output, respectively.
  • MV- 101 motorized valve
  • P-101 pump
  • Equation (2) where V denotes the volume of the tank, A denotes the cross-sectional area of the tank, and h denotes the height of the water inside the tank. Equation (2) provides a linear equation, and thus, the expression [Q in — Q out ] represents the water flow which depends on the PLC control actions implemented via the valve (MV- 101) 1212 and the pump (P-101) 1216.
  • Equation (3) Equation (3) where y k is the sensor measurement (measured sensor data) driven by the control action u k , and matrices A, B and C are the state-space matrices of appropriate dimensions.
  • Equation (3) From Equation (3), it can be seen that for a system state value at time k, and given the PLC control u k , the next state at time k + 1 can be predicted.
  • Table 2 in FIG. 9 shows a list of example design parameters for each type of control action.
  • the valve MV- 101 1212 is controlled to be in an open state and the pump P-101 1216 is controlled to be in an activated (switched on) state.
  • various example embodiments estimate the sensor data of the physical state of the state process based on design parameters associated with the state process (e.g., estimated water level in the tank based on design parameters).
  • design parameters associated with the state process e.g., estimated water level in the tank based on design parameters.
  • process offsets may be extracted, for example, how much the real process dynamics are offset from the designed physical process.
  • process offsets associated with the measured sensor data with respect to the level of the water in the above-mentioned tank 1204 for the three different state processes, namely SI, S3 and S4, can been seen.
  • the process offset at the time step may be defined as a deviation of the process dynamics due to the process inaccuracies from the design at the time step.
  • the process offset associated with the process run may be calculated or determined. All the process offsets associated with the process run may then be accumulated over the time period, to obtain process offset data (including the process offsets accumulated over the time period) associated with the process run.
  • a plurality of process offset data may thus be obtained for a plurality of process runs for a state process (e.g., for the plurality of process runs of the water emptying process (SI) as shown in FIG. 13A).
  • Process skew fingerprint information associated with the plurality of process runs for the state process may then be determined or extracted based on the plurality of process offset data obtained.
  • the process skew fingerprint information associated with a plurality of process runs may be determined based on a slope of the plurality of process offset data obtained on the plurality of process runs.
  • State process SI is a water emptying process at the water emptying state (i.e., water outflow from the tank).
  • the negative slope indicates that the real process is actually slower than designed (i.e., based on design parameters).
  • State process S2 is a static process corresponding to the process state of being static (i.e., there is no inflow or outflow), and thus, this state process is actually missing so no process offset exists.
  • State process S3 is a water filling process at the water filling state (i.e., only the inflow is present).
  • the positive slope indicates that the real process is actually faster than designed.
  • State process S4 is a water filling and emptying process at the water filling and emptying state (i.e., both the inflow and outflow are present).
  • the negative slope indicates that the real process is actually slower than designed.
  • All these state processes (or process states) scenarios may be different state processes (or process states) of the same physical process, that is, with respect to the water tank 1204 in stage 1 of the SWaT testbed. In this regard, although they may be different state processes (or process states) of the same physical process, it can be observed that based on the corresponding process offsets obtained, all the state processes (or process states) of the physical process can be distinguished from each other.
  • the method of extracting the process offsets in measured sensor data advantageously establishes a process skew fingerprint, that is, fingerprinting the process runs for a process state based on process skews to obtain a corresponding process skew fingerprint.
  • process offsets obtained are noisy, for example, due to the sensor noise.
  • various example embodiments seek to remove, or at least mitigate, the sensor noise effect without disturbing the process offsets.
  • process skew fingerprint information associated with a plurality of process runs for a state process is derived based on the plurality of process offset data produced for the plurality of process runs in the following manner.
  • Equation (4) Equation (3)
  • Equation (8) the process offset (O k ) can be extracted at each time step.
  • the process offset includes the noise from the sensor.
  • various example embodiments apply a linear regression model on the plurality of process offset data to obtain a regression coefficient, which corresponds to the process skew fingerprint information.
  • a straight line may be fitted onto the plurality of process offset data to determine the slope of the plurality of process offset data. Accordingly, for each state process (e.g., each of the three state processes shown in FIGs.
  • a straight line may be fitted onto the plurality of process offset data associated with the plurality of process runs for the state process in relation to the physical state (e.g., water level) to determine the slope thereof, which may represent the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
  • various example embodiments apply a linear regression model on the plurality of process offset data obtained for each state process. For example, from FIG. 13 A to 13C, it can be observed that the process offsets obtained for each state process are generally linear in time.
  • correlation coefficients are used to establish the linearity between the time and the progression of the process offsets. Correlation calculates the level of the linear relationship between variables. For example, a high correlation between two variables indicates that the values for the variables increase or decrease in a linear relationship. However, uncorrelated variables may still be dependent on each other, just that the relationship may be nonlinear. For example, for N scalar values of two variables, the Pearson correlation coefficient may be defined as:
  • Equation (9) where X denotes the mean of the variable X, and 7 denotes the mean of the variable Y.
  • X denotes the mean of the variable X
  • 7 denotes the mean of the variable Y.
  • a linear regression approach is adopted to obtain a data model describing the relationship between variables in a mathematical form.
  • a least squares fit is applied to obtain the model for a set of n observed values of X and Y given by respectively.
  • FIG. 14 depicts a plot of a plurality of process offset data for the water filing process in stage 1 of the water treatment system, with a linear regression model fitted thereon, according to various example embodiments of the present invention.
  • the linear regression model is used to find the slope that defines process skew fingerprint information representing the process skews associated with the plurality of process runs. For example, the accuracy of the linear regression model can be seen in FIG. 14.
  • mean square error MSE
  • RMSE root mean square error
  • MSE is the difference between measured sensor data and estimated sensor data (or modelled sensor data) squared, and in general, provides the distance between measured and estimated value, in other words, how far the estimated value is from the measured value.
  • the model accuracies for the three stages of the SWaT testbed and corresponding state processes used in the experiments conducted (from the SWaT testbed) are shown in Table 3 in FIG. 17. It can be seen that the obtained system model is very accurate, with almost zero mean error for all the process runs of a plurality of state processes.
  • Table 3 in FIG. 17 shows the mean of models created for all the process runs of the plurality of state processes.
  • the process offsets accumulated for a process run of a state process may be expressed as:
  • Equation (12) and the corresponding process skew fingerprint information may be determined as:
  • FIG. 15A depicts a plot of three process skew fingerprint information obtained with respect to three state processes, respectively, at stage 1 of the SWaT testbed
  • FIG. 15B depicts a plot of three process skew fingerprint information obtained with respect to three state processes, respectively, at stage 3 of the SWaT testbed
  • FIG. 15C depicts a plot of two process skew fingerprint information obtained with respect to two state processes, respectively, at stage 4 of the SWaT testbed.
  • FIGs. 15A to 15C depict process skew fingerprint distribution for eight state processes at three stages of the SWaT testbed. From FIGs.
  • FIGs. 15A to 15C show a visual analysis for process skew fingerprint uniqueness.
  • a mathematical proof for the process skew fingerprint uniqueness will now be described. For example, demonstrating that fingerprints are information-theoretically unique helps to negate the possibility of impersonation attacks. Let w(t) be the signal corresponding to a process skew fingerprint. In order to present an information-theoretic analysis, justification of two important criteria are provided:
  • conditional entropy of process skew fingerprints with other process skew fingerprints should be very low, «1.
  • a process skew vector (determined based on the corresponding process skew fingerprint information) is provided as an input to the anomaly detector, which for example, may be a cumulative sum (CUSUM) detector and known as the stateful detector.
  • the input (process skew vector) to the CUSUM procedure may be considered as a distance measure, that is, a measure of how far the estimated measurements is from the expected measurements.
  • a dedicated anomaly detector for each process state may be designed.
  • the index i denotes the process, , where m is the number of processes in each stage of the plant.
  • Process skew fingerprint is denoted as for easy reference, where k is the time step.
  • the standard CUSUM procedure (e.g., as described in D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 2009) is explained using the following equations.
  • Equations (14) and (15) From Equations (14) and (15), it can be observed that accumulate the distance measure over time to measure how far are the values of the residual from the target mean .
  • G is a multiplier to the standard deviation ( s ) and may typically be between 3 and 5.
  • an alarm is raised when this accumulation becomes greater or less than a predetermined threshold For example, based on a threshold that is derived from the process skew fingerprint information obtained under normal operation, an alarm may be raised when the accumulation crosses the threshold.
  • the sequence S k i is reset to the target mean value each time it becomes negative or larger than T j .
  • the CUSUM sequence S k i grows unbounded until the threshold q is reached, no matter how large q is set.
  • the slack variable jq is selected properly based on the statistical properties of the distance measure. Once k is chosen, the threshold q must be selected to achieve a required false alarm rate denotes the false alarm rate for the CUSUM procedure defined as the expected proportion of observations which are false alarms.
  • the process skew based detection method 500 is evaluated in a real water treatment testbed.
  • the following metrics are used for performance evaluation.
  • TP t denotes true positive for class q when it is rightly classified based on the ground truth
  • FN t denotes false negative for class q when it is wrongly rejected
  • FP t denotes false positive for class q when it is wrongly accepted
  • TN L denotes true negative for class q when it is rightly rejected.
  • TPR True Positive Rate
  • FPR False Positive Rate
  • FPR should be as small as possible and TPR as high as possible. Both TPR and FPR being a ratio ranging between 0 and 1.
  • Process offsets are determined for each of these process runs.
  • process offsets are noisy due to noise from the sensors.
  • a linear regression model is fitted on the plurality of process offset data obtained associated with the plurality of process runs to address the noise in the signal. After the linear regression model is fitted, a straight line is obtained for the accumulated process offsets associated with the plurality of process runs over a process time frame.
  • the rate of change of these process offsets may then be defined as the process skew fingerprint (e.g., corresponding to the reference process skew fingerprint information as described hereinbefore according to various embodiments) associated with the plurality of process runs.
  • FIGs. 13A to 13C show the process offsets for different state processes of the stage 1 of the SWaT testbed.
  • FIG. 14 shows an example of linear model fitting for the process offsets.
  • the obtained linear model may thus be used to calculate the process skew fingerprint.
  • the process skew fingerprints obtained under normal operating conditions (which may be referred to as normal or reference process skew fingerprints) may then be used by an anomaly detector (e.g., the CUSUM detector) for detecting an anomaly in a physical process.
  • the CUSUM parameters for all the stages in SWaT are shown in Table 4 in FIG. 18. In particular, Table 4 shows the design and performance of the CUSUM detector on the normal operation data.
  • Table 4 shows bias parameter K, threshold t, mean m and standard deviation s for the process skew fingerprints. In the last two rows of Table 4 in FIG. 18, performance of the CUSUM detector under the normal operating conditions are shown using the design parameters specified. It can be observed that for all the cases the desired false alarm rate is below 5%.
  • Table 4 in FIG. 18 shows a high true negative rate, which indicates that it is possible to identify each state process (or process state) with a high accuracy based on the process skew fingerprint.
  • a physical process goes through different state processes (or process states) during the operation of the process plant. For example, for the process of a fluid tank, either fluid is flowing out, flowing in, both or in a static state. Since different state processes have different process skew fingerprints according to various example embodiments of the present invention, it is possible to uniquely identify each state process (or process state) based on its associated process skew fingerprint.
  • FIG. 19 depicts a plot of the process offsets obtained on the process runs in relation to tank 4 in stage 4, including a mixture of normal process runs and a few attacks. From FIG. 19, it can be observed that the process offsets obtained associated with normal process runs are close together and follow the normal profile of the process runs. On the other hand, the process offsets obtained associated with attacks (attack start and attack stop are denoted in FIG. 19) show clear deviations. Accordingly, from FIG.
  • FIGs. 20A and 20B show the CUSUM detector for the same process.
  • FIGs. 20A and 20B depict plots in relation to an attack detection example for LIT-401, outflow process.
  • FIGs. 20A and 20B it can be observed that the process skew fingerprint according to various example embodiments of the present invention enables the detection of an attack easily and effectively.
  • a detailed analysis was carried out for all the three stages and corresponding state processes in the SWaT testbed and the results are presented in Table 5 shown in FIG. 21.
  • FIG. 21 shows the evaluation results of the process skew based detection method 500 according to various example embodiments of the present invention on the attack data from the SWaT testbed.
  • the TPR presents the attacks which were detected accurately as percentage (attacks-detection/total-attacks-executed). From FIG. 21, it can be seen that all the attacks are detected in all the scenarios with 100% TPR. Furthermore, FPR is close to the desired 5% false alarm rate except for two instances. Accordingly, the process skew based detection method 500 has shown perfect performance on attack detection.
  • a stealthy attack is an attack designed to be hidden for a system model based attack detector.
  • the objective of the attacker is to modify the physical process measurements to achieve its objective and remain hidden.
  • FIGs. 22A and 22B show the execution of such an attack in stage 1 of the SWaT testbed.
  • FIG. 22A a plot of level sensor (LIT-101) actual measurements and sensor estimates obtained using the system model is shown.
  • FIG. 22B respective residual (measured - estimated) values for the level sensor are shown. Upper and lower limits for a statistical detector are also shown.
  • LIT-101 level sensor
  • the dotted line shows the ground truth for the state process, while the attacker is spoofing the sensor values and managed to derive the system away from the normal operation over time during the attack period.
  • the spoofed values are chosen such that the residual values never grow bigger than a model-based detector threshold and hence, could not get detected. But from the ground truth, the process dynamics are not what the attacker is making the PLC believe. Accordingly, using the process offsets according to various example embodiments of the present invention, it is possible to detect the presence of such an attacker. In particular, if an attacker wants to deviate the process from its desired operation, it must defy the process dynamics and expose itself in the process offsets.
  • the process skew based detection method 500 is able to detect attacks that are stealthy for the system model based detectors.
  • the process skew based detection method 500 according to various example embodiments of the present invention has been described with respect to a water treatment plant, it will appreciated by a person skilled in the art that the present invention is not limited to a water treatment plant.
  • physical processes have been discussed herein with respect to water/fluid dynamics but it will be appreciated that other similar processes are also applicable, such as but not limited to, gas or other chemical fluids where the process skew based detection method 500 can also be applied in the same or similar manner.
  • process skew based detection method 500 may be used to fingerprint different state processes of a physical process, for example, filling, emptying or a combination of these process dynamics in a water treatment system.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

There is provided a method of detecting anomaly in a physical process associated with a networked control system. The method includes: obtaining, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing, for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information including characteristic information associated with the plurality of process offset data; and detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process. There is also provided a corresponding system for detecting anomaly in a physical process associated with a networked control system.

Description

METHOD AND SYSTEM FOR DETECTING ANOMALY IN A PHYSICAL PROCESS ASSOCIATED WITH A NETWORKED CONTROL SYSTEM
[0001] This application claims the benefit of priority of Singapore Patent Application No. 10202005543S, filed on 11 June 2020, the content of which being hereby incorporated by reference in its entirety for all purposes.
TECHNICAL FIELD
[0002] The present invention generally relates to a method and a system for detecting anomaly in a physical process associated with a networked control system, and more particularly, for detecting an attack on the networked control system.
BACKGROUND
[0003] An Industrial Control System (ICS) is a networked control system comprising sensors, actuators, controllers and communication networks configured to control one or more physical processes in an industry, such as water treatment, water distribution, smart grid, autonomous transportation, and so on. For example, connectivity in an ICS provides improved monitoring and operation of a physical process. Such advancements are helpful but also bring about challenges of secure operation of the connected devices for the ICS to operate securely. [0004] For example, an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data, and manipulate the actuators to cause anomaly of operations, which may eventually lead to physical damages to the ICS. However, traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic.
[0005] For example, sensor data may be transmitted to a Programmable Logic Controller (PLC) to perform an appropriate action based on the sensor measurement. In this regard, if an adversary can spoof sensor data in the digital or physical domain, the adversary can cause the ICS to go to an unsafe state. The focus here is not on the confidentiality of the sensor data as in legacy computer security but the integrity and trustworthiness of the sensor data. [0006] A need therefore exists to provide a method and a system for detecting anomaly in a physical process associated with a networked control system, that seek to overcome, or at least ameliorate, one or more problems relating to conventional method and a system for detecting anomaly in a physical process, and more particularly, enabling or improving anomaly detection (in particular, attack detection) in a physical process associated with a networked control system in an effective manner. It is against this background that the present invention has been developed.
SUMMARY
[0007] According to a first aspect of the present invention, there is provided a method of detecting anomaly in a physical process associated with a networked control system using at least one processor, the method comprising: obtaining, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
[0008] According to a second aspect of the present invention, there is provided a system for detecting anomaly in a physical process associated with a networked control system, the system comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; produce, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detect anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
[0009] According to a third aspect of the present invention, there is provided a computer program product, embodied in one or more non-transitory computer-readable storage mediums, comprising instructions executable by at least one processor to perform the method of detecting anomaly in a physical process according to the above-mentioned first aspect of the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS [0010] Embodiments of the present invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:
FIG. 1 depicts a schematic flow diagram of a method of detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention
FIG. 2 depicts a schematic block diagram of a system for detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention, such as corresponding to the method of detecting anomaly in a physical process as described with reference to FIG. 1; FIG. 3 depicts a schematic block diagram of an exemplary computer system which may be used to realize or implement the system as described with reference to FIG. 2;
FIG. 4 depicts a schematic drawing of a Secure Water Treatment Testbed (SWaT) Network Architecture, based on which the method of detecting anomaly in a physical process may be applied as an illustrative example, according to various example embodiments of the present invention for illustration purpose;
FIG. 5 depicts a schematic flow diagram illustrating an overview of a process skew based detection method according to various example embodiments of the present invention, such as corresponding to the method of detecting anomaly in a physical process as described with reference to FIG. 1;
FIG. 6 depicts a schematic drawing showing an abstraction of an industrial control system (ICS) in relation to a thread model;
FIGs. 7A and 7B depict a table (Table 1) providing a list of attacks on the SWaT testbed that may be carried out in experiments conducted, according to various example embodiments of the present invention;
FIG. 8 depicts a plot of level sensor (LIT-101) measurements in the SWaT testbed in stage 1 for a duration of a normal process (i.e., under normal operation);
FIG. 9 shows a table (Table 2) providing a list of example design parameters for each type of state process or control action, according to various example embodiments of the present invention;
FIGs. 10A to 10D depict four plots showing four possible state processes of a physical process, respectively, along with measured sensor data obtained with respect to the water level in a water tank (tank 1) as described with reference to Table 2, according to various example embodiments of the present invention;
FIG. 11 depicts a plot illustrating the concept of process offsets (or process skews) associated with a physical process, according to various example embodiments of the present invention;
FIG. 12 depicts a schematic diagram of an example physical system for stage 1 of the SWaT testbed, according to various example embodiments of the present invention;
FIGs. 13A to 13C depict process offset data obtained based on measured sensor data for different state processes at stage 1 of the SWaT testbed, according to various example embodiments of the present invention; FIG. 14 depicts a plot of a plurality of process offset data obtained based on measured sensor data for the water filing process in stage 1 of the SWaT testbed, with a linear regression model fitted thereon, according to various example embodiments of the present invention;
FIGs. 15A to 15C depict process skew fingerprint distributions for eight state processes at three stages of the SWaT testbed, according to various example embodiments of the present invention;
FIG. 16 illustrates mutual information across the eight process skew fingerprints discussed with reference to FIGs. 15A to 15C, according to various example embodiments of the present invention;
FIG. 17 depicts a table (Table 3) showing the model accuracies for three stages of the SWaT and corresponding state processes used in experiments conducted (from the SWaT testbed), according to various example embodiments of the present invention;
FIG. 18 depicts a table (Table 4) showing example design and performance of a cumulative sum (CUSUM) detector on the normal operation data, according to various example embodiments of the present invention;
FIG. 19 depicts a plot of process offsets obtained based on measured sensor data on process runs in relation to tank 4 in stage 4 of the SWaT testbed, including a mixture of normal process runs and a few attacks, according to various example embodiments of the present invention;
FIGs. 20A and 20B depict plots in relation to an attack detection example for an outflow process in relation to level sensor LIT-401, according to various example embodiments of the present invention;
FIG. 21 depicts a table (Table 5) showing evaluation results of the process skew based detection method, according to various example embodiments of the present invention, on attack data from the SWaT testbed; and
FIGs. 22 A and 22B show an example execution of a stealthy attack on stage 1 of the SWaT testbed, according to various example embodiments of the present invention.
DETAILED DESCRIPTION
[0011] Various embodiments of the present invention provide a method and a system for detecting anomaly in a physical process associated with a networked control system, and more particularly, for detecting an attack on the networked control system. For example, the networked control system may be implemented in any industries (industrial applications) as an Industrial Control System (ICS) as desired or as appropriate that requires an industrial process control, such but not limited to, water treatment, chemical processing, power generation, oil and gas processing, and so on. The networked control system may comprise a sensor network comprising a plurality of sensors. A sensor network is known in the art and may comprise a plurality of sensors that are spatially positioned or installed in the networked control system, each being arranged or positioned for monitoring and collecting sensor data (e.g., measurements or readings, which may be referred to herein as measured sensor data) relating to one or more process states of one or more physical processes associated with the networked control system, such as relating to a physical condition or a physical property of a surrounding environment (e.g., in relation to a medium or an object), such as but not limited to, temperature, sound, pressure, fluid flow rate, fluid level and so on. The plurality of sensors may be communicatively coupled to a processor (e.g., a central processor or a sensor data processor, such as a Programmable Logic Controller (PLC)) based on any communications technologies known in the art, such as wired communications technologies or wireless communications technologies, and need not be described herein. It will be appreciated by a person skilled in the art that each sensor may be any type of sensor known in the art configured for capturing a physical condition or physical property of a surrounding environment and outputting corresponding measured sensor data in relation to the surrounding environment. It will be appreciated by a person skilled in the art that the plurality of sensors may be of the same type or may include a variety of sensors, depending on the aspects of (or in relation to) the one or more physical processes associated with the networked control system that are desired to be monitored. In various embodiments, an attack on the networked control system may refer to any type of security or malicious attack on the networked control system known in the art and need not be described herein, such as an attack on one or more sensors in the sensor network. Various possible types of attack on a networked control system are known in the art, and for illustration purpose, example types of attack will be described later below according to various example embodiments of the present invention.
[0012] As described in the background, an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data and may manipulate the actuators, which may result in anomaly of operations and eventually lead to physical damages to the ICS. However, traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic. For example, sensor data may be transmitted to a PLC to perform an appropriate action based on the sensor measurement. In this regard, if an adversary can spoof sensor data in the digital or physical domain, the adversary can cause the ICS to go to an unsafe state. Therefore, the integrity and trustworthiness of the sensor data are important for the ICS to operate securely.
[0013] Accordingly, various embodiments of the present invention provide a method and a system for detecting anomaly in a physical process associated with a networked control system, that seek to overcome, or at least ameliorate, one or more problems relating to conventional method and a system for detecting anomaly in a physical process, and more particularly, enabling or improving anomaly detection (in particular, attack detection) in a physical process associated with a networked control system in an effective manner.
[0014] FIG. 1 depicts a schematic flow diagram of a method 100 of detecting anomaly in a physical process associated with a networked control system using at least one processor, according to various embodiments of the present invention. The method 100 comprises: obtaining (at 102), for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing (at 104), for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining (at 106) process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detecting (at 108) anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
[0015] In various embodiments, in relation to 102, the measured sensor data associated with a process run may be obtained from one or more sensors of the sensor network of the networked control system. For example, a physical process associated with a network control system may comprise one or more process states, and for each process state, there may be one or more state processes (i.e., one or more types of state processes) that may be performed (or may occur) at the process state. Furthermore, for each state process, a plurality of process runs (or process instances) for the state process may be performed (or may occur). By way of an example only and without limitation, in relation to a water treatment plant, there may be a plurality of process stages, such as a raw water storage stage to hold the raw water for treatment, a chemical dosing stage to treat the water, an ultra-filtrating stage, and so on. For example, in relation to the raw water storage stage, there may be an associated physical process comprising a plurality of process states, such as a water emptying state, a water filling state and a water filling and emptying state. In this regard, the one or more types of processes performed at each process state may thus be one or more state processes (i.e., one or more types of state processes) at the process state. Furthermore, for a state process (e.g., water emptying process) at a process state (e.g., water emptying state), a plurality of process runs (e.g., hundreds of process runs (or process instances) of the water emptying process over time) may be performed (or may occur), and measured sensor data associated with each of the plurality of process runs in relation to a physical state (e.g., water level in a water tank) associated with the state process may thus be obtained, to produce a plurality of measured sensor data associated with the plurality of process runs in relation to the physical state associated with the state process. In various embodiments, the networked control system may be configured to perform or control a physical process, and each process stage of the networked control system may thus be configured to perform or control an associated physical sub-process.
[0016] In various embodiments, in relation to 104, the modelled sensor data in relation to the physical state associated with the state process may be sensor data in relation to the physical state that is expected or intended (e.g., configured or designed to be) based on design parameters associated with the state process. For example, in the case of the physical state associated with the state process being the water level in a water tank during the water emptying process, the modelled sensor data in relation to the water level during the water emptying process may be sensor data in relation to the water level that is expected over time during the water emptying process based on design parameters associated with water emptying process in relation to the water tank. As will be described later below, according to various embodiments, sensor data in relation to a physical state that is expected may be determined by estimating the sensor data in relation to the physical state based on a system model.
[0017] In various embodiments, in relation to 106, characteristic information associated with the plurality of process offset data associated with the plurality of process runs for the state process may be determined (e.g., extracted) so as to produce the process skew fingerprint information associated with the plurality of process runs. As will be described later below, according to various embodiments, the characteristic information may be a slope parameter associated with a linear regression model applied to the plurality of process offset data.
[0018] In various embodiments, in relation to 108, using the process skew fingerprint information associated with the process runs for the state process, whether there is an anomaly in the physical process at that state process may then be detected.
[0019] It will be understood by a person skilled in the art that the method 100 of detecting anomaly in the physical process for additional one or more other state processes at the process state of the physical process, or at one or more other process states of the physical process may be performed in the same or similar manner as described herein in relation to the above- mentioned state process at the above-mentioned process state of the physical process. It will also be understood by a person skilled in the art that the method 100 of detecting anomaly in additional one or more physical processes associated with the networked control system may also be performed in the same or similar manner as described herein in relation to the above- mentioned state process at the above-mentioned process state of the physical process.
[0020] Accordingly, the method 100 of detecting anomaly in a physical process is based on a plurality of process offset data (e.g., which may be referred to herein as a plurality of process skew data) associated with a plurality of process runs for a state process at a process state of the physical process. Therefore, the method 100 of detecting anomaly in a physical process is advantageously based on inaccuracies (e.g., deviations due to an attack) in the physical process itself, and thus is based on physical process dynamics. Furthermore, a plurality of process skew data associated with a plurality of process runs for a state process of a physical process is produced and used to fingerprint the plurality of process runs for the state process, so as to produce process skew fingerprint information for detecting anomaly in the state process of the physical process. Such a technical approach in detecting anomaly in the physical process has been found to enable or improve anomaly detection (in particular, attack detection) in a physical process associated with a networked control system in an effective manner. These advantages or technical effects, and/or other advantages or technical effects, will become more apparent to a person skilled in the art as the method 100 of detecting anomaly, as well as corresponding system for detecting anomaly, is described in more detail according to various embodiments and example embodiments of the present invention.
[0021] In various embodiments, the method 100 further comprises determining the modelled sensor data in relation to the physical state associated with the state process, comprising estimating sensor data in relation to the physical state associated with the state process based on a system model representing the state process and one or more design parameters associated with the state process.
[0022] In various embodiments, the above-mentioned producing (at 104), for the above- mentioned each process run of the plurality of process runs of the state process at the process state, process offset data associated with the process run comprises: determining, for each time step of a plurality of time steps, a process offset between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain a plurality of process offset information associated with the process run at the plurality of time steps, respectively. In this regard, the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
[0023] In various embodiments, the above-mentioned determining, for the above- mentioned each time step of the plurality of time steps, the process offset comprises determining, for the above-mentioned each time step of the plurality of time steps, a difference between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain the plurality of process offset information associated with the process run at the plurality of time steps, respectively.
[0024] In various embodiments, the above-mentioned determining (at 106) the process skew fingerprint information associated with the plurality of process runs comprises determining a slope parameter associated with the plurality of process offset data associated with the plurality of process runs in relation to the physical state associated with the state process. In this regard, the process skew fingerprint information comprises the slope parameter. [0025] In various embodiments, the above-mentioned determining the slope parameter associated with the plurality of process offset data comprising applying a linear regression model to the plurality of process offset data to obtain a regression coefficient. In this regard, the slope parameter comprises the regression coefficient.
[0026] In various embodiments, the above-mentioned detecting (at 108) anomaly in the physical process comprises detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process and a reference process skew fingerprint information associated with the state process at the process state of the physical process. [0027] In various embodiments, the method 100 further comprises determining the reference process skew fingerprint information associated with the state process at the process state of the physical process, comprising: obtaining, for each reference process run of a plurality of reference process runs for the state process at the process state of the physical process, measured sensor data associated with the reference process run in relation to the physical state associated with the state process at the process state; producing, for said each reference process run of the plurality of reference process runs, reference process offset data associated with the reference process run based on the measured sensor data associated with the reference process run and the modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of reference process offset data associated with the plurality of reference process runs, respectively, in relation to the physical state associated with the state process; and determining the reference process skew fingerprint information associated with the plurality of reference process runs based on the plurality of reference process offset data, the reference process skew fingerprint information comprising characteristic information associated with the plurality of reference process offset data. In various embodiments, the reference process skew fingerprint information, the plurality of reference process runs and the plurality of reference process offset data may refer to that obtained under normal operating conditions (e.g., without attack and breakdown), that is, reference with respect to normal operating conditions.
[0028] In various embodiments, the above-mentioned reference process skew fingerprint information associated with the state process may be determined in the same or similar manner as the above-mentioned process skew fingerprint information associated with the state process determined at 106, except that the above-mentioned reference process skew fingerprint information is determined based on measured sensor data associated with the plurality of reference process runs for the state process.
[0029] In various embodiments, the above-mentioned detecting anomaly in the physical process comprises detecting an attack on the networked control system based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
[0030] FIG. 2 depicts a schematic block diagram of a system 200 for detecting anomaly in a physical process associated with a networked control system, according to various embodiments of the present invention, such as corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system as described hereinbefore with reference to FIG. 1 according to various embodiments of the present invention. The system 200 comprises: a memory 202; and at least one processor 204 communicatively coupled to the memory 202 and configured to: obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; produce, for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detect anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
[0031] It will be appreciated by a person skilled in the art that the at least one processor 204 may be configured to perform various functions or operations through set(s) of instructions (e.g., software modules) executable by the at least one processor 204 to perform various functions or operations. Accordingly, as shown in FIG. 2, the system 200 may comprise a measured sensor data module (or a measured sensor data circuit) 206 configured to obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; a process offset data producing module (or a process offset data producing circuit) 208 configured to produce, for the above-mentioned each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; a process skew fingerprint information determining module (a process skew fingerprint information determining circuit) 210 configured to determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and an anomaly detection module (or an anomaly detection circuit) 212 configured to detect anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process. [0032] It will be appreciated by a person skilled in the art that the above-mentioned modules are not necessarily separate modules, and one or more modules may be realized by or implemented as one functional module (e.g., a circuit or a software program) as desired or as appropriate without deviating from the scope of the present invention. For example, two or more of the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and the anomaly detection module 212 may be realized (e.g., compiled together) as one executable software program (e.g., software application or simply referred to as an “app”), which for example may be stored in the memory 202 and executable by the at least one processor 204 to perform various functions/operations as described herein according to various embodiments of the present invention.
[0033] In various embodiments, the system 200 corresponds to the method 100 of detecting anomaly as described hereinbefore with reference to FIG. 1, therefore, various functions or operations configured to be performed by the least one processor 204 may correspond to various steps or operations of the method 100 of detecting anomaly as described herein according to various embodiments, and thus need not be repeated with respect to the system 200 for detecting anomaly for clarity and conciseness. In other words, various embodiments described herein in context of the methods are analogously valid for the corresponding systems, and vice versa. [0034] For example, in various embodiments, the memory 202 may have stored therein the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212, which respectively correspond to various steps (or operations or functions) of the method 100 of detecting anomaly as described herein according to various embodiments, which are executable by the at least one processor 204 to perform the corresponding functions/operations as described herein.
[0035] A computing system, a controller, a microcontroller or any other system providing a processing capability may be provided according to various embodiments in the present disclosure. Such a system may be taken to include one or more processors and one or more computer-readable storage mediums. For example, the system 200 described hereinbefore may include a processor (or controller) 204 and a computer-readable storage medium (or memory) 202 which are for example used in various processing carried out therein as described herein. A memory or computer-readable storage medium used in various embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).
[0036] In various embodiments, a “circuit” may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing software stored in a memory, firmware, or any combination thereof. Thus, in an embodiment, a “circuit” may be a hard-wired logic circuit or a programmable logic circuit such as a programmable processor, e.g., a microprocessor (e.g., a Complex Instruction Set Computer (CISC) processor or a Reduced Instruction Set Computer (RISC) processor). A “circuit” may also be a processor executing software, e.g., any kind of computer program, e.g., a computer program using a virtual machine code, e.g., Java. Any other kind of implementation of the respective functions which will be described in more detail below may also be understood as a “circuit” in accordance with various alternative embodiments. Similarly, a “module” may be a portion of a system according to various embodiments in the present invention and may encompass a “circuit” as above, or may be understood to be any kind of a logic-implementing entity therefrom.
[0037] Some portions of the present disclosure are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to convey most effectively the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.
[0038] Unless specifically stated otherwise, and as apparent from the following, it will be appreciated that throughout the present specification, description or discussions utilizing terms such as “obtaining”, “producing”, “determining”, “detecting”, “estimating” or the like, refer to the actions and processes of a computer system, or similar electronic device, that manipulates and transforms data represented as physical quantities within the computer system into other data similarly represented as physical quantities within the computer system or other information storage, transmission or display devices.
[0039] The present specification also discloses a system (e.g., which may also be embodied as a device or an apparatus), such as the system 200, for performing various operations/functions of various methods described herein. Such a system may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose machines may be used with computer programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform various method steps may be appropriate.
[0040] In addition, the present specification also at least implicitly discloses a computer program or software/functional module, in that it would be apparent to the person skilled in the art that individual steps of various methods described herein may be put into effect by computer code. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the scope of the invention. It will be appreciated by a person skilled in the art that various modules described herein (e.g., the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212) may be software module(s) realized by computer program(s) or set(s) of instructions executable by a computer processor to perform the required functions, or may be hardware module(s) being functional hardware unit(s) designed to perform the required functions. It will also be appreciated that a combination of hardware and software modules may be implemented.
[0041] Furthermore, one or more of the steps of a computer program/module or method described herein may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer. The computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the methods described herein.
[0042] In various embodiments, there is provided a computer program product, embodied in one or more computer-readable storage mediums (non-transitory computer-readable storage medium(s)), comprising instructions (e.g., the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212) executable by one or more computer processors to perform the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein with reference to FIG. 1 according to various embodiments. Accordingly, various computer programs or modules described herein may be stored in a computer program product receivable by a system therein, such as the system 200 as shown in FIG. 2, for execution by at least one processor 204 of the system 200 to perform various functions.
[0043] Software or functional modules described herein may also be implemented as hardware modules. More particularly, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it can form a portion of an entire electronic circuit such as an Application Specific Integrated Circuit (ASIC). Numerous other possibilities exist. Those skilled in the art will appreciate that the software or functional module(s) described herein can also be implemented as a combination of hardware and software modules.
[0044] In various embodiments, the system 200 may be realized by any computer system (e.g., desktop or portable computer system) including at least one processor and a memory, such as a computer system 300 as schematically shown in FIG. 3 as an example only and without limitation. Various methods/steps or functional modules (e.g., the measured sensor data module 206, the process offset data producing module 208, the process skew fingerprint information determining module 210 and/or the anomaly detection module 212) may be implemented as software, such as a computer program being executed within the computer system 300, and instructing the computer system 300 (in particular, one or more processors therein) to conduct various functions or operations as described herein according to various embodiments. The computer system 300 may comprise a computer module 302, input modules, such as a keyboard and/or a touchscreen 304 and a mouse 306, and a plurality of output devices such as a display 308, and a printer 310. The computer module 302 may be connected to a computer network 312 via a suitable transceiver device 314, to enable access to e.g., the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN). The computer module 302 in the example may include a processor 318 for executing various instructions, a Random Access Memory (RAM) 320 and a Read Only Memory (ROM) 322. The computer module 302 may also include a number of Input/Output (I/O) interfaces, for example I/O interface 324 to the display 308, and I/O interface 326 to the keyboard 304. The components of the computer module 302 typically communicate via an interconnected bus 328 and in a manner known to the person skilled in the relevant art.
[0045] It will be appreciated by a person skilled in the art that the terminology used herein is for the purpose of describing various embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
[0046] Any reference to an element or a feature herein using a designation such as “first”, “second” and so forth does not limit the quantity or order of such elements or features, unless stated or the context requires otherwise. For example, such designations may be used herein as a convenient way of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not necessarily mean that only two elements can be employed, or that the first element must precede the second element. In addition, a phrase referring to “at least one of’ a list of items refers to any single item therein or any combination of two or more items therein.
[0047] In order that the present invention may be readily understood and put into practical effect, various example embodiments of the present invention will be described hereinafter by way of examples only and not limitations. It will be appreciated by a person skilled in the art that the present invention may, however, be embodied in various different forms or configurations and should not be construed as limited to the example embodiments set forth hereinafter. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present invention to those skilled in the art. [0048] In particular, for better understanding of the present invention and without limitation or loss of generality, unless stated otherwise, various example embodiments of the present invention will now be described with respect to attack detection on a networked control system for a water treatment plant. However, it will be understood by a person skilled in the art that the present invention is not limited to being applied to or implemented in a networked control system for a water treatment plant, and may be applied to or implemented in any industry as desired or as appropriate that may require an industrial process control, such but not limited to, chemical processing, power generation, oil and gas processing, and so on.
[0049] In an ICS, its complex network of sensors, actuators and controllers have raised security concerns. In this regard, various example embodiments provide a method (or technique), which may be referred to herein as a process skew based detection method, that uses process offsets (which may also be referred to as process skews, e.g., small deviations) in the ICS process for anomaly detection (e.g., attack detection). In various example embodiments, the process skew based detection method (e.g., corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein according to various embodiments) may be based on fingerprinting a physical process for anomaly detection (i.e., determining process skew fingerprint information (i.e., process skew based fingerprint information), which may be simply referred to herein as a process skew fingerprint). In this regard, various example embodiments determine the process skew fingerprint based on noise in sensor measurements due to the process fluctuations (or process offsets). Various example embodiments found that a process skew fingerprint determined such a manner is unique to a physical process due to the intrinsic operational constraints of the physical process, and is hard to be forged even for a powerful attacker knowing the process operation. As will be described later below, the process skew based detection method was validated using data from a real-world water treatment testbed. In this regard, experimental results obtained demonstrate that the process skew based detection method effectively detected anomaly in a physical process associated with a networked control system (e.g., detected process anomaly with a very low false-positive rate), as well as demonstrate that the process skew based detection method can effectively identify a process (e.g., a state process at a process state of the physical process) based on the process skew fingerprint.
[0050] An ICS is a networked control system comprising sensors, actuators, controllers and communication networks configured to control one or more physical processes in an industry. For example, connectivity in an ICS provides improved monitoring and operation of a physical process. Such advancements are helpful but also bring about challenges of secure operation of the connected devices for the ICS to operate securely.
[0051] For example, as discussed in the background, an ICS can be subject to cyber and/or physical attacks, which can be launched either remotely or locally. Attackers may tamper sensor reading or inject spoofing sensor data, and manipulate the actuators to cause anomaly of operations, which may eventually lead to physical damages to the ICS. However, traditional intrusion detection methods based on network traffic cannot detect many low layer attacks originated in the physical domain, as there may be no abnormal network traffic.
[0052] For example, sensor data may be transmitted to a PLC to perform an appropriate action based on the sensor measurement. In this regard, if an adversary can spoof sensor data in the digital or physical domain, the adversary can cause the ICS to go to an unsafe state. Therefore, according to various example embodiments, the focus is not on the confidentiality of the sensor data as in legacy computer security but the integrity and trustworthiness of the sensor data.
[0053] Furthermore, an open problem with conventional attack detection methods is that it is not possible to localize the source of attacks. This is especially an issue associated with conventional machine learning-based methods that use raw data from a process and feed them to well-known machine learning models. In this regard, various example embodiments note that, in contrast to the process skew based detection method, these conventional attack detection methods are not aware of or do not take into account process dynamics, and thus, they are unable to determine or locate the physical state where the attack occurred.
[0054] Accordingly, in various example embodiments, there is provided a process skew based detection method (e.g., corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein according to various embodiments) configured to identify a physical process (e.g., a state process of the physical process) and detect data integrity attacks in an ICS in an effective manner. In various example embodiments, the process skew based detection method uses deviations (e.g., small deviations) in a physical process (in particular, deviations in measured sensor data associated with the physical process) with respect to a modelled physical process (in particular, modelled sensor data of the physical process) modelled based on design parameters (e.g., which may be referred to as process offsets or process skews) for fingerprinting the physical process so as to obtain a process skew fingerprint associated with the physical process (in particular, a process fingerprint associated with the measured sensor data associated with the physical process). For example, these deviations in the physical process may be noise that appears in sensor measurements due to the process fluctuations (or process offsets). Advantageously, uniqueness in the process skew fingerprint obtained can be achieved due to the specified operational constraints of the physical process.
[0055] Accordingly, various example embodiments create a process skew fingerprint by extracting process offset information (or process skew information) from the sensor measurements. In this regard, various example embodiments note that for a physical process, due to inaccuracies in the physical process, it would have a skew from what it is designed for. An example is that of a water pipe delivering water to fill a tank. Pipes and tanks of two different sizes would take/store a different amount of water. Even if the pipes are of the same size, two different amounts of pumping force would result in a different amount of water flowing or being stored. In this example, the flow of water in a pipe and water storage in a tank are examples of state processes of a physical process. At the design stage, these state processes may be designed to meet certain operational requirements (i.e., based on design parameters). However, various example embodiments note that when these state processes are running, they show small deviations or offsets from the designed parameters due to the physical inaccuracies in the state processes, for example, no two water pipes can have the same diameter at a micro-scale due to manufacturing imperfections. Therefore, the process skew based detection method according to various example embodiments advantageously makes use of inaccuracies in the physical process itself. Moreover, the process skew based detection method does not depend on the specific state of the system (e.g., does not need to wait for the process to be static) and uses the dynamics of the physical process (or a state process thereof) to create a system model, and uses process skews (or process offsets) for fingerprinting the physical process (or a state process thereof) so as to obtain a process skew fingerprint associated with the physical process (or a state process thereof). Accordingly, the process skew based detection method advantageously provides a distinctive way of passively fingerprinting processes, by using process skews to determine a process skew fingerprint to detect attacks.
[0056] Accordingly, in various example embodiments of the present invention, there are provided a method of utilizing process skews to fingerprint a physical process, a method of analyzing the effects of stealthy attacks on a process skew based detection method, and a method of detecting sensor attacks under a multitude of adversarial scenarios.
[0057] By way of an example only for illustration purpose and without limitation, the Secure Water Treatment Testbed (SWaT) at the Singapore University of Technology and Design will now be described as an example ICS, along with an example implementation of the process skew based detection method thereto for detecting an attack, according to various example embodiments of the present invention.
[0058] ICS is a broad domain of connected industrial systems. A particular example of a water treatment industrial process, and more particularly, the SWaT, is considered. The SWaT is a fully functional testbed and is open for researchers to use. A brief introduction of the SWaT is provided below, but is described in detail in A. P. Mathur and N. O. Tippenhauer, "SWaT: a water treatment testbed for research and training on ICS security," 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater), 2016, pp. 31-36, doi: 10.1109/CySWater.2016.7469060. The SWaT testbed produces purified water and it is a scaled-down version of a real water treatment process. FIG. 4 depicts a schematic drawing of the SWaT Test-bed Network Architecture 400. In FIG. 4, it can be seen that the testbed is distributed and there are different stages, where each stage is labeled as Pn where n is the nth stage. There are six stages in the SWaT testbed, namely, PI through P 6. Each stage is equipped with a set of sensors and actuators. Sensors include water quantity measures, such as level, flow, and pressure and water quality measures, such as pH, ORP and conductivity. Actuators are different motorized valves and electric pumps. Stage 1 is a raw water stage configured to hold the raw water for the treatment and stage 2 is a chemical dosing stage configured to treat the water depending on the measurements from the water quality sensors. Stage 3 is an ultrafiltration stage. Stage 4 includes a de-chlorinator and stage 5 is equipped with reverse osmosis filters. Stage 6 is configured to hold the treated water for distribution. In the SWaT, data from the sensors and actuators are communicated to the PLCs using a level 0 network and PLC communicates to each other over a level 1 network as shown in FIG. 4.
[0059] According to various example embodiments, process skews (or process offsets) are extracted from measured sensor data. FIG. 5 depicts a schematic flow diagram illustrating an overview of a process skew based detection method 500 according to various example embodiments of the present invention (e.g., corresponding to the method 100 of detecting anomaly in a physical process associated with a networked control system, as described herein according to various embodiments). The method 500 comprises extracting (at 504) measurements (i.e., measured sensor data) for a specific state of the physical process (i.e., a process state of the physical process). For example, measured sensor data associated with each process state of the physical process may be obtained. In this regard, based on the actuator data, it is possible to determine the process state of the physical process. For example, if the inlet pump is switched on, then the water is being filled in a tank, and thus, by knowing the state of the inlet pump, it is possible to know the process state of the physical process (e.g., in this case, the process state is that of the tank being filled with water, and the state process is a process at that process state, such as, the process of the water filling the tank). However, such state information from the sensors and actuators may be spoofed by an attacker.
[0060] Next, at 508, based on the state process at the process state of the physical process, a system model, along with the design parameters associated with the state process at the process state of the physical process, is used to model (e.g., estimate based on modelling) the physical state associated with the state process to obtain estimated sensor data (which may be referred to as modelled sensor data since the estimated sensor data is obtained based on a system model) in relation to the physical state associated with the state process at the process state (e.g., in the above-mentioned case, the physical state may be the water level in a tank).
[0061] At 512, the difference between the estimate (modelled sensor data) and real sensor measurement (measured sensor data) establishes an offset value (or a process offset), which corresponds to an amount by which the physical process (or state process) is offset from what it should be, as designed. This process offset data may be obtained for each process run of a plurality of process runs for a state process at the process state of the physical process, thereby obtaining a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process. Furthermore, for each process run, the difference between the modelled sensor data and the measured sensor data associated with the process run may be obtained or determined at each time step of a plurality of time steps, thereby obtaining a plurality of process offsets (or a plurality of process offset information) associated with the process run at the plurality of time steps, respectively. In this regard, the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
[0062] The plurality process offset data associated with the plurality of process runs for a state process, when accumulated over time, reveal the process skews associated with the plurality of process runs. Various example embodiments note that the plurality of process offset data associated with the plurality of process runs for a state process obtained may have fluctuations, for example, due to sensor noise. In this regard, various example embodiments apply (at 516) a linear regression model to the plurality of process offset data to obtain the best fit for the state process, so as to determine process skew fingerprint information (which may simply be referred to herein as a process skew fingerprint) associated with the plurality of process runs. In various example embodiments, the process skew fingerprint may be obtained by calculating the rate of change of linear regression on the plurality of process offset data with respect to time. According to various example embodiments, a theoretical proof based on the calculated entropy of the process skew fingerprint is used to establish the uniqueness of the process skew fingerprint obtained. At 520, an anomaly detector (e.g., a CUSUM detector) may be used to detect attacks on the networked control system (e.g., the SWaT) based on the process skew fingerprint. The process skew based detection method 500 will be described in further details later below according to various example embodiments of the present invention.
[0063] In an ICS, sensors play an important role by sending physical measurements (measured sensor data) to a controller to execute an appropriate control action. However, an adversary can spoof sensor measurements either through cyber domain or physical domain intrusions. Accordingly, it is important to detect or authenticate whether the data is originating from the real physical process or is being modified in some manner. For example, various example embodiments note that due to computational limitations and legacy compliant equipment, it is not feasible to rely on cryptographic methods. Accordingly, various example embodiments provide a process skew based detection method 500 to achieve a process skew based authentication of a physical process. In this regard, various example embodiments seek to identify a physical process (or a state process of the physical process) based on its physical dynamics. In various example embodiments, specific cyber attacks are also considered on sensor measurements in a water treatment plant. FIG. 6 depicts a schematic drawing showing an abstraction of an ICS 600 in relation to a thread model, whereby may or may not be
Figure imgf000025_0002
attacked sensor measurement. In FIG. 6, it can be seen that an attacker can modify a rightful sensor measurement by an attack value . For illustration purpose, example types of attacks
Figure imgf000025_0001
launched on the SWaT in experiments conducted will be described later below. In general, the attacker model encompasses the attacker’ s intentions and capabilities. For example, the attacker may choose its goals from a set of intentions, including performance degradation, disturbing a physical property of the system, or damaging a component. In the experiments conducted according to various example embodiments, a range of attacks are considered from already published attack scenarios in the literature.
[0064] An example attacker model will now be described. In the example attacker model, it is assumed that the attacker has access to the sensor’s measurements. For example, a powerful attacker can arbitrarily change sensor measurements to the desired sensor value. A malicious insider may be an attacker with physical access to the plant and thus to its devices, such as level sensors. However, an attacker who can physically replace or tamper sensors may not necessarily be an insider, because critical infrastructures, e.g., for water and power, are generally distributed across large areas. An outsider, for example, an end user, may also carry out a physical attack on sensors, such as smart energy monitors.
[0065] For example, in data injection attacks, an attacker injects or modifies the real sensor measurement. In general, for a complex ICS, there may be many possible attack scenarios. For illustration purpose, various example embodiments consider a generic attack to show the performance of the process skew based detection method 500 according to various example embodiments of the present invention. Accordingly, it will be appreciated by a person skilled in the art that the present invention is not limited to any particular or specific type of attack. However, a stealthy attack may be a worst case scenario for a model based detection technique where an adversary tries to deceive the detection mechanism by creating attack vectors 5k based on working principle of the attack detection technique. In various example embodiments, the following types of data injection attack scenarios are considered.
[0066] A type of data injection attack scenario is a generic sensor spoofing attack. In this regard, the process skew based detection method 500 was evaluated for a range of network attack scenarios from benchmark attacks on the SWaT testbed. These benchmark attacks cover a wide range of 36 attacks on both sensors and actuators. Since the process skew based detection method 500 extracts process skew information for various physical properties, attacks on chemical sensors are thus excluded, resulting in a total of 25 attacks remaining as summarized in Table 1 shown in FIGs. 7A and 7B. In particular, Table 1 provides a list of attacks on the SWaT testbed carried out in experiments conducted according to various example embodiments of the present invention. In general, an attack vector may be defined as:
Figure imgf000026_0001
Equation (1) where yk are the real sensor measurement (measured sensor data), is sensor measurement
Figure imgf000026_0003
with a possible attack and is the data injected by an attacker at time step k. Details about
Figure imgf000026_0002
each (attack vector) is described in Table 1 shown in FIGs. 7 A and 7B, where it can be seen that it ranges from an abrupt injection of data to a more slow/stealthy change in sensor measurements.
[0067] Another type of data injection attack scenario is stealthy attacks for model based techniques. These attacks are designed to be stealthy by changing sensor measurements such that the system model based detection mechanism would fail. Since model based detectors use a system model, a Kalman filter and a statistical detector, an attacker who wants to remain stealthy may try to choose injected readings so as not to exceed the detector threshold. To do this, an attacker may learn the system model and the detector parameters. In this regard, various example embodiments assume that an attacker has the ability to do so, but does not possess the process skew knowledge.
[0068] In relation to attack execution, all the attacks which are obtained from the SWaT dataset disclosed in Goh J., et ah, “A Dataset to Support Research in the Design of SecureWater Treatment Systems”, In Critical Information Infrastructures Security, Grigore Havarneanu, Roberto Setola, Hypatia Nassopoulos, and Stephen Wolthusen (Eds.), Springer International Publishing, Cham, pages 88-99, 2017, are executed by compromising the Supervisory Control and Data Acquisition (SCAD A) system. An attack toolbox was used to inject an arbitrary value for real sensor measurement.
[0069] The process skew based detection method 500 will now be described in further details according to various example embodiments of the present invention.
[0070] A method of extracting state processes (or process states) of a physical process will now be described according to various example embodiments of the present invention. An example from the SWaT testbed is considered. FIG. 8 depicts a plot of level sensor (LIT- 101) measurements in the SWaT testbed in stage 1 labeled as LIT- 101 for a duration of a normal process (i.e., under normal operation). In particular, FIG. 8 shows multiple runs for a state process at a process state, e.g., water filling, water flowing out, both or none of the previous process. Each of these possible state processes in the water tank are labeled as SI to S4, respectively, and their descriptions are provided in Table 2 shown in FIG. 9. The design parameters in Table 2 indicate the design for the inflow and outflow processes and which state process is present in a particular process state. The tank 1 in stage 1 of the SWaT testbed has one inlet valve labeled as MV- 101 and one outlet pump labeled as P-101. There is also a secondary backup pump at the outlet labeled as P-102. Accordingly, there can be four possible state processes for the water level (physical state) in tank 1 based on input and output flow processes, namely, output flow process is present but no input flow process (SI), neither input nor output flow processes (S2), input flow process is present but no output flow process (S3), both input and output flow processes are present (S4).
[0071] In experiments conducted, the water treatment plant was operated for seven days continuously and measured sensor data for normal operations (e.g., without attack and breakdown) of the water treatment plant were collected. FIGs. 10A to 10D depict four plots showing four possible state processes of a physical process, respectively, along with measured sensor data obtained with respect to the water level in a water tank (tank 1) as described with reference to Table 2. The level sensor in the SWaT testbed in stage 1 is labeled as LIT- 101 under normal operations. Accordingly, the measured sensor data shown in FIGs. 10A to 10D present the particular state processes extracted from the seven days of normal operations. As shown, for each state process, there may be a plurality of occurrences (or process runs), for example, hundreds of occurrences. For example, for the water emptying process (SI) shown in FIG. 10A, there may be hundreds of occurrences (hundreds of process runs) of the water empty process during the observation period. From FIGs. 10A to 10D, the effects of noise (e.g., deviations or fluctuations) on the measured sensor data obtained in the plurality of process runs noted according to various example embodiments of the present invention can be seen.
[0072] Each physical process (or state process of the physical process) is expected to behave according to design parameters, such as shown in Table 2 in FIG. 9. However, as can be observed in FIGs. 10A to 10D, there are deviations due to the process noise. For example, in FIG. 10A, a first state process S 1 shows different process runs of the water emptying process from the tank 1. In this regard, the variations in each process run due to the sensor noise can be observed in FIG. 10A. This can also be observed from the static (S2) and water filling (S3, S4) state processes. In this regard, according to various example embodiments of the present invention, based on these deviations (e.g., due to noise) in the state processes, variations due to process offsets in the state processes from those expected based on corresponding design parameters are determined. In various example embodiments, to quantify the amount of process skews associated with these state processes, the process dynamics for these state processes based on the design parameters (e.g., under the designed set points) are learned or determined. [0073] A design based system model according to various example embodiments will now be described
[0074] FIG. 11 depicts a plot illustrating the concept of process skews associated with a physical process, according to various example embodiments of the present invention. In FIG. 11, the sensor measurements (measured sensor data) with respect to the water level for the water filling process and estimated sensor value (modelled sensor data) based on design parameters are shown. The difference between the measured sensor data and the modelled sensor data at each time step may be referred to as a process offset. In FIG. 11, the accumulated process offsets over time are also labelled for illustration purpose and better understanding. [0075] For illustration purpose, FIG. 12 depicts a schematic diagram of an example physical system 1200 for stage 1 according to various example embodiments of the present invention, for modelling the physical process with respect to the level sensor in a tank (Tank 1) 1204. In the example physical system 1200, the tank 1204 in stage 1 of the SWaT testbed is being used as a running example to demonstrate the process skew based detection method 500 according to various example embodiments of the present invention. In FIG. 12, as shown, the water level in the tank 1204 is measured using a level sensor 1208 and the inflow and outflow of the water are controlled by the motorized valve (MV- 101) 1212 at the input and pump (P-101) 1216 at the output, respectively. Various example embodiments model this inflow and outflow by considering the physical principles and the design parameters associated with the physical process. In particular, process skew information associated with the measured sensor data is extracted based on determining process dynamics drift from the design due to process noise. For example, for a tank, the rate of change of water inside the tank is equal to the difference between water flowing into the tank and water flowing out from the tank with respect to time. Accordingly, the rate of change of water inside the thank may be represented using a mass- balance equation, such as, = A x h
Figure imgf000029_0001
Equation (2) where V denotes the volume of the tank, A denotes the cross-sectional area of the tank, and h denotes the height of the water inside the tank. Equation (2) provides a linear equation, and thus, the expression [Qin — Qout] represents the water flow which depends on the PLC control actions implemented via the valve (MV- 101) 1212 and the pump (P-101) 1216.
[0076] Accordingly, from FIG. 12, it can be seen that using the height and diameter of the tank from design documents, it is possible to determine the volume and the cross-sectional area of the tank. Assuming that the physical state of the physical process corresponds to the height of water (i.e., water level) inside the tank 1204, then the solution of Equation (2) produces the following result, [xk+1 = xk + uk], where uk is the PLC control action. In this regard, xk represents water level in the tank at time k, and the control action uk can be either open/close (for the motorized valve 1212) or on/off (for the pump 1216). Furthermore, by defining the sensor state, the following set of system equations can be obtained:
Figure imgf000030_0001
Equation (3) where yk is the sensor measurement (measured sensor data) driven by the control action uk, and matrices A, B and C are the state-space matrices of appropriate dimensions.
[0077] From Equation (3), it can be seen that for a system state value at time k, and given the PLC control uk, the next state at time k + 1 can be predicted. Table 2 in FIG. 9 shows a list of example design parameters for each type of control action. For example, for state process S4, the valve MV- 101 1212 is controlled to be in an open state and the pump P-101 1216 is controlled to be in an activated (switched on) state. For example, based on these control information from the PFC, various example embodiments estimate the sensor data of the physical state of the state process based on design parameters associated with the state process (e.g., estimated water level in the tank based on design parameters). However, as explained hereinbefore and further described below, for example due to the process noise, there are deviations between the measured sensor data and the estimated sensor data (or modelled sensor data) with respect to the physical state.
[0078] A method of extracting process offsets in measured sensor data will now be described according to various example embodiments of the present invention. Using the process design parameters and the system of equations in Equation (3), process offsets (or process skews) may be extracted, for example, how much the real process dynamics are offset from the designed physical process. For example, in FIGs. 13A to 13C to be described later below, the process offsets associated with the measured sensor data with respect to the level of the water in the above-mentioned tank 1204 for the three different state processes, namely SI, S3 and S4, can been seen.
[0079] In various example embodiments, for each time step, the process offset at the time step may be defined as a deviation of the process dynamics due to the process inaccuracies from the design at the time step. For example, for each time step over a time period (e.g., the duration while a process run is active, which may be referred to as a process activity time frame), the process offset associated with the process run may be calculated or determined. All the process offsets associated with the process run may then be accumulated over the time period, to obtain process offset data (including the process offsets accumulated over the time period) associated with the process run. A plurality of process offset data may thus be obtained for a plurality of process runs for a state process (e.g., for the plurality of process runs of the water emptying process (SI) as shown in FIG. 13A). Process skew fingerprint information associated with the plurality of process runs for the state process may then be determined or extracted based on the plurality of process offset data obtained.
[0080] In various example embodiments, the process skew fingerprint information associated with a plurality of process runs may be determined based on a slope of the plurality of process offset data obtained on the plurality of process runs.
[0081] In FIGs. 13A to 13C, the accumulative process offsets for three different state processes, namely, SI, S3 and S4, respectively, can be seen. State process SI is a water emptying process at the water emptying state (i.e., water outflow from the tank). In this regard, the negative slope indicates that the real process is actually slower than designed (i.e., based on design parameters). State process S2 is a static process corresponding to the process state of being static (i.e., there is no inflow or outflow), and thus, this state process is actually missing so no process offset exists. State process S3 is a water filling process at the water filling state (i.e., only the inflow is present). In this regard, the positive slope indicates that the real process is actually faster than designed. State process S4 is a water filling and emptying process at the water filling and emptying state (i.e., both the inflow and outflow are present). In this regard, the negative slope indicates that the real process is actually slower than designed. All these state processes (or process states) scenarios may be different state processes (or process states) of the same physical process, that is, with respect to the water tank 1204 in stage 1 of the SWaT testbed. In this regard, although they may be different state processes (or process states) of the same physical process, it can be observed that based on the corresponding process offsets obtained, all the state processes (or process states) of the physical process can be distinguished from each other. Accordingly, the method of extracting the process offsets in measured sensor data according to various example embodiments of the present invention advantageously establishes a process skew fingerprint, that is, fingerprinting the process runs for a process state based on process skews to obtain a corresponding process skew fingerprint.
[0082] Various example embodiments note that, as can be observed in FIGs. 13A to 13C, the process offsets obtained are noisy, for example, due to the sensor noise. In this regard, various example embodiments seek to remove, or at least mitigate, the sensor noise effect without disturbing the process offsets. Accordingly, in various example embodiments, process skew fingerprint information associated with a plurality of process runs for a state process is derived based on the plurality of process offset data produced for the plurality of process runs in the following manner. [0083] Consider the linear time invariant model of the system with sensor and process noise as:
Figure imgf000032_0002
Equation (4) where yk is the sensor measurement with the measurement noise
Figure imgf000032_0001
and xk+1 is the system state.
[0084] At each time step, the difference between sensor measurements given by Equation (4) and sensor measurement estimate given by Equation (3) is calculated to obtain the process offset as,
Figure imgf000032_0003
[0085] Without wishing to be bound by theory, a proof for the above derivation of the process offset is provided as follows. The difference between Equations (4) and (3) may be expressed as:
Figure imgf000032_0004
Equation (5)
Figure imgf000032_0005
Equation (6)
Figure imgf000032_0006
Equation (7)
[0086] As the process offset is defined as the difference the real system state and the estimated state of the system it produces,
Figure imgf000032_0007
Equation (8)
Accordingly, it can be seen from Equation (8) that the process offset (Ok) can be extracted at each time step.
[0087] From Equation (8), it can be observed that the process offset includes the noise from the sensor. In this regard, various example embodiments apply a linear regression model on the plurality of process offset data to obtain a regression coefficient, which corresponds to the process skew fingerprint information. For example, a straight line may be fitted onto the plurality of process offset data to determine the slope of the plurality of process offset data. Accordingly, for each state process (e.g., each of the three state processes shown in FIGs. 13A to 13C), a straight line may be fitted onto the plurality of process offset data associated with the plurality of process runs for the state process in relation to the physical state (e.g., water level) to determine the slope thereof, which may represent the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process. Accordingly, various example embodiments apply a linear regression model on the plurality of process offset data obtained for each state process. For example, from FIG. 13 A to 13C, it can be observed that the process offsets obtained for each state process are generally linear in time.
[0088] In various example embodiments, to establish the linearity between the time and the progression of the process offsets, correlation coefficients are used. Correlation calculates the level of the linear relationship between variables. For example, a high correlation between two variables indicates that the values for the variables increase or decrease in a linear relationship. However, uncorrelated variables may still be dependent on each other, just that the relationship may be nonlinear. For example, for N scalar values of two variables, the Pearson correlation coefficient may be defined as:
Figure imgf000033_0001
Equation (9) where X denotes the mean of the variable X, and 7 denotes the mean of the variable Y. In this regard, various example embodiments found that the process data is linearly correlated with the time as the process is linearly increasing or decreasing in time. In various example embodiments, a linear regression approach is adopted to obtain a data model describing the relationship between variables in a mathematical form. In various example embodiments, a least squares fit is applied to obtain the model for a set of n observed values of X and Y given by respectively. These values for a system of linear
Figure imgf000033_0005
equations may be represented in matrix form as:
Figure imgf000033_0002
which may be simplified to:
Figure imgf000033_0003
Equation (10) where is the y-intercept, is the slope/regression coefficient and e is the model error.
Figure imgf000033_0004
[0089] FIG. 14 depicts a plot of a plurality of process offset data for the water filing process in stage 1 of the water treatment system, with a linear regression model fitted thereon, according to various example embodiments of the present invention. The linear regression model is used to find the slope that defines process skew fingerprint information representing the process skews associated with the plurality of process runs. For example, the accuracy of the linear regression model can be seen in FIG. 14. To quantify the goodness of a system model, mean square error (MSE) may be used as a metric. In particular, one minus the root mean square error (RMSE) defines the estimation accuracy or best fit of a model,
Figure imgf000034_0001
Equation (11)
[0090] MSE is the difference between measured sensor data and estimated sensor data (or modelled sensor data) squared, and in general, provides the distance between measured and estimated value, in other words, how far the estimated value is from the measured value. The model accuracies for the three stages of the SWaT testbed and corresponding state processes used in the experiments conducted (from the SWaT testbed) are shown in Table 3 in FIG. 17. It can be seen that the obtained system model is very accurate, with almost zero mean error for all the process runs of a plurality of state processes. Table 3 in FIG. 17 shows the mean of models created for all the process runs of the plurality of state processes. Accordingly, from Table 3, the use of the linear regression model to find a good fit for the plurality of process offset data to determine the corresponding process skew fingerprint information is validated. In various example embodiments, the process offsets accumulated for a process run of a state process may be expressed as:
Figure imgf000034_0002
Equation (12) and the corresponding process skew fingerprint information (ProcessSkew) may be determined as:
Figure imgf000034_0003
Equation (13)
[0091] The uniqueness of the process skew fingerprint information obtained will now be described according to various example embodiments of the present invention. FIG. 15A depicts a plot of three process skew fingerprint information obtained with respect to three state processes, respectively, at stage 1 of the SWaT testbed, FIG. 15B depicts a plot of three process skew fingerprint information obtained with respect to three state processes, respectively, at stage 3 of the SWaT testbed, and FIG. 15C depicts a plot of two process skew fingerprint information obtained with respect to two state processes, respectively, at stage 4 of the SWaT testbed. Accordingly, FIGs. 15A to 15C depict process skew fingerprint distribution for eight state processes at three stages of the SWaT testbed. From FIGs. 15A to 15C, it can be observed that all of the eight state processes can be uniquely distinguished based on the process skew fingerprint profile, according to various example embodiments of the present invention. Therefore, FIGs. 15A to 15C show a visual analysis for process skew fingerprint uniqueness. In addition, without wishing to be bound by theory, a mathematical proof for the process skew fingerprint uniqueness will now be described. For example, demonstrating that fingerprints are information-theoretically unique helps to negate the possibility of impersonation attacks. Let w(t) be the signal corresponding to a process skew fingerprint. In order to present an information-theoretic analysis, justification of two important criteria are provided:
(1) mutual information between process skew fingerprints as recorded for the same process, that is, in successive operations should be high, ~ 1, and
(2) conditional entropy of process skew fingerprints with other process skew fingerprints should be very low, «1.
[0092] In order to investigate these relations, mutual information, /(), for process i is defined
Figure imgf000035_0001
entropy of 7th attempt by a process
Figure imgf000035_0002
is conditional entropy of i* process for jth attempt, given the features of kLh attempt. For high recall, mutual information for each of the process skew fingerprints should be close to 1 (normalized). Similarly, an ICS process i should not have access to any extra information about process t given observations of its own. Mathematically this can be quantified in condition entropy as
Figure imgf000035_0003
Entropy measure and mutual information were evaluated for each of the process skew fingerprints. As can be seen in FIG. 16, mutual information across the above-mentioned eight process skew fingerprints are fairly low (less than 0.1), which demonstrates the uniqueness of the process skew fingerprints obtained. Entropy of each of the process skew fingerprints was recorded to be > 0.94. In addition, investigation of conditional entropy across different processes of the ICS system reveals that features are independent. [0093] An example anomaly detector according to various example embodiments of the present invention will now be described. As described hereinbefore, the process offsets for different runs of a particular state process at a process state (e.g., a water filling process) are accumulated. In various example embodiments, a process skew vector (determined based on the corresponding process skew fingerprint information) is provided as an input to the anomaly detector, which for example, may be a cumulative sum (CUSUM) detector and known as the stateful detector. The input (process skew vector) to the CUSUM procedure may be considered as a distance measure, that is, a measure of how far the estimated measurements is from the expected measurements. In various example embodiments, a dedicated anomaly detector for each process state may be designed. The index i denotes the process, , where
Figure imgf000036_0011
m is the number of processes in each stage of the plant. Process skew fingerprint is denoted as for easy reference, where k is the time step. The standard CUSUM procedure (e.g., as described in D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 2009) is explained using the following equations.
Figure imgf000036_0001
Equation (14)
Figure imgf000036_0003
Equation (15)
Design parameters: Bias ; threshold
Figure imgf000036_0004
Figure imgf000036_0005
Output: Alarm(
Figure imgf000036_0002
[0094] From Equations (14) and (15), it can be observed that accumulate the
Figure imgf000036_0006
distance measure over time to measure how far are the values of the residual from the target
Figure imgf000036_0007
mean . To tune the CUSUM detector, there is also a slack variable k chosen to be in
Figure imgf000036_0009
this experiment conducted. where G is a multiplier to the standard deviation ( s )
Figure imgf000036_0008
and may typically be between 3 and 5. In various example embodiments, an alarm is raised when this accumulation becomes greater or less than a predetermined threshold For example,
Figure imgf000036_0010
based on a threshold that is derived from the process skew fingerprint information obtained under normal operation, an alarm may be raised when the accumulation crosses the threshold. The sequence Sk i is reset to the target mean value each time it becomes negative or larger than Tj . If rk i is tightly bounded and Kt is not sufficiently large, the CUSUM sequence Sk i grows unbounded until the threshold q is reached, no matter how large q is set. In order to prevent such drifts, the slack variable jq is selected properly based on the statistical properties of the distance measure. Once k is chosen, the threshold q must be selected to achieve a required false alarm rate denotes the false alarm rate for the CUSUM procedure defined as
Figure imgf000037_0003
the expected proportion of observations which are false alarms.
[0095] To demonstrate the effectiveness of the process skew based detection method 500 according to various example embodiments, the process skew based detection method 500 is evaluated in a real water treatment testbed. For example, the following metrics are used for performance evaluation. TPt denotes true positive for class q when it is rightly classified based on the ground truth, FNt denotes false negative for class q when it is wrongly rejected, FPt denotes false positive for class q when it is wrongly accepted, and TNL denotes true negative for class q when it is rightly rejected. The True Positive Rate (TPR) and False Positive Rate (FPR) are defined as follows:
Figure imgf000037_0001
Equation (16)
Figure imgf000037_0002
Equation (17)
[0096] Ideally, FPR should be as small as possible and TPR as high as possible. Both TPR and FPR being a ratio ranging between 0 and 1.
[0097] In experiments conducted, normal operation data (or reference operation data, e.g., without attack or breakdown) from the SWaT testbed is collected for a period of seven days. During normal operations, the water treatment plant was run continuously under normal conditions and as it was designed to operate. The operating conditions from the design are presented in Table 2 in FIG. 9. For all the possible state processes, measured sensor data is extracted. Process offsets (e.g., corresponding to the plurality of reference process offset data as described hereinbefore according to various embodiments) are extracted for each state process in stage 1, stage 3 and stage 4 of the SWaT testbed. Stage 2 and stage 5 relate to chemical sensors and reverse osmosis process, respectively, and therefore, these two stages are not considered in the experiments. The experiments are focused on examining the physical properties of the process. During the seven days, water filling or emptying process occurred hundreds of times (i.e., hundreds of process runs for this process state, e.g., corresponding to the plurality of reference process runs as described hereinbefore according to various embodiments). Process offsets are determined for each of these process runs. As explained hereinbefore, process offsets are noisy due to noise from the sensors. In this regard, a linear regression model is fitted on the plurality of process offset data obtained associated with the plurality of process runs to address the noise in the signal. After the linear regression model is fitted, a straight line is obtained for the accumulated process offsets associated with the plurality of process runs over a process time frame. The rate of change of these process offsets (corresponding to the slope of the straight line) may then be defined as the process skew fingerprint (e.g., corresponding to the reference process skew fingerprint information as described hereinbefore according to various embodiments) associated with the plurality of process runs.
[0098] For example, as described hereinbefore, FIGs. 13A to 13C show the process offsets for different state processes of the stage 1 of the SWaT testbed. FIG. 14 shows an example of linear model fitting for the process offsets. The obtained linear model may thus be used to calculate the process skew fingerprint. The process skew fingerprints obtained under normal operating conditions (which may be referred to as normal or reference process skew fingerprints) may then be used by an anomaly detector (e.g., the CUSUM detector) for detecting an anomaly in a physical process. The CUSUM parameters for all the stages in SWaT are shown in Table 4 in FIG. 18. In particular, Table 4 shows the design and performance of the CUSUM detector on the normal operation data. All the thresholds and other parameters are designed to have a desired false alarm rate of less than 5%. Table 4 shows bias parameter K, threshold t, mean m and standard deviation s for the process skew fingerprints. In the last two rows of Table 4 in FIG. 18, performance of the CUSUM detector under the normal operating conditions are shown using the design parameters specified. It can be observed that for all the cases the desired false alarm rate is below 5%.
[0099] Table 4 in FIG. 18 shows a high true negative rate, which indicates that it is possible to identify each state process (or process state) with a high accuracy based on the process skew fingerprint. A physical process goes through different state processes (or process states) during the operation of the process plant. For example, for the process of a fluid tank, either fluid is flowing out, flowing in, both or in a static state. Since different state processes have different process skew fingerprints according to various example embodiments of the present invention, it is possible to uniquely identify each state process (or process state) based on its associated process skew fingerprint.
[00100] In experiments conducted, a particular process, for example, a water filling process started at different initial states depending on the control logic. The results presented in Table 4 in FIG. 18 is a combination of all possible initial conditions of a particular process and a process skew fingerprint is created with respect to all the process runs for the process state. It can be seen from Table 4 in FIG. 18 that the process skew based fingerprint is stable over a range of process start and end conditions making it robust for application in a real-world system. [00101] The performance of the process skew based detection method 500 as an attack detection method is evaluated under a range of attack data collected from SWaT testbed. The SWaT testbed was subject to different attack scenarios for four days. In particular, over the four days, there were many process runs of normal operation and then there were attack instances in between. A complete list of attacks is shown in Table 1 in FIGs. 7A and 7B. An example of process offsets (or process skews) obtained for the process runs in relation to tank 4 in stage 4 is shown in FIG. 19. In particular, FIG. 19 depicts a plot of the process offsets obtained on the process runs in relation to tank 4 in stage 4, including a mixture of normal process runs and a few attacks. From FIG. 19, it can be observed that the process offsets obtained associated with normal process runs are close together and follow the normal profile of the process runs. On the other hand, the process offsets obtained associated with attacks (attack start and attack stop are denoted in FIG. 19) show clear deviations. Accordingly, from FIG. 19, it is evident that using process offsets according to various example embodiments of the present invention enable easy detection of attacks since the attack scenarios clearly deviate from the normal process offsets. In some cases, when the attack was stopped, the slope of the process offsets (corresponding to process skew fingerprint) tends to return to normal as expected but the whole data offsets have deviated for the overall process.
[00102] FIGs. 20A and 20B show the CUSUM detector for the same process. In particular, FIGs. 20A and 20B depict plots in relation to an attack detection example for LIT-401, outflow process. From FIGs. 20A and 20B, it can be observed that the process skew fingerprint according to various example embodiments of the present invention enables the detection of an attack easily and effectively. A detailed analysis was carried out for all the three stages and corresponding state processes in the SWaT testbed and the results are presented in Table 5 shown in FIG. 21. In particular, FIG. 21 shows the evaluation results of the process skew based detection method 500 according to various example embodiments of the present invention on the attack data from the SWaT testbed. The TPR presents the attacks which were detected accurately as percentage (attacks-detection/total-attacks-executed). From FIG. 21, it can be seen that all the attacks are detected in all the scenarios with 100% TPR. Furthermore, FPR is close to the desired 5% false alarm rate except for two instances. Accordingly, the process skew based detection method 500 has shown perfect performance on attack detection.
[00103] A stealthy attack is an attack designed to be hidden for a system model based attack detector. For a stealthy attack, the objective of the attacker is to modify the physical process measurements to achieve its objective and remain hidden. FIGs. 22A and 22B show the execution of such an attack in stage 1 of the SWaT testbed. In FIG. 22A, a plot of level sensor (LIT-101) actual measurements and sensor estimates obtained using the system model is shown. In FIG. 22B, respective residual (measured - estimated) values for the level sensor are shown. Upper and lower limits for a statistical detector are also shown. In FIG. 22A, the dotted line shows the ground truth for the state process, while the attacker is spoofing the sensor values and managed to derive the system away from the normal operation over time during the attack period. The spoofed values are chosen such that the residual values never grow bigger than a model-based detector threshold and hence, could not get detected. But from the ground truth, the process dynamics are not what the attacker is making the PLC believe. Accordingly, using the process offsets according to various example embodiments of the present invention, it is possible to detect the presence of such an attacker. In particular, if an attacker wants to deviate the process from its desired operation, it must defy the process dynamics and expose itself in the process offsets. Accordingly, it has been demonstrated that the process skew based detection method 500 is able to detect attacks that are stealthy for the system model based detectors. [00104] Although the process skew based detection method 500 according to various example embodiments of the present invention has been described with respect to a water treatment plant, it will appreciated by a person skilled in the art that the present invention is not limited to a water treatment plant. For example, physical processes have been discussed herein with respect to water/fluid dynamics but it will be appreciated that other similar processes are also applicable, such as but not limited to, gas or other chemical fluids where the process skew based detection method 500 can also be applied in the same or similar manner. Moreover, a range of different processes and process states have been considered herein which demonstrate the scalability of the process skew based detection method 500 according to various example embodiments of the present invention. Accordingly, demonstration on a real system highlights its applicability in real-world applications. [00105] Accordingly, various example embodiments demonstrate that indeed a process skew (process offset) exists for each process (e.g., process run) due to the deviations in the process itself from designed. As described hereinbefore, the process skew based detection method 500 may be used to fingerprint different state processes of a physical process, for example, filling, emptying or a combination of these process dynamics in a water treatment system. Therefore, it is possible to detect attacks on the state processes, as well as identify the particular state process/processes (or the particular process state(s)) of the physical process being attacked when the attack occurred. In this regard, the extensive evaluation of the process skew based detection method 500 on a real-world water treatment system validated its applicability and practicality, as well as effectiveness.
[00106] While embodiments of the invention have been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

CLAIMS What is claimed is:
1. A method of detecting anomaly in a physical process associated with a networked control system using at least one processor, the method comprising: obtaining, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; producing, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determining process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
2. The method according to claim 1, further comprising determining the modelled sensor data in relation to the physical state associated with the state process, comprising estimating sensor data in relation to the physical state associated with the state process based on a system model representing the state process and one or more design parameters associated with the state process.
3. The method according to claim 1 or 2, wherein said producing, for said each process run of the plurality of process runs of the state process at the process state, process offset data associated with the process run comprises: determining, for each time step of a plurality of time steps, a process offset between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain a plurality of process offset information associated with the process run at the plurality of time steps, respectively, wherein the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
4. The method according to claim 3, wherein said determining, for said each time step of the plurality of time steps, the process offset comprises determining, for said each time step of the plurality of time steps, a difference between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain the plurality of process offset information associated with the process run at the plurality of time steps, respectively.
5. The method according to any one of claims 1 to 4, wherein said determining the process skew fingerprint information associated with the plurality of process runs comprises determining a slope parameter associated with the plurality of process offset data associated with the plurality of process runs in relation to the physical state associated with the state process, wherein the process skew fingerprint information comprises the slope parameter.
6. The method according to claim 5, wherein said determining the slope parameter associated with the plurality of process offset data comprising applying a linear regression model to the plurality of process offset data to obtain a regression coefficient, wherein the slope parameter comprises the regression coefficient.
7. The method according to any one of claims 1 to 6, wherein said detecting anomaly in the physical process comprises detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process and a reference process skew fingerprint information associated with the state process at the process state of the physical process.
8. The method according to claim 7, further comprising determining the reference process skew fingerprint information associated with the state process at the process state of the physical process, comprising: obtaining, for each reference process run of a plurality of reference process runs for the state process at the process state of the physical process, measured sensor data associated with the reference process run in relation to the physical state associated with the state process at the process state; producing, for said each reference process run of the plurality of reference process runs, reference process offset data associated with the reference process run based on the measured sensor data associated with the reference process run and the modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of reference process offset data associated with the plurality of reference process runs, respectively, in relation to the physical state associated with the state process; and determining the reference process skew fingerprint information associated with the plurality of reference process runs based on the plurality of reference process offset data, the reference process skew fingerprint information comprising characteristic information associated with the plurality of reference process offset data.
9. The method according to any one of claims 1 to 8, wherein said detecting anomaly in the physical process comprises detecting an attack on the networked control system based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
10. A system for detecting anomaly in a physical process associated with a networked control system, the system comprising: a memory; and at least one processor communicatively coupled to the memory and configured to: obtain, for each process run of a plurality of process runs for a state process at a process state of the physical process, measured sensor data associated with the process run in relation to a physical state associated with the state process at the process state; produce, for said each process run of the plurality of process runs, process offset data associated with the process run based on the measured sensor data associated with the process run and modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of process offset data associated with the plurality of process runs, respectively, in relation to the physical state associated with the state process; determine process skew fingerprint information associated with the plurality of process runs based on the plurality of process offset data, the process skew fingerprint information comprising characteristic information associated with the plurality of process offset data; and detect anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
11. The system according to claim 10, wherein the at least one processor is further configured to determine the modelled sensor data in relation to the physical state associated with the state process, comprising estimating sensor data in relation to the physical state associated with the state process based on a system model representing the state process and one or more design parameters associated with the state process.
12. The system according to claim 10 or 11, wherein said produce, for said each process run of the plurality of process runs of the state process at the process state, process offset data associated with the process run comprises: determining, for each time step of a plurality of time steps, a process offset between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain a plurality of process offset information associated with the process run at the plurality of time steps, respectively, wherein the process offset data associated with the process run comprises the plurality of process offset information associated with the process run.
13. The system according to claim 12, wherein said determining, for said each time step of the plurality of time steps, the process offset comprises determining, for said each time step of the plurality of time steps, a difference between the measured sensor data associated with the process run obtained at the time step and the modelled sensor data at the time step in relation to the physical state associated with the state process, to obtain the plurality of process offset information associated with the process run at the plurality of time steps, respectively.
14. The system according to any one of claims 10 to 13, wherein said determine the process skew fingerprint information associated with the plurality of process runs comprises determining a slope parameter associated with the plurality of process offset data associated with the plurality of process runs in relation to the physical state associated with the state process, wherein the process skew fingerprint information comprises the slope parameter.
15. The system according to claim 14, wherein said determining the slope parameter associated with the plurality of process offset data comprising applying a linear regression model to the plurality of process offset data to obtain a regression coefficient, wherein the slope parameter comprises the regression coefficient.
16. The system according to any one of claims 10 to 15, wherein said detect anomaly in the physical process comprises detecting anomaly in the physical process based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process and a reference process skew fingerprint information associated with the state process at the process state of the physical process.
17. The system according to claim 16, wherein the at least one processor is further configured to determine the reference process skew fingerprint information associated with the state process at the process state of the physical process, comprising: obtaining, for each reference process run of a plurality of reference process runs for the state process at the process state of the physical process, measured sensor data associated with the reference process run in relation to the physical state associated with the state process at the process state; producing, for said each reference process run of the plurality of reference process runs, reference process offset data associated with the reference process run based on the measured sensor data associated with the reference process run and the modelled sensor data in relation to the physical state associated with the state process, to obtain a plurality of reference process offset data associated with the plurality of reference process runs, respectively, in relation to the physical state associated with the state process; and determining the reference process skew fingerprint information associated with the plurality of reference process runs based on the plurality of reference process offset data, the reference process skew fingerprint information comprising characteristic information associated with the plurality of reference process offset data.
18. The system according to any one of claims 10 to 17, wherein said detect anomaly in the physical process comprises detecting an attack on the networked control system based on the process skew fingerprint information associated with the plurality of process runs for the state process at the process state of the physical process.
19. A computer program product, embodied in one or more non-transitory computer- readable storage mediums, comprising instructions executable by at least one processor to perform the method of detecting anomaly in a physical process associated with a networked control system according to any one of claims 1 to 9.
PCT/SG2021/050340 2020-06-11 2021-06-11 Method and system for detecting anomaly in a physical process associated with a networked control system WO2021251906A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SG10202005543S 2020-06-11
SG10202005543S 2020-06-11

Publications (1)

Publication Number Publication Date
WO2021251906A1 true WO2021251906A1 (en) 2021-12-16

Family

ID=78845828

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2021/050340 WO2021251906A1 (en) 2020-06-11 2021-06-11 Method and system for detecting anomaly in a physical process associated with a networked control system

Country Status (1)

Country Link
WO (1) WO2021251906A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115118510A (en) * 2022-06-30 2022-09-27 东北大学 Hidden cheating attack method based on leaked resources and damaged resources

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180025558A1 (en) * 2016-07-19 2018-01-25 GM Global Technology Operations LLC Detection and reconstruction of sensor faults
WO2018055616A1 (en) * 2016-09-21 2018-03-29 Aperio Technology Pte. Ltd. Method and system for detecting attacks on monitored physical systems
CN108520187A (en) * 2018-04-20 2018-09-11 西安交通大学 Industrial control system physics Network Intrusion detection method based on the analysis of serial communication bus signal
US20190257716A1 (en) * 2014-09-17 2019-08-22 International Business Machines Corporation Detecting apparatus, detection method, and program

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190257716A1 (en) * 2014-09-17 2019-08-22 International Business Machines Corporation Detecting apparatus, detection method, and program
US20180025558A1 (en) * 2016-07-19 2018-01-25 GM Global Technology Operations LLC Detection and reconstruction of sensor faults
WO2018055616A1 (en) * 2016-09-21 2018-03-29 Aperio Technology Pte. Ltd. Method and system for detecting attacks on monitored physical systems
CN108520187A (en) * 2018-04-20 2018-09-11 西安交通大学 Industrial control system physics Network Intrusion detection method based on the analysis of serial communication bus signal

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KOHNO TADAYOSHI, ANDRE BROIDO, K.C. CLAFFY: "Remote physical device fingerprinting", SECURITY AND PRIVACY, 2005 IEEE SYMPOSIUM ON OAKLAND, CA, USA 08-11 MAY 2005, PISCATAWAY, NJ, USA,IEEE, vol. 2, 25 May 2005 (2005-05-25) - 11 May 2005 (2005-05-11), pages 1 - 28, XP055886301, ISBN: 978-0-7695-2339-2, DOI: 10.1109/SP.2005.18 *
KYONG-TAK CHO AND KANG G. SHIN: "Fingerprinting Electronic Control Units for Vehicle Intrusion Detection", USENIX, USENIX, THE ADVANCED COMPUTING SYSTEMS ASSOCIATION, 12 August 2016 (2016-08-12), pages 911 - 927, XP061025127 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115118510A (en) * 2022-06-30 2022-09-27 东北大学 Hidden cheating attack method based on leaked resources and damaged resources

Similar Documents

Publication Publication Date Title
Adepu et al. Using process invariants to detect cyber attacks on a water treatment system
Ahmed et al. Noise matters: Using sensor and process noise fingerprint to detect stealthy cyber attacks and authenticate sensors in cps
Adepu et al. Distributed attack detection in a water treatment plant: Method and case study
Kurt et al. Online cyber-attack detection in smart grid: A reinforcement learning approach
Adepu et al. Distributed detection of single-stage multipoint cyber attacks in a water treatment plant
Kalech Cyber-attack detection in SCADA systems using temporal pattern recognition techniques
Adepu et al. Generalized attacker and attack models for cyber physical systems
TWI734765B (en) Method of detecting cyber attacks on a cyber physical system which includes at least one computing device coupled to at least one sensor and/or actuator for controlling a physical process
WO2020246944A1 (en) Method and system for attack detection in a sensor network of a networked control system
Adepu et al. From design to invariants: Detecting attacks on cyber physical systems
Ahmed et al. Challenges and opportunities in cyberphysical systems security: A physics-based perspective
Ahmed et al. Process skew: Fingerprinting the process for anomaly detection in industrial control systems
Umsonst et al. Security analysis of control system anomaly detectors
Palleti et al. Cascading effects of cyber-attacks on interconnected critical infrastructure
CN111698257B (en) Industrial information physical system security detection method for multi-class malicious attacks
WO2021251906A1 (en) Method and system for detecting anomaly in a physical process associated with a networked control system
WO2022015246A1 (en) Method and system for characterising a programmable logic controller (plc) and/or attack detection in a networked control system
Ahmed et al. A practical physical watermarking approach to detect replay attacks in a CPS
Umer et al. Attack rules: an adversarial approach to generate attacks for Industrial Control Systems using machine learning
Mujeeb Ahmed et al. Machine learning for cps security: applications, challenges and recommendations
Zugasti et al. Null is not always empty: Monitoring the null space for field-level anomaly detection in industrial IoT environments
Harirchi et al. Model (in) validation and fault detection for systems with polynomial state-space models
Ghaeini et al. Zero residual attacks on industrial control systems and stateful countermeasures
Du et al. Active fault isolation of nonlinear process systems
Adepu et al. Introducing cyber security at the design stage of public infrastructures: A procedure and case study

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21821793

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17/03/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21821793

Country of ref document: EP

Kind code of ref document: A1