CN109189599B - Method and device for improving real-time data dump stability of SCADA system - Google Patents

Method and device for improving real-time data dump stability of SCADA system Download PDF

Info

Publication number
CN109189599B
CN109189599B CN201810898838.5A CN201810898838A CN109189599B CN 109189599 B CN109189599 B CN 109189599B CN 201810898838 A CN201810898838 A CN 201810898838A CN 109189599 B CN109189599 B CN 109189599B
Authority
CN
China
Prior art keywords
task
data
dumping
dump
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810898838.5A
Other languages
Chinese (zh)
Other versions
CN109189599A (en
Inventor
杭小林
王学博
高松领
***
屈利康
缪博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xinzhi Energy System Control Co ltd
Original Assignee
Xinzhi Energy System Control Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xinzhi Energy System Control Co ltd filed Critical Xinzhi Energy System Control Co ltd
Priority to CN201810898838.5A priority Critical patent/CN109189599B/en
Publication of CN109189599A publication Critical patent/CN109189599A/en
Application granted granted Critical
Publication of CN109189599B publication Critical patent/CN109189599B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0706Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
    • G06F11/0736Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in functional embedded systems, i.e. in a data processing system designed as a combination of hardware and software dedicated to performing a certain function
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766Error or fault reporting or storing
    • G06F11/0778Dumping, i.e. gathering error/state information after a fault for later diagnosis

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the invention discloses a method and a device for improving the real-time data dump stability of an SCADA system, wherein the method comprises the following steps: establishing a SCADA system real-time data dump task queue, wherein the queue comprises a first dump task T1 and a second dump task T2; respectively collecting real-time data through a first dumping task T1 and a second dumping task T2 and storing the real-time data into corresponding buffer data files; applying for acquiring data dump right and executing normal data dump by a first dump task T1; when the first dumping task T1 fails during the data dumping process, according to the type of the failure, the second dumping task T2 continuously or temporarily resumes the transmission at the breakpoint according to the real-time data stored in the second buffered data file, thereby achieving automatic failure recovery. The scheme of the embodiment solves the problem of data interruption caused by faults in the real-time data dump process of the SCADA system, and improves the stability of the real-time data dump of the SCADA system.

Description

Method and device for improving real-time data dump stability of SCADA system
Technical Field
The embodiment of the invention relates to a data dump technology, in particular to a method and a device for improving the stability of real-time data dump of an SCADA (supervisory control and data acquisition) system.
Background
In the field of industrial automation Control, SCADA (Supervisory Control And Data Acquisition) is commonly used for acquiring Data And monitoring field operation equipment, various instruments And communication equipment are installed on the field, a SCADA system acquires Data through a communication protocol, And the SCADA system has a Data storage function, so that historical Data trend of a Data item to be monitored can be checked And analyzed through a SCADA system software module, And the upper Data analysis module needs to calculate And analyze bottom Data to obtain various summarized Data such as metering Data And operation indexes, And needs to dump basic Data to a relational database in real time through the SCADA system, And the upper application software accesses the Data, calculates And then performs business analysis.
The SCADA system real-time data dump operating environment generally comprises: the SCADA server runs an SCADA software system and a real-time data dump program, the database server runs a relational database, the application server runs upper-layer industry application software and the like.
The SCADA software system collects real-time data from field equipment, and a real-time data dump program accesses a real-time database through an interface or a conventional communication protocol to obtain the data and provides the data for a relational database accessible by application software in the upper industry.
At present, the commonly used data dump measures of the SCADA system include:
(1) ADO data processing script
And embedding an ADO data processing script in the SCADA timing script, and directly writing the real-time SCADA variable data into a historical library.
ADO (ActiveX Data objects): a program object for representing data structures and contained data in a user database, using ADO objects to create or modify tables and queries, check the database, or access external data sources.
(2) ODBC data dump component
The ODBC data dump component acquires data by accessing an internal interface through an SDK of an SCADA manufacturer and dumping real-time data into a relational database in a certain format at regular time through an ODBC mode.
ODBC (Open Database Connectivity Open Database interconnection) is a component of microsoft corporation Open Services Architecture (WOSA), which builds a set of specifications and provides a set of standard APIs (application programming interfaces) for Database access.
SDK (Software Development Kit Software Development tool) is a collection of Development tools used by Software engineers to create application Software for a specific Software package, Software framework, hardware platform, operating system, etc., and the patent uses an application program interface API provided for SCADA Software vendors to access SCADA real-time databases through the interface.
(3) OPC data acquisition and storage software
And deploying OPC data acquisition and storage software, acquiring SCADA real-time data by the software in an OPC mode, acquiring the real-time data by the software at regular time and writing the real-time data into a relational database in an ODBC mode.
OPC (object connection and embedding for Process Control) is a specification designed to solve a communication standard between a field management layer and a Process Control layer based on Microsoft COM/DCOM (Distributed component object model) technology and DNA (Distributed internet Application) framework, and provides a standard mechanism for data communication between a Process management Application and a field automation Control system and devices.
However, the current SCADA system data dump has the following disadvantages:
(1) ADO data processing script
Because the ADO scripts are complex and inconvenient to maintain, and the running process can affect the software running efficiency of the SCADA system.
(2) Data loss often occurs during the process of using ODBC data dump component or OPC data acquisition and storage software:
when the ODBC component dumps abnormally, the data is temporarily interrupted;
the database access is abnormal, and the data is temporarily interrupted;
maintaining or migrating the database, and interrupting data; the stored data of the OPC acquisition and storage software is abnormal (database writing failure, network abnormality and the like), and the data can be temporarily interrupted;
because the OPC data acquisition and storage software needs to be reconfigured or updated and maintained during database migration and SCADA system upgrading, data items are reconfigured and the software is restarted, so that data loss is caused. The SCADA system upgrading can comprise SCADA system data acquisition project adjustment upgrading, OPC data acquisition and storage software configuration project upgrading and software upgrading.
In order to solve the above problems, the conventional solutions are as follows:
the traditional solution is to restart and activate the software function by monitoring whether the process is abnormal or not, detecting whether the data stops dumping or not, and manually maintaining and completing the data (inquiring the data record of the SCADA historical database and manually completing the data). However, the following problems still exist with these current solutions:
(1) data are interrupted for a long time, and the workload of filling up is large;
(2) partial data are incomplete, so that the result of timing data calculation is deviated;
(3) the data maintenance cost is high;
(4) the upper layer data calculation and analysis processing application functions are influenced;
(5) increasing the work complexity of data migration.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide a method and an apparatus for improving stability of real-time data dump of a SCADA system by data acquisition and monitoring control, which can prevent data interruption after a fault occurs in a data dump process, avoid behaviors that a fault caused by a program fault, an upgrade, a database migration, and the like cannot automatically recover and complement historical data, and ensure sequential write-in of data dump, thereby improving stability of real-time data dump of the SCADA system.
In one embodiment of the present invention, there is provided:
(1) a method for improving the stability of real-time data dump of a SCADA system can comprise the following steps:
establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database;
when the first dumping task T1 fails during the data dumping process, according to the type of the failure, the second dumping task T2 continuously or temporarily resumes the transmission at the breakpoint according to the real-time data stored in the second buffered data file, thereby achieving automatic failure recovery.
In various other embodiments of the invention:
(2) the method for improving the stability of the real-time data dump of the SCADA system according to (1), wherein the establishing the real-time data dump task queue of the SCADA system may include:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
(3) The method for improving the stability of real-time data dump of the SCADA system according to (2), wherein the acquisition and dump configuration information may include one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
The method may further comprise: after a SCADA system real-time data dump task queue is established, setting the main-standby relationship between the first dump task T1 and the second dump task T2;
the first dump task T1 is a main task, and the second dump task T2 is a standby task.
(4) The method for improving the stability of the real-time data dump of the SCADA system according to (1), wherein the starting the first dump task T1 and the second dump task T2 may include:
respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
the operation information is accessed and recorded to the preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
(5) The method for improving the stability of the real-time data dump of the SCADA system according to (4), wherein the task synchronization table Y1 may include: task coding, task state, state change time and data dump time point;
wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task;
the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5.
The applying for acquiring a data dump right by the first dumping task T1 and executing a normal data dump to write the real-time data into a preset relational database may include:
setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
after the data dump right is acquired, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
(6) The method for improving the stability of the real-time data dump of the SCADA system according to (5), wherein the fault type includes any one or more of the following types: the failure of the dump program itself, the failure caused by program upgrade, and the failure caused by database migration.
(7) The method for improving the stability of real-time data dump of the SCADA system according to (6), wherein when the failure type is a failure of the dump program itself, the performing continuous or temporary breakpoint transmission by the second dump task T2 according to the failure type and according to the real-time data stored in the second buffered data file includes:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right; acquiring a first data breakpoint B1 of the first dumping task T1 from a data dumping time point and a state change time of the task synchronization table Y1; extracting a continuous data queue comprising the first data breakpoint B1 from the second buffered data file and performing a resume process for breakpoint resume;
when the second dump task T2 does not have a fault, continuing to perform breakpoint resuming by the second dump task T2; or, when the fault of the first dumping task T1 is eliminated, the first dumping task T1 reappears the data dumping right, and obtains a second data breakpoint B2 of the second dumping task T2, and performs breakpoint resuming according to the real-time data stored in the first buffered data file.
(8) The method for improving stability of real-time data dump of SCADA system according to (6), wherein when the failure type is a failure caused by program upgrade, the performing continuous or temporary breakpoint transmission by the second dump task T2 according to the failure type and according to the real-time data stored in the second buffered data file includes:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right; acquiring a third data breakpoint B3 of the first dumping task T1 from the data dumping time point and the state change time of the task synchronization table Y1; extracting a continuous data queue comprising the third data breakpoint B3 from the second buffered data file and performing a resume process for breakpoint resume;
when the first dumping task T1 is not upgraded, continuing to perform breakpoint resuming by the second dumping task T2; when the first dumping task T1 is upgraded, the first dumping task T1 reappears the data dumping right, a fourth data breakpoint B4 of the second dumping task T2 is obtained, and breakpoint continuous transmission is performed according to real-time data stored in the first buffer data file.
(9) The method for improving stability of real-time data dump of SCADA system according to (6), wherein when the failure type is a failure caused by database migration, the performing continuous or temporary breakpoint transmission by the second dump task T2 according to the failure type according to the real-time data stored in the second buffered data file includes:
when the relational database is migrated, the first dumping task T1 updates corresponding first configuration information, so that when a data dumping process fails, the second dumping task T2 uninterruptedly stores the collected real-time data into the first buffered data file according to data breakpoints generated by the migration and the first dumping task T1, so as to implement breakpoint transmission and automatic failure recovery;
when the first dumping task T1 completes updating the first configuration information and the second dumping task T2 updates the corresponding second configuration information due to the migration, the first dumping task T1 stores the collected real-time data into the first buffered data file according to a data breakpoint generated by the second dumping task T2, and writes the data stored in the first buffered data file into the migrated relational database, so as to implement breakpoint resume.
In another embodiment of the present invention, there is provided:
(10) an apparatus for improving stability of data dump of a data acquisition and supervisory control SCADA system, comprising: the system comprises a processor and a computer readable storage medium, wherein instructions are stored in the computer readable storage medium, and when the instructions are executed by the processor, the method for improving the stability of the real-time data dump of the SCADA system is realized.
The embodiment of the invention comprises the following steps: establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2; starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file; applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table; when the first dumping task T1 fails during the data dumping process, according to the type of the failure, the second dumping task T2 continuously or temporarily resumes the transmission at the breakpoint according to the real-time data stored in the second buffered data file, thereby achieving automatic failure recovery. By the scheme of the embodiment, the problem of data interruption caused by faults in the real-time data dump process of the SCADA system is solved, the data interruption after the faults occur in the data dump process is prevented, the actions that the faults caused by program faults, upgrading, database migration and the like cannot be automatically recovered and the historical data is filled are avoided, the sequential write-in of data dump is ensured, and the stability of the real-time data dump of the SCADA system is improved.
Additional features and advantages of embodiments of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the examples of the application do not constitute a limitation of the embodiments of the invention.
FIG. 1 is a flow chart of a method for improving stability of real-time data dump of a SCADA system for data acquisition and monitoring control according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a SCADA system real-time data dump method according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for initiating a first dump task T1 and a second dump task T2 according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for applying for acquiring data dump right and executing normal data dump by the first dump task T1 according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method for continuous or temporary breakpoint resuming by the second dump task T2 according to the fault type when the fault type is a fault of the dump program itself according to the embodiment of the present invention;
FIG. 6 is a flowchart of a method for performing breakpoint resuming by the first dumping task T1 when the fault type is a fault of the dumping program itself and the fault of the first dumping task T1 is cleared according to the embodiment of the present invention;
FIG. 7 is a flowchart of a method for continuous or temporary breakpoint resuming by the second dump task T2 according to the fault type when the fault type is a fault caused by program upgrade according to an embodiment of the present invention;
FIG. 8 is a flowchart of a method for continuous or temporary breakpoint resuming by the second dump task T2 according to the type of failure when the type of failure is a failure caused by database migration according to an embodiment of the present invention;
FIG. 9 is a flowchart of a method performed by the second dumping task T2 before real-time data is stored into the first buffered data file according to the database migration and the data breakpoint generated by the first dumping task T1;
FIG. 10 is a flowchart of a method performed before the first dumping task T1 stores the collected real-time data into the first buffered data file according to the data break generated by the second dumping task T2 and writes the data stored in the first buffered data file into the migrated relational database data table according to an embodiment of the present invention;
fig. 11 is a block diagram of an apparatus for improving stability of real-time data dump of SCADA system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Example one
A method for improving stability of data dump in real time of a SCADA system for data acquisition and monitoring control, as shown in FIG. 1 and FIG. 2, may include S101-S104:
s101, establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
s102, starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
s103, applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database;
s104, when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 continuously or temporarily continues to transmit at a breakpoint according to the real-time data stored in the second buffer data file according to the failure type, so as to realize automatic failure recovery.
In the embodiment of the invention, in order to solve the problems provided in the background art, a task queue of real-time data dump of the SCADA system is established, a plurality of data acquisition and dump programs are operated in parallel, a software synchronous operation mechanism is established, an online operation backup function is provided, when data acquisition or dump is abnormal, fault recording and redundancy processing are realized, and the stability of real-time data dump of the SCADA system can be greatly improved.
Example two
The embodiment provides a specific implementation way for establishing a real-time data dump task queue of the SCADA system on the basis of the first embodiment.
In this embodiment of the present invention, the establishing a real-time data dump task queue of the SCADA system may include:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
In this embodiment of the present invention, a dump task configuration table C2 may be provided, where the two dump tasks in the dump task configuration table C2 are configured as a first dump task T1 and a second dump task T2, respectively. The task numbers of the first dumping task T1 and the second dumping task T2 are different, and the collection and dumping configuration information is the same.
In an embodiment of the present invention, the collecting and dumping configuration information may include one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
In the embodiment of the present invention, the dump task configuration table C2 may include one or more of task number, task name, task type, data collection and dump program path, data collection and dump program name, collection task number, dump destination data table name, and other information items.
In the embodiment of the present invention, the method may further include: after a SCADA system real-time data dump task queue is established, setting the main-standby relationship between the first dump task T1 and the second dump task T2; wherein, the first dump task T1 is a main task (M1), and the second dump task T2 is a standby task (M2).
In the embodiment of the present invention, in the dump task configuration table C2, the task type of the first dump task T1 may be set as the main task (M1), and the task type of the second dump task T2 may be set as the standby task (M2).
In the embodiment of the present invention, before the SCADA system real-time data dump task queue is established, the collection data configuration table C1 may be preset, so as to execute uninterrupted real-time data collection tasks according to the collection data configuration table C1.
In the embodiment of the present invention, the collected data configuration table C1 may include information of data items to be acquired and dumped from the SCADA system, and specifically may include any one or more of the following information: collecting task numbers, collected input/output IO point names, data item names, collection frequency, value taking modes and the like.
EXAMPLE III
This embodiment is based on the first embodiment or the second embodiment, and a specific implementation manner of starting the first dumping task T1 and the second dumping task T2 is given.
In the embodiment of the present invention, as shown in fig. 3, the starting the first dumping task T1 and the second dumping task T2 may include S201 to S202:
s201, respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
s202, accessing and recording the running information to a preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
In the embodiment of the present invention, a first data collecting and dumping program a1 corresponding to the first dumping task T1 and a second data collecting and dumping program a2 corresponding to the second dumping task T2 may be written in advance; when the first dumping task T1 and the second dumping task T2 are restarted, the first data collecting and dumping program a1 and the second data collecting and dumping program a2 may be started, respectively, the program a1 and the program a2 access and record the running information to the task synchronization table Y1, and record the task state in the task synchronization table Y1 as the X1 (running) state.
In this embodiment of the present invention, the task synchronization table Y1 may include: task coding, task state, state change time and data dump time point;
wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task;
the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5.
In the embodiment of the present invention, task coding (or task type coding) describes: m1 (main): a main task; m2 (stock): and (5) preparing tasks. That is, when the corresponding task code column is set to M1, the corresponding dump task is the main task; when the corresponding task code column is set to M2, the corresponding dump task is referred to as a standby task.
In the embodiment of the present invention, the task code may include, but is not limited to, the letters M1 and M2, and any number, letter, character, etc. other than M1 and M2 may be used.
In the embodiment of the invention, the task state description comprises the following steps:
x1 (run): after the software is started, the task state item of the task synchronization table Y1 is written into the state;
x2 (fault): the software dump is abnormal, and the task state item in the task synchronization table Y1 is written into the state;
x3 (application): applying for executing a data dump task, and when the task state is not dumped (a non-continuous transfer state and a non-working state) or the update time is invalid (such as a record with the update time less than 60 seconds), updating the task state into an application state;
x4 (continuous transmission): executing breakpoint continuous transmission work;
x5 (working): and executing real-time dump work.
In the embodiment of the present invention, the representation manner of the task state may include, but is not limited to, letters X1-X5, and any number, letter, character, etc. other than X1-X5 may be used.
In the embodiment of the invention, the implementation mode of the data collection and dump task program can select an OPC mode or an interface calling mode.
Example four
On the basis of any of the above embodiments, this embodiment provides a specific implementation manner that the real-time data are collected continuously and stored in the corresponding first buffered data file and the corresponding second buffered data file by the first dumping task T1 and the second dumping task T2, respectively.
In the embodiment of the invention, real-time data can be acquired from the SCADA system according to the preset information of the acquisition data configuration table C1, and after the data acquisition and dump program is started, the program can execute uninterrupted real-time data acquisition tasks.
In an embodiment of the present invention, the uninterrupted collection of real-time data by the first dumping task T1 and the second dumping task T2, and the storing of the corresponding first buffered data file and the second buffered data file may specifically include: continuously executing a collecting process K1_1 and a writing buffer data process K3_1 of the first transstorage task T1; the collection process K1_2 and the write buffer data process K3_2 of the second dump task T2 are executed without interruption.
In an embodiment of the present invention, the uninterrupted execution of the collection process K1_1 and the write buffer data process K3_1 of the first dumping task T1 may include: a first data acquisition and dump program A1 of a first dumping task T1 executes an acquisition process K1_1 according to an acquisition data configuration table C1 list, and acquires real-time data from an SCADA real-time base; the K3_1 process is performed to write the real-time data to the first buffered data file F1.
In an embodiment of the present invention, the uninterrupted execution of the collection process K1_2 and the write buffered data process K3_2 of the second dump task T2 may include: a second data acquisition and dump program A2 of a second dump task T2 executes an acquisition process K1_2 according to an acquisition data configuration table C1 list, and acquires real-time data from an SCADA real-time base; the K3_2 process is performed to write the real-time data to the second buffered data file F2.
EXAMPLE five
The embodiment provides a specific implementation manner of the first dumping task T1 applying for acquiring the data dumping right on the basis of any of the above embodiments.
In the embodiment of the present invention, as shown in fig. 4, applying for obtaining a data dump right by the first dumping task T1, and performing a normal data dump to write the real-time data into a preset relational database may include S301 to S302:
s301, setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
s302, after the data dump right is obtained, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
In the embodiment of the present invention, the data dump right application process K2_1 may be executed by the first data collection and dump program a1, so that the first dump task T1, which is marked as a main task, applies for acquiring data dump right. Specifically, the corresponding task state of the first dumping task T1 in the task synchronization table Y1 may be set to the X3 (application) state by the first data collection and dumping program a1 to apply for data dumping rights.
In the embodiment of the present invention, the data dump right application process K2_2 may also be executed first by the second data acquisition and dump program a2, so that the second dump task T2, which is marked as a secondary task, applies for acquiring the data dump right. The specific definition may be self-defined according to an actual application scenario, and is not particularly limited herein.
In this embodiment of the present invention, after acquiring the data dump right, the first dumping task T1 executes a normal data dump, and writes the real-time data into the relational database data table, which may specifically include: the first data collecting and dumping program a1 of the first dumping task T1 executes a normal data dumping task process K5_1, writes the collected real-time data into a data table W1, sets the task state of the first dumping task T1 in a task synchronization table Y1 to an X5 (working) state, and updates the data dumping time point and the state change time. The first dumping task T1 continues to execute the dumping task process K5_1 without interruption.
In the embodiment of the present invention, the data table W1 may include, but is not limited to, any one or more of the following items: IO (input/output) point code, data item name, value, acquisition time and the like
EXAMPLE six
This embodiment provides several possible fault type embodiments in the data dump process on the basis of any of the above embodiments.
In the embodiment of the present invention, the failure in the data dump process may include, but is not limited to, any one or more of the following: the failure of the dump program itself, the failure caused by program upgrade, and the failure caused by database migration.
In the embodiment of the present invention, each fault type will be described in detail below:
(1) reasons for dump task failure scenarios (failure of the dump program itself) may include: program exception, data dump write exception. In this scenario, the failed dump task cannot continue to perform data dump, and then the data dump can be continued by performing breakpoint resuming through another normal dump task.
(2) Data collection and dump program upgrade scenarios (failures due to program upgrades) — reasons may include: the first data collection and dump procedure A1 is closed before being upgraded;
(3) database migration scenario (failure due to database migration) — reasons may include: shutdown before database migration.
EXAMPLE seven
Based on the sixth embodiment, the embodiment provides a specific implementation manner of performing breakpoint resuming based on the first failure type (failure of the dump program itself).
In this embodiment of the present invention, as shown in fig. 5, when the failure type is a failure of the dump program itself, the performing, by the second dump task T2, continuous or temporary breakpoint resuming according to the real-time data stored in the second buffered data file according to the failure type may include S401-S402:
s401, the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right; acquiring a first data breakpoint B1 of the first dumping task T1 from a data dumping time point and a state change time of the task synchronization table Y1; extracting a continuous data queue comprising the first data breakpoint B1 from the second buffered data file and performing a resume process for breakpoint resume;
s402, when the second dump task T2 does not have a fault, continuously performing breakpoint transmission by the second dump task T2; or, when the fault of the first dumping task T1 is eliminated, the first dumping task T1 reappears the data dumping right, and obtains a second data breakpoint B2 of the second dumping task T2, and performs breakpoint resuming according to the real-time data stored in the first buffered data file.
In the embodiment of the present invention, after the first data collecting and dumping program a1 of the first dumping task T1 performs the data dumping task process K5_1 and has a fault, the fault recording process K4_1 of the first data collecting and dumping program a1 may record the time point of the fault data into the task fault table Y2, the recovery state entry in the task fault table Y2 is marked as not recovered, the task state corresponding to the first dumping task T1 in the task synchronization table Y1 is set to be the X2 (fault) state, and the state change time is updated.
In the embodiment of the present invention, the failure of the first dumping task T1 and the failure recovery process implemented by the second dumping task T2 executing breakpoint resuming may specifically include: 1. the second dumping task T2 judges the failure of the first dumping task T1 and applies for acquiring the data dumping right; that is, the second dump task T2 accesses the task synchronization table Y1 to determine that the first dump task T1 has a fault, the second dump task T2 applies for and acquires a data dump right, and a task state corresponding to the second dump task T2 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring a data breakpoint of the first dump task T1 from the task state table Y1, and marking the data breakpoint as a first data breakpoint B1; 3. breakpoint resume is carried out on the first data breakpoint B1, and fault recovery is achieved; that is, the second data collecting and dumping program a2 executes the process of extracting buffered data K3_2, extracting buffered data from the second buffered data file F2, obtaining a continuous data queue including the first data breakpoint B1, and executing the resume process K5_2 (the resume process K5_2 is also a normal data dumping task process of the second dumping task T2); and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
In the embodiment of the present invention, during the process of executing data dump task K5_2 continuously by the second data collection and dump program a2, the collected real-time data is written into the data table W1, and the task state of the second dump task T2 in the task synchronization table Y1 is set to X5 (working).
In the embodiment of the present invention, if during the execution of the normal data dump task process K5_2 by the second dump task T2, the fault of the first dump task T1 is cleared, and the normal data dump task can be continuously executed, the first dump task T1 may reapply to acquire the data dump right; to continue writing real-time data into the data table W1 by the first dump task T1.
In the embodiment of the present invention, the recovery status in the task fault table Y2 may be represented by different status flags, which may include, but are not limited to, values 01 and 02, where 01 may represent not recovered and 02 may represent recovered; in other embodiments, any number, letter, character, etc. other than 01 and 02 may be used.
Example eight
This embodiment is based on the seventh embodiment, and a specific implementation of breakpoint resuming by the first dumping task T1 after the second dumping task T2 fails is given.
In the embodiment of the present invention, as shown in fig. 6, when the failure of the first destaging task T1 is cleared, the performing, by the first destaging task T1, breakpoint resuming according to the real-time data stored in the first buffered data file may include S501-S504:
s501, the first dumping task T1 judges the fault of the second dumping task T2 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right;
s502, acquiring a second data breakpoint B2 of the second dump task T2 from the data dump time point and the state change time of the task synchronization table Y1;
s503, extracting a continuous data queue comprising the second data breakpoint B2 from the first buffer data file and executing a continuous transmission process to perform breakpoint continuous transmission;
s504, after the breakpoint resume is completed, the preset recovery state of the task fault table Y2 is changed to be recovered, and the data dump time point and the state change time of the task synchronization table Y1 are updated.
In the embodiment of the present invention, after the second data collecting and dumping program a2 of the second dumping task T2 executes the normal data dumping task process K5_2 and has a fault, the fault recording process K4_2 of the second data collecting and dumping program a2 may record the time point of the fault data into the task fault table Y2, mark the recovery state entry in the task fault table Y2 as not recovered, set the task state corresponding to the second dumping task T2 in the task synchronization table Y1 as the X2 (fault) state, and update the state change time.
In the embodiment of the present invention, when the second dumping task T2 fails, and the first dumping task T1 implements a failure recovery process, specifically, the process may include: 1. the first dumping task T1 judges the fault of the second dumping task T2 and applies for acquiring the data dumping right; that is, the first dumping task T1 accesses the task synchronization table Y1 to determine that the second dumping task T2 has a fault, the first dumping task T1 applies for and acquires a data dumping right, and a task state corresponding to the first dumping task T1 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring a data breakpoint of a second dump task T2 from the task state table Y1, and marking the data breakpoint as a second data breakpoint B2; 3. breakpoint resume is carried out on the second data breakpoint B2, and fault recovery is achieved; that is, the first data collecting and dumping program a1 executes the process of extracting buffered data K3_1, extracting buffered data from the first buffered data file F1, obtaining a continuous data queue including the second data breakpoint B2, and executing the resume process K5_1 (the resume process K5_1 is also a normal data dumping task process of the first resume task T1); and after the breakpoint resume is completed, changing the recovery state of the task fault table Y2 into recovered state, and updating the data dump time point and the state change time.
In the embodiment of the present invention, during the first data collecting and dumping program a1 continuously executing the data dumping task process K5_1, the collected real-time data is written into the data table W1, and the task state of the first dumping task T1 in the task synchronization table Y1 is set to X5 (working).
Example nine
The embodiment provides a specific implementation manner of performing breakpoint resuming based on the second failure type (failure caused by program upgrade) on the basis of the sixth embodiment.
In this embodiment of the present invention, as shown in fig. 7, when the failure type is a failure caused by program upgrade, the performing, by the second dumping task T2, continuous or temporary breakpoint resuming according to the real-time data stored in the second buffered data file according to the failure type may include S601-S602:
s601, the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right; acquiring a third data breakpoint B3 of the first dumping task T1 from the data dumping time point and the state change time of the task synchronization table Y1; extracting a continuous data queue comprising the third data breakpoint B3 from the second buffered data file and performing a resume process for breakpoint resume;
s602, when the first dumping task T1 is not upgraded, the second dumping task T2 continues to perform breakpoint resuming; when the first dumping task T1 is upgraded, the first dumping task T1 reappears the data dumping right, a fourth data breakpoint B4 of the second dumping task T2 is obtained, and breakpoint continuous transmission is performed according to real-time data stored in the first buffer data file.
In the embodiment of the present invention, the first data collecting and dumping program a1 of the first dumping task T1 may have a failure of the first dumping task T1 due to the upgrade, before the first data collecting and dumping program a1 is upgraded, the first data collecting and dumping program a1 is turned off, and a failure of the first dumping task T1 occurs, so as to form a third data breakpoint B3.
In this embodiment of the present invention, the performing breakpoint resuming by the second dump task T2 to implement the failure recovery process may specifically include:
1. the second dumping task T2 judges the failure of the first dumping task T1 and applies for acquiring the data dumping right; that is, the second dump task T2 accesses the task synchronization table Y1 to determine that the first dump task T1 has a fault, the second dump task T2 applies for and acquires a data dump right, and a task state corresponding to the second dump task T2 in the task synchronization table Y1 is assigned as an X3 (application) state;
2. acquiring a data breakpoint of the first dump task T1 from the task state table Y1, and marking the data breakpoint as a third data breakpoint B3;
3. breakpoint resume is carried out on the third data breakpoint B3, and fault recovery is achieved; that is, the second data collecting and dumping program a2 executes the process of extracting buffered data K3_2, extracts buffered data from the second buffered data file F2, obtains a continuous data queue including the third data breakpoint B3, and executes the resume process K5_2 (the resume process K5_2 is also a normal data dumping task process of the second dumping task T2); and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
In the embodiment of the present invention, in the process of performing breakpoint resuming on the second dump task T2, after the first data collection and dump program a1 of the first dump task T1 is upgraded, the first data collection and dump program a1 may be resumed, that is, the first dump task T1 reappears the data dumping right, and obtains the fourth data breakpoint B4 of the second dump task T2, and performs a data dump process K5_1, and performs breakpoint resuming on the basis of the fourth data breakpoint B4 according to the real-time data stored in the first buffered data file; the collected real-time data is written into a data table W1, the task state of a first dumping task T1 in a task synchronization table Y1 is set to be an X5 (working) state, and the first dumping task T1 executes an uninterrupted dumping process K5_ 1.
In the embodiment of the present invention, similarly, the second data collecting and dumping program a2 of the second dumping task T2 may also have a failure of the second dumping task T2 due to the upgrade, before the second data collecting and dumping program a2 is upgraded, the second data collecting and dumping program a2 is closed, and the failure of the second dumping task T2 occurs, so as to form a data breakpoint. At this time, the first dumping task T1 which is recovered to be normal may reapply the data dumping right, acquire a data breakpoint of the second dumping task T2, execute a data dumping process K5_1, and perform breakpoint continuous transmission based on the data breakpoint according to the real-time data stored in the first buffered data file; the collected real-time data is written into a data table W1, the task state of a first dumping task T1 in a task synchronization table Y1 is set to be an X5 (working) state, and the first dumping task T1 executes an uninterrupted dumping process K5_ 1.
Example ten
The embodiment provides a specific implementation manner of performing breakpoint resuming based on the third failure type (failure caused by database migration) on the basis of the sixth embodiment.
In this embodiment of the present invention, as shown in fig. 8, when the failure type is a failure caused by database migration, the performing, by the second dumping task T2, continuous or temporary breakpoint resuming according to the real-time data stored in the second buffered data file according to the failure type may include S701-S702:
s701, when the relational database is migrated, the first dumping task T1 updates corresponding first configuration information, so that when a data dumping process fails, the second dumping task T2 uninterruptedly stores the collected real-time data into the first buffer data file according to the migration and data breakpoints generated by the first dumping task T1, so as to realize breakpoint transmission and automatic failure recovery;
s702, when the first dumping task T1 completes updating the first configuration information and the second dumping task T2 updates the corresponding second configuration information due to the migration, the first dumping task T1 stores the collected real-time data into the first buffer data file according to a data breakpoint generated by the second dumping task T2, and writes the data stored in the first buffer data file into the migrated relational database, so as to realize breakpoint transmission.
In the embodiment of the present invention, as shown in fig. 9, before the data breakpoint generated by the second dumping task T2 according to the migration and the first dumping task T1 uninterruptedly stores the collected real-time data into the first buffered data file, the method may further include S801-S802:
s801, judging the fault of the first dumping task T1 by the second dumping task T2 through the task state of the task synchronization table Y1, and applying for acquiring a data dumping right;
s802, acquiring a first data breakpoint B1 and a second data breakpoint B2 from a data dump time point and a state change time of the task synchronization table Y1; the first data breakpoint B1 is a data breakpoint formed by closing a connection between the relational databases before migration of the relational databases, and the second data breakpoint B2 is a data breakpoint formed by the first data collection and dump program a1 when updating the first configuration information after migration of the relational databases is completed.
In an embodiment of the present invention, the uninterrupted storing of the collected real-time data into the first buffered data file by the second dumping task T2 according to the data breakpoints generated by the migration and the first dumping task T1 may include:
and acquiring a continuous real-time data queue between the first data breakpoint B1 and the second data breakpoint B2, and storing the continuous real-time data queue into the first buffer data file.
In the embodiment of the present invention, during the execution of the data dump task, there may be a phenomenon of database migration, and before the database migration, the database connection is usually closed, so as to form a data breakpoint, i.e. the first data breakpoint B1 described above. After the migration of the database is completed, the database is recovered to be normal, at this time, the first data acquisition and dump program a1 of the first dumping task T1 is upgraded and run, that is, a new data breakpoint, that is, the second data breakpoint B2, is formed when the first data acquisition and dump program a1 updates configuration information and restarts the first dumping task T1.
In the embodiment of the present invention, after the first data collecting and dumping program a1 of the first dumping task T1 performs the data dumping task process K5_1 and has a fault, the fault recording process K4_1 of the first data collecting and dumping program a1 may record the time point of the fault data into the task fault table Y2, the recovery state entry in the task fault table Y2 is marked as not recovered, the task state corresponding to the first dumping task T1 in the task synchronization table Y1 is set to be the X2 (fault) state, and the state change time is updated.
In the embodiment of the present invention, when the first dumping task T1 fails due to database migration, the second dumping task T2 executes breakpoint resuming to implement a failure recovery process, which may specifically include: 1. the second dumping task T2 judges the failure of the first dumping task T1 and applies for acquiring the data dumping right; that is, the second dump task T2 accesses the task synchronization table Y1 to determine that the first dump task T1 has a fault, the second dump task T2 applies for and acquires a data dump right, and a task state corresponding to the second dump task T2 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring the first data breakpoint B1 and the second data breakpoint B2 from the task state table Y1; 3. breakpoint resuming from the first data breakpoint B1 to the second data breakpoint B2 is performed, so as to implement failure recovery; that is, the second data collecting and dumping program a2 executes a process of extracting buffered data K3_2, acquires a continuous real-time data queue from the first data breakpoint B1 to the second data breakpoint B2, executes a resume process K5_2 (the resume process K5_2 is also a normal data dumping task process of the second dumping task T2), and stores the continuous real-time data queue into the first buffered data file F1; and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
In this embodiment of the present invention, as shown in fig. 10, before the first dumping task T1 stores the collected real-time data into the first buffered data file according to the data break point generated by the second dumping task T2, and writes the data stored in the first buffered data file into the migrated relational database data table, the method may further include S901-S902:
s901, the first dumping task T1 judges the fault of the second dumping task T2 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right;
s902, acquiring a third data breakpoint B3 from the data dump time point and the state change time of the task synchronization table Y1; wherein the third data breakpoint B3 is a data breakpoint formed when the second data collection and dump program a2 performs the second configuration information update.
In an embodiment of the present invention, the storing, by the first dumping task T1, the collected real-time data into the first buffered data file according to the data break point generated by the second dumping task T2, and writing the data stored in the first buffered data file into the migrated relational database may include:
and acquiring a continuous real-time data queue between the second data breakpoint B2 and the third data breakpoint B3, storing the continuous real-time data queue into the first buffered data file, and writing the data between the first data breakpoint B1 and the third data breakpoint B3 stored in the first buffered data file into a migrated relational database.
In the embodiment of the present invention, after the migration of the database is completed, the database is recovered to normal, at this time, the second data acquisition and dump program a2 of the second dump task T2 may also be upgraded and run, that is, the second data acquisition and dump program a2 may also update the configuration information and restart the second dump task T2, so as to form a new data breakpoint, that is, the above-mentioned third data breakpoint B3.
In the embodiment of the present invention, after the second data collecting and dumping program a2 of the second dumping task T2 executes the normal data dumping task process K5_2 and has a fault, the fault recording process K4_2 of the second data collecting and dumping program a2 may record the time point of the fault data into the task fault table Y2, mark the recovery state entry in the task fault table Y2 as not recovered, set the task state corresponding to the second dumping task T2 in the task synchronization table Y1 as the X2 (fault) state, and update the state change time.
In the embodiment of the present invention, the recovery status in the task fault table Y2 may be represented by different status flags, which may include, but are not limited to, values 01 and 02, where 01 may represent not recovered and 02 may represent recovered; in other embodiments, any number, letter, character, etc. other than 01 and 02 may be used.
In the embodiment of the present invention, when the second dump task T2 fails due to the database migration, the first dump task T1 executes breakpoint resuming to implement a failure recovery process, which may specifically include: 1. the first dumping task T1 judges the fault of the second dumping task T2 and applies for acquiring the data dumping right; that is, the first dumping task T1 accesses the task synchronization table Y1 to determine that the second dumping task T2 has a fault, the first dumping task T1 applies for and acquires a data dumping right, and a task state corresponding to the first dumping task T1 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring a third data breakpoint B3 of a second dump task T2 from the task state table Y1; 3. breakpoint continuous transmission from the second data breakpoint B2 to the third data breakpoint B3 is performed, and the acquired real-time data is stored in a first buffer data file F1, so that fault recovery is realized; that is, the first data collecting and dumping program a1 executes the collecting process K1_1, acquires the continuous real-time data from the second data breakpoint B2 to the third data breakpoint B3, and executes the data buffering process K3_1 to store the continuous real-time data in the first buffered data file F1. A resume process K5_1 is executed (the resume process K5_1 is also a normal data dump task process of the first transfer task T1), a continuous data queue between a first data breakpoint B1 and a third data breakpoint B3 is extracted from the first buffer data file F1 and stored in a relational database data table W1, and breakpoint resume is realized; and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
In the embodiment of the present invention, it should be noted that, during the updating and restarting of the first dumping task T1, when the second dumping task T2 makes a breakpoint resume, real-time data between the first data breakpoint B1 and the second data breakpoint B2 may also continue to be stored in the second buffered data file F2, after the first dumping task T1 is updated and restarted, during the updating and restarting of the second dumping task T2, the first dumping task T1 may obtain the real-time data between the first data breakpoint B1 and the second data breakpoint from the second buffered data file F2, dump the real-time data into the relational database data table W1, and acquires real-time data between the second data breakpoint B2 and the third data breakpoint B3, stores the real-time data into the first buffered data file F1, and is dumped from the first buffered data file F1 to the relational database data table W1, thereby realizing the smooth migration of the relational database in the SCADA system.
EXAMPLE eleven
An apparatus 1 for improving real-time data dump of SCADA system, as shown in fig. 11, may include: a processor 11 and a computer-readable storage medium 12, wherein the computer-readable storage medium 12 stores instructions, and when the instructions are executed by the processor 11, the method for improving the stability of the real-time data dump of the SCADA system is implemented.
The embodiment of the invention comprises the following steps: establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2; starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file; applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table; when the first dumping task T1 fails during the data dumping process, according to the type of the failure, the second dumping task T2 continuously or temporarily resumes the transmission at the breakpoint according to the real-time data stored in the second buffered data file, thereby achieving automatic failure recovery. Through the scheme of the embodiment, the problem of data interruption caused by software fault in the real-time data dump process of the SCADA system is solved, the data interruption caused by abnormal data dump program is prevented, the actions that the faults caused by faults, upgrading, migration and the like cannot be automatically recovered and the historical data is filled up are avoided, the sequential write-in of the data dump is ensured, and therefore the stability of the real-time data dump of the SCADA system is improved.
The scheme of the embodiment of the invention at least comprises the following beneficial effects:
1. the embodiment of the invention can reduce the manual maintenance cost and ensure the rationality and the correctness of the upper-layer application data for the faults of the SCADA system data dump system;
the embodiment of the invention can realize the functions of breaking point data buffering and automatic dumping after the fault occurs, has better data guarantee function for guaranteeing the continuous real-time operation data of the SCADA system, improves the fault resistance of the whole application, reduces the cost of manual maintenance through the self-healing function, and can also guarantee the rationality and the correctness of the upper-layer application data.
2. For the SCADA system data service integration system, the embodiment of the invention often needs to change the data dump address and upgrade the upper-layer software of the SCADA system with the dump service function, can quickly realize service upgrade and exert a good and stable continuous dump function of historical data;
by the service functions of online redundancy of the real-time data dump task of the SCADA system, breakpoint continuous transmission of the real-time data dump of the SCADA system and the like, the invention can solve the problem of the service data breakpoint of data acquisition and dump program upgrading and database migration, and realize the automatic recovery processing and breakpoint-free upgrading effects of the service data.
3. Through the scheme of the embodiment, the upgrading process of the data acquisition and dump program has no data loss: the data dump task is implemented by means of online redundancy, when a data acquisition and dump program is closed and upgraded, the system can automatically judge the breakpoint, automatically realize breakpoint continuous transmission and realize no data loss.
4. The database migration work realizes data lossless migration through the scheme of the embodiment: after the database is migrated, a data dump task needs to be upgraded, and by sequentially operating the database migration, the data acquisition and dump program A1 upgrading and the data acquisition and dump program A2 upgrading, breakpoint continuous transmission is automatically completed, and data continuity and accuracy are guaranteed.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.

Claims (10)

1. A method for improving stability of data dump of a data acquisition and monitoring control SCADA system in real time comprises the following steps:
establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database;
when the first dumping task T1 fails during the data dumping process, according to the type of the failure, the second dumping task T2 continuously or temporarily resumes the transmission at the breakpoint according to the real-time data stored in the second buffered data file, thereby achieving automatic failure recovery.
2. The method for improving the stability of the real-time data dump of the SCADA system according to claim 1, wherein the establishing the task queue of the real-time data dump of the SCADA system comprises:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
3. The method for improving the stability of real-time data dump of a SCADA system according to claim 2, wherein the collecting and dumping configuration information comprises one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
4. The method for improving the stability of real-time data dump of SCADA system according to claim 1, wherein the starting the first dump task T1 and the second dump task T2 comprises:
respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
the operation information is accessed and recorded to the preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
5. The method for improving the stability of the real-time data dump of the SCADA system as in claim 4, wherein the task synchronization table Y1 comprises: task coding, task state, state change time and data dump time point; wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task; the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5;
the application for acquiring the data dump right by the first dump task T1 and executing normal data dump to write the real-time data into a preset relational database includes:
setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
after the data dump right is acquired, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
6. The method for improving the stability of a real-time data dump of a SCADA system according to claim 5, wherein the fault type comprises any one or more of the following: the failure of the dump program itself, the failure caused by program upgrade, and the failure caused by database migration.
7. The method of claim 6, wherein when the failure type is a failure of the dump program itself, the performing continuous or temporary breakpoint resuming by the second dump task T2 according to the failure type and the real-time data stored in the second buffered data file comprises:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right; acquiring a first data breakpoint B1 of the first dumping task T1 from a data dumping time point and a state change time of the task synchronization table Y1; extracting a continuous data queue comprising the first data breakpoint B1 from the second buffered data file and performing a resume process for breakpoint resume;
when the second dump task T2 does not have a fault, continuing to perform breakpoint resuming by the second dump task T2; or, when the fault of the first dumping task T1 is eliminated, the first dumping task T1 reappears the data dumping right, and obtains a second data breakpoint B2 of the second dumping task T2, and performs breakpoint resuming according to the real-time data stored in the first buffered data file.
8. The method of claim 6, wherein when the failure type is a failure caused by program upgrade, the performing continuous or temporary breakpoint resuming by the second dumping task T2 according to the failure type and according to the real-time data stored in the second buffered data file comprises:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right; acquiring a third data breakpoint B3 of the first dumping task T1 from the data dumping time point and the state change time of the task synchronization table Y1; extracting a continuous data queue comprising the third data breakpoint B3 from the second buffered data file and performing a resume process for breakpoint resume;
when the first dumping task T1 is not upgraded, continuing to perform breakpoint resuming by the second dumping task T2; when the first dumping task T1 is upgraded, the first dumping task T1 reappears the data dumping right, a fourth data breakpoint B4 of the second dumping task T2 is obtained, and breakpoint continuous transmission is performed according to real-time data stored in the first buffer data file.
9. The method of claim 6, wherein when the failure type is a failure caused by database migration, the performing continuous or temporary breakpoint resuming by the second dumping task T2 according to the failure type and according to the real-time data stored in the second buffered data file comprises:
when the relational database is migrated, the first dumping task T1 updates corresponding first configuration information, so that when a data dumping process fails, the second dumping task T2 uninterruptedly stores the collected real-time data into the first buffered data file according to data breakpoints generated by the migration and the first dumping task T1, so as to implement breakpoint transmission and automatic failure recovery;
when the first dumping task T1 completes updating the first configuration information and the second dumping task T2 updates the corresponding second configuration information due to the migration, the first dumping task T1 stores the collected real-time data into the first buffered data file according to a data breakpoint generated by the second dumping task T2, and writes the data stored in the first buffered data file into the migrated relational database, so as to implement breakpoint resume.
10. An apparatus for improving stability of data dump of a data acquisition and supervisory control SCADA system, comprising: processor and computer readable storage medium, wherein instructions are stored in the computer readable storage medium, and when the instructions are executed by the processor, the method for improving the stability of the real-time data dump of the SCADA system is implemented according to any one of claims 1 to 9.
CN201810898838.5A 2018-08-08 2018-08-08 Method and device for improving real-time data dump stability of SCADA system Active CN109189599B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810898838.5A CN109189599B (en) 2018-08-08 2018-08-08 Method and device for improving real-time data dump stability of SCADA system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810898838.5A CN109189599B (en) 2018-08-08 2018-08-08 Method and device for improving real-time data dump stability of SCADA system

Publications (2)

Publication Number Publication Date
CN109189599A CN109189599A (en) 2019-01-11
CN109189599B true CN109189599B (en) 2021-11-05

Family

ID=64920729

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810898838.5A Active CN109189599B (en) 2018-08-08 2018-08-08 Method and device for improving real-time data dump stability of SCADA system

Country Status (1)

Country Link
CN (1) CN109189599B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102197319A (en) * 2008-08-25 2011-09-21 沙特***石油公司 Data acquisition in an intelligent oil and gas field
CN105022774A (en) * 2014-04-15 2015-11-04 Ls产电株式会社 Apparatus, system and method for application log data processing
CN107798062A (en) * 2017-09-20 2018-03-13 中国电力科学研究院 A kind of transformer station's historical data unifies storage method and system

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4444688A1 (en) * 1994-12-15 1996-06-20 Abb Patent Gmbh Process for highly reliable and consistent message transmission
CN102023876B (en) * 2009-09-14 2013-04-17 漳州科能电器有限公司 Embedded system capable of upgrading software on line and online upgrading method
CN102495906A (en) * 2011-12-23 2012-06-13 天津神舟通用数据技术有限公司 Incremental data migration method capable of realizing breakpoint transmission
CN102945195B (en) * 2012-11-26 2015-11-18 国电南瑞科技股份有限公司 A kind of Master-standby redundancy replication method based on SQLite database
CN103559570B (en) * 2013-10-09 2016-03-09 东北电网有限公司 A kind of failure wave-recording main station information management system
KR101518720B1 (en) * 2015-02-15 2015-05-08 (주)부품디비 Method and apparatus for managing failure mode for condition based maintenance in marin resource production equipment
CN105007294B (en) * 2015-05-26 2018-06-08 华北电力大学(保定) Power transmission and transformation equipment state monitoring big data quickly receives and dissemination system
EP3278213A4 (en) * 2015-06-05 2019-01-30 C3 IoT, Inc. Systems, methods, and devices for an enterprise internet-of-things application development platform
CN108011699A (en) * 2017-12-20 2018-05-08 中国原子能科学研究院 reactor computer monitoring system network architecture method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102197319A (en) * 2008-08-25 2011-09-21 沙特***石油公司 Data acquisition in an intelligent oil and gas field
CN105022774A (en) * 2014-04-15 2015-11-04 Ls产电株式会社 Apparatus, system and method for application log data processing
CN107798062A (en) * 2017-09-20 2018-03-13 中国电力科学研究院 A kind of transformer station's historical data unifies storage method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
面向串行通信的实时数据转储网关的设计与实现;王昊轶等;《计算机***应用》;20100715(第07期);130-134 *

Also Published As

Publication number Publication date
CN109189599A (en) 2019-01-11

Similar Documents

Publication Publication Date Title
US20170046146A1 (en) Autonomously healing microservice-based applications
CN109032838B (en) Automatic verification method for consistency of backup and recovery data of virtual machine
CN111108481B (en) Fault analysis method and related equipment
CN111309734B (en) Method and system for automatically generating table data
CN112199356B (en) Fault processing method, device, server and storage medium
CN109116818B (en) Real-time data dump method and device during SCADA system upgrade
CN110063042A (en) A kind of response method and its terminal of database failure
CN105354102B (en) A kind of method and apparatus of file system maintenance and reparation
CN109150596B (en) SCADA system real-time data dump method and device
CN109241029B (en) Method and device for realizing smooth migration of SCADA system database
CN109189599B (en) Method and device for improving real-time data dump stability of SCADA system
CN111124724B (en) Node fault testing method and device of distributed block storage system
CN108804239B (en) Platform integration method and device, computer equipment and storage medium
JP5670935B2 (en) Distributed data management system and operation method thereof
CN113836223A (en) Heterogeneous database data synchronization method
CN114090173A (en) Automatic virtual machine migration method and system for heterogeneous virtualization platform
CN114003266A (en) Method and device for generating multiple differential packets based on Android flash firmware
CN116149707B (en) Method and device for detecting and avoiding upgrading risk of distributed system
CN112468327B (en) ICOS cluster-based address change method, device and medium
CN116028078B (en) Software remote upgrading method based on VPN technology
CN117632559A (en) Fault disk repairing method and device, storage medium and electronic equipment
CN116719791A (en) Information processing method, device, electronic equipment and storage medium
CN117951115A (en) Data migration method, device, equipment and medium based on Kafka cluster
CN117891563A (en) Control method and device of virtual machine, storage medium and electronic device
CN116775361A (en) Virtual cloud platform fault repairing method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant