CN112835526A - Engineering data persistence storage system and method - Google Patents
Engineering data persistence storage system and method Download PDFInfo
- Publication number
- CN112835526A CN112835526A CN202110169794.4A CN202110169794A CN112835526A CN 112835526 A CN112835526 A CN 112835526A CN 202110169794 A CN202110169794 A CN 202110169794A CN 112835526 A CN112835526 A CN 112835526A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- file
- persistence
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002688 persistence Effects 0.000 title claims abstract description 54
- 238000000034 method Methods 0.000 title claims abstract description 13
- 230000002085 persistent effect Effects 0.000 claims description 30
- 238000004891 communication Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 description 7
- 238000013461 design Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to an engineering data persistence storage system which is characterized by comprising a memory database, a data persistence storage module, a data interface module and a persistence database which are sequentially connected; and the data persistence storage module receives data sent by the memory database and stores a corresponding data set in the persistence database through the data interface module. The invention improves the stability and reliability of the industrial control system, is used for ensuring that a user can quickly inquire historical data under the condition that the system is in shutdown or failure, and ensures the data integrity in the operation process of the system and the data safety.
Description
Technical Field
The invention relates to the technical field of automation control, in particular to a system and a method for engineering data persistence storage.
Background
Because the industrial control system needs to control the equipment uninterruptedly for a long time, a user has requirements on continuous working performance and real-time performance of the control system, and in order to ensure the safety and reliability of the industrial control system, engineering data of a control process needs to be stored timely and stably so as to be inquired by the user when needed.
The engineering data persistence storage is a persistence storage function for providing real-time data points for project engineering, can store temporary data in a real-time memory database into storage equipment capable of being stored permanently, ensures that the data cannot be lost under the conditions of shutdown and failure of the engineering equipment, and provides a channel for fault tracking and data query of users.
Disclosure of Invention
In view of the above, the present invention is directed to a system and method for persistent storage of engineering data,
in order to achieve the purpose, the invention adopts the following technical scheme:
a project data persistence storage system comprises a memory database, a data persistence storage module, a data interface module and a persistence database which are connected in sequence; and the data persistence storage module receives data sent by the memory database and stores a corresponding data set in the persistence database through the data interface module.
Further, the memory database comprises a master library, a slave library and a real-time data persistence storage module.
Furthermore, the persistent database supports the storage of multiple data types, and the data of different data types are stored in different databases; the data types include: numerical data, character string data, picture data, file data, audio data.
Further, the storage mode of the persistent database comprises file stream storage and file system storage; the file stream storage converts all files into file streams and transmits the file streams to a persistent database; and the file system stores the file into the file system, the stored address is stored into the persistent database, the picture basic information is stored in the picture database, the file basic information is stored in the file database, and the audio basic information is stored in the audio database.
Further, the real-time data persistence module transfers the real-time value data of the data points in the real-time memory database to the data persistence storage module; the storage information of the data points comprises data point numbers, data point timestamps, data point values, data point labels and communication quality.
Furthermore, the data interface module comprises a picture data interface, a file data interface, an audio data interface, a data point historical data query interface, a picture data query interface, a file data query interface and an audio data query interface.
A method for persistently storing engineering data comprises the following steps:
step S1, the slave library in the memory database periodically reads the data of the corresponding data points from the master library according to the set time;
step S2, the real-time data persistence storage module distributes data point id to the data temporarily stored in the slave database according to the sequence of the data point id and sends the data point id to the data persistence storage module;
step S3, the data persistence storage module splices the received data according to different types of collection nouns of data points, pictures, files and audios and data point id according to the type of the received data, and the data is used as the name of a collection in the persistence database;
and step S4, the data persistence storage module creates a name set in the persistence database through the corresponding data interface module according to the generated set name, and adds the corresponding data into the set in sequence.
Compared with the prior art, the invention has the following beneficial effects:
the invention effectively improves the stability and reliability of the industrial control system, is used for ensuring that a user can quickly inquire historical data under the condition that the system is in shutdown or failure, and ensures the integrity of the data in the running process of the system and the safety of the data.
Drawings
FIG. 1 is a system architecture diagram of the present invention;
FIG. 2 is a flow chart of storage according to an embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
Referring to fig. 1, the present invention provides an engineering data persistent storage system, which includes a memory database, a data persistent storage module, a data interface module and a persistent database, which are connected in sequence; and the data persistence storage module receives data sent by the memory database and stores a corresponding data set in the persistence database through the data interface module.
In this embodiment, the memory database includes a master library, a slave library and a real-time data persistence storage module.
Preferably, the real-time data persistence module is responsible for transferring the real-time value data of the data point in the real-time memory database to the data persistence storage module. The storage information of a data point at least comprises information such as a data point number, a data point time stamp, a data point value, a data point label, communication quality and the like. The following persistence strategies are required:
(1) data is persisted periodically.
(2) And carrying out data persistence according to the set persistence precision.
In the embodiment, the persistent database directly performs write-in and read-out operations on the disk, supports storage of multiple data types, and performs database-based storage on data of different data types; the data types include: numerical data, character string data, picture data, file data, audio data.
There may be multiple data in the persistent database, multiple collections exist in one database, and many documents may be stored in one collection. The basic information of the data points, the picture basic information, the file basic information and the audio basic information are stored in a set in the form of documents. Basic information of the data points is stored in a data point database in the database; there are two storage modes of files: file stream storage and file system storage. And (3) file stream storage: and converting all the files into file streams and transmitting the file streams to the persistent database. And file system storage: storing the file into a file system, and storing the stored address into a persistent database; the picture basic information is stored in a picture database, the file basic information is stored in a file database, and the audio basic information is stored in an audio database.
In this embodiment, the persistent database storage mode includes file stream storage and file system storage; the file stream storage converts all files into file streams and transmits the file streams to a persistent database; and the file system stores the file into the file system, the stored address is stored into the persistent database, the picture basic information is stored in the picture database, the file basic information is stored in the file database, and the audio basic information is stored in the audio database.
In this embodiment, the real-time data persistence module migrates the real-time value data of the data point in the real-time memory database to the data persistence storage module; the storage information of the data points comprises data point numbers, data point timestamps, data point values, data point labels and communication quality.
In this embodiment, the data interface module includes a picture data interface, a file data interface, an audio data interface, a query data point historical data interface, a query picture data interface, a query file data interface, and a query audio data interface.
(1) Preferably, the interface for receiving picture data provides an interface for uploading pictures, and the interface parameters need to include additional information parameters (using character strings as parameters) in addition to the picture stream data. And storing the data in a picture type database, wherein the stored information at least comprises a data point number, a timestamp, a picture name, a picture size, a picture format and the like. And supports the following two storage modes:
1) the pictures are directly stored in a persistent database in a binary mode;
2) the pictures are stored in a file system, and the picture paths and related information are stored in a persistent database.
(2) The file data receiving interface provides a file uploading interface, and the interface parameters need to contain additional information parameters (using character strings as parameters) besides file stream data. And storing the data in a file type database, wherein the storage information at least comprises a data point number, a timestamp, a file name, a file size, a file type and the like. And supports the following two storage modes:
1) directly storing the file in a persistent database in a binary mode;
2) files are stored in a file system, file paths and related information are stored in a persistent database.
(3) The interface receives audio data and provides an audio uploading interface, and the interface parameters need to contain additional information parameters (using character strings as parameters) besides audio stream data. And storing the data in an audio type database, wherein the stored information at least comprises a data point number, a time stamp, an audio name, an audio size, an audio format and the like. And supports the following two storage modes:
1) the audio is directly stored in a persistent database in a binary mode;
2) the audio is stored in a file system, the audio path and related information is stored in a persistent database.
(4) The query data point historical data interface can query historical data (numerical data) in a specified time period through one or more data point numbers and return the historical data in a paging mode, and can specify a sorting rule of the data, such as ascending or descending time.
(5) And the image data query interface can query the image list of the latest or appointed time period according to the number of the configuration point. The functions are as follows:
1) the method can quickly inquire the latest one or more pictures of the specified one or more configuration points, allows a picture list to be returned in a paging mode under the condition of the plurality of pictures, and can specify the ordering rule of the pictures, such as time ascending or descending;
2) the picture list in the appointed time interval can be inquired according to the appointed configuration point, the picture list can be returned in a paging mode, and the ordering rule of the pictures can be appointed, such as ascending order or descending order of time.
(6) And the file data query interface can query a file list (file attribute information) of the latest or specified time period according to the number of the configuration point. The functions are as follows:
1) the method can quickly inquire one or more files which are specified and are closest to one or more configuration points, allows file data to be returned in a paging mode under the condition of the plurality of files, and can specify the ordering rule of the files, such as time ascending or descending;
2) the file list in the appointed time interval can be inquired according to the appointed configuration point, the file data can be returned in a paging mode, and the ordering rule of the files can be appointed, such as ascending order or descending order of time;
the downloading and consulting of files requires separate interface support.
(7) And the audio data query interface can query an audio list (file attribute information) of the latest or specified time period according to the number of the configuration point. The functions are as follows:
1) the method can quickly inquire one or more files which are specified and are closest to one or more configuration points, allows file data to be returned in a paging mode under the condition of the plurality of files, and can specify the ordering rule of the files, such as time ascending or descending;
2) the file list in the specified time interval can be inquired according to the specified configuration point, the file data can be returned in a paging mode, and the sorting rule of the files can be specified, such as ascending order or descending order of time.
In this embodiment, the persistent database data structure is specifically as follows:
(1) basic information structure design of data points
Stored in the data point database, the collection name uses "point" to stitch the data point id.
(2) Design of picture basic information structure
Stored in the picture database, the set name concatenates the data point ids using "image".
(3) Design of basic information structure of file
Stored in the file database, the collection name uses the "filesystems" to splice the data point id.
(4) Audio basic information structure design
Stored in the audio database, the set name uses the "audio" splice data point id.
Referring to fig. 2, in this embodiment, a method for persistently storing engineering data is further provided, which includes the following steps:
step S1, the slave library in the memory database periodically reads the data of the corresponding data points from the master library according to the set time;
step S2, the real-time data persistence storage module distributes data point id to the data temporarily stored in the slave database according to the sequence of the data point id and sends the data point id to the data persistence storage module;
step S3, the data persistence storage module splices the received data according to different types of collection nouns of data points, pictures, files and audios and data point id according to the type of the received data, and the data is used as the name of a collection in the persistence database;
and step S4, the data persistence storage module creates a name set in the persistence database through the corresponding data interface module according to the generated set name, and adds the corresponding data into the set in sequence.
The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.
Claims (7)
1. The engineering data persistence storage system is characterized by comprising a memory database, a data persistence storage module, a data interface module and a persistence database which are sequentially connected; and the data persistence storage module receives data sent by the memory database and stores a corresponding data set in the persistence database through the data interface module.
2. The engineering data persistent storage system of claim 1, wherein the in-memory database comprises a master library, a slave library and a real-time data persistent storage module.
3. The engineering data persistent storage system according to claim 1, wherein the persistent database supports storage of multiple data types, and different data types are stored in different databases; the data types include: numerical data, character string data, picture data, file data, audio data.
4. The project data persistence storage system of claim 1, wherein the persistent database storage comprises file stream storage and file system storage; the file stream storage converts all files into file streams and transmits the file streams to a persistent database; and the file system stores the file into the file system, the stored address is stored into the persistent database, the picture basic information is stored in the picture database, the file basic information is stored in the file database, and the audio basic information is stored in the audio database.
5. The system according to claim 1, wherein the real-time data persistence module migrates real-time value data of data points in a real-time memory database to the data persistence storage module; the storage information of the data points comprises data point numbers, data point timestamps, data point values, data point labels and communication quality.
6. The engineering data persistent storage system of claim 1, wherein the data interface module comprises a picture data interface, a file data interface, an audio data interface, a query data point historical data interface, a query picture data interface, a query file data interface, and a query audio data interface.
7. A method for persistently storing engineering data is characterized by comprising the following steps:
step S1, the slave library in the memory database periodically reads the data of the corresponding data points from the master library according to the set time;
step S2, the real-time data persistence storage module distributes data point id to the data temporarily stored in the slave database according to the sequence of the data point id and sends the data point id to the data persistence storage module;
step S3, the data persistence storage module splices the received data according to different types of collection nouns of data points, pictures, files and audios and data point id according to the type of the received data, and the data is used as the name of a collection in the persistence database;
and step S4, the data persistence storage module creates a name set in the persistence database through the corresponding data interface module according to the generated set name, and adds the corresponding data into the set in sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110169794.4A CN112835526A (en) | 2021-02-08 | 2021-02-08 | Engineering data persistence storage system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110169794.4A CN112835526A (en) | 2021-02-08 | 2021-02-08 | Engineering data persistence storage system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112835526A true CN112835526A (en) | 2021-05-25 |
Family
ID=75932646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110169794.4A Pending CN112835526A (en) | 2021-02-08 | 2021-02-08 | Engineering data persistence storage system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112835526A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114764326A (en) * | 2022-03-30 | 2022-07-19 | 中国石油天然气集团有限公司 | Data layer extension method and system of integrated software |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750202A (en) * | 2012-06-06 | 2012-10-24 | 宇龙计算机通信科技(深圳)有限公司 | Data protection method and device |
| CN106909641A (en) * | 2017-02-16 | 2017-06-30 | 青岛高校信息产业股份有限公司 | A kind of real-time data memory device |
| US20190251067A1 (en) * | 2012-09-24 | 2019-08-15 | Sandisk Technologies Llc | Snapshots for a non-volatile device |
| CN111462290A (en) * | 2020-04-24 | 2020-07-28 | 福州大学 | Unity 3D-based chemical virtual simulation experiment system and development method thereof |
| CN112118283A (en) * | 2020-07-30 | 2020-12-22 | 爱普（福建）科技有限公司 | Data processing method and system based on multi-level cache |
-
2021
- 2021-02-08 CN CN202110169794.4A patent/CN112835526A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102750202A (en) * | 2012-06-06 | 2012-10-24 | 宇龙计算机通信科技(深圳)有限公司 | Data protection method and device |
| US20190251067A1 (en) * | 2012-09-24 | 2019-08-15 | Sandisk Technologies Llc | Snapshots for a non-volatile device |
| CN106909641A (en) * | 2017-02-16 | 2017-06-30 | 青岛高校信息产业股份有限公司 | A kind of real-time data memory device |
| CN111462290A (en) * | 2020-04-24 | 2020-07-28 | 福州大学 | Unity 3D-based chemical virtual simulation experiment system and development method thereof |
| CN112118283A (en) * | 2020-07-30 | 2020-12-22 | 爱普（福建）科技有限公司 | Data processing method and system based on multi-level cache |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114764326A (en) * | 2022-03-30 | 2022-07-19 | 中国石油天然气集团有限公司 | Data layer extension method and system of integrated software |
| CN114764326B (en) * | 2022-03-30 | 2023-09-19 | 中国石油天然气集团有限公司 | Data layer expansion method and system of integrated software |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111400408B (en) | Data synchronization method, device, equipment and storage medium | |
| CN100375047C (en) | Management method for computer log | |
| CA2603901C (en) | System and methods for facilitating a linear grid database with data organization by dimension | |
| US7149750B2 (en) | Method, system and program product for extracting essence from a multimedia file received in a first format, creating a metadata file in a second file format and using a unique identifier assigned to the essence to access the essence and metadata file | |
| US6826726B2 (en) | Remote document updating system using XML and DOM | |
| JP4782367B2 (en) | Metadata generation device | |
| CN103067483A (en) | Remote data increment synchronization method and device based on data package | |
| CN109213820B (en) | Method for realizing fusion use of multiple types of databases | |
| CN110334326A (en) | A kind of method and system for identifying recipe file and being converted into XML file | |
| US9104689B2 (en) | Method for synchronizing documents for disconnected operation | |
| CN112835526A (en) | Engineering data persistence storage system and method | |
| CN102810115A (en) | Method for implementing multi-layer distributed document management system | |
| US20070094278A1 (en) | Data transfer services | |
| CN111625520A (en) | Universal mapping method and system for field types of heterogeneous database | |
| CN114416868A (en) | Data synchronization method, device, equipment and storage medium | |
| US11106739B2 (en) | Document structures for searching within and across messages | |
| CN111427845A (en) | Interactive modeling analysis operator data exchange method | |
| CN116821179A (en) | Dream database cross-database searching system and method | |
| EP1349166A1 (en) | System | |
| KR100436702B1 (en) | System and method for providing virtual document | |
| CN116150236A (en) | Data synchronization method and device, electronic equipment and computer readable storage medium | |
| JP2004129200A (en) | Information material production system and method | |
| CN101661277A (en) | Part processing method and device used for numerical control machine | |
| CN113641760A (en) | Data synchronization method and device | |
| WO2013160767A2 (en) | Data package format library |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210525 |
|
| RJ01 | Rejection of invention patent application after publication |