CN111475520B - Method and system for automatically monitoring and alarming block data - Google Patents
Method and system for automatically monitoring and alarming block data Download PDFInfo
- Publication number
- CN111475520B CN111475520B CN202010281052.6A CN202010281052A CN111475520B CN 111475520 B CN111475520 B CN 111475520B CN 202010281052 A CN202010281052 A CN 202010281052A CN 111475520 B CN111475520 B CN 111475520B
- Authority
- CN
- China
- Prior art keywords
- block
- data
- block data
- node
- alarm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method and a system for automatically monitoring and alarming block data, which capture the health state of block synchronous data in real time, and timely find and alarm abnormal alarms such as differential cross risk, block height abnormality, block data abnormality, malicious attack of the same node address, DoS attack of oversized block data, double-flower block attack and the like. Meanwhile, a novel attack mode of node neighbor table pollution attack can be captured in the machine learning and manual intervention scene, workers are assisted in emergency treatment timely, and accuracy and stability of block data synchronization of block chain nodes are guaranteed.
Description
Technical Field
The invention relates to the technical field of block chains, in particular to a method and a system for automatically monitoring and alarming block data.
Background
With the rapid development of the block chain technology, the items of development and innovation of the block chain as the core technology are increasing day by day, most block chain node programs do not monitor and alarm block data in block chain link points, so that a manual intervention mode is needed, and related commands are frequently operated on the block chain link point programs to inquire and record the synchronous state of the block data, the change situation of the block height and the occurrence situation of abnormal block data, so that a large amount of manpower is consumed, and real-time observation, monitoring and alarm cannot be achieved.
Currently, checking the synchronization state of block data in a block chain node generally refers to querying the current node height by running a command line in the block chain node, comparing the current node height with the block heights of other nodes, and judging and calculating the synchronization state of blocks in block chain nodes in a manual mode.
Repeated manual intervention is also needed in a scene of checking the change of the block height in the block chain link points, and the change of the block chain block height can reflect whether the current block chain network is stable enough. The block height in the blockchain node is constantly changed, usually in an incremental manner, but when the blockchain network is unstable or is hacked, the block height of the blockchain node has a high rollback situation, and there is no relevant mechanism for giving feedback to the user in the original blockchain node program, so that the user is very difficult to find the problem in time.
In a scene of checking abnormal block data in a block chain node, repeated manual intervention is also needed, and as the block chain node program does not monitor the synchronous block data in real time and does not detect whether abnormal block data exists in real time, when the block data is synchronous, abnormal block data does not have a processing mechanism for timely alarming, so that abnormal conditions of a block chain network and the operation of the block chain node are difficult to find.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and adopts the following technical scheme:
a method for automatically monitoring and alarming block data comprises the following steps:
s1: in the code of the block link point program, using Hook technology in the P2P network protocol of the node program and the code logic of block storage to automatically capture complete block data when the node automatically synchronizes the block data in the block chain network;
s2: formatting the captured block data in the Hook code, recording the synchronized block data by using a database, and recording the data of the block synchronized by the node in the node block data table, wherein the recorded data comprises: block time, block hash, block height, block difficulty, block data size, miner address, transaction root hash, workload certification, parent block hash, tertiary block hash, and peer address;
s3: selecting a plurality of stable nodes from seed nodes of a block chain, compiling a query program to query the latest block data of the seed nodes, and recording the latest block data into a seed node block data table, wherein the recorded data content needs to record the node address information of the seed nodes besides fields in the node block data table, and the seed node block data table is compared with the latest data in the node block data table, so that whether the block data synchronization condition is healthy or not is conveniently checked when the node is compared with other stable nodes;
s4: writing a data comparison program to inquire the latest block data in the local node block data table and the seed node block data table in real time, and recording the synchronization condition of the block data into a block data synchronization condition table, wherein the data to be recorded comprises: checking whether the time, the latest block data of the node, the latest block data of the seed node, the height difference between the node and the block of the seed node, the data difference between the node and the block of the seed node and the synchronization condition of the block of the node are healthy or not;
s5: writing an alarm program, wherein the alarm program is used for analyzing, judging and alarming unhealthy data in the block data synchronization condition table, and the alarm program records the analysis and judgment results into a block monitoring alarm table, wherein the recorded data comprises: analyzing the result, judging basis, alarm type, analyzing time, alarm time and informing or not;
s6: and compiling a daemon program, wherein the daemon program is used for monitoring whether the query program, the data comparison program and the alarm program run normally or not, and the daemon program, the automatic monitoring system and the alarm system are started or stopped simultaneously.
Furthermore, an analysis rule is added in the alarm program, the record in the data synchronization condition table is analyzed in real time by using the analysis rule, and the analysis result is recorded in the block monitoring alarm table.
Further, the alarm types are divided into: general alarms, severe alarms, and unknown alarms.
Further, the data of the unknown alarm type is recorded into an unknown alarm record table, wherein the recorded data includes: alarm discovery time, alarm block data, and alarm occurrence times.
Further, the data in the unknown alarm record table is analyzed and judged in a mode of combining manual examination and machine learning.
A system for executing the above method for automatically monitoring and alarming block data comprises a block data acquisition module, a block data persistence storage module, a block data comparison module and a block abnormity alarm module, wherein,
the data acquisition module is internally provided with a Hook code and is used for acquiring multi-node block data;
the block data persistence storage module is used for storing newly added block data and inquiring the block data;
the block data comparison module is used for comparing single-node block data, comparing multi-node block data and collecting machine learning samples;
the block abnormity warning module is used for setting warning rules, analyzing and notifying the warning and performing machine learning analysis.
The invention has the beneficial effects that: compared with the prior art, the method for automatically monitoring and alarming block data provided by the invention captures the health state of the block synchronous data in real time, and finds and alarms abnormal alarms such as differential cross risk, block height abnormity, block data abnormity, malicious attack of the same node address, DoS attack of oversized block data, double-flower block attack and the like in time.
Meanwhile, a novel attack mode of node neighbor table pollution attack can be captured in the machine learning and manual intervention scene, workers are assisted in emergency treatment timely, and accuracy and stability of block data synchronization of block chain nodes are guaranteed.
The features and advantages of the present invention will be described in detail by embodiments in conjunction with the accompanying drawings.
Drawings
FIG. 1 is a main flow diagram of the present invention;
FIG. 2 is a flowchart of a daemon process in the invention;
fig. 3 is a system framework diagram of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood, however, that the description herein of specific embodiments is only intended to illustrate the invention and not to limit the scope of the invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
Referring to fig. 1-3, the present embodiment provides a method and a system for performing automatic monitoring and alarm on block data, wherein:
a method for automatically monitoring and alarming block data comprises the following steps:
s1: in the code of the block link point program, using Hook technology in the P2P network protocol of the node program and the code logic of block storage to automatically capture complete block data when the node automatically synchronizes the block data in the block chain network;
s2: formatting the captured block data in the Hook code, recording the synchronized block data by using a database, and recording the data of the block synchronized by the node in the node block data table, wherein the recorded data comprises: block time, block hash, block height, block difficulty, block data size, miner address, transaction root hash, workload certification, parent block hash, tertiary block hash, and peer address;
s3: selecting a plurality of stable nodes from seed nodes of a block chain, compiling a query program to query the latest block data of the seed nodes, and recording the latest block data into a seed node block data table, wherein the recorded data content needs to record the node address information of the seed nodes besides fields in the node block data table, and the seed node block data table is compared with the latest data in the node block data table, so that whether the block data synchronization condition is healthy or not is conveniently checked when the node is compared with other stable nodes;
s4: writing a data comparison program to inquire the latest block data in the local node block data table and the seed node block data table in real time, and recording the synchronization condition of the block data into a block data synchronization condition table, wherein the data to be recorded comprises: checking whether the time, the latest block data of the node, the latest block data of the seed node, the height difference between the node and the block of the seed node, the data difference between the node and the block of the seed node and the synchronization condition of the block of the node are healthy or not;
specifically, a data comparison program is used to compare whether data obtained by querying the node block data table and the seed node block data table are consistent, and compare whether block data of the node and the seed node block data roll back, whether block data received by the node are all from the same node address, whether size change of the block data is within a preset range, and the like, for specific examples, the data comparison program includes:
a. comparing the difference of the block heights in the node block data table and the seed node block data table, and judging whether the node program height is seriously lagged or seriously advanced, if so, judging that the synchronization condition of the node program block data is unhealthy. It is considered healthy if the lagging and leading heights are within a preset interval.
b. If the block heights in the 'local node block data table' and the 'seed node block data table' are consistent, block data are checked, whether the values of fields in the 'local node block data table' and the 'seed node block data table' are consistent or not is compared, inconsistency is recorded in the data difference field of the local node and the seed node block, the node block synchronization condition is marked to be unhealthy, and if the node block synchronization condition is always considered to be healthy.
c. If the block heights in the local node block data table and the seed node block data table roll back, it is considered that a 51% computational attack is most likely to be occurring in the blockchain network. And marking the block synchronization condition of the node as unhealthy.
S5: writing an alarm program, wherein the alarm program is used for analyzing, judging and alarming unhealthy data in the block data synchronization condition table, and the alarm program records the analysis and judgment results into a block monitoring alarm table, wherein the recorded data comprises: analyzing the result, judging basis, alarm type, analyzing time, alarm time and informing or not;
s6: and compiling a daemon program, wherein the daemon program is used for monitoring whether the query program, the data comparison program and the alarm program run normally or not, and the daemon program, the automatic monitoring system and the alarm system are started or stopped simultaneously.
Specifically, referring to fig. 2, the daemon process is a separate process for monitoring the query process, the data comparison process, whether the alarm process is operating normally, is started when the automated monitoring and alarm system is started, and then resides in the background in a service manner, if the automatic monitoring and alarm system has a termination in the process, the terminated process is restarted immediately, and then the log is recorded and the relevant personnel are notified by using a mail or an instant messaging tool.
Furthermore, an analysis rule is added in the alarm program, the record in the data synchronization condition table is analyzed in real time by using the analysis rule, and the analysis result is recorded in the block monitoring alarm table.
Further, the alarm types are divided into: general alarms, severe alarms, and unknown alarms.
In particular, general alarms, such as synchronization delays due to network fluctuations; serious alarm, such as block data synchronization serious lag or lead, according to the existing analysis rule, the analysis and judgment result is the known hacking method; unknown alarms, such as: the existing analysis rules cannot detect the reason that the synchronization condition of the node block is unhealthy, and the alarm type is recorded as 'unknown alarm' because the record cannot be analyzed.
Further, the data of the unknown alarm type is recorded into an unknown alarm record table, wherein the recorded data includes: alarm discovery time, alarm block data, and alarm occurrence times.
Further, the data in the unknown alarm record table is analyzed and judged in a mode of combining manual examination and machine learning.
Specifically, after the alarm program analyzes unhealthy block data, the alarm program can adopt a mail, an instant messaging tool and other modes to carry out alarm notification, the data in the unknown alarm record table is analyzed by adopting a mode of combining manual review and machine learning, manual intervention review is needed in the early stage, analysis rules are written and updated to perfect an alarm mechanism, when the data quantity is enough in the later stage, the machine learning mode can be adopted to carry out automatic analysis on the data in the unknown alarm record table, the data with higher identification degree is output by combining the existing rules, and whether the data needs to be added into the alarm judgment rules is judged by manual review.
The application also provides a system for executing the method for automatically monitoring and alarming block data, which comprises a block data acquisition module, a block data persistence storage module, a block data comparison module and a block abnormity alarming module, wherein,
the data acquisition module is internally provided with a Hook code and is used for acquiring multi-node block data;
the block data persistence storage module is used for storing newly added block data and inquiring the block data;
the block data comparison module is used for comparing single-node block data, comparing multi-node block data and collecting machine learning samples;
the block abnormity warning module is used for setting warning rules, analyzing and notifying the warning and performing machine learning analysis.
The invention uses Hook technology to obtain the block data of the node in the P2P protocol of the open source block chain node program and the code logic of the block storage, stores the obtained block data in the database, and then uses the automatic monitoring and alarming system to automatically monitor and alarm.
The above detailed description of the embodiments of the present invention is provided as an example, and the present invention is not limited to the above described embodiments. It will be apparent to those skilled in the art that any equivalent modifications or substitutions can be made within the scope of the present invention, and thus, equivalent changes and modifications, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the scope of the present invention.
Claims (6)
1. A method for automatically monitoring and alarming block data is characterized in that: the method comprises the following steps:
s1: in the code of the block link point program, using Hook technology in the P2P network protocol of the node program and the code logic of block storage to automatically capture complete block data when the node automatically synchronizes the block data in the block chain network;
s2: formatting the captured block data in the Hook code, recording the synchronized block data by using a database, and recording the data of the block synchronized by the node in the node block data table, wherein the recorded data comprises: block time, block hash, block height, block difficulty, block data size, miner address, transaction root hash, workload certification, parent block hash, tertiary block hash, and peer address;
s3: selecting a plurality of stable nodes from seed nodes of a block chain, compiling a query program to query the latest block data of the seed nodes, and recording the latest block data into a seed node block data table, wherein the recorded data content needs to record the node address information of the seed nodes besides fields in the node block data table, and the seed node block data table is compared with the latest data in the node block data table, so that whether the block data synchronization condition is healthy or not is conveniently checked when the node is compared with other stable nodes;
s4: writing a data comparison program to inquire the latest block data in the local node block data table and the seed node block data table in real time, and recording the synchronization condition of the block data into a block data synchronization condition table, wherein the data to be recorded comprises: checking whether the time, the latest block data of the node, the latest block data of the seed node, the height difference between the node and the block of the seed node, the data difference between the node and the block of the seed node and the synchronization condition of the block of the node are healthy or not;
s5: writing an alarm program, wherein the alarm program is used for analyzing, judging and alarming unhealthy data in the block data synchronization condition table, and the alarm program records the analysis and judgment results into a block monitoring alarm table, wherein the recorded data comprises: analyzing the result, judging basis, alarm type, analyzing time, alarm time and informing or not;
s6: and compiling a daemon program, wherein the daemon program is used for monitoring whether the query program, the data comparison program and the alarm program run normally or not, and the daemon program, the automatic monitoring system and the alarm system are started or stopped simultaneously.
2. The method of claim 1, wherein the method comprises: and adding an analysis rule in the alarm program, analyzing the record in the data synchronization condition table in real time by using the analysis rule, and recording the analysis result into a block monitoring alarm table.
3. The method of claim 2, wherein the method comprises: the alarm types are divided into: general alarms, severe alarms, and unknown alarms.
4. A method for automated monitoring and alerting of block data as claimed in claim 3, wherein: and recording the data of the unknown alarm type into an unknown alarm record table, wherein the recorded data comprises: alarm discovery time, alarm block data, and alarm occurrence times.
5. The method of claim 4, wherein the method comprises: and analyzing and judging the data in the unknown alarm record table in a mode of combining manual examination and machine learning.
6. A system for performing the method for automated monitoring and alerting of tile data of claim 1, wherein: the system comprises a block data acquisition module, a block data persistence storage module, a block data comparison module and a block abnormity warning module, wherein,
the data acquisition module is internally provided with a Hook code and is used for acquiring multi-node block data;
the block data persistence storage module is used for storing newly added block data and inquiring the block data;
the block data comparison module is used for comparing single-node block data, comparing multi-node block data and collecting machine learning samples;
the block abnormity warning module is used for setting warning rules, analyzing and notifying the warning and performing machine learning analysis.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010281052.6A CN111475520B (en) | 2020-04-10 | 2020-04-10 | Method and system for automatically monitoring and alarming block data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010281052.6A CN111475520B (en) | 2020-04-10 | 2020-04-10 | Method and system for automatically monitoring and alarming block data |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111475520A CN111475520A (en) | 2020-07-31 |
CN111475520B true CN111475520B (en) | 2022-05-13 |
Family
ID=71751671
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010281052.6A Active CN111475520B (en) | 2020-04-10 | 2020-04-10 | Method and system for automatically monitoring and alarming block data |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111475520B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112052137A (en) * | 2020-08-24 | 2020-12-08 | 深圳区块大陆科技有限公司 | Monitoring and alarm implementation of blockchain applications |
CN113609230A (en) * | 2021-09-16 | 2021-11-05 | 平安国际智慧城市科技股份有限公司 | Data synchronization abnormity warning method and device, computer equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109428779A (en) * | 2017-08-29 | 2019-03-05 | 武汉安天信息技术有限责任公司 | A kind of monitoring alarm method and device of distributed service |
CN109670950A (en) * | 2018-10-29 | 2019-04-23 | 平安科技(深圳)有限公司 | Transaction monitor method, device, equipment and storage medium based on block chain |
KR20190134296A (en) * | 2018-05-25 | 2019-12-04 | 서강대학교산학협력단 | System and method for managing and tracking a supply network based on a blockchain |
CN110599728A (en) * | 2019-09-17 | 2019-12-20 | 腾讯科技(深圳)有限公司 | Fire early warning method, device, equipment and storage medium based on block chain |
-
2020
- 2020-04-10 CN CN202010281052.6A patent/CN111475520B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109428779A (en) * | 2017-08-29 | 2019-03-05 | 武汉安天信息技术有限责任公司 | A kind of monitoring alarm method and device of distributed service |
KR20190134296A (en) * | 2018-05-25 | 2019-12-04 | 서강대학교산학협력단 | System and method for managing and tracking a supply network based on a blockchain |
CN109670950A (en) * | 2018-10-29 | 2019-04-23 | 平安科技(深圳)有限公司 | Transaction monitor method, device, equipment and storage medium based on block chain |
CN110599728A (en) * | 2019-09-17 | 2019-12-20 | 腾讯科技(深圳)有限公司 | Fire early warning method, device, equipment and storage medium based on block chain |
Non-Patent Citations (1)
Title |
---|
《DataPipeline王睿:业务异常实时自动化检测 — 基于人工智能的***实战_DataPipeline数见科技的博客-CSDN博客_自动化检测》;DataPipeline数见科技;《https://blog.csdn.net/datapipeline/article/details/104557057》;20200228;1-10 * |
Also Published As
Publication number | Publication date |
---|---|
CN111475520A (en) | 2020-07-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111475520B (en) | Method and system for automatically monitoring and alarming block data | |
CN107943668A (en) | Computer server cluster daily record monitoring method and monitor supervision platform | |
CN105260866B (en) | A kind of locker Dead on Arrival detection method | |
CN105528280A (en) | Method and system capable of determining log alarm grades according to relationship between system logs and health monitoring | |
CN103595416A (en) | System and method for compressing production data stream and filtering compressed data with different criteria | |
JP2020057416A (en) | Method and device for processing data blocks in distributed database | |
CN104574219A (en) | System and method for monitoring and early warning of operation conditions of power grid service information system | |
CN111221890B (en) | Automatic monitoring and early warning method and device for universal index class | |
CN108809734A (en) | Network alarm root-cause analysis method, system, storage medium and computer equipment | |
Zhang et al. | A survey on quality assurance techniques for big data applications | |
CN115118581A (en) | Internet of things data full-link monitoring and intelligent security system based on 5G | |
CN116028887B (en) | Analysis method of continuous industrial production data | |
CN117951128A (en) | Data quality inspection method based on artificial intelligence | |
CN111597510A (en) | Power transmission and transformation operation and inspection data quality assessment method and system | |
CN117240594B (en) | Multi-dimensional network security operation and maintenance protection management system and method | |
CN108055152B (en) | Communication network information system abnormity detection method based on distributed service log | |
CN117215870A (en) | Method for monitoring system to ensure service continuity | |
CN110399242B (en) | Information maintenance management system based on Hadoop platform | |
CN113836203A (en) | Network data diagnosis detection analysis system | |
CN112148671A (en) | Data management system for Robot | |
CN116166652A (en) | Intelligent monitoring method based on data quality of scientific data center | |
CN211928569U (en) | Server maintenance system of acquisition workstation | |
CN117743473B (en) | Data management synchronization performance monitoring system | |
CN116743618B (en) | Data acquisition and analysis method, equipment and medium of station remote equipment | |
CN116192612B (en) | System fault monitoring and early warning system and method based on log analysis |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |