CN109274728B

CN109274728B - Block chain data life cycle management method

Info

Publication number: CN109274728B
Application number: CN201811022588.5A
Authority: CN
Inventors: 林东海
Original assignee: Beijing Feina Taike Information Technology Co ltd
Current assignee: Beijing Feina Taike Information Technology Co ltd
Priority date: 2018-09-03
Filing date: 2018-09-03
Publication date: 2021-08-10
Anticipated expiration: 2038-09-03
Also published as: CN109274728A

Abstract

The invention provides a block chain data life cycle management method, which adds monitoring variables to a block data structure, wherein the variables comprise block access times, block last access time, block access validity period, block lifetime, precipitation marks and precipitation pool index values. The block access times store the total number of times the block has been accessed since creation; recording the last time when the block is accessed at the last time in the last access time of the block; the block access lifetime records the maximum period of time a block can be accessed, and if a block is not accessed beyond this period, the block is deposited. The blocks after the precipitation keep the block index value, the block data is not in synchronization any more, and is temporarily stored in the precipitation pool, but can still be recovered within the lifetime of the block.

Description

Block chain data life cycle management method

Technical Field

The present invention relates to a data management method, and more particularly, to a method for managing a life cycle of blockchain data.

Background

The blockchain technology is a brand new distributed infrastructure and computing mode that uses blockchain data structures to verify and store data, uses distributed node consensus algorithms to generate and update data, uses cryptography to secure data transmission and access, and uses intelligent contracts composed of automated script codes to program and manipulate data. For example, the bitcoin is a completely distributed system, and the system consists of a plurality of nodes, and each node has accounting capability. As transactions continue and the tiles continue to be produced, the tiles on each node are clustered together in time sequence to form a chain of tiles. The system involves several core concepts, nodes, chains, blocks, records, and relationships as shown in FIG. 1.

The advent of blockchain technology provides a non-repudiatable trusted execution environment for the running of intelligent contracts. Some block chain developers combine the block chain and the intelligent contract to create a bottom system, namely a block chain platform, such as an ether house, an EOS and the like, which realizes the intelligent contract and open source on the block chain, so that the development of block chain application is greatly facilitated, and the application scene of the block chain technology is expanded.

One important problem faced by blockchain platforms today is inefficient synchronization and access of node data. In fact, today's public etherhouse networks and bitcoin networks can handle 7-10 transactions per second on average, a figure far below that of centralized payment processors like Visa, which can handle about 8000 transactions per second on average. The alliance chain and the private chain represented by the logistics block chain are also easy to get into bottleneck because the data synchronization and access in the nodes are slow, and the block chain growth speed is easy to get into the bottleneck. The data synchronization and access efficiency of the nodes is low, so that the application programs such as real-time payment are difficult to run efficiently on the blockchain, and as the time for confirming the payment is prolonged, inconvenience is brought to users, and the reason why the credit card payment mode such as PayPal and Visa is still more attractive is one of the main reasons. As more complex DApps begin to rely on the same one blockchain network, the problem caused by sluggish transaction speeds only becomes more complex.

The essential reason is that in distributed storage of the block chain, the longer the time is, the larger the data stored in each block of the node is, the longer the synchronization time of the newly-entered block is, and the longer the access time is. But in fact, much historical data is invalid or worthless data, and the operation efficiency of the block chain is greatly influenced.

From a technical point of view, the consensus protocol for all blockchains has a challenging limitation: each fully participating node in the network must verify each transaction and must be consistent with its other nodes, which is an integral part of blockchain technology that secures blockchains by creating a distributed ledger.

The block chain is divided according to the application scene and the data read-write range, and can be divided into three types: public block chains, federation block chains, and private block chains. The public blockchain refers to a blockchain on which anyone can participate and trade, read, write and know all over the world. The common block chain is completely distributed, data are public, user participation degree is high, popularization is convenient, and system operation depends on a reward mechanism. The application scenes comprise bitcoin, Ether house and the like. A federation chain means, for example, a plurality of financial institutions, each of which, as a node, together form a distributed system of blockchains. The pair of blockchains is limited to participating nodes between these organizations, or to a consensus process. A typical application scenario is like IBM's hyper ledger (HyperLedger). A fully private blockchain refers to a blockchain whose write rights are in only one organization's hand. The read right is either open to the outside or is limited to any extent.

On most blockchains like bitcoin and etherhouses, nodes are operated by the public. While decentralized consensus mechanisms provide some important advantages such as fault tolerance, security, political neutrality, and authenticity, this method of verifying the chain comes at the cost of reduced synchronization and access efficiency. As shown in fig. 2, which is an example of a block chain, two blocks in the chain are bound by a hash value of a predecessor node and a hash value of a successor node in a block header, and a block of each block stores record data. As the blockchains become longer, the number of blocks increases rapidly, requiring more and more processing power to verify the blocks, which may create bottlenecks in the network of blockchains, slowing down the speed of creating new applications. However, in the federation chain and the private chain represented by the logistics block chain, the useless data of part of the blocks can be precipitated or destroyed under the condition that the nodes reach consensus. The node where the user is located can transmit the marking information to other nodes in the network after marking the block needing to be deposited or destroyed, and request confirmation information. And if other nodes agree to precipitate or destroy the marked node, returning confirmation information to the initiating node. The specific number of nodes needed to achieve a common identification to deposit and destroy the block is set according to the specific block chain type and application context.

Disclosure of Invention

Based on the characteristics, the invention introduces a data life cycle mechanism to solve the synchronization, access and storage pressure of the block data. Data Lifecycle Management (DLM) is a policy-based method for managing the flow of data of an information system throughout the lifecycle: from creation and initial storage, to deletion after it is determined to be invalid. In the blockchain technique, all data information is always stored in different blocks of a node, but a large part of the data in the blocks is rarely accessed, and even some data is invalid and redundant. If there is no mechanism to filter and remove these data when the blocks in the node are synchronized, the block chain is continuously extended with the increase of the number of blocks, the node synchronization is difficult to be performed due to the increase of the block data in the node, and the storage of the block information of the node itself may also encounter a bottleneck. If information such as access times, last access time, block access validity period, block lifetime and the like is added to each block data structure, block screening can be performed based on the variables to determine which blocks are valid blocks. And then according to a set verification mechanism, settling some blocks which are not accessed for a long time, namely removing the blocks from the block chain and temporarily storing the blocks in a settling pool, wherein the settling pool is independent from each block in the block chain for synchronous backup. If not yet accessed during the settling period, the block is destroyed.

The invention provides a method for managing the life cycle of block chain data, which solves the problem of invalid and redundant data processing in a block chain, and adopts the following technical scheme:

a block chain data life cycle management method adds monitoring variables to a block data structure, wherein the variables comprise block access times, block last access time, block access validity period, block lifetime, precipitation marks and precipitation pool index values, and the management method comprises the following steps:

(1) before synchronizing the block data of each node, the system checks the last access time and the access times of each block, and marks the block as a sediment block if the block is not accessed again within the block validity period and the access times are less than a set threshold;

(2) the sediment block is not linked in the original block chain any more, the previous block of the sediment block is linked with the next block, and the hash value of the subsequent block and the hash value of the previous block corresponding to the headers of the two blocks are reset to form a new block chain;

(3) the blocks in the sedimentation pool and the blocks in the block chain are independently synchronized and backed up, the contents of the blocks are required to be restored at the user node, and after the nodes with the appointed number are confirmed, the blocks can be restored to the original block chain position according to the index hash value in the block head; if the sediment block is accessed in the block lifetime, the sediment mark can be automatically cancelled and the original block chain position is recovered;

(4) if the time exceeds the maximum time limit that the block can stay in the sedimentation tank, the data of the block is not synchronized to each node on the block chain, and the block is destroyed in the sedimentation tank, namely the index value of the block is removed from the sedimentation tank index chain, and the block is not synchronized and backed up.

In the step (1), the threshold value can be set by a user according to the application background of the block chain, and the user adjusts the precipitation or destruction strategy of the block by adjusting the threshold value of the minimum access times of the block synchronization.

When the block data of each node is synchronized, the speed of synchronization is adjusted to be increased, and the threshold is set to be a higher value.

In the step (2), the settled block does not participate in the block chain synchronization any more, and only the index hash value in the block header is reserved and temporarily stored in the settling tank.

Any node in the distributed network can initiate a request for precipitating and destroying useless block data, and N _ application nodes on other nodes can approve to synchronize in the whole network.

The number of nodes to be requested N _ apply is set according to factors such as the type of the block chain, the application environment of the block chain and the like.

In step (3), when a certain node is ready to perform block synchronization, the system queries the last access time of the block and the access times of the block, and based on whether the block is still within the access validity period, if the validity period is exceeded and the access times are less than a set threshold v _ total, the block is deposited to a deposition pool, the block in the deposition pool is not synchronized normally, and data is still stored, and the deposition pool is used as a whole for synchronous backup.

When the nodes are ready to perform block synchronization, the user can mark invalid blocks according to the requirement, and the system sends block precipitation or destruction requests to other nodes after marking the blocks. If the agreeing node exceeds the number N _ apply of the nodes needing to be requested, block precipitation or destruction can be carried out, and then block synchronization is carried out; if the number of the agreeing nodes is less than the number of the required nodes N _ apply, the block synchronization is directly performed.

The invention redesigns the data structure of the block by introducing a data life cycle mechanism, introduces variables such as access times, last access time, block access validity period, block lifetime and the like, and precipitates or destroys the block which is not accessed for a long time. On one hand, the method is beneficial to improving the synchronization and access efficiency of the nodes, and enables the synchronization and access of the nodes to be carried out in a shorter time, so that the condition that a network is blocked by a large number of unfinished transactions when the transactions are carried out is relieved, and the transaction speed is accelerated. On the other hand, the method is also favorable for relieving the local pressure on the storage of the block data in the nodes, and the condition that the storage space is insufficient along with the gradual extension of the block chain is avoided.

The 'data lifecycle' mechanism includes two parts, automatic lifecycle management and node-initiated lifecycle management. Invalid block precipitation and destruction in block synchronization can be automatically carried out, and can be carried out in a manual request mode, so that the operation efficiency of a block chain is greatly improved.

Drawings

FIG. 1 is a diagram illustrating the relationship between nodes, chains, blocks, and records;

FIG. 2 is an example schematic of an existing blockchain;

FIG. 3 is a basic workflow diagram of a blockchain;

FIG. 4 is a block header data structure of the present invention;

FIG. 5 is a flow chart of an automatic lifecycle management algorithm;

fig. 6 is a flow chart of a node-initiated lifecycle management algorithm.

Detailed Description

The invention introduces a 'data life cycle' mechanism, redesigns the block data structure, increases the variables of block access times, block last access time, block access validity period, block lifetime and the like, and solves the problem of low efficiency in block data synchronization and access, and the specific block data structure is shown in fig. 4. The block access times store the total number of times the block has been accessed since creation; recording the last time when the block is accessed at the last time in the last access time of the block; the block access lifetime records the maximum period of time a block can be accessed, and if a block is not accessed beyond this period, the block is deposited. The blocks after the precipitation keep the block index value, the block data is not in synchronization any more, and is temporarily stored in the precipitation pool, but can still be recovered within the lifetime of the block.

The 'data lifecycle' mechanism established by the invention comprises two parts, namely automatic lifecycle management and node-initiated lifecycle management. The basic workflow of the blockchain is shown in fig. 3: firstly, when a transaction record occurs, a node whole-network broadcast record is sent, and a whole-network node can monitor a new transaction record. And then, verifying the transaction records by the nodes, putting the transaction records into a node cache region, and starting mine excavation calculation to obtain the accounting right. If a certain node strives for the accounting right, the blocks can be packaged, recorded and generated, the blocks are identified to the whole network broadcast, the whole network node synchronizes the new blocks, and the identified blocks are added to the tail of the block chain. However, theoretically every node receives the full history, which increases the overall network load by many nodes that are already useless.

The automatic life cycle management of the invention is that when the node carries out block data synchronization, whether each block is in the access validity period (namely the longest period of time that the block can be accessed) is automatically detected. Before synchronizing the block data of each node, the system checks the last access time and the access times of each block, and marks the block as a sediment block if the block is not accessed again within the block validity period and the access times are less than a set threshold. The threshold value needs to be set by a user according to the application background of the block chain, the user can adjust the sedimentation or destruction strategy of the block by adjusting the minimum access time threshold value of the block synchronization, and if the synchronization speed needs to be adjusted to a higher level, the threshold value can be set to a higher value. The sediment block is not linked in the original block chain any more, the previous block and the next block of the sediment block are linked, and the hash value of the subsequent block and the hash value of the previous block corresponding to the headers of the two blocks are reset to form a new block chain. The settled block does not participate in the block chain synchronization any more, only the index hash value in the block header is reserved, and the index hash value is temporarily stored in the settling tank. The blocks in the sedimentation pool and the blocks in the block chain are synchronized and backed up independently, the contents of the blocks are required to be restored at the user node, and after the nodes with the appointed number are confirmed, the original block chain position can be restored again according to the index hash value in the block head. If the sediment block is accessed during the block lifetime (i.e., the block can stay within the maximum time period of the sediment pool), the sediment flag can also be automatically canceled and restored to the original block chain location. If the time exceeds the maximum time limit that the block can stay in the sedimentation tank, the data of the block is not synchronized to each node on the block chain, and the block is destroyed in the sedimentation tank, namely the index value of the block is removed from the sedimentation tank index chain, and the block is not synchronized and backed up.

The automatic life cycle management can effectively improve the synchronization and block access efficiency of each node of the block chain, and can greatly relieve the bottleneck caused by data accumulation of invalid blocks. Sometimes, however, the user needs to directly mark some useless blocks for precipitation and destruction, so the invention introduces a node-initiated lifecycle management method.

In the node-initiated lifecycle management of the present invention, any node in the distributed network may initiate a request for precipitation and destruction of useless block data, and N _ apply nodes on other nodes approve the request, so that synchronization can be performed in the whole network. The algorithm can clear block data at the first time when a node user finds a useless block, and avoids the useless block from being synchronized into a block chain again. The number of nodes to be requested N _ apply can be set according to the type of the blockchain, the application environment of the blockchain, and other factors. For example, in the private chain, the number of nodes to be requested N _ apply may be set to a higher value (e.g., 80% of the total number of nodes), and invalid block precipitation and destruction may be performed if most of the nodes approve; if the number of nodes N _ application needed to be requested in the federation chain can be optionally reduced (e.g., 60% of the total number of nodes), it is ensured that the request can obtain a feedback result in a short time.

In summary, the invention reduces the invalid block data synchronization by setting the data lifecycle mechanism of the block, effectively improves the block data synchronization and access speed in the node, further improves the operation efficiency of the block chain, and reduces the storage cost.

The first embodiment of the invention: the re-designed block header data structure of the present invention is shown in fig. 4. On the basis of the data structure of the left classical block header, the invention adds the variables of block access times, block last access time, block access validity period, block lifetime, precipitation mark, precipitation pool index value and the like. The block access times record the total access times of the block since the block is created, the block validity period records the maximum time limit allowed for the block to be accessed, the block lifetime period records the maximum allowed time which is not destroyed in the sedimentation tank after the block is sedimented, the sedimentation mark records whether the block is sedimented, and the sedimentation tank index value records the position of the block in the sedimentation tank.

Example two of the invention: the flow chart of the automatic life cycle management method of the invention is shown in fig. 5. When a certain node is ready for block synchronization, the system inquires the last access time of the block and the access times of the block, and based on whether the block is still in the access validity period or not, if the validity period is exceeded and the access times are less than a set threshold value v _ total, the block is deposited to a deposition pool, the block in the deposition pool is not normally synchronized any more, data is still stored, and the deposition pool is used as a whole for synchronous backup. The deposited block may be requested for block recovery if it needs to be accessed again. If the sediment block is not accessed in the lifetime, the sediment block is destroyed in the sediment pool, the data of the sediment block is not backed up, and recovery cannot be applied. And after the precipitation and destruction verification is carried out, the synchronization of all blocks of the nodes can be carried out.

Example three of the invention: a flow chart of the node-initiated lifecycle management method of the present invention is shown in fig. 6. When the nodes are ready to perform block synchronization, the user can mark invalid blocks according to the requirement, and the system sends block precipitation or destruction requests to other nodes after marking the blocks. If the agreeing node exceeds the number N _ apply of the nodes needing to be requested, block precipitation or destruction can be carried out, and then block synchronization is carried out; if the number of the agreeing nodes is less than the number of the required nodes N _ apply, the block synchronization is directly performed.

Claims

1. A block chain data life cycle management method adds monitoring variables to a block data structure, wherein the variables comprise block access times, block last access time, block access validity period, block lifetime, precipitation marks and precipitation pool index values, and the management method comprises the following steps:

2. The method of claim 1, wherein: in the step (1), the threshold value can be set by a user according to the application background of the block chain, and the user adjusts the precipitation or destruction strategy of the block by adjusting the threshold value of the minimum access times of the block synchronization.

3. The method of claim 2, wherein: when the block data of each node is synchronized, the speed of synchronization is adjusted to be increased, and the threshold is set to be a higher value.

4. The method of claim 1, wherein: in the step (2), the settled block does not participate in the block chain synchronization any more, and only the index hash value in the block header is reserved and temporarily stored in the settling tank.

5. The method of claim 1, wherein: any node in the distributed network can initiate a request for precipitating and destroying useless block data, and N _ application nodes on other nodes can approve to synchronize in the whole network.

6. The method of claim 5, wherein: the number of nodes to be requested N _ apply is set according to the type of the block chain and the application environment factor of the block chain.

7. The method of claim 1, wherein: in step (3), when a certain node is ready to perform block synchronization, the system queries the last access time of the block and the access times of the block, and based on whether the block is still within the access validity period, if the validity period is exceeded and the access times are less than a set threshold v _ total, the block is deposited to a deposition pool, the block in the deposition pool is not synchronized normally, and data is still stored, and the deposition pool is used as a whole for synchronous backup.

8. The method of claim 1, wherein: when the nodes are ready to carry out block synchronization, a user can mark an invalid block according to requirements, the system sends a block settling or destroying request to other nodes after marking the block, if the number of the nodes which are allowed to exceed the number N _ apply of the nodes which need to be requested is increased, the block settling or destroying can be carried out, and then the block synchronization is carried out; if the number of the agreeing nodes is less than the number of the required nodes N _ apply, the block synchronization is directly performed.