CN115422203A - Data management method, device, equipment and medium for block chain distributed system - Google Patents
Data management method, device, equipment and medium for block chain distributed system Download PDFInfo
- Publication number
- CN115422203A CN115422203A CN202211217715.3A CN202211217715A CN115422203A CN 115422203 A CN115422203 A CN 115422203A CN 202211217715 A CN202211217715 A CN 202211217715A CN 115422203 A CN115422203 A CN 115422203A
- Authority
- CN
- China
- Prior art keywords
- block
- data
- compressed
- determining
- block chain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data management method, a device, equipment and a medium for a block chain distributed system. The method is applied to the electronic equipment of the block chain nodes in the block chain distributed system, and comprises the following steps: acquiring data access records of each block in the block chain node; determining at least one block to be compressed from the blocks of the block chain node according to the data access record, wherein the block to be compressed is a first block; and compressing the data in the first block, and migrating the compressed data to a preset historical data storage area. According to the method and the device, the block to be compressed is compressed by determining the block to be compressed in the block chain node, so that the storage space of the block chain node is saved, the data operation performance of the node is greatly improved, and the operation efficiency and the stability of a system are improved.
Description
Technical Field
The present application relates to big data technologies, and in particular, to a data management method, apparatus, device, and medium for a block chain distributed system.
Background
In a distributed system of a block chain structure, there are a plurality of block chain nodes, and data is shared between the block chain nodes.
The system of block-chain architecture, due to the characteristics of decentralization and multinodes, each node stores a complete data copy of all blocks. With the lapse of time, the amount of data stored in each node on the block chain system will increase continuously, and the consumption of storage resources increases exponentially, so that the operational performance of data on the nodes is lower and lower, and the processing efficiency of data is affected.
Disclosure of Invention
The application provides a data management method, a data management device, data management equipment and a data management medium for a block chain distributed system, which are used for saving data storage space.
In a first aspect, the present application provides a data management method for a blockchain distributed system, where the method is applied to an electronic device of a blockchain node in the blockchain distributed system, and the method includes:
acquiring data access records of each block in the block chain node;
determining at least one block to be compressed from the blocks of the block chain node according to the data access record, wherein the block to be compressed is a first block;
and compressing the data in the first block, and migrating the compressed data to a preset historical data storage area.
In a second aspect, the present application provides a data management apparatus for a blockchain distributed system, where the apparatus is applied to an electronic device of a blockchain node in the blockchain distributed system, and the apparatus includes:
an access record obtaining module, configured to obtain a data access record of each block in the block chain node;
a first block determining module, configured to determine, according to the data access record, at least one block to be compressed from the blocks of the block chain node, where the block to be compressed is a first block;
and the first block compression module is used for compressing the data in the first block and transferring the compressed data to a preset historical data storage area.
In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored in the memory to implement the data management method of the blockchain distributed system according to the first aspect of the present application.
In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions for implementing the data management method of the blockchain distributed system according to the first aspect of the present application when the computer-executable instructions are executed by a processor.
In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements a data management method for a blockchain distributed system as described in the first aspect of the present application.
According to the data management method, device, equipment and medium for the block chain distributed system, a plurality of blocks to be compressed are selected from all the blocks of one block chain node by determining the data access records of all the blocks in all the block chain nodes. Compressing data in a block to be compressed, dividing a block chain link point into historical data storage areas in advance, and storing the compressed data in the historical data storage areas. The problem of among the prior art, the block data in each block chain node is constantly growing, occupies memory space is solved. By determining the data access records, the infrequently accessed data are compressed, the block data are managed, the storage resources are greatly saved, the data operation performance of each node is greatly improved, and the system operation efficiency is further improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic flowchart illustrating a data management method of a block chain distributed system according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart illustrating a data management method of a block chain distributed system according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating a relationship between a data access frequency and a time series according to an embodiment of the present application;
fig. 4 is a flowchart illustrating a data management method of a block chain distributed system according to an embodiment of the present disclosure;
fig. 5 is a block diagram illustrating a data management apparatus of a blockchain distributed system according to an embodiment of the present disclosure;
fig. 6 is a block diagram illustrating a data management apparatus of a blockchain distributed system according to an embodiment of the present disclosure;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present disclosure.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the inventive concepts to those skilled in the art by reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims.
In the description of the present application, it is to be understood that the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not necessarily used to describe a particular order or sequence, nor are they to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
It should be noted that, for the sake of brevity, this description does not exhaust all alternative embodiments, and it should be understood by those skilled in the art after reading this description that any combination of features may constitute an alternative embodiment as long as the features are not mutually inconsistent. The following examples are described in detail.
In a system with a blockchain structure, there are a plurality of blockchain nodes, and data between blockchain nodes can be synchronized, that is, each blockchain node stores a data copy of each block. Over time, the data volume of the data copy in each block link point is expanded continuously, which causes the problems of system data processing performance and network performance reduction.
That is, the distributed system of blockchain architecture, due to its decentralized and multi-node nature, each blockchain node stores a complete data copy of all blocks. With the lapse of time, the amount of data stored by each node on the block chain system will increase continuously, which causes exponential increase of the consumption of storage resources, and the data cannot be searched and processed quickly, and at the same time, the operation performance of the data on the node is lower and lower, which affects the processing efficiency of the data. Therefore, data in the blockchain node needs to be managed to save storage resources and improve system operation efficiency.
The application provides a data management method, device, equipment and medium for a block chain distributed system, and aims to solve the technical problems in the prior art.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 1 is a flowchart illustrating a data management method for a blockchain distributed system according to an embodiment of the present application, where the method is applied to an electronic device at a blockchain node in the blockchain distributed system and executed by a data management apparatus of the blockchain distributed system. As shown in fig. 1, the method comprises the steps of:
s101, obtaining data access records of each block in a block chain node.
Illustratively, a blockchain distributed system includes a plurality of blockchain nodes, each of which may represent an electronic device. The user can access and process data through each block link point, for example, the user can perform transactions through equipment at the user end. Each time a user conducts a transaction on one blockchain node, a new block can be generated in the blockchain node, and information sharing is conducted among all blockchain nodes, so that corresponding blocks can be added to other blockchain nodes.
When each block is generated, a unique block code is corresponding to each block, and the generation time of the block can be recorded. The user can access the data of each block in the block chain node at any time, and the block link point can store the data access record for the user to access the data through the block link point in real time, for example, the data access record can store the access time, the access block code, the access address, the access frequency and the like of the user to the data in each block. The access frequency of the data may be used to indicate the number of times that the user accesses the data in the block within a preset time scale, for example, the data access frequency may be the number of times that the user accesses the data in the block from a certain block generation time. For another example, the data access frequency may be the number of times data in a certain block has been accessed within the last 24 hours.
Each block link point may obtain its own stored data access record in real time or at regular time, for example, a data management period is preset, and the data access record may be obtained once a day or once a month.
S102, according to the data access record, determining at least one block to be compressed from the blocks of the block link points, wherein the block is a first block.
For example, after obtaining the data access record, each block chain node may determine one or more blocks to be compressed from each block stored in itself according to the data access record, and use the determined blocks to be compressed as the first block. The block to be compressed refers to a block which is accessed infrequently when a user accesses data through the block link point. The blocks can be compressed to save storage space, so that the process that the block chain nodes are searched from mass data is avoided, and the data processing efficiency of the block chain nodes is improved. For example, the time at which each block is accessed may be determined by the data access record, and the block with an earlier access time may be determined as the block to be compressed. That is, a block that has not been accessed for a preset period of time may be determined as a block to be compressed, for example, a block that has not been accessed for 24 hours may be determined as a block to be compressed.
The first block may also be determined according to the generation time of the block, a historical time interval may be preset, and if the generation time of the block is outside the historical time interval, the block is determined as the first block. For example, the historical time interval is one month before the current time, the block generated one month before is determined as the first block,
in this embodiment, determining at least one block to be compressed from among blocks of block link points according to a data access record includes: and determining the block with the data access frequency less than a preset access frequency threshold value in the blocks of the block chain nodes as the block to be compressed.
Specifically, the data access record may include a data access frequency, an access frequency threshold is preset, and the block link node compares the data access frequency of itself to each block with the access frequency threshold. If the data access frequency of a certain block is less than a preset access frequency threshold, determining the block as a block to be compressed; and if the data access frequency of a certain block is equal to or greater than the access frequency threshold, determining that the block is not the block to be compressed. For example, if the access frequency threshold is 3 times a day, three blocks, namely, a block one, a block two, and a block three, exist in the block chain node, the data access frequency of the block one in a day is 0 times, the data access frequency of the block two in a day is 1 time, and the data access frequency of the block three in a day is 4 times, then the blocks to be compressed may be determined to be the block one and the block two.
The beneficial effect that sets up like this lies in, through numerical value comparison, can confirm the block of treating the compression fast, is convenient for compress the block data of infrequently visiting, realizes managing the data in the block chain node, and avoids causing the influence to the data that the user visited other blocks.
S103, compressing the data in the first block, and migrating the compressed data to a preset historical data storage area.
Illustratively, after determining the first chunk, the data in the first chunk is compressed, generating a compressed packet of data in the first chunk. A compression algorithm may be preset, and in this embodiment, the preset compression algorithm is not specifically limited. The storage space in the block chain node may be divided into blocks in advance as the history data storage area. The history data storage area may be used to store compressed data, and the compressed data may be stored as compressed packets, for example, compressed packets of data in blocks that are not frequently accessed may be stored in the history data storage area, or compressed packets of data of blocks that are generated earlier in time may be stored in the history data storage area. After the compressed packet of the data of the first block is obtained, the compressed packet is migrated to the historical data storage region, so that the node storage resources are greatly saved, and the overall performance of the block chain system is improved.
The storage space in the blockchain node may also be divided into active data storage areas, which may be used to store data in frequently accessed blocks or to store data in more recently generated blocks. Data of other blocks than the first block may also be directly stored in the active data storage area. For example, the blocks with data access frequency exceeding a preset access frequency threshold value may be determined according to the data access record, and the data of the blocks may be stored in the active data storage area. The generation time of each block can also be determined, and the blocks with the generation time within the preset historical time interval are stored in the active data storage area. For example, blocks generated within one month are stored in the active data storage area. When a user accesses data through the block link points, the data in the active data storage area can be searched preferentially, and if the active data storage area does not have data to be accessed by the user, the data can be searched from the historical data storage area. When a user searches for data in the active data storage area and the historical data storage area, the data access records can be updated.
And the generated compressed packet of each block corresponds to a unique block code, a data management period is preset, and data in the historical data storage area and the active data storage area are regularly managed according to the preset data management period. For example, data in the active data storage region may be stored compressed into the historical data storage region, or data in the historical data storage region may be stored decompressed into the active data storage region.
By carrying out regional management on the data in the blockchain nodes, the space utilization rate of the blockchain nodes can be saved by at least 70%, the query and interaction performance of the blockchain nodes can be improved by at least 50%, the problem of occupation of data storage resources is effectively solved, and the access operation performance of an active data storage region is ensured.
In the data management method for the block chain distributed system provided by the embodiment of the application, a plurality of blocks to be compressed are selected from the blocks of one block chain node by determining the data access record of each block in each block chain node. Compressing data in a block to be compressed, dividing a block chain link point into historical data storage areas in advance, and storing the compressed data in the historical data storage areas. The problem of among the prior art, the block data in each block chain node is constantly growing, occupies memory space is solved. By determining the data access records, the infrequently accessed data are compressed, the block data are managed, the storage resources are greatly saved, the data operation performance of each node is greatly improved, and the system operation efficiency and stability are improved.
Fig. 2 is a schematic flowchart of a data management method of a blockchain distributed system according to an embodiment of the present disclosure, which is an alternative embodiment based on the foregoing embodiment.
In this embodiment, the data access record includes a data access frequency; according to the data access record, at least one block to be compressed is determined from the blocks of the block chain nodes, and the method can be refined as follows: sorting the data access frequency of each block in the block chain node to obtain a sorting result of each block; and determining the block at the preset sorting position from the sorting result as the block to be compressed.
As shown in fig. 2, the method comprises the steps of:
s201, obtaining data access records of each block in the block chain node.
For example, this step may refer to step S101 described above, and is not described again.
S202, sorting the data access frequency of each block in the block chain node to obtain a sorting result of each block.
For example, the data access record may include data access frequencies, and the block link node sorts the data access frequencies of its respective blocks to obtain a sorting result of the data access frequencies. For example, a small to large ordering may be performed. Each data access frequency corresponds to one block, so that the sorting result of the blocks is obtained.
S203, determining a block located at the preset sorting position from the sorting result as a block to be compressed, where the block to be compressed is a first block.
Illustratively, the sorting position of the block to be compressed is preset, and the block located at the preset sorting position is searched from the sorting result to be used as the block to be compressed, that is, the first block is obtained. For example, if the sorting order is from small to large, and the preset sorting position can be the first 2000, the first 2000 blocks with the least data access frequency are used as the blocks to be compressed. The preset sorting position may also be a ratio that the block to be compressed occupies all blocks in the block chain node, for example, if the preset ratio is 20%, the block of the top 20% in the sorting is used as the block to be compressed, so that the block data in the historical data storage area is always kept at 20%, and the performance of data processing is greatly improved.
In this embodiment, the data access frequency and the time sequence have a positive correlation data distribution characteristic, and the time sequence may refer to the sequence of the block generation time. The positive correlation data distribution characteristic means that the earlier generated block has the lower data access frequency. Fig. 3 is a schematic diagram illustrating a relationship between a data access frequency and a time series in this embodiment. In fig. 3, the horizontal axis represents the creation time of a block, the creation time may be a date, and the vertical axis represents the data access frequency of each block within a certain time. As can be seen from fig. 3, the later the block is generated, the more times it is accessed. Therefore, when determining the first tile, the first tile may be determined by the generation time of each tile. For example, the generation time of each block may be sorted in time series, and the block located at the preset sorting position may be determined as the first block to be compressed. The preset sorting position may be a plurality of blocks generated earliest, for example, the generation time is sorted from morning to evening, and the preset sorting position may be the first 2000 blocks or the first 20% blocks in the sorting.
In this embodiment, determining the block located at the preset sorting position from the sorting result as the block to be compressed includes: determining a block positioned at a preset sorting position in the sorting result as a candidate compressed block; and determining the generation time of the candidate compressed block, and determining the candidate compressed block with the generation time in a preset time period as the block to be compressed.
Specifically, although the data access frequency and the time series are normally characterized by a positive correlation data distribution, the case where the data access frequency of a block whose generation time is earlier is high or the data access frequency of a block whose generation time is later is low is not excluded. Therefore, the magnitude of the data access frequency can be sorted first, and the block located at the preset position is selected as the candidate compressed block. There may be newly generated blocks among the candidate compressed blocks, and the newly generated blocks may be temporarily accessed less frequently for practical reasons. Thus, a final block to be compressed may be determined from the candidate compressed blocks.
And determining the generation time of each candidate compressed block, and presetting a time period which is a time period of historical time. For example, the current time is 2022, 9, 15 days, and the predetermined time period may be a time range from 2022, 8, 15 days. It is determined whether the generation time of the candidate compressed block is within a preset time period, for example, whether the generation time of the candidate compressed block is one month before. If yes, determining that the candidate compressed block is a block to be compressed, and if not, determining that the candidate compressed block is a newly generated block and is not used as the block to be compressed.
If the candidate compressed blocks are screened out through the block generation time, the data access frequency of each candidate compressed block can be determined, and the candidate compressed blocks with the data access frequency lower than the preset access frequency threshold are determined as blocks to be compressed, so that the blocks with early generation time and more access times are reserved.
The method has the advantages that the block with early generation time and low access frequency is taken as the block to be compressed by combining the data access frequency and the block generation time, and the newly generated block or the frequently accessed block is prevented from being placed in the historical data storage area. The method and the device realize quick access and processing of active data and improve the access operation performance of the data.
And S204, compressing the data in the first block, and migrating the compressed data to a preset historical data storage area.
For example, this step may refer to step S103 described above, and is not described again.
In the data management method for the block chain distributed system provided by the embodiment of the application, a plurality of blocks to be compressed are selected from the blocks of one block chain node by determining the data access record of each block in each block chain node. Compressing data in a block to be compressed, dividing a block chain link point into historical data storage areas in advance, and storing the compressed data in the historical data storage areas. The problem of among the prior art, the block data in each block chain node continuously increases, occupy storage space is solved. By determining the data access records, the infrequently accessed data are compressed, the block data are managed, the storage resources are greatly saved, the data operation performance of each node is greatly improved, and the system operation efficiency and stability are improved.
Fig. 4 is a schematic flowchart of a data management method of a blockchain distributed system according to an embodiment of the present disclosure, which is an alternative embodiment based on the foregoing embodiment.
In this embodiment, the data in the first block is compressed, and may be subdivided into: determining a block code for the first block; sending the block code of the first block to each block chain node except the first block in a block chain distributed system, and receiving the block code of the second block sent by each block chain node except the first block; the second block is used for representing a block to be compressed determined by each block chain node except the second block; and determining a target block according to the block coding of the first block and the block coding of the second block, and compressing the data in the target block.
As shown in fig. 4, the method comprises the steps of:
s401, obtaining data access records of each block in the block chain node.
For example, this step may refer to step S101 described above, and is not described again.
S402, according to the data access record, determining at least one block to be compressed from the blocks of the block link points as a first block.
For example, this step may refer to step S102 described above, and is not described again.
S403, determining the block code of the first block.
Each block is generated, for example, by corresponding to a unique block code, and each block link point stores the block code of each block. After the first block is determined, the block coding of the first block is determined.
S404, sending the block code of the first block to a block chain distributed system, wherein each block chain node except the first block chain node receives the block code of the second block sent by each block chain node except the first block chain node; the second block is used for representing the block to be compressed determined by each block chain node except the second block.
Illustratively, each block link point is in a distributed system, and each block link point can access and process data stored in the block link point. The user can access the block data in each block chain node through different devices, and therefore, data access records in different block chain nodes can be different.
Each blockchain node may determine its own block to be compressed, i.e., the first block, and determine the block coding of the first block. When a blockchain node determines its first block, the block code of the first block may be sent to other blockchain nodes in the system. That is, each blockchain node may receive other blockchain node transmissions for block coding than itself.
And one block link point sends the determined block code of the first block to other block link nodes except the block link point, and receives the block codes of the second blocks sent by other block link points. The second block is the first block determined by the link points of other blocks. That is, the block codes of the respectively determined first blocks may be shared among the respective block chain nodes.
S405, determining a target block according to the block code of the first block and the block code of the second block, compressing data in the target block, and transferring the compressed data to a preset historical data storage area.
Illustratively, after obtaining the block coding of the second block sent by the link points of other blocks, the target block to be finally compressed is determined according to the block coding of the first block and the block coding of the second block. For example, a union of the block codes of the first block and the second block may be determined, and the block corresponding to the block code in the union may be used as the target block, so that the target nodes determined by the respective block chain nodes are the same. And compressing the target block according to a preset compression algorithm to obtain a compressed packet of the data in the target block. And storing the compressed packet of the data into a preset historical data storage area, and storing the uncompressed block data into a preset active data storage area. The data in each block can be preferentially stored in an active data storage area, the data which are accessed infrequently are searched from the active data storage area, the compressed packets of the data which are accessed infrequently are migrated to a historical data storage area to be stored, and the data which are accessed frequently are still stored in the active data storage area.
In this embodiment, determining the target block according to the block coding of the first block and the block coding of the second block includes: determining an intersection of block codes in the block code of the first block and the block code of the second block; and determining the block corresponding to the block code in the intersection as a target block.
In particular, combining the block code of the first block and the block code of the second block may be determining an intersection of the block codes of the first block and the second block. And determining the block corresponding to the block code in the intersection as the target block. For example, the blockchain distributed system includes three blockchain nodes, which are node one, node two, and node three, respectively. The block coding of the first block determined by node one is 001, 002 and 003, the block coding of the first block determined by node two is 002, 003 and 004, and the block coding of the first block determined by node three is 001, 003 and 004. Taking node one as an example, the second block received by node one is 002, 003, and 004 transmitted by node two, and 001, 003, and 004 transmitted by node three. The node one, the node two and the node three can respectively determine that the intersection of the block codes is 003, that is, the block corresponding to 003 is the target block.
The method has the advantages that the cross validation of the block codes is realized through the sharing of the block codes among the block chain nodes, the compressed block data are ensured to be the blocks which are not frequently accessed by the block chain nodes, the active data are prevented from being compressed, the access operation performance of the active data is ensured, and the data processing efficiency is improved.
In the data management method for the block chain distributed system provided by the embodiment of the application, a plurality of blocks to be compressed are selected from the blocks of one block chain node by determining the data access record of each block in each block chain node. Compressing data in a block to be compressed, dividing a block chain link point into historical data storage areas in advance, and storing the compressed data in the historical data storage areas. The problem of among the prior art, the block data in each block chain node is constantly growing, occupies memory space is solved. By determining the data access records, the infrequently accessed data are compressed, the block data are managed, the storage resources are greatly saved, the data operation performance of each node is greatly improved, and the system operation efficiency and stability are improved.
Fig. 5 is a block diagram of a data management apparatus of a blockchain distributed system according to an embodiment of the present disclosure. For ease of illustration, only portions that are relevant to embodiments of the present disclosure are shown. Referring to fig. 5, the apparatus includes: a record acquisition module 501, a first block determination module 502, and a first block compression module 503 are accessed.
An access record obtaining module 501, configured to obtain a data access record of each block in the block chain node;
a first block determining module 502, configured to determine, according to the data access record, at least one block to be compressed from the blocks of the block chain node, as a first block;
the first block compressing module 503 is configured to compress the data in the first block, and migrate the compressed data to a preset historical data storage area.
Fig. 6 is a block diagram of a structure of a data management apparatus of a blockchain distributed system according to an embodiment of the present application, where a data access record includes a data access frequency, and based on the embodiment shown in fig. 5, as shown in fig. 6, a first block determining module 502 includes a block sorting unit 5021 and a block determining unit 5022.
A block sorting unit 5021, configured to sort the data access frequency of each block in the block chain node to obtain a sorting result of each block;
the block determining unit 5022 is configured to determine a block located at a preset sorting position from the sorting result as a block to be compressed.
In one example, the first block determining module 502 further includes:
and the frequency comparison unit is used for determining the block with the data access frequency smaller than a preset access frequency threshold value in the blocks of the block chain node as the block to be compressed.
In one example, the block determination unit 5022 is specifically configured to:
determining a block located at a preset sorting position in the sorting result as a candidate compressed block;
and determining the generation time of the candidate compressed block, and determining the candidate compressed block with the generation time in a preset time period as a block to be compressed.
In one example, the first block compression module 503 includes:
an encoding determination unit configured to determine a block encoding of the first block;
the code receiving unit is used for sending the block codes of the first block to each block chain node except the block chain node in the block chain distributed system and receiving the block codes of the second block sent by each block chain node except the block chain node; the second block is used for representing a block to be compressed determined by each block chain node except the second block;
and the target block determining unit is used for determining a target block according to the block coding of the first block and the block coding of the second block and compressing the data in the target block.
In an example, the target block determining unit is specifically configured to:
determining an intersection of block codes in the block code of the first block and the block code of the second block;
and determining a block corresponding to the block code in the intersection as the target block.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 7, the electronic device includes: a memory 71, a processor 72; a memory 71 for storing instructions executable by the processor 72.
Wherein the processor 72 is configured to perform the methods provided in the above embodiments.
The electronic device further comprises a receiver 73 and a transmitter 74. The receiver 73 is used for receiving instructions and data transmitted from other devices, and the transmitter 74 is used for transmitting instructions and data to external devices.
FIG. 8 is a block diagram illustrating an electronic device, which may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like, in accordance with an exemplary embodiment.
The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operation at the device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
A power supply component 806 provides power to the various components of the device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.
The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front-facing camera and/or the rear-facing camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 can detect the open/closed state of the device 800, the relative positioning of components, such as a display and keypad of the device 800, the sensor assembly 814 can also detect a change in position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, orientation or acceleration/deceleration of the device 800, and a change in temperature of the device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
In an exemplary embodiment, the device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
A non-transitory computer readable storage medium, instructions in which, when executed by a processor of a terminal device, enable the terminal device to perform a data management method of a blockchain distributed system of the terminal device.
The application also discloses a computer program product comprising a computer program which, when executed by a processor, implements the method as described in the embodiments.
Various implementations of the systems and techniques described here above may be realized in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present application may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or electronic device.
In the context of this application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as data electronics), or that includes a middleware component (e.g., application electronics), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.
The computer system may include a client and an electronic device. The client and the electronic device are generally remote from each other and typically interact through a communication network. The relationship of client and electronic device arises by virtue of computer programs running on the respective computers and having a client-electronic device relationship to each other. The electronic device may be a cloud electronic device, which is also called a cloud computing electronic device or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and low service extensibility in a conventional physical host and VPS service ("Virtual Private Server", or "VPS" for short). The electronic device may also be a distributed system of electronic devices or an electronic device incorporating a blockchain. It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present invention is not limited herein.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (10)
1. A data management method for a block chain distributed system is applied to an electronic device of a block chain node in the block chain distributed system, and the method comprises the following steps:
acquiring data access records of each block in the block chain node;
determining at least one block to be compressed from the blocks of the block chain node according to the data access record, wherein the block to be compressed is a first block;
and compressing the data in the first block, and migrating the compressed data to a preset historical data storage area.
2. The method of claim 1, wherein the data access record includes a data access frequency;
determining at least one block to be compressed from the blocks of the block chain node according to the data access record, including:
sorting the data access frequency of each block in the block chain node to obtain a sorting result of each block;
and determining the block at the preset sorting position from the sorting result as the block to be compressed.
3. The method of claim 2, wherein determining at least one block to be compressed from the blocks of the blockchain node according to the data access record comprises:
and determining the block with the data access frequency smaller than a preset access frequency threshold value in the blocks of the block chain nodes as the block to be compressed.
4. The method of claim 2, wherein determining the block located at the predetermined sorting position from the sorting result as the block to be compressed comprises:
determining a block located at a preset sorting position in the sorting result as a candidate compressed block;
and determining the generation time of the candidate compressed block, and determining the candidate compressed block with the generation time in a preset time period as a block to be compressed.
5. The method of claim 1, wherein compressing the data in the first block comprises:
determining a block code for the first block;
sending the block code of the first block to each block chain node except the first block in the block chain distributed system, and receiving the block code of the second block sent by each block chain node except the first block; the second block is used for representing a block to be compressed determined by each block chain node except the second block;
and determining a target block according to the block code of the first block and the block code of the second block, and compressing data in the target block.
6. The method of claim 5, wherein determining a target block based on the block coding of the first block and the block coding of the second block comprises:
determining an intersection of block codes in the block code of the first block and the block code of the second block;
and determining a block corresponding to the block code in the intersection as the target block.
7. A data management apparatus for a blockchain distributed system, the apparatus being applied to an electronic device of a blockchain node in the blockchain distributed system, the apparatus comprising:
an access record obtaining module, configured to obtain a data access record of each block in the block chain node;
a first block determining module, configured to determine, according to the data access record, at least one block to be compressed from the blocks of the block chain node, where the block to be compressed is a first block;
and the first block compression module is used for compressing the data in the first block and transferring the compressed data to a preset historical data storage area.
8. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes the computer-executable instructions stored by the memory to implement the data management method of the blockchain distributed system according to any one of claims 1 to 6.
9. A computer-readable storage medium having stored thereon computer-executable instructions for implementing a data management method of a blockchain distributed system according to any one of claims 1 to 6 when executed by a processor.
10. A computer program product comprising a computer program which, when executed by a processor, implements a method of data management for a blockchain distributed system according to any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211217715.3A CN115422203A (en) | 2022-09-30 | 2022-09-30 | Data management method, device, equipment and medium for block chain distributed system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211217715.3A CN115422203A (en) | 2022-09-30 | 2022-09-30 | Data management method, device, equipment and medium for block chain distributed system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115422203A true CN115422203A (en) | 2022-12-02 |
Family
ID=84205873
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211217715.3A Pending CN115422203A (en) | 2022-09-30 | 2022-09-30 | Data management method, device, equipment and medium for block chain distributed system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115422203A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116662453A (en) * | 2023-08-01 | 2023-08-29 | 天津理工大学 | Digital asset storage method based on block chain |
-
2022
- 2022-09-30 CN CN202211217715.3A patent/CN115422203A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116662453A (en) * | 2023-08-01 | 2023-08-29 | 天津理工大学 | Digital asset storage method based on block chain |
CN116662453B (en) * | 2023-08-01 | 2023-11-03 | 天津理工大学 | Digital asset storage method based on block chain |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106371326B (en) | Storage method and device for equipment working scene | |
CN115422203A (en) | Data management method, device, equipment and medium for block chain distributed system | |
CN111046040A (en) | Method and device for determining index, electronic equipment and storage medium | |
US11797218B2 (en) | Method and device for detecting slow node and computer-readable storage medium | |
US20210133505A1 (en) | Method, device, and storage medium for retrieving samples | |
WO2019000468A1 (en) | User location identification method and apparatus, storage medium, and electronic device | |
CN116303303A (en) | Batch data processing method, device, equipment and medium | |
CN115834483A (en) | Flow control method, device and equipment based on cluster and storage medium | |
CN114281859A (en) | Data processing method, device and storage medium | |
CN115017406A (en) | Live broadcast picture display method and device, electronic equipment and storage medium | |
CN111144845B (en) | Mobile terminal meeting reminding method and device | |
CN114428589A (en) | Data processing method and device, electronic equipment and storage medium | |
CN113626396B (en) | Cluster detection method and device, electronic equipment, storage medium and product | |
CN113422800B (en) | Resource processing method, device, system, electronic equipment and storage medium | |
CN113689520B (en) | Graph data processing method and device, electronic equipment and storage medium | |
CN110019358B (en) | Data processing method, device and equipment and storage medium | |
CN111625536B (en) | Data access method and device | |
CN110716985B (en) | Node information processing method, device and medium | |
CN115113915A (en) | Data processing method and device, electronic equipment and storage medium | |
CN112306667B (en) | Animation playing method and device, computer equipment and storage medium | |
CN116644057A (en) | Data migration method and device based on unitized architecture and electronic equipment | |
CN117032920A (en) | Batch job priority determining method, device, equipment and storage medium | |
CN116700974A (en) | Multi-process task processing method and device, electronic equipment and storage medium | |
CN116610466A (en) | Message processing method, device, electronic equipment and storage medium | |
CN115437887A (en) | Alarm text information processing method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |