CN117472959A - Gskip list-based block chain efficient query system and dynamic construction method - Google Patents

Gskip list-based block chain efficient query system and dynamic construction method Download PDF

Info

Publication number
CN117472959A
CN117472959A CN202311447531.0A CN202311447531A CN117472959A CN 117472959 A CN117472959 A CN 117472959A CN 202311447531 A CN202311447531 A CN 202311447531A CN 117472959 A CN117472959 A CN 117472959A
Authority
CN
China
Prior art keywords
node
metad
key
subsystem
level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311447531.0A
Other languages
Chinese (zh)
Inventor
李旭东
李泽麟
任文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nankai University
Original Assignee
Nankai University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nankai University filed Critical Nankai University
Priority to CN202311447531.0A priority Critical patent/CN117472959A/en
Publication of CN117472959A publication Critical patent/CN117472959A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • G06F16/2246Trees, e.g. B+trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24564Applying rules; Deductive queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a Gskip list-based block chain efficient query system and a dynamic construction method, and belongs to the field of computers. The blockchain efficient query system based on Gskip list comprises a blockchain subsystem, a group jump table subsystem and a database, wherein the blockchain subsystem comprises all transactions and blocks, the group jump table subsystem comprises a multi-level Node tree, a plurality of MetaD Node sets and a parameter file, and the database is used for storing all data permanently. The dynamic construction method of the block chain efficient query system based on the Gskip list comprises the following steps: reading a parameter file; reading block chain data and constructing an initial preset 'group jump table' subsystem; externally starting inquiry service, and simultaneously monitoring a new block and a user inquiry request; responding to the inquiry request of the user end and dynamically maintaining the structure of the group jump table subsystem. The invention can effectively accelerate the block chain inquiry efficiency.

Description

Gskip list-based block chain efficient query system and dynamic construction method
Technical Field
The invention belongs to the field of computers, and mainly aims to improve the search efficiency of block chain inquiry.
Background
The blockchain is a decentralized and non-tamperable distributed database, and has wide prospects in finance, internet of things, public service, notarization, digital rights and traceability. As the number of users and data stored in a blockchain increases, the throughput requirements of blockchain systems continue to increase, but the following problems exist:
1. with the rapid growth of the data volume of the blockchain, the structure and algorithm of the existing blockchain system are difficult to support the data efficient query operation in a large-scale data scene. Under the existing block chain structure and algorithm, one query involves accessing a plurality of disk files at the bottom layer, so that a large number of disk IO are generated, and the performance of query operation is greatly influenced;
2. with the rapid growth of blockchain data volume, existing blockchain systems have difficulty in efficiently implementing complex queries, such as Top-k queries, range queries, equivalence queries, and the like. However, there is a great deal of real life demand for blockchain complex queries. For example, a blockchain investor wants to know the data of the last quarter transaction total, the maximum transaction total, the total transaction total, the average transaction amount and the like, and hopes to estimate the current running state of the blockchain system according to the data and judge whether the blockchain system is suitable for buying and selling the investment, so that the user hopes to provide an interface for querying the ethernet data in a certain range by the blockchain system, but the current blockchain system can only search the range data one by one from the underlying database such as a leverldb, and the query efficiency is low and the computing resource is wasted.
In order to solve the above problems, a main solution is to store blockchain data by constructing an external database, for example, etherQL synchronizes and parses the blockchain data from a blockchain system, and the process of parsing the blockchain data redesigns the storage format of the data and stores the parsing result in a mongo db. MongoDB is an efficient and extensible NoSQL database, and the efficiency of block chain data query in MongoDB can be obviously improved, and various complex queries for data are supported. Although the above two problems can be solved. But there are the following limitations to using an external database: 1) The non-tamper property and the decentralization property of the data cannot be guaranteed, and meanwhile, the additional cost of the system is increased; 2) The query efficiency is determined by an external database, and the problems of single-point failure, data loss, data tampering and the like can occur, so that huge security holes exist in out-of-chain storage.
Other typical schemes are as follows:
document 1: zheng Haohan, shen Derong, nie Tiezheng, etc. Inquiry optimization of blockchain systems for hybrid indexing [ J ]. Computer science, 2020,47 (10): 301-308.
A new solution is proposed herein: firstly, dividing the blockchain data into different attributes; then according to different data attributes, combining Merkle tree and various index structures of the blockchain, a new index-MHerkle tree is provided, namely all transactions in each block are ordered according to time or gas value when each block is generated, and the ordered and reconstructed corresponding Merkel prefix tree is stored in a block body, so that the transactions in the block can be ensured to be ordered according to a certain sequence, and a bloom filter is added for rapidly screening whether the transaction is in the block body of the region, and the structure enhances the query performance of the blockchain under the condition that the block chain is fully ensured to be non-tamper-modified; and finally, designing an index construction algorithm of the MHerkle tree, and providing a query algorithm and a range query algorithm based on different attributes according to indexes. The proposal has the advantages that local range searching and top-k searching can be realized, and the bloom filter is used to improve the query efficiency; the disadvantage is that global range search and top-k search cannot be performed, the bloom filter also has the problem of false positive, and the efficiency improvement is limited.
Document 2: zhang Hong, wei Zhongqi A blockchain system query model based on a two-level indexing mechanism [ J ]. Computer application, 2022,42 (S2): 129-134.
A query model based on account public keys and smart contracts is presented herein. The provided model enables a user to quickly and accurately search transaction information through account information by establishing a secondary index on hash values of the block and the transaction; the proposed model joins the established secondary index as a special node to the blockchain network and establishes access control mechanisms through intelligent contracts so that it can be safely maintained and shared by all users together. The scheme has the advantages that a new method is provided for rapidly acquiring historical transaction data of a user of the blockchain, and the decentralization characteristic of the index can be ensured; the disadvantage is that simply linking all of the user's transaction data together does not satisfy advanced queries on the user's historical transactions, such as the tok-k query and the range query in terms of time or gas values.
Disclosure of Invention
The invention aims to improve the search efficiency of block chain inquiry. The invention provides a Gskip list-based block chain efficient query system and a dynamic construction method.
Gskip list is a group skip list, namely, a group skip list, abbreviated as Gskip list, provided by the invention. The group skip list is a new skip list supporting a group of key value range, and its structure can be dynamically adjusted according to the ' user's inquiry requirement '. Each Node of the group jump table, namely Node, contains a value not of a Key, but a range containing a Key, namely KeyLeft, keyRight, wherein KeyLeft is a left Key and KeyRight is a right Key; thus, each "Node" will point to a "MetaD Node set" of "MetaD nodes" and each "MetaD Node" will point to a transaction ID, transactionID, and a block ID, blockID, in the blockchain; and the multi-stage linked Node tree composed of Node nodes can dynamically upgrade or downgrade a Node according to the user end inquiry requirement.
The invention establishes the 'group jump table' instance of the appointed 'query field set' on the existing platform of the blockchain, namely the blockchain and the database by adopting the 'group jump table' Gskip list technology, initializes the 'group jump table' instance by initial preset, and further dynamically adjusts the structure of the 'group jump table' instance by 'user side query requirement', thereby providing high-efficiency complex query service for the blockchain system.
The technical scheme of the invention is as follows:
a high-efficiency block chain query system based on Gskip list comprises a block chain subsystem, a group skip list subsystem and a database; the block chain subsystem comprises all 'transaction' and 'blocks', each 'transaction' at least comprises transaction ID, namely transaction ID, transaction time, transaction participators, transaction head hash and transaction content information, each 'block' comprises transactions within a given time, the 'blocks' are connected together in a chained mode according to the time generated by the block, each 'block' at least comprises block ID, namely block ID, block number, block start-stop time, previous block ID, merck tree root of a contained 'transaction' set, block head hash and contained 'transaction' set information, and all 'transaction' and 'block' data in the block chain subsystem are stored in the database; the group jump table subsystem is called Gskip list subsystem for short, which comprises a multi-stage Node tree composed of Node nodes, the Gskip list subsystem also comprises a plurality of MetaD Node sets composed of MetaD nodes, each Node at least comprises Node ID, left Key, right Key, key right, first MetaD Node ID, metaDID, search times, SCount, former Node pointer array Prev, latter Node pointer array Next information, each Node comprises a corresponding MetaD Node set in the range which is larger than or equal to the left Key and smaller than the right Key, namely [ KeyLeft, keyRight ], each Node also constructs a plurality of levels of chain association with other Node nodes through a pointer array Prev and a pointer array Next, each MetaD Node at least comprises an ID of the MetaD Node, namely MetaDID, a Key, namely Key, a NodeID of the corresponding Node, a transaction ID in the corresponding blockchain, namely TransactionID, a block ID in the corresponding blockchain, a MetaDID of the former MetaD Node, namely Prev, a MetaDID of the latter MetaD Node, namely Next, the Gskip list subsystem also comprises a Gskip list parameter file for defining a multi-level Node tree, wherein the Gskip list parameter file at least comprises an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searches, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node upgrading threshold U, a Node upgrading threshold value, a Node playing threshold value, a Node S threshold value, a maximum memory threshold value and a memory threshold value, all the data of Node and MetaD Node in Gskip list subsystem are stored in the database; the database is used for storing transaction data and block data of the block chain subsystem, and the database is also used for storing Node data and MetaD Node data of the Gskip list subsystem.
The invention also provides a dynamic construction method of the high-efficiency block chain query system based on Gskip list, which comprises the following specific steps:
step 1, starting a system, and setting a Gskip list parameter file to be read;
step 2, reading a Gskip list parameter file to obtain an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searchfields, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node tree, a Node upgrading threshold U, a Node degrading threshold D, a MetaD Node fissioning threshold S, a MetaD Node merging threshold M, a maximum occupied memory space value G and persistent storage decision threshold P information;
step 3, reading block chain data and constructing an initial preset 'group jump table' subsystem, wherein the specific steps are as follows:
step 3.1, reading blockchain data to construct a level 0 Node tree and a plurality of MetaD Node sets according to the search field set of Gskip list in the step 2, namely GSL_Searchfields;
step 3.2, constructing a Node tree which is originally preset from the 1 st level to the Q level on the basis of the Node tree of the 0 th level according to the initial preset level Q of the Node tree in the step 2;
Step 4, the system starts to provide the block chain inquiry service to the outside;
step 5, monitoring new blocks of the block chain subsystem, and simultaneously monitoring and receiving a query request of a user;
step 6, judging the monitoring response type, if the monitoring response type is 'new block generation of the block chain subsystem', executing step 17; if the query request is the query request of the user side, executing the step 7;
step 7, analyzing the inquiry request of the user side;
step 8, converting the query request in the step 7 into a key value range [ Vsb, vse ] to be queried;
step 9, gradually searching from the "highest level" of the gskipllist subsystem to the 0 th level, and searching all "Node nodes" corresponding to the key value range [ Vsb, vse ] given in the 8 th step, wherein the "highest level" of the gskipllist subsystem is K, but the "highest level" is dynamically changed along with the structural change of the gskipllist subsystem; the [ KeyLeft, keyRight) of each "Node" retrieved needs to have a non-empty intersection with the key value range [ Vsb, vse ] given in step 8;
step 10, further searching all "MetaD nodes" corresponding to the Key value range [ Vsb, vse ] given in step 8, in all "Node nodes" searched in step 9, that is, the Key, i.e., the Key, of the searched "MetaD Node" needs to be in the Key value range [ Vsb, vse) given in step 8;
Step 11, all the "MetaD nodes" retrieved in step 10 further find relevant "blockchain data" in the blockchain according to transaction ID, i.e. transactionID, blockID, information in each "MetaD node";
step 12, further calculating a final query result of the query request of the user side according to all the block chain data searched in the step 11;
step 13, adding 1 to the search times, i.e. SCount, of each Node aiming at all Node nodes searched in the step 9;
step 14, dynamically adjusting the structure of the Gskiplist subsystem in a synchronous or asynchronous mode, wherein the specific steps are as follows:
step 14.1, performing 'dynamic structure adjustment' on the Gskip list subsystem in a synchronous or asynchronous mode, namely dynamically adjusting the upgrading or degradation of 'Node' and dynamically adjusting the splitting or merging of 'MetaD Node set';
step 14.2, adopting a synchronous or asynchronous mode to make a persistent storage decision and implement on the Gskip list subsystem, and storing all memory data of the Gskip list subsystem into a database;
step 15, returning the final query result calculated in the step 12 to the user side;
Step 16, judging whether the system continues to operate, if so, jumping to the step 5 to continue to operate; if the system does not continue to run, jumping to step 18;
step 17, reading a new block of the blockchain, and carrying out content updating on the Gskip list subsystem, namely updating a plurality of MetaD Node sets and Node trees according to transaction information in the new block; further jumping to the step 5 to continue execution;
step 18, performing persistent storage implementation on the Gskip list subsystem in a synchronous mode, and storing all memory data of the Gskip list subsystem into a database;
and step 19, ending the system execution.
Further, according to the access method provided by the present invention, the step 3.1 of constructing a 0 th level "Node tree" and a plurality of "MetaD Node sets" for the gskipllist subsystem is to sequentially read the blockchain block and the transaction data, and the step 2 of constructing a 0 th level "Node tree" and a plurality of "MetaD Node sets" according to the search field set of the gsl_security fields in the step 2 may be implemented by the following steps:
step 3.1.1, sequentially reading block data from the 0 th block to the 'maximum ID block' of the blockchain, wherein each read block is abbreviated as Bi, wherein 0< = i < 'maximum ID block', and performing the following treatment on each block Bi;
Step 3.1.2, traversing each transaction in the block Bi, wherein each transaction is abbreviated as Tj, wherein the maximum transaction number of the block Bi is 0< = j < ", and processing the transaction Tj as follows;
step 3.1.3, calculating a key and a value (value) for the transaction Tj based on gsl_security fields, which is the search field set of Gskiplist in step 2, and recording the result as (Kj, vj);
step 3.1.4, creating a MetaD node MDj for the transaction Tj, wherein the Key of MetaD node MDj has a value of Kj, the transaction ID of TransactionID of MetaD node MDj is Tj, and the block ID of MetaD node MDj is the block ID of Tj;
step 3.1.5, in the Gskip list subsystem, sequentially searching a 0 th level Node tree, and searching a Node Nm with a key value range containing Kj, wherein the key value range of the Node Nm is marked as [ x, y ], wherein x < = y; the key value range of Nm when x is equal to y is represented only as x; initially, the 0 th level Node tree has only one Node, and the key value range is [ - ++infinity, ++infinity);
step 3.1.6, inserting the MetaD Node MDj information in step 3.1.4 into the "MetaD Node set" of Node Nm in step 3.1.5;
step 3.1.7, calculating the number of MetaD nodes owned by the Node Nm in the step 3.1.5, if the number of MetaD nodes owned by the Node Nm reaches the MetaD Node fission threshold S read in the step 2, calculating a t value so that the Node Nm can be split into two new Node nodes by fission, wherein the key value ranges of the two new Node nodes are [ x, t ] and [ t, y ] respectively, and the principle of selecting the t value is to make the two Node nodes approximately have the same number of MetaD nodes as much as possible;
Step 3.1.8, judging whether all transactions of the block Bi are traversed, if all transactions of the block Bi are traversed, jumping to the 3.1.9 step to continue execution; otherwise, jumping to the 3.1.2 step to continue to process the next transaction;
step 3.1.9, judging whether all blocks of the blockchain are traversed, if all blocks of the blockchain are traversed, jumping to 3.1.10 to continue execution; otherwise, jumping to the 3.1.1 step to continue processing the next block;
step 3.1.10, the program process ends.
Further, according to the access method provided by the present invention, the step 3.2 of constructing the "Node tree" of the initial preset of the 1 st to the K th levels for the gskipllist subsystem is to upgrade a part of the "Node nodes" of the 0 th level "Node tree" according to the initial preset level Q of the "Node tree" and the initial preset rule type R of the "Node tree" in the "gskipllist parameter file", so as to build the "Node tree" of the 1 st to the K th levels step by step; the initial preset rule type R of the "Node tree" may include the following rule types: a bipartite rule, a trisection rule, an N-score rule and a custom rule; the bipartite rule refers to that in the Node tree of the same level, one Node in every two continuous Node nodes performs upgrading operation; the trisection rule refers to that in the Node tree of the same level, one Node in every three continuous Node nodes performs upgrading operation; n division rules refer to that in the Node tree of the same level, one Node in each continuous N Node nodes performs upgrading operation; the custom rule refers to that in the Node tree of the same level, a plurality of Node nodes are selected according to the custom rule of a designer to carry out upgrading operation.
Furthermore, according to the access method provided by the invention, the step 14.1 of upgrading the dynamic adjustment Node for performing the dynamic structure adjustment on the Gskip list subsystem, according to the searching times of each Node, namely whether SCount meets the condition of a Node upgrading threshold U in a Gskip list parameter file, if yes, the Node is upgraded in a Node tree by one stage; the step-up method of the Node comprises the following steps:
step 14.1.U.1, obtaining the highest level of the Node to be upgraded and marking the highest level as g, and upgrading the Node to be upgraded into g+1 level;
step 14.1.U.2, in the g level of "Node tree", obtaining the left side link pointer array Prev [ g ] of "Node" to be upgraded to obtain the left side adjacent "Node"; if the left adjacent Node does not have g+1 level, continuing to acquire the left adjacent Node of the left adjacent Node until acquiring a left adjacent Node with g+1 level;
step 14.1.U.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be upgraded to acquire a right side adjacent Node; if the right adjacent Node does not have g+1 level, continuing to acquire the right adjacent Node of the right adjacent Node until acquiring a right adjacent Node with g+1 level;
Step 14.1.U.4, setting the right side link pointer array Next [ g+1] of the left side adjacent Node obtained in the step 14.1.U.2 as the Node to be upgraded; setting the left side link pointer array Prev [ g+1] of the right side adjacent Node obtained in the 14.1.U.3 step as the Node to be upgraded;
step 14.1.U.5, further setting the left side link pointer array Prev [ g+1] of the "Node" to be upgraded as the left side adjacent "Node" obtained in the 14.1.U.2 step, and setting the right side link pointer array Next [ g+1] of the "Node" to be upgraded as the right side adjacent "Node" obtained in the 14.1.U.3 step;
and step 14.1.U.6, finishing upgrading the Node to be upgraded into g+1 level, and finishing the upgrading treatment.
Further, according to the access method provided by the present invention, the degradation of the "Node" of the "dynamic structure adjustment" performed on the gskipllist subsystem in step 14.1 is based on the "retrieval times of each" Node ", that is, whether the SCount" meets the "Node" degradation threshold D condition in the "gskipllist parameter file", and if the degradation condition is met, the "Node" is reduced by one step in the "Node tree"; the step-down method of the Node comprises the following steps:
Step 14.1.D.1, obtaining the highest level of the Node to be demoted to be g, and demoting the Node to be demoted to g-1 level;
step 14.1.D.2, in the g level of the Node tree, acquiring a left side link pointer array Prev [ g ] of the Node to be degraded to acquire a left side adjacent Node;
step 14.1.D.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be demoted to acquire a right side adjacent Node;
step 14.1.D.4, setting the right side link pointer array Next [ g ] of the left side adjacent Node obtained in the step 14.1.D.2 as the right side adjacent Node obtained in the step 14.1. D.3; setting the left side link pointer array Prev [ g ] of the right side adjacent Node obtained in the 14.1.D.3 step as the left side adjacent Node obtained in the 14.1.D.2 step;
step 14.1.D.5, setting the Prev [ g ] and Next [ g ] of the Node to be demoted to be empty;
and 14.1.D.6, the Node to be upgraded is demoted to g-1 level, and the demotion process is finished.
Further, according to the access method provided by the present invention, in step 14.1, the splitting of the dynamically adjusted "MetaD Node set" of the "dynamic structure adjustment" is performed on the gskipllist subsystem, and is aimed at a given "Node" if the number of "MetaD nodes" of the "MetaD Node set" pointed by the MetaD in the "Node" is greater than or equal to the "MetaD Node" fission threshold S in the "gskipllist parameter file", then the given "Node" becomes a "Node" to be split, which can be split into two new "Node" whose key value ranges do not intersect, and the original "MetaD Node set" is further split into two new "MetaD Node sets" according to the key value ranges of the two new "Node nodes", and each new "Node" points to a new "MetaD Node set"; further, the new Node with the key value range at the left side inherits the link relation of the Node in the Node tree given by the original Node; the method steps of splitting the MetaD node set are as follows:
Step 14.1.s.1, obtaining a key value range [ KeyLeft, keyRight ] of a Node to be split, and obtaining a MetaD Node set pointed by MetaDID in the Node to be split;
step 14.1.s.2, sorting the Key keys of the MetaD nodes in the step 14.1.s.1 according to the MetaD node set, and selecting the median value Key M of the Key keys;
step 14.1.s.3, dividing the Node to be split into two new Node nodes according to the Key M in the step 14.1.s.2, wherein the key value ranges of the Node nodes are [ Key left, key M ] and [ Key M, key Right ] respectively; the new Node on the left inherits the link relation of the original Node in the Node tree, and the new Node on the right is the completely new Node;
step 14.1.s.4, traversing the "MetaD Node set" pointed by the new "Node" on the left side in the step 14.1.s.3, and migrating all the "MetaD nodes" of the Key value range [ Key left, key M) of the new "Node" with the Key Key not on the left side to the "MetaD Node set" pointed by the new "Node" on the right side in the step 14.1.s.3;
step 14.1.s.5, completing the splitting of the MetaD node set, and ending the splitting treatment.
Further, according to the access method provided by the present invention, in step 14.1, the merging of the dynamically adjusted "MetaD Node sets" for "dynamically adjusting the gskipllist subsystem" is performed for two adjacent "Node nodes" in the 0 th level "Node tree", and if the number of "MetaD nodes" of the "MetaD Node set" pointed by the MetaD in the two "Node nodes" is smaller than the merging threshold M of the "MetaD Node" in the "gskipllist parameter file", the two "Node nodes" are merged into a new "Node" and the corresponding two "MetaD Node sets" are also merged together; further, the link relation of the two Node nodes in the Node tree is inherited by the new Node, the concrete method is that the respective highest level of the two Node nodes in the Node tree is obtained first, then selecting the highest-level Node as new Node, if the levels are the same, selecting one Node as new Node, and merging the link relation of another original Node in Node tree into new Node.
Furthermore, according to the access method provided by the present invention, the specific method steps of performing "persistent storage decision" on the gskipllist subsystem and implementing "persistent storage decision" in step 14.2 are as follows:
step 14.2.1, firstly, acquiring the actual occupied memory space value of the Gskip list subsystem;
step 14.2.2, calculating the ratio r of the actual occupied memory space value of the Gskiplist subsystem to the occupied maximum memory space value G in the Gskiplist parameter file;
14.2.3, judging whether the ratio r in the 14.2.2 th step is larger than or equal to a 'persistent storage decision threshold P' in a 'Gskip list parameter file', and if so, carrying out 'persistent storage implementation'; otherwise, do not perform "persistent storage enforcement";
step 14.2.4, the "persistent store decision" process ends.
Furthermore, according to the access method provided by the invention, the step 14.2 of performing a 'persistent storage decision and implementation' on the Gskip list subsystem adopts a database as a carrier of persistent storage, and the database can adopt a key value type database or a relational database; the storage format may be in a key-value format (key, value), where key is a key and value is a value corresponding to the key; the key in the Node is NodeID, the value is RLP code, which is the "recursive length prefix code" of other fields, and the value is the result of RLP code processing on the set of other fields of all non-NodeIDs in the Node; the key in the "MetaD node" is MetaDID, value value as a result of RLP encoding of the set of all other fields in the "MetaD node" that are not MetaDID.
The specific content and the meaning of the terms are as follows:
(1) Skip list: skip lists aprobabilistic alternative to balanced trees were published by William Pugh in 1990 at Communications ofthe ACM June1990,33 (6) 668-676. The design of skip list was initially an option as an alternative to balancing trees.
(2) Group hop table Gskiplist: the invention provides a novel jump table. The group jump table is a novel jump table supporting a group of key value ranges, and the structure of the novel jump table can be dynamically adjusted according to the 'user side inquiry requirement'.
(3) Blockchain: the distributed database is a decentralized and non-tamperable distributed database, and is widely applied to the aspects of finance, the Internet of things, public service, notarization, digital rights, traceability and the like.
(4) A blockchain query system: the system is based on the public and non-tamperable data of the blockchain and provides inquiry statistical service to the outside. With the needs of actual business, blockchain query systems are required to provide reliable, authoritative, and efficient complex queries, such as Top-k queries, range queries, equivalence queries, and the like.
(5) RLP: a data encoding rule, known as full Recursive Length Prefix, a recursive length prefix encoding, may be used to encode any nested binary array of data. The result of RLP encoding is also a binary sequence. The RLP may effectively serialize and deserialize data.
The invention has the advantages and positive effects that:
the invention can effectively improve the search efficiency of the block chain query.
The efficient query system and the dynamic construction method for the blockchain based on the Gskip list realize efficient query and retrieval of the blockchain data on the premise of not affecting the decentralization characteristic of the blockchain and the original data storage mode. The invention ensures the non-tamper property, openness and authority of the block chain data, ensures the high efficiency of the block chain data query search, and provides high efficiency query service for more and more applications based on the block chain on the premise of ensuring one block chain data.
Drawings
FIG. 1 is a schematic diagram of a system architecture of a Gskip list-based blockchain efficient query system of the present invention.
FIG. 2 is a schematic diagram of the basic structure of a transaction in a blockchain in the present invention.
FIG. 3 is a block diagram illustrating the basic structure of a block in a blockchain in accordance with the present invention.
Fig. 4 is a schematic diagram of the basic structure of a Node in the Gskiplist subsystem of the present invention.
FIG. 5 is a schematic diagram of the basic structure of a "MetaD node" in the Gskip list subsystem of the present invention.
Fig. 6 is a schematic diagram of the basic structure of a Gskiplist parameter file in the Gskiplist subsystem of the present invention.
Fig. 7 is a schematic diagram of a structure before an upgrade of a Node tree in a Gskiplist subsystem according to the present invention.
Fig. 8 is a schematic diagram of an upgraded structure of a Node tree in the Gskiplist subsystem according to the present invention.
Detailed Description
The invention discloses a Gskip list-based block chain efficient query system and a dynamic construction method. As shown in fig. 1, the high-efficiency query system based on Gskiplist provided by the present invention includes: a blockchain subsystem 110, a group jump table subsystem 120, a database 100, and the like; wherein, the blockchain subsystem 110 includes all "transactions" 112 and "blocks" 111, each "transaction" 111 includes at least transaction ID, i.e. TransactionID, transaction time, transaction participant, transaction header hash, transaction content information, each "block" 112 includes transactions within a given time, the "blocks" are linked together in a chain according to the time generated by the block, each "block" includes at least block ID, i.e. BlockID, block number, block start-stop time, previous block ID, merck tree root of the "transaction" set, block header hash, and "transaction" set information, and all "transaction" 112 and "block" 111 data in the blockchain subsystem 110 are stored in the database; the group jump table subsystem 120 is called Gskip list subsystem for short, which comprises a multi-stage Node tree 121 composed of Node nodes, gskip list subsystem also comprises a plurality of MetaD Node sets 122 composed of MetaD nodes, each Node at least comprises Node ID, left Key, right Key, key right, first MetaD Node ID, metaDID, search times, previous Node pointer array Prev, next Node pointer array Next information, each Node comprises a corresponding MetaD Node set in the range which is larger than or equal to the left Key and smaller than the right Key, namely [ KeyLeft, keyRight ], each Node also constructs a plurality of levels of chain association with other Node nodes through a pointer array Prev and a pointer array Next, each MetaD Node at least comprises an ID of the MetaD Node, namely MetaDID, a Key, namely Key, a NodeID of the corresponding Node, a transaction ID in the corresponding blockchain, namely TransactionID, a block ID in the corresponding blockchain, a MetaDID of the former MetaD Node, namely Prev, a MetaDID of the latter MetaD Node, namely Next, the Gskip list subsystem also comprises a Gskip list parameter file for defining a multi-level Node tree, wherein the Gskip list parameter file at least comprises an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searches, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node upgrading threshold U, a Node upgrading threshold value, a Node playing threshold value, a Node S threshold value, a maximum memory threshold value and a memory threshold value, all the data of Node and MetaD Node in Gskip list subsystem are stored in the database; database 100 is used to store transaction data 112 and chunk data 111 for chunk chain subsystem 110, and is also used to store "Node" and "MetaD Node" data for gskipedist subsystem 120.
The method for dynamically constructing the block chain efficient query system based on Gskip list comprises the following specific steps:
step 1, starting a system, and setting a Gskip list parameter file to be read;
step 2, reading a Gskip list parameter file to obtain an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searchfields, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node tree, a Node upgrading threshold U, a Node degrading threshold D, a MetaD Node fissioning threshold S, a MetaD Node merging threshold M, a maximum occupied memory space value G and persistent storage decision threshold P information;
step 3, reading block chain data and constructing an initial preset 'group jump table' subsystem, wherein the specific steps are as follows:
step 3.1, reading blockchain data to construct a level 0 Node tree and a plurality of MetaD Node sets according to the search field set of Gskip list in the step 2, namely GSL_Searchfields;
step 3.2, constructing a Node tree which is originally preset from the 1 st level to the Q level on the basis of the Node tree of the 0 th level according to the initial preset level Q of the Node tree in the step 2;
Step 4, the system starts to provide the block chain inquiry service to the outside;
step 5, monitoring new blocks of the block chain subsystem, and simultaneously monitoring and receiving a query request of a user;
step 6, judging the monitoring response type, if the monitoring response type is 'new block generation of the block chain subsystem', executing step 17; if the query request is the query request of the user side, executing the step 7;
step 7, analyzing the inquiry request of the user side;
step 8, converting the query request in the step 7 into a key value range [ Vsb, vse ] to be queried;
step 9, gradually searching from the "highest level" of the gskipllist subsystem to the 0 th level, and searching all "Node nodes" corresponding to the key value range [ Vsb, vse ] given in the 8 th step, wherein the "highest level" of the gskipllist subsystem is K, but the "highest level" is dynamically changed along with the structural change of the gskipllist subsystem; the [ KeyLeft, keyRight) of each "Node" retrieved needs to have a non-empty intersection with the key value range [ Vsb, vse ] given in step 8;
step 10, further searching all "MetaD nodes" corresponding to the Key value range [ Vsb, vse ] given in step 8, in all "Node nodes" searched in step 9, that is, the Key, i.e., the Key, of the searched "MetaD Node" needs to be in the Key value range [ Vsb, vse) given in step 8;
Step 11, all the "MetaD nodes" retrieved in step 10 further find relevant "blockchain data" in the blockchain according to transaction ID, i.e. transactionID, blockID, information in each "MetaD node";
step 12, further calculating a final query result of the query request of the user side according to all the block chain data searched in the step 11;
step 13, adding 1 to the search times, i.e. SCount, of each Node aiming at all Node nodes searched in the step 9;
step 14, dynamically adjusting the structure of the Gskiplist subsystem in a synchronous or asynchronous mode, wherein the specific steps are as follows:
step 14.1, performing 'dynamic structure adjustment' on the Gskip list subsystem in a synchronous or asynchronous mode, namely dynamically adjusting the upgrading or degradation of 'Node' and dynamically adjusting the splitting or merging of 'MetaD Node set';
step 14.2, adopting a synchronous or asynchronous mode to make a persistent storage decision and implement on the Gskip list subsystem, and storing all memory data of the Gskip list subsystem into a database;
step 15, returning the final query result calculated in the step 12 to the user side;
Step 16, judging whether the system continues to operate, if so, jumping to the step 5 to continue to operate; if the system does not continue to run, jumping to step 18;
step 17, reading a new block of the blockchain, and carrying out content updating on the Gskip list subsystem, namely updating a plurality of MetaD Node sets and Node trees according to transaction information in the new block; further jumping to the step 5 to continue execution;
step 18, performing persistent storage implementation on the Gskip list subsystem in a synchronous mode, and storing all memory data of the Gskip list subsystem into a database;
and step 19, ending the system execution.
For further understanding of the technical content, features and effects of the present invention, the following examples of application are set forth in detail below, along with the accompanying drawings:
application example 1
The dynamic construction method of the blockchain efficient query system based on the Gskip list comprises the following specific steps:
step 1, starting a system, and setting a Gskip list parameter file to be read;
for example: setting Gskip list parameter file as gsskip list_gas_config.yaml file
Step 2, reading a Gskip list parameter file to obtain an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searchfields, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node tree, a Node upgrading threshold U, a Node degrading threshold D, a MetaD Node fissioning threshold S, a MetaD Node merging threshold M, a maximum occupied memory space value G and persistent storage decision threshold P information;
For example:
gskipllist_gas_config.yaml file, wherein parameter gsl_id is 1, gsl_security fields is "Gas", maximum level N of Node tree "is 20, initial preset level Q of Node tree" is 14, initial preset rule type R of Node tree "is default value 1, node upgrade threshold U is 10, node degradation threshold D is 3, metaD Node fission threshold S is 10, metaD Node merge threshold M is 3, occupied maximum memory space value G is 128 units is MB, and persistent storage decision threshold P is 0.7.
The "github.com/spf13/viper" component may be used to manage the memory configuration, call the ReadInConfig () method of the viper component, transfer the absolute path of the parameter file on the disk, read the configuration stored in the parameter file, and then call the Unmarshal () method of the viper component to deserialize the configuration into the memory object gskiplist_gas_config.
Step 3, reading block chain data and constructing an initial preset 'group jump table' subsystem, wherein the specific steps are as follows:
step 3.1, reading blockchain data to construct a level 0 Node tree and a plurality of MetaD Node sets according to the search field set of Gskip list in the step 2, namely GSL_Searchfields;
For example:
firstly, installing a plug-in unit gateway/ethernet/go-ethernet, calling an ethclient Dial () method and transmitting an Rpc call address of an Ethernet, successfully creating an Ethernet client object to interact with the Ethernet, then transmitting a parameter block number through a method global EthRpcclient.
The transaction structure of the blockchain is shown in fig. 2, and the blockchain structure is shown in fig. 3.
Then, the GSL_Searchfields are read to judge that the Gskip list established at this time is ordered by taking the Gas value of the transaction as a key, so that the data of the response piece object of the Ethernet transaction is read, and a corresponding MetaD object is created, as shown in FIG. 5.
As shown in FIG. 5, metaDID is the globally unique self-increment id of the MetaD object, and Key corresponds to the Gas value of the transaction, that is, effectiveGasPrice, transactionID in Receipt corresponds to TxHash in Receipt, and BlockID corresponds to BlockNumber in Receipt.
Then call the Search () method of Gskiplist and pass in the Key of parameter MetaD, find out that [ KeyLeft, keyRight) of a Node in Gskiplist contains the Key of MetaD, then link MetaD to MetaD set in the corresponding Node object, if there is no Node, create a new Node object, as shown in fig. 4.
As shown in fig. 4, the Node id is the globally unique self-increment id of the Node object, keyLeft represents the left boundary of the Key range of the Node, keyRight represents the right boundary of the Key range of the Node, metaDID represents the first MetaDID of the MetaD set corresponding to the Node, the Prev array stores the previous Node id of each level of the Node in the Gskiplist, and the Next array stores the Next Node id of each level of the Node in the Gskiplist.
After the creation is successful, the pre_add () method of the Gskiplist is called, and the method finds the linked list position where the Key of the new Node is located in the 0 th level Node tree with ordered Key, inserts the new Node into the position of the 0 th level Node tree, and links the new MetaD into the MetaD set of the new Node.
Step 3.2, constructing a Node tree which is originally preset from the 1 st level to the Q level on the basis of the Node tree of the 0 th level according to the initial preset level Q of the Node tree in the step 2;
for example:
and reading a variable Q in the gskiplist_gas_config object, and acquiring an initial preset level of 14. We design the initial ideal model of Gskip list to be that the number of nodes of each stage is 1/2 of the Next stage, so we use the conditional probability algorithm to calculate that each Node should exist between 0 and N stages (N.ltoreq.Q) of Gskip list, we assume that every time the probability of adding 1 to N is 1/2, calculate the size of N, then construct the Node to the "Node tree" of 1 to N stages, specifically edit the Prev and Next array of the Node, such as now construct the Node to the "Node tree" of 1 st stage, then traverse all the nodes of the "Node tree" of 1 st stage, find the position where the new Node should be stored according to the Key order of the previous Node, i.e. Key right of the previous Node is less than or equal to Key left of the new Node and Key right of the Next Node is less than or equal to Key right of the new Node, then modify [1] Next Node and [1] Next Node, and update [1] Next Node and [1] Prev of the new Node, and so on the order of 1 st Node, and so on from "Node tree to" Node tree of N ".
Step 4, the system starts to provide the block chain inquiry service to the outside;
for example:
modifying Flag bit in memory configuration to 1 indicates providing blockchain query service, each query request will access Flag first, and only if 1 is determined, the query can be executed.
Step 5, monitoring new blocks of the block chain subsystem, and simultaneously monitoring and receiving a query request of a user;
for example:
and using a time packet of golang to start a timing task to check the Block number of the latest block of the Ethernet at fixed time, simultaneously establishing TCP connection with the client based on socket programming, and receiving a query request of the client.
Step 6, judging the monitoring response type, if the monitoring response type is 'new block generation of the block chain subsystem', executing step 17; if the query request is the query request of the user side, executing the step 7;
step 7, analyzing the inquiry request of the user side;
for example:
judging the query type of the user according to the mark bit of the user request parameter, wherein the query type can be Key equivalent query, key range query and Top-K query of Key.
Step 8, converting the query request in the step 7 into a key value range [ Vsb, vse ] to be queried;
for example:
the range of key values to be queried for the blockchain Gas fee [ Vsb, vse) is [0.01, 0.05).
Step 9, gradually searching from the "highest level" of the gskipllist subsystem to the 0 th level, and searching all "Node nodes" corresponding to the key value range [ Vsb, vse ] given in the 8 th step, wherein the "highest level" of the gskipllist subsystem is K, but the "highest level" is dynamically changed along with the structural change of the gskipllist subsystem; the [ KeyLeft, keyRight) of each "Node" retrieved needs to have a non-empty intersection with the key value range [ Vsb, vse ] given in step 8;
for example:
the [ KeyLeft, keyRight) of each "Node" retrieved should have a non-empty intersection with the example [0.01,0.05 ] in step 8), i.e., the key value of [ KeyLeft, keyRight) needs to contain or semi-contain [0.01,0.05 ].
Step 10, further searching all "MetaD nodes" corresponding to the Key value range [ Vsb, vse ] given in step 8, in all "Node nodes" searched in step 9, that is, the Key, i.e., the Key, of the searched "MetaD Node" needs to be in the Key value range [ Vsb, vse) given in step 8;
step 11, all the "MetaD nodes" retrieved in step 10 further find relevant "blockchain data" in the blockchain according to transaction ID, i.e. transactionID, blockID, information in each "MetaD node";
For example:
the transaction information and the block information of the block chain are shown in fig. 2 and 3 respectively.
Step 12, further calculating a final query result of the query request of the user side according to all the block chain data searched in the step 11;
for example:
according to different query types, such as Key equivalent query, key range query and Top-K query of Key, further calculation such as accumulation or sorting is carried out on all the searched block chain data, and finally a final query result of a query request of a user terminal is generated.
Step 13, adding 1 to the search times, i.e. SCount, of each Node aiming at all Node nodes searched in the step 9;
for example:
the "search times" of each "Node", i.e., SCount ", is incremented by 1, thereby preparing for further dynamic adjustment of the" Node tree ".
Step 14, dynamically adjusting the structure of the Gskiplist subsystem in a synchronous or asynchronous mode, wherein the specific steps are as follows:
step 14.1, performing 'dynamic structure adjustment' on the Gskip list subsystem in a synchronous or asynchronous mode, namely dynamically adjusting the upgrading or degradation of 'Node' and dynamically adjusting the splitting or merging of 'MetaD Node set';
For example:
the Node which is frequently accessed can be upgraded, so that the retrieval speed is increased;
the Node which is not frequently accessed can be degraded, so that the retrieval speed is increased, and the memory scale of the Gskip list is reduced as much as possible;
for a large number of "MetaD node sets" it is possible to split to reduce the cost of searching within the "MetaD node sets";
aiming at the 'MetaD node set' with extremely small quantity, the 'MetaD node set' is possibly converged to reduce the quantity, and the memory scale of Gskip list is also beneficial to be reduced as much as possible;
step 14.2, adopting a synchronous or asynchronous mode to make a persistent storage decision and implement on the Gskip list subsystem, and storing all memory data of the Gskip list subsystem into a database;
for example:
the Gskip list subsystem can determine whether to immediately implement 'persistent storage', namely 'persistent storage implementation', according to the memory utilization rate of the current Gskip list, so that the operation efficiency of the Gskip list subsystem is ensured, and the data security of the Gskip list subsystem is also ensured.
Step 15, returning the final query result calculated in the step 12 to the user side;
step 16, judging whether the system continues to operate, if so, jumping to the step 5 to continue to operate; if the system does not continue to run, jumping to step 18;
Step 17, reading a new block of the blockchain, and carrying out content updating on the Gskip list subsystem, namely updating a plurality of MetaD Node sets and Node trees according to transaction information in the new block; further jumping to the step 5 to continue execution;
for example:
the latest Block Numbers of the current Ethernet are obtained through the EthRpcClient.Block Numbers () method of the go-ethernet plugin, and then are compared with the LastBlock Numbers stored in our memory, if the data of the current Gskip list are the latest, if the data are the same, the current Gskip list needs to update the LastBlock Numbers to all blocks between the latest Block Numbers, if the data are larger, the current Gskip list needs to update the LastBlock Numbers. The specific updating step is similar to step 3.1, all transaction receipts of the Block are taken by the Ethernet client object, then a corresponding MetaD is created according to the transaction receipt content, then the MetaD is linked to a corresponding Node, if no corresponding Node exists, a new Node is created and inserted into the Gskip list, and the 1 to N stages of the Gskip list which should be inserted into the Gskip list are calculated according to a conditional probability algorithm. And then jumps to step 5 to continue execution.
Step 18, performing persistent storage implementation on the Gskip list subsystem in a synchronous mode, and storing all memory data of the Gskip list subsystem into a database;
For example:
just before the system ends, it is necessary to "persist" the Gskiplist subsystem, i.e., persist the updated Node and MetaD in memory to the database. The Ethernet bottom layer realizes a Leveldb and is used as a storage engine for persisting all data of the Ethernet, and a Batch () method of plug-in go-ethernet/ethdb/Leveldb can be used for executing the Put () method in batches, so that Node and MetaD are persisted into the Leveldb, and the decentralization characteristic of Gskip list index data is ensured not to be lost.
And step 19, ending the system execution.
Further, in the step 3.1, constructing a level 0 "Node tree" and a plurality of "MetaD Node sets" for the Gskiplist subsystem, which are sequentially reading blockchain blocks and transaction data, and constructing a level 0 "Node tree" and a plurality of "MetaD Node sets" according to the gsl_searchfields, which are the search field sets of the Gskiplist in the step 2, may be implemented by the following method steps:
step 3.1.1, sequentially reading block data from the 0 th block to the 'maximum ID block' of the blockchain, wherein each read block is abbreviated as Bi, wherein 0< = i < 'maximum ID block', and performing the following treatment on each block Bi;
For example:
the block information of the blockchain is shown in fig. 3.
Step 3.1.2, traversing each transaction in the block Bi, wherein each transaction is abbreviated as Tj, wherein the maximum transaction number of the block Bi is 0< = j < ", and processing the transaction Tj as follows;
for example:
the transaction information for the blockchain is shown in fig. 2.
Step 3.1.3, calculating a key and a value (value) for the transaction Tj based on gsl_security fields, which is the search field set of Gskiplist in step 2, and recording the result as (Kj, vj);
for example:
the Gas fee is used as the search field set.
Step 3.1.4, creating a MetaD node MDj for the transaction Tj, wherein the Key of MetaD node MDj has a value of Kj, the transaction ID of TransactionID of MetaD node MDj is Tj, and the block ID of MetaD node MDj is the block ID of Tj;
for example:
as shown in FIG. 5, the structure of the MetaD node.
Step 3.1.5, in the Gskip list subsystem, sequentially searching a 0 th level Node tree, and searching a Node Nm with a key value range containing Kj, wherein the key value range of the Node Nm is marked as [ x, y ], wherein x < = y; the key value range of Nm when x is equal to y is represented only as x; initially, the 0 th level Node tree has only one Node, and the key value range is [ - ++infinity, ++infinity);
For example:
as shown in fig. 4, the structure of the Node.
Initially, the 0 th level Node tree has only one Node, and the key value range is [ - ++infinity, ++infinity); then through the "split" operation, one Node will be dynamically split into several Node nodes.
Step 3.1.6, inserting the MetaD Node MDj information in step 3.1.4 into the "MetaD Node set" of Node Nm in step 3.1.5;
for example:
the MetaDID of a "Node" points to the first "MetaD Node" of a "MetaD Node set", and then other "MetaD nodes" are found from the Next, prev of this "MetaD Node".
Step 3.1.7, calculating the number of MetaD nodes owned by the Node Nm in the step 3.1.5, if the number of MetaD nodes owned by the Node Nm reaches the MetaD Node fission threshold S read in the step 2, calculating a t value so that the Node Nm can be split into two new Node nodes by fission, wherein the key value ranges of the two new Node nodes are [ x, t ] and [ t, y ] respectively, and the principle of selecting the t value is to make the two Node nodes approximately have the same number of MetaD nodes as much as possible;
for example:
this step implements the splitting, i.e., fission, of the "MetaD nodes" and the corresponding "Node nodes", thereby dynamically increasing the number of "Node nodes" and the number of "MetaD nodes".
Step 3.1.8, judging whether all transactions of the block Bi are traversed, if all transactions of the block Bi are traversed, jumping to the 3.1.9 step to continue execution; otherwise, jumping to the 3.1.2 step to continue to process the next transaction;
step 3.1.9, judging whether all blocks of the blockchain are traversed, if all blocks of the blockchain are traversed, jumping to 3.1.10 to continue execution; otherwise, jumping to the 3.1.1 step to continue processing the next block;
step 3.1.10, the program process ends.
Further, in the step 3.2, the step 1 of constructing the "Node tree" of the first to the fourth levels of initial presettings for the gskipllist subsystem is to upgrade a part of the "Node nodes" of the 0 th level "Node tree" according to the initial preset level Q of the "Node tree" and the initial preset rule type R of the "Node tree" in the "gskipllist parameter file", so as to build the "Node tree" of the first to the fourth levels of stages step by step; the initial preset rule type R of the "Node tree" may include the following rule types: a bipartite rule, a trisection rule, an N-score rule and a custom rule; the bipartite rule refers to that in the Node tree of the same level, one Node in every two continuous Node nodes performs upgrading operation; the trisection rule refers to that in the Node tree of the same level, one Node in every three continuous Node nodes performs upgrading operation; n division rules refer to that in the Node tree of the same level, one Node in each continuous N Node nodes performs upgrading operation; the custom rule refers to that in the Node tree of the same level, a plurality of Node nodes are selected according to the custom rule of a designer to carry out upgrading operation.
For example:
the Node tree of the 0 th level adopts a bipartite rule, namely, in the Node tree of the 0 th level, one Node in every two continuous Node nodes is updated to become the Node tree of the 1 st level, so that the Node tree of the 1 st level has half Node nodes in the Node tree of the 0 th level, and each Node tree of the 0 th level is not required to be traversed in searching, thereby accelerating the searching efficiency.
Further, the step 14.1 is to upgrade the dynamic adjustment Node of the dynamic structure adjustment for the gskipllist subsystem, according to the searching times of each Node, namely whether SCount meets the condition of a Node upgrading threshold U in a Gskip list parameter file, if yes, the Node is upgraded in a Node tree by one stage; the step-up method of the Node comprises the following steps:
step 14.1.U.1, obtaining the highest level of the Node to be upgraded and marking the highest level as g, and upgrading the Node to be upgraded into g+1 level;
for example:
as shown in fig. 7, the Node to be upgraded is upgraded from g level to g+1 level, and the result shown in fig. 8 is obtained.
Step 14.1.U.2, in the g level of "Node tree", obtaining the left side link pointer array Prev [ g ] of "Node" to be upgraded to obtain the left side adjacent "Node"; if the left adjacent Node does not have g+1 level, continuing to acquire the left adjacent Node of the left adjacent Node until acquiring a left adjacent Node with g+1 level;
step 14.1.U.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be upgraded to acquire a right side adjacent Node; if the right adjacent Node does not have g+1 level, continuing to acquire the right adjacent Node of the right adjacent Node until acquiring a right adjacent Node with g+1 level;
for example:
before upgrading the Node, the Node nodes adjacent to the left and the right need to be found, and the Node nodes adjacent to the left and the right need to have g+1 level, so the Node nodes adjacent to the left and the right meeting the requirement are found in the steps 14.1.U.2 and 14.1. U.3.
Step 14.1.U.4, setting the right side link pointer array Next [ g+1] of the left side adjacent Node obtained in the step 14.1.U.2 as the Node to be upgraded; setting the left side link pointer array Prev [ g+1] of the right side adjacent Node obtained in the 14.1.U.3 step as the Node to be upgraded;
Step 14.1.U.5, further setting the left side link pointer array Prev [ g+1] of the "Node" to be upgraded as the left side adjacent "Node" obtained in the 14.1.U.2 step, and setting the right side link pointer array Next [ g+1] of the "Node" to be upgraded as the right side adjacent "Node" obtained in the 14.1.U.3 step;
for example:
in upgrading the Node, it is necessary to implement link interconnection on the Node nodes adjacent to each other on the g+1 level at the g+1 level.
And step 14.1.U.6, finishing upgrading the Node to be upgraded into g+1 level, and finishing the upgrading treatment.
Further, the step 14.1 of performing the degradation of the "Node" of the "dynamic structure adjustment" on the gskipllist subsystem is based on whether the "retrieval number" of each "Node" is equal to the "Node" degradation threshold D condition in the "gskipllist parameter file", and if the degradation condition is equal to the "Node" is reduced by one level in the "Node tree"; the step-down method of the Node comprises the following steps:
step 14.1.D.1, obtaining the highest level of the Node to be demoted to be g, and demoting the Node to be demoted to g-1 level;
Step 14.1.D.2, in the g level of the Node tree, acquiring a left side link pointer array Prev [ g ] of the Node to be degraded to acquire a left side adjacent Node;
step 14.1.D.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be demoted to acquire a right side adjacent Node;
for example:
before the "Node" is downgraded, the left and right adjacent "Node nodes" need to be found, so steps 14.1.D.2 and 14.1.D.3 find the right and left adjacent "Node nodes" meeting the requirement.
Step 14.1.D.4, setting the right side link pointer array Next [ g ] of the left side adjacent Node obtained in the step 14.1.D.2 as the right side adjacent Node obtained in the step 14.1. D.3; setting the left side link pointer array Prev [ g ] of the right side adjacent Node obtained in the 14.1.D.3 step as the left side adjacent Node obtained in the 14.1.D.2 step;
for example:
in the degradation "Node" it is necessary to implement, at the g-th level, link interconnection of the "Node nodes" adjacent to the degradation "Node" on the left and right.
Step 14.1.D.5, setting the Prev [ g ] and Next [ g ] of the Node to be demoted to be empty;
For example:
the Prev and Next of the g-th level of the demotion "Node" are set to null, meaning that the "Node" has no g-th level.
And 14.1.D.6, the Node to be upgraded is demoted to g-1 level, and the demotion process is finished.
Further, the splitting of the dynamic adjustment "MetaD Node set" of the "dynamic structure adjustment" for the gskipllist subsystem in step 14.1 is for a given "Node", if the number of "MetaD nodes" of the "MetaD Node set" pointed by the MetaDID in the "Node" is greater than or equal to the "MetaD Node" fission threshold S in the "gskipllist parameter file", the given "Node" becomes a "Node" to be split, which can be split into two new "Node nodes" whose key value ranges do not intersect, and further splitting the original "MetaD Node set" into two new "MetaD Node sets" according to the key value ranges of the two new "Node nodes", where each new "Node" points to a new "MetaD Node set"; further, the new Node with the key value range at the left side inherits the link relation of the Node in the Node tree given by the original Node; the method steps of splitting the MetaD node set are as follows:
Step 14.1.s.1, obtaining a key value range [ KeyLeft, keyRight ] of a Node to be split, and obtaining a MetaD Node set pointed by MetaDID in the Node to be split;
step 14.1.s.2, sorting the Key keys of the MetaD nodes in the step 14.1.s.1 according to the MetaD node set, and selecting the median value Key M of the Key keys;
for example:
the median value KeyM of the Key Key is generally selected as a split Key value point, and an intermediate value and a self-selected value can be selected as the split Key value point, so that the split of the MetaD node set is as uniform as possible, and the number of MetaD nodes of two new MetaD node sets is as much as possible.
Step 14.1.s.3, dividing the Node to be split into two new Node nodes according to the Key M in the step 14.1.s.2, wherein the key value ranges of the Node nodes are [ Key left, key M ] and [ Key M, key Right ] respectively; the new Node on the left inherits the link relation of the original Node in the Node tree, and the new Node on the right is the completely new Node;
step 14.1.s.4, traversing the "MetaD Node set" pointed by the new "Node" on the left side in the step 14.1.s.3, and migrating all the "MetaD nodes" of the Key value range [ Key left, key M) of the new "Node" with the Key Key not on the left side to the "MetaD Node set" pointed by the new "Node" on the right side in the step 14.1.s.3;
For example:
in fact, the new "Node" on the left side and the new "Node" on the right side can inherit the link relation of the originally given "Node" in the "Node tree".
Step 14.1.s.5, completing the splitting of the MetaD node set, and ending the splitting treatment.
Further, the step 14.1 of merging the dynamically adjusted "MetaD Node sets" of the "dynamic structure adjustment" performed on the gskipllist subsystem is aimed at two adjacent "Node nodes" in the level 0 "Node tree", and if the number of "MetaD nodes" of the "MetaD Node sets" pointed by the MetaD in the two "Node nodes" is smaller than the merging threshold M of the "MetaD nodes" in the "gskipllist parameter file", the two "Node nodes" are merged into a new "Node" and the corresponding two "MetaD Node sets" are also merged together; further, the link relation of the two Node nodes in the Node tree is inherited by the new Node, the concrete method is that the respective highest level of the two Node nodes in the Node tree is obtained first, then selecting the highest-level Node as new Node, if the levels are the same, selecting one Node as new Node, and merging the link relation of another original Node in Node tree into new Node.
For example:
the merging of the MetaD Node sets mainly reduces the memory usage of the Gskip list subsystem and accelerates the retrieval speed by reducing the number of the Node nodes.
Further, the specific method steps of the "persistence decision" for making and implementing "persistence decision" for the gskipllist subsystem in step 14.2 are as follows:
step 14.2.1, firstly, acquiring the actual occupied memory space value of the Gskip list subsystem;
step 14.2.2, calculating the ratio r of the actual occupied memory space value of the Gskiplist subsystem to the occupied maximum memory space value G in the Gskiplist parameter file;
14.2.3, judging whether the ratio r in the 14.2.2 th step is larger than or equal to a 'persistent storage decision threshold P' in a 'Gskip list parameter file', and if so, carrying out 'persistent storage implementation'; otherwise, do not perform "persistent storage enforcement";
for example:
the "persistent storage decision threshold P" in the "Gskiplist parameter file" typically takes a value of about 0.7. If the ratio r in step 14.2.2 is greater than or equal to 0.7, this means that a specific "persistent store implementation" can be performed to save the in-memory modified data to the database.
Step 14.2.4, the "persistent store decision" process ends.
Further, in the step 14.2, "persistent storage decision and implementation" is performed on the gskipllist subsystem, the invention adopts a database as a carrier of persistent storage, and the database can adopt a key value type database or a relational type database; the storage format may be in a key-value format (key, value), where key is a key and value is a value corresponding to the key; the key in the Node is NodeID, the value is RLP code, which is the "recursive length prefix code" of other fields, and the value is the result of RLP code processing on the set of other fields of all non-NodeIDs in the Node; the key in the "MetaD node" is MetaDID, value value as a result of RLP encoding of the set of all other fields in the "MetaD node" that are not MetaDID.
For example:
in implementing RLP coding, it is first necessary to map the object into two forms, byte arrays or lists. Taking the go language code "Node" as an example, it will be mapped into a list [ NodeID, keyLeft, keyRight, metaDID, SCount, prev, next ];
for a "Node", its value calculation process:
node:=Node{KeyLeft:"0.01",KeyRight:"0.03",MetaDID:"1234",SCount:"5",Prev:"101",Next:"151"}
value:=bytes.Buffer{}
rpl.Encode(&value,&node)
For a "MetaD node," its value calculation process:
node:=MetaD{Key:"0.012",NodeID:19,TransactionID:"0x22e11567c38d6e7ea45c0761c13b08a6cbabdb98461ff05cfdfeb5e1611579b7",BlockID:"9888333",Prev:"1003",Next:"1202"}
value:=bytes.Buffer{}
rpl.Encode(&value,&node)
other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope of the invention being indicated by the following claims.

Claims (10)

1. A high-efficiency block chain query system based on Gskip list is characterized by comprising a block chain subsystem, a group skip list subsystem and a database; the block chain subsystem comprises all 'transaction' and 'blocks', each 'transaction' at least comprises transaction ID, namely transaction ID, transaction time, transaction participators, transaction head hash and transaction content information, each 'block' comprises transactions within a given time, the 'blocks' are connected together in a chained mode according to the time generated by the block, each 'block' at least comprises block ID, namely block ID, block number, block start-stop time, previous block ID, merck tree root of a contained 'transaction' set, block head hash and contained 'transaction' set information, and all 'transaction' and 'block' data in the block chain subsystem are stored in the database; the group jump table subsystem is called Gskip list subsystem for short, which comprises a multi-stage Node tree composed of Node nodes, the Gskip list subsystem also comprises a plurality of MetaD Node sets composed of MetaD nodes, each Node at least comprises Node ID, left Key, right Key, key right, first MetaD Node ID, metaDID, search times, SCount, former Node pointer array Prev, latter Node pointer array Next information, each Node comprises a corresponding MetaD Node set in the range which is larger than or equal to the left Key and smaller than the right Key, namely [ KeyLeft, keyRight ], each Node also constructs a plurality of levels of chain association with other Node nodes through a pointer array Prev and a pointer array Next, each MetaD Node at least comprises an ID of the MetaD Node, namely MetaDID, a Key, namely Key, a NodeID of the corresponding Node, a transaction ID in the corresponding blockchain, namely TransactionID, a block ID in the corresponding blockchain, a MetaDID of the former MetaD Node, namely Prev, a MetaDID of the latter MetaD Node, namely Next, the Gskip list subsystem also comprises a Gskip list parameter file for defining a multi-level Node tree, wherein the Gskip list parameter file at least comprises an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searches, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node upgrading threshold U, a Node upgrading threshold value, a Node playing threshold value, a Node S threshold value, a maximum memory threshold value and a memory threshold value, all the data of Node and MetaD Node in Gskip list subsystem are stored in the database; the database is used for storing transaction data and block data of the block chain subsystem, and the database is also used for storing Node data and MetaD Node data of the Gskip list subsystem.
2. A method for dynamically constructing a Gskiplist-based blockchain efficient query system as in claim 1, the method comprising the specific steps of:
step 1, starting a system, and setting a Gskip list parameter file to be read;
step 2, reading a Gskip list parameter file to obtain an ID of the Gskip list, namely GSL_ID, a search field set of the Gskip list, namely GSL_Searchfields, a maximum level N of the Node tree, an initial preset level Q of the Node tree, an initial preset rule type R of the Node tree, a Node upgrading threshold U, a Node degrading threshold D, a MetaD Node fissioning threshold S, a MetaD Node merging threshold M, a maximum occupied memory space value G and persistent storage decision threshold P information;
step 3, reading block chain data and constructing an initial preset 'group jump table' subsystem, wherein the specific steps are as follows:
step 3.1, reading blockchain data to construct a level 0 Node tree and a plurality of MetaD Node sets according to the search field set of Gskip list in the step 2, namely GSL_Searchfields;
step 3.2, constructing a Node tree which is originally preset from the 1 st level to the Q level on the basis of the Node tree of the 0 th level according to the initial preset level Q of the Node tree in the step 2;
Step 4, the system starts to provide the block chain inquiry service to the outside;
step 5, monitoring new blocks of the block chain subsystem, and simultaneously monitoring and receiving a query request of a user;
step 6, judging the monitoring response type, if the monitoring response type is 'new block generation of the block chain subsystem', executing step 17; if the query request is the query request of the user side, executing the step 7;
step 7, analyzing the inquiry request of the user side;
step 8, converting the query request in the step 7 into a key value range [ Vsb, vse ] to be queried;
step 9, gradually searching from the "highest level" of the gskipllist subsystem to the 0 th level, and searching all "Node nodes" corresponding to the key value range [ Vsb, vse ] given in the 8 th step, wherein the "highest level" of the gskipllist subsystem is K, but the "highest level" is dynamically changed along with the structural change of the gskipllist subsystem; the [ KeyLeft, keyRight) of each "Node" retrieved needs to have a non-empty intersection with the key value range [ Vsb, vse ] given in step 8;
step 10, further searching all "MetaD nodes" corresponding to the Key value range [ Vsb, vse ] given in step 8, in all "Node nodes" searched in step 9, that is, the Key, i.e., the Key, of the searched "MetaD Node" needs to be in the Key value range [ Vsb, vse) given in step 8;
Step 11, all the "MetaD nodes" retrieved in step 10 further find relevant "blockchain data" in the blockchain according to transaction ID, i.e. transactionID, blockID, information in each "MetaD node";
step 12, further calculating a final query result of the query request of the user side according to all the block chain data searched in the step 11;
step 13, adding 1 to the search times, i.e. SCount, of each Node aiming at all Node nodes searched in the step 9;
step 14, dynamically adjusting the structure of the Gskiplist subsystem in a synchronous or asynchronous mode, wherein the specific steps are as follows:
step 14.1, performing 'dynamic structure adjustment' on the Gskip list subsystem in a synchronous or asynchronous mode, namely dynamically adjusting the upgrading or degradation of 'Node' and dynamically adjusting the splitting or merging of 'MetaD Node set';
step 14.2, adopting a synchronous or asynchronous mode to make a persistent storage decision and implement on the Gskip list subsystem, and storing all memory data of the Gskip list subsystem into a database;
step 15, returning the final query result calculated in the step 12 to the user side;
Step 16, judging whether the system continues to operate, if so, jumping to the step 5 to continue to operate; if the system does not continue to run, jumping to step 18;
step 17, reading a new block of the blockchain, and carrying out content updating on the Gskip list subsystem, namely updating a plurality of MetaD Node sets and Node trees according to transaction information in the new block; further jumping to the step 5 to continue execution;
step 18, performing persistent storage implementation on the Gskip list subsystem in a synchronous mode, and storing all memory data of the Gskip list subsystem into a database;
and step 19, ending the system execution.
3. The method according to claim 2, wherein the constructing the "Node tree" and the plurality of "MetaD Node sets" for the gskipllist subsystem in step 3.1 is sequentially reading blockchain blocks and transaction data, and the "Node tree" and the plurality of "MetaD Node sets" in step 0 are constructed according to the gsl_searchfields, which are the search field set of the gskipllist in step 2, by the following method steps:
step 3.1.1, sequentially reading block data from the 0 th block to the 'maximum ID block' of the blockchain, wherein each read block is abbreviated as Bi, wherein 0< = i < 'maximum ID block', and performing the following treatment on each block Bi;
Step 3.1.2, traversing each transaction in the block Bi, wherein each transaction is abbreviated as Tj, wherein the maximum transaction number of the block Bi is 0< = j < ", and processing the transaction Tj as follows;
step 3.1.3, calculating a key and a value (value) for the transaction Tj based on gsl_security fields, which is the search field set of Gskiplist in step 2, and recording the result as (Kj, vj);
step 3.1.4, creating a MetaD node MDj for the transaction Tj, wherein the Key of MetaD node MDj has a value of Kj, the transaction ID of TransactionID of MetaD node MDj is Tj, and the block ID of MetaD node MDj is the block ID of Tj;
step 3.1.5, in the Gskip list subsystem, sequentially searching a 0 th level Node tree, and searching a Node Nm with a key value range containing Kj, wherein the key value range of the Node Nm is marked as [ x, y ], wherein x < = y; the key value range of Nm when x is equal to y is represented only as x; initially, the 0 th level Node tree has only one Node, and the key value range is [ - ++infinity, ++infinity);
step 3.1.6, inserting the MetaD Node MDj information in step 3.1.4 into the "MetaD Node set" of Node Nm in step 3.1.5;
step 3.1.7, calculating the number of MetaD nodes owned by the Node Nm in the step 3.1.5, if the number of MetaD nodes owned by the Node Nm reaches the MetaD Node fission threshold S read in the step 2, calculating a t value so that the Node Nm can be split into two new Node nodes by fission, wherein the key value ranges of the two new Node nodes are [ x, t ] and [ t, y ] respectively, and the principle of selecting the t value is to make the two Node nodes approximately have the same number of MetaD nodes as much as possible;
Step 3.1.8, judging whether all transactions of the block Bi are traversed, if all transactions of the block Bi are traversed, jumping to the 3.1.9 step to continue execution; otherwise, jumping to the 3.1.2 step to continue to process the next transaction;
step 3.1.9, judging whether all blocks of the blockchain are traversed, if all blocks of the blockchain are traversed, jumping to 3.1.10 to continue execution; otherwise, jumping to the 3.1.1 step to continue processing the next block;
step 3.1.10, the program process ends.
4. The method as claimed in claim 2, wherein the constructing the "Node tree" of the initial presets of the 1 st to the K th levels for the gskipllist subsystem in step 3.2 is to perform the upgrade operation for the part of the "Node nodes" of the 0 th level "Node tree" according to the initial preset level Q of the "Node tree" in the "gskipllist parameter file" and the initial preset rule type R of the "Node tree", so as to build the "Node tree" of the 1 st to the K th levels step by step; the initial preset rule type R of the "Node tree" may include the following rule types: a bipartite rule, a trisection rule, an N-score rule and a custom rule; the bipartite rule refers to that in the Node tree of the same level, one Node in every two continuous Node nodes performs upgrading operation; the trisection rule refers to that in the Node tree of the same level, one Node in every three continuous Node nodes performs upgrading operation; n division rules refer to that in the Node tree of the same level, one Node in each continuous N Node nodes performs upgrading operation; the custom rule refers to that in the Node tree of the same level, a plurality of Node nodes are selected according to the custom rule of a designer to carry out upgrading operation.
5. The method according to claim 2, wherein the step 14.1 of dynamically adjusting "Node" of "dynamic structure adjustment" of the gskipllist subsystem is performed according to whether "Node" updating threshold U condition in "gskipllist parameter file" is satisfied by "Node" as the number of times of searching "Node" or by "SCount" of each "Node", and if the updating condition is satisfied, the "Node" is updated by one step in "Node tree"; the step-up method of the Node comprises the following steps:
step 14.1.U.1, obtaining the highest level of the Node to be upgraded and marking the highest level as g, and upgrading the Node to be upgraded into g+1 level;
step 14.1.U.2, in the g level of "Node tree", obtaining the left side link pointer array Prev [ g ] of "Node" to be upgraded to obtain the left side adjacent "Node"; if the left adjacent Node does not have g+1 level, continuing to acquire the left adjacent Node of the left adjacent Node until acquiring a left adjacent Node with g+1 level;
step 14.1.U.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be upgraded to acquire a right side adjacent Node; if the right adjacent Node does not have g+1 level, continuing to acquire the right adjacent Node of the right adjacent Node until acquiring a right adjacent Node with g+1 level;
Step 14.1.U.4, setting the right side link pointer array Next [ g+1] of the left side adjacent Node obtained in the step 14.1.U.2 as the Node to be upgraded; setting the left side link pointer array Prev [ g+1] of the right side adjacent Node obtained in the 14.1.U.3 step as the Node to be upgraded;
step 14.1.U.5, further setting the left side link pointer array Prev [ g+1] of the "Node" to be upgraded as the left side adjacent "Node" obtained in the 14.1.U.2 step, and setting the right side link pointer array Next [ g+1] of the "Node" to be upgraded as the right side adjacent "Node" obtained in the 14.1.U.3 step;
and step 14.1.U.6, finishing upgrading the Node to be upgraded into g+1 level, and finishing the upgrading treatment.
6. The method according to claim 2, wherein the step 14.1 of performing the degradation of the "Node" of the "dynamic structure adjustment" on the gskipllist subsystem is based on whether the "retrieval number" of each "Node" is equal to the "Node" degradation threshold D condition in the "gskipllist parameter file", and if the degradation condition is equal to the "Node" is reduced by one level in the "Node tree"; the step-down method of the Node comprises the following steps:
Step 14.1.D.1, obtaining the highest level of the Node to be demoted to be g, and demoting the Node to be demoted to g-1 level;
step 14.1.D.2, in the g level of the Node tree, acquiring a left side link pointer array Prev [ g ] of the Node to be degraded to acquire a left side adjacent Node;
step 14.1.D.3, in the g level of the Node tree, acquiring a right side link pointer array Next [ g ] of the Node to be demoted to acquire a right side adjacent Node;
step 14.1.D.4, setting the right side link pointer array Next [ g ] of the left side adjacent Node obtained in the step 14.1.D.2 as the right side adjacent Node obtained in the step 14.1. D.3; setting the left side link pointer array Prev [ g ] of the right side adjacent Node obtained in the 14.1.D.3 step as the left side adjacent Node obtained in the 14.1.D.2 step;
step 14.1.D.5, setting the Prev [ g ] and Next [ g ] of the Node to be demoted to be empty;
and 14.1.D.6, the Node to be upgraded is demoted to g-1 level, and the demotion process is finished.
7. The method according to claim 2, wherein the splitting of the dynamically adjusted "MetaD Node set" for the Gskiplist subsystem in step 14.1 is for a given "Node", and if the number of "MetaD nodes" of the "MetaD Node set" pointed by the MetaD in the "Node" is greater than or equal to the "MetaD Node" fission threshold S in the "Gskiplist parameter file", the given "Node" becomes a "Node" to be split, which can be split into two new "Node nodes" with disjoint key value ranges, and further splitting the original "MetaD Node set" into two new "MetaD Node sets" according to the key value ranges of the two new "Node nodes", wherein each new "Node" points to a new "MetaD Node set"; further, the new Node with the key value range at the left side inherits the link relation of the Node in the Node tree given by the original Node; the method steps of splitting the MetaD node set are as follows:
Step 14.1.s.1, obtaining a key value range [ KeyLeft, keyRight ] of a Node to be split, and obtaining a MetaD Node set pointed by MetaDID in the Node to be split;
step 14.1.s.2, sorting the Key keys of the MetaD nodes in the step 14.1.s.1 according to the MetaD node set, and selecting the median value Key M of the Key keys;
step 14.1.s.3, dividing the Node to be split into two new Node nodes according to the Key M in the step 14.1.s.2, wherein the key value ranges of the Node nodes are [ Key left, key M ] and [ Key M, key Right ] respectively; the new Node on the left inherits the link relation of the original Node in the Node tree, and the new Node on the right is the completely new Node;
step 14.1.s.4, traversing the "MetaD Node set" pointed by the new "Node" on the left side in the step 14.1.s.3, and migrating all the "MetaD nodes" of the Key value range [ Key left, key M) of the new "Node" with the Key Key not on the left side to the "MetaD Node set" pointed by the new "Node" on the right side in the step 14.1.s.3;
step 14.1.s.5, completing the splitting of the MetaD node set, and ending the splitting treatment.
8. The method according to claim 2, wherein the step 14.1 of merging the dynamically adjusted "MetaD Node sets" of the gskipllist subsystem is performed for two adjacent "Node nodes" in the level 0 "Node tree", and if the number of "MetaD nodes" of the "MetaD Node sets" pointed to by the MetaD in the two "Node nodes" is smaller than the "MetaD Node" merging threshold M in the "gskipllist parameter file", the two "Node nodes" are merged into a new "Node", and the corresponding two "MetaD Node sets" are also merged together; further, the link relation of the two Node nodes in the Node tree is inherited by the new Node, the concrete method is that the respective highest level of the two Node nodes in the Node tree is obtained first, then selecting the highest-level Node as new Node, if the levels are the same, selecting one Node as new Node, and merging the link relation of another original Node in Node tree into new Node.
9. The method according to claim 2, wherein the specific method steps of the step 14.2 for making and implementing the "persistent storage decision" for the Gskiplist subsystem are as follows:
step 14.2.1, firstly, acquiring the actual occupied memory space value of the Gskip list subsystem;
step 14.2.2, calculating the ratio r of the actual occupied memory space value of the Gskiplist subsystem to the occupied maximum memory space value G in the Gskiplist parameter file;
14.2.3, judging whether the ratio r in the 14.2.2 th step is larger than or equal to a 'persistent storage decision threshold P' in a 'Gskip list parameter file', and if so, carrying out 'persistent storage implementation'; otherwise, do not perform "persistent storage enforcement";
step 14.2.4, the "persistent store decision" process ends.
10. The method according to claim 2, wherein the step 14.2 of performing "persistent storage decision and implementation" on the Gskiplist subsystem "is implemented by using a database as a carrier of persistent storage, wherein the database may be a key-value type database or a relational database; the storage format may be in a key-value format (key, value), where key is a key and value is a value corresponding to the key; the key in the Node is NodeID, the value is RLP code, which is the "recursive length prefix code" of other fields, and the value is the result of RLP code processing on the set of other fields of all non-NodeIDs in the Node; the key in the "MetaD node" is MetaDID, value value as a result of RLP encoding of the set of all other fields in the "MetaD node" that are not MetaDID.
CN202311447531.0A 2023-11-02 2023-11-02 Gskip list-based block chain efficient query system and dynamic construction method Pending CN117472959A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311447531.0A CN117472959A (en) 2023-11-02 2023-11-02 Gskip list-based block chain efficient query system and dynamic construction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311447531.0A CN117472959A (en) 2023-11-02 2023-11-02 Gskip list-based block chain efficient query system and dynamic construction method

Publications (1)

Publication Number Publication Date
CN117472959A true CN117472959A (en) 2024-01-30

Family

ID=89639293

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311447531.0A Pending CN117472959A (en) 2023-11-02 2023-11-02 Gskip list-based block chain efficient query system and dynamic construction method

Country Status (1)

Country Link
CN (1) CN117472959A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725348A (en) * 2024-02-07 2024-03-19 蓝象智联(杭州)科技有限公司 Thread management method and system in GPU computing large-scale array summation process

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117725348A (en) * 2024-02-07 2024-03-19 蓝象智联(杭州)科技有限公司 Thread management method and system in GPU computing large-scale array summation process
CN117725348B (en) * 2024-02-07 2024-05-10 蓝象智联(杭州)科技有限公司 Thread management method and system in GPU computing large-scale array summation process

Similar Documents

Publication Publication Date Title
CN107092656B (en) A kind of tree data processing method and system
CN105975587B (en) A kind of high performance memory database index organization and access method
CN103150394B (en) Distributed file system metadata management method facing to high-performance calculation
US9575976B2 (en) Methods and apparatuses to optimize updates in a file system based on birth time
CN109471905B (en) Block chain indexing method supporting time range and attribute range compound query
US20010042186A1 (en) Compression of nodes in a trie structure
CN107016071B (en) A kind of method and system using simple path characteristic optimization tree data
CN117472959A (en) Gskip list-based block chain efficient query system and dynamic construction method
CN1881210A (en) Method and apparatus for search
Hadian et al. Interpolation-friendly B-trees: Bridging the Gap Between Algorithmic and Learned Indexes.
CN108134739B (en) Route searching method and device based on index trie
CN113961514A (en) Data query method and device
CN107239485A (en) Database operation method, apparatus and system
US20170116242A1 (en) Evaluating sql expressions on dictionary encoded vectors
CN115718819A (en) Index construction method, data reading method and index construction device
CN102567419B (en) Mass data storage device and method based on tree structure
CN115114294A (en) Self-adaption method and device of database storage mode and computer equipment
US20050240595A1 (en) Dynamic redistribution of a distributed memory index when individual nodes have different lookup indexes
CN113722274A (en) Efficient R-tree index remote sensing data storage model
CN113688130A (en) Memory database storage engine management method
CN116301656A (en) Data storage method, system and equipment based on log structure merging tree
CN116501760A (en) Efficient distributed metadata management method combining memory and prefix tree
US20220215021A1 (en) Data Query Method and Apparatus, Computing Device, and Storage Medium
CN113076330B (en) Query processing method, device, database system, electronic equipment and storage medium
CN113961568A (en) Block chain-based block fast searching method for chain data structure

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination