CN117595997A - Method, apparatus, device, storage medium and program product for data processing - Google Patents

Method, apparatus, device, storage medium and program product for data processing Download PDF

Info

Publication number
CN117595997A
CN117595997A CN202311570233.0A CN202311570233A CN117595997A CN 117595997 A CN117595997 A CN 117595997A CN 202311570233 A CN202311570233 A CN 202311570233A CN 117595997 A CN117595997 A CN 117595997A
Authority
CN
China
Prior art keywords
target
consensus
block
data
consensus node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311570233.0A
Other languages
Chinese (zh)
Inventor
张爽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to CN202311570233.0A priority Critical patent/CN117595997A/en
Publication of CN117595997A publication Critical patent/CN117595997A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1061—Peer-to-peer [P2P] networks using node-based peer discovery mechanisms
    • H04L67/1065—Discovery involving distributed pre-established resource-based relationships among peers, e.g. based on distributed hash tables [DHT] 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/50—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using hash chains, e.g. blockchains or hash trees
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Embodiments of the present disclosure provide a data processing method, apparatus, device, computer readable medium, and computer program product. The method described herein is performed at a consensus node that lacks ledger data. The method comprises the following steps: acquiring block information of the latest block stored by other consensus nodes in the consensus network, wherein the block information comprises block heights; determining a group of target consensus nodes according to the acquired block information, wherein the block information of the latest block stored in the group of target consensus nodes is consistent; determining a plurality of target blocks for acquiring missing ledger data by comparing the block heights of the locally stored latest blocks with the block heights of the latest blocks stored in a set of target consensus nodes; and concurrently acquiring ledger data in a plurality of target blocks from some or all of the target consensus nodes. Therefore, the efficiency of acquiring the missing ledger data by the consensus node of the missing ledger data is improved, and then the consensus node can participate in new consensus as early as possible.

Description

Method, apparatus, device, storage medium and program product for data processing
Technical Field
Example embodiments of the present disclosure relate generally to the field of computers and, more particularly, relate to methods, apparatuses, devices, computer-readable storage media and computer program products for data processing.
Background
Blockchains are a completely new distributed infrastructure and computing paradigm that use block-chained data structures to validate and store data, distributed node consensus algorithms to generate and update data, cryptography to secure data transfer and access control, and intelligent contracts composed of automated script code to program and manipulate data. Currently, blockchain technology is widely used in the fields of finance, the internet of things, supply chains and the like.
A consensus mechanism is used to ensure the correctness of the data, so that all nodes agree on the data and prevent malicious nodes from submitting false data. The consensus mechanism means that one transaction needs to be verified by a sufficient number of consensus nodes, and the transaction is recorded and synchronized to all the consensus nodes in the consensus network under the condition of no error in verification. The consensus node generates a block (block) based on consensus data (e.g., transaction data resulting from transactions) for which consensus is complete. A plurality of blocks in the consensus node are linked to form a blockchain. Each blockchain in the consensus node may be considered a distributed ledger. The data described by each block in the blockchain may be considered ledger data.
In practical applications, account book data may be missing (also referred to as account book data lag) at a certain common node due to network reasons, equipment failures, and the like. If a consensus node missing ledger data wants to continue to participate in a new consensus, it is necessary to synchronize the missing ledger data from other consensus nodes.
Disclosure of Invention
In a first aspect of the present disclosure, a method for data processing at a consensus node missing ledger data is provided. The method comprises the following steps: obtaining block information of the latest block stored by other consensus nodes in a consensus network, wherein the consensus network comprises the consensus node with missing account book data and other consensus nodes, and the block information comprises block height; determining a group of target consensus nodes according to the block information of the latest block stored in the rest consensus nodes, wherein the block information of the latest block stored in the group of target consensus nodes is consistent; determining a plurality of target blocks for acquiring missing ledger data by comparing the block heights of the locally stored latest blocks with the block heights of the latest blocks stored in a set of target consensus nodes; and concurrently acquiring ledger data in a plurality of target blocks from some or all of a set of target consensus nodes, ledger data obtained from different target blocks from different target consensus nodes.
In a second aspect of the present disclosure, an apparatus for data processing at a consensus node of missing ledger data is provided. The device comprises: the block information acquisition module is configured to acquire block information of the latest block stored by other consensus nodes in the consensus network, wherein the consensus network comprises the consensus node with missing account book data and other consensus nodes, and the block information comprises block heights; the target consensus node determining module is configured to determine a group of target consensus nodes according to the block information of the latest block stored in the rest of consensus nodes, and the block information of the latest block stored in the group of target consensus nodes is consistent; a target block determination module configured to determine a plurality of target blocks for acquiring missing ledger data by comparing a block height of a locally stored latest block with a block height of a latest block stored in a set of target consensus nodes; and a ledger data acquisition module configured to concurrently acquire ledger data in a plurality of target blocks from some or all of the target consensus nodes in the set of target consensus nodes, the ledger data obtained from different target consensus nodes being from different target blocks.
In a third aspect of the present disclosure, an electronic device is provided. The electronic device comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the electronic device to perform the method of the first aspect of the disclosure.
In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer readable storage medium has stored thereon a computer program executable by a processor to perform the method according to the first aspect of the present disclosure.
In a fifth aspect of the present disclosure, a computer program product is provided. The computer program product comprises computer executable instructions which, when executed by a processor, implement a method according to the first aspect of the present disclosure.
It should be understood that what is described in this summary is not intended to limit the critical or essential features of the embodiments of the disclosure nor to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The above and other features, advantages, and aspects of various implementations of the present disclosure will become more apparent hereinafter with reference to the following detailed description in conjunction with the accompanying drawings. In the drawings, wherein like or similar reference numerals denote like or similar elements, in which:
FIG. 1 illustrates a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented;
FIG. 2 illustrates a flow chart of a process for data processing according to some embodiments of the present disclosure;
FIG. 3 illustrates a flow chart of a state transfer protocol according to some embodiments of the present disclosure;
FIG. 4 illustrates an exemplary process diagram for ledger and consensus synchronization in accordance with some embodiments of the present disclosure;
FIG. 5 illustrates a block diagram of an apparatus for data processing according to some embodiments of the present disclosure; and
fig. 6 illustrates a block diagram of an electronic device in which one or more embodiments of the disclosure may be implemented.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the embodiments of the present disclosure have been illustrated in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather, embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
In describing embodiments of the present disclosure, the term "comprising" and its like should be taken to be open-ended, i.e., including, but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions are also possible below.
As mentioned above, if a consensus node missing ledger data wants to continue to participate in a new consensus, it needs to synchronize the missing ledger data from other consensus nodes. In the current scheme, the consensus node of the missing ledger data synchronizes the missing ledger data from a consensus node with complete ledger data. The synchronization missing ledger data generally refers to ledger data in blocks where two consensus nodes differ. But this scheme is suitable for the scene that the difference of the blocks between the consensus nodes of the missing ledger data and the consensus nodes of the complete ledger data is less, and no running transaction needs to be consensus and blocks are generated. However, if there are more blocks with phase difference and new transactions need to be continuously identified, the identified node of the missing ledger data may be always in the state of missing ledger data, so that the identified node cannot be quickly added into the identification of the new transactions.
Example Environment and basic operating principles
Example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
FIG. 1 illustrates a schematic diagram of an example environment in which embodiments of the present disclosure may be implemented. The environment includes a consensus network 100. The consensus network is a network of consensus nodes in a blockchain. A synchronous network (not shown) may also be included in the blockchain network. The synchronization network is a network of synchronization nodes (not shown in the figure). In the example of fig. 1, consensus network 100 includes consensus node 110, consensus node 120, consensus node 130, and consensus node 140. The disclosed embodiments do not limit the number of consensus nodes in the consensus network 100.
The consensus node may complete the consensus of the transaction or information, etc., using any suitable consensus algorithm, such as the bayer consensus algorithm. The consensus node may also generate a block (block) based on consensus data (e.g., transaction data generated by a transaction) for which the consensus is complete. The generated blocks are then processed to link the blocks, e.g., to link the generated blocks into locally stored blocks. In the example of fig. 1, a plurality of consensus nodes in consensus network 100 each store a generated block, and links between the blocks result in a blockchain. For example, the block chain 115 obtained by linking a plurality of blocks is stored in the consensus node 110, the block chain 125 obtained by linking a plurality of blocks is stored in the consensus node 120, the block chain 135 obtained by linking a plurality of blocks is stored in the consensus node 130, and the block chain 145 obtained by linking a plurality of blocks is stored in the consensus node 140.
The block is a data packet carrying transaction data, and is a data structure marked with a time stamp and a hash value corresponding to a preceding block. The hash value of a block may be used as a unique identification of the block. The block includes a block header and a block body. The block header may record meta information of the current block, such as the current version number, hash value corresponding to the previous block, timestamp, random number, hash value of the merck root, etc. The block may record detailed data generated over a period of time, including all transaction records or other information generated during the creation of the block for which the current block is verified. As mentioned previously, blockchains in consensus nodes may be considered a distributed ledger. The data stored in the block may be regarded as ledger data.
The position of a block in a blockchain may be represented by the block height of the block. The first block in the blockchain is the creative block. The height of the creative block is generally defined as 0. The height of the rest blocks is higher than that of the created blocks, namely, a few blocks higher than the created blocks. Blocks may be numbered, e.g., block 0, block 1, block N (N is a positive integer), etc. The number of the created block is usually number 0, and the numbers of the remaining blocks are sequentially ordered. In general, the number of a block may represent the height of the block.
The consensus nodes in the consensus network 100 are in a state of normal operation, and the blockchains stored by the consensus nodes are consistent. The blockchain consistency includes the number of blocks, the height of the latest block, and the hash value of the latest block. In the event that a particular consensus node lacks ledger data, for example, the consensus node fails and needs to be restarted, the consensus node needs to be consistent with ledger data of other consensus nodes to participate in the new consensus. In this case, the consensus node missing ledger data needs to obtain ledger data from other consensus nodes in consensus network 100.
In some embodiments of the present disclosure, the consensus node of the missing ledger data may concurrently obtain the missing ledger from a plurality of other consensus nodes satisfying the condition. This process is also referred to hereinafter as the synchronization of ledger data (simply "ledger synchronization"). The specific implementation of this process is described in detail below, and is not described here.
In some embodiments of the present disclosure, the synchronization of ledger data and the synchronization of consensus data may also be achieved concurrently if other consensus nodes in the consensus network complete a new consensus during the synchronization of ledger data. The specific implementation of this process is described in detail below, and is not described here.
The consensus network 100 may be implemented using any suitable network. As one example, consensus network 100 is implemented in a private network environment. There are data connections between the consensus nodes in the consensus network 100, through which data can be transmitted. The data connection may take any suitable form, for example, the consensus nodes may be directly or indirectly connected by a wired communication method, may be directly or indirectly connected by a wireless communication method, and may also be connected by other connection methods, which is not limited in this disclosure.
Each consensus node in the consensus network 100 has its corresponding node identification. The node identification may be an Internet Protocol (IP) address of an interconnection between networks, and any other information that may be used to identify nodes in a consensus network. Each consensus node may store node identifications of other consensus nodes connected to itself, such that acquired data or generated blocks may be broadcast to other consensus nodes based on the node identifications of the other consensus nodes. For example, the consensus node 110 may maintain a list of node identities that stores node names and node identities of other consensus nodes.
In the example of fig. 1, the consensus node in consensus network 100 may be any type of device with computing capabilities, including a terminal device or a server device. The terminal device may be any type of mobile terminal, fixed terminal, or portable terminal, including a mobile handset, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal Communication System (PCS) device, personal navigation device, personal Digital Assistant (PDA), audio/video player, digital camera/camcorder, positioning device, television receiver, radio broadcast receiver, electronic book device, game device, or any combination of the preceding, including accessories and peripherals for these devices, or any combination thereof. The server devices may include, for example, computing systems/servers, such as mainframes, edge computing nodes, electronic devices in a cloud environment, and so forth.
Example procedure
Fig. 2 illustrates a flow chart of a process 200 for data processing according to some embodiments of the present disclosure. Process 200 may be implemented at any consensus node in consensus network 100 where ledger data is missing. The process 200 may be performed when each consensus node is restarted. The following description will take a first consensus node missing ledger data as an example.
At block 210, the first consensus node obtains block information for the most recent block stored by the remaining consensus nodes in the consensus network.
A consensus network in an embodiment of the present disclosure includes a plurality of consensus nodes. The plurality of consensus nodes include a consensus node missing ledger data and the remaining consensus nodes. The number of consensus nodes missing ledger data may be one or more, and the disclosure is not limited.
The latest block in the embodiment of the present disclosure refers to the block that was recently generated according to the generation time of the block. That is, the block is generated at the time closest to the current time. The block information may include a block height, such as a number of blocks. Further, the block information may also include a block identification, such as a hash value of the block.
At block 220, the first consensus node determines a set of target consensus nodes based on the block information.
The first consensus node needs to determine a consensus node (abbreviated as "ledger data complete node") storing complete ledger data according to the obtained block information. The blockchains stored by the plurality of consensus nodes storing the complete ledger data are consistent. The blockchain consistency may be the block information consistency of the latest block. Thus, the first consensus node may determine such a set of target consensus nodes that satisfy the condition from the block information obtained in block 210.
Specifically, the block information consistency of the latest block may be a block height consistency of the latest block. For example, the number of the latest blocks is 5, indicating that the common nodes store blocks 0 through 5. The block information of the latest block is consistent, and the identification of the latest block is consistent. For example, the hash values of the latest blocks are identical, indicating that the latest blocks are the same block. In this case, the blocks are typically generated by a consensus on the same transaction. Of course, the block information of the latest block is consistent, and the height and the identification of the latest block can be consistent.
As will be illustrated below in connection with consensus network 100 in fig. 1. Referring to fig. 3, the consensus network 100 includes a consensus node 110, a consensus node 120, a consensus node 130, and a consensus node 140. The four consensus nodes are mutually connected in a communication way, so that consensus can be achieved for new transactions at any time. Assume that the latest block heights of the consensus nodes 110, 120, 140 are set to M (M is a positive integer), and belong to the ledger-data complete node. And the consensus node 130 loses account book data due to a period of time of shutdown caused by a fault in the operation process, that is, there is a situation that the account book data is behind.
In this scenario, since consensus node 110, consensus node 120, and consensus node 140 are all in a normal operating state, a reboot is not required. And consensus node 130 needs to be restarted.
Referring to fig. 3, during the restart, the consensus node 130 may broadcast a message to other consensus nodes in the consensus network 100 that inquires about ledger update information (e.g., block height of the latest block, block hash (hash) of the latest block, latest consensus ID). I.e., process 310 in fig. 3 (query consensus ID). Accordingly, as in process 320 of fig. 3 (replying to the consensus ID), after receiving the query message from consensus node 130, consensus node 110, consensus node 120, consensus node 140 returns the local up-to-date ledger information as a response to consensus node 130.
Further, the consensus node 130 node parses the latest account information from other nodes, and adds the consensus node having the same latest block height and block hash to the data complete node set. Assuming here that consensus node 110, consensus node 120, and consensus node 140 return the latest ledger information with consistent block height, block hash, then consensus node 110, consensus node 120, consensus node 140 are added to the data complete node set.
Next, the first consensus node determines a height difference between the block stored by itself and the block stored by the ledger data completion node. Then, the first consensus node may acquire ledger data in the block of phase differences from the plurality of ledger data complete nodes.
With continued reference to fig. 2. At block 230, the first consensus node determines a plurality of target blocks for acquiring missing ledger data by comparing the block heights of the locally stored latest blocks with the block heights of the latest blocks stored in the set of target consensus nodes determined at block 220.
For example, the latest block stored locally by consensus node 130 is block 0, and the latest block in these consensus nodes is block 4 based on the latest ledger information of ledger data complete nodes (e.g., consensus node 110, consensus node 120, and consensus node 140). It is apparent that there is a difference in height between the blocks stored by consensus node 130 and the ledger data completion node. In this example, the blocks of phase difference between consensus node 130 and the ledger data complete node are block 1, block 2, block 3, and block 4. The consensus node 130 may treat these blocks as target blocks.
After determining the target block, the first consensus node may concurrently acquire missing ledger data from the plurality of ledger data complete nodes, that is, ledger data stored in the target block.
At block 240, the first consensus node concurrently obtains ledger data in a plurality of target blocks from some or all of a set of target consensus nodes.
In the disclosed embodiment, the ledger data obtained by the first consensus node from different target consensus nodes comes from different target blocks. The ledger data in these different target blocks constitute ledger data that is missing at the first consensus node.
In some embodiments, the ledger data acquired by the first consensus node from the target consensus node may be data prior to being performed a predetermined operation, such as transaction data prior to performing a transaction. In this case, the first consensus node obtains the data and then locally performs a predetermined operation to generate a block. In some embodiments, the ledger data acquired by the first consensus node from the target consensus node may also be data for which a predetermined operation has been performed. For example, ledger data stored in a KV structure is used. The first consensus node obtains the data, and then directly generates and stores the block.
Concurrency may also be understood as parallelism in the disclosed embodiments.
Block 240 may also be understood that the first consensus node may obtain ledger data in a plurality of target blocks from at least a portion of the target nodes concurrently. In this way, the missing ledger data of the first consensus node is quickly synchronized to the local, and the first consensus node can then quickly participate in the new consensus. Referring to fig. 3, in the example of fig. 3, block 240 may be represented as two processes, a request state 330 and a reply state 340. The request state may be understood as a send request message hereinafter. The reply state may be understood as reply ledger data below.
Still taking consensus node 110, consensus node 120, and consensus node 140 as complete nodes of ledger data, consensus node 130 is a node missing ledger data, and target blocks are block 1, block 2, block 3, and block 4 as examples. Common node 130 may obtain ledger data in block 1 from common node 110, ledger data in block 2 from common node 120, and ledger data in block 4 from common node 140. The process of obtaining ledger data for these three blocks may all be concurrent. For example, ledger data in blocks 1-4 are acquired concurrently. The process of obtaining ledger data for these three blocks may also be partially concurrent. For example, ledger data in block 1 and block 2 are acquired concurrently. Then, the ledger data in block 3 and block 4 are further acquired. How to concurrency can be set according to specific situations (such as network conditions and processing performance of nodes, etc.), which is not described herein.
Some possible implementations of the first consensus node concurrently acquiring ledger data in multiple target blocks are described below.
In some embodiments, the first consensus node indicates which target block or blocks each target consensus node is to transmit ledger data in, respectively. For example, the first consensus node may send a request message to each target consensus node separately, where the request message corresponding to a given target consensus node includes the location of the target block in the blockchain, e.g., the height or number of the target block, etc. The request message is used for indicating account book data of the target block for transmitting the position indication. In addition, the locations in different request messages corresponding to different target consensus nodes are different, i.e. the ledger data that different target nodes need to provide come from different target blocks.
Thus, after each target consensus node receives the request message from the first consensus node, the ledger data in the target block indicated by the request message can be sent to the first consensus node concurrently. Accordingly, the first consensus node may concurrently receive ledger data for the target blocks indicated by the request message from each target consensus node.
In some embodiments, the first consensus node may instruct each target consensus node to send ledger data for one or more given target blocks. The plurality of target block ledger data synchronization tasks may be uniformly divided into the target consensus nodes, or the tasks may be unevenly divided into the target consensus nodes. The uniformity or non-uniformity here is for the number of target blocks. For example, the set of target consensus nodes may include a first target consensus node and a second target consensus node. It will be appreciated that a set of target consensus nodes may also include more consensus nodes, such as a third target consensus node, a fourth target consensus node, etc.
In this case, the first consensus node may send a first request message to the first target consensus node, the first request message including a first number of different locations therein. The first consensus node may also send a second request message to the second target consensus node, the second request message including a second number of different locations, the second number and the first number may be the same or different. For example, if consensus node 130 is a first consensus node missing ledger data, consensus node 110 is a first target consensus node, consensus node 120 is a second target consensus node, and target blocks are block 1, block 2, block 3, and block 4. The consensus node 130 may send a first request message to the consensus node 110, the first request message including the locations of block 1 and block 2, i.e. ledger data indicating that the consensus node 110 sent both blocks. Accordingly, the consensus node may obtain ledger data in block 1 and block 2 from consensus node 110. Consensus node 130 may also send a second request message to consensus node 120 that includes the locations of block 3 and block 4, i.e., ledger data that instructs consensus node 120 to send both blocks. Accordingly, consensus node 130 obtains ledger data in block 3 and block 4 from consensus node 120. In this example, the first request message and the second request message each include the locations of two blocks. In other examples, the number of locations carried in the request message sent by the first consensus node to each target consensus node may also be different. The present disclosure does not limit the number of locations in each request message and the implementation form of the locations.
In some embodiments, the first consensus node may obtain its performance parameters, such as processor (CPU) resource occupancy, storage resource occupancy, bandwidth, etc., from the respective target consensus node. The first consensus node may determine which target consensus node provides ledger data for which target block or blocks based on the obtained performance parameters of the respective target consensus node. The first consensus node may instruct the better performing target consensus node to provide ledger data for more target blocks. In particular, the first consensus node may also receive a first performance parameter from the first target consensus node, the first performance parameter being indicative of a performance of the first target consensus node. Similarly, the first consensus node may also receive a second performance parameter from the second target consensus node, the second performance parameter being indicative of a performance of the second target consensus node. If it is determined that the performance of the first target consensus node is better than the performance of the second target consensus node based on the first performance parameter and the second performance parameter, the first consensus node may determine that the first number is greater than the second number.
For example, if consensus node 130 is a first consensus node missing ledger data, consensus node 110 is a first target consensus node, consensus node 120 is a second target consensus node, and target blocks are block 1, block 2, block 3, and block 4. The consensus node 130 may obtain performance parameters of the consensus node 110 and the consensus node 120, respectively, and if it is determined that the performance of the consensus node 110 is better than the performance of the consensus node 120, the consensus node may instruct the consensus node 110 to provide ledger data for more target blocks. For example, the request message sent by consensus node 130 to consensus node 110 may include the locations of block 1, block 2, and block 3, i.e., ledger data that instructs consensus node 110 to provide the three blocks. And the request message sent by consensus node 130 to consensus node 120 may include the location of block 4, i.e., ledger data that instructs consensus node 120 to provide this one block.
As mentioned previously, the location of the target block in the blockchain may be the height of the target block. Moreover, for a new block to be generated for a new transaction, the generation of the new block depends on the previous block adjacent thereto. For example, the hash of the new block depends on the ledger data stored in the previous block. In order for the first consensus node to quickly participate in the new consensus, the highest target block may be preferentially synchronized, i.e. the latest target block is generated. In this way, it is possible to participate in the new consensus based on the ledger data in the target block with the greatest height having been obtained.
In some embodiments, the first consensus node may preferentially obtain ledger data in the highest-altitude target block from a third target consensus node (which may be the same as or different from the first target consensus node, the second target consensus node mentioned earlier). For the remaining target blocks of the plurality of target blocks except the target block with the largest height, the first consensus node may obtain ledger data in the remaining target blocks of the plurality of target blocks except the target block with the largest height from a set of target consensus nodes (which may or may not include a third target consensus node).
In some embodiments, if a consensus node in the consensus network completes a new consensus in the process of acquiring ledger data in a plurality of target blocks, the first consensus node caches the consensus data associated with the consensus.
Further, the first consensus node may further perform a predetermined operation (e.g., a transaction operation) in a consensus sequence corresponding to consensus data (e.g., transaction data) based on the cached consensus data after acquiring the ledger data in the target block having the greatest height. Further, if the predetermined operation is performed, a new block for storing ledger data generated after the predetermined operation is performed may be generated.
Referring to fig. 4, consensus node 110, consensus node 120, and consensus node 140 are ledger data complete nodes. Consensus node 130 obtains ledger data in block 3 from consensus node 110. Consensus node 130 obtains ledger data in block 1 and block 2 from consensus node 120. Consensus node 130 obtains ledger data in block 4 from consensus node 140. In synchronizing ledger data in blocks 1-4 from these ledger data complete nodes by consensus node 130 (e.g., 410 ledger concurrence in fig. 4), new transactions may be continually being sent into the consensus network. The consensus node 110, the consensus node 120, and the consensus node 140 may consensus the new business transaction and generate a new block. For consensus node 130, if it has not completed the synchronization of ledger data in block 4, consensus node 130 may first cache the consensus data associated with the consensus for the new business transaction, i.e., the synchronization of ledger data and the storage of consensus data may be concurrent (i.e., the 420 ledger in fig. 4 is synchronized with the consensus). However, the consensus node 130 has not been able to perform predetermined operations on the consensus data, such as transactions.
If the consensus node 130 has completed synchronizing the ledger data in the block 4, it is not necessary to wait for the ledger data in the block 3 to be synchronized, and the consensus data (e.g. transaction data) can be fetched from the cache, and executed in the consensus sequence (e.g. transaction sequence) of other nodes to perform transaction operations, and a new block is generated after the execution of the operations, that is, blocks 5 to N-1 are generated. At this time, the block heights of the four nodes all reach N-1. The new business transaction can be commonly participated in consensus, and a new block N is generated and stored for account book data.
In the embodiment of the disclosure, during the process of concurrent synchronization of ledger data, consensus data associated with the new consensus may be cached. That is, the account book data and the common identification data are stored in parallel and synchronized, and when the account book data is synchronized, the cached common identification data can be executed, and then a new block is generated. Thus, the consensus node missing the ledger data can quickly join in new consensus.
Example apparatus
Fig. 5 illustrates a block diagram of an apparatus 500 for data processing according to some embodiments of the present disclosure. The apparatus 500 may be implemented in a consensus node of the consensus network 100. The various modules/components in apparatus 500 may be implemented in hardware, software, firmware, or any combination thereof.
The apparatus 500 includes a block information obtaining module 510 configured to obtain block information of a latest block stored by remaining consensus nodes in a consensus network, the consensus network including the consensus node and the remaining consensus nodes of the missing ledger data, the block information including a block height. The apparatus 500 further comprises a target consensus node determination module 520 configured to determine a set of target consensus nodes based on the block information of the latest block stored in the remaining consensus nodes, the block information of the latest block stored in the set of target consensus nodes being identical. Apparatus 500 further includes a target tile determination module 530 configured to determine a plurality of target tiles for acquiring missing ledger data by comparing the tile heights of the locally stored latest tiles with the tile heights of the latest tiles stored in a set of target consensus nodes. Apparatus 500 further includes a ledger data acquisition module 540 configured to concurrently acquire ledger data in a plurality of target blocks from some or all of a set of target consensus nodes, ledger data obtained from different target consensus nodes from different target blocks.
In some embodiments, apparatus 500 further includes a caching module configured to cache consensus data associated with the consensus if a consensus node in the consensus network completes a new consensus during the course of the ledger data acquisition module 540 acquiring ledger data in the plurality of target blocks.
In some embodiments, ledger data acquisition module 540 is further configured to send a request message to each target consensus node, respectively, the request message corresponding to a given target consensus node including a location of the target block in the blockchain, the request message being for ledger data indicating the target block for which the location indication was sent, the location being different in different request messages corresponding to different target consensus nodes; and concurrently receiving ledger data for the target blocks indicated by the request message from each target consensus node.
In some embodiments, the set of target consensus nodes includes a first target consensus node and a second target consensus node. The ledger data acquisition module 540 is further configured to send a first request message to the first target consensus node, the first request message including a first number of different locations; and sending a second request message to the second target consensus node, the second request message including a second number of different locations, the second number being the same as or different from the first number.
In some embodiments, the apparatus 500 further comprises a quantity determination module configured to receive a first performance parameter from the first target consensus node, the first performance parameter being indicative of a performance of the first target consensus node; receiving a second performance parameter from the second target consensus node, the second performance parameter being indicative of a performance of the second target consensus node; and if the performance of the first target consensus node is determined to be better than the performance of the second target consensus node based on the first performance parameter and the second performance parameter, determining that the first number is greater than the second number.
In some embodiments, the location is the height of the target block, and ledger data acquisition module 540 is further configured to preferentially acquire ledger data in the target block with the greatest height from the third target consensus node; and acquiring account book data in the rest target blocks except the target block with the largest height from a group of target consensus nodes.
In some embodiments, the apparatus 500 further includes a block generation module configured to perform a predetermined operation in a consensus sequence corresponding to the consensus data based on the cached consensus data after acquiring the ledger data in the target block having the greatest height; and generating a new block for storing ledger data generated after the predetermined operation is performed in response to the predetermined operation being performed.
In some embodiments, the tile information further includes a tile identifier.
The elements included in apparatus 500 may be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and/or firmware, such as machine executable instructions stored on a storage medium. In addition to or in lieu of machine-executable instructions, some or all of the elements in apparatus 500 may be at least partially implemented by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standards (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.
Fig. 6 illustrates a block diagram of an electronic device 600 in which one or more embodiments of the disclosure may be implemented. It should be understood that the electronic device 600 illustrated in fig. 6 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 600 shown in fig. 6 may be used to implement the electronic device 130 or the model training device 125 of fig. 1.
As shown in fig. 6, the electronic device 600 is in the form of a general-purpose electronic device. The components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to programs stored in the memory 620. In a multiprocessor system, multiple processing units concurrently execute computer-executable instructions to increase the concurrent processing capability of electronic device 600.
The electronic device 600 typically includes a number of computer storage media. Such a medium may be any available medium that is accessible by electronic device 600, including, but not limited to, volatile and non-volatile media, removable and non-removable media. The memory 620 may be volatile memory (e.g., registers, cache, random Access Memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 may be a removable or non-removable media and may include machine-readable media such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (e.g., training data for training) and may be accessed within electronic device 600.
The electronic device 600 may further include additional removable/non-removable, volatile/nonvolatile storage media. Although not shown in fig. 6, a magnetic disk drive for reading from or writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data medium interfaces. Memory 620 may include a computer program product 625 having one or more program modules configured to perform the various methods or acts of the various embodiments of the disclosure.
The communication unit 640 enables communication with other electronic devices through a communication medium. Additionally, the functionality of the components of the electronic device 600 may be implemented in a single computing cluster or in multiple computing machines capable of communicating over a communication connection. Thus, the electronic device 600 may operate in a networked environment using logical connections to one or more other servers, a network Personal Computer (PC), or another network node.
The input device 650 may be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 660 may be one or more output devices such as a display, speakers, printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., with one or more devices that enable a user to interact with the electronic device 600, or with any device (e.g., network card, modem, etc.) that enables the electronic device 600 to communicate with one or more other electronic devices, as desired, via the communication unit 640. Such communication may be performed via an input/output (I/O) interface (not shown).
According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that are executed by a processor to implement the method described above.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable apparatus for data processing, thereby producing a machine, such that the instructions, when executed by the processing unit of the computer or other programmable apparatus for data processing, produce an apparatus that implements the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, apparatus programmable data processing, and/or other devices to function in a particular manner, such that the computer-readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable apparatus for data processing, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus for data processing, or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus for data processing, or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of implementations of the present disclosure has been provided for illustrative purposes, is not exhaustive, and is not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various implementations described. The terminology used herein was chosen in order to best explain the principles of each implementation, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand each implementation disclosed herein.

Claims (11)

1. A method for data processing at a consensus node of missing ledger data, comprising:
obtaining block information of the latest block stored by other consensus nodes in a consensus network, wherein the consensus network comprises the consensus node of the missing ledger data and the other consensus nodes, and the block information comprises block height;
determining a group of target consensus nodes according to the block information, wherein the block information of the latest block stored in the group of target consensus nodes is consistent;
determining a plurality of target blocks for acquiring missing ledger data by comparing the block heights of the locally stored latest blocks with the block heights of the latest blocks stored in the set of target consensus nodes; and
And concurrently acquiring ledger data in the plurality of target blocks from some or all target consensus nodes in the group of target consensus nodes, the ledger data acquired from different target consensus nodes being from different target blocks.
2. The method of claim 1, further comprising:
and if the consensus nodes in the consensus network complete new consensus in the process of acquiring the account book data in the target blocks, caching the consensus data associated with the consensus.
3. The method of claim 1 or 2, wherein concurrently acquiring ledger data in the plurality of target blocks comprises:
respectively sending a request message to each target consensus node, wherein the request message corresponding to a given target consensus node comprises the position of a target block in a blockchain, and the positions in different request messages corresponding to different target consensus nodes are different, and the request message is used for indicating account book data of the target block for sending the position indication; and
and concurrently receiving account book data of the target block indicated by the request message from each target consensus node.
4. A method according to claim 3, the set of target consensus nodes comprising a first target consensus node and a second target consensus node, and the sending request messages to each of the target consensus nodes respectively comprising:
transmitting a first request message to the first target consensus node, wherein the first request message comprises a first number of different positions; and
and sending a second request message to the second target consensus node, wherein the second request message comprises a second number of different positions, and the second number is the same as or different from the first number.
5. The method of claim 4, further comprising:
receiving a first performance parameter from the first target consensus node, the first performance parameter being indicative of a performance of the first target consensus node;
receiving a second performance parameter from the second target consensus node, the second performance parameter being indicative of a performance of the second target consensus node; and
if the performance of the first target consensus node is determined to be better than the performance of the second target consensus node based on the first performance parameter and the second performance parameter, the first number is determined to be greater than the second number.
6. The method of any of claims 3-5, wherein the location is a height of the target tile, and the concurrently acquiring ledger data in the plurality of target tiles comprises:
preferentially acquiring account book data in the target block with the largest height from a third target consensus node; and
ledger data in the remaining target blocks of the plurality of target blocks excluding the target block having the largest height is acquired from the set of target consensus nodes.
7. The method of claim 6, further comprising:
in response to acquiring ledger data in the target block with the maximum height, performing a predetermined operation in a consensus sequence corresponding to the consensus data based on the cached consensus data; and
in response to the predetermined operation being performed, a new block is generated, the new block being used to store ledger data generated after the predetermined operation is performed.
8. The method of claim 1, the tile information further comprising a tile identification.
9. An apparatus for data processing at a consensus node of missing ledger data, the apparatus comprising:
the block information acquisition module is configured to acquire block information of the latest block stored by other consensus nodes in a consensus network, wherein the consensus network comprises the consensus node of the missing account book data and the other consensus nodes, and the block information comprises block heights;
The target consensus node determining module is configured to determine a group of target consensus nodes according to the block information of the latest block stored in the rest of consensus nodes, wherein the block information of the latest block stored in the group of target consensus nodes is consistent;
a target block determination module configured to determine a plurality of target blocks for acquiring missing ledger data by comparing a block height of a locally stored latest block with a block height of a latest block stored in the set of target consensus nodes; and
and the account book data acquisition module is configured to acquire account book data in the plurality of target blocks from part or all of the target consensus nodes in the group of target consensus nodes in a concurrent mode, wherein the account book data acquired from different target consensus nodes come from different target blocks.
10. An electronic device, comprising:
at least one processing unit; and
at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which when executed by the at least one processing unit, cause the electronic device to perform the method of any one of claims 1-8.
11. A computer readable storage medium having stored thereon a computer program executable by a processor to implement the method of any of claims 1 to 8.
CN202311570233.0A 2023-11-22 2023-11-22 Method, apparatus, device, storage medium and program product for data processing Pending CN117595997A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311570233.0A CN117595997A (en) 2023-11-22 2023-11-22 Method, apparatus, device, storage medium and program product for data processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311570233.0A CN117595997A (en) 2023-11-22 2023-11-22 Method, apparatus, device, storage medium and program product for data processing

Publications (1)

Publication Number Publication Date
CN117595997A true CN117595997A (en) 2024-02-23

Family

ID=89914651

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311570233.0A Pending CN117595997A (en) 2023-11-22 2023-11-22 Method, apparatus, device, storage medium and program product for data processing

Country Status (1)

Country Link
CN (1) CN117595997A (en)

Similar Documents

Publication Publication Date Title
CN110535969B (en) Data storage method, device, storage medium and equipment based on block chain network
CN110263035A (en) Data storage, querying method and device and electronic equipment based on block chain
US20190205314A1 (en) Method for parallel maintenance of data consistency
KR20120018178A (en) Swarm-based synchronization over a network of object stores
US10069942B2 (en) Method and apparatus for changing configurations
US6487678B1 (en) Recovery procedure for a dynamically reconfigured quorum group of processors in a distributed computing system
CN108616574B (en) Management data storage method, device and storage medium
EP4300323A1 (en) Data processing method and apparatus for blockchain network, computer device, computer readable storage medium, and computer program product
US6542929B1 (en) Relaxed quorum determination for a quorum based operation
US20070255823A1 (en) Method for low-overhead message tracking in a distributed messaging system
US20140059315A1 (en) Computer system, data management method and data management program
US20090049172A1 (en) Concurrent Node Self-Start in a Peer Cluster
US10474185B2 (en) Timestamp alignment across a plurality of computing devices
CN111522874B (en) Block chain consensus method, apparatus, computer device and storage medium
CN105069152B (en) data processing method and device
EP4198861A1 (en) Information processing method and apparatus for blockchain network, and device and storage medium
CN113238996A (en) Block chain data archiving method based on DHT, electronic equipment and storage medium
CN112015595B (en) Master-slave database switching method, computing device and storage medium
CN111061813B (en) Method, apparatus and computing device for data synchronization in blockchain network
CN111681011B (en) Data processing method, blockchain system, computer system and medium
US7933962B1 (en) Reducing reliance on a central data store while maintaining idempotency in a multi-client, multi-server environment
WO2023207529A1 (en) Data processing method and apparatus, device, medium, and product
CN117595997A (en) Method, apparatus, device, storage medium and program product for data processing
WO2023134160A1 (en) Blockchain network-based consensus method and apparatus, and electronic device and storage medium
CN115510161A (en) Data synchronization method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination