WO2022083690A1 - Procédé et appareil de gestion de données, ainsi que dispositif, support de stockage par ordinateur et programme - Google Patents

Procédé et appareil de gestion de données, ainsi que dispositif, support de stockage par ordinateur et programme Download PDF

Info

Publication number
WO2022083690A1
WO2022083690A1 PCT/CN2021/125290 CN2021125290W WO2022083690A1 WO 2022083690 A1 WO2022083690 A1 WO 2022083690A1 CN 2021125290 W CN2021125290 W CN 2021125290W WO 2022083690 A1 WO2022083690 A1 WO 2022083690A1
Authority
WO
WIPO (PCT)
Prior art keywords
cluster
sub
master node
node
data
Prior art date
Application number
PCT/CN2021/125290
Other languages
English (en)
Chinese (zh)
Inventor
焦宏宇
邱路达
丁易元
张若君
孙芮
邓虹雨
Original Assignee
深圳前海微众银行股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海微众银行股份有限公司 filed Critical 深圳前海微众银行股份有限公司
Publication of WO2022083690A1 publication Critical patent/WO2022083690A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/30—Decision processes by autonomous network management units using voting and bidding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
    • H04L41/0668—Management of faults, events, alarms or notifications using network fault recovery by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • This application relates to the distributed structure technology of financial technology (Fintech), and relates to, but is not limited to, a data management method, apparatus, electronic device, computer storage medium and computer program.
  • data management can be implemented based on a single-layer distributed architecture.
  • the single-layer distributed architecture is not suitable for super-large distributed clusters.
  • the premise of data management in a distributed cluster is to determine the master node through cluster election.
  • cluster election is performed based on a single-layer distributed architecture, each node must send election messages to other nodes in the cluster, which is likely to cause a message storm. , so that most of the traffic is occupied by the packets sent by the cluster election, which affects the transmission of normal packets.
  • the embodiments of the present application provide a data management method, apparatus, electronic device, computer storage medium and computer, which can solve the problem of message storm caused by electing a master node in the prior art.
  • the embodiment of the present application provides a data management method, the method includes:
  • the data storage architecture includes a parent cluster (Parent Cluster) and a plurality of sub-clusters (Child Cluster), each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, the parent cluster's
  • the node is the master node of the multiple sub-clusters, and the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • establishing a data storage architecture includes:
  • the data storage architecture is established based on the master node of each sub-cluster and the master node of the parent cluster.
  • the method further includes:
  • the health indicator of the master node of each subcluster is derived based on at least one of the following: the number of failures, the duration of at least one failure, and the length of time since the last failure.
  • the embodiment of the present application can accurately obtain the health index of the master node of each sub-cluster based on the fault information, thus, it is beneficial to accurately select a node with a higher health degree among the master nodes of the sub-cluster As the master node of the parent cluster,
  • selecting a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health degree indicators of the respective sub-clusters including:
  • a master node is selected as the master node of the parent cluster; wherein the first score
  • the value is a preconfigured value.
  • the selection of the master node of the parent cluster is not only based on the ID of the node, but the selection of the master node of the parent cluster can be based on comprehensive consideration of the score corresponding to the ID of the node and the health index.
  • the master node is beneficial to reduce the problem of frequent replacement of master nodes caused by frequent addition of nodes with larger ID scores to the cluster.
  • a master node is selected as the master node of the parent cluster ,include:
  • the master node of each sub-cluster Based on the data interaction between the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value ;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the embodiment of the present application can obtain the master node of the parent cluster through the election of the sub-cluster based on the second score value, and the second score value is not only related to the ID of the node, but also related to the health degree index of the node. Therefore, The embodiment of the present application can reduce the problem of frequent replacement of master nodes caused by nodes with larger ID scores frequently joining the cluster.
  • selecting a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health degree indicators of the respective sub-clusters including:
  • the master node of each sub-cluster Based on the data interaction of the master node of each sub-cluster, the master node of each sub-cluster acquires the first message of each sub-cluster, where the first message represents a join message (Join Message) of the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, among the master nodes of each sub-cluster, one master node is selected as the The master node of the parent cluster.
  • the embodiment of the present application can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the master node of the cluster can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • enabling the master node of each sub-cluster to obtain the first message of each sub-cluster based on the data interaction between the master nodes of each sub-cluster includes:
  • the master node (Seed Cluster Master) of the seed cluster receives the first message sent by the master node of other sub-clusters, it interacts with the data of the other sub-clusters through the seed cluster, so that the The master node obtains the first message of each sub-cluster; wherein, the other sub-clusters are sub-clusters of the multiple sub-clusters except the seed cluster; the first message is the master of the other sub-clusters.
  • the node is sent based on the address of the master node of the seed cluster, and the address of the master node of the seed cluster is information predetermined by the master nodes of the other sub-clusters.
  • the embodiment of the present application can realize the data interaction of the master nodes of each sub-cluster through the selection of seed clusters, and further, is conducive to selecting the master node of the parent cluster from the master nodes of each sub-cluster.
  • the method further includes:
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, it joins the master node of the first sub-cluster in the parent cluster, and sends a message of the parent cluster to each node of the parent cluster.
  • Member change information the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster;
  • the first message represents a message for joining the parent cluster.
  • the embodiment of the present application can update the member change information of the parent cluster in time when the slave node of the parent cluster fails, which is beneficial to accurately realize the master node of the parent cluster when the master node of the parent cluster is elected subsequently. election.
  • the method further includes:
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the second sub-cluster represents the sub-cluster to which the faulty master node belongs, and the first message represents the message of joining the parent cluster.
  • the main node information of the parent cluster can be re-elected in time, and the main node of the second sub-cluster can be received after the main node of the second sub-cluster is selected.
  • the join message is conducive to the subsequent re-election of the master node of the parent cluster.
  • the method further includes:
  • the writing data to each node of each sub-cluster based on the data storage architecture includes:
  • the master node of the parent cluster sends the data writing instruction to the master node of each sub-cluster, and the master node of each sub-cluster will The data writing instruction is sent to each slave node of the sub-cluster, so that each node of each sub-cluster writes the data to be written.
  • the embodiment of the present application can write the same data to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • the data to be written includes at least two levels of tag data
  • Writing the data to be written by each node of each sub-cluster includes:
  • a hash table is established for each level of label data, and a multi-fork tree data structure is constructed based on the hash table corresponding to each level of label data;
  • the at least two levels of label data are written into each node of each sub-cluster based on the multi-tree data structure.
  • the embodiment of the present application can implement hierarchical storage of multi-level tag data, which is beneficial to the subsequent management of multi-level tag data.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • the label data of the current level can be quickly queried to the label data of the next level.
  • the method further includes:
  • At least one of the following operations is performed on the at least two-level tag data: adding, deleting, modifying, and querying.
  • An embodiment of the present application provides a data management device, and the device includes:
  • the establishment module is configured to establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the nodes of the parent cluster are all the sub-clusters.
  • master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • the processing module is configured to write data to each node of each sub-cluster based on the data storage architecture.
  • An embodiment of the present application provides an electronic device, and the electronic device includes:
  • a memory configured to store executable instructions
  • the processor when configured to execute the executable instructions stored in the memory, implements any one of the above data management methods.
  • Embodiments of the present application provide a computer-readable storage medium storing executable instructions for implementing any of the foregoing data management methods when executed by a processor.
  • Embodiments of the present application further provide a computer program product, where the computer program product includes computer-executable instructions, where the computer-executable instructions are used to implement any one of the data management methods provided by the embodiments of the present application.
  • a data storage architecture is established, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the nodes of the parent cluster are The master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster; based on the data storage architecture, to each node of each sub-cluster data input.
  • the data storage architecture of the embodiment of the present application is a two-layer distributed architecture, which can realize data interaction based on the master node of each sub-cluster. It reduces the negotiation message data, limits the propagation range of negotiation messages, and helps to alleviate the problems that affect normal packet transmission caused by message storms.
  • FIG. 2 is a schematic diagram of a data storage architecture according to an embodiment of the present application.
  • Fig. 3 is a schematic diagram of cluster establishment in the embodiment of the present application.
  • FIG. 4 is a schematic diagram of electing a parent cluster master node in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a multi-tree data structure according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of an operation of adding data to at least two levels of tag data in an embodiment of the present application
  • FIG. 7 is a schematic flowchart of a data query operation on at least two levels of tag data in an embodiment of the present application
  • FIG. 8 is a schematic diagram of an optional composition structure of a data management apparatus according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an optional composition structure of an electronic device provided by an embodiment of the present application.
  • the single-layer distributed architecture is not suitable for super-large distributed clusters, because there are many super-large distributed cluster nodes, and the premise of data management based on super-large distributed clusters is to determine the master node through cluster election.
  • cluster election each node must send election messages to other nodes in the cluster, which can easily cause message storms, and cause most of the traffic to be occupied by the packets sent by the cluster election, affecting normal packet transmission.
  • the election algorithm does not balance the election speed and stability. For example, the bully algorithm, the cluster election is simply judged according to the size of the node ID, which will cause frequent elections
  • the problem of the master node; such as the raft algorithm, the majority voting mechanism is used for cluster election. This mechanism can avoid frequent election of the master, but it will cause the election time to become longer, and more than half of the nodes must vote to elect the master node. .
  • each node uses a unified Hash table structure for data storage.
  • the hash value of all key values (key) leads to a longer expansion time.
  • the embodiments of the present application provide a data management method, an apparatus, an electronic device, and a computer storage medium.
  • the data management methods of the embodiments of the present application can be applied to electronic devices.
  • the following describes exemplary applications of the electronic devices provided by the embodiments of the present application.
  • the electronic device provided by the embodiments of the present application may be implemented as a server, and the server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, Cloud servers for basic cloud computing services such as cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.
  • CDN Content Delivery Network
  • FIG. 1 is an optional flowchart of a data management method provided by an embodiment of the present application. As shown in FIG. 1 , the flowchart may include:
  • Step 101 establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, the node of the parent cluster is the master node of the multiple sub-clusters, and the parent cluster
  • the nodes include one master node of the parent cluster and at least one slave node of the parent cluster.
  • a sub-cluster represents a small cluster obtained by dividing the nodes of all clusters, each sub-cluster can independently elect a sub-cluster master node (Child Cluster Master), and the remaining nodes in the sub-cluster except the sub-cluster master node are Child Cluster Slave.
  • Child Cluster Master sub-cluster master node
  • the parent cluster represents a cluster composed of master nodes of each sub-cluster. You can select a node from the nodes of the parent cluster as the parent cluster master node (Parent Cluster Master), and the rest of the nodes in the parent cluster except the parent cluster master node are the parent cluster slave nodes. Node (Parent Cluster Slave).
  • FIG. 2 is a schematic diagram of a data storage architecture according to an embodiment of the present application.
  • sub-cluster 0, sub-cluster 1 and sub-cluster 2 represent three different sub-clusters
  • master node 01 represents the master node of sub-cluster 0, and slave Node 02 and slave node 03 represent different slave nodes in sub-cluster 0
  • master node 11 represents the master node of sub-cluster 1
  • slave node 12 and slave node 13 represent different slave nodes in sub-cluster 1
  • master node 21 represents the sub-cluster
  • master node of 2 slave node 22 and slave node 23 represent different slave nodes in sub-cluster 2
  • master node 01, master node 11 and master node 21 form the parent cluster
  • master node 21 is the master node of the parent cluster
  • master node 11 and The master node 01 is a different slave node in the parent cluster.
  • the data storage architecture of the embodiment of the present application is a two-layer structure, wherein the first layer is composed of multiple sub-clusters, and the sub-clusters of the first layer include sub-cluster 0, sub-cluster 1 and sub-cluster 2; the second layer represents The parent cluster consists of master nodes of multiple sub-clusters, that is, the parent cluster consists of master node 01 , master node 11 and master node 21 . After each sub-cluster is established, it will independently elect its own master node. Then, the master node of each sub-cluster can obtain the master ground of the parent cluster through election, so that one super-large cluster is realized through a two-layer structure.
  • Step 102 Write data to each node of each sub-cluster based on the data storage architecture.
  • the same data can be written to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • data can be read from any node because the data in each node is consistent.
  • steps 101 to 102 may be implemented based on a processor of an electronic device, and the above-mentioned processor may be an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital Signal Processing Device (Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), CPU, Controller, Microcontroller, Microprocessor at least one of them.
  • ASIC application specific integrated circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • Field Programmable Gate Array Field Programmable Gate Array
  • FPGA Field Programmable Gate Array
  • CPU Controller
  • Microcontroller Microprocessor at least one of them.
  • the data storage architecture of the embodiment of the present application is a two-layer distributed architecture, which can realize data interaction based on the master node of each sub-cluster. It reduces the negotiation message data, limits the propagation range of negotiation messages, and helps to alleviate problems that affect normal packet transmission caused by message storms.
  • establishing a data storage architecture may include: determining a master node and a health index of each sub-cluster; based on the health index of each sub-cluster, selecting a One master node is used as the master node of the parent cluster; the data storage architecture is established based on the master nodes of each sub-cluster and the master node of the parent cluster.
  • the health index is used to reflect the health of the master node of the sub-cluster; after determining the master node of each sub-cluster, the slave nodes of each sub-cluster can be determined, and after determining the master node of the parent cluster, the parent node can be determined.
  • the slave node of the cluster thus, the above data storage architecture can be obtained.
  • the embodiment of the present application can select the child nodes of the parent cluster based on the health index of the master node of each sub-cluster, which is similar to the bully algorithm.
  • the raft algorithm even if a node with a large ID is added frequently, it will not cause frequent replacement of the master node of the parent cluster, and the stability is stronger; and compared with the raft algorithm, it is necessary to elect the master node based on the half voting mechanism to determine the speed of the master node. Faster, to a certain extent, achieves a balance between election speed and stability.
  • the health index of the master node of each sub-cluster can be obtained according to at least one of the following: the number of failures, the duration of at least one failure, and the duration from the last failure to the current time.
  • the health index is negatively correlated with the number of failures
  • the health index is negatively correlated with the duration of at least one failure
  • the health index is the length of time from the last failure to the current time; that is, the longer the duration of at least one failure If it is short, the value of the health index is larger; the longer the time from the last fault to the current time, the larger the value of the health index; the greater the number of failures, the smaller the value of the health index.
  • the duration of the at least one failure may include the duration of the last failure.
  • failure start time and failure recovery time can be recorded to determine the failure duration.
  • the health index of the master node of each sub-cluster can be obtained according to the following formula (1):
  • S0 represents the health index of the master node of the sub-cluster
  • t1 represents the duration of the last failure from the current time
  • fault_count represents the number of failures
  • t2 represents the duration of the last failure.
  • the embodiment of the present application can accurately obtain the health index of the master node of each sub-cluster based on the fault information. Therefore, it is beneficial to accurately select a node with a higher health degree among the master nodes of the sub-cluster.
  • the master node of the parent cluster As the master node of the parent cluster,
  • one master node may be selected as the master node of the parent cluster among the master nodes of each sub-cluster; wherein, the first The scoring value is a preconfigured numerical value.
  • the first score value may be a score value corresponding to the ID of the node; in practical applications, the first score value may be predetermined, that is, the first score value of each node is a fixed value.
  • the selection of the master node of the parent cluster is not only based on the ID of the node, but the selection of the master node of the parent cluster can be based on comprehensive consideration of the score corresponding to the ID of the node and the health index.
  • the master node is beneficial to reduce the problem of frequent replacement of master nodes caused by frequent addition of nodes with larger ID scores to the cluster.
  • selecting a main node as the main node of the parent cluster among the main nodes of each sub-cluster may include:
  • the master node of each sub-cluster Based on the data interaction of the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the master node of each sub-cluster can calculate the second score value of this node, and then, through the data interaction of the master node of each sub-cluster, the second score value of the master node of other sub-clusters can be obtained, and then, The magnitude relationship between the second score values of the current node and the master nodes of other sub-clusters can be judged, and it can be judged whether the current node can serve as the master node of the parent cluster.
  • the second score value of this node is greater than or equal to the second score value of the master node of other sub-clusters, it means that this node can serve as the master node of the parent cluster; if the second score value of this node is less than The second score value of the master node of any other sub-cluster indicates that this node cannot serve as the master node of the parent cluster.
  • the embodiment of the present application can obtain the master node of the parent cluster through the election of the sub-cluster based on the second score value, and the second score value is not only related to the ID of the node, but also related to the health degree index of the node. Therefore, The embodiment of the present application can reduce the problem of frequent replacement of master nodes caused by nodes with larger ID scores frequently joining the cluster.
  • selecting a master node as the master node of the parent cluster may include:
  • the master node of each sub-cluster obtains the first message of each sub-cluster, where the first message represents the message of joining the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, one master node is selected as the master node of the parent cluster among the master nodes of each sub-cluster.
  • the master node of each sub-cluster after generating the first message, that is, the message of joining the parent cluster, can send the first message to the master nodes of other sub-clusters; the master node of each sub-cluster can also receive other sub-clusters.
  • the embodiment of the present application can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the master node of the cluster can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the above-mentioned data interaction based on the master node of each sub-cluster, enabling the master node of each sub-cluster to obtain the first message of each sub-cluster includes:
  • the master node of the seed cluster After the master node of the seed cluster receives the first message sent by the master node of other sub-clusters, it interacts with the data of other sub-clusters through the seed cluster, so that the master node of each sub-cluster obtains the first message of each sub-cluster; wherein, The other sub-clusters are sub-clusters of multiple sub-clusters except the seed cluster; the first message is sent by the master node of other sub-clusters based on the address of the master node of the seed cluster, and the address of the master node of the seed cluster is the master node of the other sub-clusters. Node predetermined information.
  • one sub-cluster may be arbitrarily selected from each sub-cluster as a seed cluster, or a seed cluster may be selected from each sub-cluster according to a preset selection method.
  • the address of the master node of the seed cluster may be pre-configured in the master nodes of other sub-clusters, so that the master nodes of other sub-clusters may send the first message to the seed cluster; the seed cluster receives the first message after receiving the first message. After the message is sent, the sending address of the first message, that is, the address of the master node of other sub-clusters can be recorded, and further, the seed cluster can send the received first message to the master node of each sub-cluster.
  • the embodiment of the present application can realize the data interaction of the master nodes of each sub-cluster through the selection of seed clusters, and further, is conducive to selecting the master node of the parent cluster from the master nodes of each sub-cluster.
  • FIG. 3 is a schematic diagram of cluster establishment in an embodiment of the application.
  • a seed cluster is selected first, and the seed cluster is a sub-cluster in each sub-cluster.
  • the seed cluster, sub-cluster 1 and sub-cluster 2 represent three different
  • the master node 01 represents the master node of the seed cluster
  • the slave node 02 and slave node 03 represent the slave nodes (Seed Cluster Slave) in the seed cluster;
  • the meaning of master node 11, slave node 12 and slave node 13 is the same as that of Figure 2
  • the meanings of the master node 21 , the slave node 22 and the slave node 23 are the same as those in FIG. 2 , and will not be repeated here.
  • the master node of the seed cluster can be selected, and then the IDs of all sub-clusters (including sub-cluster 1 and sub-cluster 2) at the initial moment can be written into the master node of the seed cluster, so that the seed cluster can know ID of each subcluster.
  • sub-clusters except the seed cluster can selectively select the master node, and the ID and address of the seed cluster can be pre-configured in other sub-clusters.
  • the master node of each sub-cluster can send a first message to the master node of the seed cluster (master node 01 shown in FIG. 3 ), where the first message includes the ID of the sub-cluster and the ID of the sub-cluster Information such as the address of the master node.
  • the master node of the seed cluster After receiving the first message sent by the other sub-cluster, the master node of the seed cluster can store the ID of the sub-cluster as the sub-cluster information according to the first message, and record the address of the master node of the corresponding sub-cluster.
  • the seed cluster After the seed cluster receives the first messages of all other sub-clusters, it can broadcast the sub-cluster information to the master node of each other sub-cluster, where the sub-cluster information may include the received first messages of all other sub-clusters, so that, Each sub-cluster can obtain the information of the master node of each sub-cluster.
  • the election process of the master node of the parent cluster can be initiated; here, in determining the election process of the master node that initiates the parent cluster, the seed cluster has no meaning and needs to be Fair competition with other subclusters.
  • the seed cluster is the coordinator at the initialization moment of the cluster establishment, and each sub-cluster can obtain the information of the master node of each sub-cluster and then degenerate into a sub-cluster.
  • FIG. 4 is a schematic diagram of electing a master node of a parent cluster in an embodiment of the application.
  • sub-cluster 0, sub-cluster 1 and sub-cluster 2 represent three different sub-clusters; master node 01, slave node 02, slave node 03 , the master node 11 , the slave node 12 , the slave node 13 , the master node 21 , the slave node 22 and the slave node 23 have the same meanings as in FIG. 2 , and will not be repeated here.
  • the master node of each sub-cluster first determines the second score value of the node, and can send a success (victory) message to the master nodes of other sub-clusters, for example, the master node 21 can send the master node 11 and master node 11 and the master node 01 to send a victory message; the victory message represents a message announcing that this node is the master node, and the victory message can carry the second score value of this node.
  • the master node of each sub-cluster can receive the victory message sent by the master node of other sub-clusters. After receiving the victory message, it compares the second score value in the victory message with the second score value of this node. The second score value of the node is greater than or equal to the second score value of this node, then an election message can be sent to the source node of the victory message. If the second score value in the message is less than the second score value, a response (alive) message may be sent to the source node of the victory message, where the alive message for the victory message is a message indicating re-election of the parent cluster master node.
  • the master node 21 sends the victory message to the master node 11 and the master node 01
  • the master node 11 can send the master node 11 to the master node.
  • 21 sends an election message; if the second score value in the victory message is greater than the second score value of the master node 01, the master node 01 may send an election message to the master node 21.
  • the master node of each sub-cluster can reply to the alive message.
  • the alive message for the election message indicates that the reply information of the election message has been received.
  • the master node 21 may send an alive message to the master node 11 and the master node 01, respectively.
  • the master node of each sub-cluster may not process the alive message and wait to receive messages sent by the master nodes of other sub-clusters.
  • the master node of any sub-cluster After the master node of any sub-cluster sends the master node of the victory to the master node of other sub-clusters, if it does not receive the alive message sent by the master node of other sub-clusters within a given time, it is considered that the master node is the parent cluster At this time, the master node of the other sub-clusters can send a victory message to the master nodes of other sub-clusters, so that the master nodes of other sub-clusters can determine the master node of the parent cluster. In this way, the election process of the master node of the parent cluster ends.
  • the given time may be set according to an actual application scenario, for example, the given time may be determined according to the maximum communication delay between each sub-cluster.
  • a node of the parent cluster fails, targeted processing needs to be performed, and two situations in which the parent node fails are respectively described below.
  • the slave node of the parent cluster fails, delete the failed slave node from the parent cluster, and use the master node of the parent cluster to send the member change information of the parent cluster to the slave node of the parent cluster;
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, the master node of the first sub-cluster is added to the parent cluster, and the member change information of the parent cluster is sent to each node of the parent cluster;
  • the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster.
  • the member change information of the parent cluster may include information such as IDs of each node of the parent cluster.
  • the master node of the parent cluster can delete the faulty slave node from the parent cluster, mark the ID of the corresponding child cluster as unavailable, and update the member information of the parent cluster, and then the parent cluster can be deleted. Membership change information is broadcast to each slave node of the parent cluster.
  • the master node of the first sub-cluster After the master node of the first sub-cluster fails, the master node can be re-selected in the first sub-cluster, and then the master node of the first sub-cluster can send the first message to the master node of the parent cluster; the master node of the parent cluster receives the first message. After the first message is received, the above-mentioned unavailable mark can be cleared, and then the member information of the parent cluster can be updated, and then the member change information of the parent cluster can be broadcasted to each slave node of the parent cluster.
  • the master node of the parent cluster may also update the failure information related to the first sub-cluster for use in electing the master node of the parent cluster.
  • the embodiment of the present application can update the member information of the parent cluster in time when the slave node of the parent cluster fails, and is conducive to accurately realizing the master node of the parent cluster when the election of the master node of the parent cluster is performed subsequently. election.
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the master node of the second sub-cluster After receiving the first message sent by the master node of the second sub-cluster, the master node of the second sub-cluster is added to the parent cluster, and the member change information of the parent cluster is sent to each node of the parent cluster; the second sub-cluster indicates The subcluster to which the failed master belongs.
  • the master node of the parent cluster fails, it is necessary to elect a node from each slave node of the parent cluster as the master node of the parent cluster, and the process of electing the master node of the parent cluster has been described above. The description is made in the content and will not be repeated here.
  • the master node of the second sub-cluster fails, it can be considered that the master node of the second sub-cluster is separated from the parent cluster. In this case, after the master node is selected from the second sub-cluster, it needs to join the parent cluster.
  • the specific process is as follows:
  • the master node of the parent cluster When the master node of the parent cluster fails, other nodes in the second sub-cluster can delete the information of the master node of the parent cluster, and re-select a node in the second sub-cluster as the master node of the second sub-cluster; After the master node of the second sub-cluster is selected, the master node of the second sub-cluster may send the first message to any node of the parent cluster; in some embodiments, a hash ring may be established for all nodes of the parent cluster in advance, and the first message may be The master node of the two sub-clusters may select a node adjacently in a clockwise or counterclockwise direction based on the hash ring, and send the first message to the selected node.
  • the master node of the parent cluster may directly record the ID of the second sub-cluster and the address information of the current master node of the second sub-cluster.
  • the node of any one of the parent clusters can return the master node information of the parent cluster to the current master node of the second sub-cluster, and the master node of the second sub-cluster
  • the node may send the first message to the master node of the parent cluster based on the master node information of the parent cluster.
  • the main node information of the parent cluster can be re-elected in time, and the main node of the second sub-cluster can be received after the main node of the second sub-cluster is selected.
  • the join message is conducive to the subsequent re-election of the master node of the parent cluster.
  • the above-mentioned writing data to each node of each sub-cluster based on the data storage architecture may include:
  • the data write command carries the data to be written
  • the master node of the parent cluster sends the data write instruction to the master node of each sub-cluster
  • the master node of each sub-cluster sends the data write instruction to the sub-cluster.
  • Each slave node enables each node of each sub-cluster to write the data to be written.
  • the data write instruction can be sent to any node in the parent cluster.
  • a node if the node receiving the data write command is not the master node of the parent cluster, the node receiving the data write command will reply the information of the master node of the parent cluster to the client, and then the client can send the data write command
  • the master node of the parent cluster can send the data write command to the master node of each sub-cluster; if the node receiving the data write command is the master node of the parent cluster, the master node of the parent cluster can write the data to the master node of the parent cluster.
  • Write commands are sent to the master node of each subcluster.
  • the master node of a sub-cluster when the master node of a sub-cluster determines that all nodes in the sub-cluster have successfully written the data to be written, it will return successful writing information to the master node of the parent cluster; otherwise, it will return unsuccessful writing information to the master node of the parent cluster; when the master node of the parent cluster receives the successful write information sent by the master nodes of each sub-cluster, it can return the successful write information to the client; when the master node of the parent cluster receives at least one When the master node of the sub-cluster sends the unsuccessful writing information, it can be determined that the node of the at least one sub-cluster is faulty, or it can be determined that the data has not been successfully written to the at least one sub-cluster. The master node of a sub-cluster sends a data write instruction until the successful write information sent by the master node of the at least one sub-cluster is received.
  • the embodiment of the present application can write the same data to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • a data read instruction can also be obtained, and the data read instruction carries the data read address of any node; then, based on the data read address, data can be read from any of the above nodes .
  • the above-mentioned data to be written includes at least two levels of tag data
  • writing the data to be written by each node of each sub-cluster includes: establishing a hash table for each level of label data, and building a multi-fork tree data structure based on the hash table corresponding to the label data at each level; A multi-tree data structure, the at least two levels of label data are written into each node of each sub-cluster.
  • micro-service technology has been widely used.
  • the characteristic of micro-service is dynamic.
  • IP Internet Protocol
  • a data management solution based on tag data is proposed instead of an IP-based data management solution.
  • the data to be written is application instance data
  • the application instance data may include the application cluster and the system to which the application belongs
  • the application instance data may be represented as /cluster/system/app, where the symbol / represents a fixed separator , cluster, system, and app represent clusters, systems, and applications, respectively, cluster represents the first-level label, system represents the second-level label, and app represents the third-level label.
  • FIG. 5 is a schematic diagram of a multi-fork tree data structure according to an embodiment of the present application.
  • hashtable1-0 represents a hash address of cluster data
  • bucket0 and bucket1 in hashtable1-0 represent different stored data.
  • the key is cluster0 and the value is hashtable1-0;
  • bucket1 in hashtable1-0 the key is cluster1 and the value is hashtable1-1.
  • hashtable1-0 and hashtable1-1 represent hash addresses of different system data
  • bucket0 and bucket1 in hashtable1-0 represent different storage data
  • the key is system0
  • the value is hashtable2-0
  • bucket1 in hashtable1-0 the key is system1
  • the value is hashtable2-1
  • bucket0 in hashtable1-1 represents the storage data
  • the key is system0
  • the value is hashtable2-2.
  • hashtable2-0, hashtable2-1 and hashtable2-2 respectively represent hash addresses of different application data; bucket0 in hashtable2-0 represents storage data, in bucket0 in hashtable2-0, the key is app0, The value is ip1; bucket0 and bucket1 in hashtable2-1 represent different storage data, in bucket0 in hashtable2-0, the key is app1 and the value is ip2; in bucket1 in hashtable2-0, the key is app2 and the value is ip2; in bucket0 in hashtable2-2, the key is app0 and the value is ip3; here, app0, app1, and app2 are different applications, and ip1, ip2, and ip3 represent different IP addresses.
  • label-to-IP conversion needs to be performed, and the actual physical-level communication can be completed by converting the logical concept label into physical concept IP when actually performing network-level communication; For example, referring to the third-level label data in FIG. 5 , a corresponding relationship between labels and IPs can be established, thus facilitating efficient storage and search of the corresponding relationships between labels and IPs.
  • each tree node in the multi-tree data structure is correspondingly established with a hash table, and the hash table is used to store corresponding first-level label data.
  • the embodiment of the present application can implement hierarchical storage of multi-level tag data, which is beneficial to the subsequent management of multi-level tag data.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • multi-level label data may be divided according to fixed separators, each level of label data corresponds to a tree node of a multi-tree data structure, and the tree node in the multi-tree data structure is not a leaf node (there is no child node), the key in the tree node is an element of the corresponding hash table, and the value is the hash address of the next-level hash table; here, the hash address of the next-level hash table is the above-mentioned lower hash table.
  • the hash address corresponding to the first-level tag data is divided according to fixed separators, each level of label data corresponds to a tree node of a multi-tree data structure, and the tree node in the multi-tree data structure is not a leaf node (there is no child node), the key in the tree node is an element of the corresponding hash table, and the value is the hash address of the next-level hash table; here, the hash address of the next-level hash table is the
  • the label data of the current level can be quickly queried to the label data of the next level.
  • Tag data with the same prefix string can share the same hash table node, which can reduce storage space usage.
  • bucket0 and bucket1 in hashtable2-1 have the same The prefix string, that is, the prefix strings of bucket0 and bucket1 in hashtable2-1 are both cluster0/system1; thus, bucket0 and bucket1 in hashtable2-1 share the same hash table node hashtable2-1.
  • the number of strings separated by a fixed separator is a fixed number, so the height of the multi-fork tree is also fixed, and the lookup time of each hash table is a constant level , the overall hash table-based query time complexity is also a constant level.
  • suffix wildcard matching such as /cluster0/system0/*
  • the embodiment of the present application does not need to traverse the hash table corresponding to the entire data to be written, which can further reduce the time complexity of suffix wildcard matching.
  • a hash table needs to be used uniformly to realize data storage. Therefore, when the data volume of the hash table is large, the more keys that need to be recalculated during expansion, the longer the expansion time; When data expansion is required, it is only necessary to expand the hash table of the node corresponding to the new element that needs to be placed, which reduces the expansion time to a certain extent.
  • At least one of the following operations may be performed on the at least two-level tag data: adding, deleting, modifying, and querying.
  • FIG. 6 is a schematic flowchart of an operation of adding data to at least two levels of label data in an embodiment of the present application. As shown in FIG. 6 , the process may include:
  • Step 601 Acquire label data at all levels.
  • the multi-level label data can be divided according to the fixed delimiter described above to obtain the label data of each level.
  • Step 602 Take out the first-level label.
  • cluster represents the first-level label
  • system represents the second-level label
  • app represents the third-level label
  • Step 603 Determine whether the hash table node corresponding to the currently fetched tag exists, if not, execute step 604; if yes, execute step 605.
  • Step 604 Create a corresponding hash table node, and perform step 605.
  • a corresponding hash table node can be created for the currently fetched tag.
  • Step 605 Determine whether there is a corresponding label in the hash table node, if not, go to step 606, and if so, go to step 607.
  • Step 606 insert the corresponding label, and then perform step 607 .
  • the currently fetched tag can be inserted into the hash table node, and then step 607 is performed.
  • Step 607 Determine whether the traversal of the label data at all levels is completed, if yes, end the process, if not, go to Step 608.
  • Step 608 take out the next-level label data, and then return to step 603 .
  • FIG. 7 is a schematic flowchart of a data query operation for at least two levels of tag data in an embodiment of the present application. As shown in FIG. 7 , the process may include:
  • Step 701 Acquire label data at all levels.
  • Step 702 Take out the first-level label.
  • Step 703 Determine whether the hash table node corresponding to the currently fetched tag exists, if yes, go to Step 704, if not, go to Step 708.
  • Step 704 Determine whether there is a corresponding label in the hash table node, if yes, go to Step 705, if not, go to Step 708.
  • Step 705 Determine whether the traversal of the label data at all levels is completed, if yes, go to Step 706, if not, go to Step 707.
  • Step 706 It is determined that the corresponding tag is queried, and then the process ends.
  • the corresponding tag data can be read; in practical applications, the corresponding data can be read from the hash table node where the tag data is located.
  • Step 707 take out the next-level label, and then return to step 703 .
  • Step 708 It is determined that the corresponding tag is not queried, and then the process ends.
  • the corresponding data when the data deletion operation is performed on the at least two-level tag data, the corresponding data may be queried first, and then the corresponding data may be deleted in the corresponding hash node.
  • the corresponding data when the data modification operation is performed on the at least two-level tag data, the corresponding data may be queried first, and then the corresponding data may be modified in the corresponding hash node.
  • FIG. 8 is a schematic diagram of an optional composition structure of the data management apparatus according to the embodiment of the present application, as shown in FIG. 8 .
  • the data management apparatus 800 may include:
  • the establishment module 801 is configured to establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the node of the parent cluster is master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • the processing module 802 is configured to write data to each node of each sub-cluster based on the data storage architecture.
  • the establishing module 801 configured to establish a data storage architecture, includes:
  • the data storage architecture is established based on the master node of each sub-cluster and the master node of the parent cluster.
  • the establishing module 801 is further configured to obtain the health index of the master node of each sub-cluster according to at least one of the following: the number of failures, the duration of at least one failure, the last time The length of time from the fault to the current time.
  • the establishing module 801 is configured to select a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health index of the respective sub-clusters nodes, including:
  • a master node is selected as the master node of the parent cluster; wherein the first score
  • the value is a preconfigured value.
  • the establishing module 801 is configured to, based on the health index and the first score value of the respective sub-clusters, select one master node as the primary node among the master nodes of the respective sub-clusters.
  • the master node of the parent cluster including:
  • the master node of each sub-cluster Based on the data interaction between the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value ;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the establishing module 801 is configured to select a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health index of the respective sub-clusters nodes, including:
  • the master node of each sub-cluster acquires the first message of each sub-cluster, where the first message represents the message of joining the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, among the master nodes of each sub-cluster, one master node is selected as the The master node of the parent cluster.
  • the establishing module 801 is configured to enable the master node of each sub-cluster to obtain the first message of each sub-cluster based on data interaction between the master nodes of each sub-cluster, including:
  • the master node of the seed cluster After the master node of the seed cluster receives the first message sent by the master node of other sub-clusters, the master node of each sub-cluster interacts with the data of the other sub-clusters, so that the master node of each sub-cluster obtains the The first message of each sub-cluster; wherein, the other sub-clusters are sub-clusters in the multiple sub-clusters except the seed cluster; the first message is that the master node of the other sub-clusters is based on the seed
  • the address of the master node of the cluster is sent from the address of the master node of the seed cluster, and the address of the master node of the seed cluster is the information predetermined by the master nodes of the other sub-clusters.
  • the establishing module 801 is further configured to:
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, it joins the master node of the first sub-cluster in the parent cluster, and sends a message of the parent cluster to each node of the parent cluster.
  • Member change information the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster,
  • the first message represents a message for joining the parent cluster.
  • the establishing module 801 is further configured to:
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the second sub-cluster represents the sub-cluster to which the faulty master node belongs, and the first message represents the message of joining the parent cluster.
  • processing module 802 is further configured to:
  • the processing module 802 is configured to write data to each node of each sub-cluster based on the data storage architecture, including:
  • the master node of the parent cluster sends the data writing instruction to the master node of each sub-cluster, and the master node of each sub-cluster will The data writing instruction is sent to each slave node of the sub-cluster, so that each node of each sub-cluster writes the to-be-written data.
  • the data to be written includes at least two levels of tag data
  • Writing the data to be written by each node of each sub-cluster includes:
  • a hash table is established for each level of label data, and a multi-fork tree data structure is constructed based on the hash table corresponding to each level of label data;
  • the at least two levels of label data are written into each node of each sub-cluster based on the multi-tree data structure.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • the processing module is further configured to perform at least one of the following operations on the at least two-level tag data based on the multi-tree data structure: adding, deleting, modifying, and querying.
  • both the establishment module 801 and the processing module 802 can be implemented by a processor, and the above-mentioned processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. kind. It can be understood that the electronic device that implements the function of the above processor may also be other, which is not limited in the embodiment of the present application.
  • the embodiments of the present application if the above data management method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the computer software products are stored in a storage medium and include several instructions for A computer device (which may be a terminal, a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), magnetic disk or optical disk and other media that can store program codes.
  • the embodiments of the present application are not limited to any specific combination of hardware and software.
  • the embodiments of the present application further provide a computer program product, where the computer program product includes computer-executable instructions, and the computer-executable instructions are used to implement any one of the data management methods provided by the embodiments of the present application.
  • an embodiment of the present application further provides a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and the computer-executable instructions are used to implement any one of the data management methods provided in the foregoing embodiments.
  • FIG. 9 is an optional structural schematic diagram of the electronic device provided by the embodiment of the present application.
  • the electronic device 900 includes:
  • the processor 902 is configured to implement any one of the above data management methods when executing the executable instructions stored in the memory 901 .
  • the above-mentioned processor 902 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.
  • the above-mentioned computer-readable storage medium/memory can be a read-only memory (Read Only Memory, ROM), a programmable read-only memory (Programmable Read-Only Memory, PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory) Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, optical disk, or memory such as Compact Disc Read-Only Memory (CD-ROM); it can also be various terminals including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital Assistant etc.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.
  • the unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.
  • each functional unit in each embodiment of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.
  • the above-mentioned integrated units of the present application are implemented in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application may be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to make The automatic test line of the device performs all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un procédé et un appareil de gestion de données, ainsi qu'un dispositif électronique et un support de stockage par ordinateur. Le procédé comprend : l'établissement d'une architecture de stockage de données, l'architecture de stockage de données comprenant un groupe parent et une pluralité de groupes enfants, chaque groupe enfant dans la pluralité de groupes enfants comprenant un nœud maître et au moins un nœud esclave, les nœuds du groupe parent étant les nœuds maîtres de la pluralité de groupes enfants, et les nœuds du groupe parent comprenant un nœud maître du groupe parent et au moins un nœud esclave du groupe parent ; et l'écriture de données dans chaque nœud de chaque groupe enfant sur la base de l'architecture de stockage de données.
PCT/CN2021/125290 2020-10-22 2021-10-21 Procédé et appareil de gestion de données, ainsi que dispositif, support de stockage par ordinateur et programme WO2022083690A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011139659.7 2020-10-22
CN202011139659.7A CN112311596B (zh) 2020-10-22 2020-10-22 数据管理方法、装置、设备及计算机存储介质

Publications (1)

Publication Number Publication Date
WO2022083690A1 true WO2022083690A1 (fr) 2022-04-28

Family

ID=74327069

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/125290 WO2022083690A1 (fr) 2020-10-22 2021-10-21 Procédé et appareil de gestion de données, ainsi que dispositif, support de stockage par ordinateur et programme

Country Status (2)

Country Link
CN (1) CN112311596B (fr)
WO (1) WO2022083690A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242812A (zh) * 2022-07-25 2022-10-25 济南浪潮数据技术有限公司 一种节点数据同步方法、装置及计算机可读存储介质
CN115793992A (zh) * 2023-02-07 2023-03-14 天翼云科技有限公司 数据存储方法、装置、电子设备及可读存储介质
CN115811525A (zh) * 2023-02-09 2023-03-17 杭州合众数据技术有限公司 基于分布式架构的数据交换与处理方法
CN116389786A (zh) * 2023-03-29 2023-07-04 深圳市安飞信息有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN117666970A (zh) * 2024-01-30 2024-03-08 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112311596B (zh) * 2020-10-22 2023-05-12 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质
CN113067883B (zh) * 2021-03-31 2023-07-28 建信金融科技有限责任公司 数据传输方法、装置、计算机设备及存储介质
CN113660339B (zh) * 2021-08-18 2023-08-04 北京百度网讯科技有限公司 用于去中心化集群的方法和装置
CN114357049B (zh) * 2022-01-07 2024-01-19 苏州浪潮智能科技有限公司 一种存储集群互联方法、装置、计算机设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105721545A (zh) * 2016-01-20 2016-06-29 浪潮(北京)电子信息产业有限公司 一种多层次集群管理实现方法
CN108134706A (zh) * 2018-01-02 2018-06-08 中国工商银行股份有限公司 区块链多活高可用***、计算机设备以及方法
CN110169040A (zh) * 2018-07-10 2019-08-23 深圳花儿数据技术有限公司 基于多层一致性哈希的分布式数据存储方法与***
CN110308984A (zh) * 2019-04-30 2019-10-08 北京航空航天大学 一种用于处理地理分布式数据的跨集群计算***
US10719417B2 (en) * 2018-01-30 2020-07-21 EMC IP Holding Company, LLC Data protection cluster system supporting multiple data tiers
CN112311596A (zh) * 2020-10-22 2021-02-02 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8675672B1 (en) * 2011-12-30 2014-03-18 Emc Corporation Hierarchical cluster tree overlay network
CN105007193A (zh) * 2015-08-19 2015-10-28 浪潮(北京)电子信息产业有限公司 一种多层信息处理方法、***及集群管理节点
CN107426038A (zh) * 2017-09-12 2017-12-01 郑州云海信息技术有限公司 一种采集数据的分布式集群***及数据采集方法
CN108540310B (zh) * 2018-03-15 2021-05-11 南京邮电大学 一种基于无线网络协同感知的行为预测方法
CN108803967B (zh) * 2018-04-18 2020-12-01 李世伟 聊天窗口展示方法、装置及电子设备
CN110543410A (zh) * 2019-09-05 2019-12-06 曙光信息产业(北京)有限公司 一种处理集群指标的方法、查询集群指标的方法和装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105721545A (zh) * 2016-01-20 2016-06-29 浪潮(北京)电子信息产业有限公司 一种多层次集群管理实现方法
CN108134706A (zh) * 2018-01-02 2018-06-08 中国工商银行股份有限公司 区块链多活高可用***、计算机设备以及方法
US10719417B2 (en) * 2018-01-30 2020-07-21 EMC IP Holding Company, LLC Data protection cluster system supporting multiple data tiers
CN110169040A (zh) * 2018-07-10 2019-08-23 深圳花儿数据技术有限公司 基于多层一致性哈希的分布式数据存储方法与***
CN110308984A (zh) * 2019-04-30 2019-10-08 北京航空航天大学 一种用于处理地理分布式数据的跨集群计算***
CN112311596A (zh) * 2020-10-22 2021-02-02 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242812A (zh) * 2022-07-25 2022-10-25 济南浪潮数据技术有限公司 一种节点数据同步方法、装置及计算机可读存储介质
CN115793992A (zh) * 2023-02-07 2023-03-14 天翼云科技有限公司 数据存储方法、装置、电子设备及可读存储介质
CN115811525A (zh) * 2023-02-09 2023-03-17 杭州合众数据技术有限公司 基于分布式架构的数据交换与处理方法
CN116389786A (zh) * 2023-03-29 2023-07-04 深圳市安飞信息有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN116389786B (zh) * 2023-03-29 2024-04-05 南京浮点智算数字科技有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN117666970A (zh) * 2024-01-30 2024-03-08 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置
CN117666970B (zh) * 2024-01-30 2024-05-07 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置

Also Published As

Publication number Publication date
CN112311596B (zh) 2023-05-12
CN112311596A (zh) 2021-02-02

Similar Documents

Publication Publication Date Title
WO2022083690A1 (fr) Procédé et appareil de gestion de données, ainsi que dispositif, support de stockage par ordinateur et programme
US8073972B2 (en) System and method for location discovery based on DNS
RU2400806C2 (ru) Организация стыковки запросов на ресурс с соответствующими ресурсами
US9405781B2 (en) Virtual multi-cluster clouds
RU2431184C2 (ru) Межблизостная связь в федерации рандеву
US9104560B2 (en) Two level addressing in storage clusters
EP2095248B1 (fr) Cohérence à l'intérieur d'une infrastructure en fédération
US8874708B2 (en) Location discovery based on DNS
WO2020063763A1 (fr) Procédé, appareil et système de stockage de données, et serveur, nœud de commande et support
JP7270755B2 (ja) 分散システムでのメタデータルーティング
TW200803303A (en) Inter-proximity communication within a rendezvous federation
JP2008533564A (ja) データ管理のための方法および装置
US8296420B2 (en) Method and apparatus for constructing a DHT-based global namespace
US9407687B2 (en) Method, apparatus, and network system for acquiring content
WO2023050986A1 (fr) Maintenance d'informations d'architecture de réseau de système de chaîne de blocs
CN109818948B (zh) 应用部署方法、装置、***、设备及介质
CN106687943A (zh) 组织具有多个计算机的计算***、在计算机之中分配计算任务、以及在计算机***中维护数据完整性与冗余的***和方法
WO2024140698A1 (fr) Procédé, système et appareil de traitement de session, et support de stockage
WO2008089616A1 (fr) Serveur, système réseau p2p et procédé d'acheminement et de transfert de l'affectation de la clé de ressource de ce dernier.......
WO2021218984A1 (fr) Procédé de routage de données et appareil associé
Yu et al. Granary: A sharing oriented distributed storage system
CN115865844B (zh) 基于sdn与ndn的虚实结合动态流量调度方法及装置
WO2023160040A1 (fr) Procédé et appareil de traitement de données basés sur une chaîne de blocs, et dispositif et support de stockage lisible
WO2022078012A1 (fr) Procédé et système de stockage en continu de données stockées dans une chaîne de blocs, dispositif et support
US20040122944A1 (en) Method and system of locating computers in distributed computer system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21882105

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21882105

Country of ref document: EP

Kind code of ref document: A1