WO2022083690A1 - 数据管理方法、装置、设备、计算机存储介质和程序 - Google Patents

数据管理方法、装置、设备、计算机存储介质和程序 Download PDF

Info

Publication number
WO2022083690A1
WO2022083690A1 PCT/CN2021/125290 CN2021125290W WO2022083690A1 WO 2022083690 A1 WO2022083690 A1 WO 2022083690A1 CN 2021125290 W CN2021125290 W CN 2021125290W WO 2022083690 A1 WO2022083690 A1 WO 2022083690A1
Authority
WO
WIPO (PCT)
Prior art keywords
cluster
sub
master node
node
data
Prior art date
Application number
PCT/CN2021/125290
Other languages
English (en)
French (fr)
Inventor
焦宏宇
邱路达
丁易元
张若君
孙芮
邓虹雨
Original Assignee
深圳前海微众银行股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳前海微众银行股份有限公司 filed Critical 深圳前海微众银行股份有限公司
Publication of WO2022083690A1 publication Critical patent/WO2022083690A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/30—Decision processes by autonomous network management units using voting and bidding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
    • H04L41/0668—Management of faults, events, alarms or notifications using network fault recovery by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • This application relates to the distributed structure technology of financial technology (Fintech), and relates to, but is not limited to, a data management method, apparatus, electronic device, computer storage medium and computer program.
  • data management can be implemented based on a single-layer distributed architecture.
  • the single-layer distributed architecture is not suitable for super-large distributed clusters.
  • the premise of data management in a distributed cluster is to determine the master node through cluster election.
  • cluster election is performed based on a single-layer distributed architecture, each node must send election messages to other nodes in the cluster, which is likely to cause a message storm. , so that most of the traffic is occupied by the packets sent by the cluster election, which affects the transmission of normal packets.
  • the embodiments of the present application provide a data management method, apparatus, electronic device, computer storage medium and computer, which can solve the problem of message storm caused by electing a master node in the prior art.
  • the embodiment of the present application provides a data management method, the method includes:
  • the data storage architecture includes a parent cluster (Parent Cluster) and a plurality of sub-clusters (Child Cluster), each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, the parent cluster's
  • the node is the master node of the multiple sub-clusters, and the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • establishing a data storage architecture includes:
  • the data storage architecture is established based on the master node of each sub-cluster and the master node of the parent cluster.
  • the method further includes:
  • the health indicator of the master node of each subcluster is derived based on at least one of the following: the number of failures, the duration of at least one failure, and the length of time since the last failure.
  • the embodiment of the present application can accurately obtain the health index of the master node of each sub-cluster based on the fault information, thus, it is beneficial to accurately select a node with a higher health degree among the master nodes of the sub-cluster As the master node of the parent cluster,
  • selecting a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health degree indicators of the respective sub-clusters including:
  • a master node is selected as the master node of the parent cluster; wherein the first score
  • the value is a preconfigured value.
  • the selection of the master node of the parent cluster is not only based on the ID of the node, but the selection of the master node of the parent cluster can be based on comprehensive consideration of the score corresponding to the ID of the node and the health index.
  • the master node is beneficial to reduce the problem of frequent replacement of master nodes caused by frequent addition of nodes with larger ID scores to the cluster.
  • a master node is selected as the master node of the parent cluster ,include:
  • the master node of each sub-cluster Based on the data interaction between the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value ;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the embodiment of the present application can obtain the master node of the parent cluster through the election of the sub-cluster based on the second score value, and the second score value is not only related to the ID of the node, but also related to the health degree index of the node. Therefore, The embodiment of the present application can reduce the problem of frequent replacement of master nodes caused by nodes with larger ID scores frequently joining the cluster.
  • selecting a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health degree indicators of the respective sub-clusters including:
  • the master node of each sub-cluster Based on the data interaction of the master node of each sub-cluster, the master node of each sub-cluster acquires the first message of each sub-cluster, where the first message represents a join message (Join Message) of the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, among the master nodes of each sub-cluster, one master node is selected as the The master node of the parent cluster.
  • the embodiment of the present application can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the master node of the cluster can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • enabling the master node of each sub-cluster to obtain the first message of each sub-cluster based on the data interaction between the master nodes of each sub-cluster includes:
  • the master node (Seed Cluster Master) of the seed cluster receives the first message sent by the master node of other sub-clusters, it interacts with the data of the other sub-clusters through the seed cluster, so that the The master node obtains the first message of each sub-cluster; wherein, the other sub-clusters are sub-clusters of the multiple sub-clusters except the seed cluster; the first message is the master of the other sub-clusters.
  • the node is sent based on the address of the master node of the seed cluster, and the address of the master node of the seed cluster is information predetermined by the master nodes of the other sub-clusters.
  • the embodiment of the present application can realize the data interaction of the master nodes of each sub-cluster through the selection of seed clusters, and further, is conducive to selecting the master node of the parent cluster from the master nodes of each sub-cluster.
  • the method further includes:
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, it joins the master node of the first sub-cluster in the parent cluster, and sends a message of the parent cluster to each node of the parent cluster.
  • Member change information the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster;
  • the first message represents a message for joining the parent cluster.
  • the embodiment of the present application can update the member change information of the parent cluster in time when the slave node of the parent cluster fails, which is beneficial to accurately realize the master node of the parent cluster when the master node of the parent cluster is elected subsequently. election.
  • the method further includes:
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the second sub-cluster represents the sub-cluster to which the faulty master node belongs, and the first message represents the message of joining the parent cluster.
  • the main node information of the parent cluster can be re-elected in time, and the main node of the second sub-cluster can be received after the main node of the second sub-cluster is selected.
  • the join message is conducive to the subsequent re-election of the master node of the parent cluster.
  • the method further includes:
  • the writing data to each node of each sub-cluster based on the data storage architecture includes:
  • the master node of the parent cluster sends the data writing instruction to the master node of each sub-cluster, and the master node of each sub-cluster will The data writing instruction is sent to each slave node of the sub-cluster, so that each node of each sub-cluster writes the data to be written.
  • the embodiment of the present application can write the same data to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • the data to be written includes at least two levels of tag data
  • Writing the data to be written by each node of each sub-cluster includes:
  • a hash table is established for each level of label data, and a multi-fork tree data structure is constructed based on the hash table corresponding to each level of label data;
  • the at least two levels of label data are written into each node of each sub-cluster based on the multi-tree data structure.
  • the embodiment of the present application can implement hierarchical storage of multi-level tag data, which is beneficial to the subsequent management of multi-level tag data.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • the label data of the current level can be quickly queried to the label data of the next level.
  • the method further includes:
  • At least one of the following operations is performed on the at least two-level tag data: adding, deleting, modifying, and querying.
  • An embodiment of the present application provides a data management device, and the device includes:
  • the establishment module is configured to establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the nodes of the parent cluster are all the sub-clusters.
  • master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • the processing module is configured to write data to each node of each sub-cluster based on the data storage architecture.
  • An embodiment of the present application provides an electronic device, and the electronic device includes:
  • a memory configured to store executable instructions
  • the processor when configured to execute the executable instructions stored in the memory, implements any one of the above data management methods.
  • Embodiments of the present application provide a computer-readable storage medium storing executable instructions for implementing any of the foregoing data management methods when executed by a processor.
  • Embodiments of the present application further provide a computer program product, where the computer program product includes computer-executable instructions, where the computer-executable instructions are used to implement any one of the data management methods provided by the embodiments of the present application.
  • a data storage architecture is established, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the nodes of the parent cluster are The master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster; based on the data storage architecture, to each node of each sub-cluster data input.
  • the data storage architecture of the embodiment of the present application is a two-layer distributed architecture, which can realize data interaction based on the master node of each sub-cluster. It reduces the negotiation message data, limits the propagation range of negotiation messages, and helps to alleviate the problems that affect normal packet transmission caused by message storms.
  • FIG. 2 is a schematic diagram of a data storage architecture according to an embodiment of the present application.
  • Fig. 3 is a schematic diagram of cluster establishment in the embodiment of the present application.
  • FIG. 4 is a schematic diagram of electing a parent cluster master node in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a multi-tree data structure according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of an operation of adding data to at least two levels of tag data in an embodiment of the present application
  • FIG. 7 is a schematic flowchart of a data query operation on at least two levels of tag data in an embodiment of the present application
  • FIG. 8 is a schematic diagram of an optional composition structure of a data management apparatus according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of an optional composition structure of an electronic device provided by an embodiment of the present application.
  • the single-layer distributed architecture is not suitable for super-large distributed clusters, because there are many super-large distributed cluster nodes, and the premise of data management based on super-large distributed clusters is to determine the master node through cluster election.
  • cluster election each node must send election messages to other nodes in the cluster, which can easily cause message storms, and cause most of the traffic to be occupied by the packets sent by the cluster election, affecting normal packet transmission.
  • the election algorithm does not balance the election speed and stability. For example, the bully algorithm, the cluster election is simply judged according to the size of the node ID, which will cause frequent elections
  • the problem of the master node; such as the raft algorithm, the majority voting mechanism is used for cluster election. This mechanism can avoid frequent election of the master, but it will cause the election time to become longer, and more than half of the nodes must vote to elect the master node. .
  • each node uses a unified Hash table structure for data storage.
  • the hash value of all key values (key) leads to a longer expansion time.
  • the embodiments of the present application provide a data management method, an apparatus, an electronic device, and a computer storage medium.
  • the data management methods of the embodiments of the present application can be applied to electronic devices.
  • the following describes exemplary applications of the electronic devices provided by the embodiments of the present application.
  • the electronic device provided by the embodiments of the present application may be implemented as a server, and the server may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, Cloud servers for basic cloud computing services such as cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.
  • CDN Content Delivery Network
  • FIG. 1 is an optional flowchart of a data management method provided by an embodiment of the present application. As shown in FIG. 1 , the flowchart may include:
  • Step 101 establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, the node of the parent cluster is the master node of the multiple sub-clusters, and the parent cluster
  • the nodes include one master node of the parent cluster and at least one slave node of the parent cluster.
  • a sub-cluster represents a small cluster obtained by dividing the nodes of all clusters, each sub-cluster can independently elect a sub-cluster master node (Child Cluster Master), and the remaining nodes in the sub-cluster except the sub-cluster master node are Child Cluster Slave.
  • Child Cluster Master sub-cluster master node
  • the parent cluster represents a cluster composed of master nodes of each sub-cluster. You can select a node from the nodes of the parent cluster as the parent cluster master node (Parent Cluster Master), and the rest of the nodes in the parent cluster except the parent cluster master node are the parent cluster slave nodes. Node (Parent Cluster Slave).
  • FIG. 2 is a schematic diagram of a data storage architecture according to an embodiment of the present application.
  • sub-cluster 0, sub-cluster 1 and sub-cluster 2 represent three different sub-clusters
  • master node 01 represents the master node of sub-cluster 0, and slave Node 02 and slave node 03 represent different slave nodes in sub-cluster 0
  • master node 11 represents the master node of sub-cluster 1
  • slave node 12 and slave node 13 represent different slave nodes in sub-cluster 1
  • master node 21 represents the sub-cluster
  • master node of 2 slave node 22 and slave node 23 represent different slave nodes in sub-cluster 2
  • master node 01, master node 11 and master node 21 form the parent cluster
  • master node 21 is the master node of the parent cluster
  • master node 11 and The master node 01 is a different slave node in the parent cluster.
  • the data storage architecture of the embodiment of the present application is a two-layer structure, wherein the first layer is composed of multiple sub-clusters, and the sub-clusters of the first layer include sub-cluster 0, sub-cluster 1 and sub-cluster 2; the second layer represents The parent cluster consists of master nodes of multiple sub-clusters, that is, the parent cluster consists of master node 01 , master node 11 and master node 21 . After each sub-cluster is established, it will independently elect its own master node. Then, the master node of each sub-cluster can obtain the master ground of the parent cluster through election, so that one super-large cluster is realized through a two-layer structure.
  • Step 102 Write data to each node of each sub-cluster based on the data storage architecture.
  • the same data can be written to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • data can be read from any node because the data in each node is consistent.
  • steps 101 to 102 may be implemented based on a processor of an electronic device, and the above-mentioned processor may be an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital Signal Processing Device (Digital Signal Processing Device, DSPD), Programmable Logic Device (Programmable Logic Device, PLD), Field Programmable Gate Array (Field Programmable Gate Array, FPGA), CPU, Controller, Microcontroller, Microprocessor at least one of them.
  • ASIC application specific integrated circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • Field Programmable Gate Array Field Programmable Gate Array
  • FPGA Field Programmable Gate Array
  • CPU Controller
  • Microcontroller Microprocessor at least one of them.
  • the data storage architecture of the embodiment of the present application is a two-layer distributed architecture, which can realize data interaction based on the master node of each sub-cluster. It reduces the negotiation message data, limits the propagation range of negotiation messages, and helps to alleviate problems that affect normal packet transmission caused by message storms.
  • establishing a data storage architecture may include: determining a master node and a health index of each sub-cluster; based on the health index of each sub-cluster, selecting a One master node is used as the master node of the parent cluster; the data storage architecture is established based on the master nodes of each sub-cluster and the master node of the parent cluster.
  • the health index is used to reflect the health of the master node of the sub-cluster; after determining the master node of each sub-cluster, the slave nodes of each sub-cluster can be determined, and after determining the master node of the parent cluster, the parent node can be determined.
  • the slave node of the cluster thus, the above data storage architecture can be obtained.
  • the embodiment of the present application can select the child nodes of the parent cluster based on the health index of the master node of each sub-cluster, which is similar to the bully algorithm.
  • the raft algorithm even if a node with a large ID is added frequently, it will not cause frequent replacement of the master node of the parent cluster, and the stability is stronger; and compared with the raft algorithm, it is necessary to elect the master node based on the half voting mechanism to determine the speed of the master node. Faster, to a certain extent, achieves a balance between election speed and stability.
  • the health index of the master node of each sub-cluster can be obtained according to at least one of the following: the number of failures, the duration of at least one failure, and the duration from the last failure to the current time.
  • the health index is negatively correlated with the number of failures
  • the health index is negatively correlated with the duration of at least one failure
  • the health index is the length of time from the last failure to the current time; that is, the longer the duration of at least one failure If it is short, the value of the health index is larger; the longer the time from the last fault to the current time, the larger the value of the health index; the greater the number of failures, the smaller the value of the health index.
  • the duration of the at least one failure may include the duration of the last failure.
  • failure start time and failure recovery time can be recorded to determine the failure duration.
  • the health index of the master node of each sub-cluster can be obtained according to the following formula (1):
  • S0 represents the health index of the master node of the sub-cluster
  • t1 represents the duration of the last failure from the current time
  • fault_count represents the number of failures
  • t2 represents the duration of the last failure.
  • the embodiment of the present application can accurately obtain the health index of the master node of each sub-cluster based on the fault information. Therefore, it is beneficial to accurately select a node with a higher health degree among the master nodes of the sub-cluster.
  • the master node of the parent cluster As the master node of the parent cluster,
  • one master node may be selected as the master node of the parent cluster among the master nodes of each sub-cluster; wherein, the first The scoring value is a preconfigured numerical value.
  • the first score value may be a score value corresponding to the ID of the node; in practical applications, the first score value may be predetermined, that is, the first score value of each node is a fixed value.
  • the selection of the master node of the parent cluster is not only based on the ID of the node, but the selection of the master node of the parent cluster can be based on comprehensive consideration of the score corresponding to the ID of the node and the health index.
  • the master node is beneficial to reduce the problem of frequent replacement of master nodes caused by frequent addition of nodes with larger ID scores to the cluster.
  • selecting a main node as the main node of the parent cluster among the main nodes of each sub-cluster may include:
  • the master node of each sub-cluster Based on the data interaction of the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the master node of each sub-cluster can calculate the second score value of this node, and then, through the data interaction of the master node of each sub-cluster, the second score value of the master node of other sub-clusters can be obtained, and then, The magnitude relationship between the second score values of the current node and the master nodes of other sub-clusters can be judged, and it can be judged whether the current node can serve as the master node of the parent cluster.
  • the second score value of this node is greater than or equal to the second score value of the master node of other sub-clusters, it means that this node can serve as the master node of the parent cluster; if the second score value of this node is less than The second score value of the master node of any other sub-cluster indicates that this node cannot serve as the master node of the parent cluster.
  • the embodiment of the present application can obtain the master node of the parent cluster through the election of the sub-cluster based on the second score value, and the second score value is not only related to the ID of the node, but also related to the health degree index of the node. Therefore, The embodiment of the present application can reduce the problem of frequent replacement of master nodes caused by nodes with larger ID scores frequently joining the cluster.
  • selecting a master node as the master node of the parent cluster may include:
  • the master node of each sub-cluster obtains the first message of each sub-cluster, where the first message represents the message of joining the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, one master node is selected as the master node of the parent cluster among the master nodes of each sub-cluster.
  • the master node of each sub-cluster after generating the first message, that is, the message of joining the parent cluster, can send the first message to the master nodes of other sub-clusters; the master node of each sub-cluster can also receive other sub-clusters.
  • the embodiment of the present application can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the master node of the cluster can initiate the process of selecting the master node of the parent cluster at an appropriate time based on the first message of the master node of each sub-cluster, which is conducive to accurately selecting the parent node from the master nodes of each sub-cluster.
  • the above-mentioned data interaction based on the master node of each sub-cluster, enabling the master node of each sub-cluster to obtain the first message of each sub-cluster includes:
  • the master node of the seed cluster After the master node of the seed cluster receives the first message sent by the master node of other sub-clusters, it interacts with the data of other sub-clusters through the seed cluster, so that the master node of each sub-cluster obtains the first message of each sub-cluster; wherein, The other sub-clusters are sub-clusters of multiple sub-clusters except the seed cluster; the first message is sent by the master node of other sub-clusters based on the address of the master node of the seed cluster, and the address of the master node of the seed cluster is the master node of the other sub-clusters. Node predetermined information.
  • one sub-cluster may be arbitrarily selected from each sub-cluster as a seed cluster, or a seed cluster may be selected from each sub-cluster according to a preset selection method.
  • the address of the master node of the seed cluster may be pre-configured in the master nodes of other sub-clusters, so that the master nodes of other sub-clusters may send the first message to the seed cluster; the seed cluster receives the first message after receiving the first message. After the message is sent, the sending address of the first message, that is, the address of the master node of other sub-clusters can be recorded, and further, the seed cluster can send the received first message to the master node of each sub-cluster.
  • the embodiment of the present application can realize the data interaction of the master nodes of each sub-cluster through the selection of seed clusters, and further, is conducive to selecting the master node of the parent cluster from the master nodes of each sub-cluster.
  • FIG. 3 is a schematic diagram of cluster establishment in an embodiment of the application.
  • a seed cluster is selected first, and the seed cluster is a sub-cluster in each sub-cluster.
  • the seed cluster, sub-cluster 1 and sub-cluster 2 represent three different
  • the master node 01 represents the master node of the seed cluster
  • the slave node 02 and slave node 03 represent the slave nodes (Seed Cluster Slave) in the seed cluster;
  • the meaning of master node 11, slave node 12 and slave node 13 is the same as that of Figure 2
  • the meanings of the master node 21 , the slave node 22 and the slave node 23 are the same as those in FIG. 2 , and will not be repeated here.
  • the master node of the seed cluster can be selected, and then the IDs of all sub-clusters (including sub-cluster 1 and sub-cluster 2) at the initial moment can be written into the master node of the seed cluster, so that the seed cluster can know ID of each subcluster.
  • sub-clusters except the seed cluster can selectively select the master node, and the ID and address of the seed cluster can be pre-configured in other sub-clusters.
  • the master node of each sub-cluster can send a first message to the master node of the seed cluster (master node 01 shown in FIG. 3 ), where the first message includes the ID of the sub-cluster and the ID of the sub-cluster Information such as the address of the master node.
  • the master node of the seed cluster After receiving the first message sent by the other sub-cluster, the master node of the seed cluster can store the ID of the sub-cluster as the sub-cluster information according to the first message, and record the address of the master node of the corresponding sub-cluster.
  • the seed cluster After the seed cluster receives the first messages of all other sub-clusters, it can broadcast the sub-cluster information to the master node of each other sub-cluster, where the sub-cluster information may include the received first messages of all other sub-clusters, so that, Each sub-cluster can obtain the information of the master node of each sub-cluster.
  • the election process of the master node of the parent cluster can be initiated; here, in determining the election process of the master node that initiates the parent cluster, the seed cluster has no meaning and needs to be Fair competition with other subclusters.
  • the seed cluster is the coordinator at the initialization moment of the cluster establishment, and each sub-cluster can obtain the information of the master node of each sub-cluster and then degenerate into a sub-cluster.
  • FIG. 4 is a schematic diagram of electing a master node of a parent cluster in an embodiment of the application.
  • sub-cluster 0, sub-cluster 1 and sub-cluster 2 represent three different sub-clusters; master node 01, slave node 02, slave node 03 , the master node 11 , the slave node 12 , the slave node 13 , the master node 21 , the slave node 22 and the slave node 23 have the same meanings as in FIG. 2 , and will not be repeated here.
  • the master node of each sub-cluster first determines the second score value of the node, and can send a success (victory) message to the master nodes of other sub-clusters, for example, the master node 21 can send the master node 11 and master node 11 and the master node 01 to send a victory message; the victory message represents a message announcing that this node is the master node, and the victory message can carry the second score value of this node.
  • the master node of each sub-cluster can receive the victory message sent by the master node of other sub-clusters. After receiving the victory message, it compares the second score value in the victory message with the second score value of this node. The second score value of the node is greater than or equal to the second score value of this node, then an election message can be sent to the source node of the victory message. If the second score value in the message is less than the second score value, a response (alive) message may be sent to the source node of the victory message, where the alive message for the victory message is a message indicating re-election of the parent cluster master node.
  • the master node 21 sends the victory message to the master node 11 and the master node 01
  • the master node 11 can send the master node 11 to the master node.
  • 21 sends an election message; if the second score value in the victory message is greater than the second score value of the master node 01, the master node 01 may send an election message to the master node 21.
  • the master node of each sub-cluster can reply to the alive message.
  • the alive message for the election message indicates that the reply information of the election message has been received.
  • the master node 21 may send an alive message to the master node 11 and the master node 01, respectively.
  • the master node of each sub-cluster may not process the alive message and wait to receive messages sent by the master nodes of other sub-clusters.
  • the master node of any sub-cluster After the master node of any sub-cluster sends the master node of the victory to the master node of other sub-clusters, if it does not receive the alive message sent by the master node of other sub-clusters within a given time, it is considered that the master node is the parent cluster At this time, the master node of the other sub-clusters can send a victory message to the master nodes of other sub-clusters, so that the master nodes of other sub-clusters can determine the master node of the parent cluster. In this way, the election process of the master node of the parent cluster ends.
  • the given time may be set according to an actual application scenario, for example, the given time may be determined according to the maximum communication delay between each sub-cluster.
  • a node of the parent cluster fails, targeted processing needs to be performed, and two situations in which the parent node fails are respectively described below.
  • the slave node of the parent cluster fails, delete the failed slave node from the parent cluster, and use the master node of the parent cluster to send the member change information of the parent cluster to the slave node of the parent cluster;
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, the master node of the first sub-cluster is added to the parent cluster, and the member change information of the parent cluster is sent to each node of the parent cluster;
  • the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster.
  • the member change information of the parent cluster may include information such as IDs of each node of the parent cluster.
  • the master node of the parent cluster can delete the faulty slave node from the parent cluster, mark the ID of the corresponding child cluster as unavailable, and update the member information of the parent cluster, and then the parent cluster can be deleted. Membership change information is broadcast to each slave node of the parent cluster.
  • the master node of the first sub-cluster After the master node of the first sub-cluster fails, the master node can be re-selected in the first sub-cluster, and then the master node of the first sub-cluster can send the first message to the master node of the parent cluster; the master node of the parent cluster receives the first message. After the first message is received, the above-mentioned unavailable mark can be cleared, and then the member information of the parent cluster can be updated, and then the member change information of the parent cluster can be broadcasted to each slave node of the parent cluster.
  • the master node of the parent cluster may also update the failure information related to the first sub-cluster for use in electing the master node of the parent cluster.
  • the embodiment of the present application can update the member information of the parent cluster in time when the slave node of the parent cluster fails, and is conducive to accurately realizing the master node of the parent cluster when the election of the master node of the parent cluster is performed subsequently. election.
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the master node of the second sub-cluster After receiving the first message sent by the master node of the second sub-cluster, the master node of the second sub-cluster is added to the parent cluster, and the member change information of the parent cluster is sent to each node of the parent cluster; the second sub-cluster indicates The subcluster to which the failed master belongs.
  • the master node of the parent cluster fails, it is necessary to elect a node from each slave node of the parent cluster as the master node of the parent cluster, and the process of electing the master node of the parent cluster has been described above. The description is made in the content and will not be repeated here.
  • the master node of the second sub-cluster fails, it can be considered that the master node of the second sub-cluster is separated from the parent cluster. In this case, after the master node is selected from the second sub-cluster, it needs to join the parent cluster.
  • the specific process is as follows:
  • the master node of the parent cluster When the master node of the parent cluster fails, other nodes in the second sub-cluster can delete the information of the master node of the parent cluster, and re-select a node in the second sub-cluster as the master node of the second sub-cluster; After the master node of the second sub-cluster is selected, the master node of the second sub-cluster may send the first message to any node of the parent cluster; in some embodiments, a hash ring may be established for all nodes of the parent cluster in advance, and the first message may be The master node of the two sub-clusters may select a node adjacently in a clockwise or counterclockwise direction based on the hash ring, and send the first message to the selected node.
  • the master node of the parent cluster may directly record the ID of the second sub-cluster and the address information of the current master node of the second sub-cluster.
  • the node of any one of the parent clusters can return the master node information of the parent cluster to the current master node of the second sub-cluster, and the master node of the second sub-cluster
  • the node may send the first message to the master node of the parent cluster based on the master node information of the parent cluster.
  • the main node information of the parent cluster can be re-elected in time, and the main node of the second sub-cluster can be received after the main node of the second sub-cluster is selected.
  • the join message is conducive to the subsequent re-election of the master node of the parent cluster.
  • the above-mentioned writing data to each node of each sub-cluster based on the data storage architecture may include:
  • the data write command carries the data to be written
  • the master node of the parent cluster sends the data write instruction to the master node of each sub-cluster
  • the master node of each sub-cluster sends the data write instruction to the sub-cluster.
  • Each slave node enables each node of each sub-cluster to write the data to be written.
  • the data write instruction can be sent to any node in the parent cluster.
  • a node if the node receiving the data write command is not the master node of the parent cluster, the node receiving the data write command will reply the information of the master node of the parent cluster to the client, and then the client can send the data write command
  • the master node of the parent cluster can send the data write command to the master node of each sub-cluster; if the node receiving the data write command is the master node of the parent cluster, the master node of the parent cluster can write the data to the master node of the parent cluster.
  • Write commands are sent to the master node of each subcluster.
  • the master node of a sub-cluster when the master node of a sub-cluster determines that all nodes in the sub-cluster have successfully written the data to be written, it will return successful writing information to the master node of the parent cluster; otherwise, it will return unsuccessful writing information to the master node of the parent cluster; when the master node of the parent cluster receives the successful write information sent by the master nodes of each sub-cluster, it can return the successful write information to the client; when the master node of the parent cluster receives at least one When the master node of the sub-cluster sends the unsuccessful writing information, it can be determined that the node of the at least one sub-cluster is faulty, or it can be determined that the data has not been successfully written to the at least one sub-cluster. The master node of a sub-cluster sends a data write instruction until the successful write information sent by the master node of the at least one sub-cluster is received.
  • the embodiment of the present application can write the same data to each node of each sub-cluster based on the data write instruction, so that the data of each sub-cluster can be consistent.
  • a data read instruction can also be obtained, and the data read instruction carries the data read address of any node; then, based on the data read address, data can be read from any of the above nodes .
  • the above-mentioned data to be written includes at least two levels of tag data
  • writing the data to be written by each node of each sub-cluster includes: establishing a hash table for each level of label data, and building a multi-fork tree data structure based on the hash table corresponding to the label data at each level; A multi-tree data structure, the at least two levels of label data are written into each node of each sub-cluster.
  • micro-service technology has been widely used.
  • the characteristic of micro-service is dynamic.
  • IP Internet Protocol
  • a data management solution based on tag data is proposed instead of an IP-based data management solution.
  • the data to be written is application instance data
  • the application instance data may include the application cluster and the system to which the application belongs
  • the application instance data may be represented as /cluster/system/app, where the symbol / represents a fixed separator , cluster, system, and app represent clusters, systems, and applications, respectively, cluster represents the first-level label, system represents the second-level label, and app represents the third-level label.
  • FIG. 5 is a schematic diagram of a multi-fork tree data structure according to an embodiment of the present application.
  • hashtable1-0 represents a hash address of cluster data
  • bucket0 and bucket1 in hashtable1-0 represent different stored data.
  • the key is cluster0 and the value is hashtable1-0;
  • bucket1 in hashtable1-0 the key is cluster1 and the value is hashtable1-1.
  • hashtable1-0 and hashtable1-1 represent hash addresses of different system data
  • bucket0 and bucket1 in hashtable1-0 represent different storage data
  • the key is system0
  • the value is hashtable2-0
  • bucket1 in hashtable1-0 the key is system1
  • the value is hashtable2-1
  • bucket0 in hashtable1-1 represents the storage data
  • the key is system0
  • the value is hashtable2-2.
  • hashtable2-0, hashtable2-1 and hashtable2-2 respectively represent hash addresses of different application data; bucket0 in hashtable2-0 represents storage data, in bucket0 in hashtable2-0, the key is app0, The value is ip1; bucket0 and bucket1 in hashtable2-1 represent different storage data, in bucket0 in hashtable2-0, the key is app1 and the value is ip2; in bucket1 in hashtable2-0, the key is app2 and the value is ip2; in bucket0 in hashtable2-2, the key is app0 and the value is ip3; here, app0, app1, and app2 are different applications, and ip1, ip2, and ip3 represent different IP addresses.
  • label-to-IP conversion needs to be performed, and the actual physical-level communication can be completed by converting the logical concept label into physical concept IP when actually performing network-level communication; For example, referring to the third-level label data in FIG. 5 , a corresponding relationship between labels and IPs can be established, thus facilitating efficient storage and search of the corresponding relationships between labels and IPs.
  • each tree node in the multi-tree data structure is correspondingly established with a hash table, and the hash table is used to store corresponding first-level label data.
  • the embodiment of the present application can implement hierarchical storage of multi-level tag data, which is beneficial to the subsequent management of multi-level tag data.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • multi-level label data may be divided according to fixed separators, each level of label data corresponds to a tree node of a multi-tree data structure, and the tree node in the multi-tree data structure is not a leaf node (there is no child node), the key in the tree node is an element of the corresponding hash table, and the value is the hash address of the next-level hash table; here, the hash address of the next-level hash table is the above-mentioned lower hash table.
  • the hash address corresponding to the first-level tag data is divided according to fixed separators, each level of label data corresponds to a tree node of a multi-tree data structure, and the tree node in the multi-tree data structure is not a leaf node (there is no child node), the key in the tree node is an element of the corresponding hash table, and the value is the hash address of the next-level hash table; here, the hash address of the next-level hash table is the
  • the label data of the current level can be quickly queried to the label data of the next level.
  • Tag data with the same prefix string can share the same hash table node, which can reduce storage space usage.
  • bucket0 and bucket1 in hashtable2-1 have the same The prefix string, that is, the prefix strings of bucket0 and bucket1 in hashtable2-1 are both cluster0/system1; thus, bucket0 and bucket1 in hashtable2-1 share the same hash table node hashtable2-1.
  • the number of strings separated by a fixed separator is a fixed number, so the height of the multi-fork tree is also fixed, and the lookup time of each hash table is a constant level , the overall hash table-based query time complexity is also a constant level.
  • suffix wildcard matching such as /cluster0/system0/*
  • the embodiment of the present application does not need to traverse the hash table corresponding to the entire data to be written, which can further reduce the time complexity of suffix wildcard matching.
  • a hash table needs to be used uniformly to realize data storage. Therefore, when the data volume of the hash table is large, the more keys that need to be recalculated during expansion, the longer the expansion time; When data expansion is required, it is only necessary to expand the hash table of the node corresponding to the new element that needs to be placed, which reduces the expansion time to a certain extent.
  • At least one of the following operations may be performed on the at least two-level tag data: adding, deleting, modifying, and querying.
  • FIG. 6 is a schematic flowchart of an operation of adding data to at least two levels of label data in an embodiment of the present application. As shown in FIG. 6 , the process may include:
  • Step 601 Acquire label data at all levels.
  • the multi-level label data can be divided according to the fixed delimiter described above to obtain the label data of each level.
  • Step 602 Take out the first-level label.
  • cluster represents the first-level label
  • system represents the second-level label
  • app represents the third-level label
  • Step 603 Determine whether the hash table node corresponding to the currently fetched tag exists, if not, execute step 604; if yes, execute step 605.
  • Step 604 Create a corresponding hash table node, and perform step 605.
  • a corresponding hash table node can be created for the currently fetched tag.
  • Step 605 Determine whether there is a corresponding label in the hash table node, if not, go to step 606, and if so, go to step 607.
  • Step 606 insert the corresponding label, and then perform step 607 .
  • the currently fetched tag can be inserted into the hash table node, and then step 607 is performed.
  • Step 607 Determine whether the traversal of the label data at all levels is completed, if yes, end the process, if not, go to Step 608.
  • Step 608 take out the next-level label data, and then return to step 603 .
  • FIG. 7 is a schematic flowchart of a data query operation for at least two levels of tag data in an embodiment of the present application. As shown in FIG. 7 , the process may include:
  • Step 701 Acquire label data at all levels.
  • Step 702 Take out the first-level label.
  • Step 703 Determine whether the hash table node corresponding to the currently fetched tag exists, if yes, go to Step 704, if not, go to Step 708.
  • Step 704 Determine whether there is a corresponding label in the hash table node, if yes, go to Step 705, if not, go to Step 708.
  • Step 705 Determine whether the traversal of the label data at all levels is completed, if yes, go to Step 706, if not, go to Step 707.
  • Step 706 It is determined that the corresponding tag is queried, and then the process ends.
  • the corresponding tag data can be read; in practical applications, the corresponding data can be read from the hash table node where the tag data is located.
  • Step 707 take out the next-level label, and then return to step 703 .
  • Step 708 It is determined that the corresponding tag is not queried, and then the process ends.
  • the corresponding data when the data deletion operation is performed on the at least two-level tag data, the corresponding data may be queried first, and then the corresponding data may be deleted in the corresponding hash node.
  • the corresponding data when the data modification operation is performed on the at least two-level tag data, the corresponding data may be queried first, and then the corresponding data may be modified in the corresponding hash node.
  • FIG. 8 is a schematic diagram of an optional composition structure of the data management apparatus according to the embodiment of the present application, as shown in FIG. 8 .
  • the data management apparatus 800 may include:
  • the establishment module 801 is configured to establish a data storage architecture, the data storage architecture includes a parent cluster and multiple sub-clusters, each sub-cluster in the multiple sub-clusters includes a master node and at least one slave node, and the node of the parent cluster is master nodes of the multiple sub-clusters, the nodes of the parent cluster include a master node of the parent cluster and at least one slave node of the parent cluster;
  • the processing module 802 is configured to write data to each node of each sub-cluster based on the data storage architecture.
  • the establishing module 801 configured to establish a data storage architecture, includes:
  • the data storage architecture is established based on the master node of each sub-cluster and the master node of the parent cluster.
  • the establishing module 801 is further configured to obtain the health index of the master node of each sub-cluster according to at least one of the following: the number of failures, the duration of at least one failure, the last time The length of time from the fault to the current time.
  • the establishing module 801 is configured to select a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health index of the respective sub-clusters nodes, including:
  • a master node is selected as the master node of the parent cluster; wherein the first score
  • the value is a preconfigured value.
  • the establishing module 801 is configured to, based on the health index and the first score value of the respective sub-clusters, select one master node as the primary node among the master nodes of the respective sub-clusters.
  • the master node of the parent cluster including:
  • the master node of each sub-cluster Based on the data interaction between the master nodes of each sub-cluster, the master node of each sub-cluster obtains the second score value of the master node of each sub-cluster, where the second score value is the sum of the health index and the first score value ;
  • the master node of each sub-cluster elects the master node of the parent cluster.
  • the establishing module 801 is configured to select a master node as the master node of the parent cluster from the master nodes of the respective sub-clusters based on the health index of the respective sub-clusters nodes, including:
  • the master node of each sub-cluster acquires the first message of each sub-cluster, where the first message represents the message of joining the parent cluster;
  • each sub-cluster acquires the first message of each sub-cluster, based on the health index of each sub-cluster, among the master nodes of each sub-cluster, one master node is selected as the The master node of the parent cluster.
  • the establishing module 801 is configured to enable the master node of each sub-cluster to obtain the first message of each sub-cluster based on data interaction between the master nodes of each sub-cluster, including:
  • the master node of the seed cluster After the master node of the seed cluster receives the first message sent by the master node of other sub-clusters, the master node of each sub-cluster interacts with the data of the other sub-clusters, so that the master node of each sub-cluster obtains the The first message of each sub-cluster; wherein, the other sub-clusters are sub-clusters in the multiple sub-clusters except the seed cluster; the first message is that the master node of the other sub-clusters is based on the seed
  • the address of the master node of the cluster is sent from the address of the master node of the seed cluster, and the address of the master node of the seed cluster is the information predetermined by the master nodes of the other sub-clusters.
  • the establishing module 801 is further configured to:
  • the master node of the parent cluster After the master node of the parent cluster receives the first message sent by the master node of the first sub-cluster, it joins the master node of the first sub-cluster in the parent cluster, and sends a message of the parent cluster to each node of the parent cluster.
  • Member change information the first sub-cluster represents the sub-cluster to which the faulty slave node belongs, and the master node of the first sub-cluster represents the master node re-selected from each node of the first sub-cluster,
  • the first message represents a message for joining the parent cluster.
  • the establishing module 801 is further configured to:
  • the master node of the parent cluster fails, select a node from other nodes of the parent cluster as the master node of the parent cluster;
  • the second sub-cluster represents the sub-cluster to which the faulty master node belongs, and the first message represents the message of joining the parent cluster.
  • processing module 802 is further configured to:
  • the processing module 802 is configured to write data to each node of each sub-cluster based on the data storage architecture, including:
  • the master node of the parent cluster sends the data writing instruction to the master node of each sub-cluster, and the master node of each sub-cluster will The data writing instruction is sent to each slave node of the sub-cluster, so that each node of each sub-cluster writes the to-be-written data.
  • the data to be written includes at least two levels of tag data
  • Writing the data to be written by each node of each sub-cluster includes:
  • a hash table is established for each level of label data, and a multi-fork tree data structure is constructed based on the hash table corresponding to each level of label data;
  • the at least two levels of label data are written into each node of each sub-cluster based on the multi-tree data structure.
  • the hash table of the tag data of any level includes the hash address corresponding to the tag data of the next level.
  • the processing module is further configured to perform at least one of the following operations on the at least two-level tag data based on the multi-tree data structure: adding, deleting, modifying, and querying.
  • both the establishment module 801 and the processing module 802 can be implemented by a processor, and the above-mentioned processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. kind. It can be understood that the electronic device that implements the function of the above processor may also be other, which is not limited in the embodiment of the present application.
  • the embodiments of the present application if the above data management method is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the computer software products are stored in a storage medium and include several instructions for A computer device (which may be a terminal, a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), magnetic disk or optical disk and other media that can store program codes.
  • the embodiments of the present application are not limited to any specific combination of hardware and software.
  • the embodiments of the present application further provide a computer program product, where the computer program product includes computer-executable instructions, and the computer-executable instructions are used to implement any one of the data management methods provided by the embodiments of the present application.
  • an embodiment of the present application further provides a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and the computer-executable instructions are used to implement any one of the data management methods provided in the foregoing embodiments.
  • FIG. 9 is an optional structural schematic diagram of the electronic device provided by the embodiment of the present application.
  • the electronic device 900 includes:
  • the processor 902 is configured to implement any one of the above data management methods when executing the executable instructions stored in the memory 901 .
  • the above-mentioned processor 902 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.
  • the above-mentioned computer-readable storage medium/memory can be a read-only memory (Read Only Memory, ROM), a programmable read-only memory (Programmable Read-Only Memory, PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory) Memory, EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (FRAM), Flash Memory (Flash Memory), Magnetic Surface Memory, optical disk, or memory such as Compact Disc Read-Only Memory (CD-ROM); it can also be various terminals including one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital Assistant etc.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.
  • the unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solutions of the embodiments of the present application.
  • each functional unit in each embodiment of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration
  • the unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.
  • the above-mentioned integrated units of the present application are implemented in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application may be embodied in the form of software products in essence or the parts that contribute to related technologies.
  • the computer software products are stored in a storage medium and include several instructions to make The automatic test line of the device performs all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Environmental & Geological Engineering (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请实施例提供了一种数据管理方法、装置、电子设备及计算机存储介质；该方法包括：建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；基于所述数据存储架构，向每个子集群的每个节点写入数据。

Description

数据管理方法、装置、设备、计算机存储介质和程序
相关申请的交叉引用
本申请基于申请号为202011139659.7、申请日为2020年10月22日、名称为“数据管理方法、装置、设备及计算机存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及金融科技(Fintech)的分布式结构技术，涉及但不限于一种数据管理方法、装置、电子设备、计算机存储介质和计算机程序。
背景技术
随着计算机技术的发展，越来越多的技术应用在金融领域，传统金融业正在逐步向金融科技转变，但由于金融行业的安全性、实时性要求，也对技术提出了更高的要求。
目前，在金融科技领域中，可以基于单层的分布式架构来实现数据管理，单层的分布式架构不适合超大型的分布式集群，具体原因是，超大分布式集群节点众多，基于超大型的分布式集群进行数据管理的前提是：通过集群选举确定主节点，然而，在基于单层的分布式架构进行集群选举时，各个节点都要向集群中其他节点发送选举消息，容易造成消息风暴，导致大部分流量都被集群选举发送的报文占据，影响正常报文的传输。
发明内容
本申请实施例提供一种数据管理方法、装置、电子设备、计算机存储介质和计算机，可以解决现有技术中选举主节点导致的消息风暴的问题。
本申请实施例的技术方案是这样实现的：
本申请实施例提供一种数据管理方法，所述方法包括：
建立数据存储架构，所述数据存储架构包括父集群(Parent Cluster)和多个子集群(Child Cluster)，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；
基于所述数据存储架构，向每个子集群的每个节点写入数据。
在本申请的一些实施例中，所述建立数据存储架构包括：
确定各个子集群的主节点和健康程度指标；
基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；
基于所述各个子集群的主节点和所述父集群的主节点，建立所述数据存储架构。
在本申请的一些实施例中，所述方法还包括：
根据以下至少一项得出每个子集群的主节点的健康程度指标：出现故障的次数、至少一次故障的持续时间、上次故障距离当前时间的时长。
可以看出，本申请实施例可以基于故障信息，准确地得出每个子集群的主节点的健康程度指标，因而，有利于在子集群的主节点中，准确地选取出健康程度较高的节点作为父集群的主节点，
在本申请的一些实施例中，所述基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于所述各个子集群的主节点的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；其中，所述第一评分值为预先配 置的数值。
可以看出，本申请实施例中不仅仅基于节点的ID来进行父集群的主节点的选取，而是可以在综合考虑节点的ID对应的分值和健康程度指标的基础上，选取父集群的主节点，有利于降低ID的分值较大的节点频繁加入集群导致的主节点频繁更换的问题。
在本申请的一些实施例中，所述基于所述各个子集群的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的主节点的第二评分值，所述第二评分值为所述健康程度指标和第一评分值之和；
基于各个子集群的主节点的第二评分值，由各个子集群的主节点选举出所述父集群的主节点。
可以理解地，本申请实施例可以基于第二评分值，通过子集群的选举得到父集群的主节点，而第二评分值不仅与节点的ID有关，还与节点的健康程度指标相关，因而，本申请实施例可以降低ID的分值较大的节点频繁加入集群导致的主节点频繁更换的问题。
在本申请的一些实施例中，所述基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，所述第一消息表示加入父集群的消息(Join Message)；
在所述每个子集群的主节点获取各个子集群的第一消息的情况下，基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点。
可以看出，本申请实施例可以基于各个子集群的主节点的第一消息，在合适的时机发起选取父集群的主节点的流程，有利于准确地在各个子集群的主节点中选取出父集群的主节点。
在本申请的一些实施例中，所述基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，包括：
在所述各个子集群中选取种子集群(Seed Cluster)；
在所述种子集群的主节点(Seed Cluster Master)接收到其它子集群的主节点发送的第一消息后，通过所述种子集群与所述其它子集群的数据交互，使所述每个子集群的主节点获取所述各个子集群的第一消息；其中，所述其它子集群为所述多个子集群中除所述种子集群外的子集群；所述第一消息是所述其它子集群的主节点基于所述种子集群的主节点的地址发送的，所述种子集群的主节点的地址是所述其它子集群的主节点预先确定的信息。
可以看出，本申请实施例可以通过种子集群的选取，实现各个子集群的主节点的数据交互，进而，有利于在各个子集群的主节点中选取出父集群的主节点。
在本申请的一些实施例中，所述方法还包括：
在所述父集群的从节点发生故障的情况下，将发生故障的从节点从所述父集群中删除，利用父集群的主节点向父集群的从节点发送父集群的成员变更信息；
在父集群的主节点接收到第一子集群的主节点发送的第一消息后，在所述父集群中加入所述第一子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述第一子集群表示所述发生故障的从节点所属的子集群，第一子集群的主节点表示在所述述第一子集群的各节点中重新选取出的主节点；所述第一消息表示加入父集群的消息。
可以理解地，本申请实施例可以在父集群的从节点发生故障时，及时更新父集群的成员变更信息，在后续进行父集群的主节点的选举时，有利于准确地实现父集群的主节点的选举。
在本申请的一些实施例中，所述方法还包括：
在所述父集群的主节点发生故障的情况下，在所述父集群的其它节点中选取一个节点作为所述父集群的主节点；
在接收到第二子集群的主节点发送的第一消息后，在所述父集群中加入所述第二子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述第二子集群表示发生故障的主节点所属的子集群，所述第一消息表示加入父集群的消息。
可以看出，本申请实施例可以在父集群的主节点发生故障时，及时重新选举父集群的主节点信息，并可以在第二子集群选取出主节点后，接收第二子集群的主节点的加入消息，有利于实现后续的父集群的主节点的重新选举。
在本申请的一些实施例中，所述方法还包括：
获取数据读取指令，所述数据读取指令中携带有任意一个节点的数据读取地址；
基于所述数据读取地址，从所述任意一个节点中读取数据。
可以看出，由于各个子集群中的数据是一致的，因而，本申请实施例可以基于任意一个子集群中的任意一个节点读取数据，具有易于实现的特点。
在本申请的一些实施例中，所述基于所述数据存储架构，向每个子集群的每个节点写入数据，包括：
获取数据写入指令，所述数据写入指令中携带有待写入数据；
将所述数据写入指令发送至所述父集群的主节点，所述父集群的主节点将所述数据写入指令发送至各个子集群的主节点，所述每个子集群的主节点将所述数据写入指令发送至本子集群的每个从节点，使每个子集群的每个节点写入所述待写入数据。
可以看出，本申请实施例可以基于数据写入指令向各个子集群的各个节点写入相同的数据，从而可以实现各个子集群的数据一致。
在本申请的一些实施例中，所述待写入数据包括至少两级标签数据；
所述每个子集群的每个节点写入所述待写入数据包括：
针对每级标签数据建立哈希表，基于各级标签数据对应的哈希表构建多叉树数据结构；
基于所述多叉树数据结构，向每个子集群的每个节点中写入所述至少两级标签数据。
可以看出，本申请实施例基于多叉树数据结构，可以实现多级标签数据的分级存储，有利于后续对多级标签数据进行管理。
在本申请的一些实施例中，在任意一级标签数据不是最低一级标签数据的情况下，所述任意一级标签数据的哈希表包括下一级标签数据对应的哈希地址。
可以看出，基于本申请实施例的多叉树数据结构，可以从当前一级的标签数据，快速查询到下一级标签数据。
在本申请的一些实施例中，所述方法还包括：
基于所述多叉树数据结构，对所述至少两级标签数据执行以下至少一项操作：增加、删除、修改、查询。
可以看出，基于多叉树数据结构的特点，有利于快速实现至少两级标签数据的增加、删除、修改或查询。
本申请实施例提供一种数据管理装置，所述装置包括：
建立模块，配置为建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；
处理模块，配置为基于所述数据存储架构，向每个子集群的每个节点写入数据。
本申请实施例提供一种电子设备，所述电子设备包括：
存储器，配置为存储可执行指令；
处理器，配置为执行所述存储器中存储的可执行指令时，实现上述任意一种数据管理方法。
本申请实施例提供一种计算机可读存储介质，存储有可执行指令，用于被处理器执行时，实现上述任意一种数据管理方法。
本申请实施例还提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令，该计算机可执行指令用于实现本申请实施例提供的任意一种数据管理方法。
本申请实施例中，建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；基于所述数据存储架构，向每个子集群的每个节点写入数据。可以看出，本申请实施例的数据存储架构为双层分布式架构，可以基于各个子集群的主节点实现数据交互，在进行集群选举时，不需要所有的子节点发送选举的协商消息，减少了协商消息数据，限制了协商消息的传播范围，有利于减缓消息风暴导致的影响正常报文传输的问题。
附图说明
图1本申请实施例提供的数据管理方法的一个可选的流程图；
图2是本申请实施例的数据存储架构的示意图；
图3是本申请实施例中集群建立的一个示意图；
图4是本申请实施例中选举父集群主节点的一个示意图；
图5是本申请实施例的多叉树数据结构的示意图；
图6是本申请实施例中对至少两级标签数据执行增加数据操作的流程示意图；
图7是本申请实施例中对至少两级标签数据执行查询数据操作的流程示意图；
图8是本申请实施例的数据管理装置的一个可选的组成结构示意图；
图9是本申请实施例提供的电子设备的一个可选的组成结构示意图。
具体实施方式
在相关技术中，可以基于单层的分布式架构来实现数据管理，这种方案存在以下缺陷：
1)单层的分布式架构不适合超大型的分布式集群，因为超大分布式集群节点众多，基于超大型的分布式集群进行数据管理的前提是：通过集群选举确定主节点，然而，在进行集群选举时，各个节点都要向集群中其他节点发送选举消息，容易造成消息风暴，导致大部分流量都被集群选举发送的报文占据，影响正常报文的传输。
2)在通过集群选举确定主节点，选举算法并没有在选举速度与稳定性方面做到平衡，如bully算法，进行集群选举时是单纯的根据节点ID的大小来判断的，这会造成频繁选举主节点的问题；如raft算法，进行集群选举使用的是多数投票机制，这种机制可以避免频繁选主，但是会导致选举时间变得更长，必须有半数以上节点进行投票才能选举出主节点。
3)使用单层的分布式架构来实现数据存储时，每个节点使用统一的一个哈希(Hash)表结构进行数据存储，当存储数据较多需要进行扩容时，需要计算节点中所有涉及的所有键值(key)的哈希值，导致扩容时间变长。
针对上述技术问题，提出本申请实施例的技术方案。
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，所描述的实施例不应视为对本申请的限制，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
本申请实施例提供一种数据管理方法、装置、电子设备及计算机存储介质，本申请实施例的数据管理方法可以应用于电子设备中，下面说明本申请实施例提供的电子设备的示例性应用，本申请实施例提供的电子设备可以实施为服务器，服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式***，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。
下面对本申请实施例的数据管理方法进行示例性说明。
图1为本申请实施例提供的数据管理方法的一个可选的流程图，如图1所示，该流程可以包括：
步骤101：建立数据存储架构，数据存储架构包括父集群和多个子集群，多个子集群中每个子集群包括一个主节点和至少一个从节点，父集群的节点为多个子集群的主节点，父集群的节点包括父集群的一个主节点和父集群的至少一个从节点。
本申请实施例中，子集群表示对所有集群的节点划分得出的小集群，每个子集群可以独立选举子集群主节点(Child Cluster Master)，子集群中除子集群主节点外的其余节点为子集群从节点(Child Cluster Slave)。
父集群表示由各个子集群主节点组成的集群，可以在父集群的节点中选取一个节点作为父集群主节点(Parent Cluster Master)，父集群中除父集群主节点外的其余节点为父集群从节点(Parent Cluster Slave)。
图2为本申请实施例的数据存储架构的示意图，如图2所示，子集群0、子集群1和子集群2表示3个不同的子集群；主节点01表示子集群0的主节点，从节点02和从节点03表示子集群0中不同的从节点；主节点11表示子集群1的主节点，从节点12和从节点13表示子集群1中不同的从节点；主节点21表示子集群2的主节点，从节点22和从节点23表示子集群2中不同的从节点；主节点01、主节点11和主节点21组成父集群，主节点21为父集群主节点，主节点11和主节点01为父集群中不同的从节点。
参照图2，本申请实施例的数据存储架构为双层结构，其中，第一层有多个子集群构成，第一层的子集群包括子集群0、子集群1和子集群2；第二层表示父集群，由多个子集群的主节点组成，即，父集群由主节点01、主节点11和主节点21组成。每个子集群在建立完成之后会独立选举出各自的主节点，然后，各个子集群的主节点通过选举可以得出父集群的主接地那，从而，将一个一个超大集群通过双层结构实现。
步骤102：基于数据存储架构，向每个子集群的每个节点写入数据。
本申请实施例中，基于上述数据存储结构，在接收到数据写入指令后，可以基于数据写入指令向各个子集群的各个节点写入相同的数据，从而可以实现各个子集群的数据一致。基于上述数据存储结构，可以从任意一个节点中读取数据，这是因为各个节点中的数据是一致的。
在实际应用中，步骤101至步骤102可以基于电子设备的处理器实现，上述处理器可以是特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)、CPU、控制器、微控制器、微处理器中的至少一种。可以理解地，实现上述处理器功能的电子器件还可以为其它，本申请实施例不作限制。
可以看出，本申请实施例的数据存储架构为双层分布式架构，可以基于各个子集群的主节点实现数据交互，在进行集群选举时，不需要所有的子节点发送选举的协商消息，减少了协商消息数据，限制了协商消息的传播范围，有利于减缓消息风暴导致的影响正常报文传输的问题。
本申请的一些实施例中，建立数据存储架构可以包括：确定各个子集群的主节点和健康程度指标；基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；基于所述各个子集群的主节点和所述父集群的主节点，建立所述数据存储架构。
这里，健康程度指标用于反映子集群的主节点的健康程度；在确定各个子集群的主节点后，可以确定出各个子集群的从节点，在确定父集群的主节点后，可以确定出父集群的从节点，从而，可以得到上述数据存储架构。
可以理解地，与相关技术中仅仅根据节点的ID进行主节点选举的方案不同，本申请实施例可以基于各个子集群的主节点的健康程度指标进行父集群的子节点的选取，与bully算法相比，即使ID较大的节点频繁加入也不会引起父集群主节点的频繁更换，稳定性较强；并且与raft算法相比，需要基于半数投票机制进行主节点的选举，确定主节点的速度更快，在一定程度上实现了选举速度与稳定性的平衡。
本申请的一些实施例中，可以根据以下至少一项得出每个子集群的主节点的健康程度指标：出现故障的次数、至少一次故障的持续时间、上次故障距离当前时间的时长。
这里，健康程度指标与出现故障的次数成负相关，健康程度指标与至少一次故障的持续时间成负相关，健康程度指标与上次故障距离当前时间的时长；即，至少一次故障的持续时间越短，则健康程度指标的数值越大；上次故障距离当前时间的时长越长，则说明健康程度指标的数值越大；出现故障的次数越大，则说明健康程度指标的数值越小。
在一些实施例中，至少一次故障的持续时间可以包括上次故障的持续时间。
在实际应用中，可以在节点每次故障时，记录故障开始时间和故障恢复时间，从而确定故障持续时间。
在一些实施例中，可以根据以下公式(1)得出每个子集群的主节点的健康程度指标：
S0＝t1-(fault_count*t2)          (1)
其中，S0表示子集群的主节点的健康程度指标，t1表示上次故障距离当前时间的时长，fault_count表示出现故障的次数，t2表示上次故障的持续时间。
可以理解地，本申请实施例可以基于故障信息，准确地得出每个子集群的主节点的健康程度指标，因而，有利于在子集群的主节点中，准确地选取出健康程度较高的节点作为父集群的主节点，
本申请的一些实施例中，可以基于各个子集群的主节点的健康程度指标和第一评分值，在各个子集群的主节点中，选取一个主节点作为父集群的主节点；其中，第一评分值为预先配置的数值。
在一些实施例中，第一评分值可以是节点的ID对应的分值；在实际应用中，可以预先确定第一评分值，即，每个节点的第一评分值为固定不变的数值。
可以看出，本申请实施例中不仅仅基于节点的ID来进行父集群的主节点的选取，而是可以在综合考虑节点的ID对应的分值和健康程度指标的基础上，选取父集群的主节点，有利于降低ID的分值较大的节点频繁加入集群导致的主节点频繁更换的问题。
本申请的一些实施例中，上述基于各个子集群的健康程度指标和第一评分值，在各个子集群的主节点中，选取一个主节点作为父集群的主节点，可以包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的主节点的第二评分值，第二评分值为所述健康程度指标和第一评分值之和；
基于各个子集群的主节点的第二评分值，由各个子集群的主节点选举出所述父集群的主节点。
在实际应用中，每个子集群的主节点可以计算本节点的第二评分值，然后，通过各个子集群的主节点的数据交互，可以获取其它子集群的主节点的第二评分值，进而，可以判断本节点和其它子集群的主节点的第二评分值的大小关系，判断，本节点是否可以 作为父集群的主节点。
在一些实施例中，如果本节点的第二评分值大于或等于其它子集群的主节点的第二评分值，则说明本节点可以作为父集群的主节点；如果本节点的第二评分值小于任意一个其它子集群的主节点的第二评分值，则说明本节点不能作为父集群的主节点。
可以理解地，本申请实施例可以基于第二评分值，通过子集群的选举得到父集群的主节点，而第二评分值不仅与节点的ID有关，还与节点的健康程度指标相关，因而，本申请实施例可以降低ID的分值较大的节点频繁加入集群导致的主节点频繁更换的问题。
在本申请的一些实施例中，上述基于各个子集群的健康程度指标，在各个子集群的主节点中，选取一个主节点作为父集群的主节点，可以包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，第一消息表示加入(join)父集群的消息；
在每个子集群的主节点获取各个子集群的第一消息的情况下，基于各个子集群的健康程度指标，在各个子集群的主节点中，选取一个主节点作为所述父集群的主节点。
在实际应用中，每个子集群的主节点在生成第一消息即加入父集群的消息后，可以将第一消息发送至其它子集群的主节点；每个子集群的主节点也可以接收其它子集群的主节点发送的第一消息；在每个子集群的主节点获取到各个子集群的第一消息后，便可以发起选取父集群的主节点的流程。
可以看出，本申请实施例可以基于各个子集群的主节点的第一消息，在合适的时机发起选取父集群的主节点的流程，有利于准确地在各个子集群的主节点中选取出父集群的主节点。
在本申请的一些实施例中，上述基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，包括：
在各个子集群中选取种子集群；
在种子集群的主节点接收到其它子集群的主节点发送的第一消息后，通过种子集群与其它子集群的数据交互，使每个子集群的主节点获取各个子集群的第一消息；其中，其它子集群为多个子集群中除种子集群外的子集群；第一消息是其它子集群的主节点基于种子集群的主节点的地址发送的，种子集群的主节点的地址是其它子集群的主节点预先确定的信息。
在一些实施例中，可以在各个子集群中任意选取一个子集群作为种子集群，或者，可以在各个子集群中按照预设的选取方式选取种子集群。
在一些实施例中，可以在其它子集群的主节点中预先配置种子集群的主节点的地址，这样，其它子集群的主节点可以将第一消息发送至种子集群；种子集群在接收到第一消息后，可以记录第一消息的发送地址即其它子集群的主节点的地址，进而，种子集群可以将接收到的第一消息发送至各个子集群的主节点。
可以看出，本申请实施例可以通过种子集群的选取，实现各个子集群的主节点的数据交互，进而，有利于在各个子集群的主节点中选取出父集群的主节点。
图3为本申请实施例中集群建立的一个示意图，如图3所示，首先选取种子集群，种子集群为各个子集群中的一个子集群，种子集群、子集群1和子集群2表示3个不同的子集群；主节点01表示种子集群的主节点，从节点02和从节点03表示种子集群中的从节点(Seed Cluster Slave)；主节点11、从节点12和从节点13的含义与图2相同，主节点21、从节点22和从节点23的含义与图2相同，这里不再赘述。
下面参照图3说明集群建立的过程。
在种子集群选取完毕后，可以选取种子集群的主节点，然后，可以向种子集群的主节点中写入初始时刻所有子集群(包括子集群1和子集群2)的ID，这样，种子集群可以获知各个子集群的ID。
除种子集群外的其它子集群可以进选取主节点，可以预先在其它子集群中配置种子集群的ID和地址。
在每个子集群选取完主节点后，每个子集群的主节点可以向种子集群的主节点(如图3所示的主节点01)发送第一消息，第一消息包含子集群的ID和子集群的主节点的地址等信息。
种子集群的主节点收到其它子集群发送的第一消息后，可以根据第一消息将子集群的ID作为子集群信息进行存储，并记录对应的子集群的主节点的地址。
当种子集群接收到所有其它子集群的第一消息后，可以向各个其它子集群的主节点广播子集群信息，这里的子集群信息可以包括接收到的所有其它子集群的第一消息，这样，每个子集群就可以获取各个子集群的主节点的信息，此时，可以发起父集群的主节点的选举流程；这里，在确定发起父集群的主节点的选举流程，种子集群已经没有意义，需要与其它子集群进行公平竞争。
这里，种子集群在集群建立的初始化时刻为协调者，在每个子集群就可以获取各个子集群的主节点的信息后退化为子集群。
图4为本申请实施例中选举父集群主节点的一个示意图，图4中，子集群0、子集群1和子集群2表示3个不同的子集群；主节点01、从节点02、从节点03、主节点11、从节点12、从节点13、主节点21、从节点22和从节点23的含义与图2相同，这里不再赘述。
参照图4，每个子集群的主节点首先确定本节点的第二评分值，并可以向其它子集群的主节点发送成功(victory)消息，例如，主节点21可以向主节点11和主节点11和主节点01发送victory消息；victory消息表示宣布本节点是主节点的消息，在victory消息中可以携带本节点的第二评分值。
每个子集群的主节点可以接收其它子集群的主节点发送的victory消息，在接收到victory消息后，将victory消息中的第二评分值与本节点的第二评分值进行比较，如果victory消息中的第二评分值大于或等于本节点的第二评分值，则可以向victory消息的来源节点发送选举(election)消息，election消息表示选取victory消息的来源节点为父集群主节点的消息；如果victory消息中的第二评分值小于的第二评分值，则可以向victory消息的来源节点发送响应(alive)消息，这里，针对victory消息的alive消息为指示重新选举父集群主节点的消息。
例如，按照图4，在主节点21向主节点11和主节点01发送victory消息后，如果victory消息中的第二评分值大于主节点11的第二评分值，则主节点11可以向主节点21发送election消息；如果victory消息中的第二评分值大于主节点01的第二评分值，则主节点01可以向主节点21发送election消息。
每个子集群的主节点在收到election消息后，可以回复alive消息，这里，针对election消息的alive消息表示已经接收到election消息的回复信息。例如，参照图4，主节点21在收到主节点11和主节点01发送的election消息后，可以分别向主节点11和主节点01发送alive消息。
每个子集群的主节点在收到针对victory消息的alive消息后，可以不针对该alive消息进行处理，等待接收其它子集群的主节点发送的消息。
任意一个子集群的主节点在向其它子集群的主节点发送victory的主节点后，如果在给定时间内未收到其它子集群的主节点发送的alive消息，则认为办呢节点为父集群的主节点，此时，可以向其它子集群的主节点发送victory消息，使其它子集群的主节点确定出父集群的主节点，这样，父集群的主节点的选举流程结束。
在一些实施例中，给定时间可以根据实际应用场景进行设置，例如，给定时间可以根据各个子集群之间的最大通信时延确定。
在本申请的一些实施例中，在父集群的节点发生故障时，需要进行针对性的处理，下面分别对父节点发生故障的两种情况进行说明。
示例性地，在父集群的从节点发生故障的情况下，将发生故障的从节点从父集群中删除，利用父集群的主节点向父集群的从节点发送父集群的成员变更信息；
在父集群的主节点接收到第一子集群的主节点发送的第一消息后，在父集群中加入第一子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；第一子集群表示发生故障的从节点所属的子集群，第一子集群的主节点表示在所述述第一子集群的各节点中重新选取出的主节点。
这里，父集群的成员变更信息可以包括父集群的各个节点的ID等信息。
在一些实施例中，父集群的主节点可以将发生故障的从节点从父集群中删除，标记对应的子集群的ID为不可用，并更新父集群的成员信息，然后，可以将父集群的成员变更信息广播至父集群的各个从节点。
第一子集群的主节点发生故障后，可以在第一子集群中重新选取出主节点，然后，第一子集群的主节点可以父集群的主节点发送第一消息；父集群的主节点接收到第一消息后，可以清楚上述不可用的标记，然后，更新父集群的成员信息，然后，可以将父集群的成员变更信息广播至父集群的各个从节点。
在一些实施例中，父集群的主节点还可以更新第一子集群相关的故障信息，以便于在选举父集群的主节点时使用。
可以理解地，本申请实施例可以在父集群的从节点发生故障时，及时更新父集群的成员信息，在后续进行父集群的主节点的选举时，有利于准确地实现父集群的主节点的选举。
示例性地，在父集群的主节点发生故障的情况下，在父集群的其它节点中选取一个节点作为父集群的主节点；
在接收到第二子集群的主节点发送的第一消息后，在父集群中加入第二子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；第二子集群表示发生故障的主节点所属的子集群。
在一些实施例中，在父集群的主节点发生故障的情况下，需要在父集群的各个从节点中选举一个节点作为父集群的主节点，选举父集群的主节点的过程已经在上述记载的内容中作出说明，这里不再赘述。
由于第二子集群的主节点发生故障，可以认为第二子集群的主节点与父集群脱离，此时，需要在第二子集群选取出主节点后，加入父集群，具体流程如下：
当父集群的主节点发生故障时，第二子集群中的其它节点可以删除父集群的主节点的信息，在第二子集群中重新选取出一个节点作为第二子集群的主节点；在选取出第二子集群的主节点后，第二子集群的主节点可以向任意一个父集群的节点发送第一消息；在一些实施例中，可以预先针对父集群的所有节点建立哈希环，第二子集群的主节点可以基于哈希环按照顺时针或逆时针方向临近选择一个节点，向选择的节点发送第一消息。
在上述任意一个父集群的节点为父集群当前的主节点的情况下，父集群的主节点可以直接记录第二子集群的ID和第二子集群当前的主节点的地址信息。在上述任意一个父集群的节点不是父集群当前的主节点的情况下，上述任意一个父集群的节点可以向第二子集群当前的主节点返回父集群的主节点信息，第二子集群的主节点可以基于父集群的主节点信息，向父集群的主节点发送第一消息。
可以看出，本申请实施例可以在父集群的主节点发生故障时，及时重新选举父集群的主节点信息，并可以在第二子集群选取出主节点后，接收第二子集群的主节点的加入消息，有利于实现后续的父集群的主节点的重新选举。
本申请的一些实施例中，上述基于数据存储架构，向每个子集群的每个节点写入数 据，可以包括：
获取数据写入指令，数据写入指令中携带有待写入数据；
将数据写入指令发送至父集群的主节点，父集群的主节点将所述数据写入指令发送至各个子集群的主节点，每个子集群的主节点将数据写入指令发送至本子集群的每个从节点，使每个子集群的每个节点写入所述待写入数据。
在一些实施例中，当接收到客户端发送的数据写入指令后，说明需要向数据存储架构中的每个节点写入数据，此时，可以将数据写入指令发送至父集群中的任意一个节点；如果接收数据写入指令的节点不是父集群的主节点，则接收数据写入指令的节点会回复父集群的主节点的信息至客户端，然后，客户端可以将数据写入指令发送至父集群的主节点，父集群的主节点可以将数据写入指令发送至各个子集群的主节点；如果接收数据写入指令的节点是父集群的主节点，父集群的主节点可以将数据写入指令发送至各个子集群的主节点。
在一些实施例中，当一个子集群的主节点确定子集群内所有节点均成功写入待写入数据后，会返回成功写入信息至父集群的主节点；否则，会返回未成功写入信息至父集群的主节点；当父集群的主节点收到各个子集群的主节点发送的成功写入信息时，可以向客户端返回成功写入信息；当父集群的主节点收到至少一个子集群的主节点发送的未成功写入信息时，可以确定上述至少一个子集群的节点发生故障，或者，可以确定向上述至少一个子集群未能成功写入数据，此时，可以重新向至少一个子集群的主节点发送数据写入指令，直至收到上述至少一个子集群的主节点发送的成功写入信息。
可以看出，本申请实施例可以基于数据写入指令向各个子集群的各个节点写入相同的数据，从而可以实现各个子集群的数据一致。
本申请的一些实施例中，还可以获取数据读取指令，数据读取指令中携带有任意一个节点的数据读取地址；然后，可以基于数据读取地址，从上述任意一个节点中读取数据。
可以看出，由于各个子集群中的数据是一致的，因而，本申请实施例可以基于任意一个子集群中的任意一个节点读取数据，具有易于实现的特点。
本申请的一些实施例中，上述待写入数据包括至少两级标签数据；
相应地，每个子集群的每个节点写入所述待写入数据包括：针对每级标签数据建立哈希表，基于各级标签数据对应的哈希表构建多叉树数据结构；基于所述多叉树数据结构，向每个子集群的每个节点中写入所述至少两级标签数据。
在相关技术中，微服务技术得到了广泛的应用，微服务的特点就是动态化，对于使用传统互联网协议(Internet Protocol，IP)来管理数据(例如可以是应用实例数据)来说是不能满足微服务动态化要求的；因而，在本申请实施例中提出了基于标签数据的数据管理方案，代替基于IP的数据管理方案。
在一些实施例中，待写入数据为应用实例数据，应用实例数据可以包括应用所述集群和应用所属***，应用实例数据可以表示为/cluster/system/app，这里，符号/表示固定分隔符，cluster、system和app分别表示集群、***和应用，cluster表示第一级标签，system表示第二级标签，app表示第三级标签。
图5为本申请实施例的多叉树数据结构的示意图，参照图5，在集群中，hashtable1-0表示集群数据的哈希地址，hashtable1-0中的bucket0和bucket1表示不同的存储数据，在hashtable1-0中的bucket0中，键(key)为cluster0，值(value)为hashtable1-0；在hashtable1-0中的bucket1中，键为cluster1，值为hashtable1-1。
参照图5，在***中，hashtable1-0和hashtable1-1表示不同的***数据的哈希地址，hashtable1-0中的bucket0和bucket1表示不同的存储数据，在hashtable1-0中的bucket0中，键为system0，值为hashtable2-0；在hashtable1-0中的bucket1中，键为system1，值 为hashtable2-1；hashtable1-1中的bucket0表示存储数据，在hashtable1-1中的bucket0中，键为system0，值为hashtable2-2。
参照图5，hashtable2-0、hashtable2-1和hashtable2-2分别表示不同的应用数据的哈希地址；在hashtable2-0中的bucket0表示存储数据，在hashtable2-0中的bucket0中，键为app0，值为ip1；hashtable2-1中的bucket0和bucket1表示不同的存储数据，在hashtable2-0中的bucket0中，键为app1，值为ip2；在hashtable2-0中的bucket1中，键为app2，值为ip2；在hashtable2-2中的bucket0中，键为app0，值为ip3；这里，app0、app1和app2为不同的应用，ip1、ip2和ip3表示不同的IP地址。
在一些实施例中，为了兼容传统的基于IP的数据管理架构，需要进行标签到IP的转换，在实际进行网络层面通信时将逻辑概念标签转换成物理概念IP即可完成实际物理层面的通信；例如，参照图5的第三级标签数据，可以建立标签和IP的对应关系，因而便于有效地存储和查找标签和IP的对应关系。
在一些实施例中，多叉树数据结构中的每个树节点均对应建立有哈希表，该哈希表用于存储对应的一级标签数据。
可以看出，本申请实施例基于多叉树数据结构，可以实现多级标签数据的分级存储，有利于后续对多级标签数据进行管理。
在本申请的一些实施例中，在任意一级标签数据不是最低一级标签数据的情况下，所述任意一级标签数据的哈希表包括下一级标签数据对应的哈希地址。
本申请实施例中，可以按照固定分隔符划分出多级标签数据，每级标签数据均对应一个多叉树数据结构的一个树节点，在多叉树数据结构的树节点不是叶子节点(不存在子节点的节点)时，树节点中的键为对应的哈希表的一个元素，值为下一级哈希表的哈希地址；这里，下一级哈希表的哈希地址为上述下一级标签数据对应的哈希地址。
可以看出，基于本申请实施例的多叉树数据结构，可以从当前一级的标签数据，快速查询到下一级标签数据。
基于多叉树数据结构的数据存储方案具有以下特点：
1)具有相同前缀字符串的标签数据可以共享同一个哈希表节点，可以降低存储空间使用率，例如，参照图5，在应用级标签数据中，hashtable2-1中的bucket0和bucket1具有相同的前缀字符串，即，hashtable2-1中的bucket0和bucket1的前缀字符串均为cluster0/system1；这样，hashtable2-1中的bucket0和bucket1共享相同的哈希表节点hashtable2-1。
2)基于多级标签数据的形式，采用固定分隔符分隔出来的字符串个数为固定的个数，所以多叉树的高度也是固定的，并且每个哈希表的查找时间都是常数级别，整体基于哈希表的查询时间的复杂度也是常数级别。
3)需要进行后缀通配符匹配时，如/cluster0/system0/*，只需要找到完全匹配前缀的节点，它所指向的哈希表节点就包含了后缀通配符所匹配到的所有信息，相比统一使用一个哈希表实现数据存储的方案相比，本申请实施例无需遍历与整个待写入数据对应的哈希表，可以使进行后缀通配符匹配的时间复杂度进一步降低。
4)在相关技术中，需要统一使用一个哈希表实现数据存储，因而，在该哈希表的数据量较大时，进行扩容时需要重新计算的键越多，导致扩容时间变长；在需要进行数据扩容时，只需要扩容当前需要放置新元素对应节点的哈希表即可，在一定程度上降低了扩容时间。
在本申请的一些实施例中，可以基于多叉树数据结构，对至少两级标签数据执行以下至少一项操作：增加、删除、修改、查询。
图6为本申请实施例中对至少两级标签数据执行增加数据操作的流程示意图，如图6所示，该流程可以包括：
步骤601：获取各级标签数据。
这里，可以根据前述记载的固定分隔符对多级标签数据进行分割，得到各级标签数据。
步骤602：取出第一级标签。
例如，对于数据/cluster/system/app，cluster表示第一级标签，system表示第二级标签，app表示第三级标签。
步骤603：判断当前取出的标签对应的哈希表节点是否存在，如果否，则执行步骤604；如果是，则执行步骤605。
步骤604：创建对应的哈希表节点，执行步骤605。
这里，可以针对当前取出的标签，创建对应的哈希表节点。
步骤605：判断哈希表节点中是否存在对应的标签，如果否，则执行步骤606，如果是，则执行步骤607。
步骤606：***对应的标签，然后，执行步骤607。
这里，可以在哈希表节点中***当前取出的标签，然后，执行步骤607。
步骤607：判断各级标签数据是否遍历完毕，如果是，则结束流程，如果否，则执行步骤608。
这里，如果已经取出了各级标签数据，这说明各级标签数据遍历完毕，否则，说明各级标签并没有遍历完毕。
步骤608：取出下一级标签数据，然后，返回至步骤603。
图7为本申请实施例中对至少两级标签数据执行查询数据操作的流程示意图，如图7所示，该流程可以包括：
步骤701：获取各级标签数据。
步骤702：取出第一级标签。
步骤703：判断当前取出的标签对应的哈希表节点是否存在，如果是，则执行步骤704，如果否，则执行步骤708。
步骤704：判断哈希表节点中是否存在对应的标签，如果是，则执行步骤705，如果否，则执行步骤708。
步骤705：判断各级标签数据是否遍历完毕，如果是，则执行步骤706，如果否，则执行步骤707。
步骤706：确定查询到对应的标签，然后结束流程。
这里，在确定查询到对应的标签后，可以读取出对应的标签数据；在实际应用中，可以在标签数据所在的哈希表节点中读取出相应的数据。
步骤707：取出下一级标签，然后返回至步骤703。
步骤708：确定未查询到对应的标签，然后，结束流程。
在一些实施例中，在对上述至少两级标签数据执行删除数据操作时，可以先查询到对应的数据，然后在对应的哈希节点中删除对应的数据。
在一些实施例中，在对上述至少两级标签数据执行修改数据操作时，可以先查询到对应的数据，然后在对应的哈希节点中修改对应的数据。
可以看出，基于多叉树数据结构的特点，有利于快速实现至少两级标签数据的增加、删除、修改或查询。
在前述实施例提出的数据管理方法的基础上，本申请实施例还提出了一种数据管理装置；图8为本申请实施例的数据管理装置的一个可选的组成结构示意图，如图8所示，该数据管理装置800可以包括：
建立模块801，配置为建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节 点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；
处理模块802，配置为基于所述数据存储架构，向每个子集群的每个节点写入数据。
在本申请的一些实施例中，所述建立模块801，配置为建立数据存储架构包括：
确定各个子集群的主节点和健康程度指标；
基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；
基于所述各个子集群的主节点和所述父集群的主节点，建立所述数据存储架构。
在本申请的一些实施例中，所述建立模块801，还配置为根据以下至少一项得出每个子集群的主节点的健康程度指标：出现故障的次数、至少一次故障的持续时间、上次故障距离当前时间的时长。
在本申请的一些实施例中，所述建立模块801，配置为基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于所述各个子集群的主节点的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；其中，所述第一评分值为预先配置的数值。
在本申请的一些实施例中，所述建立模块801，配置为基于所述各个子集群的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的主节点的第二评分值，所述第二评分值为所述健康程度指标和第一评分值之和；
基于各个子集群的主节点的第二评分值，由各个子集群的主节点选举出所述父集群的主节点。
在本申请的一些实施例中，所述建立模块801，配置为基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，所述第一消息表示加入父集群的消息；
在所述每个子集群的主节点获取各个子集群的第一消息的情况下，基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点。
在本申请的一些实施例中，所述建立模块801，配置为基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，包括：
在所述各个子集群中选取种子集群；
在所述种子集群的主节点接收到其它子集群的主节点发送的第一消息后，通过所述种子集群与所述其它子集群的数据交互，使所述每个子集群的主节点获取所述各个子集群的第一消息；其中，所述其它子集群为所述多个子集群中除所述种子集群外的子集群；所述第一消息是所述其它子集群的主节点基于所述种子集群的主节点的地址发送的，所述种子集群的主节点的地址是所述其它子集群的主节点预先确定的信息。
在本申请的一些实施例中，所述建立模块801，还配置为：
在所述父集群的从节点发生故障的情况下，将发生故障的从节点从所述父集群中删除，利用父集群的主节点向父集群的从节点发送父集群的成员变更信息；
在父集群的主节点接收到第一子集群的主节点发送的第一消息后，在所述父集群中加入所述第一子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述 第一子集群表示所述发生故障的从节点所属的子集群，第一子集群的主节点表示在所述述第一子集群的各节点中重新选取出的主节点，所述第一消息表示加入父集群的消息。
在本申请的一些实施例中，所述建立模块801，还配置为：
在所述父集群的主节点发生故障的情况下，在所述父集群的其它节点中选取一个节点作为所述父集群的主节点；
在接收到第二子集群的主节点发送的第一消息后，在所述父集群中加入所述第二子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述第二子集群表示发生故障的主节点所属的子集群，所述第一消息表示加入父集群的消息。
在本申请的一些实施例中，所述处理模块802，还配置为：
获取数据读取指令，所述数据读取指令中携带有任意一个节点的数据读取地址；
基于所述数据读取地址，从所述任意一个节点中读取数据。
在本申请的一些实施例中，所述处理模块802，配置为基于所述数据存储架构，向每个子集群的每个节点写入数据，包括：
获取数据写入指令，所述数据写入指令中携带有待写入数据；
将所述数据写入指令发送至所述父集群的主节点，所述父集群的主节点将所述数据写入指令发送至各个子集群的主节点，所述每个子集群的主节点将所述数据写入指令发送至本子集群的每个从节点，使每个子集群的每个节点写入所述待写入数据。
在本申请的一些实施例中，所述待写入数据包括至少两级标签数据；
所述每个子集群的每个节点写入所述待写入数据包括：
针对每级标签数据建立哈希表，基于各级标签数据对应的哈希表构建多叉树数据结构；
基于所述多叉树数据结构，向每个子集群的每个节点中写入所述至少两级标签数据。
在本申请的一些实施例中，在任意一级标签数据不是最低一级标签数据的情况下，所述任意一级标签数据的哈希表包括下一级标签数据对应的哈希地址。
在本申请的一些实施例中，所述处理模块，还配置为基于所述多叉树数据结构，对所述至少两级标签数据执行以下至少一项操作：增加、删除、修改、查询。
在实际应用中，建立模块801和处理模块802均可以利用处理器实现，上述处理器可以是ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。可以理解地，实现上述处理器功能的电子器件还可以为其它，本申请实施例不作限制。
需要说明的是，以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请装置实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
需要说明的是，本申请实施例中，如果以软件功能模块的形式实现上述的数据管理方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是终端、服务器等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本申请实施例不限制于任何特定的硬件和软件结合。
对应地，本申请实施例再提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令，该计算机可执行指令用于实现本申请实施例提供的任意一种数据管理方法。
相应的，本申请实施例再提供一种计算机存储介质，所述计算机存储介质上存储有 计算机可执行指令，该计算机可执行指令用于实现上述实施例提供的任意一种数据管理方法。
本申请实施例还提供一种电子设备，图9为本申请实施例提供的电子设备的一个可选的组成结构示意图，如图9所示，所述电子设备900包括：
存储器901，用于存储可执行指令；
处理器902，用于执行所述存储器901中存储的可执行指令时，实现上述任意一种数据管理方法。
上述处理器902可以为ASIC、DSP、DSPD、PLD、FPGA、CPU、控制器、微控制器、微处理器中的至少一种。
上述计算机可读存储介质/存储器可以是只读存储器(Read Only Memory，ROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、磁性随机存取存储器(Ferromagnetic Random Access Memory，FRAM)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(Compact Disc Read-Only Memory，CD-ROM)等存储器；也可以是包括上述存储器之一或任意组合的各种终端，如移动电话、计算机、平板设备、个人数字助理等。
这里需要指出的是：以上存储介质和设备实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请存储介质和设备实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
应理解，说明书通篇中提到的“一些实施例”意味着与实施例有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各处出现的“在一些实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个***，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本申请实施例方案的目的。
另外，在本申请各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
或者，本申请上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得设备自动测试线执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。
本申请所提供的几个方法实施例中所揭露的方法，在不冲突的情况下可以任意组合，得到新的方法实施例。
本申请所提供的几个方法或设备实施例中所揭露的特征，在不冲突的情况下可以任意组合，得到新的方法实施例或设备实施例。
以上所述，仅为本申请的实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (18)

  1. 一种数据管理方法，所述方法包括：
    建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；
    基于所述数据存储架构，向每个子集群的每个节点写入数据。
  2. 根据权利要求1所述的方法，其中，所述建立数据存储架构包括：
    确定各个子集群的主节点和健康程度指标；
    基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；
    基于所述各个子集群的主节点和所述父集群的主节点，建立所述数据存储架构。
  3. 根据权利要求2所述的方法，其中，所述方法还包括：
    根据以下至少一项得出每个子集群的主节点的健康程度指标：出现故障的次数、至少一次故障的持续时间、上次故障距离当前时间的时长。
  4. 根据权利要求2所述的方法，其中，所述基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
    基于所述各个子集群的主节点的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点；其中，所述第一评分值为预先配置的数值。
  5. 根据权利要求4所述的方法，其中，所述基于所述各个子集群的健康程度指标和第一评分值，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
    基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的主节点的第二评分值，所述第二评分值为所述健康程度指标和第一评分值之和；
    基于各个子集群的主节点的第二评分值，由各个子集群的主节点选举出所述父集群的主节点。
  6. 根据权利要求2所述的方法，其中，所述基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点，包括：
    基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，所述第一消息表示加入父集群的消息；
    在所述每个子集群的主节点获取各个子集群的第一消息的情况下，基于所述各个子集群的健康程度指标，在所述各个子集群的主节点中，选取一个主节点作为所述父集群的主节点。
  7. 根据权利要求6所述的方法，其中，所述基于各个子集群的主节点的数据交互，使每个子集群的主节点获取各个子集群的第一消息，包括：
    在所述各个子集群中选取种子集群；
    在所述种子集群的主节点接收到其它子集群的主节点发送的第一消息后，通过所述种子集群与所述其它子集群的数据交互，使所述每个子集群的主节点获取所述各个子集群的第一消息；其中，所述其它子集群为所述多个子集群中除所述种子集群外的子集群；所述第一消息是所述其它子集群的主节点基于所述种子集群的主节点的地址发送的，所述种子集群的主节点的地址是所述其它子集群的主节点预先确定的信息。
  8. 根据权利要求1所述的方法，其中，所述方法还包括：
    在所述父集群的从节点发生故障的情况下，将发生故障的从节点从所述父集群中删除，利用父集群的主节点向父集群的从节点发送父集群的成员变更信息；
    在父集群的主节点接收到第一子集群的主节点发送的第一消息后，在所述父集群中加入所述第一子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述第一子集群表示所述发生故障的从节点所属的子集群，所述第一子集群的主节点表示在所述述第一子集群的各节点中重新选取出的主节点，所述第一消息表示加入父集群的消息。
  9. 根据权利要求1所述的方法，其中，所述方法还包括：
    在所述父集群的主节点发生故障的情况下，在所述父集群的其它节点中选取一个节点作为所述父集群的主节点；
    在接收到第二子集群的主节点发送的第一消息后，在所述父集群中加入所述第二子集群的主节点，并向父集群的各节点发送父集群的成员变更信息；所述第二子集群表示发生故障的主节点所属的子集群，所述第一消息表示加入父集群的消息。
  10. 根据权利要求1所述的方法，其中，所述方法还包括：
    获取数据读取指令，所述数据读取指令中携带有任意一个节点的数据读取地址；
    基于所述数据读取地址，从所述任意一个节点中读取数据。
  11. 根据权利要求1-10任一项所述的方法，其中，所述基于所述数据存储架构，向每个子集群的每个节点写入数据，包括：
    获取数据写入指令，所述数据写入指令中携带有待写入数据；
    将所述数据写入指令发送至所述父集群的主节点，所述父集群的主节点将所述数据写入指令发送至各个子集群的主节点，所述每个子集群的主节点将所述数据写入指令发送至本子集群的每个从节点，使每个子集群的每个节点写入所述待写入数据。
  12. 根据权利要求11所述的方法，其中，所述待写入数据包括至少两级标签数据；
    所述每个子集群的每个节点写入所述待写入数据包括：
    针对每级标签数据建立哈希表，基于各级标签数据对应的哈希表构建多叉树数据结构；
    基于所述多叉树数据结构，向每个子集群的每个节点中写入所述至少两级标签数据。
  13. 根据权利要求12所述的方法，其中，在任意一级标签数据不是最低一级标签数据的情况下，所述任意一级标签数据的哈希表包括下一级标签数据对应的哈希地址。
  14. 根据权利要求12所述的方法，其中，所述方法还包括：
    基于所述多叉树数据结构，对所述至少两级标签数据执行以下至少一项操作：增加、删除、修改、查询。
  15. 一种数据管理装置，所述装置包括：
    建立模块，用于建立数据存储架构，所述数据存储架构包括父集群和多个子集群，所述多个子集群中每个子集群包括一个主节点和至少一个从节点，所述父集群的节点为所述多个子集群的主节点，所述父集群的节点包括所述父集群的一个主节点和所述父集群的至少一个从节点；
    处理模块，用于基于所述数据存储架构，向每个子集群的每个节点写入数据。
  16. 一种电子设备，所述电子设备包括：
    存储器，用于存储可执行指令；
    处理器，用于执行所述存储器中存储的可执行指令时，实现权利要求1至14任一项所述的数据管理方法。
  17. 一种计算机可读存储介质，存储有可执行指令，用于被处理器执行时，实现权利要求1至14任一项所述的数据管理方法。
  18. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1至14任一所述的数据管理方法。
PCT/CN2021/125290 2020-10-22 2021-10-21 数据管理方法、装置、设备、计算机存储介质和程序 WO2022083690A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011139659.7A CN112311596B (zh) 2020-10-22 2020-10-22 数据管理方法、装置、设备及计算机存储介质
CN202011139659.7 2020-10-22

Publications (1)

Publication Number Publication Date
WO2022083690A1 true WO2022083690A1 (zh) 2022-04-28

Family

ID=74327069

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/125290 WO2022083690A1 (zh) 2020-10-22 2021-10-21 数据管理方法、装置、设备、计算机存储介质和程序

Country Status (2)

Country Link
CN (1) CN112311596B (zh)
WO (1) WO2022083690A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242812A (zh) * 2022-07-25 2022-10-25 济南浪潮数据技术有限公司 一种节点数据同步方法、装置及计算机可读存储介质
CN115793992A (zh) * 2023-02-07 2023-03-14 天翼云科技有限公司 数据存储方法、装置、电子设备及可读存储介质
CN115811525A (zh) * 2023-02-09 2023-03-17 杭州合众数据技术有限公司 基于分布式架构的数据交换与处理方法
CN116389786A (zh) * 2023-03-29 2023-07-04 深圳市安飞信息有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN117666970A (zh) * 2024-01-30 2024-03-08 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112311596B (zh) * 2020-10-22 2023-05-12 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质
CN113067883B (zh) * 2021-03-31 2023-07-28 建信金融科技有限责任公司 数据传输方法、装置、计算机设备及存储介质
CN113660339B (zh) * 2021-08-18 2023-08-04 北京百度网讯科技有限公司 用于去中心化集群的方法和装置
CN114357049B (zh) * 2022-01-07 2024-01-19 苏州浪潮智能科技有限公司 一种存储集群互联方法、装置、计算机设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105721545A (zh) * 2016-01-20 2016-06-29 浪潮(北京)电子信息产业有限公司 一种多层次集群管理实现方法
CN108134706A (zh) * 2018-01-02 2018-06-08 中国工商银行股份有限公司 区块链多活高可用***、计算机设备以及方法
CN110169040A (zh) * 2018-07-10 2019-08-23 深圳花儿数据技术有限公司 基于多层一致性哈希的分布式数据存储方法与***
CN110308984A (zh) * 2019-04-30 2019-10-08 北京航空航天大学 一种用于处理地理分布式数据的跨集群计算***
US10719417B2 (en) * 2018-01-30 2020-07-21 EMC IP Holding Company, LLC Data protection cluster system supporting multiple data tiers
CN112311596A (zh) * 2020-10-22 2021-02-02 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8675672B1 (en) * 2011-12-30 2014-03-18 Emc Corporation Hierarchical cluster tree overlay network
CN105007193A (zh) * 2015-08-19 2015-10-28 浪潮(北京)电子信息产业有限公司 一种多层信息处理方法、***及集群管理节点
CN107426038A (zh) * 2017-09-12 2017-12-01 郑州云海信息技术有限公司 一种采集数据的分布式集群***及数据采集方法
CN108540310B (zh) * 2018-03-15 2021-05-11 南京邮电大学 一种基于无线网络协同感知的行为预测方法
CN108874489A (zh) * 2018-04-18 2018-11-23 李世伟 一种多功能业务处理方法和装置
CN110543410A (zh) * 2019-09-05 2019-12-06 曙光信息产业(北京)有限公司 一种处理集群指标的方法、查询集群指标的方法和装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105721545A (zh) * 2016-01-20 2016-06-29 浪潮(北京)电子信息产业有限公司 一种多层次集群管理实现方法
CN108134706A (zh) * 2018-01-02 2018-06-08 中国工商银行股份有限公司 区块链多活高可用***、计算机设备以及方法
US10719417B2 (en) * 2018-01-30 2020-07-21 EMC IP Holding Company, LLC Data protection cluster system supporting multiple data tiers
CN110169040A (zh) * 2018-07-10 2019-08-23 深圳花儿数据技术有限公司 基于多层一致性哈希的分布式数据存储方法与***
CN110308984A (zh) * 2019-04-30 2019-10-08 北京航空航天大学 一种用于处理地理分布式数据的跨集群计算***
CN112311596A (zh) * 2020-10-22 2021-02-02 深圳前海微众银行股份有限公司 数据管理方法、装置、设备及计算机存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242812A (zh) * 2022-07-25 2022-10-25 济南浪潮数据技术有限公司 一种节点数据同步方法、装置及计算机可读存储介质
CN115793992A (zh) * 2023-02-07 2023-03-14 天翼云科技有限公司 数据存储方法、装置、电子设备及可读存储介质
CN115811525A (zh) * 2023-02-09 2023-03-17 杭州合众数据技术有限公司 基于分布式架构的数据交换与处理方法
CN116389786A (zh) * 2023-03-29 2023-07-04 深圳市安飞信息有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN116389786B (zh) * 2023-03-29 2024-04-05 南京浮点智算数字科技有限公司 基于节点扩容的视频云存储方法、装置和电子设备
CN117666970A (zh) * 2024-01-30 2024-03-08 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置
CN117666970B (zh) * 2024-01-30 2024-05-07 杭州阿里云飞天信息技术有限公司 数据存储方法以及数据存储装置

Also Published As

Publication number Publication date
CN112311596A (zh) 2021-02-02
CN112311596B (zh) 2023-05-12

Similar Documents

Publication Publication Date Title
WO2022083690A1 (zh) 数据管理方法、装置、设备、计算机存储介质和程序
US8073972B2 (en) System and method for location discovery based on DNS
RU2400806C2 (ru) Организация стыковки запросов на ресурс с соответствующими ресурсами
US9405781B2 (en) Virtual multi-cluster clouds
RU2431184C2 (ru) Межблизостная связь в федерации рандеву
US9104560B2 (en) Two level addressing in storage clusters
US8874708B2 (en) Location discovery based on DNS
JP7270755B2 (ja) 分散システムでのメタデータルーティング
TW200803303A (en) Inter-proximity communication within a rendezvous federation
WO2020063763A1 (zh) 数据存储方法、装置、***、服务器、控制节点及介质
JP2008533564A (ja) データ管理のための方法および装置
US8296420B2 (en) Method and apparatus for constructing a DHT-based global namespace
US9407687B2 (en) Method, apparatus, and network system for acquiring content
CN112468525B (zh) 一种基于区块链的域名管理***
WO2023050986A1 (zh) 维护区块链***的网络架构信息
CN109818948B (zh) 应用部署方法、装置、***、设备及介质
CN106687943A (zh) 组织具有多个计算机的计算***、在计算机之中分配计算任务、以及在计算机***中维护数据完整性与冗余的***和方法
WO2008089616A1 (fr) Serveur, système réseau p2p et procédé d'acheminement et de transfert de l'affectation de la clé de ressource de ce dernier.......
Yu et al. Granary: A sharing oriented distributed storage system
CN115865844B (zh) 基于sdn与ndn的虚实结合动态流量调度方法及装置
WO2022078012A1 (zh) 区块链存储的数据持续保存的方法、***、设备及介质
US20040122944A1 (en) Method and system of locating computers in distributed computer system
WO2024108493A1 (zh) 基于sdn与ndn的虚实结合动态流量调度方法及装置
WO2023160040A1 (zh) 基于区块链的数据处理方法、装置、设备及可读存储介质
CN107846290B (zh) 一种拓补关系管理方法和管理设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21882105

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21882105

Country of ref document: EP

Kind code of ref document: A1