CN113760468A

CN113760468A - Distributed election method, device, system and medium

Info

Publication number: CN113760468A
Application number: CN202110069889.9A
Authority: CN
Inventors: 藏茱飞
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2021-01-19
Filing date: 2021-01-19
Publication date: 2021-12-07

Abstract

The invention discloses a distributed election method, a device, a system and a medium, wherein the method comprises the following steps: when the trigger election condition is met, the uniquely existing child node in the cluster receives an election message, and the election message is sent to the child node from the slave node so as to promote the child node to become a new master node; determining whether the number of election messages received by a child node exceeds a predetermined threshold; when the number of election messages exceeds a preset threshold value, the child node is switched to a new main node; and the new master node selects a new child node from the slave nodes. The invention can enable the cluster to rapidly elect the main node by establishing the role of the child node, thereby avoiding a large amount of RPC communication, avoiding election failure caused by the fact that votes are sorted by the scores, and improving the working efficiency of the distributed system.

Description

Distributed election method, device, system and medium

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a distributed election method, apparatus, system, and medium.

Background

One core problem solved by the distributed technology is to improve the concurrent access capability of the system. When a system has a large number of requests, the overall service capacity of the system is improved by adding nodes of the system. And the data among all the nodes needs to be kept consistent, so that the whole system can be ensured to provide consistent data to the outside. To ensure consistent node data, one of the solutions is for one node to receive all write requests and then synchronize to the other nodes, i.e., the master-slave model. This makes the focus of the solution into how to elect the master node.

In the process of implementing the invention, the inventor finds that the existing distributed election method has many defects such as a large amount of RPC (remote procedure call) communication, election failure caused by votes being melon, or very complicated election process.

Disclosure of Invention

In view of this, embodiments of the present invention provide a distributed election method, apparatus and medium, which enable a cluster to rapidly elect a master node by establishing a child node role, so that a large amount of RPC communications can be avoided, and election failure caused by votes being voted up is avoided, so as to improve the working efficiency of a distributed system.

To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a distributed election method, including:

when a triggering election condition is met, receiving an election message by a child node which only exists in the cluster, wherein the election message is sent to the child node from a slave node so as to promote the child node to become a new master node;

determining whether the number of election messages received by the child node exceeds a predetermined threshold;

when the number of the election messages exceeds the preset threshold value, the child node is switched to a new main node; and

and the new main node selects a new child node from the slave nodes.

Further, the child node, after becoming the new master node, sends a broadcast to the cluster and suggests heartbeat communications with other nodes in the cluster.

Further, the new master node randomly selects the new child node from the slave nodes.

Further, the election triggering condition includes that a master node in the cluster is down or that the master node loses heartbeat communication with more than half of the slave nodes and the child nodes in the cluster.

Further, when the master node and the child nodes are down at the same time, or when the number of the election messages does not exceed half of the number of the slave nodes in the cluster, all the slave nodes in the cluster elect a new master node by using a Raft algorithm.

Further, after a new master node is elected by using the Raft algorithm, a new child node is randomly selected by the new master node.

Further, the predetermined threshold is half the number of slave nodes in the cluster.

According to another aspect of the present invention, there is provided a distributed election device, comprising:

the system comprises a receiving unit, a sending unit and a receiving unit, wherein the receiving unit is used for receiving election messages when triggering election conditions are met, and the election messages are sent from slave nodes to child nodes which exist only in the cluster so as to promote the child nodes to become new main nodes;

a determining unit, configured to determine whether the number of election messages received by the child node exceeds a predetermined threshold;

a switching unit, configured to switch the child node to a new master node when the number of election messages exceeds the predetermined threshold; and

a selecting unit, the first selecting unit being configured to select a new child node from the slave node.

Further, the distributed election device further comprises:

a broadcasting unit for sending out a broadcast to the cluster after the child node becomes the new master node.

According to yet another aspect of the present invention, there is provided a distributed election system, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement a method as described in one aspect above.

According to yet another aspect of the invention, a computer-readable medium is provided, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the method of the previous aspect.

One embodiment of the above invention has the following advantages or benefits: by arranging the child nodes in the cluster, the cluster is enabled to quickly elect the main nodes after the main nodes are down, so that the defects of a large amount of RPC communication in the existing election method and election failure caused by votes being sorted can be overcome, and the working efficiency of a distributed system can be improved.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of a main flow of a distributed election method according to an embodiment of the invention;

FIG. 2 is a schematic illustration of the election process of the nodes involved in the distributed election method according to the present invention;

FIG. 3 is a schematic diagram of the main modules of a distributed election device according to an embodiment of the present invention;

fig. 4 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Fig. 5 is a schematic diagram for explaining the ZAB algorithm election process.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Before proceeding with a detailed description of the present invention, a brief description will first be given of several distributed election methods that are improved upon by the present invention, in order to facilitate understanding of the technical problems to be solved and the technical effects that can be achieved.

The existing distributed election method applies more of contemporary Raft and ZAB (zookeeper Atomic broadcast), and the election of zookeeper applies ZAB algorithm, and the Raft algorithm can be said to be the preferred algorithm of the current distributed system, such as Consul, Etcd, and the like.

The implementation of the Raft algorithm is simple, votes are based on the principle of 'first come first get', a voting request initiated by a person is received first, the votes are cast to the person, and finally nodes with more than half votes are promoted to be main nodes. The drawback of the Raft algorithm is that votes may be scored, and no node obtains more than half of votes, resulting in failure of election, and then the next round of election is performed.

The process of the ZAB algorithm for electing the master node is relatively complex, candidates are needed to be compared, and finally, the candidate with the most complete data is elected as a leader. Specifically, referring to fig. 5, the voting information 2, 1, 10, B represents a node sequence number, an expiration, a data transaction Id (reflecting the update condition of data), and a voting object. After the master node a goes down, the node B, C casts one vote respectively, and then broadcasts the vote to other followers, the node C receives two votes, and finally casts the votes to the node C according to the principle that the deadline and the transaction ID are large, namely the node C becomes a new master node.

As described above, the Raft algorithm has the problems of melon sorting tickets and a large number of RPC communications, while the ZAB algorithm election process is time-division complex. The invention is provided for overcoming the defects, and can avoid the problems under normal conditions by establishing the child nodes in the cluster, thereby improving the working efficiency of the distributed system.

The distributed election method of the present invention will be described in detail below with reference to the accompanying drawings.

Fig. 1 is a main flow of a distributed election method according to an embodiment of the present invention, and as shown in fig. 1, the distributed election method according to the present invention includes a receiving step S101, a determining step S102, a switching step S103, a switching step S104, a selecting step S105, and a broadcasting step S106.

Before describing the above steps in detail, the identity of each node in the cluster to which the invention relates will be explained first.

A master node: there is only one master node in the cluster and all write requests are executed on the master node.

And (3) child nodes: only one child node is arranged in the cluster, and when the main node is down, the child node can be upgraded to the main node.

The slave node: all nodes except the main node and the sub-nodes in the cluster are slave nodes, the slave nodes only receive a client read request, and the write request needs to be forwarded to the main node.

After the identities of the above nodes are clarified, the above steps of the distributed election method of the present invention will be described in detail below with reference to fig. 2.

Step S101, receiving step

When the trigger election condition is met, the child node which exists only in the cluster receives an election message, and the election message is sent to the child node from the slave node so as to promote the child node to become a new master node.

Specifically, when a master node in the cluster goes down or more than half of the master nodes and other nodes (slave nodes and child nodes) in the cluster lose heartbeat communication, the trigger election condition is satisfied. At this time, the slave node communicates to the child node to send the election message, thereby electing the child node as the master node, and the child node receives the election message from the slave node.

Step S102, determining step

It is determined whether the number of election messages received by the child node exceeds a predetermined threshold, e.g., half the number of slave nodes in the cluster.

In particular, the child node counts the election messages received from the slave nodes and determines that the number of election messages received to elect itself to be the master node is more than half the number of slave nodes into the cluster.

Step S103, switching step

When the number of received election messages exceeds a predetermined threshold, for example half the number of slave nodes in the cluster, the child node switches to the new master node.

Step S104, selecting step

The new master node selects a new child node from the slave nodes.

Specifically, the new master node may select a new child node from the slave nodes according to a preset condition. For example, a new master node may randomly choose a new child node from the slave nodes.

Step S105, broadcasting step

After becoming the new master node, the child node sends out a broadcast (a broadcast in which the master node is switched successfully) to the cluster to notify each node in the cluster that it has become the new master node and establish heartbeat communication with other nodes.

It is to be noted that the following abnormal situations exist:

if the main node is down and the child nodes are also down, all the remaining slave nodes select a new main node by using a Raft algorithm, and the selected new main node executes the step S104 and the step S105 to select a new child node and send a broadcast.

If the number of election messages received through a plurality of polling sub-nodes does not exceed half of the number of the slave nodes, all the remaining slave nodes elect a new master node by using the Raft algorithm, and the elected new master node executes the steps S104 and S105 to select a new sub-node and send a broadcast.

And when the main node A goes down or is in heartbeat communication with more than half of the sub-nodes and the slave nodes, the trigger election condition is met. At this time, the slave nodes C1, C2, C3 … … Cn send election messages to the child node B to promote the child node B, and when the child node B receives more than half of the election messages from the slave nodes C1, C2, C3 … … Cn, the child node B switches to a new master node B ', and a new child node C' is randomly selected from the slave nodes C1, C2, C3 … … Cn.

Therefore, according to the distributed election method provided by the embodiment of the invention, the cluster can rapidly elect the main node by establishing the child node role, so that a large amount of RPC communication can be avoided, election failure caused by the fact that votes are sorted by the votes is avoided, and the working efficiency of a distributed system is improved.

According to another aspect of the invention, a distributed apparatus is provided. A distributed election device 200 according to an embodiment of the present invention will now be described with reference to fig. 3. The distributed election device 200 according to an embodiment of the present invention includes: a receiving unit 201, a determining unit 202, a switching unit 203, an extracting unit 204 and a broadcasting unit 205. The above-described respective units will be described in detail below.

Receiving unit 201

When a trigger election condition is met, the receiving unit 201 receives an election message that is sent from a slave node to a child node to promote the child node to become a new master node.

Specifically, when a master node in the cluster goes down or more than half of the master nodes and other nodes (slave nodes and child nodes) in the cluster lose heartbeat communication, the trigger election condition is satisfied. At this time, the slave node communicates to the child node to transmit the election message, thereby electing the child node as the master node, and the receiving unit 201 receives the election message from the slave node.

Determination unit 202

The determining unit 202 is configured to determine whether the number of election messages received by a slave node is above a predetermined threshold, e.g. more than half the number of slave nodes in the cluster.

Specifically, the determining unit 202 counts the election messages received from the slave nodes and determines whether the number of received election messages that elect itself to be the master node exceeds half the number of slave nodes in the cluster.

Switching unit 203

When the number of received election messages exceeds a predetermined threshold, e.g. half the number of slave nodes in the cluster, the switching unit 203 switches the child node to a new master node.

Selecting unit 204

The selecting unit 204 selects a new child node from the slave nodes.

Specifically, the selecting unit 204 may select a new child node from the slave nodes according to a preset condition. Alternatively, the selecting unit 204 may randomly select a new child node from the slave nodes.

Broadcast unit 205

After the child node becomes the new master node, the broadcasting unit 205 sends out a broadcast (a broadcast in which the master node is successfully switched) to the cluster to notify the nodes in the cluster that the original child node has become the new master node and establishes heartbeat communication with other nodes.

It should be noted that, if the child node is down while the master node is down, or the number of election messages received by the child node does not exceed half of the number of slave nodes, all the remaining slave nodes elect a new master node using the Raft algorithm, and the elected new master node uses the selecting unit and the broadcasting unit to select a new child node and send a broadcast.

The distributed election device according to the embodiment of the present invention is described above, and the distributed election device according to the embodiment of the present invention can enable the cluster to rapidly elect the master node by establishing the child node role, so that a large number of RPC communications can be avoided, and election failure caused by the fact that votes are sorted by melons can be avoided, so as to improve the working efficiency of the distributed system.

Referring now to FIG. 4, a block diagram of a computer system 400 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 4, the computer system 400 includes a Central Processing Unit (CPU)401 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. In the RAM 403, various programs and data necessary for the operation of the system 400 are also stored. The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section 407 including a display device such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the internet. A driver 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 410 as necessary, so that a computer program read out therefrom is mounted into the storage section 408 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 409, and/or installed from the removable medium 411. The computer program performs the above-described functions defined in the system of the present invention when executed by a Central Processing Unit (CPU) 401.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present invention may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a receiving unit, a determining unit, a switching unit, a selecting unit, and a broadcasting unit. Where the names of these elements do not in some cases constitute a limitation on the elements themselves, for example, a determination element may also be described as an "element that determines whether the number of election messages is more than half".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise:

and the new main node selects a new child node from the slave nodes.

According to the technical scheme of the embodiment of the invention, the cluster can rapidly elect the main node by establishing the child node role, so that a large amount of RPC communication can be avoided, election failure caused by the fact that votes are sorted is avoided, and the working efficiency of a distributed system is improved.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed election method, comprising:

and the new main node selects a new child node from the slave nodes.

2. The method of claim 1, further comprising:

the child node, after becoming the new master node, broadcasts to the cluster and suggests heartbeat communications with other nodes in the cluster.

3. The method according to claim 1 or 2, wherein,

and the new main node randomly selects the new child node from the slave nodes.

4. The method of claim 3, wherein,

the election triggering condition comprises that a main node in the cluster is down or that the main node and more than half of the slave nodes and the sub-nodes in the cluster lose heartbeat communication.

5. The method of claim 4, wherein,

when the main nodes and the child nodes are down at the same time or the number of the election messages does not exceed the preset threshold value, all the slave nodes in the cluster elect a new main node by using a Raft algorithm.

6. The method of claim 5, wherein,

after a new master node is elected by using the Raft algorithm, a new child node is randomly selected by the new master node.

7. The method of claim 6, wherein,

the predetermined threshold is half the number of slave nodes in the cluster.

8. A distributed election device, comprising:

9. The apparatus of claim 8, further comprising:

10. A distributed election system, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

11. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-7.