CN111245924A - Load balancing method and device and computer storage medium - Google Patents
Load balancing method and device and computer storage medium Download PDFInfo
- Publication number
- CN111245924A CN111245924A CN202010019102.3A CN202010019102A CN111245924A CN 111245924 A CN111245924 A CN 111245924A CN 202010019102 A CN202010019102 A CN 202010019102A CN 111245924 A CN111245924 A CN 111245924A
- Authority
- CN
- China
- Prior art keywords
- server node
- virtual nodes
- load balancing
- node
- load
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/101—Server selection for load balancing based on network conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Computer And Data Communications (AREA)
Abstract
The disclosure relates to a load balancing method and device and a computer storage medium, belonging to the field of distributed technology; the load balancing method comprises the following steps: acquiring load data of each server node; determining the number of virtual nodes contained in each server node in the distributed system according to the load data; sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information, wherein the number information indicates the number of the virtual nodes; thus, load balancing is achieved without changing the architecture of the existing distributed system based on consistent hashing.
Description
Technical Field
The present disclosure relates to the field of distributed technologies, and in particular, to a load balancing method and apparatus, and a computer storage medium.
Background
The DHT provides a mapping rule between different modules and nodes, a common coordination scheme for decentralized DHT, and provides a hash table that, when resized, only part of the key values (keys) on average need to be remapped for the hash slot, unlike the traditional hash table, which requires nearly all of the keys to be remapped for the hash slot. The consistent hashing algorithm can ensure that the mapping relation between the key and the node is not greatly changed due to addition and deletion of the node in a distributed environment. However, for a distributed system that already uses a consistent hash algorithm, in order to solve the problem of unbalanced load of the distributed system, the original scheduling architecture needs to be completely changed, which is costly.
Disclosure of Invention
To overcome the problems in the related art, the present disclosure provides a load balancing method and apparatus, and a computer storage medium.
According to a first aspect of the embodiments of the present disclosure, there is provided a load balancing method, including:
acquiring load data of each server node;
determining the number of virtual nodes contained in each server node in the distributed system according to the load data;
and sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information, wherein the number information indicates the number of the virtual nodes.
In the above scheme, the method further comprises:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
In the above scheme, the method further comprises:
determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
the sending of the number information of the allocated virtual nodes to each server node includes:
and sending the number information to the first server node.
In the foregoing solution, the sending the number information of the allocated virtual nodes to each server node includes:
sending change information to a second server node, the number of which does not change, in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
In the above scheme, the load data includes one or more of the following data:
network card flow;
a Central Processing Unit (CPU) memory;
number of requests per second (QPS).
According to a second aspect of the embodiments of the present disclosure, there is provided a load balancing apparatus, the apparatus including:
the acquisition module is configured to acquire load data of each server node;
the determining module is configured to determine the number of virtual nodes contained in each server node in the distributed system according to the load data;
a notification module configured to send the number information of the allocated virtual nodes to each server node, so that each server node adjusts the current virtual node according to the number information, where the number information indicates the number of the virtual nodes.
In the foregoing solution, the determining module is configured to:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
In the foregoing solution, the determining module is further configured to:
determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
the notification module configured to:
and sending the number information to the first server node.
In the foregoing solution, the notification module is configured to:
sending change information to a second server node in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
In the above scheme, the load data includes one or more of the following data:
network card flow;
a CPU memory;
QPS。
according to a third aspect of the embodiments of the present disclosure, there is provided a load balancing apparatus, including:
a processor;
a memory for storing executable instructions;
wherein the processor is configured to: the executable instructions are executed to implement the load balancing method of any one of the preceding aspects.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer storage medium having stored therein executable instructions, which when executed by a processor, cause the processor to execute the load balancing method according to any one of the foregoing aspects.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
acquiring load data; determining the number of virtual nodes contained in each server node in the distributed system according to the load data; sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information, wherein the number information indicates the number of the virtual nodes; thus, load balancing is achieved without changing the architecture of the existing distributed system based on consistent hashing.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. In the drawings:
FIG. 1 is a flow diagram illustrating a method of load balancing in accordance with an exemplary embodiment;
FIG. 2 is a schematic diagram illustrating a virtual node of server nodes employing consistent hashing in the prior art in accordance with an illustrative embodiment;
FIG. 3 is a schematic diagram illustrating virtual nodes of server nodes employing consistent hashing according to an example embodiment;
FIG. 4 is a flow diagram illustrating another method of load balancing according to an example embodiment;
FIG. 5 is a diagram illustrating the effects of traffic after a servlet cluster comes online in accordance with an illustrative embodiment;
FIG. 6(a) is a graph showing the daily average traffic profile before a large cluster comes online in accordance with an exemplary embodiment;
FIG. 6(b) is a graph showing the average daily traffic profile after the line is completed for a large cluster in accordance with an exemplary embodiment;
FIG. 7 is a block diagram illustrating a load balancing apparatus in accordance with an exemplary embodiment;
fig. 8 is a block diagram illustrating an apparatus 800 to implement load balancing processing in accordance with an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the examples of the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the embodiments of the application, as detailed in the appended claims.
The terminology used in the embodiments of the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the embodiments of the present disclosure. As used in the disclosed embodiments and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information in the embodiments of the present disclosure, such information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of embodiments of the present disclosure. The words "if" and "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination", depending on the context.
For better understanding of the technical solution of the present application, the following technical background is described in detail.
As traffic grows, the bandwidth tends to become a bottleneck as Talos serves as a high-throughput message queue. If the network card flow is not balanced, a barrel effect of cluster network card resources is generated, namely when the machine using the most network cards in the cluster reaches the resource bottleneck, expansion is triggered, even if other network cards of other nodes of the cluster have a large margin at the moment, the machine cost is wasted. And as the traffic volume is more and more, the cluster scale is continuously increased, and the problem becomes more obvious. Taking an on-line cluster of millet as an example, 150 machines of the cluster draw the daily flow average value of each node into a scatter diagram so as to conveniently observe the flow balance degree of the nodes. The highest flow rate 324M and the lowest flow rate 134M differ by more than two times. Analyzing the problem, the size of the network card flow of the machine node is positively correlated with the sum of all Partition flows served by the node (service). To solve the problem of unbalanced network card load between nodes, the distribution of Partition in each node needs to be adjusted (i.e. the scheduling relationship of Partition is changed).
The scheduling mechanism used by Talos before optimization will be briefly described below.
Talos uses consistent hashing to agree on a Partition scheduling relationship. In brief, a consistent hash may be understood as a function, Y ═ consistenhash (x). Any value of X can be mapped to a unique value Y within a fixed interval. For example, when each node in a 10-node cluster is mapped, the interval is divided into 10 parts. The Partition map is also mapped to the ring, and the node to which the Partition map serves is determined according to the position interval. All nodes of the cluster follow this convention so that the scheduling information is consistent. In fact, in order to make the ring division more uniform and the node up-down line more stable, each machine node will correspond to multiple virtual nodes to strive for averaging. In the conventional consistent hash, the number of virtual nodes mapped by each node is consistent. For example, the number of virtual nodes corresponding to each machine node of Talos is 2000. It can be understood that the scheduling mode of consistent hashing is used, the distribution of Partition numbers is balanced as much as possible, the weights of the partitions in scheduling are the same, the difference of other indexes (such as QPS and flow) of the partitions is not concerned, and the mapping relation of the consistent hashing algorithm cannot be interfered and adjusted.
Based on this, the present embodiment provides a load balancing method, which is applied to servers of a distributed system, for example, the servers include but are not limited to a common server or a cloud server. Referring to fig. 1, fig. 1 is a flowchart illustrating a load balancing method according to an exemplary embodiment, as shown in fig. 1, the load balancing method includes the following steps:
in step S11, load data of each server node is acquired.
In this embodiment, the load data includes one or more of the following data:
network card flow;
a CPU memory;
number of requests per second (QPS).
In some embodiments, the load data may be acquired by a Druid. Among them, the Druid is an efficient data query system.
The present application does not limit the manner in which load data is acquired.
In step S12, the number of virtual nodes is determined for each server node in the distributed system according to the load data.
In some embodiments, determining the number of virtual nodes for each server node in the distributed system according to the load data includes:
constructing a consistent hash function according to the load data of each server node;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
In this embodiment, the preset load balancing standard includes a target value of load balancing. For example, if the target value is set to 10%, the target is to adjust the cluster maximum daily average traffic and the cluster minimum daily average traffic to be within 10% of the cluster daily average traffic.
It should be noted that the preset load balancing standard may be set or adjusted according to requirements.
Therefore, a consistent hash function is constructed according to the load data, the number of the virtual nodes of the consistent hash function is adjusted in a simulation mode to calculate the load until the load is in line with expectation, namely the number distribution of the needed virtual nodes is achieved, and then each server node is triggered to adjust the current virtual node according to the number of the corresponding virtual nodes in the distribution.
In practical application, load data is introduced to participate in scheduling decision of creating partitions (partitions) in a consistent hash function. By introducing a load data feedback mechanism into the scheduling strategy, the number of the consistent hash virtual nodes is adjusted, and load balance is achieved. Specifically, monitored load data is obtained, and the load value of a server node is compared with the average load value of each server node of a cluster; if the node load value is larger, reducing the provided Partition number by reducing the virtual node number of the node; if the load value of the node is smaller, the number of partitions provided by increasing the number of virtual nodes of the node is increased, and the feedback is continuously carried out. Through monitoring feedback and continuous adjustment of the virtual nodes, consistent Hash virtual node distribution which accords with load balanced distribution of each node can be found finally. It should be noted that the Partition number of each migration is controlled, so as to ensure the overall stability of the cluster; the variance is calculated to determine whether to adjust to prevent unnecessary oscillation adjustment or adjustment in worse directions. The variance can reflect the discrete degree, and the adjustment is triggered only when the variance after the analog adjustment becomes smaller; the maximum number of adjustments is set to set infinite simulation calculations when the adjustment target cannot be reached.
Referring to fig. 2, fig. 2 shows a schematic diagram of virtual nodes of server nodes using consistent hashing in the prior art. As shown in fig. 2, the number of virtual nodes included in each server node using the conventional consistent hash is the same, and each of the server nodes RS0, RS1, RS2, RS3, and RS4 includes two virtual nodes. In other words, the number of virtual nodes per server node is the same.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating a virtual node of each server node using consistent hashing in the present embodiment. As shown in fig. 3, the number of virtual nodes included in each server node is different. For example, the server node RS0 contains two virtual nodes; RS1 contains a virtual node; RS2 contains two virtual nodes; RS3 contains two virtual nodes; RS4 contains three virtual nodes.
It should be noted that, the present application does not limit the number of server nodes in the distributed system.
In practical applications, the initial values of the number of virtual nodes configured for each server node in the distributed system may be the same. Of course, different numbers of virtual nodes can be configured for each server node in the distributed system according to different priorities of each server.
In step S13, the number of virtual nodes allocated to each server node is notified so that each server node adjusts the current virtual node according to the number of virtual nodes allocated.
The method described in this embodiment can be applied to any one server in a distributed system, and the server running the method is responsible for completing logic such as analog calculation according to a threshold value and a monitored value, controlling migration volume, triggering according to a specified time interval, and the like. The availability of the cluster does not depend on the server, the hanging of the server only stops the load balancing adjustment of the cluster, and the cluster can continue to perform scheduling operation according to the current virtual node distribution without influencing the normal operation of the cluster.
According to the technical scheme provided by the embodiment, load data of each server node is obtained; determining the number of virtual nodes contained in each server node in the distributed system according to the load data; sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information; thus, load balancing is achieved without changing the architecture of the existing distributed system based on consistent hashing.
The present embodiment also provides another load balancing method. The load balancing method is applied to servers of a distributed system, for example, the servers include but are not limited to ordinary servers or cloud servers. Referring to fig. 4, fig. 4 shows a flow chart of another load balancing method, and as shown in fig. 4, the method shown in fig. 1 further includes the following steps. In step S41, the first server node whose number of virtual nodes has changed is determined based on the number of virtual nodes included in each server node. Here, the first server node is a server node in which the number of virtual nodes in the distributed system is to be changed.
In some embodiments, the determining the first server node with the changed number of virtual nodes includes:
acquiring the initial number of the virtual nodes of each server node before determining the number of the virtual nodes according to the load data;
and determining the first server node with the changed number of the virtual nodes according to the initial number and the number.
Referring to fig. 4, in the present embodiment, step S13 shown in fig. 1 includes the following steps:
in step S42, the number information is sent to the first server node.
The number information is used for representing the number of virtual nodes to which the first server node should be adjusted.
Therefore, the first server node is informed of the virtual node change notification, so that the first service node can adjust the number of the virtual nodes according to the node change notification.
Referring to fig. 4, in the present embodiment, step S13 shown in fig. 1 includes the following steps:
in step S43, change information is sent by the first server node to a second server node in the distributed system, where the number of virtual nodes has not changed, and the change information indicates that the number of virtual nodes of the first server node has changed.
Here, the second server node is a server node in which the number of virtual nodes in the distributed system has not changed.
Fig. 5 is a flow effect diagram after the servlet cluster is online, where a curve represents a flow value of each server, and as can be seen from fig. 5, the flow value of each server tends to be in a balanced state after the servlet cluster is online.
Taking a large cluster such as a Talos cluster as an example, the daily traffic average of each server node is made into a scatter diagram. FIG. 6(a) is a graph showing the daily average flow before the line in a large cluster, according to an exemplary embodiment, where the flow distribution before the line is less concentrated as can be seen in FIG. 6 (a); still taking the above-mentioned large cluster such as Talos cluster as an example, the average daily traffic of each optimized server node is made into a scatter diagram, and fig. 6(b) is a daily average traffic distribution diagram after the large cluster is online according to an exemplary embodiment, and as can be seen from fig. 6(b), the online traffic distribution is more concentrated, that is, the traffic values of each optimized node are obviously aggregated. Through calculation, 35 machines needing capacity expansion due to the fact that the network card reaches the bottleneck can be saved by the 150 machines in the cluster.
Fig. 7 is a block diagram illustrating a load balancing apparatus in accordance with an example embodiment. The load balancing device is applied to a distributed system, and referring to fig. 7, the device comprises an acquisition module 10, a determination module 20 and a notification module 30.
An obtaining module 10 configured to obtain load data of each server node;
a determining module 20, configured to determine, according to the load data, the number of virtual nodes included in each server node in the distributed system;
a notification module 30 configured to send the number information of the allocated virtual nodes to each server node, so that each server node adjusts the current virtual node according to the number information, where the number information indicates the number of the virtual nodes.
In the foregoing solution, the determining module 20 is further configured to:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
In some embodiments, the determining module 20 is further configured to: determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
the notification module 30 configured to:
and sending the number information to the first server node.
In some embodiments, the notification module 30 is configured to:
sending change information to a second server node in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
In some embodiments, the load data comprises one or more of the following:
network card flow;
a CPU memory;
QPS。
with regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In practical applications, the specific structures of the obtaining module 10, the determining module 20 and the notifying module 30 can be implemented by a Central Processing Unit (CPU), a Micro Controller Unit (MCU), a Digital Signal Processor (DSP), a Programmable Logic Controller (PLC), and the like in the load balancing apparatus or the first device to which the load balancing apparatus belongs.
The load balancing apparatus of this embodiment may be disposed in any server in the distributed system, such as the main server.
Of course, the load balancing means may also exist as a stand-alone device.
It should be understood by those skilled in the art that the functions of the processing modules in the load balancing apparatus according to the embodiment of the present disclosure may be understood by referring to the foregoing description of the load balancing method applied to the smart home system side, and the processing modules in the load balancing apparatus according to the embodiment of the present disclosure may be implemented by an analog circuit that implements the functions described in the embodiment of the present disclosure, or may be implemented by running software that executes the functions described in the embodiment of the present disclosure on an electronic device.
The load balancing device disclosed by the embodiment of the disclosure realizes load balancing without changing the architecture of the existing distributed system based on consistent hash.
The embodiment of the present disclosure also describes a load balancing apparatus, which includes: the load balancing method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein when the processor executes the program, the load balancing method provided by any one of the above technical schemes applied to the first device is realized.
As an embodiment, the processor, when executing the program, implements:
acquiring load data of each server node;
determining the number of virtual nodes contained in each server node in the distributed system according to the load data;
and sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information, wherein the number information indicates the number of the virtual nodes.
As an embodiment, the processor, when executing the program, implements:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
As an embodiment, the processor, when executing the program, implements:
determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
and sending the number information to the first server node.
As an embodiment, the processor, when executing the program, implements:
sending change information to a second server node, the number of which does not change, in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
The load balancing device provided by the embodiment of the application realizes load balancing without changing the existing structure of the distributed system based on the consistent hash.
The embodiment of the present application further describes a computer storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used for executing the load balancing method described in the foregoing embodiments. That is, after the computer executable instructions are executed by the processor, the load balancing method provided by any one of the foregoing technical solutions can be implemented.
Those skilled in the art should understand that the functions of the programs in the computer storage medium of the present embodiment can be understood by referring to the related description of the load balancing method described in the foregoing embodiments.
Fig. 8 is a block diagram illustrating an apparatus 900 for implementing load balancing processing in accordance with an example embodiment. For example, the apparatus 900 may be provided as a server. Referring to fig. 8, the apparatus 900 includes a processing component 922, which further includes one or more processors, and memory resources, represented by memory 932, for storing instructions, such as applications, that are executable by the processing component 922. The application programs stored in memory 932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 922 is configured to execute instructions to perform the data processing method applied to the server side described above.
The device 900 may also include a power component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input output (I/O) interface 958. The apparatus 900 may operate based on an operating system stored in the memory 932, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
The technical solutions described in the embodiments of the present disclosure can be arbitrarily combined without conflict.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the invention is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (12)
1. A method of load balancing, the method comprising:
acquiring load data of each server node;
determining the number of virtual nodes contained in each server node in the distributed system according to the load data;
and sending the number information of the distributed virtual nodes to each server node so that each server node adjusts the current virtual node according to the number information, wherein the number information indicates the number of the virtual nodes.
2. The method according to claim 1, wherein the determining the number of virtual nodes included in each server node in the distributed system according to the load data comprises:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
3. The method of load balancing according to claim 1, further comprising:
determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
the sending of the number information of the allocated virtual nodes to each server node includes:
and sending the number information to the first server node.
4. The load balancing method according to claim 3, wherein the sending information on the number of the allocated virtual nodes to each server node includes:
sending change information to a second server node, the number of which does not change, in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
5. The load balancing method according to any one of claims 1 to 4, wherein the load data comprises one or more of the following data:
network card flow;
a Central Processing Unit (CPU) memory;
number of requests per second QPS.
6. A load balancing apparatus, the apparatus comprising:
the acquisition module is configured to acquire load data of each server node;
the determining module is configured to determine the number of virtual nodes contained in each server node in the distributed system according to the load data;
a notification module configured to send the number information of the allocated virtual nodes to each server node, so that each server node adjusts the current virtual node according to the number information, where the number information indicates the number of the virtual nodes.
7. The load balancing apparatus of claim 6, wherein the determining module is further configured to:
constructing a consistent hash function according to the load data;
adjusting the number of virtual nodes contained in each server node in the consistent hash function, and simulating and calculating the load data of each server node by using the consistent hash function after the number of the virtual nodes is adjusted until the load data obtained by the simulated calculation meets a preset load balancing standard;
and determining the number of the virtual nodes contained in each server node in the distributed system according to the number of the virtual nodes contained in each server node in the consistent hash function meeting the preset load balancing standard.
8. The load balancing apparatus of claim 6, wherein the determining module is further configured to: determining a first server node with the changed number of virtual nodes based on the number of the virtual nodes contained in each server node;
the notification module configured to:
and sending the number information to the first server node.
9. The load balancing apparatus of claim 8, wherein the notification module is configured to:
sending change information to a second server node in the distributed system through the first server node, wherein the change information indicates that the number of the virtual nodes of the first server node changes.
10. The load balancing device according to any one of claims 7 to 9, wherein the load data comprises one or more of the following data:
network card flow;
a Central Processing Unit (CPU) memory;
number of requests per second QPS.
11. A load balancing apparatus comprising:
a processor;
a memory for storing executable instructions;
wherein the processor is configured to: executing the executable instructions to implement the load balancing method of any one of claims 1 to 5.
12. A computer storage medium having stored therein executable instructions that, when executed by a processor, cause the processor to perform the load balancing method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010019102.3A CN111245924A (en) | 2020-01-08 | 2020-01-08 | Load balancing method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010019102.3A CN111245924A (en) | 2020-01-08 | 2020-01-08 | Load balancing method and device and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111245924A true CN111245924A (en) | 2020-06-05 |
Family
ID=70865060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010019102.3A Pending CN111245924A (en) | 2020-01-08 | 2020-01-08 | Load balancing method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111245924A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112306688A (en) * | 2020-10-30 | 2021-02-02 | 天地伟业技术有限公司 | Innovative hash consistency algorithm suitable for cloud storage |
| CN113778645A (en) * | 2021-08-18 | 2021-12-10 | 煤炭科学研究总院 | Task scheduling method, device and equipment based on edge calculation and storage medium |
| CN114428585A (en) * | 2020-10-29 | 2022-05-03 | 北京奇艺世纪科技有限公司 | Data storage method and device and electronic equipment |
| CN114466019A (en) * | 2022-04-11 | 2022-05-10 | 阿里巴巴（中国）有限公司 | Distributed computing system, load balancing method, device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103124299A (en) * | 2013-03-21 | 2013-05-29 | 杭州电子科技大学 | Distributed block-level storage system in heterogeneous environment |
| CN106559448A (en) * | 2015-09-28 | 2017-04-05 | 北京国双科技有限公司 | Server load balancing method and apparatus |
| WO2018059032A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Data migration method for virtual node, and virtual node |
| CN107920129A (en) * | 2017-12-07 | 2018-04-17 | 郑州云海信息技术有限公司 | A kind of method, apparatus, equipment and the cloud storage system of data storage |
| CN110336891A (en) * | 2019-07-24 | 2019-10-15 | 中南民族大学 | Data cached location mode, equipment, storage medium and device |
-
2020
- 2020-01-08 CN CN202010019102.3A patent/CN111245924A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103124299A (en) * | 2013-03-21 | 2013-05-29 | 杭州电子科技大学 | Distributed block-level storage system in heterogeneous environment |
| CN106559448A (en) * | 2015-09-28 | 2017-04-05 | 北京国双科技有限公司 | Server load balancing method and apparatus |
| WO2018059032A1 (en) * | 2016-09-30 | 2018-04-05 | 华为技术有限公司 | Data migration method for virtual node, and virtual node |
| CN107920129A (en) * | 2017-12-07 | 2018-04-17 | 郑州云海信息技术有限公司 | A kind of method, apparatus, equipment and the cloud storage system of data storage |
| CN110336891A (en) * | 2019-07-24 | 2019-10-15 | 中南民族大学 | Data cached location mode, equipment, storage medium and device |
Non-Patent Citations (2)
| Title |
|---|
| HIKOTOSHI NAKAZATO: "《Data Allocation Method considering Server Performance and Data Access Frequency with Consistent Hashing》", 《2012 14TH ASIA-PACIFIC NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM (APNOMS)》 * |
| 曾发: "《飞行器自主保障***海量数据存储设计与研究》", 《宇航计测技术》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114428585A (en) * | 2020-10-29 | 2022-05-03 | 北京奇艺世纪科技有限公司 | Data storage method and device and electronic equipment |
| CN112306688A (en) * | 2020-10-30 | 2021-02-02 | 天地伟业技术有限公司 | Innovative hash consistency algorithm suitable for cloud storage |
| CN113778645A (en) * | 2021-08-18 | 2021-12-10 | 煤炭科学研究总院 | Task scheduling method, device and equipment based on edge calculation and storage medium |
| CN114466019A (en) * | 2022-04-11 | 2022-05-10 | 阿里巴巴（中国）有限公司 | Distributed computing system, load balancing method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111245924A (en) | Load balancing method and device and computer storage medium | |
| CN110858161B (en) | Resource allocation method, device, system, equipment and medium | |
| CN109218355B (en) | Load balancing engine, client, distributed computing system and load balancing method | |
| WO2017167025A1 (en) | Method and device for realizing task scheduling, and computer storage medium | |
| EP3281359B1 (en) | Application driven and adaptive unified resource management for data centers with multi-resource schedulable unit (mrsu) | |
| EP2923479B1 (en) | Method and apparatus for controlling utilization in a horizontally scaled software application | |
| US8713125B2 (en) | Method and system for scaling usage of a social based application on an online social network | |
| CN103516807A (en) | Cloud computing platform server load balancing system and method | |
| CN111131486B (en) | Load adjustment method and device of execution node, server and storage medium | |
| JP2012099062A (en) | Service cooperation system and information processing system | |
| CN105703927A (en) | Resource allocation method, network device and network system | |
| CN112714029A (en) | Method, device and equipment for scheduling content distribution network bandwidth | |
| CN112711479A (en) | Load balancing system, method and device of server cluster and storage medium | |
| CN115396377B (en) | Method, device, equipment and storage medium for optimizing service quality of object storage | |
| CN116909735A (en) | Calculation power scheduling method and device, server and storage medium | |
| Moens et al. | Design and evaluation of a hierarchical application placement algorithm in large scale clouds | |
| CN111338750A (en) | Pressure adjusting method and device for execution node, server and storage medium | |
| CN109815204A (en) | A kind of metadata request distribution method and equipment based on congestion aware | |
| CN109413117B (en) | Distributed data calculation method, device, server and computer storage medium | |
| CN115168017B (en) | Task scheduling cloud platform and task scheduling method thereof | |
| Kumar et al. | BMAQR: balanced multi attribute QoS aware replication in HDFS | |
| Zhu et al. | Load balancing algorithm for web server based on weighted minimal connections | |
| CN113377866A (en) | Load balancing method and device for virtualized database proxy service | |
| CN108234354B (en) | SDN controller and SDN switch connection control method and SDN controller system | |
| Li et al. | A dynamic load balancing algorithm based on consistent hash |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100085 unit C, building C, lin66, Zhufang Road, Qinghe, Haidian District, Beijing Applicant after: Beijing Xiaomi pinecone Electronic Co.,Ltd. Address before: 100085 unit C, building C, lin66, Zhufang Road, Qinghe, Haidian District, Beijing Applicant before: BEIJING PINECONE ELECTRONICS Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200605 |
|
| RJ01 | Rejection of invention patent application after publication |