CN109617826B - Storm dynamic load balancing method based on cuckoo search - Google Patents

Storm dynamic load balancing method based on cuckoo search Download PDF

Info

Publication number
CN109617826B
CN109617826B CN201811636906.7A CN201811636906A CN109617826B CN 109617826 B CN109617826 B CN 109617826B CN 201811636906 A CN201811636906 A CN 201811636906A CN 109617826 B CN109617826 B CN 109617826B
Authority
CN
China
Prior art keywords
node
vector
weight
task
performance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811636906.7A
Other languages
Chinese (zh)
Other versions
CN109617826A (en
Inventor
龙笑
周良
郑洪源
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN201811636906.7A priority Critical patent/CN109617826B/en
Publication of CN109617826A publication Critical patent/CN109617826A/en
Application granted granted Critical
Publication of CN109617826B publication Critical patent/CN109617826B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/12—Avoiding congestion; Recovering from congestion
    • H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/29—Flow control; Congestion control using a combination of thresholds
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004—Server selection for load balancing
    • H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1044—Group management mechanisms 

Abstract

The invention relates to a storm cluster dynamic load balancing method based on a cuckoo search algorithm. The invention gives consideration to the real-time utilization conditions of resources such as CPU, memory, network and the like of the cluster, realizes the optimization process of the performance weight vector of the cluster node and completes the dynamic allocation of tasks; by the self-adaptive adjustment of the step size factor in the cuckoo search, the optimal weight vector can be found more quickly in the optimizing process, and the reasonable distribution of resources can be realized, so that the cluster response time is reduced, and the cluster optimization method has higher cluster throughput and smaller system delay.

Description

Storm dynamic load balancing method based on cuckoo search
Technical Field
The invention belongs to the field of big data real-time processing, and particularly relates to a storm dynamic load balancing method based on a cuckoo search algorithm.
Background
With the rapid development of the internet of things and the social network, the scale of flow data is continuously increased, and flow processing is widely applied to the fields of traffic monitoring, meteorological observation, bank transaction management and the like. By taking the real-time flow data of the freight of the unmanned aerial vehicle as an example, the generated flow data has the characteristics of high arrival speed, continuous arrival time, dynamic change and the like, and the traditional large data batch processing framework cannot meet the real-time requirement. The Storm distributed real-time computing system is widely applied to real-time processing of mass data by the characteristics of low delay, high performance, distribution, expandability, high fault tolerance and the like as a typical representative of streaming computing.
Under the condition that the requirements of data processing on real-time performance and high efficiency are higher and higher, management of nodes in a cluster and allocation of resources occupy more and more important positions in cluster management, and a load balancing technology is an effective means for ensuring high performance and high throughput of Storm real-time computing application. Storm real-time computing applications are typically compute intensive applications, and load balancing plays a crucial role in the performance of Storm real-time computing applications.
However, Storm default scheduling does not perform satisfactorily in terms of load balancing, and has more problems, so that some systems with high requirements on real-time performance and high efficiency cannot process data in time. Firstly, the storm platform adopts a polling scheduling (Round Robin scheduling) algorithm as a default task scheduling, that is, tasks included in a topology submitted by a user are uniformly distributed to each work process, and then each work process is uniformly distributed to each work node. Secondly, nodes in the storm cluster can be frequently and dynamically added or deleted, and worker processes can also be dynamically added or deleted, so that cluster computing resources are changed, and after the cluster nodes or the processes are dynamically added or deleted, the storm cannot make an effective adjustment strategy according to the changed available resources, so that load balance is influenced. Third, the default scheduling focuses more on CPU resources for node resources, and ignores other types of resources such as memory, disk, network, etc., which may cause problems such as insufficient memory of working nodes, network congestion, etc. In summary, in order to meet the requirements of a system with higher real-time performance and higher efficiency, a new method needs to be provided to ensure load balancing of storm scheduling, and the new method can make response time of a cluster faster, throughput higher, and system delay lower.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the technical problems and lay a foundation for the real-time storage of the next-step streaming data, the invention provides a storm dynamic load balancing method based on a cuckoo search algorithm.
The technical scheme is as follows:
the invention relates to a storm cluster dynamic load balancing method based on a cuckoo search algorithm, which comprises the following steps:
the method comprises the following steps: acquiring cluster node load information, including node performance vectors and completion rates of tasks on nodes, initializing relevant parameters, including the number of host bird nests, namely the number of tasks to be distributed, the probability of finding bird eggs, namely the elimination probability of solutions generated by iteration, the global maximum iteration times, step initial values and fitness threshold values;
step two: judging whether an iteration stopping condition is reached, namely the maximum iteration times Max is reached or the fitness threshold is larger than the threshold set in the step one, if the iteration stopping condition is reached, stopping the iteration, otherwise, continuing;
step three: calculating a load vector of the cluster node through the node performance vector and the performance weight vector alpha, and calculating the distribution weight and fitness function value of the task on the node;
step four: judging whether the iteration fitness function value is more optimal, namely the function value is smaller, if so, performing the next step, otherwise, keeping the node performance weight vector obtained last time;
step five: calculating a step factor of a cuckoo search algorithm according to the product of the current node performance weight vector and the average value and initial value of the initial distance, and calculating a new node performance weight vector according to a node performance weight vector updating formula;
step six: randomly generating a number K (0< K <1), judging whether the random number K is larger than the elimination probability set in the step one, if so, randomly updating the solution, otherwise, keeping the original solution unchanged;
step seven: calculating a node load vector according to the weight value of the performance vector obtained in the last iteration round, wherein the node load vector calculation method is the same as that in the third step;
step eight: and calculating the distribution weight of the tasks on the nodes according to the latest node load, and distributing the tasks according to the calculated distribution weight through a selection function.
Further, calculating the load vector L of the ith node of the cluster in step threeiThe method comprises the following steps:
Figure BDA0001930252550000021
wherein i is 1,2, …, m; j is 1,2, …, n; n is the number of tasks, and m is the number of nodes; k is 1,2,3,4, and denotes the number of the resource whose usage rate is to be monitored, and represents the CPU resourcesSource, memory, disk, network bandwidth, U is the performance vector of the cluster node, U ═1,u2,u3…um]Performance vector of a node in the cluster, i.e. component U of UiIs provided with
Figure BDA0001930252550000031
Figure BDA0001930252550000032
The vector represents the performance vector of the ith task at the jth node, having
Figure BDA0001930252550000033
Figure BDA0001930252550000034
The utilization rate of the kth resource of the ith task on the jth node is represented and comprises real-time monitored CPU resource occupancy rate, memory occupancy rate, disk I/O occupancy rate and network bandwidth occupancy rate, wherein alpha is the weight of a node performance vector, and alpha is [ alpha ]1,α2,α3,…αn]For the weight vector α, there is αj∈(0,1)∩∑aj1, i.e. each component vector a of the vector weights for the node performancejHas ajGreater than 0 and less than 1, and all alphajThe sum is equal to 1.
Further, in step three, the distribution weight W of the tasks on the nodes is calculatediComprises the following steps:
Figure BDA0001930252550000035
where Wi is the assigned weight W ═ W1，W2，W3…，Wm]The component vector of (a).
Further, the fitness function value calculating method in the third step comprises the following steps:
3a) according to the task distribution weight W, tasks are planned to be distributed to the nodes, and theoretical completion time is obtained
Figure BDA0001930252550000036
Reflecting the theoretical time required by the completion of the task i (i ═ 1,2 … n) on the jth node at the time t, the update formula of the theoretical completion time of the task is defined as:
Figure BDA0001930252550000037
wherein the content of the first and second substances,
Figure BDA0001930252550000038
the theoretical completion time of a new task T 'on a node j at the moment T', n 'represents the number of new tasks on the node j at the moment T', and theta is the monitored task completion rate;
3b) calculating a fitness function value:
Figure BDA0001930252550000039
wherein
Figure BDA00019302525500000310
For the solution generated by iteration, i.e. the current position of the bird nest, n is the number of tasks and m is the number of nodes.
Further, the step size factor a in step five is calculated by the following method:
Figure BDA00019302525500000311
wherein, a0Is the initial value of the step-size factor,
Figure BDA0001930252550000041
in order to be the current solution,
Figure BDA0001930252550000042
is the initial solution.
Further, in the step five, the node performance weight vector updates the formula:
Figure BDA0001930252550000043
wherein a is the changed step-size factor,
Figure BDA0001930252550000044
representing the solution resulting from the t-th search;
Figure BDA0001930252550000045
indicating a point product, Levy (β) is a search path, and indicating a random vector generated by a Levy distribution, wherein Levy (β) -u-t-βIt means that the random vector Levy (β) follows a binomial distribution.
Further, the step of allocating tasks in the step eight includes:
8a) calculating a judgment vector ThrVal to calculate a selection function value, wherein the judgment component calculation method of a single node su comprises the following steps:
ThrValsu＝Wsu×ResTol-∑W×Hissu
where ResTol represents the total load request, His, currently accumulatedsuRepresenting a component of the load distribution history vector, i.e. the load distribution history of a single node su, WsuRepresenting the task distribution weight of a single node su, wherein sigma W is the sum of the weight vectors distributed to all the node tasks;
8b) selecting a suitable node δ to process the current task according to a selection function:
{δ＝Sel(T,h,W)|ThrValδ＝max(ThrVal)}
wherein Sel (T, h, W) is a selection function, and T ═ T1,t2,…tnH is a load distribution history vector, and W is a node task distribution weight vector
Has the advantages that: the invention comprehensively considers the resource utilization conditions of CPU, memory, network bandwidth, disk IO and the like of the cluster nodes, and avoids the limitation that default scheduling focuses more on CPU resources to possibly cause the problems of insufficient memory, network blockage and the like. Meanwhile, the invention monitors the operation condition of the cluster in real time, calculates the weight of the performance of the cluster node according to the node load data monitored in real time, completes the dynamic allocation of resources, achieves the balance of the node load and enables the response time of the cluster to be faster, the throughput to be higher and the system delay to be lower.
Drawings
Fig. 1 is a diagram of a dynamic load balancing model.
Fig. 2 is a control flow chart of the storm dynamic load balancing method based on the cuckoo search algorithm.
Detailed Description
The invention is further explained below with reference to the drawings.
The environment of the invention is a Storm fully distributed cluster consisting of 5 physical machines, and the specific hardware configuration of each node is shown in table 1.
TABLE 1
Figure BDA0001930252550000051
Each node runs the Ubuntu 12.04 operating system. One of the nodes is used as a main node to run a Nimbus process and is responsible for resource allocation and task scheduling; and the slave node is responsible for receiving tasks distributed by Nimbus and starting and stopping Worker processes belonging to the slave node. And meanwhile, Zookeeper clusters are built and are always in a running state. Ganglia is used in the process to monitor the state of each node of the Storm cluster.
The model for the dynamic load balancing of strom is as follows:
the task assignment process in Strom can be described as a process performed to assign n tasks in one Topology to m Supervisors worker nodes,
T＝{t1,t2,…tndenotes n task sets, where ti(i ═ 1,2, …, n) denotes the ith task;
S＝{s1,s2,…smdenotes a set of m Supervisor nodes, where sj(j ═ 1,2, …, m) denotes the jth node;
L＝[L1,L2,…Lm]representing m nodesThe dynamic load vector of (2);
U＝[u1,u2,u3…um]and α ═ α1,α2,α3,…αn]A node performance vector and a weight vector of performance are represented, respectively.
The load value of the node can be calculated according to the performance vector and the performance weight vector, then the distribution weight of the tasks on the node can be calculated according to the load value of the node, and finally the tasks are effectively distributed to the node according to the calculated distribution weight vector through a selection function, so that the dynamic distribution of the tasks is realized.
The performance weight calculation is a dynamic process, and the optimal performance weight is found in step-by-step iteration through the optimization process of the cuckoo search algorithm, wherein the weight can enable the average response time of the cluster to be the lowest. The concrete model is represented as shown in fig. 1.
As shown in fig. 2, the present invention is a storm dynamic load balancing method based on cuckoo search algorithm, which specifically includes the following steps:
the first step is as follows: acquiring cluster node load information including node performance vectors and completion rates of tasks on nodes, initializing relevant parameters, determining the number of host bird nests as the number n of the tasks, finding the probability P of probability bird eggs as 0.75, setting the global maximum iteration number Max as 220, and setting the initial step length value a as an initial value00.1, the fitness threshold Val is 103。
The second step is that: judging whether an algorithm iteration stopping condition is reached, namely the maximum iteration frequency Max is reached or the fitness threshold is larger than the threshold Val set in the step one, stopping iteration if the algorithm iteration stopping condition is reached, and otherwise, continuing the iteration;
the third step: calculating a load vector of the cluster node through the node performance vector and the performance weight vector alpha, and calculating the distribution weight and fitness function value of the task on the node;
load L of ith nodeiThe calculation formula is as follows,
Figure BDA0001930252550000061
wherein i is 1,2, …, m; j is 1,2, …, n; n is the number of tasks, and m is the number of nodes; k is 1,2,3,4, which indicates the number of the resource whose usage rate is to be monitored, and represents the CPU resource, the memory, the disk, and the network bandwidth, respectively, U is the performance vector of the cluster node, and U is [ U ═ U [ [ U [ ] [, U [ ], and1,u2,u3…um]performance vector of a node in the cluster, i.e. component U of UiIs provided with
Figure BDA0001930252550000062
Figure BDA0001930252550000063
The vector represents the performance vector of the ith task at the jth node, having
Figure BDA0001930252550000064
Figure BDA0001930252550000065
The utilization rate of the kth resource of the ith task on the jth node is represented and comprises real-time monitored CPU resource occupancy rate, memory occupancy rate, disk I/O occupancy rate and network bandwidth occupancy rate, wherein alpha is the weight of a node performance vector, and alpha is [ a ═ a [1,α2,α3,…αn]For the weight vector α, there is αj∈(0,1)∩∑αj1, i.e. each component vector a of the node performance vector weightsjHaving a ofjGreater than 0 and less than 1, and all alphajThe sum being equal to 1, ajHas an initial value of
Figure BDA0001930252550000066
The calculation formula of the distribution weight W is as follows;
Figure BDA0001930252550000071
where Wi is the assigned weight W ═ W1，W2，W3…，Wm]The component vector of (a).
Fitness function value
Figure BDA0001930252550000072
The calculation steps are as follows:
a) according to the task distribution weight W, tasks are planned to be distributed to the nodes, and the theoretical completion time can be obtained
Figure BDA0001930252550000073
Reflecting the theoretical time required by the completion of a task i (i is 1,2 … n) at the jth node at the moment t, the initial theoretical time is obtained by averaging according to several test rounds, and the update formula of the theoretical completion time of the task is defined as:
Figure BDA0001930252550000074
wherein the content of the first and second substances,
Figure BDA0001930252550000075
the theoretical completion time of a new task T 'on a node j at the time T', n 'represents the number of new tasks on the node j at the time T', and theta is the monitored task completion rate.
b) Calculating a fitness function value:
Figure BDA0001930252550000076
wherein
Figure BDA0001930252550000077
For the solution generated by iteration, i.e. the current position of the bird nest, n is the number of tasks and m is the number of nodes.
The fourth step: and judging whether the fitness function value of the iteration is better or not, namely comparing the fitness function value of the iteration with the function value obtained in the previous iteration, if the fitness function value of the iteration is smaller, performing the next step, and if not, keeping the node performance weight vector obtained in the previous iteration.
The fifth step: calculating a step factor of the cuckoo search algorithm according to the product of the current node performance weight vector alpha and the average value of the initial distance and the initial value of the step length, calculating a new node performance weight vector according to a node performance weight vector updating formula,
the step-size factor is calculated as follows,
Figure BDA0001930252550000078
wherein, a0Is the initial value of the step-size factor,
Figure BDA0001930252550000079
in order to be the current solution,
Figure BDA00019302525500000710
is the initial solution.
New node performance weight vector
Figure BDA00019302525500000711
As calculated by the following formula,
Figure BDA00019302525500000712
wherein a is the changed step factor;
Figure BDA0001930252550000081
representing the solution resulting from the t-th search;
Figure BDA0001930252550000082
representing a point product, Levy (β) is a search path, and represents a random vector generated by Levy distribution, where Levy (β) -u-t-βIt means that the random vector Levy (β) follows a binomial distribution.
And a sixth step: randomly generating a number K (0< K <1), judging whether the random number K is greater than the elimination probability P, if so, indicating that the current bird egg is easy to be found in the cuckoo search algorithm, namely, the current solution is not good enough, so that the node performance weight vector needs to be randomly updated again, otherwise, keeping the original node performance weight vector unchanged;
the seventh step: after iteration is finished, the node load L is calculated according to the weight optimal solution, the node load calculation method is the same as that in the third step, the load vectors are planned to be distributed in the third step, and the current solution is to calculate the real load of the cluster node according to the performance weight optimal solution obtained by algorithm iteration
Figure BDA0001930252550000083
Eighth step: calculating the distribution weight W of the task on the node according to the latest node load L, wherein the method is the same as the third step, the third step is the planned distribution of the distribution weight, and the solution calculates the distribution weight of the task on the node according to the real load of the cluster node)
Figure BDA0001930252550000084
Distributing tasks according to the distribution weights obtained by calculation through a selection function, and the specific steps are as follows:
a) the judgment vector ThrVal is calculated to describe the selection function, and the judgment component calculation method of a single node comprises the following steps:
ThrValsu＝Wsu×ResTol-∑w×Hissu
where ResTol represents the total load request, His, currently accumulatedsuRepresenting a component of a load distribution history vector, i.e. the load distribution history of a single node, WsuAnd E, representing the task distribution weight of a single node su, and sigma W is the sum of weight vectors distributed to all node tasks.
b) Selecting a suitable node δ to process the current task according to a selection function:
{δ＝Sel(T,h,W)|ThrValδ＝max(ThrVal)}
wherein Sel (T, h, W) is a selection function, and T ═ T1,t2,…tnAnd h is a load distribution history vector, and W is a node task distribution weight vector.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (4)

1. A storm dynamic load balancing method based on cuckoo search is characterized by comprising the following steps:
the method comprises the following steps: acquiring cluster node load information, including node performance vectors and completion rates of tasks on nodes, initializing relevant parameters, including the number of host bird nests, namely the number of tasks to be distributed, the probability of finding bird eggs, namely the elimination probability of solutions generated by iteration, the global maximum iteration times, step initial values and fitness threshold values;
step two: judging whether an iteration stopping condition is reached, namely the maximum iteration times Max is reached or the fitness threshold is larger than the threshold set in the step one, if the iteration stopping condition is reached, stopping the iteration, otherwise, continuing;
step three: calculating a load vector of the cluster node through the node performance vector and the performance weight vector, and calculating the distribution weight and fitness function value of the task on the node;
load vector L of ith nodeiThe calculation formula is as follows,
Figure FDA0003514351930000011
wherein i is 1,2, …, m; j is 1,2, …, n; m is the number of nodes, and n is the number of tasks; k is 1,2,3,4, which indicates the number of the resource whose usage rate is to be monitored, and represents the CPU resource, the memory, the disk, and the network bandwidth, respectively, U is the performance vector of the cluster node, and U is [ U ═ U [ [ U [ ] [, U [ ], and1，u2，u3...um]performance vector of a node in the cluster, i.e. component U of UiIs provided with
Figure FDA0003514351930000012
Figure FDA0003514351930000013
The vector represents the performance vector of the jth task on the ith node, having
Figure FDA0003514351930000014
Figure FDA0003514351930000015
The utilization rate of the kth resource of the jth task on the ith node is represented and comprises real-time monitored CPU resource occupancy rate, memory occupancy rate, disk I/O occupancy rate and network bandwidth occupancy rate, wherein alpha is the weight of a node performance vector, and alpha is [ alpha ]1，α2，α3，...αn]For the weight vector α, there is αj∈(0，1)∩∑αj1, i.e. each component vector a of the node performance vector weightsjHaving a ofjGreater than 0 and less than 1, and all alphajThe sum being equal to 1, alphajHas an initial value of
Figure FDA0003514351930000016
The calculation formula of the distribution weight W is as follows;
Figure FDA0003514351930000017
wherein WiTo assign a weight W ═ W1，W2，W3…，Wm]Component of, LiIs the load vector of the ith node,
Figure FDA0003514351930000021
loading for all nodes;
the fitness function value calculating method in the third step comprises the following steps:
3a) according to the task distribution weight W, tasks are planned to be distributed to the nodes, and theoretical completion time is obtained
Figure FDA0003514351930000022
Reflecting the theoretical time required by the completion of the task j on the ith node at the moment t, and defining an updating formula of the theoretical completion time of the task as follows:
Figure FDA0003514351930000023
wherein the content of the first and second substances,
Figure FDA0003514351930000024
the theoretical completion time of a new task T 'on a node i at the moment T', n 'represents the number of new tasks on the node i at the moment T', and theta is the monitored task completion rate;
Figure FDA0003514351930000025
for the initial theoretical time of the task, TtIs the current task time;
3b) calculating a fitness function value:
Figure FDA0003514351930000026
wherein
Figure FDA0003514351930000027
For the solution generated by iteration, namely the current position of the bird nest, m is the number of nodes, and n is the number of tasks;
step four: judging whether the iteration fitness function value is more optimal, namely the function value is smaller, if so, performing the next step, otherwise, keeping the node performance weight vector obtained last time;
step five: calculating a step factor of a cuckoo search algorithm according to the product of the current node performance weight vector and the average value and initial value of the initial distance, and calculating a new node performance weight vector according to a node performance weight vector updating formula;
step six: randomly generating a number K, wherein K is more than 0 and less than 1, judging whether the random number K is more than the elimination probability set in the step one, if so, randomly updating the solution, otherwise, keeping the original solution unchanged;
step seven: calculating a node load vector according to the weight value of the performance vector obtained in the last iteration round;
step eight: and calculating the distribution weight of the tasks on the nodes according to the latest node load, and distributing the tasks according to the calculated distribution weight through a selection function.
2. The storm dynamic load balancing method based on cuckoo search according to claim 1, wherein the step-size factor a in step five is calculated by:
Figure FDA0003514351930000031
wherein, a0Is the initial value of the step-size factor,
Figure FDA0003514351930000032
in order to be the current solution,
Figure FDA0003514351930000033
is the initial solution.
3. The storm dynamic load balancing method based on cuckoo search as claimed in claim 1, wherein the node performance weight vector update formula in step five:
Figure FDA0003514351930000034
wherein a is the changed step-size factor,
Figure FDA0003514351930000035
representing the solution resulting from the t-th search;
Figure FDA0003514351930000036
indicating a point product, Levy (β) is a search path, and indicating a random vector generated by a Levy distribution, wherein Levy (β) -u-t-βIt means that the random vector Levy (β) follows a binomial distribution.
4. The storm dynamic load balancing method based on cuckoo search as claimed in claim 1, wherein the step of allocating tasks in step eight comprises:
8a) calculating a judgment vector ThrVal to calculate a selection function value, wherein the judgment component calculation method of a single node su comprises the following steps:
ThrValsu＝Wsu×ResTol-∑W×Hissu
where ResTol represents the total load request, His, currently accumulatedsuRepresenting a component of the load distribution history vector, i.e. the load distribution history of a single node su, WsuRepresenting the task distribution weight of a single node su, wherein sigma W is the sum of the weight vectors distributed to all the node tasks;
8b) selecting a suitable node δ to process the current task according to a selection function:
{δ＝Sel(T，h，W)|ThrValδ＝max(ThrVal)}
wherein Sel (T, h, W) is a selection function, and T ═ T1，t2，...tnAnd h is a load distribution history vector, and W is a node task distribution weight vector.
CN201811636906.7A 2018-12-29 2018-12-29 Storm dynamic load balancing method based on cuckoo search Active CN109617826B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811636906.7A CN109617826B (en) 2018-12-29 2018-12-29 Storm dynamic load balancing method based on cuckoo search

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811636906.7A CN109617826B (en) 2018-12-29 2018-12-29 Storm dynamic load balancing method based on cuckoo search

Publications (2)

Publication Number Publication Date
CN109617826A CN109617826A (en) 2019-04-12
CN109617826B true CN109617826B (en) 2022-05-03

Family

ID=66015405

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811636906.7A Active CN109617826B (en) 2018-12-29 2018-12-29 Storm dynamic load balancing method based on cuckoo search

Country Status (1)

Country Link
CN (1) CN109617826B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110688219B (en) * 2019-09-05 2022-03-18 浙江理工大学 Adaptive weight load balancing algorithm based on reverse chaotic cuckoo search
CN110429667B (en) * 2019-09-06 2022-04-22 国网甘肃省电力公司经济技术研究院 Photovoltaic-thermal bundling power generation system capacity configuration method based on cuckoo search
CN110855762B (en) * 2019-10-31 2022-07-22 云南电网有限责任公司信息中心 Data block distribution method for heterogeneous cluster nodes in power grid system
CN111858029B (en) * 2020-06-16 2023-06-27 国网福建省电力有限公司信息通信分公司 Storm cluster load balancing method and system based on discrete particle swarm
CN111782626A (en) * 2020-08-14 2020-10-16 工银科技有限公司 Task allocation method and device, distributed system, electronic device and medium
CN112333255A (en) * 2020-10-27 2021-02-05 重庆紫光华山智安科技有限公司 Load balancing scheduling method and device, scheduling server and storage medium
CN112787948B (en) * 2020-12-30 2023-07-18 上海微盟企业发展有限公司 Traffic load balancing method and related device
CN113141317B (en) * 2021-03-05 2022-09-30 西安电子科技大学 Streaming media server load balancing method, system, computer equipment and terminal
CN113282405B (en) * 2021-04-13 2023-09-15 福建天泉教育科技有限公司 Load adjustment optimization method and terminal
CN115633041A (en) * 2022-10-31 2023-01-20 中科星图数字地球合肥有限公司 Multi-cluster management method and device, electronic equipment and readable storage medium
CN117697265B (en) * 2024-02-06 2024-05-03 厦门锋元机器人有限公司 Automatic control management system for aluminum welding production line

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102387071A (en) * 2011-10-12 2012-03-21 苏州阔地网络科技有限公司 Network load balancing method, processor and system
US9608913B1 (en) * 2014-02-24 2017-03-28 Google Inc. Weighted load balancing in a multistage network
CN108182115A (en) * 2017-12-28 2018-06-19 福州大学 A kind of virtual machine load-balancing method under cloud environment
CN108965014A (en) * 2018-07-25 2018-12-07 北京智芯微电子科技有限公司 The service chaining backup method and system of QoS perception

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102387071A (en) * 2011-10-12 2012-03-21 苏州阔地网络科技有限公司 Network load balancing method, processor and system
US9608913B1 (en) * 2014-02-24 2017-03-28 Google Inc. Weighted load balancing in a multistage network
CN108182115A (en) * 2017-12-28 2018-06-19 福州大学 A kind of virtual machine load-balancing method under cloud environment
CN108965014A (en) * 2018-07-25 2018-12-07 北京智芯微电子科技有限公司 The service chaining backup method and system of QoS perception

Also Published As

Publication number Publication date
CN109617826A (en) 2019-04-12

Similar Documents

Publication Publication Date Title
CN109617826B (en) Storm dynamic load balancing method based on cuckoo search
CN108829494B (en) Container cloud platform intelligent resource optimization method based on load prediction
WO2021104096A1 (en) Method and apparatus for task scheduling in container cloud environment, and server and storage apparatus
CN108566659B (en) 5G network slice online mapping method based on reliability
CN110297699B (en) Scheduling method, scheduler, storage medium and system
CN105491138B (en) Distributed load scheduling method based on load rate graded triggering
CN110688219B (en) Adaptive weight load balancing algorithm based on reverse chaotic cuckoo search
US8843929B1 (en) Scheduling in computer clusters
CN109947574B (en) Fog network-based vehicle big data calculation unloading method
Al-maamari et al. Task scheduling using hybrid algorithm in cloud computing environments
CN113141317B (en) Streaming media server load balancing method, system, computer equipment and terminal
CN107832129B (en) Dynamic task scheduling optimization method for distributed stream computing system
Ashouraei et al. A new SLA-aware load balancing method in the cloud using an improved parallel task scheduling algorithm
CN109656713B (en) Container scheduling method based on edge computing framework
CN115629865B (en) Deep learning inference task scheduling method based on edge calculation
CN107566535B (en) Self-adaptive load balancing method based on concurrent access timing sequence rule of Web map service
CN116467082A (en) Big data-based resource allocation method and system
CN111131447A (en) Load balancing method based on intermediate node task allocation
CN109460301B (en) Method and system for configuring elastic resources of streaming data load
CN116302404B (en) Resource decoupling data center-oriented server non-perception calculation scheduling method
Jain et al. An algorithm for dynamic load balancing in distributed systems with multiple supporting nodes by exploiting the interrupt service
CN107948330A (en) Load balancing based on dynamic priority under a kind of cloud environment
CN116302578B (en) QoS (quality of service) constraint stream application delay ensuring method and system
Datta A new task scheduling method for 2 level load balancing in homogeneous distributed system
CN111488208A (en) Edge cloud cooperative computing node scheduling optimization method based on variable step length bat algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant