CN104935523A - Load balancing processing method and equipment - Google Patents
Load balancing processing method and equipment Download PDFInfo
- Publication number
- CN104935523A CN104935523A CN201410108066.2A CN201410108066A CN104935523A CN 104935523 A CN104935523 A CN 104935523A CN 201410108066 A CN201410108066 A CN 201410108066A CN 104935523 A CN104935523 A CN 104935523A
- Authority
- CN
- China
- Prior art keywords
- task
- information
- migrated
- migration
- working node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a load balancing processing method and equipment. The method comprises the following steps of acquiring operation time information of each task in one iteration period of one working node; according to the obtained operation time information of each task and an analysis strategy of a task to be migrated, determining a task to be migrated of the working node in the iteration period; when times of the task to be migrated exceed a setting value, migrating the task to be migrated to a working node whose task processing quantity is less than a setting threshold except for the working node. By using a task mode, a task migration in multiple iterative operations of working points is executed so that task load balancing between the working points is effectively realized; a problem that time consumption exists in a load balancing strategy in Giraph is avoided and simultaneously a problem that message communication network cost is increased in a speculative execution strategy in Hadoop is avoided; task migration efficiency in a BSP model is increased and a load balance property of the system is improved.
Description
Technical field
The present invention relates to wireless communication technology field, particularly relate to a kind of processing method and equipment of load balancing.
Background technology
BSP(Bulk-Synchronous Parallel, Large Copacity is run simultaneously) model is model based on large data iterative processing, is different from Map Reduce model.In BSP model, data processing is divided into some tasks, and each task experiencings same iteration phase: synchronous phase, calculation stages and message communicating stage.
For the multitask of executed in parallel in BSP model, dynamic load leveling is a key point in parallel computation.But under parallel environment medium-high frequency iteration scene, inevitably there is the situation of load imbalance in BSP model.
Two kinds of solutions are proposed at present: a kind of is load balancing in Giraph for this problem of load imbalance; Another kind is the speculating type implementation strategy in Hadoop.
Wherein, the load balancing in Giraph, is the dynamic partition strategy of data.Three kinds of processing modes are adopted: the first is static division when carrying out the dynamic division of data; The second balances according to limit number; The third balances according to number of vertex.
Particularly, a kind of simple heuritic approach is used to realize the dynamic division of data between working point balanced.Comprise: the first step, all data fragmentations are sorted according to equilibrium valve is descending; Second step, according to the overall balance value of each working point, carries out sequence and forms working point heap by working point; 3rd step, data fragmentation is put into from big to small successively the heap top of working point heap, after often discharging a data fragmentation, and the working point of putting data fragmentation is put into again working point heap, ensure like this data fragmentation to be put into the working point using capacity minimum at every turn; 4th step, returns the information list of the data fragmentation after balance; 5th step, according to the information list of the data fragmentation after the balance returned, carries out the dynamic division migration of data.
Speculating type implementation strategy in Hadoop, mainly Hadoop detects the slow task existed in one-stop operation, and respectively speculating type execution is carried out to the slow task start Map task detected, from slow task, finally choose the input of output as Reduce task of the fastest task of execution.
In sum, adopt the load balancing in Giraph in BSP model, all need to carry out Data Migration before each iterative computation, adjustment requires a great deal of time as the cost of load balancing like this; In BSP model, adopt the speculating type implementation strategy in Hadoop, because supposition task is from 0 superledge, belong to different superledge from corresponding slow task, the message communicating of current superledge cannot be carried out, therefore, need to increase message communication network expense.
Summary of the invention
In view of this, embodiments provide a kind of processing method and equipment of load balancing, in BSP model, adopt the load balancing in Giraph to there is time consuming problem at present for solving, and in BSP model, adopt the speculating type implementation strategy in Hadoop to there is the problem increasing message communication network expense.
A processing method for load balancing, comprising:
Obtain information running time of an iteration cycle each task interior of a working node;
According to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle;
When the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node.
According to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle, comprising:
According to acquisition, information running time of each task, sorts information running time of each task described;
Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle; And
Utilize migration Cost Model, calculate the migration financial value of the standard migration task determined;
Migration financial value is greater than to the standard migration task of setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle, comprising:
For each of the information in sequencing information, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle running time;
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
Utilize migration Cost Model, calculate the migration financial value of the standard migration task determined, comprising:
Calculate the migration financial value of the standard migration task determined in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunTime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
When the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node, comprises:
Judge whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle;
When there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
A treatment facility for load balancing, comprising:
Acquisition module, for obtain a working node an iteration cycle in information running time of each task;
Determination module, for the information and the analysis strategy of task to be migrated running time according to each task described in obtaining, determines the to be migrated task of described working node in described iteration cycle;
Transferring module, during for exceeding setting numerical value when the number of times being defined as task to be migrated, is less than the working node of setting threshold to the task treating capacity except described working node by described task immigration to be migrated.
Described determination module, specifically for information running time of each task according to acquisition, sorts information running time of each task described; Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle; And utilize migration Cost Model, calculate the migration financial value of the standard migration task determined; Migration financial value is greater than to the standard migration task of setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
Described determination module, specifically for for each of the information in sequencing information running time, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle;
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
Described determination module, specifically for calculating the migration financial value of the standard migration task determined in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunTime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
Described transferring module, specifically for judging whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle;
When there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
Beneficial effect of the present invention is as follows:
The embodiment of the present invention obtains information running time of an iteration cycle each task interior of a working node, according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle, when the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node, like this by the information and the analysis strategy of task to be migrated running time according to task, filter out the task to be migrated of described working node, and when the number of times being defined as task to be migrated exceedes setting numerical value, the task treating capacity started outside by this task immigration to be migrated to described working node is less than the working node of setting threshold, that is task immigration mode is utilized, a task immigration is performed in successive ignition operation in working point, effectively realize task load between working point balanced, avoid the problem adopting the load balancing life period in Giraph to consume, and adopt the mode of Real-Time Monitoring Runtime information, avoid and adopt the speculating type implementation strategy in Hadoop to there is the problem increasing message communication network expense, effectively improve the efficiency of task immigration in BSP model, improve the load equilibrium of system.
Accompanying drawing explanation
The schematic flow sheet of the method for a kind of load balancing that Fig. 1 provides for the embodiment of the present invention one;
Fig. 2 is the schematic flow sheet determining the to be migrated task of described working node in described iteration cycle;
The schematic flow sheet of the method for a kind of load balancing that Fig. 3 provides for the embodiment of the present invention two.
Embodiment
In order to realize object of the present invention, embodiments provide a kind of method and apparatus of load balancing, by information running time of each task in an iteration cycle obtaining a working node, according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle, when the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node, like this by the information and the analysis strategy of task to be migrated running time according to task, filter out the task to be migrated of described working node, and when the number of times being defined as task to be migrated exceedes setting numerical value, the task treating capacity started outside by this task immigration to be migrated to described working node is less than the working node of setting threshold, that is task immigration mode is utilized, a task immigration is performed in successive ignition operation in working point, effectively realize task load between working point balanced, avoid the problem adopting the load balancing life period in Giraph to consume, and adopt the mode of Real-Time Monitoring Runtime information, avoid and adopt the speculating type implementation strategy in Hadoop to there is the problem increasing message communication network expense, effectively improve the efficiency of task immigration in BSP model, improve the load equilibrium of system.
Below in conjunction with Figure of description, each embodiment of the present invention is described in detail.
Embodiment one:
As shown in Figure 1, the schematic flow sheet of the method for a kind of load balancing provided for the embodiment of the present invention one.Described method can be as described below.
Step 101: information running time obtaining an iteration cycle each task interior of a working node.
In a step 101, when the task of working node brings into operation, information running time of each task in the statistics current iteration cycle, the data amount information of local task and the size of message for next iteration cycle produced in the current iteration cycle is carried out that pre-estimation obtains estimate metrical information (can also be represented by the temporal information of read/write checkpoint).
It should be noted that, that adds up the data amount information of local task that obtains and carry out that pre-estimation obtains to the size of message for next iteration cycle produced in the current iteration cycle estimates metrical information, will relate to migration cost during calculation task migration.
Performing within the current iteration cycle when working point of task has M, and information running time of so adding up each task obtained is: T
1, T
2, T
3..., T
m.
Step 102: according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle.
In a step 102, according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine that the mode of the to be migrated task of described working node in described iteration cycle includes but not limited to under type:
As shown in Figure 2, for determining the schematic flow sheet of the to be migrated task of described working node in described iteration cycle.
Step 201: information running time of each task according to acquisition, sorts information running time of each task described.
In step 201, when getting information running time of each task described, according to the ordering rule of setting, information running time of each task described is sorted.
Such as: the ordering rule of setting is according to information running time order from small to large; Or be according to information running time order from big to small; Here do not limit.
Particularly, suppose that information running time (wherein, subscript is expressed as task identification) getting each task described is T
1=1s, T
2=2s, T
3=4s, T
4=5s, T
5=7s, T
6=3.5s, T
7=1.5s, T
8=9s, T
9=3s, T
10=0.5s, T
11=4.5s, T
12=6s, sort according to information running time order from small to large, the sequence obtained is: { T
10, T
1, T
7, T
2, T
9, T
6, T
3, T
11, T
4, T
12, T
5, T
8.
Step 202: utilize the sequencing information obtained, determines the standard migration task of described working node in described iteration cycle.
In step 202., for each of the information in sequencing information running time, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle.
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
Particularly, first, according to the sequencing information obtained, to determine in sequencing information 1/4th places corresponding first running time information and 3/4ths places corresponding second running time information.
Still for above-mentioned example, the sequence obtained is: { T
10, T
1, T
7, T
2, T
9, T
6, T
3, T
11, T
4, T
12, T
5, T
8, according to the sequence information obtained, 1/4th places corresponding first running time information be T
7, namely task identification is information running time of 7 correspondences, 3/4ths places corresponding second running time information be T
4, namely task identification is information running time of 4 correspondences.
Secondly, for each of the information in sequencing information running time, judge running time, whether information was greater than the operation threshold of setting, if running time, information was greater than the operation threshold of setting, then determine that task corresponding to this of information is the standard migration task of described working node in described iteration cycle running time; Otherwise, determine that task corresponding to this of information is not the standard migration task of described working node in described iteration cycle running time.
Wherein, the operation threshold of setting can be second running time information with (second running time information-the first information running time) * 1.5 sums, data can also determine by experiment, or be determined by practical experience, not limit here.
Particularly, when determining that running time, information was greater than [second running time information+(second running time information-the first information running time) * 1.5], the task of determining to be greater than information running time of [second running time information+(second running time information-the first information running time) * 1.5] corresponding is the standard migration task of described working node in described iteration cycle.
Such as: still for above-mentioned example, according to the sequence information obtained, 1/4th places corresponding first running time information be T
7, namely task identification is information running time of 7 correspondences, 3/4ths places corresponding second running time information be T
4, namely task identification is information running time of 4 correspondences, and the operation threshold of the setting obtained is 7.75s, therefore, running time information to be greater than the task of 7.75s be task 8, that is, in this example, the standard migration task of the described working node determined in described iteration cycle is task 8.
Step 203: utilize migration Cost Model, calculates the migration financial value of the standard migration task determined.
In step 203, the migration financial value of the standard migration task determined is calculated in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunTime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
It should be noted that, T.
migrateCostthe migration cost information obtained when being information running time obtaining each task in a step 101, can be obtained by the time of the read/write checkpoint of diagram data and message.Wherein, the access time of diagram data can with the time of read-write check point diagram data (or 2 of the diagram data load time times) approximate simulation; Carry out the read/write time of evaluate message data by reading current iteration message data volume size used, computing formula is: read-write message data time=read-write diagram data time/the total message data volume size (byte) of diagram data size (byte) *.
Step 204: standard migration task migration financial value being greater than to setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
Wherein, setting threshold value is generally 0, also just means that migration financial value may be negative value, but is tasks that migration financial value is greater than 0 correspondence for the research object of step 204.
In step 204, after obtaining migration financial value and being greater than the standard migration task of setting threshold value, the task treating capacity of working point in setting-up time of aiming at migration task place is needed to judge, if the task treating capacity of working point in setting-up time at accurate migration task place is little, make this working point idling-resource many, the standard migration task now determined will not considered as task to be migrated; Only when the task treating capacity of working point in setting-up time at standard migration task place is greater than setting threshold, mean that in setting-up time, the duty ratio of working point is heavier, task is moved by necessity, alleviate the task processing pressure of working point, so the standard migration task now determined is using by as the to be migrated task of described working node in described iteration cycle.
Wherein, setting-up time can be treated a little that task average operating time is and amasss (i.e. avgRunTime*T. with working for residue superledge running time of standard migration task of determining
remainSuperStep) 20% time, also can be experimental data or determined by actual needs, not limit here.
Setting threshold can be 1/2nd of working point task treating capacity, also can be experimental data or is determined by actual needs, does not limit here.
Effectively prevent the possibility of considerable task migration in working point like this, reduce the consumption of system resource, improve the accuracy of task immigration.
Step 103: when the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node.
In step 103, first, judge whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle.
Wherein, N is natural number, and is less than total iteration cycle number of working point, task place to be migrated.
Here, avoid an Iterative statistical error in data and cause determining that migration task exists error, improve the accuracy of task immigration.
Secondly, when there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
By the scheme of the embodiment of the present invention one, obtain information running time of an iteration cycle each task interior of a working node, according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle, when the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node, like this by the information and the analysis strategy of task to be migrated running time according to task, filter out the task to be migrated of described working node, and when the number of times being defined as task to be migrated exceedes setting numerical value, the task treating capacity started outside by this task immigration to be migrated to described working node is less than the working node of setting threshold, that is task immigration mode is utilized, a task immigration is performed in successive ignition operation in working point, effectively realize task load between working point balanced, avoid the problem adopting the load balancing life period in Giraph to consume, and adopt the mode of Real-Time Monitoring Runtime information, avoid and adopt the speculating type implementation strategy in Hadoop to there is the problem increasing message communication network expense, effectively improve the efficiency of task immigration in BSP model, improve the load equilibrium of system.
Embodiment two:
As shown in Figure 3, the structural representation of the treatment facility of a kind of load balancing provided for the embodiment of the present invention two, the embodiment of the present invention two is the inventions belonged to the embodiment of the present invention one under same inventive concept, and described equipment comprises: acquisition module 11, determination module 12 and transferring module 13, wherein:
Acquisition module 11, for obtain a working node an iteration cycle in information running time of each task;
Determination module 12, for the information and the analysis strategy of task to be migrated running time according to each task described in obtaining, determines the to be migrated task of described working node in described iteration cycle;
Transferring module 13, during for exceeding setting numerical value when the number of times being defined as task to be migrated, is less than the working node of setting threshold to the task treating capacity except described working node by described task immigration to be migrated.
Particularly, described determination module 12, specifically for information running time of each task according to acquisition, sorts information running time of each task described; Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle; And utilize migration Cost Model, calculate the migration financial value of the standard migration task determined; Migration financial value is greater than to the standard migration task of setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
Described determination module 12, specifically for for each of the information in sequencing information running time, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle;
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
Described determination module 12, specifically for calculating the migration financial value of the standard migration task determined in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunT
ime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
Described transferring module 13, specifically for judging whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle;
When there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
Wherein, N is natural number, and is less than total iteration cycle number of working point, task place to be migrated.
It should be noted that, the treatment facility described in the embodiment of the present invention can be realized by hardware mode, also can be realized by software mode, not limit here.
It will be understood by those skilled in the art that embodiments of the invention can be provided as method, device (equipment) or computer program.Therefore, the present invention can adopt the form of complete hardware embodiment, completely software implementation or the embodiment in conjunction with software and hardware aspect.And the present invention can adopt in one or more form wherein including the upper computer program implemented of computer-usable storage medium (including but not limited to magnetic disc store, CD-ROM, optical memory etc.) of computer usable program code.
The present invention describes with reference to according to the flow chart of the method for the embodiment of the present invention, device (equipment) and computer program and/or block diagram.Should understand can by the combination of the flow process in each flow process in computer program instructions realization flow figure and/or block diagram and/or square frame and flow chart and/or block diagram and/or square frame.These computer program instructions can being provided to the processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing device to produce a machine, making the instruction performed by the processor of computer or other programmable data processing device produce device for realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be stored in can in the computer-readable memory that works in a specific way of vectoring computer or other programmable data processing device, the instruction making to be stored in this computer-readable memory produces the manufacture comprising command device, and this command device realizes the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, make on computer or other programmable devices, to perform sequence of operations step to produce computer implemented process, thus the instruction performed on computer or other programmable devices is provided for the step realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
Although describe the preferred embodiments of the present invention, those skilled in the art once obtain the basic creative concept of cicada, then can make other change and amendment to these embodiments.So claims are intended to be interpreted as comprising preferred embodiment and falling into all changes and the amendment of the scope of the invention.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.
Claims (10)
1. a processing method for load balancing, is characterized in that, comprising:
Obtain information running time of an iteration cycle each task interior of a working node;
According to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determine the to be migrated task of described working node in described iteration cycle;
When the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node.
2. processing method as claimed in claim 1, is characterized in that, according to the information and the analysis strategy of task to be migrated running time of each task described in obtaining, determines the to be migrated task of described working node in described iteration cycle, comprising:
According to acquisition, information running time of each task, sorts information running time of each task described;
Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle; And
Utilize migration Cost Model, calculate the migration financial value of the standard migration task determined;
Migration financial value is greater than to the standard migration task of setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
3. processing method as claimed in claim 2, is characterized in that, utilizes the sequencing information obtained, and determines the standard migration task of described working node in described iteration cycle, comprising:
For each of the information in sequencing information, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle running time;
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
4. processing method as claimed in claim 2 or claim 3, is characterized in that, utilizes migration Cost Model, calculates the migration financial value of the standard migration task determined, comprising:
Calculate the migration financial value of the standard migration task determined in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunTime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
5. processing method as claimed in claim 4, it is characterized in that, when the number of times being defined as task to be migrated exceedes setting numerical value, described task immigration to be migrated is less than the working node of setting threshold to the task treating capacity except described working node, comprises:
Judge whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle;
When there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
6. a treatment facility for load balancing, is characterized in that, comprising:
Acquisition module, for obtain a working node an iteration cycle in information running time of each task;
Determination module, for the information and the analysis strategy of task to be migrated running time according to each task described in obtaining, determines the to be migrated task of described working node in described iteration cycle;
Transferring module, during for exceeding setting numerical value when the number of times being defined as task to be migrated, is less than the working node of setting threshold to the task treating capacity except described working node by described task immigration to be migrated.
7. treatment facility as claimed in claim 6, is characterized in that,
Described determination module, specifically for information running time of each task according to acquisition, sorts information running time of each task described; Utilize the sequencing information obtained, determine the standard migration task of described working node in described iteration cycle; And utilize migration Cost Model, calculate the migration financial value of the standard migration task determined; Migration financial value is greater than to the standard migration task of setting threshold value, when move financial value be greater than setting threshold value standard migration task place the task treating capacity of working node in setting-up time be greater than setting threshold time, determine move financial value be greater than setting threshold value standard migration task be the to be migrated task of described working node in described iteration cycle.
8. treatment facility as claimed in claim 7, is characterized in that,
Described determination module, specifically for for each of the information in sequencing information running time, when determining that running time, information was greater than setting information running time, determine that being greater than task corresponding to setting information running time is the standard migration task of described working node in described iteration cycle;
Wherein, described setting information running time is determined in the following manner:
T=T
2+ (T
2-T
1) * 1.5; T is setting information running time, T
2for the sequencing information that basis obtains, determine the temporal information that in sequencing information, 3/4ths places are corresponding; T
1for the sequencing information that basis obtains, determine information running time that in sequencing information, 1/4th places are corresponding.
9. treatment facility as claimed in claim 7 or 8, is characterized in that,
Described determination module, specifically for calculating the migration financial value of the standard migration task determined in the following manner:
G(T)＝T.
remainSuperStep*(T.
runTime-avgRunTime)-T.
migrateCost；
Wherein, G (T) for determine standard migration task migration financial value, T.
remainSuperStepfor the standard determined moves residue superledge information running time of task, T.
runTimefor the standard determined moves information running time of task, avgRunTime is the average operating time information of described working node non-accurate migration task in described iteration cycle, T.
migrateCostfor the migration cost temporal information of standard migration task determined, equal the time span sum that the time span of Data import and message are read or write.
10. treatment facility as claimed in claim 9, is characterized in that,
Described transferring module, specifically for judging whether described working node exists the number of times being defined as task to be migrated and exceed the task to be migrated setting numerical value in a N continuous iteration cycle;
When there is the number of times being defined as task to be migrated and exceeding the task to be migrated of setting numerical value, judge that the number of times being defined as task to be migrated exceedes the task to be migrated that set numerical value as migration task, and the task treating capacity of the described migration task immigration judged extremely except described working node is less than in the working node of setting threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410108066.2A CN104935523B (en) | 2014-03-21 | 2014-03-21 | The processing method and equipment of a kind of load balancing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410108066.2A CN104935523B (en) | 2014-03-21 | 2014-03-21 | The processing method and equipment of a kind of load balancing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104935523A true CN104935523A (en) | 2015-09-23 |
| CN104935523B CN104935523B (en) | 2018-06-15 |
Family
ID=54122497
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410108066.2A Active CN104935523B (en) | 2014-03-21 | 2014-03-21 | The processing method and equipment of a kind of load balancing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104935523B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107517273A (en) * | 2017-09-21 | 2017-12-26 | 郑州云海信息技术有限公司 | Method, system, computer-readable recording medium and the server of Data Migration |
| CN108073446A (en) * | 2016-11-10 | 2018-05-25 | 华为技术有限公司 | Overtime pre-judging method and device |
| CN108268321A (en) * | 2016-12-30 | 2018-07-10 | 三星电子株式会社 | For migrating the method for workload and machine frame system |
| CN111104209A (en) * | 2019-11-25 | 2020-05-05 | 华为技术有限公司 | Method for processing task and related equipment |
| CN111176814A (en) * | 2019-12-29 | 2020-05-19 | 山东英信计算机技术有限公司 | Task execution method and related device |
| CN107346262B (en) * | 2017-06-06 | 2020-12-15 | 华为技术有限公司 | Task migration method and controller |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050185646A1 (en) * | 2004-02-25 | 2005-08-25 | Nec Corporation | Communication processing system, packet processing load balancing device and packet processing load balancing method therefor |
| CN101446910A (en) * | 2008-12-08 | 2009-06-03 | 哈尔滨工程大学 | AEDF task scheduling method based on SMP |
| CN101986272A (en) * | 2010-11-05 | 2011-03-16 | 北京大学 | Task scheduling method under cloud computing environment |
| CN103336808A (en) * | 2013-06-25 | 2013-10-02 | 中国科学院信息工程研究所 | System and method for real-time graph data processing based on BSP (Board Support Package) model |
-
2014
- 2014-03-21 CN CN201410108066.2A patent/CN104935523B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050185646A1 (en) * | 2004-02-25 | 2005-08-25 | Nec Corporation | Communication processing system, packet processing load balancing device and packet processing load balancing method therefor |
| CN101446910A (en) * | 2008-12-08 | 2009-06-03 | 哈尔滨工程大学 | AEDF task scheduling method based on SMP |
| CN101986272A (en) * | 2010-11-05 | 2011-03-16 | 北京大学 | Task scheduling method under cloud computing environment |
| CN103336808A (en) * | 2013-06-25 | 2013-10-02 | 中国科学院信息工程研究所 | System and method for real-time graph data processing based on BSP (Board Support Package) model |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073446A (en) * | 2016-11-10 | 2018-05-25 | 华为技术有限公司 | Overtime pre-judging method and device |
| CN108073446B (en) * | 2016-11-10 | 2020-11-17 | 华为技术有限公司 | Timeout prejudging method and device |
| CN108268321A (en) * | 2016-12-30 | 2018-07-10 | 三星电子株式会社 | For migrating the method for workload and machine frame system |
| CN108268321B (en) * | 2016-12-30 | 2023-11-07 | 三星电子株式会社 | Method and rack system for migrating workload |
| CN107346262B (en) * | 2017-06-06 | 2020-12-15 | 华为技术有限公司 | Task migration method and controller |
| CN107517273A (en) * | 2017-09-21 | 2017-12-26 | 郑州云海信息技术有限公司 | Method, system, computer-readable recording medium and the server of Data Migration |
| CN111104209A (en) * | 2019-11-25 | 2020-05-05 | 华为技术有限公司 | Method for processing task and related equipment |
| CN111104209B (en) * | 2019-11-25 | 2023-07-11 | 华为技术有限公司 | Task processing method and related equipment |
| CN111176814A (en) * | 2019-12-29 | 2020-05-19 | 山东英信计算机技术有限公司 | Task execution method and related device |
| CN111176814B (en) * | 2019-12-29 | 2022-06-17 | 山东英信计算机技术有限公司 | Task execution method and related device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104935523B (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104935523A (en) | Load balancing processing method and equipment | |
| Acun et al. | Parallel programming with migratable objects: Charm++ in practice | |
| CN103914365B (en) | Method and device for acquiring power consumption of mobile terminal application | |
| CN105022670A (en) | Heterogeneous distributed task processing system and processing method in cloud computing platform | |
| CN105446979A (en) | Data mining method and node | |
| CN112433819A (en) | Heterogeneous cluster scheduling simulation method and device, computer equipment and storage medium | |
| CN104765589A (en) | Grid parallel preprocessing method based on MPI | |
| CN106980571A (en) | The construction method and equipment of a kind of test use cases | |
| CN111813517B (en) | Task queue allocation method and device, computer equipment and medium | |
| CN114936085A (en) | ETL scheduling method and device based on deep learning algorithm | |
| CN103049516A (en) | Method and device for processing data | |
| CN106648839A (en) | Method and device for processing data | |
| Fan et al. | A heterogeneity-aware data distribution and rebalance method in Hadoop cluster | |
| CN111984414A (en) | Data processing method, system, equipment and readable storage medium | |
| CN113031954A (en) | Code compiling method and device, electronic equipment, storage medium and heterogeneous system | |
| CN108139929B (en) | Task scheduling apparatus and method for scheduling a plurality of tasks | |
| US20240005446A1 (en) | Methods, systems, and non-transitory storage media for graphics memory allocation | |
| CN112990461B (en) | Method, device, computer equipment and storage medium for constructing neural network model | |
| CN116069603B (en) | Performance test method of application, method and device for establishing performance test model | |
| CN115841047A (en) | Screening method and device for engine balanced major cycle and storage medium | |
| CN115309502A (en) | Container scheduling method and device | |
| CN106844037B (en) | KNL-based test method and system | |
| CN102063308B (en) | Method for controlling processing flow of seismic prospecting data | |
| CN115456188A (en) | Quantum computing task optimization processing method and device and quantum computer | |
| CN111582464B (en) | Neural network processing method, computer system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |