CN104184813B

CN104184813B - The load-balancing method and relevant device and group system of virtual machine

Info

Publication number: CN104184813B
Application number: CN201410412412.6A
Authority: CN
Inventors: 柳永强; 张伟
Original assignee: Hangzhou Huawei Digital Technologies Co Ltd
Current assignee: Hangzhou Huawei Digital Technologies Co Ltd
Priority date: 2014-08-20
Filing date: 2014-08-20
Publication date: 2018-03-09
Anticipated expiration: 2034-08-20
Also published as: CN104184813A

Abstract

The invention discloses a kind of load-balancing method of virtual machine and relevant device and group system, to improve network performance of the group system after load balancing.In some feasible embodiments of the invention, method includes：Management node determines to need the multiple virtual machines for carrying out load balancing in group system；According to the network traffics relation between the multiple virtual machine, the multiple virtual machine is divided at least one virtual machine cluster, so that, the network traffics between at least one other virtual machine in each virtual machine and same virtual machine cluster are more than or equal to network traffics threshold value, and the network traffics between any one other virtual machine in each virtual machine and different virtual machine cluster are less than network traffics threshold value；Generation migration suggests that migration is proposed to be used in whole virtual machines that instruction includes each virtual machine cluster, moves to or host is on same target computing nodes；The source calculate node of the one or more virtual machine hosts included to virtual machine cluster sends the migration and suggested.

Description

The load-balancing method and relevant device and group system of virtual machine

Technical field

The present invention relates to computer and communication technical field, and in particular to the load-balancing method and correlation of a kind of virtual machine Equipment and group system.

Background technology

Server virtualization technology is the key technology of cloud computing, and the technology is by (or being physics to physical server Main frame) virtualized, realize and more virtual machines (Virtual Machine, VM) are disposed on separate unit physical server, to carry The resource utilization of high physical server.The physical server of more virtualizations can form a virtual cluster, virtualization collection Physical resource in cluster is abstracted as the resource pool of the various resource compositions such as storage, calculating by group, passes through on-demand application resource Mode provides virtual machine to user.

One key property of virtual cluster be dynamic resource scheduling (Dynamic Resources Scheduling, DRS).The resources such as the calculating of more physical hosts in cluster, storage can be uniformly presented to user by DRS, be moved using virtual machine Shifting technology, do not influenceing customer service or in the case where user does not perceive, virtual machine is being moved in different physical hosts Move, use focus to eliminate resource in time, improve resource utilization.

In the prior art, such as farsighted (VMware) DRS of prestige, can by calculate central processing unit (CPU), internal memory (Memory, MEM) resource load of two dimensions carrys out calculate node in gauge cluster (i.e. above described physical server or physical host) Computing resource load.By in virtual cluster calculate resource load measurement and calculating, VMware DRS realize with Lower function：(1) initial placement：When starting virtual machine, virtual machine is placed on computing resource and loaded in relatively low calculate node； (2) load balancing：The computing resource loading condition of each calculate node in virtual cluster is monitored in real time, generates virtual machine (vm) migration It is recommended that by the calculate node of the virtual machine (vm) migration of high capacity calculate node to low-load；(3) energy optimization：With reference to DPM (Distributed Power Management, distributed power supply management) function, according to the load of each calculate node, to meter Operator node carries out lower electricity or power on operation, optimizes energy consumption.

Practice finds that the DRS functions of prior art carry out load balancing according to computing resource loading condition, following lack be present Fall into：If there is performance bottleneck in cluster network link, it is easy to influences the network performance of virtual machine in cluster；It is inappropriate virtual The decline that can cause virtual machine network performance is suggested in machine migration.

The content of the invention

The embodiment of the present invention provides a kind of load-balancing method and relevant device and group system of virtual machine, to improve void Network performance of the planization group system after load balancing.

First aspect present invention provides a kind of load-balancing method of virtual machine, methods described be used to including management node with The group system of multiple calculate nodes, each calculate node in the multiple calculate node include hardware layer, operate in it is described Host on hardware layer and the virtual machine operated on the host；Methods described includes：The management node Determine to need the multiple virtual machines for carrying out load balancing in the group system, operate in the multiple virtual machine distribution described In the part or all of calculate node of the multiple calculate node in group system；According to the net between the multiple virtual machine Network discharge relation, the multiple virtual machine is divided at least one virtual machine cluster so that, each virtual machine and same virtual machine The network traffics between at least one other virtual machine in cluster are more than or equal to network traffics threshold value, each virtual machine from it is different The network traffics between any one other virtual machine in virtual machine cluster are less than network traffics threshold value；For each virtual machine fasciation Suggest into migration, the migration is proposed to be used in whole virtual machines that instruction includes each virtual machine cluster, moves to or host exists On same target computing nodes；The source calculate node hair of the one or more virtual machine hosts included to the virtual machine cluster The migration is sent to suggest, in order to which whole virtual machines that the virtual machine cluster includes are migrated to or host calculates in the target On node.

With reference to first aspect present invention, in the first possible implementation, it is described according to the multiple virtual machine it Between network traffics relation, the multiple virtual machine is divided into at least one virtual machine cluster includes：According to the multiple virtual Network traffics relation between machine, build the network service connection topology of the multiple virtual machine；The network service is connected Topology is split, and each group of virtual machine that mutual network traffics are more than or equal to network traffics threshold value is defined as one Individual virtual machine cluster, and, each is isolated into virtual machine and is individually determined as a virtual machine cluster, the isolated virtual machine with it is other The network traffics of any one virtual machine are both less than network traffics threshold value.

With reference to the possible implementation of the first of first aspect present invention or first aspect, in second of possible realization In mode, multiple virtual machines that the management node determines to need to carry out load balancing in the group system include：From described In the whole virtual machines run in group system, the virtual machine between each other with network traffics relation is filtered out, it is determined that described The virtual machine with network traffics relation is the multiple virtual machines for needing to carry out load balancing in the group system between each other.

, can at the third with reference to the first or second of possible implementation of first aspect present invention or first aspect Can implementation in, the management node determine to need in the group system to carry out load balancing multiple virtual machines it Before, in addition to：The resource that the management node obtains the group system uses data, and the resource is included described using data The cpu busy percentage and memory usage of each calculate node in group system, and each bar link between each calculate node Network resource utilization；Data computational load equilibrium desired value is used according to the resource, the load balancing desired value is used In the load balancing state for representing the group system；If the load balancing desired value is more than load balancing threshold value, institute is judged Stating group system needs to carry out load balancing.

With reference to the third possible implementation of first aspect present invention, in the 4th kind of possible implementation, institute State is included according to the resource using data computational load equilibrium desired value：Using following formula computational load equilibrium desired values；T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET)+ω₄×δ_VM(S)；

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) represent each calculate node cpu busy percentage variance, σ (Util_MEM) represent that the internal memory of each calculate node utilizes The variance of rate, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) Represent the average weighted delay of all virtual machines run in the group system, ω₁、ω₂、ω₃、ω₄It is weighting system respectively Number.

, can at the 5th kind with reference to the first or second of possible implementation of first aspect present invention or first aspect Described to be directed to each virtual machine fasciation into migration suggestion in the implementation of energy, the migration, which is proposed to be used in, to be indicated each void Whole virtual machines that plan machine cluster includes, are moved to or host includes on same target computing nodes：For each virtual machine Cluster selects multiple candidate's calculate nodes, and each virtual machine cluster of simulation calculating is migrated to or host calculates in the multiple candidate and saved The load balancing desired value of the group system after candidate's calculate node in point；For each virtual machine cluster, from described A target computing nodes are determined in multiple candidate's calculate nodes so that whole virtual machine (vm) migrations in each virtual machine cluster arrive or The load balancing desired value of host's group system after selected target computing nodes is minimum；For each virtual machine cluster Generation migration suggests that the migration is proposed to be used in instruction whole virtual machines for including each virtual machine cluster, moves to or host On the target computing nodes of the selection.

With reference to the 5th kind of possible implementation of first aspect present invention, in the 6th kind of possible implementation, institute State simulation and calculate that each virtual machine cluster is migrated to or a candidate of the host in the multiple candidate's calculate node calculates section The load balancing desired value of the group system includes after point：Using following formula computational load equilibrium desired values；

T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET)+ω₄×δ_VM(S)+ω₅× cost(S)

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) represent each calculate node cpu busy percentage variance, σ (Util_MEM) represent that the internal memory of each calculate node utilizes The variance of rate, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) The average weighted delay of all virtual machines run in the group system is represented, cost (S) represents group system mapping cost, ω₁、ω₂、ω₃、ω₄、ω₅It is weight coefficient respectively.

Second aspect of the present invention provides a kind of management node, for the collection including multiple calculate nodes and the management node Group's system, each calculate node in the multiple calculate node include hardware layer, the host operated on the hardware layer Machine and the virtual machine operated on the host；The management node includes：

Determining module, it is the multiple for determining to need the multiple virtual machines for carrying out load balancing in the group system Operate in the part or all of calculate node of the multiple calculate node in the group system to virtual machine distribution；

Sub-clustering module, for according to the network traffics relation between the multiple virtual machine, the multiple virtual machine to be drawn It is divided at least one virtual machine cluster so that, between at least one other virtual machine in each virtual machine and same virtual machine cluster Network traffics be more than or equal to network traffics threshold value, each virtual machine and any one other virtual machine in different virtual machine cluster Between network traffics be less than network traffics threshold value；

Suggestion module, for suggesting for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will be each Whole virtual machines that virtual machine cluster includes, are moved to or host is on same target computing nodes；

Sending module, for the source calculate node hair of the one or more virtual machine hosts included to the virtual machine cluster The migration is sent to suggest, in order to which whole virtual machines that the virtual machine cluster includes are migrated to or host calculates in the target On node.

With reference to second aspect of the present invention, in the first possible implementation, the sub-clustering module includes：

Construction unit, for according to the network traffics relation between the multiple virtual machine, building the multiple virtual machine Network service connection topology；

Division unit, for splitting to the network service connected graph, mutual network traffics are more than or The each group of virtual machine equal to network traffics threshold value is divided into a virtual machine cluster, and, it is independent that each is isolated into virtual machine It is defined as a virtual machine cluster, the network traffics of the isolated virtual machine and any one other virtual machine are both less than network traffics threshold Value.

With reference to the possible implementation of the first of second aspect of the present invention or second aspect, in second of possible realization In mode, the determining module is specifically used for：In the whole virtual machines run from the group system, filter out between each other Virtual machine with network traffics relation, determine that the virtual machine with network traffics relation between each other is the cluster system Empty multiple plan machines of progress load balancing are needed in system.

, can at the third with reference to the first or second of possible implementation of second aspect of the present invention or second aspect In the implementation of energy, management node also includes：Acquisition module, the resource for obtaining the group system use data, institute Stating resource includes the cpu busy percentage and memory usage of each calculate node in the group system using data, and each The network resource utilization of each bar link between calculate node；Computing module, for being calculated according to the resource using data Load balancing desired value, the load balancing desired value are used for the load balancing state for representing the group system；Judge module, If being more than load balancing threshold value for the load balancing desired value, judge that the group system needs to carry out load balancing.

With reference to the third possible implementation of second aspect of the present invention, in the 4th kind of possible implementation, institute Computing module is stated to be specifically used for using following formula computational load equilibrium desired values；

T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET)+ω₄×δ_VM(S)

, can at the 5th kind with reference to the first or second of possible implementation of second aspect of the present invention or second aspect In the implementation of energy, the suggestion module includes：

Analogue computer, for selecting multiple candidate's calculate nodes for each virtual machine cluster, simulation calculates each empty Plan machine cluster be migrated to or a candidate calculate node of the host in the multiple candidate's calculate node after the group system Load balancing desired value；

Determining unit, for for each virtual machine cluster, a target meter to be determined from the multiple candidate's calculate node Operator node so that whole virtual machine (vm) migrations in each virtual machine cluster arrive or host is described after selected target computing nodes The load balancing desired value of group system is minimum；

It is recommended that generation unit, for suggesting for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will Whole virtual machines that each virtual machine cluster includes, move to or host the selection target computing nodes.

With reference to the 5th kind of possible implementation of second aspect of the present invention, in the 6th kind of possible implementation, institute Analogue computer is stated to be specifically used for：Using following formula computational load equilibrium desired values；

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) represent each calculate node cpu busy percentage variance, σ (Util_MEM) represent that the internal memory of each calculate node utilizes The variance of rate, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) The average weighted delay of all virtual machines run in the group system is represented, cost (S) represents system mapping cost, ω₁、 ω₂、ω₃、ω₄、ω₅It is weight coefficient respectively.

Third aspect present invention provides a kind of virtual cluster, including：Multiple calculate nodes and such as second party of the present invention Management node described in face；The calculate node is used to receive the migration suggestion that the management node is sent, according to the migration It is recommended that instruction perform virtual machine (vm) migration operation.

Therefore in some embodiments of the invention, according to the network traffics relation between virtual machine, it would be desirable to carry out Multiple virtual machines of load balancing are divided at least one virtual machine cluster so that, in each virtual machine and same virtual machine cluster Network traffics between at least one other virtual machine are more than or equal to network traffics threshold value, each virtual machine and different virtual machine The network traffics between any one other virtual machine in cluster are less than network traffics threshold value；For in the multiple virtual machine cluster Each virtual machine fasciation suggests that the migration is proposed to be used in whole virtual machines that instruction includes each virtual machine cluster into migration, Move to or host is on same target computing nodes；Achieve following technique effect：

When carrying out load balancing, it is contemplated that the influence of network dimension, by by with the same of network traffics relation Multiple virtual machines in virtual machine cluster are focused in same calculate node, so as to multiple virtual in same virtual machine cluster Network traffics between machine will not take the load of the link between calculate node, and each bar chain in group system can be reduced with this There is performance bottleneck in the network load on road, prevention link；Also, by the way that multiple virtual machines in same virtual machine cluster are concentrated Onto same calculate node, network bandwidth between the virtual machine improved in same virtual machine cluster can be improved and communication can By property, so as to lift the network performance of the virtual machine cluster of traffic-intensive.

Brief description of the drawings

Technical scheme in order to illustrate the embodiments of the present invention more clearly, below will be to institute in embodiment and description of the prior art The accompanying drawing needed to use is briefly described, it should be apparent that, drawings in the following description are only some implementations of the present invention Example, for those of ordinary skill in the art, on the premise of not paying creative work, can also be obtained according to these accompanying drawings Obtain other accompanying drawings.

Fig. 1 is a kind of schematic diagram of group system of the embodiment of the present invention；

Fig. 2 a are a kind of schematic flow sheets of the load-balancing method of virtual machine provided in an embodiment of the present invention；

Fig. 2 b are the schematic flow sheets of the load-balancing method of another virtual machine provided in an embodiment of the present invention；

Fig. 3 a are a kind of structural representations of management node provided in an embodiment of the present invention；

Fig. 3 b are the structural representations of another management node provided in an embodiment of the present invention；

Fig. 4 a are the structure of one group system of prior art and the schematic diagram of virtual machine therein distribution；

Fig. 4 b are the structure of one group system of the embodiment of the present invention and the schematic diagram of virtual machine therein distribution；

Fig. 5 is the schematic diagram for the data center that the embodiment of the present invention is applied；

Fig. 6 a are the structural representations for the load balancing apparatus disposed in the management node of the embodiment of the present invention；

Fig. 6 b are the schematic diagrames for the cluster resource management system run in the management node of the embodiment of the present invention；

Fig. 7 is that the management node of the embodiment of the present invention performs the flow chart of load balance scheduling；

Fig. 8 is that the management node of the embodiment of the present invention judges whether to load balancing and generation virtual machine (vm) migration suggestion Flow chart；

Fig. 9 is the schematic flow sheet of partition virtual machines cluster in the embodiment of the present invention；

Figure 10 is the schematic diagram of partition virtual machines cluster in a scene example of the invention；

Figure 11 is the structural representation of another management node provided in an embodiment of the present invention.

Embodiment

The embodiment of the present invention provides a kind of load-balancing method and relevant device and group system of virtual machine, to improve collection Network performance of group's system after load balancing.

In order that those skilled in the art more fully understand the present invention program, below in conjunction with the embodiment of the present invention Accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only The embodiment of a part of the invention, rather than whole embodiments.Based on the embodiment in the present invention, ordinary skill people The every other embodiment that member is obtained under the premise of creative work is not made, it should all belong to the model that the present invention protects Enclose.

To facilitate understanding of the present embodiment of the invention, system and the description that the embodiment of the present invention is applied are introduced herein first The middle several key elements that can be introduced.

Technical scheme of the embodiment of the present invention, applied to system of virtual cluster (abbreviation virtual cluster or group system or Cluster), as shown in figure 1, the group system may include management node 310 and calculate node 320；Management node can have one or It is multiple, such as can be two, it is divided into main management node and standby management node；Calculate node can have multiple.Described management Node 310 and calculate node 320 are all computer equipments, and management node 310 is alternatively referred to as management server, calculate node 320 Alternatively referred to as physical host.

Wherein, calculate node 320 may include hardware layer 3201, the host 3202, the Yi Jiyun that operate on hardware layer At least one virtual machine VM of the row on the host 3202.The following detailed description of：

Virtual machine (Virtual Machine, VM)：

Virtual machine refer to by software simulate have complete hardware system function, operate in a completely isolated environment Computer system.One or more virtual machines can be simulated on a physical host by software virtual machine, and this A little virtual machines are operated just as real computer, can be with installation operation system and application program, virtually on virtual machine Machine may also access Internet resources.It is like to enter in real computer for the application program run in virtual machine Row work.

Hardware layer：

The hardware platform of virtualized environment operation.Wherein, hardware layer may include multiple hardwares, for example, certain calculate node or The hardware layer of management node may include processor (such as CPU) and memory (such as internal memory), can also include network interface card, memory Etc. the input/output of high speed/low speed (I/O, Input/Output) equipment, and with particular procedure function miscellaneous equipment, such as Input and output memory management unit (IOMMU, Input/Output Memory Management Unit), wherein IOMMU can use In the conversion of virtual machine physical address and Host physical address.

Host (Host)：

As management level, to complete the management of hardware resource, distribution；Virtual hardware platform is presented for virtual machine；Realize The scheduling and isolation of virtual machine.Wherein, Host is probably monitor of virtual machine (VMM, Virtual Machine Monitor)； In addition, VMM and 1 privileged virtual machine coordinates sometimes, both combine composition Host.Wherein, virtual hardware platform to running thereon Each virtual machine various hardware resources are provided, such as offer virtual cpu, internal memory, virtual disk, Microsoft Loopback Adapter.Wherein, should Virtual disk can correspond to Host a file or a logic block device.Virtual machine then operates in the void that Host is its preparation Intend on hardware platform, one or more virtual machines are run on Host.

Below by specific embodiment, it is described in detail respectively.

Fig. 2 a are refer to, the embodiment of the present invention provides a kind of load-balancing method of virtual machine.This method can be used for including The group system of management node and multiple calculate nodes, wherein, calculate node may include hardware layer, operate on hardware layer Host and the virtual machine operated on host.

For in the prior art, system of virtual cluster loads according only to computing resource, the virtual machine in cluster is carried out Load balancing, it is impossible to the problem of improving the network performance of virtual machine in cluster, in technical scheme of the embodiment of the present invention, introduce pair The utilization of Internet resources, when carrying out equally loaded in view of the influence of network factors, improved with this virtual after load balancing Change the network performance of cluster.

Fig. 2 a are refer to, present invention method may include：

110th, management node determines to need the multiple virtual machines for carrying out load balancing in the group system；The multiple void Operate in the part or all of calculate node of the multiple calculate node in the group system to the distribution of plan machine；

120th, management node divides the multiple virtual machine according to the network traffics relation between the multiple virtual machine For at least one virtual machine cluster；So that between at least one other virtual machine in each virtual machine and same virtual machine cluster Network traffics are more than or equal to network traffics threshold value, each virtual machine and any one other virtual machine in different virtual machine cluster it Between network traffics be less than network traffics threshold value；

130th, management node is suggested for each virtual machine fasciation into migration, and the migration is proposed to be used in instruction will be each empty Whole virtual machines that plan machine cluster includes, are moved to or host is on same target computing nodes；

140th, moved described in the source calculate node transmission of the one or more virtual machine hosts included to the virtual machine cluster Move and suggest, in order to which whole virtual machines that the virtual machine cluster includes are migrated to or host is on the target computing nodes.

It refer to Fig. 2 b, in some embodiments of the invention, can also include before 110：

100th, management node judges whether group system needs to carry out load balancing.

Illustrate in detail further below：

As the manager of group system, management node provides calculate node and life cycle management, the resource of virtual machine The functions such as management and running, O＆M, it is the brain of whole group system.Management node can gather the various property of group system in real time Can data, including resource is using data and configuration parameter etc., and wherein resource refers to that each resource (including is calculated and provided using data Source, Internet resources, storage resource etc.) use data (such as utilization rate), such as may include the computing resource of each calculate node As CPU, Memory use data, the computing resource such as CPU, Memory of each virtual machine use data are each virtual The Internet usage data of machine, the network traffics relation between virtual machine, the network of the link between each two calculate node Load etc., and historical data can be saved as.

Management node can realize dynamic resource scheduling (DRS) function according to the system performance information gathered in real time, to collection Virtual machine in group's system carries out load balancing.First, load-balancing algorithm is can perform, judges whether group system needs to carry out Load balancing, if it is desired, just start to perform subsequent step.Judge whether to need the method for carrying out load balancing to have a variety of, one As, the judgment step may include：

The resource for obtaining the group system uses data；The resource is included each in the group system using data The cpu busy percentage and memory usage of calculate node, and the network resource usage of each bar link between each calculate node Rate；Data computational load equilibrium desired value is used according to the resource, the load balancing desired value is used to represent the cluster The load balancing state of system；If the load balancing desired value is more than load balancing threshold value, judge that group system needs to carry out Load balancing.

In a kind of embodiment, management node can specifically obtain CPU of the resource using each calculate node in data Utilization rate, and then, according to the cpu busy percentage of each calculate node, the cpu busy percentage of each calculate node in computing cluster system Variance, as load balancing desired value；If load balancing desired value is more than load balancing threshold value, it is judged as that needs are loaded It is balanced.Popular point says that variance is exactly the degree deviateed with central value, and variance is bigger, illustrates the cpu busy percentage of each calculate node Mutual difference it is bigger, more need carry out load balancing.Illustrate, it is assumed that the CPU capacity of a calculate node is 10GHz, the cpu resource that the every virtual machine run in the calculate node averagely takes within the past period is 2GHz, such as Operation has 3 such virtual machines in the fruit calculate node, then the cpu busy percentage of the calculate node is 60%.Under limiting case, Assuming that the cpu busy percentage of certain calculate node has reached 100%, the cpu busy percentage of another calculate node is 0, and variance at this moment is just May be very big, more than threshold value, it is necessary to carry out load balancing.

In another embodiment, management node can specifically obtain resource and use each calculate node in data Cpu busy percentage and internal memory (MEM) utilization rate, and then, it is each in computing cluster system according to the cpu busy percentage of each calculate node The variance of the cpu busy percentage of individual calculate node, and, the variance of the MEM utilization rates of each calculate node, with two calculated The weighted sum of variance, as load balancing desired value., can to two dimensions of CPU and MEM according to the difference of practical application scene To set different weights.Illustrate, it is assumed that the memory size of a calculate node is 10GB, is run in the calculate node every The memory source that platform virtual machine averagely takes within the past period is 2GB；If operation has 3 so in the calculate node Virtual machine, then the memory source utilization rate of the calculate node be 60%.Assuming that the weight of two dimensions of CPU and MEM is a respectively And b, with σ (Util_CPU) represent each calculate node cpu busy percentage variance, with σ (Util_MEM) represent each calculate node MEM utilization rates variance, then load balancing desired value can be equal to a × σ (Util_CPU)+b×σ(Util_MEM)。

In a kind of preferred embodiment of the present invention, the following resource that management node can specifically obtain group system uses Data：The cpu busy percentage and memory usage of each calculate node, and the network of each bar link between each calculate node Resource utilization；Following formula computational load equilibrium desired values can be used；

Wherein, T (S) represents load balancing desired value；S represents the Topological Mapping relation of virtual machine and calculate node, i.e. which A little virtual machines are operated in that calculate node；

σ represents variance, specifically, σ (Util_CPU) represent each calculate node cpu busy percentage variance, σ (Util_MEM) Represent the variance of the memory usage of each calculate node, σ (Util_NET) represent each bar link between each calculate node The variance of network resource utilization；Link mentioned here, refer to the link that any two calculate node is connected in group system；

δ_VM(S) the average weighted delay of all virtual machines run in group system is represented,

Cost (S) represents group system mapping cost, virtual relative to this time after specifically can referring to each virtual machine (vm) migration Before machine migration, the mapping cost of group system；In this step, in order to judge whether to need to carry out load balancing and computational load During balanced desired value, due to not yet carrying out virtual machine (vm) migration, cost (S)=0, therefore, above-mentioned formula can be made not include+ω₅ × cost (S) this or make cost (S)=0.

ω₁、ω₂、ω₃、ω₄、ω₅It is weight coefficient respectively, can be according to being actually needed artificial setting.

In above-mentioned formula, using σ (Util_CPU)、σ(Util_MEM)、σ(Util_NET)、δ_VM(S), this five parameters of cost (S) Computational load equilibrium desired value, it can more accurately reflect the load balance degree of group system.The T that the formula calculates (S) it is smaller, represent that load balance degree is more excellent, T (S) is bigger, represents that load balance degree is poorer.When T (S) is more than load balancing During threshold value, need to carry out load balancing with regard to explanation.

In some embodiments of the invention, δ can not also be considered_VM(S), one or two in cost (S) the two parameters, Load balancing desired value can be：T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET).Other realities Apply in example, can also increase other parameters according to being actually needed in above-mentioned calculation formula.In other embodiment, it can also not have to Variance, but calculated using standard deviation, then the symbol σ in above-mentioned formula may also mean that standard deviation.

Below, so that the σ in formula above represents standard deviation as an example, further explain how according to above-mentioned formula computational load Balanced desired value.

(1) standard deviation sigma (Util of the cpu busy percentage of each calculate node is calculated_CPU)

Assuming that the calculate node PM that numbering is i_iCPU actual use amount be a_i, the capacity of the CPU is c_i, then the calculating Node PM_iCpu busy percentage be ρ_i=a_i/c_i, it is assumed that n calculate node is shared in group system, then this n calculate node CPU average utilization μ_cpuFor：

Accordingly, the standard deviation of the cpu busy percentage of n calculate node is：

(2) standard deviation sigma (Util of the memory usage of each calculate node is calculated_MEM)

Assuming that the calculate node PM that numbering is i_iThe actual use amount of internal memory be a_i, the capacity of the internal memory is c_i, then the meter Operator node PM_iMemory usage be ρ_i=a_i/c_i, it is assumed that n calculate node is shared in group system, then this n calculate node Internal memory average utilization μ_cpuFor：

Accordingly, the standard deviation of the memory usage of n calculate node is：

(3) standard deviation sigma (Util of the resource utilization of each bar link is calculated_MEM)

Link set in group system can be divided into core link and edge link, use LK_coreCore link is represented, is used LK_edgeEdge link is represented, and assumes LK_coreIn have n bar links, LK_edgeIn have a m bar links, remember LK_coreIn i-th link Resource utilization is v_i, then LK_coreLink circuit resource utilization rate average be:

LK_coreThe standard deviation of link circuit resource utilization rate be：

Remember LK_edgeIn the resource utilization of i-th link be v_i, then LK_edgeLink circuit resource utilization rate average be:

LK_edgeThe standard deviation of link circuit resource utilization rate be：

The standard deviation of total link utilization is：

σ(Util_NET)=σ (Util_core)+σ(Util_edge)

(4) average weighted of all virtual machines postpones δ in computing cluster system_VM(S)

The virtual machine vm that note numbering is i_iCalculate node pm (the vm at place_i), the virtual machine vm for being j with numbering_jThe meter at place Operator node pm (vm_j), the time delay of network service between the two isBoth weight coefficients are empirically set to λ_i,j, then, the average weighted of whole virtual machines postpones to be represented by group system：

Wherein,, i.e. vm_i、vm_jIt is any two virtual machine in group system.

(5) computing cluster system mapping cost cost (S)

Wherein PC_vmRepresent the performance loss after virtual machine (vm) migration, PMC_vmRepresent performance damage of the virtual machine in transition process Consumption, PB_vmThe performance boost of virtual machine after migrating is represented, α, β are weight coefficients.

Group system mapping cost cost (S) is used for influence of the virtual machine (vm) migration operation to whole group system performance.

More than, σ (Util are calculated respectively_CPU)、σ(Util_MEM)、σ(Util_MEM)、δ_VM(S), after cost (S), substitute into public Formula：

Obtain load balancing desired value T (S).

In other embodiment of the present invention, other manner computational load equilibrium desired value can also be used, herein to this not It is construed as limiting.As long as can be used for representing the parameter of the load balance degree of group system, load balancing index can be used as Value.As long as the load balancing desired value calculated is more than load balancing threshold value, need to carry out load balancing with regard to explanation.The judgement is No the step of needing to carry out load balancing is optional step.

110th, management node determines to need the multiple virtual machines for carrying out load balancing, the multiple virtual machine in group system Operate in the part or all of calculate node of the multiple calculate node in the group system to distribution.

When being judged as needing to carry out load balancing, management node can determine to need to be born in group system according to strategy Carry balanced multiple virtual machines.The determination method for carrying out multiple virtual machines of load balancing is needed to have a variety of, such as：

In a kind of embodiment, the virtual machine for needing to carry out load balancing can be screened from computing resource angle.Management node The computing resource that each node can specifically be obtained uses data, such as including cpu busy percentage and memory usage, can first sieve Calculate node (i.e. computing resource load higher calculate node) of the computing resource utilization rate more than given threshold is selected, from these Select a part of virtual machine to be defined as needing the virtual machine for carrying out load balancing in the calculate node filtered out, be added to be migrated In virtual machine list.For example, from the calculate node filtered out computing resource utilization rate highest can be selected one or more Virtual machine, it is defined as needing the virtual machine for carrying out load balancing.

In another embodiment, the virtual machine for needing to carry out load balancing can be screened from Internet resources angle.Management section Point can filter out between each other according to the Internet usage data of acquisition, in the whole virtual machines run from group system Virtual machine with network traffics relation, it is defined as needing the virtual machine for carrying out load balancing, i.e.,：For the institute in group system There is virtual machine, all rejected therein with other any virtual machines without the virtual machine of network traffics relation, will be remaining virtual Machine is defined as needing the virtual machine for carrying out load balancing, is added in virtual machine list to be migrated.

In the embodiment of the present invention, it is preferred to use the combination of above two mode, to determine to need to be born in group system Balanced multiple virtual machines are carried, i.e.,：What will be screened from computing resource angle needs to carry out the virtual machine of load balancing, and from network The virtual machine for needing to carry out load balancing of resource view screening, is added in virtual machine list to be migrated, is needed as final Carry out the virtual machine of load balancing.

Certainly, in other embodiment of the present invention, can also there are other screening modes, for example, it is also possible to from storage resource angle Degree screening needs to carry out the virtual machine of load balancing, is also added in virtual machine list to be migrated.Herein for determination need into The mode of multiple virtual machines of row load balancing does not limit.

120th, management node divides the multiple virtual machine according to the network traffics relation between the multiple virtual machine For at least one virtual machine cluster so that, between at least one other virtual machine in each virtual machine and same virtual machine cluster Network traffics are more than or equal to network traffics threshold value, each virtual machine and any one other virtual machine in different virtual machine cluster it Between network traffics be less than network traffics threshold value.

In the embodiment of the present invention, virtual machine generation migration suggestion of the management node in virtual machine list to be migrated Before, according to the network traffics relation between virtual machine, the multiple virtual machines for needing load balancing can be clustered as one or more Virtual machine cluster, each virtual machine cluster include at least one virtual machine.

Described virtual machine cluster refers to one group of virtual machine with particular characteristic requirement, herein, each virtual machine with it is same The network traffics between at least one other virtual machine in one virtual machine cluster are more than or equal to network traffics threshold value, each virtual The network traffics between any one other virtual machine in machine and different virtual machine cluster are less than network traffics threshold value.

Or it is understood that：The multiple virtual machine clusters marked off can be divided into according to the number of included virtual machine Two classes, the virtual machine cluster including at least two virtual machines is referred to as the first virtual machine cluster, the virtual of virtual machine will be only included Machine cluster is referred to as the second virtual machine cluster, wherein, in any one virtual machine and same first virtual machine cluster in the first virtual machine cluster At least one other virtual machine between network traffics be more than network traffics threshold value, the only one in the second virtual machine cluster is empty The network traffics of plan machine and any one other virtual machine in the multiple virtual machine are less than network traffics threshold value.

In the embodiment of the present invention, using network traffics compatibility algorithm, it would be desirable to multiple virtual machines division of load balancing For at least one virtual machine cluster, specifically may include：The network traffics between multiple virtual machines of load balancing are carried out as needed Relation, build the network service connection topology of described multiple virtual machines；Then network service connection topology is split, will The each group of virtual machine that mutual network traffics are more than or equal to network traffics threshold value is divided into a virtual machine cluster, with And each is isolated into virtual machine and is individually determined as a virtual machine cluster, the isolated virtual machine and any one other virtual machine Network traffics be both less than network traffics threshold value.In other words, topology is connected for network service, if a network traffics threshold value, If the network traffics between two virtual machines are less than the network traffics threshold value, it is considered as between the two virtual machines and net is not present Network discharge relation, the connected relation between the two virtual machines is deleted；To all virtual machines in network service connection topology After carrying out above-mentioned processing, all virtual machines are naturally divided into several groups, and each of which group is exactly a virtual machine cluster.Together Multiple virtual machines in one virtual machine cluster may be operated in same calculate node, it is also possible to operate in different calculating sections Point on.After marking off each virtual machine cluster, the corresponding relation list of virtual machine cluster and main frame can be further generated, in list Corresponding relation between calculate node including each virtual machine in virtual machine cluster and each virtual machine host, so as to for after Continuous generation migration suggestion is prepared.

130th, management node is suggested for each virtual machine fasciation into migration, and the migration is proposed to be used in instruction will be each empty Whole virtual machines that plan machine cluster includes, are moved to or host is on same target computing nodes.

In the embodiment of the present invention, when management node generation virtual machine (vm) migration is suggested, in units of virtual machine cluster, generation is moved Move and be proposed to be used in whole virtual machines that instruction includes each virtual machine cluster, move to or host is in same target computing nodes On.In general, it may include for each virtual machine fasciation into the step of suggesting is migrated：

Multiple candidate's calculate nodes are selected for each virtual machine cluster, each virtual machine cluster of simulation calculating is migrated to or place The load balancing desired value of the group system after main candidate's calculate node in the multiple candidate's calculate node；Pin To each virtual machine cluster, a target computing nodes are determined from the multiple candidate's calculate node so that in the virtual machine cluster Whole virtual machine (vm) migrations arrive or the load balancing desired value of host's group system after selected target computing nodes is minimum； Suggest for each virtual machine fasciation into migration, the migration be proposed to be used in instruction each virtual machine cluster is included it is all virtual Machine, is moved to or host is on the target computing nodes of the selection.

In a kind of embodiment, multiple candidate's calculate nodes are selected to may include for each virtual machine cluster：Filter out calculating Resource utilization is less than at least one calculate node of computing resource load threshold as candidate's calculate node.Another embodiment party In formula, multiple candidate's calculate nodes are selected to may include for each virtual machine cluster：Assuming that virtual machine cluster is moved to or host exists One in multiple random calculate nodes, simulation calculating is carried out, after filtering out the migration of virtual machine cluster, load balancing desired value is less than At least one calculate node of load threshold, as candidate's calculate node.In other embodiment, other methods can also be used Candidate's calculate node is screened, contrasts do not limit herein.

Illustrate, it is assumed that group system includes three calculate nodes altogether, is represented respectively with A, B, C；Need to be loaded A balanced virtual machine cluster includes virtual machine VM1, VM2, VM3, VM4, VM5, and wherein VM1, VM2 is operated on calculate node A, VM4, VM5 are operated on calculate node B, and VM3 is operated on calculate node C；Candidate's calculate node can be first filtered out, such as is calculated Node B and C；Then, according to the principle for focusing on whole virtual machines in same virtual machine cluster in same calculate node, Following candidate can be generated and migrated and suggested：That is, virtual machine VM1, VM2, VM3 are moved to the candidate on calculate node B and migrated and is suggested, And the candidate moved to VM1, VM2, VM4, VM5 on calculate node C migrates and suggested；Then, management node can be directed to each Candidate migrates suggestion and carries out simulation calculating, it is assumed that after each candidate migrates suggestion execution, the load for simulating computing cluster system is equal Weighed desired value, and the minimum candidate of load balancing desired value is migrated and suggests being defined as pending migration suggestion, for example, will be virtual The candidate that machine VM1, VM2, VM3 are moved on calculate node B migrates the migration cost suggested and will may migrated than another candidate It is recommended that migration cost it is small, the load balancing desired value after migration can be smaller, therefore, calculate node B can be defined as to target meter Operator node, virtual machine VM1, VM2, VM3 can be moved to the candidate on calculate node B and migrated and suggested, be defined as pending move Move and suggest.Finally, suggested by being migrated determined by execution, it would be desirable to the virtual machine (vm) migration of migration to target computing nodes, i.e., It can be achieved to move to the partial virtual machine in high capacity calculate node in low-load calculate node, while realize will be same Multiple virtual machines in virtual machine cluster are focused in same virtual machine, so as to reduce the negative of communication link between calculate node Carry.

After management node is suggested for each virtual machine fasciation into migration, the migration for each virtual machine cluster can be built View is sent to each source calculate node of one or more virtual machine hosts included by virtual machine cluster, by each source calculate node Perform migration to suggest, whole virtual machine (vm) migrations that a virtual machine cluster includes are arrived or host is in same target computing nodes On.

It is appreciated that such scheme of the embodiment of the present invention for example can be in the computer of the management node as group system Equipment is embodied.

More than, the embodiment of the invention discloses a kind of load-balancing method of virtual machine, this method is used according to virtual machine Between network traffics relation, it would be desirable to the multiple virtual machines for carrying out load balancing are divided at least one virtual machine cluster so that, The network traffics between at least one other virtual machine in each virtual machine and same virtual machine cluster are more than or equal to network flow Threshold value is measured, the network traffics between any one other virtual machine in each virtual machine and different virtual machine cluster are less than network traffics Threshold value；Suggest for each virtual machine fasciation in the multiple virtual machine cluster into migration, the migration is proposed to be used in instruction will Whole virtual machines that each virtual machine cluster includes, are moved to or technical scheme of the host on same target computing nodes；Take Obtained following technique effect：

When carrying out load balancing, it is contemplated that the influence of network dimension, by monitoring the network flow between virtual machine in real time Magnitude relation, the virtual machine cluster with network traffics relation is marked off, as the thread for being subsequently generated virtual machine (vm) migration suggestion；

Multiple virtual machines in same virtual machine cluster can be focused in same calculate node, so as to which one virtual The flow between multiple virtual machines in machine cluster will not take the load of the link between calculate node, and cluster can be reduced with this The network load of each bar link in system, can prevent link and performance bottleneck occur；

Also, by the way that multiple virtual machines in same virtual machine cluster are focused in same calculate node, Ke Yiti Height improves the network bandwidth and communication reliability between the virtual machine in same virtual machine cluster, so as to lift traffic-intensive Virtual machine cluster network performance.

In order to preferably implement the such scheme of the embodiment of the present invention, it is also provided below and implements such scheme for coordinating Relevant apparatus.

Fig. 3 a are refer to, the embodiment of the present invention provides a kind of management node 200, applied to including multiple calculate nodes and institute State the group system of management node 200, each calculate node in the multiple calculate node includes hardware layer, operates in hardware Host and at least one virtual machine for operating on host on layer；The management node 200 may include：

Determining module 210 is described more for determining to need the multiple virtual machines for carrying out load balancing in the group system Operate in the part or all of calculate node of the multiple calculate node in the group system to individual virtual machine distribution；

Sub-clustering module 220, for according to the network traffics relation between the multiple virtual machine, by the multiple virtual machine Be divided at least one virtual machine cluster so that, at least one other virtual machine in each virtual machine and same virtual machine cluster it Between network traffics be more than or equal to network traffics threshold value, each virtual machine and any one in different virtual machine cluster are other virtual Network traffics between machine are less than network traffics threshold value；

Suggestion module 230, for suggesting for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will be every Whole virtual machines that individual virtual machine cluster includes, are moved to or host is on same target computing nodes；

Sending module 240, the source of one or more virtual machine hosts for including to the virtual machine cluster calculate section Point, which sends the migration, suggests, in order to which whole virtual machines that the virtual machine cluster includes are migrated to or host is in the target In calculate node.

In some embodiments of the invention, the sub-clustering module 220 may include：

In some embodiments of the invention, the determining module 210 can be specifically used for：That is run from the group system is complete In portion's virtual machine, the virtual machine between each other with network traffics relation is filtered out, determines described there is network flow between each other The virtual machine of magnitude relation is the empty multiple plan machines for needing to carry out load balancing in the group system.

It refer to Fig. 3 b, in some embodiments of the invention, management node 200 can also include：

Acquisition module 250, the resource for obtaining the group system use data, and the resource includes institute using data State the cpu busy percentage and memory usage of each calculate node in group system, and each bar chain between each calculate node The network resource utilization on road；

Computing module 260, for using data computational load equilibrium desired value according to the resource, the load balancing refers to Scale value is used for the load balancing state for representing the group system；

Judge module 270, if being more than load balancing threshold value for the load balancing desired value, judge the group system Need to carry out load balancing.

In some embodiments of the invention, computing module 260 can be specifically used for using following formula computational load equilibrium indexs Value；

T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET)+ω₄×δ_VM(S)；

In some embodiments of the invention, it is proposed that module 230 may include：

In some embodiments of the invention, analogue computer, it can be specifically used for referring to using following formula computational load equilibriums Scale value；

It is appreciated that the function of each functional module of the management node of the embodiment of the present invention can be according to Fig. 2 a or 2b Embodiment of the method in method specific implementation, its specific implementation process can refer to the associated description in above method embodiment, Here is omitted.

The management node of the embodiment of the present invention can be specifically computer equipment.

Therefore in some feasible embodiments of the present invention, management node can be according to the net between virtual machine Network discharge relation, it would be desirable to which the multiple virtual machines for carrying out load balancing are divided at least one multiple virtual machine clusters so that, each The network traffics between at least one other virtual machine in virtual machine and same virtual machine cluster are more than or equal to network traffics threshold It is worth, the network traffics between any one other virtual machine in each virtual machine and different virtual machine cluster are less than network traffics threshold Value；Suggest for each virtual machine fasciation in the multiple virtual machine cluster into migration, the migration is proposed to be used in instruction will be every Whole virtual machines that individual virtual machine cluster includes, are moved to or host is on same target computing nodes；Achieve following technology Effect：

Fig. 1 is refer to, the embodiment of the present invention also provides a kind of group system, and the group system includes multiple calculate nodes 320 and management node 310, management node 310 be the management node as shown in Fig. 3 a or Fig. 3 b embodiments.Wherein, calculate node 320 include hardware layer 3201, the host 3202 operated on hardware layer and operate at least one on host Virtual machine VM.The calculate node 320 is used to receive the migration suggestion that the management node 310 is sent, and is built according to the migration The instruction of view performs virtual machine (vm) migration operation.The group system 300 can be specifically data center or virtual cluster.

The group system is when carrying out load balancing, it is contemplated that the influence of network dimension, by monitor in real time virtual machine it Between network traffics relation, the virtual machine cluster with network traffics relation is marked off, as being subsequently generated virtual machine (vm) migration suggestion Thread；

For ease of being better understood from technical scheme provided in an embodiment of the present invention, below by the reality under a concrete scene Exemplified by applying mode, technical scheme of the embodiment of the present invention is introduced.

It refer to Fig. 4 a, it is assumed that virtual cluster includes four calculate nodes, has been separately operable in four calculate nodes Host host, and represented respectively with host1, host2, host3, host4；Wherein, the calculate node where host1, host2 Connected by interchanger switch2, the calculate node where host3, host4 is connected by interchanger switch3, switch2 It is connected by switch1 with switch3.Assuming that virtual machine App Server, DB Server, the Mail of four different applications Higher flow between Server, Web Server be present, while this four virtual machines are distributed in different calculate nodes again When, then, it is therefore apparent that core link switch2 to switch1 and switch1 to switch3 network load easily produces Bottleneck, while the network performance between virtual machine can also be affected.

According to the core concept of the present invention, if the virtual machine cluster with discharge relation allowed in computing resource Under migrate to same calculate node, such as shown in Fig. 4 b, by App Server, DB Server, Mail Server, Web The virtual machine with high flow relation is considered as a virtual machine cluster to Server this four between each other, all moves to Host1 On, then, the load of core link can be not only reduced, while the network performance between virtual machine can be improved.

Fig. 5 is refer to, is the schematic diagram for the data center 500 that the embodiment of the present invention is applied, the data center 500 Including management node 502 and multiple calculate nodes 501.Wherein, several calculate nodes 501 can form a cluster (Cluster) system 510 (as shown in 510a in figure or 510b), the work(for being used to manage the group system 510 of management node 502 Energy entity can be considered as a part for the group system 510, in other words, it is believed that each group system also has oneself Management node.In addition, described management node 502 can have two, a primary management node of conduct, another can be used as standby Use management node.Whole data center (is now considered as a cluster by present invention method available for whole data center System), it can be used for one of group system.

Group system is a logical concept, is made up of multiple calculate nodes, and provides some advanced work(as granularity Can, such as scheduling of resource, High Availabitity etc..Calculate node is physical host (i.e. physical server), there is provided unit virtualization, Multiple virtual machines would generally be run thereon.Management node manages whole data center, there is provided calculate node and virtual machine Life Cycle The functions such as period management, resource scheduling management, O＆M, it is the brain of whole data center.Calculate node and management node are all meters Calculate machine equipment.

As shown in Figure 6 a, it is the schematic diagram for the load balancing apparatus 600 disposed in management node 502, the load balancing fills Putting 600 may include：

Performance data collection module 601, for gathering the performance data of whole data center in real time, such as resource uses number According to, it may include the computing resource such as CPU, Memory of each calculate node use data, the computing resource of each virtual machine Such as CPU, Memory use data, the Internet usage data of each virtual machine, the network traffics between virtual machine are closed It is network load of link between each two calculate node etc., and historical data can be saved as.The performance data collection module 601 correspond to the acquisition module 250 described in Fig. 3 b embodiments.

DRS control modules 602, for other modules to be controlled and dispatched, to reach to data center or wherein The resource of virtual cluster enter Mobile state management.For example, the performance data of the collection of performance data acquisition module 601 is obtained, under Hair control instruction and corresponding performance data give other modules, instruct other modules to be handled accordingly.The DRS controls mould Block 602 corresponds to the determining module 210 described in Fig. 3 a or 3b embodiments, computing module 260 and judge module 270.

Virtual machine traffic compatibility identification module 603, for the performance number gathered according to performance data collection module 601 According to identify has all virtual machines of network traffics relation between each other in group system, be additionally operable to be born as needed Carry the network traffics relation between balanced multiple virtual machines, it would be desirable to which the multiple virtual machines for carrying out load balancing are divided at least One virtual machine cluster so that, the network flow between at least one other virtual machine in each virtual machine and same virtual machine cluster Measure and be more than or equal to network traffics threshold value, the net between any one other virtual machine in each virtual machine and different virtual machine cluster Network flow is less than network traffics threshold value.The virtual machine traffic compatibility identification module 603 corresponds to institute in Fig. 3 a or 3b embodiments The sub-clustering module 220 stated.

Load balancing suggests generation module 604, may particularly include the migration of network dimension and suggests generation module 604a and calculating Resource dimension suggests generation module 604b, wherein：Computing resource dimension suggests generation module 604b, for from computing resource angle Screening needs to carry out the virtual machine of load balancing, and generation candidate, which migrates, to suggest；Generation module 604a is suggested in the migration of network dimension, uses In the virtual machine that progress load balancing is needed from the screening of Internet resources angle, generation candidate, which migrates, to suggest；Detailed description please join Examine the description in Fig. 1 embodiments.The load balancing suggests that generation module 604 may correspond to building described in Fig. 3 a or 3b embodiments Discuss module 230.

Load balancing suggests execution module 605, migrates suggestion progress Profit Assessment to the candidate of generation, such as calculate Whether load balancing desired value is minimum or whether is less than the threshold value of setting, if by Profit Assessment, performs migration and suggests, That is, virtual machine (vm) migration operation is performed, on the target computing nodes that virtual machine (vm) migration to be migrated is specified to migration suggestion.This is negative Balanced suggestion execution module 605 is carried, corresponding to the sending module 240 described in Fig. 3 a or 3b embodiments.

As shown in Figure 6 b, be group system in the embodiment of the present invention management node 502 on the cluster resource management that runs The schematic diagram of system.The input that the performance data collection module 601 of hereinbefore load balancing apparatus 600 corresponds in Fig. 6 b Data process subsystem, the control subsystem that DRS control modules 602 correspond in Fig. 6 b, load balancing suggest generation module 604 Corresponding to the algorithm subsystem (load-balancing algorithm especially therein) in Fig. 6 b, load balancing suggests that execution module 605 is right Should be in the output subsystem in Fig. 6 b.Virtual machine traffic compatibility identification module 603, it is that the embodiment of the present invention is newly added to cluster The module of resource management system, the algorithm subsystem also corresponded in Fig. 6 b.

Therefore by disposing load balancing apparatus 600, management node 502 can implement resource tune to group system Degree management, the influence of network dimension when carrying out load balancing, can be considered, by monitoring the network flow between virtual machine in real time Magnitude relation, the virtual machine cluster with network traffics relation is marked off, as the thread for being subsequently generated virtual machine (vm) migration suggestion.

Fig. 6 a and Fig. 7 are refer to, is that the management node 502 for being deployed with load balancing apparatus 600 performs load balance scheduling Flow, exemplified by system of virtual cluster, load balance scheduling flow may include：

701：The management node 502 of virtual cluster enables the DRS functions of load balancing apparatus 600；

702：The timing of DRS control modules 602 periodically obtains performance data, example from performance data collection module 601 Such as include cluster network topology information, CPU, Memory and network performance data and parameter and rule configuration；

703：DRS control devices 602 perform load-balancing algorithm, according to CPU, Memory and network topology and bandwidth Etc. performance data, the load balancing desired value of virtual cluster is calculated；

704：DRS control devices 602 judge whether to need to carry out load balancing according to the load balancing desired value calculated, Wherein, if the load balancing desired value of cluster is less than or equal to load balancing threshold value, illustrate that cluster is at equilibrium, then tie Beam control flow, if load balancing desired value is more than load balancing threshold value, group system is in non-balanced state, it is necessary to enter Row load balancing, then enter in next step；

705：DRS control devices 602 indicate that load balancing is suggested generation module 604 and known using virtual machine traffic compatibility Other module 603, the virtual machine for needing to carry out load balancing, composition are filtered out from computing resource dimension and Internet resources dimension respectively Virtual machine list to be migrated；

706：It is affine to perform virtual machine traffic using virtual machine traffic compatibility identification module 603 for DRS control devices 602 Property recognizer, multiple virtual machine clusters to be migrated are clustered out from virtual machine list to be migrated；

707：DRS control devices 602 indicate that load balancing suggests generation module 604, and it is low to filter out computing resource utilization rate In computing resource load threshold at least one calculate node as candidate's calculate node, counted for each virtual machine cluster from candidate A target computing nodes are selected in operator node so that whole virtual machine (vm) migrations in the virtual machine cluster arrive or host is selected Target computing nodes after group system load balancing desired value it is minimum；

708：DRS control devices 602 indicate that load balancing suggests generation module 604, for each virtual machine fasciation Cheng Qian Move and suggest, migration suggests that whole virtual machine (vm) migrations of each virtual machine cluster are arrived in instruction or host calculates in same target and saved Point；

709：DRS control devices 602 indicate that load balancing suggests execution module 605, and virtual machine cluster is migrated and suggests sending To the source calculate node of each virtual machine host in virtual machine cluster, realize whole virtual machine (vm) migrations of each virtual machine cluster to same One target computing nodes；

Then, into next iteration control.In the embodiment of the present invention, management node 502 is periodically or timing is carried out Load balancing, each iteration control cycle carry out a wheel load balancing.

By above-mentioned flow, management node can be by multiple void in the same virtual machine cluster with network traffics relation Plan machine is focused in same calculate node, so as to which the network traffics between multiple virtual machines in same virtual machine cluster are not The load of the link between calculate node can be taken, the network load of each bar link in group system can be reduced with this, is prevented There is performance bottleneck in link；Also, by the way that multiple virtual machines in same virtual machine cluster are focused on into same calculate node On, the network bandwidth and communication reliability between the virtual machine improved in same virtual machine cluster can be improved, so as to carry Rise the network performance of the virtual machine cluster of traffic-intensive.

Refer to Fig. 8, in some embodiments of the invention, DRS control devices 602 perform load-balancing algorithm in step 703, Judge whether to load balancing and generate the flow of virtual machine (vm) migration suggestion, specifically may include：

801：Problem of load balancing is converted into optimization problem, unified optimization object function is established and is used for computational load Balanced desired value, optimization object function are as follows：

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) represent each calculate node cpu busy percentage variance, σ (Util_MEM) represent that the internal memory of each calculate node utilizes The variance of rate, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) The weighted delay of virtual machine is represented, cost (S) represents group system mapping cost, ω₁、ω₂、ω₃、ω₄、ω₅It is weighting respectively Coefficient.

802：By timer or manual triggering collection properties data, calculated using above-mentioned optimization object function negative Carry balanced desired value；

803：When judging that load balancing desired value is less than load balancing threshold value, load balance scheduling is exited, otherwise, is judged as Need to carry out load balancing, continue executing with following steps；

804：Need to carry out the virtual machine of load balancing from the selection of computing resource (such as CPU, Memory) dimension, be added to In virtual machine list to be migrated；

805：Need to carry out the virtual machine of load balancing from the selection of Internet resources dimension, be added to virtual machine list to be migrated In；

806：Virtual machine list to be migrated is traveled through, it is one or more using virtual machine traffic compatibility recognizer, division Virtual machine cluster, according to optimization object function, to virtual machine cluster selection target calculate node；

807：Selection migration back loading equilibrium desired value minimum virtual machine cluster and target computing nodes, are generated pending Virtual machine (vm) migration suggestion.

It refer to Fig. 9, in some embodiments of the invention, virtual machine list to be migrated traveled through in step 806, utilizes virtual machine Flow compatibility recognizer, the flow of one or more virtual machine clusters is divided, specifically may include：

S1：According to the network traffics relation between virtual machine, the logical of all virtual machines in virtual machine list to be migrated is constructed Believe matrix；

S2：It is each in figure shown in virtual machine constructor communication connection figure, such as Figure 10 (a) according to the communication matrix of virtual machine Circle represents a virtual machine VM, is connected between two virtual machines with network traffics relation with line segment, the numeral on line segment For characterizing the size of flow；

S3：Virtual machine traffic compatibility identification problem is converted into weighted graph segmentation problem；

S4：Segmentation threshold is set, line segment of the weights in section [0, h] is cut, successively to virtual machine communication connection Figure is split；For example, set interval [0,5], line clipping flow value being located between section [0,5] removes, and only retains Flow value is more than 5 line segment, as shown in Figure 10 (b), obtains four virtual machine clusters (Cluster), wherein Cluster1 includes void Plan machine VM1, VM2, VM3, VM5, VM6, Cluster2, which includes virtual machine VM4, Cluster3, includes virtual machine VM9, VM10, Cluster4 includes virtual machine VM7, VM8, VM11, VM12.

S5：According to virtual machine and the attaching relation of calculate node, secondary splitting is carried out to the communication connection figure of virtual machine, such as Shown in Figure 10 (c), the corresponding relation list of virtual machine cluster and calculate node is generated.

Above-mentioned flow describes a kind of method of partition virtual machines cluster, this method by building virtual machine communication connection figure, Virtual machine communication connection figure is split, can be achieved the multiple virtual machines for needing equally loaded being divided into multiple virtual machines Cluster.

Subsequently, management node can is according to the virtual machine cluster and virtual machine cluster and the corresponding relation of calculate node of division List, generation virtual machine (vm) migration suggested.For example, it is directed to virtual machine cluster Cluster1, its virtual machine VM1, VM2, VM3 for including Operate on calculate node Host1, virtual machine VM5, VM6 are operated on calculate node Host3, then, can be generated：By virtual machine The candidate that VM1, VM2, VM3 are moved on calculate node Host3, which migrates, to suggest, or, virtual machine VM5, VM6 are moved into calculating Candidate on node Host1, which migrates, to suggest, or, VM1, VM2, VM3, VM4, VM5 are moved into another calculate node for example Candidate on Host2, which migrates, to suggest.

To sum up, exemplified by by the embodiment under a concrete scene, technical scheme of the embodiment of the present invention is situated between Continue, it is unclear that part refers to the description of other parts above.

Therefore in some feasible embodiments of the present invention, when carrying out load balancing, it is contemplated that network is tieed up The influence of degree, by the way that multiple virtual machines in the same virtual machine cluster with network traffics relation are focused on into same calculating On node, so as to which the network traffics between multiple virtual machines in same virtual machine cluster will not be taken between calculate node The load of link, the network load of each bar link in group system can be reduced with this, and performance bottleneck occurs in prevention link；And And by the way that multiple virtual machines in same virtual machine cluster are focused in same calculate node, it is same that raising can be improved The network bandwidth and communication reliability between virtual machine in individual virtual machine cluster, so as to lift the virtual machine cluster of traffic-intensive Network performance.

The embodiment of the present invention also provides a kind of computer-readable storage medium, and the computer-readable storage medium can have program stored therein, should The portion of load-balancing method including the virtual machine described in the above-mentioned embodiment of the method as shown in Fig. 2 a or 2b when program performs Point or Overall Steps.

Figure 11 is refer to, the embodiment of the present invention also provides a kind of management node 900.

The management node 900 of the embodiment of the present invention can be applied to the collection for including the management node 900 and multiple calculate nodes Group's system, and as the management node in the group system, it should be appreciated that calculate node in the group system can be with Including processor and memory；In some embodiments, the memory storage that the calculate node in the group system includes Following element, it can perform module either data structure or their subset, or their superset:

Host:As management level, to complete the management of hardware resource, distribution；Virtual hardware is presented for virtual machine to put down Platform；Realize the scheduling and isolation of virtual machine.Wherein, Host is probably monitor of virtual machine (VMM)；In addition, VMM and 1 sometimes Privileged virtual machine coordinates, and both combine composition Host.Wherein, virtual hardware platform is provided each virtual machine run thereon each Kind hardware resource, such as provides virtual processor, internal memory, virtual disk, Microsoft Loopback Adapter.Wherein, the virtual disk can correspond to A Host file or a logic block device.Virtual machine is then operated on the virtual hardware platform that Host is its preparation, One or more virtual machines are run on Host.

One or more virtual machines：One or more can be simulated on a physical computer by software virtual machine The virtual computer of platform, and these virtual machines are operated just as real computer, can be with installation operation on virtual machine System and application program, virtual machine may also access Internet resources.For the application program run in virtual machine, virtual machine It is like to be operated in real computer.

The management node 900 of the embodiment of the present invention may include that input equipment 901 (optional), output equipment 904 are (optional ), processor 902 and memory 903.Wherein, memory 903 can include read-only storage and random access memory, and Instruction and data is provided to processor 902.The a part of of memory 903 can also include nonvolatile RAM (NVRAM)。

Memory 903 stores following element, can perform module either data structure or their subset, or Their superset:

Operational order：Including various operational orders, for realizing various operations.

Operating system：Including various system programs, for realizing various basic businesses and the hardware based task of processing.

In embodiments of the present invention, processor 902 is by calling operational order (operational order that memory 903 stores It is storable in operating system), perform following operation：Determine to need to carry out the multiple virtual of load balancing in the group system Machine, the part or all of calculating of the multiple calculate node operated in the group system to the multiple virtual machine distribution On node；According to the network traffics relation between the multiple virtual machine, the multiple virtual machine is divided at least one void Plan machine cluster so that, the network traffics between at least one other virtual machine in each virtual machine and same virtual machine cluster are more than Or equal to network traffics threshold value, the network traffics between any one other virtual machine in each virtual machine and different virtual machine cluster Less than network traffics threshold value；Suggest for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will be each virtual Whole virtual machines that machine cluster includes, are moved to or host is on same target computing nodes；And wrapped into the virtual machine cluster The source calculate node of the one or more virtual machine hosts included sends the migration and suggested, includes in order to the virtual machine cluster Whole virtual machines are migrated to or host is on the target computing nodes.

In the embodiment of the present invention, when carrying out load balancing, it is contemplated that the influence of network dimension, by that will have network flow Multiple virtual machines in the same virtual machine cluster of magnitude relation are focused in same calculate node, so as to same virtual machine The network traffics between multiple virtual machines in cluster will not take the load of the link between calculate node, and collection can be reduced with this There is performance bottleneck in the network load of each bar link in group's system, prevention link；Also, by by same virtual machine cluster Multiple virtual machines are focused in same calculate node, can improve the net between the virtual machine improved in same virtual machine cluster Network bandwidth and communication reliability, so as to lift the network performance of the virtual machine cluster of traffic-intensive.

The operation of the control computer equipment 900 of processor 902, processor 902 can also be referred to as CPU (Central Processing Unit, CPU).Memory 903 can include read-only storage and random access memory, and Instruction and data is provided to processor 902.The a part of of memory 903 can also include nonvolatile RAM (NVRAM).In specific application, each component of gateway 600 is coupled by bus system 905, wherein bus system 905 in addition to including data/address bus, can also include power bus, controlling bus and status signal bus in addition etc..But in order to clear For the sake of Chu's explanation, various buses are all designated as bus system 905 in figure.

The method that the embodiments of the present invention disclose can apply in processor 902, or be realized by processor 902. Processor 902 is probably a kind of IC chip, has the disposal ability of signal.In implementation process, the above method it is each Step can be completed by the integrated logic circuit of the hardware in processor 902 or the instruction of software form.Above-mentioned processing Device 902 can be general processor, digital signal processor (DSP), application specific integrated circuit (ASIC), ready-made programmable gate array (FPGA) either other PLDs, discrete gate or transistor logic, discrete hardware components.Can realize or Person performs disclosed each method, step and the logic diagram in the embodiment of the present invention.General processor can be microprocessor or Person's processor can also be any conventional processor etc..The step of method with reference to disclosed in the embodiment of the present invention, can be straight Connect and be presented as that hardware decoding processor performs completion, or performed with the hardware in decoding processor and software module combination Into.Software module can be located at random access memory, and flash memory, read-only storage, programmable read only memory or electrically-erasable can In the ripe storage medium in this areas such as programmable memory, register.The storage medium is located at memory 903, and processor 902 is read Information in access to memory 903, with reference to the step of its hardware completion above method.

Alternatively, according to the network traffics relation between the multiple virtual machine, the multiple virtual machine is divided into The aspect of at least one virtual machine cluster, the processor 902 are specifically used for：According to the network traffics between the multiple virtual machine Relation, build the network service connection topology of the multiple virtual machine；Network service connection topology is split, by phase The each group of virtual machine that network traffics between mutually are more than or equal to network traffics threshold value is defined as a virtual machine cluster, and, Each is isolated into virtual machine to be individually determined as a virtual machine cluster, the net of the isolated virtual machine and any one other virtual machine Network flow is both less than network traffics threshold value.

Alternatively, in terms of it is determined that needing to carry out multiple virtual machines of load balancing in the group system, the place Reason device 902 is specifically used in whole virtual machines for being run from the group system, and filter out has network traffics between each other The virtual machine of relation, determine that the virtual machine with network traffics relation between each other is to need to carry out in the group system Multiple virtual machines of load balancing.

Alternatively, the processor 902 determine to need in the group system to carry out load balancing multiple virtual machines it Before, it is additionally operable to：The resource for obtaining the group system uses data, and the resource is included each in the group system using data The cpu busy percentage and memory usage of individual calculate node, and the Internet resources profit of each bar link between each calculate node With rate；Data computational load equilibrium desired value is used according to the resource, the load balancing desired value is used to represent the collection The load balancing state of group's system；If the load balancing desired value is more than load balancing threshold value, judge that the group system needs Carry out load balancing.

Alternatively, in terms of data computational load equilibrium desired value is used according to the resource, the processor 902 has Body is used for：

Using following formula computational load equilibrium desired values；

Alternatively, suggest for each virtual machine fasciation into migration, the migration is proposed to be used in instruction by each virtual machine Whole virtual machines that cluster includes, are moved to or host is on same target computing nodes, and the processor 902 is specifically used for： Multiple candidate's calculate nodes are selected for each virtual machine cluster, each virtual machine cluster of simulation calculating is migrated to or host is at one The load balancing desired value of the group system after candidate's calculate node；For each virtual machine cluster, counted from the multiple candidate A target computing nodes are determined in operator node so that whole virtual machine (vm) migrations in each virtual machine cluster arrive or host is selected The load balancing desired value of the group system is minimum after the target computing nodes selected；Built for each virtual machine fasciation into migration View, the migration are proposed to be used in whole virtual machines that instruction includes each virtual machine cluster, move to or host is in the selection Target computing nodes on.

Alternatively, each virtual machine cluster is calculated in simulation to be migrated to or host's collection after candidate's calculate node The aspect of the load balancing desired value of group's system, the processor 902 are specifically used for：

Using following formula computational load equilibrium desired values；

Therefore in some embodiments of the invention, management node, applied to including this management node and multiple calculating The group system of node, can be according to the network traffics relation between virtual machine in group system, it would be desirable to carry out load balancing Multiple virtual machines are divided at least one virtual machine cluster so that, in each virtual machine and same virtual machine cluster it is at least one its Network traffics between its virtual machine are more than or equal to network traffics threshold value, each virtual machine and any in different virtual machine cluster Network traffics between individual other virtual machines are less than network traffics threshold value；For each virtual machine in the multiple virtual machine cluster Fasciation suggests that the migration is proposed to be used in instruction whole virtual machines for including each virtual machine cluster into migration, moves to or place Master is on same target computing nodes；Achieve following technique effect：

In the above-described embodiments, the description to each embodiment all emphasizes particularly on different fields, and is not described in some embodiment Part, may refer to the associated description of other embodiments.

It should be noted that for foregoing each method embodiment, in order to be briefly described, therefore it is all expressed as a series of Combination of actions, but those skilled in the art should know, the present invention is not limited by described sequence of movement because according to According to the present invention, some steps can use other orders or carry out simultaneously.Secondly, those skilled in the art should also know, Embodiment described in this description belongs to preferred embodiment, and not necessarily the present invention must for involved action and module Must.

One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is can To instruct the hardware (such as processor) of correlation to complete by program, the program can be stored in a computer-readable storage In medium, storage medium can include：ROM, RAM, disk or CD etc..

The load-balancing method and relevant device of a kind of virtual machine provided above the embodiment of the present invention and cluster system System is described in detail, and specific case used herein is set forth to the principle and embodiment of the present invention, the above The explanation of embodiment is only intended to help the method and its core concept for understanding the present invention；Meanwhile for the general skill of this area Art personnel, according to the thought of the present invention, there will be changes in specific embodiments and applications, in summary, this Description should not be construed as limiting the invention.

Claims

1. a kind of load-balancing method of virtual machine, it is characterised in that methods described is used to include management node and multiple calculating The group system of node, each calculate node in the multiple calculate node include hardware layer, operate in the hardware layer it On host and the virtual machine that operates on the host；

Methods described includes：

The management node determines to need the multiple virtual machines for carrying out load balancing, the multiple virtual machine in the group system Operate in the part or all of calculate node of the multiple calculate node in the group system to distribution；

According to the network traffics relation between the multiple virtual machine, the multiple virtual machine is divided at least one virtual machine Cluster so that, the network traffics between at least one other virtual machine in each virtual machine and same virtual machine cluster are more than or waited The network traffics between any one other virtual machine in network traffics threshold value, each virtual machine and different virtual machine cluster are less than Network traffics threshold value；

Suggest for each virtual machine fasciation into migration, the migration is proposed to be used in the whole that instruction includes each virtual machine cluster Virtual machine, is moved to or host is on same target computing nodes；

The source calculate node of the one or more virtual machine hosts included to the virtual machine cluster sends the migration and suggested, with Whole virtual machines that being easy to the virtual machine cluster includes are migrated to or host is on the target computing nodes；

Wherein, the network traffics relation according between the multiple virtual machine, the multiple virtual machine is divided at least One virtual machine cluster includes：

According to the network traffics relation between the multiple virtual machine, the network service connection for building the multiple virtual machine is opened up Flutter；

Network service connection topology is split, mutual network traffics are more than or equal to network traffics threshold value Each group of virtual machine be defined as a virtual machine cluster, and, each is isolated into virtual machine and is individually determined as a virtual machine Cluster, the network traffics of the isolated virtual machine and any one other virtual machine are both less than network traffics threshold value.

2. according to the method for claim 1, it is characterised in that the management node determine to need in the group system into Multiple virtual machines of row load balancing include：

In the whole virtual machines run from the group system, filter out virtual with network traffics relation between each other Machine, it is to need to carry out load balancing in the group system to determine the virtual machine with network traffics relation between each other Multiple virtual machines.

3. according to any described method in claim 1 to 2, it is characterised in that the management node determines the cluster system Before multiple virtual machines that progress load balancing is needed in system, in addition to：

The resource that the management node obtains the group system uses data, and the resource includes the cluster system using data The cpu busy percentage and memory usage of each calculate node in system, and the network of each bar link between each calculate node Resource utilization；

Data computational load equilibrium desired value is used according to the resource, the load balancing desired value is used to represent the cluster The load balancing state of system；

If the load balancing desired value is more than load balancing threshold value, judge that the group system needs to carry out load balancing.

4. according to the method for claim 3, it is characterised in that described balanced using data computational load according to the resource Desired value includes：

Using following formula computational load equilibrium desired values；

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) Represent the variance of the cpu busy percentage of each calculate node, σ (Util_MEM) represent each calculate node memory usage side Difference, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) institute is represented State the average weighted delay of all virtual machines run in group system, ω₁、ω₂、ω₃、ω₄It is weight coefficient respectively.

5. according to any described method in claim 1 to 2, it is characterised in that described to be directed to each virtual machine fasciation Cheng Qian Move and suggest, the migration is proposed to be used in whole virtual machines that instruction includes each virtual machine cluster, moves to or host is same Include on individual target computing nodes：

Multiple candidate's calculate nodes are selected for each virtual machine cluster, each virtual machine cluster of simulation calculating is migrated to or host exists The load balancing desired value of the group system after candidate's calculate node in the multiple candidate's calculate node；

For each virtual machine cluster, a target computing nodes are determined from the multiple candidate's calculate node so that Mei Gexu Whole virtual machine (vm) migrations in plan machine cluster arrive or the load of host's group system after selected target computing nodes is equal The desired value that weighs is minimum；

Suggest for each virtual machine fasciation into migration, the migration is proposed to be used in the whole that instruction includes each virtual machine cluster Virtual machine, is moved to or host is on the target computing nodes of the selection.

6. according to the method for claim 5, it is characterised in that each virtual machine cluster of the simulation calculating is migrated to or place The load balancing desired value bag of the group system after main candidate's calculate node in the multiple candidate's calculate node Include：

Using following formula computational load equilibrium desired values；

T (S)=ω₁×σ(Util_CPU)+ω₂×σ(Util_MEM)+ω₃×σ(Util_NET)+ω₄×δ_VM(S)+ω₅×cos t(S)

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) Represent the variance of the cpu busy percentage of each calculate node, σ (Util_MEM) represent each calculate node memory usage side Difference, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) institute is represented The average weighted delay of all virtual machines run in group system is stated, cos t (S) represent group system mapping cost, ω₁、 ω₂、ω₃、ω₄、ω₅It is weight coefficient respectively.

7. a kind of management node, it is characterised in that for the group system including multiple calculate nodes and the management node, institute The each calculate node stated in multiple calculate nodes includes hardware layer, the host operated on the hardware layer, Yi Jiyun Virtual machine of the row on the host；The management node includes：

Determining module, it is the multiple virtual for determining to need the multiple virtual machines for carrying out load balancing in the group system Operate in the part or all of calculate node of the multiple calculate node in the group system to machine distribution；

Sub-clustering module, for according to the network traffics relation between the multiple virtual machine, the multiple virtual machine to be divided into At least one virtual machine cluster so that, the net between at least one other virtual machine in each virtual machine and same virtual machine cluster Network flow is more than or equal to network traffics threshold value, between any one other virtual machine in each virtual machine and different virtual machine cluster Network traffics be less than network traffics threshold value；

Suggestion module, for suggesting for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will be each virtual Whole virtual machines that machine cluster includes, are moved to or host is on same target computing nodes；

Sending module, the source calculate node of one or more virtual machine hosts for including to the virtual machine cluster send institute State migration to suggest, in order to which whole virtual machines that the virtual machine cluster includes are migrated to or host is in the target computing nodes On；

Wherein, the sub-clustering module includes：

Construction unit, for according to the network traffics relation between the multiple virtual machine, building the net of the multiple virtual machine Network communication connection topology；

Division unit, for splitting to the network service connected graph, mutual network traffics are more than or equal to Each group of virtual machine of network traffics threshold value is divided into a virtual machine cluster, and, each is isolated into virtual machine and is individually determined For a virtual machine cluster, the network traffics of the isolated virtual machine and any one other virtual machine are both less than network traffics threshold value.

8. management node according to claim 7, it is characterised in that the determining module is specifically used for：From the cluster In the whole virtual machines run in system, the virtual machine between each other with network traffics relation is filtered out, is determined described mutual Between have network traffics relation virtual machine be the group system in need carry out load balancing empty multiple plan machines.

9. according to any described management node in claim 7 to 8, it is characterised in that also include：

Acquisition module, the resource for obtaining the group system use data, and the resource includes the cluster using data The cpu busy percentage and memory usage of each calculate node in system, and the net of each bar link between each calculate node Network resource utilization；

Computing module, for using data computational load equilibrium desired value according to the resource, the load balancing desired value is used In the load balancing state for representing the group system；

Judge module, if being more than load balancing threshold value for the load balancing desired value, judge the group system need into Row load balancing.

10. management node according to claim 9, it is characterised in that the computing module is specifically used for using following public affairs Formula computational load equilibrium desired value；

11. according to any described management node in claim 7 to 8, it is characterised in that the suggestion module includes：

Analogue computer, for selecting multiple candidate's calculate nodes for each virtual machine cluster, simulation calculates each virtual machine Cluster be migrated to or a candidate calculate node of the host in the multiple candidate's calculate node after the group system it is negative Carry balanced desired value；

Determining unit, for for each virtual machine cluster, determining that a target calculates section from the multiple candidate's calculate node Point so that whole virtual machine (vm) migrations in each virtual machine cluster arrive or host's cluster after selected target computing nodes The load balancing desired value of system is minimum；

It is recommended that generation unit, for suggesting for each virtual machine fasciation into migration, the migration is proposed to be used in instruction will be each Whole virtual machines that virtual machine cluster includes, move to or host the selection target computing nodes.

12. management node according to claim 11, it is characterised in that the analogue computer is specifically used for：Using Following formula computational load equilibrium desired values；

Wherein, S represents the Topological Mapping relation of virtual machine and calculate node, and T (S) represents load balancing desired value, σ (Util_CPU) Represent the variance of the cpu busy percentage of each calculate node, σ (Util_MEM) represent each calculate node memory usage side Difference, σ (Util_NET) represent each bar link between each calculate node network resource utilization variance, δ_VM(S) institute is represented The average weighted delay of all virtual machines run in group system is stated, cos t (S) represent system mapping cost, ω₁、ω₂、 ω₃、ω₄、ω₅It is weight coefficient respectively.

13. a kind of group system, it is characterised in that including multiple calculate nodes and as described in any in claim 7-12 Management node；

The calculate node is used to receive the migration suggestion that the management node is sent, and is performed according to the instruction that the migration is suggested Virtual machine (vm) migration operates.