CN113204429A - Resource scheduling method and system of data center, scheduling equipment and medium - Google Patents
Resource scheduling method and system of data center, scheduling equipment and medium Download PDFInfo
- Publication number
- CN113204429A CN113204429A CN202110597755.4A CN202110597755A CN113204429A CN 113204429 A CN113204429 A CN 113204429A CN 202110597755 A CN202110597755 A CN 202110597755A CN 113204429 A CN113204429 A CN 113204429A
- Authority
- CN
- China
- Prior art keywords
- server
- energy consumption
- data center
- servers
- resource scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000005265 energy consumption Methods 0.000 claims abstract description 111
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 37
- 238000004364 calculation method Methods 0.000 claims abstract description 25
- 230000002068 genetic effect Effects 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 238000013468 resource allocation Methods 0.000 claims description 7
- 210000000349 chromosome Anatomy 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 230000035772 mutation Effects 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000008901 benefit Effects 0.000 description 6
- 241000254158 Lampyridae Species 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000002922 simulated annealing Methods 0.000 description 2
- 241000257303 Hymenoptera Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002431 foraging effect Effects 0.000 description 1
- 230000017525 heat dissipation Effects 0.000 description 1
- 238000009396 hybridization Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Genetics & Genomics (AREA)
- Physiology (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a resource scheduling method and system, scheduling equipment and a medium of a data center, which comprise the following steps: acquiring energy consumption data of each server of a data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server; calculating according to the energy consumption data and a preset energy consumption calculation model to obtain an energy consumption value of each server; and calculating by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server to find an optimal resource scheduling scheme, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme. By the method and the device, the resource scheduling of the cloud computing data center is carried out, and the energy consumption of the server of the cloud computing data center can be reduced.
Description
Technical Field
The invention relates to the technical field of machine room monitoring, in particular to a resource scheduling method and system of a data center, scheduling equipment and a computer readable storage medium.
Background
Cloud computing is an emerging commercialization model for allocating computing resources and storage resources on demand, and has been developed on a large scale and widely used in view of its advantages such as low cost, virtualization technical support, diversified services, and the like. The computer cluster of the cloud computing data center stores cloud computing software and hardware resources and has perfect application service and specialized management; in the face of the ever-increasing scale and dynamic change requirements of cloud computing data centers, how to dynamically manage and timely and effectively allocate virtual resources to users in need is a problem that must be solved.
At present, the resource scheduling process of a cloud computing data center mainly considers the problem of resource utilization rate, in the scheduling process, when a virtual machine monitor monitors that a certain server is overloaded, a virtual machine meeting a certain requirement is directly migrated, and when the resource utilization rate of the certain server is too low, the virtual machine is also directly migrated, and the server is shut down to reduce energy consumption. Although the resource scheduling process can perform virtual machine migration quickly according to the resource utilization rate and is fast in response, the scheduling process does not consider the energy consumption caused by migration from the global perspective, and selects direct closing of the servers needing to be closed, and the servers may be frequently opened or closed, which causes more energy consumption.
Disclosure of Invention
The invention aims to provide a resource scheduling method and system of a data center, scheduling equipment and a computer readable storage medium, which are used for scheduling resources of the cloud computing data center and reducing the energy consumption of a server of the cloud computing data center.
In order to achieve the above object, an embodiment of the present invention provides a resource scheduling method for a data center, including the following steps:
step S1, acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
step S2, calculating to obtain the energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model;
and step S3, calculating and searching an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
Optionally, the energy consumption calculation model is expressed by the following expression:
E＝ECPU+EMem+Ekeep
wherein E iskeepRepresenting the value of the energy consumption, k, generated by the server remaining in the ready state1Representing the ratio of the power of the server in the ready state to its full power, PCPUIndicating the operating power, P, of the server CPUMemIndicating the operating power of the server memory, PmaxIndicating the operating power of the server in full load, UCPU(t) the profit of the server CPU at time tRate of utilization, UMem(t) represents the utilization rate of the server memory at time t, and t1 and t2 are the initial time and the end time when the server maintains the ready state.
Optionally, the step S3 includes the following sub-steps:
s31, coding the architecture of the data center by adopting a tree structure to establish a population; the root node of the tree structure represents a predefined data center controller, the second layer of nodes represent N servers, the leaf nodes represent M virtual machines and represent chromosomes, N is greater than 0, and M is greater than 0;
s32, setting the population scale, and generating an initial population based on a spanning tree algorithm;
step S33, calculating the number of servers to be started according to the virtual machines distributed according to actual needs;
step S34, if the number N of the servers started in the process of initializing the population is equal to the number N' of the servers to be started, selecting operation, cross operation and mutation operation of genetic algorithm are carried out on the mapping relation of the N servers and the corresponding virtual machines in the tree structure, and the individuals meeting the requirements are added into the set { S*And step S35 is executed; if the number N of servers started in the process of initializing the population is greater than the number N' of servers that should be started, executing step S36;
step S35, judging the set { S ] of the step S34*Whether the preset calculation termination condition is met or not is judged, and if the preset calculation termination condition is met, the set { S } is respectively calculated*Energy consumption of each allocation scheme in the station; the allocation scheme with the lowest energy consumption is the optimal resource allocation scheme, and virtual machine scheduling is carried out according to the optimal resource allocation scheme;
step S36, sorting the servers according to the number of the placed virtual machines from a small number to a large number, sequentially adding all the virtual machines on the first Δ N servers to the server with the largest current load value among the N 'servers that should be started, so that N is equal to N', and then returning to execute step S34 to recalculate; where Δ N ═ N-N'.
Optionally, the step S33 specifically calculates the number N' of servers that should be turned on according to the following expression:
wherein the molecular terms
The load sum of all the virtual machines is represented, j represents the jth virtual machine, the denominator rho represents the load threshold of one server, and tau represents a preset fine adjustment constant.
Optionally, the step S36 further includes:
and judging the closing time of the no-load server according to a preset time threshold, and closing the no-load server when the duration of the ready state of the no-load server reaches the time threshold.
Optionally, the time threshold is a time when a sum of energy consumptions generated by turning on and off the server is equal to an energy consumption generated by the server being idle but in a ready state.
Optionally, the population size set in step S32 is 50-200.
A second aspect of the present invention provides a resource scheduling system for a data center, including:
the data acquisition module is used for acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
the energy consumption calculation module is used for calculating to obtain an energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model; and
and the analysis scheduling module is used for calculating and searching an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
A third aspect of the present invention provides a scheduling apparatus, including:
a memory for storing a computer program; and
a processor for implementing the steps of the method for resource scheduling in a data center according to the first aspect when executing the computer program.
A fourth aspect of the present invention proposes a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method for resource scheduling in a data center according to the first aspect.
The resource scheduling method and system, the scheduling device and the computer readable storage medium of the data center have the following beneficial effects:
the energy consumption data of each server of the data center are obtained, the energy consumption value of each server is obtained through an energy consumption calculation model based on the energy consumption data, an optimal mapping scheme of a virtual machine and the servers is found through a genetic algorithm based on the resource utilization rate and the energy consumption value of each server, the load balancing effect is good, the resources of each server are fully utilized, meanwhile, the energy consumption is reduced to a certain degree, the energy consumption problem and the resource utilization rate problem are balanced, and the economic and performance benefits of resource scheduling of the cloud computing data center are improved.
Additional features and advantages of the invention will be set forth in the description which follows.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a resource scheduling method of a data center according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a framework of a resource scheduling system of a data center according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of a framework structure of a scheduling device according to an embodiment of the present invention.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In addition, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present invention. It will be understood by those skilled in the art that the present invention may be practiced without some of these specific details. In some instances, well known means have not been described in detail so as not to obscure the present invention.
Referring to fig. 1, an embodiment of the present invention provides a resource scheduling method for a data center, including the following steps:
step S1, acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
specifically, the energy consumption data of the server mainly includes performance data of each component of the server and resource utilization rate of the server. The energy consumption data of the server can be acquired in two modes, namely direct measurement and indirect measurement. The direct measurement mode is as follows: the energy consumption measuring equipment is installed on each component (such as a CPU, an internal memory, a disk storage, a network interface, a heat dissipation assembly and the like) of the server, and the measuring method is accurate and good in real-time performance, but needs a large amount of additional equipment and is not suitable for being applied to a large computer cluster. Therefore, the invention preferably adopts an indirect measurement mode, and specifically comprises the following steps: the embedded sensor is installed in the server to measure the energy consumption data in real time, the server usually adopts IPMI (Intelligent Platform Management Interface) and ACPI (Advanced Configuration and Power Management Interface) Interface specification, the performance data and the energy data of each component can be processed in a centralized way after being collected, and the interfaces can support the Intelligent ammeter to monitor the energy consumption of the components such as CPU, memory, disk storage and the like in real time and send the energy consumption data to the energy consumption measuring Platform through the network link characteristic of the Intelligent ammeter;
step S2, calculating to obtain the energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model;
and step S3, calculating and searching an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
According to the implementation method, the energy consumption data of each server of the data center is obtained, the energy consumption value of each server is obtained through the energy consumption calculation model based on the energy consumption data, the optimal mapping scheme of the virtual machine and the server is found through the genetic algorithm based on the resource utilization rate and the energy consumption value of each server, the load balancing effect is good, the resources of each server are fully utilized, meanwhile, the energy consumption is reduced to a certain degree, the energy consumption problem and the resource utilization rate problem are balanced, and the economic and performance benefits of resource scheduling of the cloud computing data center are improved.
In one embodiment, the energy consumption of the server at that time or the predicted energy consumption next can be calculated by using a relation model of the resource utilization rate and the energy consumption value, and in the current conventional energy consumption prediction model, the total energy consumption of each server can be expressed as:
E＝ECPU+EMem+EDisk+EMainboard+ENIC
wherein E isCPURepresenting the power consumption generated by the CPU, EMemIndicating the power consumption of the memory generation, EDiskRepresenting the power consumption generated by the disk, EMainbordRepresenting the power consumption generated by the motherboard, ENICRepresenting the power consumption generated by the network card. However, in the actual calculation process, it is difficult to calculate the energy consumption values generated by the disk, the motherboard and the network card, and the model does not consider the energy consumption generated by the server in the ready state, so that the accuracy of the calculation result is poor;
in the embodiment of the invention, the CPU and the memory are considered as main parts of energy consumption generated by the server, so in order to simplify the model, the embodiment of the invention mainly adoptsThe energy consumption of the CPU and the memory is considered. In addition, the conventional method for calculating the energy consumption of the CPU and the energy consumption of the memory directly adopts a linear energy consumption model for calculation, but under the current novel software and hardware environment, the simple linear energy consumption model cannot obtain an accurate calculation result. Thus, embodiments of the present invention employ a non-linear model, specifically, a server at time (t)1，t2) The internal energy consumption calculation model is shown by the following expression:
E＝ECPU+EMem+Ekeep
wherein E iskeepRepresenting the value of the energy consumption, k, generated by the server remaining in the ready state1Representing the ratio of the power of the server in the ready state to its full power, typically 60%, PCPUIndicating the operating power, P, of the server CPUMemIndicating the operating power of the server memory, PmaxIndicating the operating power of the server in full load, UCPU(t) represents the utilization rate of the server CPU at time t, UMem(t) represents the utilization rate of the server memory at time t, and t1 and t2 are the initial time and the end time when the server maintains the ready state.
In a data center, a resource scheduling problem is a process of solving a mapping scheme from a virtual machine to a server which meets a target, and a heuristic algorithm is needed to solve and realize, wherein the heuristic algorithm is a key technology for whether the cloud computing data center can carry out effective resource scheduling. The heuristic algorithm evaluates according to each state in the state space, and selects the state which is most consistent with the target condition according to the evaluation value to determine the final position point, thereby avoiding the heavy workload of directly searching a large number of position points. The state space search process refers to a process of finding a path from an initial state to an end state according to a certain rule. The path has various uncertainties, is long or short, has branches or is only one, is a straight line or a curve, and the common search algorithm, such as a depth-first algorithm and a breadth-first algorithm, has poor effect when the state space is uncertain.
For heuristic algorithms, currently, a simulated annealing algorithm, a firefly algorithm, an ant colony algorithm, a genetic algorithm and the like are mainly used, and the latter two algorithms are also intelligent algorithms for solving the optimization problem. Each algorithm has its own solving method and advantages, for example, the simulated annealing algorithm combines probability characteristics to search a local optimal solution according to an objective function and obtain a global optimal solution; searching a position which enables the brightness degree to be maximum according to the relation between the brightness degree of the firefly and the fitness of the position of the firefly in a firefly algorithm, namely a local optimal solution; the ant colony algorithm simulates parallel, feedback (positive and negative feedback) and cooperative modes in the foraging activity of ants, searches for the optimal solution in a solution space, and has good adaptability and solution finding capability; the genetic algorithm has various coding modes, objects similar to tree and graph structures can be coded to simplify the complex problems, the solution evaluation is carried out according to a fitness function, the search direction is determined in a probability mode in a self-adaptive mode, and information of too much solution space does not need to be searched, so that the genetic algorithm is more convenient to realize. Therefore, the embodiment of the present invention improves the genetic algorithm to solve the optimal resource scheduling scheme in view of its advantages, that is, step S3 of the present embodiment preferably includes the following sub-steps:
s31, coding the architecture of the data center by adopting a tree structure to establish a population; the root node of the tree structure represents a predefined data center controller, the second layer of nodes represent N servers, the leaf nodes represent M virtual machines and represent chromosomes, N is greater than 0, and M is greater than 0;
specifically, tree coding utilizes a tree data structure, each virtual machine corresponds to a tree and is represented as a chromosome, each chromosome is a unique solution in a solution space, and each locus corresponds to one node, so that a genetic algorithm can operate the chromosome according to the operation thought of the tree data structure by adopting the tree coding. Because each node in the tree structure stores the information of the parent node and the child node, the operation can be directly carried out by utilizing the parent-child relationship information among the nodes without decoding. The information of each node includes type information (whether a leaf node is judged to determine whether the leaf node is a virtual machine or a server), parent information (whether a parent relationship is judged), child list (all child nodes are listed), order information (the order of the nodes in the same layer, if the virtual machines in the same server are not sorted, virtual machine objects cannot be correctly extracted during operations such as hybridization, and load information (load value);
s32, setting the population scale, and generating an initial population based on a spanning tree algorithm;
specifically, when the population size is too small, the diversity is insufficient, the search is easy to fall into a local optimal solution, and when the population size is too large, a large amount of computing resources are consumed, so the size is appropriate, and the population size in the invention is set to be 50-200. In addition, there are two main ways to initialize a population:
(1) continuously and randomly generating individuals, judging whether the individuals meet a given standard according to the fitness of the individuals, if not, eliminating the individuals, and if so, adding the individuals into the initial population until the number of the individuals reaches a set scale;
(2) firstly, determining the range of the optimal solution according to the relevant requirements, and then directly setting each initial individual in the range.
The embodiment of the invention preferably adopts a mode (1) to initialize the population and adopts a spanning tree method to generate, wherein the spanning tree consists of a server set and a virtual machine set. The specific process can be expressed as follows: opening a server node each time, randomly adding virtual machines one by one, modifying the childrenList information and the load information of the server node and the parent information and the order information of the currently added virtual machines, and if the current server node is overloaded (the load value exceeds a set server load threshold), re-opening a new server node and adding the virtual machine to the server node; when a plurality of server nodes are opened, the server nodes are selected to join the virtual machine, and then the load information is closer to the fully loaded server, and the processes are repeated until all the virtual machines are distributed. And after each distribution, calculating whether the individual correspondingly meets the load balancing condition and calculating the fitness, adding the individuals meeting the requirements into the initial population according to the result, eliminating the individuals if the individual does not meet the requirement, and generating new individuals again according to the process until the total number of the individuals reaches the scale of the initial population. To simplify the computation, the embodiments of the present invention assume that the performance (CPU, memory, bandwidth, etc.) of all servers is the same.
Step S33, calculating the number of servers to be started according to the virtual machines distributed according to actual needs;
the number N' of servers that should be turned on is calculated specifically according to the following expression:
wherein the molecular terms
Representing the sum of the loads of all the virtual machines, j representing the jth virtual machine, the denominator rho representing the load threshold of one server, and τ representing a preset fine adjustment constant, wherein in order to enable each server to be close to full load, the value of τ cannot be too large, a numerical value close to 0 is generally taken, and the whole formula is rounded up to obtain the number of the servers required to be opened;
step S34, if the number N of the servers which are correspondingly started by a plurality of individual genes is equal to the number N of the servers which should be started in the process of initializing the populationBased on the N servers and the corresponding virtual machines, the selection operation, the cross operation and the mutation operation of the genetic algorithm are carried out on the mapping relation of the tree structure, and the individuals meeting the requirements are added into the set { S*And step S35 is executed; if the number N of servers started in the process of initializing the population is greater than the number N' of servers that should be started, executing step S36;
it should be noted that the specific selection operation, crossover operation, and mutation operation are the same as those of the conventional genetic algorithm, and therefore, are not described herein again. Setting a load threshold value of a server in population initialization, and performing resource allocation based on high resource utilization rate on the basis;
the requirement for meeting the requirement is a rule for meeting load balance, the resource utilization rate of each server is high as much as possible, and the number of working servers is small as possible;
step S35, judging the set { S ] of the step S34*Whether the preset calculation termination condition is met or not is judged, and if the preset calculation termination condition is met, the set { S } is respectively calculated*Energy consumption of each allocation scheme in the station; the allocation scheme with the lowest energy consumption is the optimal resource allocation scheme, and virtual machine scheduling is carried out according to the optimal resource allocation scheme;
wherein the termination condition is specifically: the generated individual state is basically stable or the hereditary algebra exceeds the given maximum hereditary algebra;
step S36, sorting the servers according to the number of the placed virtual machines from a small number to a large number, sequentially adding all the virtual machines on the first Δ N servers to the server with the largest current load value among the N 'servers that should be started, so that N is equal to N', and then returning to execute step S34 to recalculate; where Δ N ═ N-N'.
And meanwhile, judging the closing time of the related servers according to a preset time threshold, namely closing the no-load servers which have migrated the virtual machines. A server is shut down once the duration of its ready state reaches a time threshold, where the time threshold represents the time at which the turning on/off of the server generates energy consumption equal to the energy consumption it would have if it had been unloaded but in the ready state. The time threshold for server shutdown is set as the following threshold:
wherein, PoffRepresenting the energy consumed by the server shutdown, PonRepresenting the energy consumed by the server at startup, EkeepThe energy consumption required when the server is idle is shown, t shows the moment when the total energy consumption of the idle time of the server is equal to the total energy consumption of one-time closing/opening of the server, namely the preset time threshold value of the waiting state. When the duration of the waiting state is less than the time threshold, the total idle time energy consumption of the server is less than the total one-time closing/opening energy consumption of the server, and the waiting state is maintained; and once the time reaches the time threshold, closing the server and waiting for the next opening.
Although the genetic algorithm of the embodiment of the invention calculates the total number of servers to be opened in the initialization stage, the servers cannot be immediately closed if the servers need to be closed, because the virtual machines on the servers cannot be immediately called into the target server, and because the target server may be overloaded after being migrated into the virtual machines, a transition server is needed. If the transition servers are exactly those servers needing to be shut down, the judgment method is adopted to determine whether the transition servers need to be shut down, and the energy consumption is reduced to a certain extent. The method for judging the overtime measures the energy consumption generated by closing/opening the server and the energy consumption caused by keeping the server in an idle state all the time, so that the working state of the corresponding server is in a state with the lowest energy consumption, and the method is easy to realize and has low cost.
Referring to fig. 2, another embodiment of the present invention provides a resource scheduling system of a data center, which can be used to implement the resource scheduling method of the data center in the foregoing embodiment, where the system of this embodiment includes:
the data acquisition module 11 is used for acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
the energy consumption calculation module 12 is configured to calculate an energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model; and
and the analysis scheduling module 13 is configured to calculate and find an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and perform scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
The above-described system embodiments are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
It should be noted that the system described in the foregoing embodiment corresponds to the method described in the foregoing embodiment, and therefore, a part of the system described in the foregoing embodiment that is not described in detail can be obtained by referring to the content of the method described in the foregoing embodiment, that is, the specific step content described in the method of the foregoing embodiment can be understood as the function that can be realized by the system of the present embodiment, and is not described herein again.
Moreover, when the resource scheduling system of the data center according to the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, the resource scheduling system may be stored in a computer-readable storage medium.
Another embodiment of the present invention provides a scheduling apparatus, including:
a memory for storing a computer program; and
a processor for implementing the steps of the method for resource scheduling in a data center according to the first aspect when executing the computer program.
For example, referring to fig. 3, the scheduling device may include a memory 21, a processor 22, and a bus 23;
the memory 21 includes at least one type of readable storage medium, which includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 21 may in some embodiments be an internal storage unit of the scheduling device. The memory 21 may also be an external storage device of the dispatching device in other embodiments, such as a plug-in hard disk equipped on the dispatching device, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and so on. Further, the memory 21 may also include both an internal storage unit of the scheduling device and an external storage device. The memory 21 can be used for storing application software installed in the scheduling device and various data, such as: the code of the program that executes the vehicle refueling behavior recognition method, etc., may also be used to temporarily store data that has been output or is to be output.
The bus 23 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 3, but this does not mean only one bus or one type of bus.
Further, the scheduling device may further include a network interface 24, and the network interface 24 may optionally include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used to establish a communication connection between the scheduling device and other electronic devices.
Optionally, the scheduling device may further comprise a user interface 25, the user interface 25 may comprise a Display (Display), an input unit such as a key, and the optional user interface 25 may further comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the scheduling apparatus and for displaying a visualized user interface.
Fig. 3 only shows the scheduling device with the components 21-25, and it will be understood by those skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the scheduling device, and may comprise fewer or more components than shown, or some components in combination, or a different arrangement of components.
Another embodiment of the present invention proposes a computer-readable storage medium, on which computer program instructions are stored, which, when executed by a processor, implement the steps of the resource scheduling method of a data center according to the first aspect.
Specifically, the computer-readable storage medium may include: any entity or device capable of carrying the computer program instructions, recording media, U-disks, removable hard disks, magnetic disks, optical disks, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signals, telecommunications signals, software distribution media, and the like.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A resource scheduling method of a data center is characterized by comprising the following steps:
step S1, acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
step S2, calculating to obtain the energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model;
and step S3, calculating and searching an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
2. The resource scheduling method of the data center according to claim 1, wherein the energy consumption calculation model is expressed by the following expression:
E＝ECPU+EMem+Ekeep
wherein E iskeepRepresenting the value of the energy consumption, k, generated by the server remaining in the ready state1Representing the ratio of the power of the server in the ready state to its full power, PCPUIndicating the operating power, P, of the server CPUMemIndicating the operating power of the server memory, PmaxIndicating the operating power of the server in full load, UCPU(t) represents the utilization rate of the server CPU at time t, UMem(t) represents the utilization rate of the server memory at time t, and t1 and t2 are the initial time and the end time when the server maintains the ready state.
3. The resource scheduling method of the data center according to claim 1, wherein the step S3 comprises the following sub-steps:
s31, coding the architecture of the data center by adopting a tree structure to establish a population; the root node of the tree structure represents a predefined data center controller, the second layer of nodes represent N servers, the leaf nodes represent M virtual machines and represent chromosomes, N is greater than 0, and M is greater than 0;
s32, setting the population scale, and generating an initial population based on a spanning tree algorithm;
step S33, calculating the number of servers to be started according to the virtual machines distributed according to actual needs;
step S34, if the number N of the servers started in the process of initializing the population is equal to the number N' of the servers to be started, selecting operation, cross operation and mutation operation of genetic algorithm are carried out on the mapping relation of the N servers and the corresponding virtual machines in the tree structure, and the individuals meeting the requirements are added into the set { S*And step S35 is executed; if the number N of servers started in the process of initializing the population is greater than the number N' of servers that should be started, executing step S36;
step S35, judging the set { S ] of the step S34*Whether the preset calculation termination condition is met or not is judged, and if the preset calculation termination condition is met, the set { S } is respectively calculated*Energy consumption of each allocation scheme in the station; the allocation scheme with the lowest energy consumption is the optimal resource allocation scheme, and virtual machine scheduling is carried out according to the optimal resource allocation scheme;
step S36, sorting the servers according to the number of the placed virtual machines from a small number to a large number, sequentially adding all the virtual machines on the first Δ N servers to the server with the largest current load value among the N 'servers that should be started, so that N is equal to N', and then returning to execute step S34 to recalculate; where Δ N ═ N-N'.
4. The resource scheduling method of the data center according to claim 3, wherein the step S33 is specifically configured to calculate the number N' of servers that should be turned on according to the following expression:
wherein the molecular terms
The load sum of all the virtual machines is represented, j represents the jth virtual machine, the denominator rho represents the load threshold of one server, and tau represents a preset fine adjustment constant.
5. The resource scheduling method of the data center according to claim 3, wherein the step S36 further comprises:
and judging the closing time of the no-load server according to a preset time threshold, and closing the no-load server when the duration of the ready state of the no-load server reaches the time threshold.
6. The resource scheduling method of the data center according to claim 5, wherein the time threshold is a time when a sum of energy consumptions generated by turning on and off the server is equal to an energy consumption generated by the server in a ready state without load.
7. The resource scheduling method of the data center according to claim 3, wherein the population size set in the step S32 is 50-200.
8. A resource scheduling system of a data center, comprising:
the data acquisition module is used for acquiring energy consumption data of each server of the data center; the energy consumption data comprises resource utilization rate of the server and performance data of each component of the server;
the energy consumption calculation module is used for calculating to obtain an energy consumption value of each server according to the energy consumption data and a preset energy consumption calculation model; and
and the analysis scheduling module is used for calculating and searching an optimal resource scheduling scheme by using a genetic algorithm according to the resource utilization rate and the energy consumption value of each server, and performing scheduling operation or closing operation on each server of the data center according to the optimal resource scheduling scheme.
9. A server, comprising:
a memory for storing a computer program; and
processor for implementing the steps of the resource scheduling method of a data center according to any of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the resource scheduling method of a data center according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110597755.4A CN113204429A (en) | 2021-05-31 | 2021-05-31 | Resource scheduling method and system of data center, scheduling equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110597755.4A CN113204429A (en) | 2021-05-31 | 2021-05-31 | Resource scheduling method and system of data center, scheduling equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113204429A true CN113204429A (en) | 2021-08-03 |
Family
ID=77023658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110597755.4A Pending CN113204429A (en) | 2021-05-31 | 2021-05-31 | Resource scheduling method and system of data center, scheduling equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113204429A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115373826A (en) * | 2022-10-26 | 2022-11-22 | 北京创新乐知网络技术有限公司 | Task scheduling method and device based on cloud computing |
| CN116755982A (en) * | 2023-05-15 | 2023-09-15 | 合芯科技(苏州)有限公司 | Method and system for calculating energy consumption ratio of server |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209484A (en) * | 2019-05-30 | 2019-09-06 | 华南理工大学 | Based on cloudy and Security mapping credible cloud task scheduling system and method |
| US20200310885A1 (en) * | 2019-03-27 | 2020-10-01 | International Business Machines Corporation | Cloud data center with reduced energy consumption |
| CN111752678A (en) * | 2020-06-16 | 2020-10-09 | 杭州电子科技大学 | Low-power-consumption container placement method for distributed collaborative learning in edge computing |
| CN111966447A (en) * | 2020-07-08 | 2020-11-20 | 华南理工大学 | Container placing method based on double-row genetic algorithm |
| CN112380005A (en) * | 2020-11-10 | 2021-02-19 | 深圳供电局有限公司 | Data center energy consumption management method and system |
-
2021
- 2021-05-31 CN CN202110597755.4A patent/CN113204429A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200310885A1 (en) * | 2019-03-27 | 2020-10-01 | International Business Machines Corporation | Cloud data center with reduced energy consumption |
| CN110209484A (en) * | 2019-05-30 | 2019-09-06 | 华南理工大学 | Based on cloudy and Security mapping credible cloud task scheduling system and method |
| CN111752678A (en) * | 2020-06-16 | 2020-10-09 | 杭州电子科技大学 | Low-power-consumption container placement method for distributed collaborative learning in edge computing |
| CN111966447A (en) * | 2020-07-08 | 2020-11-20 | 华南理工大学 | Container placing method based on double-row genetic algorithm |
| CN112380005A (en) * | 2020-11-10 | 2021-02-19 | 深圳供电局有限公司 | Data center energy consumption management method and system |
Non-Patent Citations (1)
| Title |
|---|
| 程春玲 等: "云环境下一种节能的资源调度算法", ***工程与电子技术, vol. 35, no. 11, 6 August 2013 (2013-08-06), pages 2417 - 2423 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115373826A (en) * | 2022-10-26 | 2022-11-22 | 北京创新乐知网络技术有限公司 | Task scheduling method and device based on cloud computing |
| CN115373826B (en) * | 2022-10-26 | 2022-12-30 | 北京创新乐知网络技术有限公司 | Task scheduling method and device based on cloud computing |
| CN116755982A (en) * | 2023-05-15 | 2023-09-15 | 合芯科技(苏州)有限公司 | Method and system for calculating energy consumption ratio of server |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3847549B1 (en) | Minimizing impact of migrating virtual services | |
| CN110869909B (en) | System and method for applying machine learning algorithms to calculate health scores for workload scheduling | |
| CN106899660A (en) | Cloud data center energy-saving distribution implementation method based on trundle gray forecast model | |
| CN108701060A (en) | Method for computing system adjust automatically | |
| Rajabzadeh et al. | Energy-aware framework with Markov chain-based parallel simulated annealing algorithm for dynamic management of virtual machines in cloud data centers | |
| CN113204429A (en) | Resource scheduling method and system of data center, scheduling equipment and medium | |
| CN105426241A (en) | Cloud computing data center based unified resource scheduling energy-saving method | |
| CN112835698A (en) | Heterogeneous cluster-based dynamic load balancing method for request classification processing | |
| Li et al. | Cost-aware automatic scaling and workload-aware replica management for edge-cloud environment | |
| CN115794341A (en) | Task scheduling method, device, equipment and storage medium based on artificial intelligence | |
| CN109976879B (en) | Cloud computing virtual machine placement method based on resource usage curve complementation | |
| EP3948537A1 (en) | Compute platform optimization over the life of a workload in a distributed computing environment | |
| CN111565216A (en) | Back-end load balancing method, device, system and storage medium | |
| CN111176784A (en) | Virtual machine integration method based on extreme learning machine and ant colony system | |
| US20220050716A1 (en) | Virtual machine placement method and virtual machine placement device implementing the same | |
| CN106844175B (en) | A kind of cloud platform method for planning capacity based on machine learning | |
| CN117311973A (en) | Computing device scheduling method and device, nonvolatile storage medium and electronic device | |
| Garg et al. | Optimal virtual machine scheduling in virtualized cloud environment using VIKOR method | |
| Wang et al. | Gmpr: a two-phase heuristic algorithm for virtual machine placement in large-scale cloud data centers | |
| CN116647560A (en) | Method, device, equipment and medium for coordinated optimization control of Internet of things computer clusters | |
| Kanagaraj et al. | Uniform distribution elephant herding optimization (UDEHO) based virtual machine consolidation for energy-efficient cloud data centres | |
| CN108073449B (en) | Dynamic virtual machine placement method | |
| Tang | Load balancing optimization in cloud computing based on task scheduling | |
| CN114741160A (en) | Dynamic virtual machine integration method and system based on balanced energy consumption and service quality | |
| CN111598390B (en) | Method, device, equipment and readable storage medium for evaluating high availability of server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |