CN111722912B - Data center virtual machine management method oriented to balance of performance and power consumption - Google Patents

Data center virtual machine management method oriented to balance of performance and power consumption Download PDF

Info

Publication number
CN111722912B
CN111722912B CN202010587622.4A CN202010587622A CN111722912B CN 111722912 B CN111722912 B CN 111722912B CN 202010587622 A CN202010587622 A CN 202010587622A CN 111722912 B CN111722912 B CN 111722912B
Authority
CN
China
Prior art keywords
virtual machine
data center
neural network
output
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010587622.4A
Other languages
Chinese (zh)
Other versions
CN111722912A (en
Inventor
莫毓昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guofurui Fujian Information Technology Industrial Park Co ltd
Huaqiao University
Original Assignee
Guofurui Fujian Information Technology Industrial Park Co ltd
Huaqiao University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guofurui Fujian Information Technology Industrial Park Co ltd, Huaqiao University filed Critical Guofurui Fujian Information Technology Industrial Park Co ltd
Priority to CN202010587622.4A priority Critical patent/CN111722912B/en
Publication of CN111722912A publication Critical patent/CN111722912A/en
Application granted granted Critical
Publication of CN111722912B publication Critical patent/CN111722912B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention discloses a data center virtual machine management method oriented to performance and power consumption balance, and relates to the technical field of computers. The data center virtual machine management method facing to the balance of performance and power consumption comprises the following steps: s1, actually measuring to obtain operation parameters of a virtual machine of a data center, wherein the parameters comprise: 1.1, actually obtaining the starting time of the virtual machine on the physical machineT start . According to the data center virtual machine management method for balancing performance and power consumption, the number of virtual machines is increased and decreased appropriately in advance by learning the law of customer request achievement, when the number of customer requests is continuously increased, idle virtual machines are started in advance, and when the number of customer requests is continuously decreased, the minimum virtual machines are kept to be started, so that the number of reasonable starting virtual machines is maintained, the balance between the data center customer request service quality and the virtual machine energy consumption is realized, and the higher customer request processing response rate and the lower virtual machine energy consumption are achieved.

Description

Data center virtual machine management method oriented to balance of performance and power consumption
Technical Field
The invention relates to the technical field of computers, in particular to a data center virtual machine management method oriented to balance of performance and power consumption.
Background
Virtual machines refer to complete computer systems that run in a completely isolated environment with complete hardware system functionality through software emulation. Work that can be done in a physical computer can be done in a virtual machine. When creating a virtual machine in a computer, a part of hard disk and memory capacity of the physical machine are required to be used as the hard disk and memory capacity of the virtual machine. Each virtual machine is provided with an independent CMOS, a hard disk and an operating system, and can operate the virtual machine like a physical machine, and when the virtual machine is used, the performance and the power consumption of the virtual machine need to be balanced and managed through a data center.
In the prior art, a supervised learning method is adopted in the method for training the neural network model, namely, labeled sample data is adopted for model training, but a large amount of labeled sample data is required to be adopted in the method, and meanwhile, the balance between the service quality requested by a user of a data center and the virtual energy consumption is difficult to realize well.
Disclosure of Invention
(one) solving the technical problems
Aiming at the defects of the prior art, the invention provides a data center virtual machine management method oriented to balance performance and power consumption, which solves the problems that in the prior art, a supervised learning method is adopted in a method for training a neural network model, namely, labeled sample data is adopted for model training, but a large amount of labeled sample data is required to be adopted in the method, and meanwhile, the balance between the service quality requested by a data center user and the virtual machine energy is difficult to realize well.
(II) technical scheme
In order to achieve the above purpose, the invention is realized by the following technical scheme: a data center virtual machine management method oriented to balance performance and power consumption comprises the following steps:
s1, actually measuring to obtain operation parameters of a virtual machine of a data center, wherein the parameters comprise:
1.1, actually obtaining the starting time of the virtual machine on the physical machineT start ；
1.2, according to the QoS requirement of the client to the virtual machine (such as the maximum time the virtual machine can process the client request)T reqmax ) Actually measured maximum number of client requests that can be processed on a single virtual machineR max ；
1.3 measured average processing time required for virtual machine obtained for single guest requestT reqmean ；
1.4 actual measurement acquisition management periodT g (e.g. 1 year) ithT start Number of customer requests during a time periodR gi ；
S2, determining a virtual machine optimization scheduling target, wherein the method comprises the following steps:
2.1, improving the response rate of user request processing, namely, when a client request arrives, the probability of processing the client request immediately;
2.2, reducing the number of started virtual machines;
s3, defining a working state of a virtual machine of the data center, wherein the specific operation is as follows:
an array S1. M is used]To define the working state of the virtual machine of the data center, the dimension M of the array, namely the number of elements of the array, M=T reqmean /T start . From the measured data, it is known thatT start In about 1 minute, but is typically web-appliedT reqmean Within 10 minutes. So typically M is within 10, each element S [ i ] in the state array S]I is greater than or equal to 1 and less than or equal to M, which indicates that i is processed on all started virtual machines in the current data centerT start The number of user requests that are minute and not yet processed;
s4, defining a data center virtual machine management strategy, which comprises the following steps:
4.1, starting 1 virtual machine;
4.2, maintaining the number of the virtual machines currently working unchanged;
4.3, closing 1 virtual machine;
s5, constructing a neural network model for virtual machine management, generating a neural network model for describing a mapping relation between the working state of a data center virtual machine and the management strategy of the data center virtual machine, wherein the input of the model is the working state S [1..M ] of the data center virtual machine, and the output is the probability of each of the 3 virtual machine management strategies, wherein the specific conditions are as follows:
5.1, an input layer of the neural network model comprises M inputs corresponding to M elements in the working state S1. M of the virtual machine of the data center;
5.2, the first hidden layer of the neural network model is a fully connected layer containing 64 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.3, the second hidden layer of the neural network model is a fully connected layer containing 32 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.4, the output layer of the neural network model is a full-connection layer containing 3 neurons, and the activation function is softmax, which is the probability adopted by each of the 3 virtual machine management strategies;
s6, training a neural network model for virtual machine management, and unifying the process of generating a high-quality training sample and training the neural network model by adopting a reinforcement learning method. The specific process is as follows;
6.1, randomly initializing parameters of a neural network model;
6.2, reference measured dataR g1 Generating a first quantity X of customer requests 1 =R g1 If X 1 /R max Dividing by integer, the starting number is X 1 /R max If X is 1 /R max With remainder, the number of starts is X 1 /R max Virtual machines of +1, e.g. X 1 =203，R max =100, then 3 virtual machines need to be started, 2 of which are fully loaded, 1 running 3 customer requests and their idle processing capacity is able to process 97 next batch of customer requests. At this time, the virtual machine of the data center works in a state S 1 [1..M]=[X 1 ,0,0,…,0]；
6.3, reference measured dataR g2 And add the randomness to the next batch of customer request quantity X 2 =0.9*R g2 +Δ, Δ is 1 and 0.1R g2 The random number in between, the addition of delta is mainly used to introduce randomness in multiple rounds of training. Operating state S of data center virtual machine 1 [1..M]Right shift by one position to obtain S 2 [1..M]，S 2 [1..M]Left end first position S 2 [1]Filling in the newly added number X of customer requests 2 ，S 1 [1..M]Removed right first position S 1 [M]Discarded as a processed client request, i.e. a new numberWorking state S of virtual machine in data center 2 [1..M]=[X 2 ,X 1 ,0,…,0]；
6.4 operating State S of data center virtual machine 2 [1..M]=[X 2 ,X 1 ,0,…,0]And inputting the Output data to a neural network model to obtain Output of the neural network model. Since Output is the probability that each of the 3 virtual machine management policies is adopted, the management policy H with the highest probability can be obtained. For example, output= {0.2,0.6,0.2}, then the highest probability Output max The management policy H corresponding to=0.6 is the 2 nd policy: maintaining the number of the virtual machines currently working unchanged;
6.5, reference measured dataR g3 And adding randomness to generate the next customer request quantity X 3 =0.9*R g3 +Δ, Δ is in the range 1 and 0.1R g3 The random number in between, the addition of delta is mainly used to introduce randomness in multiple rounds of training. Operating state S of data center virtual machine 2 [1..M]Right shift by one position to obtain S 3 [1..M]，S 3 [1..M]Left end first position S 3 [1]Filling in the newly added number X of customer requests 3 ，S 2 [1..M]Removed right first position S 2 [M]Discarded as a processed client request, i.e. new data center virtual machine working state S 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]；
6.6, for data center virtual machine working State from S 2 [1..M]=[X 2 ,X 1 ,0,…,0]Transition to S 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]Analysis of the obtained returns when different strategies are selected in step 6.4Reward=β*1/R idle -α*R wait I.e. the number of client requests waiting for virtual machine processing obtained after executing different virtual machine management policiesR wait And idle processing capability of the working virtual machineR idle . Beta and alpha are corresponding adjustable weight parameters;
6.7, operating the new data center virtual machine in the S state 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]Inputting the Output value into the neural network model, obtaining the Output value in the new state of the neural network model as Output, and obtaining the maximum value Output in the Output max ；
6.8, obtaining three return values under three different strategies by using the method in the step 6.6Reward 1 ，Reward 2 ，Reward 3 Output obtained in step 6.7 max Constructing a data center virtual machine operating state S 2 [1..M]=[X 2 ,X 1 ,0,…,0]The following is more true of Output ', output' =softmax ([ γ ] Output) max +η*Reward 1 ，γ*Output max +η*Reward 2 ，γ*Output max +η*Reward 3 ]) Where γ is the discount rate parameter and η is the learning rate parameter;
6.9, respectively S 2 [1..M]=[X 2 ,X 1 ,0,…,0]And Output' is used as input and Output to construct a new high-quality training sample, and the weight of the neural network is adjusted by using a classical neural network backward propagation algorithm;
6.10, circularly executing the steps 6.3 to 6.9, namely generating the next customer request quantity, constructing a new state by right shifting the state array S1. M, calculating the input S1. M and the Output' which are more in line with the actual samples, and adjusting the weight of the neural network by utilizing a classical neural network backward propagation algorithm until the cycle times meet a preset threshold Num, wherein Num=10000;
s7, verifying the effect of the neural network model for virtual machine management, wherein the trained neural network model is applied to a small data center, the data center comprises 10 physical machines, each physical machine runs 10 virtual machines at most, and the maximum number of client requests which can be processed by each virtual machine is calculatedR max 18000; start time of virtual machine on physical machineT start 1 minute; average processing time required for virtual machines requested by a single guestT reqmean For 5 minutes; the data center virtual machine work state array dimension m=5,the verification test compares three data center virtual machine management methods:
case1. The virtual machine management method based on the neural network of the patent;
case2, a virtual machine management method adopting a minimum virtual machine starting strategy, namely, no idle virtual machine exists at any moment.
Case3, a virtual machine management method adopting a 1-mode redundancy strategy, namely keeping 1 idle virtual machine before a new user request is added;
the data center virtual machine management method verifies that the test duration is 6 months, and the indexes compared by the test comprise: each virtual machine management cycleT start In, the number of client requests waiting for virtual machine processing is averagedR waitmean And the idle processing capacity of the average working virtual machineR idlemean 。
Preferably, in step S2.1, if only the objective in step S2.1 is considered, the data center may use a "start all virtual machines on all physical machines" policy, where the response rate of the user request processing is the highest, that is, the user request processing can be performed as long as there is an idle virtual machine. However, this strategy can result in significant power wastage because the idle virtual machines consume a large amount of power.
Preferably, in step S2.2, if only the objective in step S2.2 is considered, the data center may "start the least number of virtual machines" policy, i.e. the number of virtual machines started is processing the client request, and there are no redundant free virtual machines. However, this strategy often reduces the response rate of user request processing, i.e. when a large number of client requests arrive, the virtual machine needs to be started due to insufficient processing power, and during the period of time the virtual machine is started, the user request is in a waiting state and is not processed in time.
Preferably, in step S2, the final objective of comprehensively considering the objective in 2.1 and the objective in 2.2 is: through an effective virtual machine management strategy, the reasonable starting number of virtual machines is maintained, the balance between the user response rate of the data center and the virtual machine energy consumption is realized, and the higher response rate of user request processing and lower virtual machine energy consumption are achieved.
(III) beneficial effects
The invention provides a data center virtual machine management method oriented to balance of performance and power consumption. The beneficial effects are as follows:
according to the data center virtual machine management method for balancing performance and power consumption, the number of virtual machines is increased and decreased appropriately in advance by learning the law of customer request achievement, when the number of customer requests is continuously increased, idle virtual machines are started in advance, and when the number of customer requests is continuously decreased, the minimum virtual machines are kept to be started, so that the number of reasonable starting virtual machines is maintained, the balance between the data center customer request service quality and the virtual machine energy consumption is realized, and the higher customer request processing response rate and the lower virtual machine energy consumption are achieved.
Drawings
FIG. 1 is a schematic diagram of the results of verification tests of three virtual machine management methods according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the present invention provides a technical solution: a data center virtual machine management method oriented to balance performance and power consumption comprises the following steps:
s1, actually measuring to obtain operation parameters of a virtual machine of a data center, wherein the parameters comprise:
1.1, actually obtaining the starting time of the virtual machine on the physical machineT start ；
1.2, according to the QoS requirement of the client to the virtual machine (such as the maximum time the virtual machine can process the client request)T reqmax ) The obtained single virtual machine can be measuredMaximum number of customer requests processedR max ；
1.3 measured average processing time required for virtual machine obtained for single guest requestT reqmean ；
1.4 actual measurement acquisition management periodT g (e.g. 1 year) ithT start Number of customer requests during a time periodR gi ；
S2, determining a virtual machine optimization scheduling target, and comprehensively considering the target in 2.1 and the target in 2.2 as follows: through an effective virtual machine management strategy, the reasonable starting number of virtual machines is maintained, the balance between the user response rate of the data center and the virtual machine energy consumption is realized, and the higher response rate of user request processing and lower virtual machine energy consumption are achieved, wherein the method comprises the following steps:
2.1, improving the response rate of user request processing, namely, when the client request arrives, the probability of processing the client request immediately, if only the target in 2.1 is considered, the data center can adopt a strategy of starting all virtual machines on all physical machines, and at the moment, the response rate of user request processing is highest, namely, the client request processing can be performed as long as the idle virtual machines exist. However, this strategy can result in significant power wastage, as the idle virtual machines consume a large amount of power;
2.2, reducing the number of started virtual machines, if only the target in 2.2 is considered, the data center can start a minimum number of virtual machines, namely the number of started virtual machines is processing the client request, and no redundant idle virtual machines exist. However, this policy often reduces the response rate of user request processing, i.e. when a large number of client requests arrive, the virtual machine needs to be started because of insufficient processing power, and during the period of time when the virtual machine is started, the user request is in a waiting state and is not processed in time;
s3, defining a working state of a virtual machine of the data center, wherein the specific operation is as follows:
an array S1. M is used]To define the data center virtual machine operating state, the dimension M of the array,i.e. the number of elements of the array, M =T reqmean /T start . From the measured data, it is known thatT start In about 1 minute, but is typically web-appliedT reqmean Within 10 minutes. So typically M is within 10, each element S [ i ] in the state array S]I is greater than or equal to 1 and less than or equal to M, which indicates that i is processed on all started virtual machines in the current data centerT start The number of user requests that are minute and not yet processed;
s4, defining a data center virtual machine management strategy, which comprises the following steps:
4.1, starting 1 virtual machine;
4.2, maintaining the number of the virtual machines currently working unchanged;
4.3, closing 1 virtual machine;
s5, constructing a neural network model for virtual machine management, generating a neural network model for describing a mapping relation between the working state of a data center virtual machine and the management strategy of the data center virtual machine, wherein the input of the model is the working state S [1..M ] of the data center virtual machine, and the output is the probability of each of the 3 virtual machine management strategies, wherein the specific conditions are as follows:
5.1, an input layer of the neural network model comprises M inputs corresponding to M elements in the working state S1. M of the virtual machine of the data center;
5.2, the first hidden layer of the neural network model is a fully connected layer containing 64 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.3, the second hidden layer of the neural network model is a fully connected layer containing 32 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.4, the output layer of the neural network model is a full-connection layer containing 3 neurons, and the activation function is softmax, which is the probability adopted by each of the 3 virtual machine management strategies;
s6, training a neural network model for virtual machine management, and unifying the process of generating a high-quality training sample and training the neural network model by adopting a reinforcement learning method. The specific process is as follows;
6.1, randomly initializing parameters of a neural network model;
6.2, reference measured dataR g1 Generating a first quantity X of customer requests 1 =R g1 If X 1 /R max Dividing by integer, the starting number is X 1 /R max If X is 1 /R max With remainder, the number of starts is X 1 /R max Virtual machines of +1, e.g. X 1 =203，R max =100, then 3 virtual machines need to be started, 2 of which are fully loaded, 1 running 3 customer requests and their idle processing capacity is able to process 97 next batch of customer requests. At this time, the virtual machine of the data center works in a state S 1 [1..M]=[X 1 ,0,0,…,0]；
6.3, reference measured dataR g2 And add the randomness to the next batch of customer request quantity X 2 =0.9*R g2 +Δ, Δ is 1 and 0.1R g2 The random number in between, the addition of delta is mainly used to introduce randomness in multiple rounds of training. Operating state S of data center virtual machine 1 [1..M]Right shift by one position to obtain S 2 [1..M]，S 2 [1..M]Left end first position S 2 [1]Filling in the newly added number X of customer requests 2 ，S 1 [1..M]Removed right first position S 1 [M]Discarded as a processed client request, i.e. new data center virtual machine working state S 2 [1..M]=[X 2 ,X 1 ,0,…,0]；
6.4 operating State S of data center virtual machine 2 [1..M]=[X 2 ,X 1 ,0,…,0]And inputting the Output data to a neural network model to obtain Output of the neural network model. Since Output is the probability that each of the 3 virtual machine management policies is adopted, the management policy H with the highest probability can be obtained. For example, output= {0.2,0.6,0.2}, then the highest probability Output max The management policy H corresponding to=0.6 is the 2 nd policyThe following is omitted: maintaining the number of the virtual machines currently working unchanged;
6.5, reference measured dataR g3 And adding randomness to generate the next customer request quantity X 3 =0.9*R g3 +Δ, Δ is in the range 1 and 0.1R g3 The random number in between, the addition of delta is mainly used to introduce randomness in multiple rounds of training. Operating state S of data center virtual machine 2 [1..M]Right shift by one position to obtain S 3 [1..M]，S 3 [1..M]Left end first position S 3 [1]Filling in the newly added number X of customer requests 3 ，S 2 [1..M]Removed right first position S 2 [M]Discarded as a processed client request, i.e. new data center virtual machine working state S 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]；
6.6, for data center virtual machine working State from S 2 [1..M]=[X 2 ,X 1 ,0,…,0]Transition to S 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]Analysis of the obtained returns when different strategies are selected in step 6.4Reward=β*1/R idle -α*R wait I.e. the number of client requests waiting for virtual machine processing obtained after executing different virtual machine management policiesR wait And idle processing capability of the working virtual machineR idle . Beta and alpha are corresponding adjustable weight parameters;
6.7, operating the new data center virtual machine in the S state 3 [1..M]=[X 3 ,X 2 ,X 1 ,…,0]Inputting the Output value into the neural network model, obtaining the Output value in the new state of the neural network model as Output, and obtaining the maximum value Output in the Output max ；
6.8, obtaining three return values under three different strategies by using the method in the step 6.6Reward 1 ，Reward 2 ，Reward 3 Output obtained in step 6.7 max Constructing a data center virtual machine operating state S 2 [1..M]=[X 2 ,X 1 ,0,…,0]The following is more true of Output ', output' =softmax ([ γ ] Output) max +η*Reward 1 ，γ*Output max +η*Reward 2 ，γ*Output max +η*Reward 3 ]) Where γ is the discount rate parameter and η is the learning rate parameter;
6.9, respectively S 2 [1..M]=[X 2 ,X 1 ,0,…,0]And Output' is used as input and Output to construct a new high-quality training sample, and the weight of the neural network is adjusted by using a classical neural network backward propagation algorithm;
6.10, circularly executing the steps 6.3 to 6.9, namely generating the next customer request quantity, constructing a new state by right shifting the state array S1. M, calculating the input S1. M and the Output' which are more in line with the actual samples, and adjusting the weight of the neural network by utilizing a classical neural network backward propagation algorithm until the cycle times meet a preset threshold Num, wherein Num=10000;
s7, verifying the effect of the neural network model for virtual machine management, wherein the trained neural network model is applied to a small data center, the data center comprises 10 physical machines, each physical machine runs 10 virtual machines at most, and the maximum number of client requests which can be processed by each virtual machine is calculatedR max 18000; start time of virtual machine on physical machineT start 1 minute; average processing time required for virtual machines requested by a single guestT reqmean For 5 minutes; the data center virtual machine operating state array dimension m=5, where the validation test compares three data center virtual machine management methods:
case1. The virtual machine management method based on the neural network of the patent;
case2, a virtual machine management method adopting a minimum virtual machine starting strategy, namely, no idle virtual machine exists at any moment.
Case3, a virtual machine management method adopting a 1-mode redundancy strategy, namely keeping 1 idle virtual machine before a new user request is added;
data center virtual machine management sideThe duration of the method verification test is 6 months, and the indexes compared by the test comprise: each virtual machine management cycleT start In, the number of client requests waiting for virtual machine processing is averagedR waitmean And the idle processing capacity of the average working virtual machineR idlemean 。
Analysis of results:
the results of the verification tests of the three virtual machine management methods are shown in fig. 1;
for Case 3', a 1-mode redundancy strategy is adopted, namely, 1 idle virtual machine is kept before the new user request, and only when the next new user request quantity minus the user request quantity S [ M ] processed in the period]"exceedR max When a client request waiting for processing by the virtual machine is likely to occur, therefore, whenR gi At average 20000, the number of client requests waiting for virtual machine processing is minimal, whereR waitmean 3. On the other hand, since 1 idle virtual machine is maintained before the new user request is added, and the idle processing capacity of the working virtual machine is added, the method can be used for executing the following stepsR idlemean Will exceedR max Here, whereR idlemean 29764. The virtual machine management method has better performance but larger energy consumption.
For Case 2', the least virtual machine starting strategy is adopted, i.e. no idle virtual machine exists at any moment, as long as the next newly added user request quantity is larger than the user request quantity S [ M ] processed in the period]"when a guest request waiting for virtual machine processing may occur, so whenR gi When the mean square error is 537, waiting for the number of client requests processed by the virtual machineR waitmean 9305 whenR gi When the mean square error increases to 1093, the number of client requests waiting for processing by the virtual machineR waitmean The increase is 24831. On the other hand, since there is no free virtual machine at any time, there is no free virtual machineR idlemean Not exceedR max Here, whereR idlemean 7433 and 6832. The virtual machine management method has lower power consumption but poor service quality.
As for the Case1 "virtual machine management method based on neural network", the virtual machines are increased or decreased appropriately in advance by learning the rule of the number of client requests. From the verification data, the virtual machine management method well realizes the balance between the service quality requested by the user of the data center and the virtual machine energy consumption, and achieves higher response rate of the user request processing and lower virtual machine energy consumption. Specifically, by accurately predicting that the number of next user requests will continuously increase, 1 idle virtual machine is started before the newly increased user requests, so that the number of client requests waiting for processing by the virtual machine is reducedR waitmean On the other hand, by accurately predicting that the number of next user requests will continuously decrease, the minimum starting of the virtual machines is maintained, and the processing capacity of the idle virtual machines is reducedR idlemean 。
In summary, according to the data center virtual machine management method for balancing performance and power consumption, through learning the rule that the number of customer requests is reached, the virtual machines are increased and decreased appropriately in advance, when the number of customer requests is continuously increased, idle virtual machines are started in advance, and when the number of customer requests is continuously decreased, the minimum virtual machines are kept to be started, so that the number of reasonable starting virtual machines is maintained, the balance between the data center customer request service quality and the virtual machine energy consumption is realized, and the higher customer request processing response rate and the lower virtual machine energy consumption are achieved.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (4)

1. A data center virtual machine management method oriented to balance performance and power consumption is characterized in that: the method comprises the following steps:
s1, actually measuring to obtain operation parameters of a virtual machine of a data center, wherein the parameters comprise:
1.1, actually obtaining the starting time T of the virtual machine on the physical machine start ；
1.2, according to the service quality requirement of the client to the virtual machine, the service quality requirement comprises: maximum time T for virtual machine to process client request reqmax Actually measured maximum number of client requests R which can be processed on single virtual machine max ；
1.3 measured average processing time T required by the obtained virtual machine for a single guest request reqmean ；
1.4, actual measurement obtaining management period T g T of (i) start Number of customer requests R during a time period gi ；
S2, determining a virtual machine optimization scheduling target, wherein the method comprises the following steps:
2.1, improving the response rate of user request processing, namely, when a client request arrives, the probability of processing the client request immediately;
2.2, reducing the number of started virtual machines;
s3, defining a working state of a virtual machine of the data center, wherein the specific operation is as follows:
an array S1. M is used]To define the working state of the virtual machine of the data center, the dimension M of the array, namely the number of elements of the array, M=T reqmean /T start From the measured data, it can be seen that T is the common condition start For 1 minute up and down, but in the usual case web application T reqmean Within 10 minutes, so typically M is within 10, each element S [ i ] in the state array S]I is greater than or equal to 1 and less than or equal to M, indicating that i T has been processed on all started virtual machines in the current data center start The number of user requests that are minute and not yet processed;
s4, defining a data center virtual machine management strategy, which comprises the following steps:
4.1, starting 1 virtual machine;
4.2, maintaining the number of the virtual machines currently working unchanged;
4.3, closing 1 virtual machine;
s5, constructing a neural network model for virtual machine management, generating a neural network model for describing a mapping relation between the working state of a data center virtual machine and the management strategy of the data center virtual machine, wherein the input of the model is the working state S [1..M ] of the data center virtual machine, and the output is the probability of each of the 3 virtual machine management strategies, wherein the specific conditions are as follows:
5.1, an input layer of the neural network model comprises M inputs corresponding to M elements in the working state S1. M of the virtual machine of the data center;
5.2, the first hidden layer of the neural network model is a fully connected layer containing 64 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.3, the second hidden layer of the neural network model is a fully connected layer containing 32 neurons, the activation function is sigmoid, and dropout is set to 0.1;
5.4, the output layer of the neural network model is a full-connection layer containing 3 neurons, and the activation function is softmax, which is the probability adopted by each of the 3 virtual machine management strategies;
s6, training a neural network model for virtual machine management, and unifying the process of generating a high-quality training sample and training the neural network model by adopting a reinforcement learning method, wherein the specific process is as follows;
6.1, randomly initializing parameters of a neural network model;
6.2、reference to measured data R g1 Generating a first quantity X of customer requests 1 ＝R g1 If X 1 /R max Dividing by integer, the starting number is X 1 /R max If X is 1 /R max With remainder, the number of starts is X 1 /R max +1 virtual machine, if: x is X 1 ＝203，R max When=100, 3 virtual machines need to be started, wherein 2 virtual machines are fully loaded, 1 virtual machine runs 3 client requests and its idle processing capacity can process 97 next client requests, and the data center virtual machine is in a working state S 1 [1..M]＝[X 1 ,0,0,…,0]；
6.3, reference measured data R g2 And add the randomness to the next batch of customer request quantity X 2 ＝0.9*R g2 +Δ, Δ is 1 and 0.1×R g2 The random numbers in between are added delta, which is mainly used for introducing randomness in multiple rounds of training and enabling the virtual machine of the data center to work in a state S 1 [1..M]Right shift by one position to obtain S 2 [1..M]，S 2 [1..M]Left end first position S 2 [1]Filling in the newly added number X of customer requests 2 ，S 1 [1..M]Removed right first position S 1 [M]Discarded as a processed client request, i.e. new data center virtual machine working state S 2 [1..M]＝[X 2 ,X 1 ,0,…,0]；
6.4 operating State S of data center virtual machine 2 [1..M]＝[X 2 ,X 1 ,0,…,0]The probability of each of the 3 virtual machine management strategies is adopted, so that the management strategy H with the highest probability can be obtained, and if: output= {0.2,0.6,0.2}, then highest probability Output max The management policy H corresponding to=0.6 is the 2 nd policy: maintaining the number of the virtual machines currently working unchanged;
6.5, reference measured data R g3 And adding randomness to generate the next customer request quantity X 3 ＝0.9*R g3 +Δ, Δ is in the range 1 and 0.1×R g3 Random number in between, adding delta is mainly used for introducing random in multiple rounds of trainingMechanically, the working state S of the virtual machine of the data center 2 [1..M]Right shift by one position to obtain S 3 [1..M]，S 3 [1..M]Left end first position S 3 [1]Filling in the newly added number X of customer requests 3 ，S 2 [1..M]Removed right first position S 2 [M]Discarded as a processed client request, i.e. new data center virtual machine working state S 3 [1..M]＝[X 3 ,X 2 ,X 1 ,…,0]；
6.6, for data center virtual machine working State from S 2 [1..M]＝[X 2 ,X 1 ,0,…,0]Transition to S 3 [1..M]＝[X 3 ,X 2 ,X 1 ,…,0]When different strategies are selected in the analysis of 6.4 steps, the obtained return report=β×1/R idle -α*R wait I.e. the number of client requests R waiting for virtual machine processing after executing different virtual machine management policies wait And idle processing capability R of working virtual machine idle Beta and alpha are corresponding adjustable weight parameters;
6.7, operating the new data center virtual machine in the S state 3 [1..M]＝[X 3 ,X 2 ,X 1 ,…,0]Inputting the Output value into the neural network model, obtaining the Output value in the new state of the neural network model as Output, and obtaining the maximum value Output in the Output max ；
6.8, obtaining three return values Reward under three different strategies by utilizing the method in the step 6.6 1 ，Reward 2 ，Reward 3 Output obtained in step 6.7 max Constructing a data center virtual machine operating state S 2 [1..M]＝[X 2 ,X 1 ,0,…,0]The following is more true of Output ', output' =softmax ([ γ ] Output) max +η*Reward 1 ，γ*Output max +η*Reward 2 ，γ*Output max +η*Reward 3 ]) Where γ is the discount rate parameter and η is the learning rate parameter;
6.9, respectively S 2 [1..M]＝[X 2 ,X 1 ,0,…,0]And Output' as input and Output to construct a new high quality training sampleThe weight of the neural network is adjusted by using a classical neural network backward propagation algorithm;
6.10, circularly executing the steps 6.3 to 6.9, namely generating the next customer request quantity, constructing a new state by right shifting the state array S1. M, calculating the input S1. M and the Output' which are more in line with the actual samples, and adjusting the weight of the neural network by utilizing a classical neural network backward propagation algorithm until the cycle times meet a preset threshold Num, wherein Num=10000 in the common case;
s7, verifying the effect of a neural network model for virtual machine management, wherein the trained neural network model is applied to a small data center, the data center comprises 10 physical machines, each physical machine runs 10 virtual machines at most, and the maximum client request number R which can be processed by each virtual machine max 18000; starting time T of virtual machine on physical machine start 1 minute; average processing time T required by virtual machine of single customer request reqmean For 5 minutes; the data center virtual machine operating state array dimension m=5, where the validation test compares three data center virtual machine management methods:
case1. The virtual machine management method based on the neural network of the patent;
case2, a virtual machine management method adopting a minimum virtual machine starting strategy, namely, no idle virtual machine exists at any moment;
case3, a virtual machine management method adopting a 1-mode redundancy strategy, namely keeping 1 idle virtual machine before a new user request is added;
the data center virtual machine management method verifies that the test duration is 6 months, and the indexes compared by the test comprise: each virtual machine management period T start In, the number R of client requests waiting for virtual machine processing is average waitmean And the idle processing capacity R of the average working virtual machine idlemean 。
2. The data center virtual machine management method for performance and power consumption balancing of claim 1, wherein: in step S2.1, if only the objective in step S2.1 is considered, the data center may use the "start all virtual machines on all physical machines" policy, where the response rate of the user request processing is highest, that is, the user request processing can be performed as long as there is an idle virtual machine, but the policy may cause serious power waste because the idle virtual machine consumes a large amount of power.
3. The data center virtual machine management method for performance and power consumption balancing of claim 1, wherein: in step S2.2, if only the objective in step S2.2 is considered, the data center may "start the minimum number of virtual machines" policy, that is, the number of virtual machines started is processing the client request, and there is no redundant virtual machine, but the policy often reduces the response rate of processing the user request, that is, after a large number of client requests arrive, since there is insufficient processing power, the virtual machine needs to be started, and during the period of starting the virtual machine, the user request is in a waiting state and is not processed in time.
4. The data center virtual machine management method for performance and power consumption balancing of claim 1, wherein: in step S2, the final objective of comprehensively considering the objective in 2.1 and the objective in 2.2 is: through an effective virtual machine management strategy, the reasonable starting number of virtual machines is maintained, the balance between the user response rate of the data center and the virtual machine energy consumption is realized, and the higher response rate of user request processing and lower virtual machine energy consumption are achieved.
CN202010587622.4A 2020-06-24 2020-06-24 Data center virtual machine management method oriented to balance of performance and power consumption Active CN111722912B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010587622.4A CN111722912B (en) 2020-06-24 2020-06-24 Data center virtual machine management method oriented to balance of performance and power consumption

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010587622.4A CN111722912B (en) 2020-06-24 2020-06-24 Data center virtual machine management method oriented to balance of performance and power consumption

Publications (2)

Publication Number Publication Date
CN111722912A CN111722912A (en) 2020-09-29
CN111722912B true CN111722912B (en) 2023-05-02

Family

ID=72568772

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010587622.4A Active CN111722912B (en) 2020-06-24 2020-06-24 Data center virtual machine management method oriented to balance of performance and power consumption

Country Status (1)

Country Link
CN (1) CN111722912B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102662750A (en) * 2012-03-23 2012-09-12 上海交通大学 Virtual machine resource optimal control method and control system based on elastic virtual machine pool
CN110795208A (en) * 2019-10-11 2020-02-14 南京航空航天大学 Mobile cloud computing self-adaptive virtual machine scheduling method based on improved particle swarm

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8185893B2 (en) * 2006-10-27 2012-05-22 Hewlett-Packard Development Company, L.P. Starting up at least one virtual machine in a physical machine by a load balancer

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102662750A (en) * 2012-03-23 2012-09-12 上海交通大学 Virtual machine resource optimal control method and control system based on elastic virtual machine pool
CN110795208A (en) * 2019-10-11 2020-02-14 南京航空航天大学 Mobile cloud computing self-adaptive virtual machine scheduling method based on improved particle swarm

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于负载不确定性的虚拟机整合方法;李双俐;李志华;喻新荣;闫成雨;;计算机应用(06);全文 *

Also Published As

Publication number Publication date
CN111722912A (en) 2020-09-29

Similar Documents

Publication Publication Date Title
Huang et al. Keeping dataset biases out of the simulation: A debiased simulator for reinforcement learning based recommender systems
WO2020237729A1 (en) Virtual machine hybrid standby dynamic reliability assessment method based on mode transfer
FR3080477A1 (en) METHODS AND SYSTEMS FOR DETERMINING AND OPTIMIZING THE PERFORMANCE OF A RESERVOIR SIMULATOR IN A CLOUD COMPUTING ENVIRONMENT
CN110781947B (en) Power load prediction model training and power load prediction method and device
CN110413657B (en) Average response time evaluation method for seasonal non-stationary concurrency
CN109445906B (en) Method for predicting quantity of virtual machine demands
JP6718500B2 (en) Optimization of output efficiency in production system
CN103489034A (en) Method and device for predicting and diagnosing online ocean current monitoring data
CN110543978A (en) Traffic flow data prediction method and device based on wavelet neural network
CN115983485A (en) Prediction method and system for carbon emission of power plant
Li et al. Resource usage prediction based on BiLSTM-GRU combination model
CN111722912B (en) Data center virtual machine management method oriented to balance of performance and power consumption
CN112330112A (en) User energy evaluation method of new energy combined cooling heating and power system based on cloud model
CN115528750B (en) Power grid safety and stability oriented data model hybrid drive unit combination method
CN114648178B (en) Operation and maintenance strategy optimization method of electric energy metering device based on DDPG algorithm
CN104537224B (en) Multi-state System Reliability analysis method and system based on adaptive learning algorithm
US20230004870A1 (en) Machine learning model determination system and machine learning model determination method
CN115938496A (en) Quality Estimation Method Based on XGBoost Algorithm
CN113259163B (en) Web service quality prediction method and system based on network topology perception
Sun et al. Deep reinforcement-learning-guided backup for energy harvesting powered systems
Yu et al. A combination forecasting model of extreme learning machine based on genetic algorithm optimization
CN116402249B (en) Recommendation system overflow effect evaluation method, recommendation system overflow effect evaluation equipment and storage medium
Jin et al. Application Load prediction model based on SA-IPSO-BiLSTM
CN115022194B (en) Network security situation prediction method based on SA-GRU
CN112328355B (en) Adaptive optimal memory reservation estimation method for long-life container

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant