CN106970824B - Virtual machine migration compression method and system based on bandwidth sensing - Google Patents
Virtual machine migration compression method and system based on bandwidth sensing Download PDFInfo
- Publication number
- CN106970824B CN106970824B CN201710129704.2A CN201710129704A CN106970824B CN 106970824 B CN106970824 B CN 106970824B CN 201710129704 A CN201710129704 A CN 201710129704A CN 106970824 B CN106970824 B CN 106970824B
- Authority
- CN
- China
- Prior art keywords
- compression
- migration
- speed
- virtual machine
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
- G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/563—Data redirection of data network streams
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Abstract
the invention discloses a virtual machine migration compression method and system based on bandwidth perception, and belongs to the field of computer virtualization. The method comprises the steps of detecting network bandwidth by using a preset frequency, calculating a migration speed by using each pair of compression ratio and compression speed in a bandwidth and compression strategy table, selecting a compression method corresponding to the maximum migration speed for compression and migration, merging a plurality of memory pages into one data packet before memory data compression, and performing overall compression and migration on the data packet until the compression and migration are completed; the technical scheme of the invention enables the migration system to obtain higher migration speed according to the bandwidth dynamic adjustment compression method, thereby obtaining shorter migration time, reducing data transmission quantity and saving network resources.
Description
Technical Field
The invention belongs to the field of computer virtualization, and particularly relates to a virtual machine migration compression method and system based on bandwidth perception.
background
in recent years, with the development of cloud computing and virtualization technologies, virtual machines are being more and more widely deployed in data centers and cluster environments. Because the virtual machine can carry out abstract simulation on computer resources and simulate virtual hardware resources on the basis of the existing computer hardware resources, the virtual machine has the advantages of simulating different platforms, improving the utilization rate of the computer resources, facilitating management, isolating application and the like.
Virtual machine migration refers to migration of virtual machines among different physical hosts while ensuring normal operation of services in the virtual machines. To ensure availability of virtual machine services during migration, the migration process has very short downtime. Since the down time for the switchover is very short, the user does not perceive the interruption of service, and the migration process is transparent to the user. The virtual machine migration is suitable for a plurality of scenes such as load balancing, energy saving and system maintenance of a data center, and therefore is a very important characteristic of a virtualization technology.
the virtual machine migration is usually performed in a local area network, and in such an environment, the virtual machine accesses the external memory in a shared storage manner, so that only the memory data of the virtual machine and the device states such as the virtual cpu need to be migrated, and the virtual machine memory accounts for most of the data to be migrated. The pre-copy approach is the most dominant migration algorithm widely adopted by each virtualization platform. The migration process of pre-copy is to copy the complete memory image of the virtual machine to the target host first. In this process, since the virtual machine is still running, a part of the memory pages may be modified, and these modified memory dirty pages need to be transmitted to the target host again in the next iteration. Dirty pages generated in each iteration process need to be retransmitted in the next iteration process, so that the consistency of the memory state is ensured. Through multiple iterations, the number of the remaining dirty pages is small, and when the number of the remaining dirty pages reaches a preset threshold value, halt copying can be performed, and the iterative copying process is ended.
Although the existing pre-copy migration method can realize short downtime, the following problems exist: because the memory data needs multiple rounds of iterative transmission, the network transmission data volume is large, and the migration time is long; in addition, if the load running in the virtual machine is write-memory intensive, the speed of dirtying the memory of the virtual machine may be too high, and the migration manner of the pre-copy cannot be normally converged to enter the shutdown copy stage, so that the migration process cannot be normally completed. These problems greatly affect the performance of virtual machine migration, resulting in failure to achieve the intended effect when virtual machine migration techniques are used in a data center.
Disclosure of Invention
the invention aims to detect the compression rate and the compression speed of a plurality of compression methods respectively used for a plurality of typical loads, establish a compression cable policy table, sense the bandwidth at a preset frequency, calculate the migration speed corresponding to each compression method under the current bandwidth, and perform compression migration by using the compression method corresponding to the maximum migration speed, thereby solving the technical problem of conventional compression migration.
In order to achieve the above object, according to an aspect of the present invention, a virtual machine migration compression method based on bandwidth sensing is provided, where the method detects a network bandwidth at a preset frequency, calculates a migration speed by using each pair of compression ratio and compression speed in a bandwidth and compression policy table, and selects a compression method corresponding to a maximum migration speed to perform compression migration on current memory data of a virtual machine.
Further, the method of the invention specifically comprises the following steps:
(1) Monitoring network bandwidth to obtain real-time network bandwidth St which can be utilized by virtual machine migration;
(2) utilizing the compression ratio rho corresponding to each compression method in the compression strategy tableiAnd a compression speed SciCalculating migration velocity Smgti，
Smgti＝min(Sci,St×ρi)，
obtaining a plurality of migration speeds, and comparing to obtain the maximum migration speed;
(3) And finding out the compression rate and the compression speed of the maximum migration speed, and performing compression migration on the current memory data of the virtual machine by using the corresponding compression method in the compression policy table.
Further, the compression policy table is obtained by adopting the following method in advance:
Selecting a plurality of typical loads to sequentially operate in a virtual machine under the operation environment of a data center, respectively carrying out compression detection on memory data by using a plurality of compression methods, obtaining a pair of compression ratio and compression speed by each compression method, and forming a compression strategy table by all the compression methods and the compression ratios and the compression speeds corresponding to the compression methods.
Further, the method for obtaining the compression policy table specifically includes the following sub-steps:
(31) Selecting a typical load in a data center, and operating the typical load in a virtual machine;
(32) selecting one compression method for compression migration, setting all memory pages as dirty pages after each compression, and replacing the other compression method for compression migration, wherein m compression modes are iterated for m times; recording the total compression time and the compressed data size required by each round of compression;
(33) Replacing another typical load, and returning to the step (31) until the compression of the typical loads in the n data centers is completed;
(34) Calculating the compression ratio rho of the ith compression method to the jth loadij，
ρijdata size before compression/data size after compression,
calculating the compression speed Sc of the ith compression method for the jth loadij，
Scijdata size before compression/total time of compression;
Wherein i is more than or equal to 1 and less than or equal to m; j is more than or equal to 1 and less than or equal to n;
(35) calculating the average compression rate rho of the ith compression method for n loadsi，
ρi＝(ρi1+ρi2+…+ρin)/n，
calculating the average compression speed Sc of the ith compression method for n loadsi，
Sci＝(Sci1+Sci2+…+Scin)/n
M pairs of average compression ratios rho corresponding to m compression methods are obtainediand average compression speed Scithe compression method and the corresponding average compression rate and average compression speed jointly form a compression strategy table.
further, the method of the present invention further comprises a merging step:
and (3) merging steps: before the memory data compression is carried out, a plurality of memory pages are merged into one data packet, and then the data packet is compressed integrally.
according to another aspect of the present invention, a virtual machine migration compression system based on bandwidth sensing is provided, where the system is configured to detect a network bandwidth at a preset frequency, calculate a migration speed by using each pair of compression rate and compression speed in a bandwidth and compression policy table, and select a compression method corresponding to a maximum migration speed to perform compression migration on current memory data of a virtual machine.
Further, the system of the invention specifically comprises the following parts:
the bandwidth detection module is used for monitoring the network bandwidth and acquiring the real-time network bandwidth St which can be utilized by the virtual machine migration;
a migration velocity calculation module for utilizing the compression ratios rho corresponding to the various compression methods in the compression policy tableiAnd a compression speed SciCalculating migration velocity Smgti，
Smgti＝min(Sci,St×ρi)，
obtaining a plurality of migration speeds, and comparing to obtain the maximum migration speed;
And the compression migration module is used for finding out the compression rate and the compression speed of the maximum migration speed and performing compression migration on the current memory data of the virtual machine by using the compression method corresponding to the compression migration rate and the compression speed in the compression policy table.
further, the compression policy table is obtained by adopting the following modules in advance:
and the compression strategy table module is used for selecting a plurality of typical loads to sequentially operate in the virtual machine under the operation environment of the data center, respectively carrying out compression detection on the memory data by using a plurality of compression methods, wherein each compression method obtains a pair of compression rate and compression speed, and all the compression methods and the compression rates and the compression speeds corresponding to the compression methods form the compression strategy table.
Further, the compression policy table module specifically includes the following components:
the load operation unit is used for selecting a typical load in the data center and operating the typical load in the virtual machine;
the iteration compression unit is used for selecting one compression method for compression and migration, setting all the memory pages as dirty pages after each compression, and replacing the other compression method for compression and migration, wherein m compression methods are used for iterating m times; recording the total compression time and the compressed data size required by each round of compression;
The load replacing unit is used for replacing another typical load and returning to the load operation unit until the compression of the typical loads in the n data centers is completed;
a calculation unit for calculating a compression ratio ρ of the ith compression method to the jth loadij，
ρijdata size before compression/data size after compression,
Calculating the compression speed Sc of the ith compression method for the jth loadij，
ScijData size before compression/total time of compression;
wherein i is more than or equal to 1 and less than or equal to m; j is more than or equal to 1 and less than or equal to n;
A policy table constructing unit for calculating the average compression ratio rho of the ith compression method for the n loadsi，
ρi＝(ρi1+ρi2+…+ρin)/n，
Calculating the average compression speed Sc of the ith compression method for n loadsi，
Sci＝(Sci1+Sci2+…+Scin)/n
Obtaining m compression methodsM to average compression ratio ρiAnd average compression speed Scithe compression method and the corresponding average compression rate and average compression speed jointly form a compression strategy table.
Further, the system of the present invention further comprises a merging module:
the merging module is used for merging a plurality of memory pages into one data packet before the memory data compression, and then integrally compressing the data packet.
Generally, compared with the prior art, the technical scheme of the invention has the following technical characteristics and beneficial effects:
(1) According to the method, the compression method is dynamically adjusted according to the migration real-time bandwidth, compared with the method using a single fixed compression method, the method can obtain shorter migration time, and when the network bandwidth is higher, the method can select a compression method which is quick and has lower compression ratio; on the contrary, when the bandwidth is lower, a slow compression method with a higher compression ratio is selected, and the dynamic adjustment process can enable the migration system to obtain a higher throughput rate, so that a shorter migration time is obtained;
(2) Because the data content in the memory of the virtual machine is greatly different when different loads operate in the virtual machine, the obtained compression ratio and the compression speed have certain difference when the memory of different loads is compressed by the same compression method, the average compression ratio and the compression speed are obtained through a plurality of typical loads, so that a method for forming a compression strategy table is formed, the adoption of different compression strategy tables for different loads is avoided, and the specific implementation of the method is more feasible and simple;
(3) The method for packing and compressing the memory pages of the virtual machines provided by the invention excavates the characteristic that the compression window of the compression algorithm is far larger than that of a single memory page, and the compression algorithm searches redundant data in the range of the compression window to compress, so that the compression rate of memory data can be further improved by the larger compression granularity provided by the invention, thereby reducing the data transmission quantity and saving network resources.
drawings
FIG. 1 is a schematic flow diagram of the process of the present invention;
FIG. 2 is a system block diagram of a method embodiment of the present invention;
FIG. 3 is a graph showing the variation of compression rate and compression speed of LZ4 compression algorithm;
FIG. 4a is a graph illustrating a comparison of migration times for a prior art pre-copy migration using the method of the present invention;
FIG. 4b is a graph illustrating a comparison of migration data volumes using the method of the present invention and an existing pre-copy migration.
Detailed Description
in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
as shown in fig. 1, the method of the present invention detects a network bandwidth with a preset frequency, calculates a migration speed by using each pair of compression ratio and compression speed in a bandwidth and compression policy table, and selects a compression method corresponding to a maximum migration speed to perform compression migration on current memory data of a virtual machine; at an emigration end, merging a plurality of memory pages into a data packet before compression, and then integrally compressing the data packet; and at the immigration end, decompressing the received network data packet, thereby obtaining the memory page of the virtual machine.
The method comprises the following specific substeps:
(1) monitoring network bandwidth to obtain real-time network bandwidth St which can be utilized by virtual machine migration;
(2) utilizing the compression ratio rho corresponding to each compression method in the compression strategy tableiand a compression speed SciCalculating migration velocity Smgti，
Smgti＝min(Sci,St×ρi)，
obtaining a plurality of migration speeds, and comparing to obtain the maximum migration speed;
(3) finding out the compression ratio and the compression speed of the maximum migration speed, and performing compression migration on the current memory data of the virtual machine by using the corresponding compression method in the compression policy table;
Setting a buffer area at the emigration end, firstly copying a memory page of the virtual machine to be transmitted into the buffer area, compressing the whole buffer area once when the buffer area is full, and transmitting a compressed data packet to the emigration end;
setting a buffer area with the same size as that of the migration end at the migration end, placing memory pages obtained after decompressing the received data packets into the buffer area, and then placing each memory page into a corresponding virtual machine address space;
The compression method selected in this way is the one that maximizes the migration speed at the current stage; the compression method is dynamically adjusted according to the network bandwidth, so that the compression method with the maximum throughput rate at the current stage is used at each stage of the whole migration process, and the migration time is remarkably shortened.
The compression policy table in the above content is obtained by adopting the following method in advance: selecting a plurality of typical loads to sequentially operate in a virtual machine under the operation environment of a data center, respectively performing compression detection on memory data by using a plurality of compression methods, obtaining a pair of compression ratio and compression speed by each compression method, and forming a compression strategy table by all the compression methods and the compression ratios and the compression speeds corresponding to the compression methods
the method comprises the following steps:
(31) selecting a typical load in a data center, and operating the typical load in a virtual machine;
(32) selecting a compression method for compression migration, setting a buffer area before compression, firstly copying memory pages of a virtual machine to be transmitted into the buffer area, compressing the whole buffer area once after the buffer area is full, setting all the memory pages as dirty pages after each compression, replacing another compression method for compression migration, and iterating m compression modes in total; recording the total compression time and the compressed data size required by each round of compression;
Therefore, the modification of the pre-copy migration can enable all memory pages in the address space of the virtual machine to be transmitted for multiple times, and a compression method is selected to measure the compression rate and the compression speed during each transmission, so that the compression rate and the compression speed of all compression methods to a certain load can be measured only by one migration operation, and the process of obtaining the compression policy table is simplified;
in the embodiment, an LZ4 algorithm is adopted, 16 different LZ4 acceleration values are selected, and are respectively 1,3,5 and 7 … … 31, so that 16 compression methods are formed, and because the difference between the compression ratio and the compression speed generated by two adjacent LZ4 acceleration values (such as 1 and 2) is small, the embodiment chooses to adopt 2 as the step number to select the acceleration value;
(33) Replacing another typical load, and returning to the step (31) until the compression of the typical loads in the n data centers is completed;
(34) Calculating the compression ratio rho of the ith compression method to the jth loadij，
ρijData size before compression/data size after compression,
Calculating the compression speed Sc of the ith compression method for the jth loadij，
ScijData size before compression/total time of compression;
wherein i is more than or equal to 1 and less than or equal to m; j is more than or equal to 1 and less than or equal to n;
(35) calculating the average compression rate rho of the ith compression method for n loadsi，
ρi＝(ρi1+ρi2+…+ρin)/n，
Calculating the average compression speed Sc of the ith compression method for n loadsi，
Sci＝(Sci1+Sci2+…+Scin)/n
m pairs of average compression ratios rho corresponding to m compression methods are obtainediand average compression speed Scithe compression method and the corresponding average compression rate and average compression speed jointly form a compression strategy table.
As shown in fig. 2, in this embodiment, the embodiment of the present invention is implemented on a KVM/QEMU open source virtualization platform, and at an egress end, a bandwidth monitoring module obtains a current migration network bandwidth at a frequency of 1 time per 1 second and submits the current migration network bandwidth to a bandwidth sensing and compressing module; after the compression module obtains the network bandwidth, a compression method is dynamically adjusted by using a compression policy table; meanwhile, a buffer area is arranged at the emigration end, the memory page of the virtual machine to be transmitted is firstly copied into the buffer area, and when the buffer area is full, the data in the buffer area is packed and compressed; transmitting the compressed data packet to an immigration end; and after the migration end receives the packed and compressed data into the buffer area, the decompression module decompresses the packed and compressed data and submits the obtained memory pages to the corresponding virtual machine address space.
The method for dynamically adjusting the compression according to the migration real-time bandwidth can greatly improve the throughput rate of a migration system, thereby obviously shortening the migration time. The compression rates and compression speeds of different compression algorithms are different, in general, a compression algorithm with a high compression rate has a lower compression speed, and vice versa; in addition, many compression algorithms provide parameters for user selection to adjust between compression rate and compression speed. Taking the LZ4 compression algorithm used in the implementation of the present invention as an example, the algorithm provides an acceleration value parameter, and fig. 3 shows the change of the compression effect of LZ4 on the memory of the virtual machine when different acceleration values are used. As the acceleration value becomes larger, the compression speed is increased continuously at the cost of a continuous decrease in compression ratio.
as can be seen from this, in a certain bandwidth, a compression method with a different compression rate and compression speed is used in the migration, and in this embodiment, the LZ4 algorithm with a different acceleration value is used, and the system migration speed is different. The method provided by the invention can find out the compression method which can bring the maximum migration speed to the migration system under a certain bandwidth, and dynamically selects the optimal compression method along with the change of the bandwidth, thereby obtaining the shortest migration time.
the invention obtains the average compression rate and the compression speed through a plurality of typical loads, thereby forming the method of the compression strategy table, avoiding adopting different compression strategy tables for different loads and ensuring that the specific implementation of the invention is simpler and more feasible. When different loads run in a virtual machine, the data content in the memory of the virtual machine is different, so when the memory of different loads is compressed by using LZ4 with a certain acceleration value, the obtained compression rate and the compression speed have certain difference, and the migration speed of the system is different. However, our experiments show that different loads achieve maximum system migration velocities at the same or nearby acceleration values. Therefore, the invention provides a method for selecting a plurality of typical loads, measuring the compression rates and the compression speeds of the typical loads under different compression methods, averaging the data to obtain the average compression speed and the compression rate, forming a compression strategy table, and calculating to obtain the optimal compression method under a certain bandwidth. Experiments show that the compression strategy table obtained in the way is suitable for all loads within a certain error range. Therefore, the invention only needs to store one group of compression strategy table during implementation, thereby simplifying the system implementation and improving the feasibility of the invention.
As shown in FIG. 4a, compared with the migration time of the existing pre-copy migration, the migration time of the method of the present invention is less;
Compared with the migration data volume of the existing pre-copy migration, the migration data volume of the method of the present invention is smaller than that of the existing pre-copy migration, as shown in fig. 4 b.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (4)
1. A virtual machine migration compression method based on bandwidth perception is characterized in that the method of the invention is as follows: detecting network bandwidth at a preset frequency, calculating a migration speed by using the bandwidth and each pair of compression ratio and compression speed in a compression policy table, and selecting a compression method corresponding to the maximum migration speed to perform compression migration on the current memory data of the virtual machine;
the method specifically comprises the following steps:
(1) monitoring network bandwidth to obtain real-time network bandwidth St which can be utilized by virtual machine migration;
(2) Utilizing the compression ratio rho corresponding to each compression method in the compression strategy tableiand a compression speed Scicalculating migration velocity Smgti，
Smgti＝min(Sci,St×ρi)，
obtaining a plurality of migration speeds, and comparing to obtain the maximum migration speed;
(3) Finding out the compression ratio and the compression speed of the maximum migration speed, and performing compression migration on the current memory data of the virtual machine by using the corresponding compression method in the compression policy table;
The compression policy table is obtained by adopting the following method in advance:
selecting a plurality of typical loads to sequentially operate in a virtual machine under the operation environment of a data center, and respectively carrying out compression detection on memory data by using a plurality of compression methods, wherein each compression method obtains a pair of compression rate and compression speed, and all the compression methods and the compression rates and the compression speeds corresponding to the compression methods form a compression strategy table;
the method for obtaining the compression policy table specifically comprises the following sub-steps:
(31) selecting a typical load in a data center, and operating the typical load in a virtual machine;
(32) Selecting one compression method for compression migration, setting all memory pages as dirty pages after each compression, and replacing the other compression method for compression migration, wherein m compression modes are iterated for m times; recording the total compression time and the compressed data size required by each round of compression;
(33) Replacing another typical load, and returning to the step (31) until the compression of the typical loads in the n data centers is completed;
(34) calculating the compression ratio rho of the ith compression method to the jth loadij，
ρijData size before compression/data size after compression,
calculating the compression speed Sc of the ith compression method for the jth loadij，
Scijdata size before compression/total time of compression;
wherein i is more than or equal to 1 and less than or equal to m; j is more than or equal to 1 and less than or equal to n;
(35) Calculating the average compression rate rho of the ith compression method for n loadsi，
ρi＝(ρi1+ρi2+…+ρin)/n，
Calculating the average compression speed Sc of the ith compression method for n loadsi，
Sci＝(Sci1+Sci2+…+Scin)/n
m pairs of average compression ratios rho corresponding to m compression methods are obtainediAnd average compression speed SciThe compression method and the corresponding average compression rate and average compression speed jointly form a compression strategy table.
2. the virtual machine migration compression method based on bandwidth awareness as claimed in claim 1, wherein the method further comprises a merging step:
And (3) merging steps: before the memory data compression is carried out, a plurality of memory pages are merged into one data packet, and then the data packet is compressed integrally.
3. A virtual machine migration compression system based on bandwidth perception is characterized in that: the system is used for detecting the network bandwidth at a preset frequency, calculating the migration speed by utilizing each pair of compression ratio and compression speed in the bandwidth and compression strategy table, and selecting the compression method corresponding to the maximum migration speed to perform compression migration on the current memory data of the virtual machine; the method specifically comprises the following steps:
The bandwidth detection module is used for monitoring the network bandwidth and acquiring the real-time network bandwidth St which can be utilized by the virtual machine migration;
a migration velocity calculation module for utilizing the compression ratios rho corresponding to the various compression methods in the compression policy tableiAnd a compression speed SciCalculating migration velocity Smgti，
Smgti＝min(Sci,St×ρi)，
obtaining a plurality of migration speeds, and comparing to obtain the maximum migration speed;
The compression migration module is used for finding out the compression rate and the compression speed of the maximum migration speed and performing compression migration on the current memory data of the virtual machine by using a corresponding compression method in the compression policy table;
The compression strategy table is obtained by adopting the following modules in advance:
The compression strategy table module is used for selecting a plurality of typical loads to sequentially operate in the virtual machine under the operation environment of the data center, and respectively carrying out compression detection on the memory data by using a plurality of compression methods, wherein each compression method obtains a pair of compression rate and compression speed, and all the compression methods and the compression rates and the compression speeds corresponding to the compression methods form a compression strategy table;
The compression policy table module specifically comprises the following parts:
the load operation unit is used for selecting a typical load in the data center and operating the typical load in the virtual machine;
The iteration compression unit is used for selecting one compression method for compression and migration, setting all the memory pages as dirty pages after each compression, and replacing the other compression method for compression and migration, wherein m compression methods are used for iterating m times; recording the total compression time and the compressed data size required by each round of compression;
the load replacing unit is used for replacing another typical load and returning to the load operation unit until the compression of the typical loads in the n data centers is completed;
A calculation unit for calculating a compression ratio ρ of the ith compression method to the jth loadij，
ρijData size before compression/data size after compression,
Calculating the compression speed Sc of the ith compression method for the jth loadij，
ScijData size before compression/total time of compression;
Wherein i is more than or equal to 1 and less than or equal to m; j is more than or equal to 1 and less than or equal to n;
a policy table constructing unit for calculating the average compression ratio rho of the ith compression method for the n loadsi，
ρi＝(ρi1+ρi2+…+ρin)/n，
calculating the average compression speed Sc of the ith compression method for n loadsi，
Sci＝(Sci1+Sci2+…+Scin)/n
m pairs of average compression ratios rho corresponding to m compression methods are obtainediAnd average compression speed SciThe compression method and the corresponding average compression rate and average compression speed jointly form a compression strategy table.
4. The system according to claim 3, wherein the system further comprises a merging module:
the merging module is used for merging a plurality of memory pages into one data packet before the memory data compression, and then integrally compressing the data packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710129704.2A CN106970824B (en) | 2017-03-07 | 2017-03-07 | Virtual machine migration compression method and system based on bandwidth sensing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710129704.2A CN106970824B (en) | 2017-03-07 | 2017-03-07 | Virtual machine migration compression method and system based on bandwidth sensing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106970824A CN106970824A (en) | 2017-07-21 |
| CN106970824B true CN106970824B (en) | 2019-12-17 |
Family
ID=59329092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710129704.2A Active CN106970824B (en) | 2017-03-07 | 2017-03-07 | Virtual machine migration compression method and system based on bandwidth sensing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106970824B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108469982B (en) * | 2018-03-12 | 2021-03-26 | 华中科技大学 | Online migration method for container |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609361A (en) * | 2012-01-16 | 2012-07-25 | 北京邮电大学 | Method and device for transferring storage data of virtual machine |
| CN103353850A (en) * | 2013-06-13 | 2013-10-16 | 华为技术有限公司 | Virtual machine thermal migration memory processing method, device and system |
| CN103685256A (en) * | 2013-12-06 | 2014-03-26 | 华为技术有限公司 | Virtual machine migration management method, device and system |
| CN105302494A (en) * | 2015-11-19 | 2016-02-03 | 浪潮(北京)电子信息产业有限公司 | Compression strategy selecting method and device |
| US9336042B1 (en) * | 2015-11-19 | 2016-05-10 | International Business Machines Corporation | Performing virtual machine live migration within a threshold time by adding available network path in multipath network |
| CN106293870A (en) * | 2015-06-29 | 2017-01-04 | 联发科技股份有限公司 | Computer system and strategy thereof guide compression method |
-
2017
- 2017-03-07 CN CN201710129704.2A patent/CN106970824B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102609361A (en) * | 2012-01-16 | 2012-07-25 | 北京邮电大学 | Method and device for transferring storage data of virtual machine |
| CN103353850A (en) * | 2013-06-13 | 2013-10-16 | 华为技术有限公司 | Virtual machine thermal migration memory processing method, device and system |
| CN103685256A (en) * | 2013-12-06 | 2014-03-26 | 华为技术有限公司 | Virtual machine migration management method, device and system |
| CN106293870A (en) * | 2015-06-29 | 2017-01-04 | 联发科技股份有限公司 | Computer system and strategy thereof guide compression method |
| CN105302494A (en) * | 2015-11-19 | 2016-02-03 | 浪潮(北京)电子信息产业有限公司 | Compression strategy selecting method and device |
| US9336042B1 (en) * | 2015-11-19 | 2016-05-10 | International Business Machines Corporation | Performing virtual machine live migration within a threshold time by adding available network path in multipath network |
Non-Patent Citations (2)
| Title |
|---|
| Petter Svärd,et,al.Evaluation of Delta Compression Techniques for Efficient Live Migration of Large Virtual Machines.《Proceedings of the 7th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments》.2011,Pages 111-120. * |
| 云计算环境下虚拟机在线迁移优化;马路路等;《信息技术》;20141130(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106970824A (en) | 2017-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113011602B (en) | Federal model training method and device, electronic equipment and storage medium | |
| US10289451B2 (en) | Method, apparatus, and system for adjusting deployment location of virtual machine | |
| US10209908B2 (en) | Optimization of in-memory data grid placement | |
| US9632839B2 (en) | Dynamic virtual machine consolidation | |
| US9477532B1 (en) | Graph-data partitioning for workload-balanced distributed computation with cost estimation functions | |
| CN110798517B (en) | Decentralized cluster load balancing method and system, mobile terminal and storage medium | |
| US10165048B2 (en) | Data copy storage method, terminal apparatus, and server apparatus | |
| WO2018000991A1 (en) | Data balancing method and device | |
| US20180285294A1 (en) | Quality of service based handling of input/output requests method and apparatus | |
| EP3014444A1 (en) | Computing connected components in large graphs | |
| CN108196935B (en) | Cloud computing-oriented virtual machine energy-saving migration method | |
| US20160328257A1 (en) | Probabilistic deduplication-aware workload migration | |
| US20190356530A1 (en) | Moving nodes in a distributed system | |
| CN107220108B (en) | Method and system for realizing load balance of cloud data center | |
| US7817562B1 (en) | Methods and systems for back end characterization using I/O sampling | |
| US20220413965A1 (en) | Data recovery method, apparatus and device, and readable storage medium | |
| US10866934B1 (en) | Token-based data flow control in a clustered storage system | |
| CN110597598B (en) | Control method for virtual machine migration in cloud environment | |
| CN106970824B (en) | Virtual machine migration compression method and system based on bandwidth sensing | |
| CN112988892B (en) | Distributed system hot spot data management method | |
| CN105208096A (en) | Distributed cache system and method | |
| CN111400241B (en) | Data reconstruction method and device | |
| TW201327205A (en) | Managing method for hardware performance and cloud computing system | |
| CN116962441A (en) | Cloud storage control method and device and cloud storage system | |
| JP2016010124A (en) | Management device, management program, and information processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |