CN104156255A - Virtual machine migration method, virtual machine migration device and source physical host - Google Patents

Virtual machine migration method, virtual machine migration device and source physical host Download PDF

Info

Publication number
CN104156255A
CN104156255A CN201410374581.5A CN201410374581A CN104156255A CN 104156255 A CN104156255 A CN 104156255A CN 201410374581 A CN201410374581 A CN 201410374581A CN 104156255 A CN104156255 A CN 104156255A
Authority
CN
China
Prior art keywords
bandwidth
page
virtual machine
main frame
target physical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410374581.5A
Other languages
Chinese (zh)
Other versions
CN104156255B (en
Inventor
藏洪永
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Gaohang Intellectual Property Operation Co ltd
Haining hi tech Zone Science and Innovation Center Co.,Ltd.
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201410374581.5A priority Critical patent/CN104156255B/en
Publication of CN104156255A publication Critical patent/CN104156255A/en
Application granted granted Critical
Publication of CN104156255B publication Critical patent/CN104156255B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

An embodiment of the invention provides a virtual machine migration method, a virtual machine migration device and a source physical host and relates to the technical field of virtual machine migration, aiming to shortening power-off time of a virtual machine on a target physical host to improve migration performance of the virtual machine. The virtual machine migration method includes that after the virtual machine migrates to the target physical host from the source physical host and recovers to run on the target physical host, the source physical host sends a to-be-synchronized memory page of the virtual machine to the target physical host through a network interface on the basis of a first band width; the source physical host receives a page fault request sent from the target physical host through the network interface; the source physical host finds out the faulted memory page according to information of the faulted memory page contained in the page fault request, and sends the faulted memory page to the target physical host through the network interface according to a second band width; and the source physical host continues to send the to-be-synchronized memory page of the virtual machine to the target physical host through the network interface according to a third band width.

Description

A kind of virtual machine migration method, virtual machine (vm) migration device and source physical host
Technical field
The present invention relates to virtual machine (vm) migration technical field, relate in particular to a kind of virtual machine migration method, virtual machine (vm) migration device and source physical host.
Background technology
Utilize Intel Virtualization Technology can realize the resource management of high efficient and flexible, therefore on cloud platform, be used widely.Virtual machine (vm) migration is a key areas of Intel Virtualization Technology, is widely used in cluster, data center and cloud plateform system, and shared hardware and file system.Virtual machine (vm) migration technology can guarantee the virtual machine moving to move on another physical host from the current physical host moving in unbroken mode, thereby realize the flexible management of virtual machine on cloud platform, for the operations such as cluster on-line maintenance, resource dynamic scheduling, SiteServer LBS provide technical support.
Virtual machine (vm) migration is that virtual machine is moved to the process on target physical main frame from source physical host.At present, conventional rear copy virtual machine (vm) migration technology mainly comprises two stages:
First stage, source physical host stops the operation of virtual machine, and the critical data of virtual machine (as the data such as status information, facility information of CPU) is transferred to target physical main frame.
Subordinate phase, the virtual machine on target physical main frame after operation migration.In this one-phase, between source physical host and target physical main frame, can carry out the page transmission of two kinds of forms.A kind of form is: source physical host sends not synchronous page (this process is called as: backstage active transmission) by network interface between the two to target physical main frame; Another kind of form is: move the virtual machine after migration on target physical main frame time; if it (is the page that virtual machine need to be used that generation internal memory skips leaf; not yet be sent to target physical main frame from source physical host); virtual machine is shut down; and send page faults by target physical main frame to source physical host; source physical host sends the page of this page faults instruction to target physical main frame by network interface, after target physical main frame is received this disappearance page, virtual machine restarts.
Transmitted data amount during due to backstage active transmission is larger, and the transmission time is longer, thereby backstage active transmission tends to take more or whole bandwidth between source physical host and target physical main frame.Bandwidth block up will cause target physical main frame receive that the time of disappearance page is longer, and then the time that virtual machine on target physical main frame is shut down is also longer.And one of key index of weighing virtual machine (vm) migration is exactly the stop time of virtual machine, stop time is longer, and the migration performance of virtual machine is poorer.
Summary of the invention
The embodiment of the present invention provides a kind of virtual machine migration method, virtual machine (vm) migration device and source physical host; ensureing under the prerequisite of virtual machine (vm) migration T.T. in zone of reasonableness; reduce the stop time of virtual machine on target physical main frame, thereby improve the migration performance of virtual machine.
For achieving the above object, the embodiment of the present invention adopt technical scheme be,
First aspect, discloses a kind of virtual machine migration method, and described method is applied to source physical host, comprising:
Virtual machine moves to target physical main frame from source physical host, and after resuming operation on described target physical main frame, and what described source physical host utilized by network interface that the first bandwidth sends virtual machine to target physical main frame treats isochronous memory page; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Described source physical host receives by described network interface the page faults that described target physical main frame sends; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
Described source physical host, according to the information of the described disappearance page comprising in described page faults, finds described disappearance page, utilizes the second bandwidth to send described disappearance page to described target physical main frame by described network interface; What described source physical host utilized by described network interface that the 3rd bandwidth continues to send virtual machine to described target physical main frame treats isochronous memory page, and wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
In conjunction with first aspect, in the possible implementation of the first of first aspect,
Described method also comprises:
Described source physical host is determined the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
In conjunction with the possible implementation of the first of first aspect, in the possible implementation of the second of first aspect, described source physical host is according to bandwidth self-adaption allocation strategy, parameter preset M and parameter preset N, determine the ratio of described the second bandwidth in described the first bandwidth, obtain described the second bandwidth, and determine the ratio of described the 3rd bandwidth in described the first bandwidth, and obtain described the 3rd bandwidth, specifically comprise:
Described source physical host determines that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, and determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
In conjunction with first aspect, in the third possible implementation of first aspect,
Described method also comprises:
Described source physical host is according to the information of the described disappearance page comprising in described page faults, find described disappearance page, utilize the second bandwidth to send after described disappearance page to described target physical main frame by described network interface, what described source physical host utilized by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
In conjunction with first aspect, first aspect first to the third possible implementation, in the 4th kind of possible implementation of first aspect,
Described source physical host utilize by network interface the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, described method also comprises:
Described source physical host records when described virtual machine (vm) migration is front to be moved on the physical host of described source, the number of times that each page is accessed or frequency values;
Correspondingly, described source physical host utilizes described the first bandwidth to send the isochronous memory page for the treatment of of virtual machine to target physical main frame by network interface to comprise: described source physical host is according to the descending order of the accessed number of times of described each page or frequency values, and that utilizes by network interface that described the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
In conjunction with the 4th kind of possible implementation of first aspect, in the 5th kind of possible implementation of first aspect,
Described source physical host, according to the descending order of the accessed number of times of described each page or frequency values, utilizes the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, comprising:
When described source physical host utilizes described the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface at every turn, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
In conjunction with first aspect, in the 6th kind of possible implementation of first aspect,
Virtual machine moves to target physical main frame from source physical host, and before resuming operation on described target physical main frame, described method also comprises:
Described source physical host stops the operation of described virtual machine, sends the critical data of described virtual machine to described target physical main frame, so that described virtual machine resumes operation on described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
Second aspect, discloses a kind of virtual machine (vm) migration device, is deployed on the physical host of source, and described virtual machine (vm) migration device comprises:
Background transfer module, treats isochronous memory page for what utilize by network interface that the first bandwidth sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Page faults receiver module, the page faults sending for receive described target physical main frame by described network interface; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
Remote Paging module, for the information of the described disappearance page that comprises according to described page faults, finds described disappearance page, utilizes the second bandwidth to send described disappearance page to described target physical main frame by described network interface; Described background transfer module, also treats isochronous memory page for what utilize the 3rd bandwidth to continue to send virtual machine by described network interface to described target physical main frame, and wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
In conjunction with second aspect, in the possible implementation of the first of second aspect,
Described virtual machine (vm) migration device also comprises: bandwidth self-adaption module,
Described bandwidth self-adaption module is used for, and determines the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
In conjunction with the possible implementation of the first of second aspect, in the possible implementation of the second of second aspect,
Described bandwidth self-adaption module specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, and determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
In conjunction with second aspect, in the third possible implementation of second aspect,
Described background transfer module also for, utilizing the second bandwidth by described network interface when described Remote Paging module sends after described disappearance page to described target physical main frame, that utilizes by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
In conjunction with second aspect, second aspect first to the third possible implementation, in the 4th kind of possible implementation of second aspect,
Described background transfer module also comprises:
Heat page record sub module, for described background transfer module utilize by network interface described the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, when recording described virtual machine and moving on the physical host of described source, the number of times that each page is accessed or frequency values;
The heat page submodule of looking ahead, for the descending order according to the accessed number of times of described each page or frequency values, that utilizes by network interface that described the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
In conjunction with the 4th kind of possible implementation of second aspect, in the 5th kind of possible implementation of second aspect,
The described hot page of submodule of looking ahead, when at every turn utilizing described the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
In conjunction with second aspect, in the 6th kind of possible implementation of second aspect,
Virtual machine state administration module, for stopping the operation of described virtual machine;
Key message sending module, for sending the critical data of described virtual machine, so that described virtual machine resumes operation on described target physical main frame to described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
The third aspect, discloses a provenance physical host, and described source physical host comprises: hardware layer; At the virtual machine monitoring unit VMM of described hardware layer operation; At least one of described VMM controlling run migrates to the virtual machine of described target physical main frame from described source physical host, wherein:
Described VMM, treats isochronous memory page for what utilize by network interface that the first bandwidth sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Receive by described network interface the page faults that described target physical main frame sends; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
According to the information of the described disappearance page comprising in described page faults, find described disappearance page, utilize the second bandwidth to send described disappearance page to described target physical main frame by described network interface; What described source physical host utilized by described network interface that the 3rd bandwidth continues to send virtual machine to described target physical main frame treats isochronous memory page, and wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
In conjunction with the third aspect, in the possible implementation of the first of the third aspect,
Described VMM, also, for determine the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
In conjunction with the possible implementation of the first of the third aspect, in the possible implementation of the second of the third aspect,
Described VMM specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, and determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
In conjunction with the third aspect, in the third possible implementation of the third aspect,
Described VMM, also for the information of the described disappearance page that comprises according to described page faults, find described disappearance page, utilize the second bandwidth to send after described disappearance page to described target physical main frame by described network interface, what described source physical host utilized by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
In conjunction with the third aspect, the third aspect first to the third possible implementation, in the 4th kind of possible implementation of the third aspect,
Described VMM, utilize by network interface described the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, also for recording while moving on the physical host of described source before described virtual machine (vm) migration, the number of times that each page is accessed or frequency values; According to the descending order of the accessed number of times of described each page or frequency values, that utilizes by network interface that the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
In conjunction with the 4th kind of possible implementation of the third aspect, in the 5th kind of possible implementation of the third aspect,
Described VMM, according to the descending order of the accessed number of times of described each page or frequency values, utilize described the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, when at every turn utilizing the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
In conjunction with the third aspect, in the 6th kind of possible implementation of the third aspect,
Described VMM, move to target physical main frame at virtual machine from source physical host, and before resuming operation on described target physical main frame, also for stopping the operation of described virtual machine, send the critical data of described virtual machine to described target physical main frame, so that described virtual machine resumes operation on described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
Virtual machine migration method, virtual machine (vm) migration device and source physical host that the embodiment of the present invention provides, virtual machine moves to target physical main frame from source physical host, and after resuming operation on target physical main frame, source physical host can be to the not synchronous page of target physical main frame transmission.In this course, the scheme of utilizing the embodiment of the present invention to provide, once virtual machine produces and skips leaf on target physical main frame, source physical host will be with the disappearance page of the second bandwidth for transmission page faults instruction, with the not synchronous page of the 3rd bandwidth continuation transmission after receiving page faults.Like this, do not affect too much on the one hand the transmission of not synchronous page, make total duration of virtual machine (vm) migration in the reasonable scope; On the other hand, owing to there being the second special bandwidth, ensure the fast transport of disappearance page from source physical host to target physical main frame, reduced the stop time of virtual machine on target physical main frame, thereby improved on the whole the transport efficiency of virtual machine.Therefore; the embodiment of the present invention can overcome to a certain extent in existing virtual machine migration method, lack page because of bandwidth congestion cause can not fast transport to target physical main frame, and then make the virtual machine problem that waiting time of machine halt is grown on target physical main frame.
Brief description of the drawings
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.
The virtual machine (vm) migration schematic diagram that Fig. 1 a provides for the invention process;
The schematic flow sheet of the virtual machine migration method that Fig. 1 b provides for the invention process;
The schematic flow sheet of a kind of virtual machine migration method that Fig. 2 provides for the embodiment of the present invention 1;
The schematic flow sheet of the another kind of virtual machine migration method that Fig. 3 provides for the embodiment of the present invention 2;
The bandwidth adjusting method schematic diagram that Fig. 4 a provides for the embodiment of the present invention 2;
The another kind of bandwidth adjusting method schematic diagram that Fig. 4 b provides for the embodiment of the present invention 2;
A kind of structured flowchart of the virtual machine (vm) migration device that Fig. 5 provides for the embodiment of the present invention 3;
The another kind of structured flowchart of the virtual machine (vm) migration device that Fig. 6 provides for the embodiment of the present invention 3;
The another kind of structured flowchart of the virtual machine (vm) migration device that Fig. 7 provides for the embodiment of the present invention 3;
The source physical host structured flowchart that Fig. 8 provides for the embodiment of the present invention 4.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiment.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.
To facilitate understanding of the present embodiment of the invention, first introduce at this several key elements that can introduce in embodiment of the present invention description.
Virtual machine: can simulate one or many virtual computing machines by software virtual machine on a physical computer, and these virtual machines can carry out work as real physical computer.
Hardware layer: the hardware platform of virtualized environment operation.Wherein, hardware layer can comprise multiple hardwares, high speed/low speed I/O (the I/O such as such as CPU, internal memory, network interface card, storer, Input/Output) equipment, and there is the miscellaneous equipment of particular procedure function, as input and output memory management unit, (Input/Output Memory Management Unit is called for short: IOMMU), wherein IOMMU can be used for the conversion of virtual machine physical address and host (Host) physical address.
Host (Host): as administration and supervision authorities, in order to complete management, the distribution of hardware resource; For virtual machine presents virtual hardware platform, realize scheduling and the isolation of virtual machine, Host may be monitor of virtual machine (VMM).Wherein, virtual hardware platform provides various hardware resources to each virtual machine of operation on it, as virtual cpu, internal memory, virtual disk, Microsoft Loopback Adapter etc. are provided.Wherein, virtual machine operates on the virtual hardware platform that Host is its preparation, the one or more virtual machines of the upper operation of Host.
As shown in Figure 1a, on physical host 1, move multiple fictitious host computer 1-1,1-2,1-N, on physical host 2, moved multiple virtual machine 2-1,2-M.Wherein, on physical host 1, the virtual machine 1-2 of operation will migrate on physical host 2 from physical host 1, usually, physical host 1 will be called to source physical host, and physical host 2 is called to target physical main frame.
As shown in Figure 1 b, expression is copy migration schematic flow sheet after existing virtual machine.Its process is roughly: first, source physical host stops the operation of virtual machine to be migrated on it.Then, by network interface, the critical data of virtual machine (as facility information of the status information of source physical host CPU, source physical host etc.) is transferred on target physical main frame.Afterwards, target physical main frame recovers the operation of virtual machine on target physical main frame according to above-mentioned critical data.When virtual machine moves on target physical main frame, there will be the problem (being the processing of skipping leaf shown in Fig. 1 b) that lacks page, now between source physical host and target physical main frame, will carry out the transmission of page.This transmitting procedure is divided into two parts, Part I: the page (i.e. not synchronous page) that source physical host is initiatively stored in virtual machine in its region of memory sends to target physical main frame, and this process is also referred to as backstage active transmission.Part II: virtual machine when page, carries out remote request paging in the target physical main frame disappearance that runs in service, and now virtual machine can send page faults to source physical host by target physical main frame.In this page faults, can carry the information of disappearance page, as address information etc.Source physical host receives after this page faults, finds the required page of virtual machine according to the information of disappearance page, and this disappearance page is sent to target physical main frame, and virtual machine is received after this disappearance page, resumed operation.
Usually, the transmitted data amount during due to backstage active transmission is larger, and the transmission time is longer, thereby backstage active transmission tends to take more or whole bandwidth between source physical host and target physical main frame.Blocking up of bandwidth will cause target physical main frame to receive that time of disappearance page of page faults instruction is longer, and then the time of the shutdown of virtual machine on target physical main frame is also longer.And one of key index weighing virtual machine (vm) migration is the stop time of virtual machine, stop time is longer, and the migration performance of virtual machine is poorer; Another is total duration of virtual machine (vm) migration, and total duration is longer, and the migration performance of virtual machine is poorer.
Total duration that following each embodiment of the present invention is intended to ensure virtual machine (vm) migration in the reasonable scope time, reduces the stop time of virtual machine, improves the migration performance of virtual machine.
Embodiment 1:
The embodiment of the present invention provides a kind of virtual machine migration method, is applied to source physical host, and as shown in Figure 2, the method comprises the following steps:
101, virtual machine moves to target physical main frame from source physical host, and after resuming operation on target physical main frame, and what source physical host utilized by network interface that the first bandwidth sends virtual machine to target physical main frame treats isochronous memory page; The isochronous memory page for the treatment of of this virtual machine is: be stored in the physical host of source, be not sent to the page of target physical main frame in virtual machine (vm) migration process.
Preferably, the first bandwidth in this step can be between source physical host and target physical main frame, and current can be used in transmitted whole bandwidth for the treatment of isochronous memory page.For example, current whole free time of the bandwidth of a 100M, this 100M can all treat isochronous memory page for transmission, and the first bandwidth is the bandwidth of this 100M; If the bandwidth of a 100M, the current 10M of having is transmitting other data, has the 90M free time, can all treat isochronous memory page for transmission, and this first bandwidth is the bandwidth of this 90M.
102, the page faults that source physical host sends by network interface receiving target physical host; This page faults includes the information that lacks page, and this disappearance page is that virtual machine (vm) migration arrives after target physical main frame, needs but be not sent to the page of target physical main frame from source physical host after resuming operation.
Preferably, the information of the disappearance page comprising in page faults, can be the address information of disappearance page, can certainly be other information that can find this page, for example, lack the identification information of page.
103, source physical host, according to the information of the disappearance page comprising in page faults, finds this disappearance page, utilizes the second bandwidth to send this disappearance page to target physical main frame by network interface; What source physical host utilized by network interface that the 3rd bandwidth continues to send virtual machine to target physical main frame treats isochronous memory page, and wherein, the second bandwidth, the 3rd bandwidth are all less than the first bandwidth.
Preferably, in this step, source physical host is to utilize the first bandwidth for transmission to treat (being backstage active transmission) of isochronous memory page before the page that sends page faults instruction to target physical main frame.And receive after page faults, while sending the page of page faults instruction to target physical main frame, the current whole bandwidth that can be used in transmission page are divided for the second bandwidth and the 3rd bandwidth, and with the page of the second bandwidth for transmission page faults instruction, proceed backstage active transmission by the 3rd bandwidth.For example, the bandwidth of a 100M, before all for backstage active transmission, now can be with the page of the second bandwidth 50M (can be also other quantitative values) transmission page faults instruction, proceed backstage active transmission with the 3rd bandwidth 50M (can be also other quantitative values).If the bandwidth of a 100M, there is 10M transmitting other data before, there is the 90M free time, this 90M is used for carrying out backstage active transmission, now other data have passed, 100M is all available, now can be with the page of the second bandwidth 50M (can be also other quantitative values) transmission page faults instruction, proceed backstage active transmission with the 3rd bandwidth 50M (can be also other quantitative values).
The virtual machine migration method that the embodiment of the present invention provides, virtual machine moves to target physical main frame from source physical host, and after resuming operation on target physical main frame, source physical host can be to the not synchronous page of target physical main frame transmission.In this course, the scheme of utilizing the embodiment of the present invention to provide, once virtual machine produces and skips leaf on target physical main frame, source physical host will be with the disappearance page of the second bandwidth for transmission page faults instruction, with the not synchronous page of the 3rd bandwidth continuation transmission after receiving page faults.Like this, do not affect too much on the one hand the transmission of not synchronous page, make total duration of virtual machine (vm) migration in the reasonable scope; On the other hand, owing to there being the second special bandwidth, ensure the fast transport of disappearance page from source physical host to target physical main frame, reduced the stop time of virtual machine on target physical main frame, thereby improved on the whole the transport efficiency of virtual machine.Therefore; the embodiment of the present invention can overcome to a certain extent in existing virtual machine migration method, lack page because of bandwidth congestion cause can not fast transport to target physical main frame, and then make the virtual machine problem that waiting time of machine halt is grown on target physical main frame.
Preferably, in the preferred embodiment of the present invention, described source physical host is determined the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth.
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section; Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth; Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth; Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
Preferably, in the preferred embodiment of the present invention, source physical host determines that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
Preferably, in the preferred embodiment of the present invention, source physical host is according to the information of the disappearance page comprising in page faults, find disappearance page, utilize the second bandwidth to send after this disappearance page to target physical main frame by network interface, what source physical host utilized by network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page.
At this, suppose the bandwidth of a 100M, wherein 50M (i.e. the second bandwidth) is for the page of transmission deletion page instruction, another 50M (i.e. the 3rd bandwidth) is for transmitting not synchronous page, send after this disappearance page at physical host, if this 100M bandwidth is used in transmitting page, while not having other data to transmit, source physical host recovers with the not synchronous page (carrying out backstage active transmission) of 100M transmission.It should be noted that, the first bandwidth that now this 100M bandwidth is assert before can being, is likely also no longer the first original bandwidth according to current actual conditions.
Preferably, in the preferred embodiment of the present invention, 101, source physical host utilize by network interface the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, the method also comprises:
Source physical host records when virtual machine (vm) migration is front to be moved on the physical host of source, the number of times that each page is accessed or frequency values.Source physical host is according to the descending order of the accessed number of times of each page or frequency values, and that utilizes by network interface that the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.Like this, the descending order by accessed number of times or frequency values, can preferentially be sent to the most frequently used page on target physical main frame, thereby reduces the probability that virtual machine skips leaf on target physical main frame, and then minimizing virtual machine stop time.
Preferably, in the preferred embodiment of the present invention, source physical host is according to the descending order of the accessed number of times of each page or frequency values, utilize the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, while at every turn utilizing the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, by the 3rd page one after the second page before this first page address, this first page address in the same way target physical main frame send.Like this, also the most frequently used page preferentially can be sent on target physical main frame, thereby reduce the probability that skips leaf of virtual machine on target physical main frame, and then minimizing virtual machine stop time.
Preferably, in the preferred embodiment of the present invention, 101, virtual machine moves to target physical main frame from source physical host, and before resuming operation on target physical main frame, also comprise: source physical host stops the operation of virtual machine, send the critical data of this virtual machine to target physical main frame, so that virtual machine resumes operation on target physical main frame; The critical data of this virtual machine comprises status information, the facility information of source physical host etc. of source physical host CPU.
Embodiment 2:
The embodiment of the present invention provides a kind of virtual machine migration method, describes as an example of copy migration after virtual machine example.As shown in Figure 3, the method comprises the following steps:
201, when source physical host records virtual machine to be migrated and moves on this source physical host, accessed number of times or the frequency values of each page of virtual machine.
Preferably, this step can adopt bitmap (bitmap) counter to record access times or the frequency values of virutal machine memory page.The numerical value of a certain page recording in bitmap counter is larger, represent this page accessed often or frequency values high, correspondingly, the numerical value of a certain page recording in bitmap counter is less, represents that the accessed number of times of this page is few or frequency values is little.
Table 1
Before table 1 represents that virtual machine moves on the physical host of source, the bitmap Counter Value of each page, preliminary examination is " 0 ".
Table 2
Page allocation index Bitmap Counter Value
Page 1 3
Page 2 5
Page 3 1
Page 4 6
Page 5 18
Page 6 23
Page 7 11
Page 8 6
Page 9 9
Page 10 16
Table 2 is for after virtual machine operation, the numerical value of each page of bitmap counter records.
Further, can carry out descending sort according to the numerical value of each page of bitmap counter records, obtain as shown in table 3.
Table 3
Page allocation index Bitmap Counter Value
Page 6 23
Page 5 18
Page 10 16
Page 7 11
Page 9 9
Page 4 6
Page 8 6
Page 2 5
Page 1 3
Page 3 1
Page often accessed or that frequency values is high has obtained higher priority after descending sort, can after process in by limited transmission, so just can reduce the probability that skips leaf of virtual machine after migration, minimizing stop time.
202, source physical host stops the operation of this virtual machine to be migrated, sends the critical data of this virtual machine to target physical main frame, so that this virtual machine resumes operation on target physical main frame.Wherein, the critical data of virtual machine can comprise status information, the facility information of source physical host etc. of source physical host CPU.
Here it should be noted that, because each page now is not yet transferred to target physical main frame from source physical host, the virtual machine that therefore target physical main frame resumes operation does not also possess all functions of this virtual machine.
What 203, source physical host utilized by network interface that the first bandwidth sends virtual machine to target physical main frame treats isochronous memory page.The isochronous memory page for the treatment of of this virtual machine is: be stored in the physical host of source, be not sent to the page of target physical main frame in virtual machine (vm) migration process.This process is also referred to as backstage active transmission.
In the present embodiment, suppose that the bandwidth between source physical host and target physical main frame is 100M, and this 100M is now idle, all can be used for transmitting page, this 100M bandwidth is regarded as the first bandwidth by source physical host, carries out backstage active transmission.As shown in Fig. 4 a.
Preferably, due in step 201, exemplary bitmap counter records accessed number of times or the frequency values of each page, and obtained the order shown in table 3 through descending sort.So in this step, source physical host can be according to the order of table 3 order with page 6, page 5, page 10...... page 3, send to target physical main frame.
Further, when source physical host utilizes the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface at every turn, can also by the 3rd page one after the second page before this first page address, the first page address in the same way target physical main frame send.Still taking table 3 as example, when source physical host is during to target physical main frame transmission page 8, can be by the target physical main frame transmission in the same way of the page 9 one after the page 7 before page 8 addresses, page 8 addresses.
204, when virtual machine moves on target physical main frame, produce and skip leaf, virtual machine sends page faults by target physical main frame to source physical host.Wherein, in this page faults, can comprise the address information that lacks page, this disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of target physical main frame from described source physical host after resuming operation.
For example, when virtual machine moves on target physical main frame, need to use page 100, but now page 100 not yet transfers to target physical main frame from source physical host, target physical main frame generates the page faults that comprises these page 100 addresses " XXXXXXXX ", is sent to source physical host.
205, after the page faults that source physical host receiving target physical host sends, according to the address information of the disappearance page comprising in this page faults, find described disappearance page, utilize the second bandwidth to send disappearance page to target physical main frame by network interface; What source physical host utilized by network interface that the 3rd bandwidth sends virtual machine to target physical main frame treats isochronous memory page.
Particularly, source physical host is determined the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth.
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, parameter preset N sends and treats the flow that isochronous memory page is used in Preset Time section.In addition, the relation between the first bandwidth, the second bandwidth, the 3rd this three of bandwidth comprises following situation: the second bandwidth and the 3rd bandwidth sum equal described the first bandwidth, or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth.Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
In realization, after the page faults that source physical host receiving target physical host sends, find page 100 according to address information " XXXXXXXX ".The current bandwidth for backstage active transmission is divided into the second bandwidth and the 3rd bandwidth, for example current have the bandwidth of 100M for backstage active transmission, 50M in this 100M bandwidth (i.e. the second bandwidth) can be used for transmitting page 100 so, another 50M (i.e. the 3rd bandwidth) is used for proceeding backstage active transmission, transmits not synchronous page.As shown in Figure 4 b.
Preferably, above-mentioned the second bandwidth and the 3rd bandwidth sum can equal the first bandwidth.This is that the first bandwidth is being carried out backstage active transmission at the very start, this first bandwidth value never changes, so now, just this first bandwidth can be divided into second bandwidth and the 3rd bandwidth of proceeding backstage active transmission of the page of transmission page faults instruction.
Further, the ratio of the second bandwidth and the 3rd bandwidth can be: source physical host is by first bandwidth be defined as the second bandwidth, by the first bandwidth be defined as the 3rd bandwidth; Wherein, M sends disappearance page flow used in the certain hour section of adding up in advance, and N sends and treats the flow that isochronous memory page is used in this section.
So far, can find out, if do not adjust bandwidth, there will be background transfer to take the whole network bandwidth, and the page faults producing in causing during this period of time can not get timely solution, the virtual machine moving can be stopped running, and waits for that the disappearance page transmits from source physical host.The time that will cause the virtual machine shutdown on target physical main frame to be waited for is elongated.If in order to meet in time the demand that skips leaf, by the first bandwidth all for transmission deletion page to target physical main frame, backstage active transmission will be restricted, and causes the gross migration time longer.Therefore, when isochronous memory page is treated in physical host transmission in source, the page faults of reasonable distribution bandwidth processing target physical main frame can address the above problem well.
206, source physical host utilizes the second bandwidth to send after disappearance page to target physical main frame by network interface, and what source physical host utilized by this network interface that current available bandwidth continues to send virtual machine to target physical main frame treats isochronous memory page.
Wherein, current available bandwidth be more than or equal to the second bandwidth and the 3rd bandwidth bandwidth and.
For example, the bandwidth of a 100M all for backstage active transmission, in step 205, with the page 100 of the second bandwidth 50M transmission page faults instruction, is proceeded backstage active transmission with the 3rd bandwidth 50M in step 203.In this step, after page 100 is transmitted, the current bandwidth that can utilize is used for carrying out backstage active transmission again.For example, after the complete page 100 of this 50M bandwidth for transmission, there is no other purposes, can be used for carrying out background transfer yet, now will have 100M bandwidth for carrying out backstage active transmission.If after the complete page 100 of this 50M bandwidth for transmission, have 10M to be used as other data transmission, remaining 40M can be used for carrying out background transfer, now will have 90M bandwidth for carrying out backstage active transmission.Can find out, in this step, the bandwidth for backstage active transmission can equate with original the first bandwidth again, and also may not wait, this depends on and current transmission situation.
The virtual machine migration method that the embodiment of the present invention provides, virtual machine moves to target physical main frame from source physical host, and after resuming operation on target physical main frame, source physical host can be to the not synchronous page of target physical main frame transmission.In this course, the scheme of utilizing the embodiment of the present invention to provide, once virtual machine produces and skips leaf on target physical main frame, source physical host will be with the disappearance page of the second bandwidth for transmission page faults instruction, with the not synchronous page of the 3rd bandwidth continuation transmission after receiving page faults.Like this, do not affect too much on the one hand the transmission of not synchronous page, make total duration of virtual machine (vm) migration in the reasonable scope; On the other hand, owing to there being the second special bandwidth, ensure the fast transport of disappearance page from source physical host to target physical main frame, reduced the stop time of virtual machine on target physical main frame, thereby improved on the whole the transport efficiency of virtual machine.
In addition; because the accessed situation to page is in advance added up and sorts; conventional page has obtained higher transmission priority; thereby can ensure that conventional page is preferentially transferred on target physical main frame for virtual machine; and then reduced the probability that virtual machine skips leaf, also to reducing stop time, reduce the gross migration time and play a role.
In this explanation, the virtual machine migration method that the embodiment of the present invention provides rear copy migration, is also applicable to mix copy migration except being adapted to, and does not again limit.Also can think, the virtual machine migration method that the embodiment of the present invention provides is applicable to any Remote Paging (source physical host is to the process of target physical main frame transmission deletion page) and the simultaneous data transmission scene of backstage active transmission (source physical host initiatively transmits the process of the page of not synchronizeing between the two to target physical main frame).
Embodiment 3:
The embodiment of the present invention provides a kind of virtual machine (vm) migration device 3, is deployed on the physical host of source, can be applied in said method embodiment the corresponding steps in manner of execution embodiment.As shown in Figure 5, this virtual machine (vm) migration device 3 comprises:
Background transfer module 301, treats isochronous memory page for what utilize by network interface that the first bandwidth that bandwidth self-adaption module 304 determines sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of this virtual machine is: be stored in the physical host of source, be not sent to the page of target physical main frame in virtual machine (vm) migration process.
Page faults receiver module 302, for the page faults sending by network interface receiving target physical host; This page faults includes the information that lacks page, and this disappearance page is that virtual machine (vm) migration arrives after target physical main frame, needs but be not sent to the page of target physical main frame from source physical host after resuming operation.
Remote Paging module 303, for the information of the disappearance page that comprises according to page faults, finds disappearance page, and the second bandwidth of utilizing bandwidth self-adaption module 304 to determine by network interface send this disappearance page to target physical main frame; Background transfer module 301, also treats isochronous memory page for what utilize by network interface that the 3rd bandwidth that bandwidth self-adaption module 304 determines sends virtual machine to target physical main frame.Wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
In the present embodiment, the virtual machine that background transfer module 301, Remote Paging module 303 and bandwidth self-adaption module 304 can form virtual machine (vm) migration device 3 administration module 30 that skips leaf.
Move to target physical main frame at virtual machine from source physical host, and after resuming operation, source physical host can be to the not synchronous page of target physical main frame transmission on target physical main frame.In this course, the virtual machine (vm) migration device that utilizes the embodiment of the present invention to provide, once virtual machine produces and skips leaf on target physical main frame, virtual machine (vm) migration device will be with the disappearance page of the second bandwidth for transmission page faults instruction, with the not synchronous page of the 3rd bandwidth continuation transmission after receiving page faults.Like this, do not affect too much on the one hand the transmission of not synchronous page, make total duration of virtual machine (vm) migration in the reasonable scope; On the other hand, owing to there being the second special bandwidth, ensure the fast transport of disappearance page from source physical host to target physical main frame, reduced the stop time of virtual machine on target physical main frame, thereby improved on the whole the transport efficiency of virtual machine.
Preferably, this bandwidth self-adaption module 304 for, determine the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtain described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth.
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section; Relation between the first bandwidth, the second bandwidth, the 3rd this three of bandwidth comprises following situation: the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth.Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth.Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
Preferably, this bandwidth self-adaption module 304 specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
Preferably, this background transfer module 301 also for, utilize the second bandwidth by network interface when Remote Paging module 303 and send to target physical main frame after this disappearance page, that utilizes by network interface that current available bandwidth continues to send virtual machine to target physical main frame treats isochronous memory page.Current available bandwidth is more than or equal to the second bandwidth and the 3rd bandwidth sum.
Preferably, as shown in Figure 6, this background transfer module 301 also comprises:
Heat page record sub module 3011, for utilize by network interface in background transfer module 301 the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, when recording this virtual machine and moving on the physical host of source, the number of times that each page is accessed or frequency values.
The heat page submodule 3012 of looking ahead, for the descending order according to the accessed number of times of each page or frequency values, that utilizes by network interface that the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
At this, page often accessed or that frequency values is high has obtained higher priority after descending sort, can after process in by limited transmission, so just can reduce the probability that skips leaf of virtual machine after migration, minimizing stop time.
Preferably, this heat page submodule 3012 of looking ahead, when at every turn utilizing the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, by the 3rd page one after the second page before this first page address, this first page address in the same way target physical main frame send.
Preferably, as shown in Figure 7, virtual machine (vm) migration device 3 also comprises:
Virtual machine state administration module 305, for stopping the operation of virtual machine.
Key message sending module 306, for sending the critical data of this virtual machine, so that this virtual machine resumes operation on target physical main frame to target physical main frame; The critical data of virtual machine comprises status information, the facility information of source physical host etc. of source physical host CPU.
Embodiment 4:
The embodiment of the present invention provides a provenance physical host 4, can be applied to said method embodiment, the operation of correspondence in manner of execution embodiment.As shown in Figure 8, this source physical host 4 comprises:
Hardware layer 40, this hardware layer generally comprises CPU, internal memory, network interface card etc.
The virtual machine monitoring unit VMM401 moving at hardware layer 40, described VMM401, treats isochronous memory page for what utilize by network interface that the first bandwidth sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of virtual machine is: be stored in the physical host of source, be not sent to the page of target physical main frame in virtual machine (vm) migration process.
The page faults sending by network interface receiving target physical host; This page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after target physical main frame, needs but be not sent to the page of target physical main frame from source physical host after resuming operation.
According to the information of the disappearance page comprising in page faults, find disappearance page, utilize the second bandwidth to send this disappearance page to target physical main frame by network interface; What source physical host utilized by network interface that the 3rd bandwidth sends virtual machine to target physical main frame treats isochronous memory page.Wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
In addition, this source physical host 4 also comprises: at least one of VMM401 controlling run migrates to the virtual machine 41 of target physical main frame from source physical host.
Move to target physical main frame at virtual machine from source physical host, and after resuming operation, source physical host can be to the not synchronous page of target physical main frame transmission on target physical main frame.In this course, the source physical host that utilizes the embodiment of the present invention to provide, once virtual machine produces and skips leaf on target physical main frame, source physical host will be with the disappearance page of the second bandwidth for transmission page faults instruction, with the not synchronous page of the 3rd bandwidth continuation transmission after receiving page faults.Like this, do not affect too much on the one hand the transmission of not synchronous page, make total duration of virtual machine (vm) migration in the reasonable scope; On the other hand, owing to there being the second special bandwidth, ensure the fast transport of disappearance page from source physical host to target physical main frame, reduced the stop time of virtual machine on target physical main frame, thereby improved on the whole the transport efficiency of virtual machine.
Preferably, described VMM401, also, for determine the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth.
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section; Relation between the first bandwidth, the second bandwidth, the 3rd bandwidth meets following situation: described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth.Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth.Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
Preferably, this VMM401 specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
Preferably, described VMM401, also for the information of the disappearance page that comprises according to page faults, find this disappearance page, utilize the second bandwidth to continue to send after this disappearance page to described target physical main frame by network interface, what source physical host utilized by network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page.Wherein, described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
Preferably, described VMM401, utilize by network interface the first bandwidth send to target physical main frame virtual machine treat isochronous memory page before, also for recording while moving on the physical host of source before virtual machine (vm) migration, the number of times that each page is accessed or frequency values; According to the descending order of the accessed number of times of each page or frequency values, that utilizes by network interface that the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
Preferably, described VMM401, according to the descending order of the accessed number of times of each page or frequency values, utilize the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, when at every turn utilizing the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, the described in the same way target physical main frame of the 3rd page one after the second page before this first page address, this first page address is sent.
Preferably, described VMM401, move to target physical main frame at virtual machine from source physical host, and before resuming operation on target physical main frame, also for stopping the operation of this virtual machine, send the critical data of virtual machine to target physical main frame, so that this virtual machine resumes operation on target physical main frame; The critical data of virtual machine comprises the status information of source physical host central processor CPU, the facility information of source physical host.
One of ordinary skill in the art will appreciate that: all or part of step that realizes said method embodiment can complete by the relevant hardware of programmed instruction, aforesaid program can be stored in a computer read/write memory medium, this program, in the time carrying out, is carried out the step that comprises said method embodiment; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CDs.
The above; be only the specific embodiment of the present invention, but protection scope of the present invention is not limited to this, any be familiar with those skilled in the art the present invention disclose technical scope in; can expect easily changing or replacing, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of described claim.

Claims (21)

1. a virtual machine migration method, is characterized in that, described method is applied to source physical host, comprising:
Virtual machine moves to target physical main frame from source physical host, and after resuming operation on described target physical main frame, and what described source physical host utilized by network interface that the first bandwidth sends virtual machine to target physical main frame treats isochronous memory page; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Described source physical host receives by described network interface the page faults that described target physical main frame sends; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
Described source physical host, according to the information of the described disappearance page comprising in described page faults, finds described disappearance page, utilizes the second bandwidth to send described disappearance page to described target physical main frame by described network interface; What described source physical host utilized by described network interface that the 3rd bandwidth continues to send virtual machine to described target physical main frame treats isochronous memory page; Wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
2. method according to claim 1, is characterized in that, described method also comprises:
Described source physical host is determined the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; And, determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
3. method according to claim 2, it is characterized in that, described source physical host is according to bandwidth self-adaption allocation strategy, determine the ratio of described the second bandwidth in described the first bandwidth, obtain described the second bandwidth, and determine the ratio of described the 3rd bandwidth in described the first bandwidth, and obtain described the 3rd bandwidth, specifically comprise:
Described source physical host determines that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
4. method according to claim 1, it is characterized in that, described source physical host is according to the information of the described disappearance page comprising in described page faults, find described disappearance page, utilize the second bandwidth to send after described disappearance page to described target physical main frame by described network interface, described method also comprises:
What described source physical host utilized by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
5. according to the method described in claim 1-4 any one, it is characterized in that, described source physical host utilize by network interface the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, described method also comprises:
Described source physical host records when described virtual machine (vm) migration is front to be moved on the physical host of described source, the number of times that each page is accessed or frequency values;
Correspondingly, described source physical host utilizes described the first bandwidth to send the isochronous memory page for the treatment of of virtual machine to target physical main frame by network interface to comprise: described source physical host is according to the descending order of the accessed number of times of described each page or frequency values, and that utilizes by network interface that described the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
6. method according to claim 5, it is characterized in that, described source physical host, according to the descending order of the accessed number of times of described each page or frequency values, utilizes the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, comprising:
When described source physical host utilizes described the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface at every turn, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
7. method according to claim 1, is characterized in that, virtual machine moves to target physical main frame from source physical host, and before resuming operation on described target physical main frame, described method also comprises:
Described source physical host stops the operation of described virtual machine, sends the critical data of described virtual machine to described target physical main frame, so that described virtual machine resumes operation on described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
8. a virtual machine (vm) migration device, is characterized in that, is deployed on the physical host of source, and described virtual machine (vm) migration device comprises:
Background transfer module, treats isochronous memory page for what utilize by network interface that the first bandwidth sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Page faults receiver module, the page faults sending for receive described target physical main frame by described network interface; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
Remote Paging module, for the information of the described disappearance page that comprises according to described page faults, finds described disappearance page, utilizes the second bandwidth to send described disappearance page to described target physical main frame by described network interface; Described background transfer module, also treats isochronous memory page for what utilize the 3rd bandwidth to continue to send virtual machine by described network interface to described target physical main frame; Wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
9. virtual machine (vm) migration device according to claim 8, is characterized in that, described virtual machine (vm) migration device also comprises: bandwidth self-adaption module,
Described bandwidth self-adaption module is used for, and determines the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; And determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
10. virtual machine (vm) migration device according to claim 9, is characterized in that,
Described bandwidth self-adaption module specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, and determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
11. virtual machine (vm) migration devices according to claim 8, is characterized in that,
Described background transfer module also for, utilizing the second bandwidth by described network interface when described Remote Paging module sends after described disappearance page to described target physical main frame, that utilizes by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
Virtual machine (vm) migration device described in 12. according to Claim 8-11 any one, is characterized in that, described background transfer module also comprises:
Heat page record sub module, for described background transfer module utilize by network interface described the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, when recording described virtual machine and moving on the physical host of described source, the number of times that each page is accessed or frequency values;
The heat page submodule of looking ahead, for the descending order according to the accessed number of times of described each page or frequency values, that utilizes by network interface that described the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
13. virtual machine (vm) migration devices according to claim 12, is characterized in that,
The described hot page of submodule of looking ahead, when at every turn utilizing described the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
14. virtual machine (vm) migration devices according to claim 8, is characterized in that, described virtual machine (vm) migration device also comprises:
Virtual machine state administration module, for stopping the operation of described virtual machine;
Key message sending module, for sending the critical data of described virtual machine, so that described virtual machine resumes operation on described target physical main frame to described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
15. 1 provenance physical hosts, is characterized in that, described source physical host comprises: hardware layer; At the virtual machine monitoring unit VMM of described hardware layer operation; At least one of described VMM controlling run migrates to the virtual machine of described target physical main frame from described source physical host, wherein:
Described VMM, treats isochronous memory page for what utilize by network interface that the first bandwidth sends virtual machine to target physical main frame; The isochronous memory page for the treatment of of described virtual machine is: be stored in the physical host of described source, be not sent to the page of described target physical main frame in described virtual machine (vm) migration process;
Receive by described network interface the page faults that described target physical main frame sends; Described page faults includes the information that lacks page, and described disappearance page is that virtual machine (vm) migration arrives after described target physical main frame, needs but be not sent to the page of described target physical main frame from described source physical host after resuming operation;
According to the information of the described disappearance page comprising in described page faults, find described disappearance page, utilize the second bandwidth to send described disappearance page to described target physical main frame by described network interface; What described source physical host utilized by described network interface that the 3rd bandwidth continues to send virtual machine to described target physical main frame treats isochronous memory page, and wherein, described the second bandwidth, described the 3rd bandwidth are all less than described the first bandwidth.
16. source according to claim 15 physical hosts, is characterized in that,
Described VMM, also, for determine the ratio of described the second bandwidth in described the first bandwidth according to bandwidth self-adaption allocation strategy, obtains described the second bandwidth; Determine the ratio of described the 3rd bandwidth in described the first bandwidth, obtain described the 3rd bandwidth;
Wherein, described bandwidth self-adaption allocation strategy comprises: parameter preset M and parameter preset N, described parameter preset M sends disappearance page flow used in the Preset Time section of adding up in advance, described parameter preset N sends and treats the flow that isochronous memory page is used in described Preset Time section;
Described the second bandwidth and described the 3rd bandwidth sum equal described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are greater than described the first bandwidth;
Or described the second bandwidth and described the 3rd bandwidth sum are less than described the first bandwidth.
17. source according to claim 16 physical hosts, is characterized in that,
Described VMM specifically for, determine that according to bandwidth self-adaption allocation strategy described the second bandwidth ratio in described the first bandwidth is obtain described the second bandwidth, and determine that the ratio of described the 3rd bandwidth in described the first bandwidth is obtain described the 3rd bandwidth.
18. source according to claim 15 physical hosts, is characterized in that,
Described VMM, also for the information of the described disappearance page that comprises according to described page faults, find described disappearance page, utilize the second bandwidth to send after described disappearance page to described target physical main frame by described network interface, what described source physical host utilized by described network interface that current available bandwidth sends virtual machine to target physical main frame treats isochronous memory page, and described current available bandwidth is more than or equal to described the second bandwidth and described the 3rd bandwidth sum.
19. according to the source physical host described in claim 15-18 any one, it is characterized in that,
Described VMM, utilize by network interface described the first bandwidth to target physical main frame send virtual machine treat isochronous memory page before, also for recording while moving on the physical host of described source before described virtual machine (vm) migration, the number of times that each page is accessed or frequency values; According to the descending order of the accessed number of times of described each page or frequency values, that utilizes by network interface that the first bandwidth sends virtual machine to target physical main frame successively treats isochronous memory page.
20. source according to claim 19 physical hosts, is characterized in that,
Described VMM, according to the descending order of the accessed number of times of described each page or frequency values, utilize described the first bandwidth to send in the process for the treatment of isochronous memory page of virtual machine to target physical main frame successively by network interface, when at every turn utilizing the first bandwidth to send not synchronous the first page of of virtual machine to target physical main frame by network interface, the described in the same way target physical main frame of the 3rd page one after the second page before described the first page address, described the first page address is sent.
21. source according to claim 15 physical hosts, is characterized in that,
Described VMM, move to target physical main frame at virtual machine from source physical host, and before resuming operation on described target physical main frame, also for stopping the operation of described virtual machine, send the critical data of described virtual machine to described target physical main frame, so that described virtual machine resumes operation on described target physical main frame; The critical data of described virtual machine comprises the facility information of the status information of described source physical host central processor CPU, described source physical host.
CN201410374581.5A 2014-07-31 2014-07-31 A kind of virtual machine migration method, virtual machine (vm) migration device and source physical host Active CN104156255B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410374581.5A CN104156255B (en) 2014-07-31 2014-07-31 A kind of virtual machine migration method, virtual machine (vm) migration device and source physical host

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410374581.5A CN104156255B (en) 2014-07-31 2014-07-31 A kind of virtual machine migration method, virtual machine (vm) migration device and source physical host

Publications (2)

Publication Number Publication Date
CN104156255A true CN104156255A (en) 2014-11-19
CN104156255B CN104156255B (en) 2017-10-17

Family

ID=51881761

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410374581.5A Active CN104156255B (en) 2014-07-31 2014-07-31 A kind of virtual machine migration method, virtual machine (vm) migration device and source physical host

Country Status (1)

Country Link
CN (1) CN104156255B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105808383A (en) * 2016-03-11 2016-07-27 重庆大学 Method for storing inspection point of virtual machine with quick response
CN106155782A (en) * 2015-04-09 2016-11-23 联想(北京)有限公司 Virtual machine migration method and device
WO2016205978A1 (en) * 2015-06-26 2016-12-29 Intel Corporation Techniques for virtual machine migration
CN107577518A (en) * 2017-09-14 2018-01-12 郑州云海信息技术有限公司 A kind of virtual machine deployment method and device
CN108776617A (en) * 2018-06-08 2018-11-09 山东超越数控电子股份有限公司 It is a kind of that target identification method is prefetched based on access frequency and dynamic priority
CN108932149A (en) * 2017-05-22 2018-12-04 中兴通讯股份有限公司 Data transmission method and device
CN109189545A (en) * 2018-07-06 2019-01-11 烽火通信科技股份有限公司 A kind of realization method and system improving live migration of virtual machine reliability
CN110188026A (en) * 2019-05-31 2019-08-30 龙芯中科技术有限公司 The determination method and device of fast table default parameters
CN111611055A (en) * 2020-05-27 2020-09-01 上海有孚智数云创数字科技有限公司 Virtual equipment optimal idle time migration method and device and readable storage medium
CN113590269A (en) * 2021-08-12 2021-11-02 深圳威科软件科技有限公司 Method, device and equipment for connecting cloud desktop and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101398768A (en) * 2008-10-28 2009-04-01 北京航空航天大学 Construct method of distributed virtual machine monitor system
CN102262557A (en) * 2010-05-25 2011-11-30 运软网络科技(上海)有限公司 Method for constructing virtual machine monitor by bus architecture and performance service framework
US20120272244A1 (en) * 2011-03-28 2012-10-25 International Business Machines Corporation Virtual machine placement to improve memory utilization
CN103257929A (en) * 2013-04-18 2013-08-21 中国科学院计算技术研究所 Virtual machine memory mapping method and system
CN103744725A (en) * 2013-12-24 2014-04-23 杭州华为数字技术有限公司 Virtual machine management method and device
CN103955399A (en) * 2014-04-30 2014-07-30 华为技术有限公司 Migrating method and device for virtual machine, as well as physical host

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101398768A (en) * 2008-10-28 2009-04-01 北京航空航天大学 Construct method of distributed virtual machine monitor system
CN102262557A (en) * 2010-05-25 2011-11-30 运软网络科技(上海)有限公司 Method for constructing virtual machine monitor by bus architecture and performance service framework
US20120272244A1 (en) * 2011-03-28 2012-10-25 International Business Machines Corporation Virtual machine placement to improve memory utilization
CN103257929A (en) * 2013-04-18 2013-08-21 中国科学院计算技术研究所 Virtual machine memory mapping method and system
CN103744725A (en) * 2013-12-24 2014-04-23 杭州华为数字技术有限公司 Virtual machine management method and device
CN103955399A (en) * 2014-04-30 2014-07-30 华为技术有限公司 Migrating method and device for virtual machine, as well as physical host

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MICHAEL R. HINES: "Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning", 《ACM PRESS》 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106155782A (en) * 2015-04-09 2016-11-23 联想(北京)有限公司 Virtual machine migration method and device
CN106155782B (en) * 2015-04-09 2019-06-25 联想(北京)有限公司 Virtual machine migration method and device
WO2016205978A1 (en) * 2015-06-26 2016-12-29 Intel Corporation Techniques for virtual machine migration
US10467048B2 (en) 2015-06-26 2019-11-05 Intel Corporation Techniques for virtual machine migration
CN105808383A (en) * 2016-03-11 2016-07-27 重庆大学 Method for storing inspection point of virtual machine with quick response
CN105808383B (en) * 2016-03-11 2020-04-07 重庆大学 Quick-response virtual machine checkpoint storage method
CN108932149B (en) * 2017-05-22 2023-11-17 中兴通讯股份有限公司 Data transmission method and device
CN108932149A (en) * 2017-05-22 2018-12-04 中兴通讯股份有限公司 Data transmission method and device
CN107577518A (en) * 2017-09-14 2018-01-12 郑州云海信息技术有限公司 A kind of virtual machine deployment method and device
CN108776617A (en) * 2018-06-08 2018-11-09 山东超越数控电子股份有限公司 It is a kind of that target identification method is prefetched based on access frequency and dynamic priority
CN109189545A (en) * 2018-07-06 2019-01-11 烽火通信科技股份有限公司 A kind of realization method and system improving live migration of virtual machine reliability
CN110188026A (en) * 2019-05-31 2019-08-30 龙芯中科技术有限公司 The determination method and device of fast table default parameters
CN110188026B (en) * 2019-05-31 2023-05-12 龙芯中科技术股份有限公司 Method and device for determining missing parameters of fast table
CN111611055B (en) * 2020-05-27 2020-12-18 上海有孚智数云创数字科技有限公司 Virtual equipment optimal idle time migration method and device and readable storage medium
CN111611055A (en) * 2020-05-27 2020-09-01 上海有孚智数云创数字科技有限公司 Virtual equipment optimal idle time migration method and device and readable storage medium
CN113590269A (en) * 2021-08-12 2021-11-02 深圳威科软件科技有限公司 Method, device and equipment for connecting cloud desktop and storage medium

Also Published As

Publication number Publication date
CN104156255B (en) 2017-10-17

Similar Documents

Publication Publication Date Title
CN104156255A (en) Virtual machine migration method, virtual machine migration device and source physical host
CN101436966B (en) Network monitoring and analysis system under virtual machine circumstance
US8645733B2 (en) Virtualized application power budgeting
CN102073462B (en) Virtual storage migration method and system and virtual machine monitor
US9104498B2 (en) Maximizing server utilization within a datacenter
CN102521038B (en) Virtual machine migration method and device based on distributed file system
US10411953B2 (en) Virtual machine fault tolerance method, apparatus, and system
CN102567072B (en) Resource allocation method, resource allocation device and resource allocation system
CN103365704B (en) Internal memory pre-copy method in virtual machine (vm) migration and perform device and the system of the method
US20160378570A1 (en) Techniques for Offloading Computational Tasks between Nodes
CN103399778A (en) Method and device for on-line integral migration of virtual machine
CN101616194B (en) System and method for optimizing host network performance
US20110107344A1 (en) Multi-core apparatus and load balancing method thereof
CN103631633A (en) Virtual machine total-system online migration method, device and system
CN104714846A (en) Resource processing method, operating system and equipment
CN102693162A (en) Method for process communication among multiple virtual machines on multi-core platform based on shared memory and intercore interruption
CN105659212A (en) Detection of hot pages for partition hibernation
US8364877B2 (en) Implementing gang interrupts
CN102136993A (en) Data transfer method, device and system
CN110162396A (en) Method for recovering internal storage, device, system and storage medium
CN102929769A (en) Virtual machine internal-data acquisition method based on agency service
CN115202836A (en) Computational power pooling system for improving GPU utilization efficiency
CN106293945A (en) A kind of resource perception method and system across virtual machine
US20210248000A1 (en) Virtual machine migration to multiple destination nodes
US20230273801A1 (en) Method for configuring compute mode, apparatus, and computing device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20191223

Address after: 314400 No.11, Weisan Road, Nongfa District, Chang'an Town, Haining City, Jiaxing City, Zhejiang Province

Patentee after: Haining hi tech Zone Science and Innovation Center Co.,Ltd.

Address before: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China

Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd.

Effective date of registration: 20191223

Address after: 510000 unit 2414-2416, building, No. five, No. 371, Tianhe District, Guangdong, China

Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd.