CN113490254B - VNF migration method based on bidirectional GRU resource demand prediction in federal learning - Google Patents

VNF migration method based on bidirectional GRU resource demand prediction in federal learning Download PDF

Info

Publication number
CN113490254B
CN113490254B CN202110919085.3A CN202110919085A CN113490254B CN 113490254 B CN113490254 B CN 113490254B CN 202110919085 A CN202110919085 A CN 202110919085A CN 113490254 B CN113490254 B CN 113490254B
Authority
CN
China
Prior art keywords
vnf
network
resource
physical node
energy consumption
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110919085.3A
Other languages
Chinese (zh)
Other versions
CN113490254A (en
Inventor
唐伦
吴婷
周鑫隆
陈前斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202110919085.3A priority Critical patent/CN113490254B/en
Publication of CN113490254A publication Critical patent/CN113490254A/en
Application granted granted Critical
Publication of CN113490254B publication Critical patent/CN113490254B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W48/00—Access restriction; Network selection; Access point selection
    • H04W48/08—Access restriction or access information delivery, e.g. discovery data delivery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/0215—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices
    • H04W28/0221—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices power availability or consumption
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Abstract

The invention relates to a VNF migration method based on bidirectional GRU resource demand prediction in federal learning, and belongs to the technical field of mobile communication. The method comprises the following steps: s1: in a network slice scene, a VNF migration problem caused by time-varying network flow and a VNF migration delay problem caused by the fact that the resource demand of the VNF is lack of prediction are considered, and the resource demand of the VNF is predicted by adopting a FedBi-GRU algorithm; s2: according to the resource demand prediction result, calculating the resource utilization rate of the physical node, judging the physical node with overloaded resource use or underloaded resource use in the network system, and realizing system energy consumption optimization and load balance while ensuring network performance through VNF migration; s3: and obtaining the optimal decision of VNF migration by adopting a DPPO deep reinforcement learning method. The invention can reduce the migration times of the virtual network function and the energy consumption of the network system, and can ensure the load balance of the network system.

Description

VNF migration method based on bidirectional GRU resource demand prediction in federal learning
Technical Field
The invention belongs to the technical field of mobile communication, and relates to a VNF migration method based on federal learning bidirectional GRU resource demand prediction.
Background
Software Defined Networking (SDN) and Network Function Virtualization (NFV) are two new architectures for managing network systems, the main idea behind SDN is to make the network directly programmable and separate the control plane from the data plane, provide a centralized view for the network, provide manageability for complex network systems, NFV converts physical layer resources into virtual resources, separates software instances from underlying dedicated hardware, and makes the network flexible.
By using the NFV technology, traditional network hardware resources can be virtualized into a plurality of virtual machines, various network element function software of an operator network can be instantiated into a Virtual Network Function (VNF), and the VNF can be deployed in a general server, so that the flexibility of network service deployment is greatly improved, the investment cost of network equipment of the operator is reduced, and the high-efficiency network resource utilization rate is realized. The dynamic change of the network flow causes that the available resources of the physical nodes are not matched with the resources required by the VNF, if the new VNF is re-instantiated, the state information of the original VNF is lost, the VNF can be migrated to solve the problem of resource mismatching, and in addition, the VNF is migrated by utilizing the VNF to realize the integration of the VNF, so that the energy consumption of a system can be reduced.
The existing VNF migration problem is that VNF migration is carried out based on real-time network information, few documents are used for carrying out resource demand prediction work of the VNF, a VNF migration plan is made in advance, and prediction methods adopted by the documents are all centralized machine learning. In addition, in the face of high-dimensional complexity of a VNF migration remapping space, an existing heuristic algorithm is not enough to find an optimal VNF migration scheme.
Disclosure of Invention
In view of this, the present invention provides a VNF migration method based on federal learning bidirectional GRU resource demand prediction, which reduces the training burden of the existing centralized prediction mode, improves the generalization of the prediction model, and simultaneously can realize network system energy consumption optimization and load balancing under the limitation of the bottom CPU, storage and bandwidth resources, and end-to-end network delay.
In order to achieve the purpose, the invention provides the following technical scheme:
a VNF migration method based on bidirectional GRU resource demand forecasting of federal learning specifically comprises the following steps:
s1: under a network slice scene, a Virtual Network Function (VNF) migration problem caused by time-varying network flow and a VNF migration delay problem caused by the lack of prediction of VNF resource demand are considered, and the resource demand of the VNF is predicted by adopting a Federal learning-based bidirectional gating cycle unit network (FedBi-GRU) algorithm;
s2: calculating the resource utilization rate of the physical node according to the resource demand prediction result obtained in the step S1, determining the physical node with overloaded resource use or underloaded resource use in the network system, and migrating through the VNF, thereby realizing system energy consumption optimization and load balancing while ensuring network performance;
s3: in consideration of the complexity and high dimensionality of the virtual network function migration decision problem, the optimal decision of VNF migration is obtained by adopting a deep reinforcement learning method of distributed near-end policy optimization (DPPO).
Further, in step S1, the network slice scene includes three layers of architectures: the physical infrastructure layer provides physical resources of the system and mainly comprises a plurality of physical nodes; the virtualization layer abstracts physical resources such as an access network, a core network, a frequency spectrum and the like into virtual resources and provides resource support for the slice layer; the network slice layer is used for processing user network services and is processed by SFC formed by arranging a series of VNFs according to a set sequence; wherein SFC represents a service function chain;
the VNF migration problem is selecting a remapped physical node for a VNF;
the resource requirements of the VNF are CPU, storage and bandwidth resources required by the VNF at a future time.
Further, in step S1, the FedBi-GRU algorithm performs local Bi-GRU resource demand prediction model training for VNFs on SFCs on respective deployed physical nodes, performs aggregation and averaging of prediction model parameters at the SDN controller, and then issues the prediction model parameters to each VNF for continuing model training, where Bi-GRU represents a bidirectional gating cycle unit, and SDN represents a software defined network; the method specifically comprises the following steps:
s11: issuing initial global parameters by the SDN controller;
s12: each VNF on the SFC receives the initial parameters to carry out local Bi-GRU resource demand prediction model training;
s13: uploading parameters of a local Bi-GRU resource demand prediction model by each VNF on the SFC;
s14: the SDN controller performs federal averaging on the collected parameters to obtain new global parameters;
s15: the VNF on the SFC receives the new global parameters and continues to train the Bi-GRU resource demand prediction model;
s16: and repeating the steps S11-S15 until the Bi-GRU resource demand forecasting models of all VNFs on the SFC converge.
Further, in step S2, the resource utilization rate of the physical node is a ratio of the sum of CPU resource requirements of all VNFs on the physical node to the CPU capacity of the physical node;
the resource utilization rate of the physical node with the resource use overload or the resource use underload is higher than the highest CPU resource use threshold value of the physical node or lower than the lowest CPU resource use threshold value;
the network performance is to satisfy the network service quality of the user.
Further, in step S2, the system energy consumption is optimized to minimize the network system energy consumption after VNF migration, where the network system energy consumption includes the operation state energy consumption and the state switching energy consumption of the bottom physical node; the operation state energy consumption comprises basic energy consumption of physical nodes capable of operating and load energy consumption of the nodes, the load energy consumption is in direct proportion to the utilization rate of CPU resources, and the operation energy consumption of the physical nodes at the moment t is represented as:
Figure BDA0003206787730000021
wherein Z isn(t) a binary variable indicating whether physical node n is on, where physical node n remains on as long as VNF is mapped to physical node n, and Z is at this timen(t) 1, otherwise the physical node enters the sleep state, Zn(t) ═ 0; is provided with
Figure BDA0003206787730000031
For the fundamental energy consumption of the operation of the physical node n,
Figure BDA0003206787730000032
the energy consumption is the energy consumption that the CPU resource of the physical node is occupied;
Figure BDA0003206787730000033
the CPU resource utilization rate of the physical node n at time t is expressed as:
Figure BDA0003206787730000034
wherein the content of the first and second substances,
Figure BDA0003206787730000035
indicating whether VNF j in the ith SFC at time instant tth maps to a binary variable of physical node n,
Figure BDA0003206787730000036
indicating that VNF j maps onto physical node n; f is a set of SFCs,
Figure BDA0003206787730000037
all VNFs on the ith SFC are collected; cnRepresenting the computing resources owned by node n;
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) is represented as follows:
Figure BDA0003206787730000038
the switching energy consumption generated by the change of the working state of the node n at the moment t is expressed as
Figure BDA0003206787730000039
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system is expressed as:
Figure BDA00032067877300000310
wherein N isPIs a set of underlying physical nodes.
Further, in step S2, the load balancing is to minimize a variance of network resources, where the variance of network resources is a variance of utilization rates of CPUs and storage resources of all physical nodes of the network system;
assuming that at time t, VNF j on SFC i triggers a migration condition, after VNF j is migrated from physical node n to physical node m, load change of physical node m may be caused, and the load of physical node m at time t is represented as:
Figure BDA00032067877300000311
Figure BDA00032067877300000312
wherein the content of the first and second substances,
Figure BDA00032067877300000313
and
Figure BDA00032067877300000314
respectively representing the CPU and storage resource loads of the physical node m at the moment t;
Figure BDA00032067877300000315
a binary variable indicating whether VNF j in the ith SFC maps to physical node n,
Figure BDA00032067877300000316
indicating that VNF j maps onto physical node n;
Figure BDA00032067877300000317
respectively representing the computing resource requirement and the storage resource requirement of the jth VNF on the ith SFC at time instant tth,
Figure BDA00032067877300000318
all VNFs on the ith SFC are collected; the network resource variance of a single physical node cannot measure the load balance of the whole network system, and the load balance of the whole network system can only be measuredEnsuring good network performance. The mean value of the CPUs of the entire network system at time t
Figure BDA0003206787730000041
Mean value of storage load
Figure BDA0003206787730000042
Is represented as follows:
Figure BDA0003206787730000043
Figure BDA0003206787730000044
wherein N isPIs a set of underlying physical nodes, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m;
the CPU and storage resource variance of the entire network system is represented as:
Figure BDA0003206787730000045
Figure BDA0003206787730000046
the resource variance of the entire network system is expressed as:
Figure BDA0003206787730000047
further, in step S3, the complexity and high-dimensional property of the virtual network function migration decision problem is that the VNF mapping decision space is high-dimensional and complex due to the diversity of network traffic flows and the dynamic variability of the network traffic environment.
Further, in step S3, the deep reinforcement learning method for distributed near-end policy optimization (DPPO) has a structure jointly trained by a global network and a plurality of agent local networks, and the global network and the agent local networks have the same near-end policy optimization (PPO) network structure, and specifically includes the following steps:
s31: taking an SFC as an agent, and independently and synchronously carrying out PPO algorithm training on each agent in different threads to obtain different PPO network parameters; wherein PPO represents near-end policy optimization;
s32: each agent regularly pushes a loss function gradient to the global PPO network;
s33: when the gradient of each agent is accumulated to the updating frequency times of the global network by collecting the gradient of each agent, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
s34: pushing parameters obtained by the global network to each agent, enabling the agents to interactively update the gradient with the slice network system, pushing the parameters to the global network to update global network parameters, and repeating the steps until the DPPO algorithm is converged; where DPPO represents distributed near-end policy optimization.
Further, in step S3, the optimal decision of VNF migration is to find a VNF mapping policy that achieves approximately the minimum system energy consumption and network resource variance by using a DPPO algorithm.
The invention has the beneficial effects that: aiming at the problem of VNF migration delay caused by lack of effective resource prediction in the existing VNF migration problem, the invention provides a distributed Federal learning bidirectional gating unit resource prediction algorithm, the distributed prediction architecture relieves the problems of centralized prediction data training pressure and memory shortage, and an intelligent VNF migration algorithm based on distributed near-end strategy optimization is provided based on the resource demand prediction result, so that energy consumption optimization and load balance of a network system are realized.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a VNF resource demand prediction flow diagram for a Federal bidirectional gated cycle Unit (FedBi-GRU) of the present invention;
fig. 2 is a flowchart of a VNF migration algorithm for distributed near-end policy optimization.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Referring to fig. 1-2, the physical layer of the network slice of the embodiment is defined as a full-connection undirected graph GP＝(NP,LP)，NPIs a set of underlying physical nodes, LPIs the set of underlying physical links. The invention considers two resource use conditions, namely computing resources and storage resources. Let CnRepresenting the computational resources owned by node n, MnRepresenting the storage resources owned by node N, where N, m ∈ Np，lnmRepresents a physical link between physical nodes n and m, and lnm∈LP。
A network slice layer is defined as a set of SFCs, denoted as F ═ Fi1,2, I, the ith SFC chain is abstracted as a directed graph
Figure BDA0003206787730000051
For all VNFs on the ith SFC,
Figure BDA0003206787730000052
is the set of all virtual links on the ith SFC. The computational resource requirement of the jth VNF on the ith SFC is
Figure BDA0003206787730000053
The storage resource needs are
Figure BDA0003206787730000054
Virtual link between jth VNF and kth VNF on SFC i
Figure BDA0003206787730000055
The bandwidth requirement is
Figure BDA0003206787730000056
Is provided with
Figure BDA0003206787730000057
A binary variable indicating whether VNF j in the ith SFC maps to physical node n,
Figure BDA0003206787730000058
meaning that VNF j maps onto physical node n,
Figure BDA0003206787730000059
representing a virtual link between the jth VNF and the kth VNF on SFC i
Figure BDA00032067877300000510
Whether or not to map to physical link/nmIn the above-mentioned manner,
Figure BDA00032067877300000511
to represent
Figure BDA00032067877300000512
Mapping to lnmIn the above-mentioned manner,
Figure BDA0003206787730000061
in this embodiment, when predicting VNF resource demand, each physical node acquires CPU, storage, and bandwidth resource characteristics of a VNF thereon, and for a VNF j on an SFC i, the network resource demand characteristics may be expressed as
Figure BDA0003206787730000062
Respectively, the CPU, storage, and bandwidth resource characteristics of VNF j.
Referring to fig. 1, fig. 1 is a flowchart of a VNF resource demand prediction algorithm of the present invention, including the following steps:
step 1): parameter aggregator SDN controller issuing initial parameter w0To each VNF on SFC i.
Step 2): the VNF on the SFC collects network resource usage history data, and for the VNF j on the SFC i, a series of data feature sample sets o ═ o of the VNF j can be obtained by collecting receipts1,o2,...,os,., inputting the feature sample set into a local Bi-GRU model of VNF j in combination with an initial parameter w0And (5) training.
Step 3): each VNF on SFC i will train the resulting parameter w1,w2,...,wjUploading to an SDN controller for federal averaging to obtain new global parameters
Figure BDA0003206787730000063
Step 4): SDN controller assigns new global parameters
Figure BDA0003206787730000064
And issuing all VNFs on the SFC i, and continuing Bi-GRU model training by the local VNFs until the training times are reached.
In the embodiment, the energy consumption of the physical node running state and the energy consumption of state switching are mainly considered when the energy consumption of the network system is calculated. The operation state energy consumption comprises the basic energy consumption of the operation of the physical node and the load energy consumption of the node, the load energy consumption is in direct proportion to the load of the CPU, and the operation energy consumption of the physical node at the moment t is represented as follows:
Figure BDA0003206787730000065
wherein Z isn(t) a binary variable indicating whether physical node n is on, where physical node n remains on as long as VNF is mapped to physical node n, and Z is at this timen(t) 1, otherwise the physical node enters the sleep state, Zn(t) is 0. Is provided with
Figure BDA0003206787730000066
For the fundamental energy consumption of the operation of the physical node n,
Figure BDA0003206787730000067
the energy consumption that the CPU resource is full is the physical node.
Figure BDA0003206787730000068
The CPU resource utilization rate of the physical node n at time t is expressed as:
Figure BDA0003206787730000069
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) may be expressed as follows:
Figure BDA00032067877300000610
the switching energy consumption generated by the change of the working state of the node n at the time t can be expressed as
Figure BDA00032067877300000611
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system can be expressed as:
Figure BDA0003206787730000071
load balancing model:
the closer the average network resource variance usage is, the stronger the network load balancing capability of the physical node is, assuming that at time t, the VNF j on the SFC i triggers the migration condition, then the VNF j may cause load change of the physical node m after being migrated from the physical node n to the physical node m, and the load of the physical node m at time t may be represented as:
Figure BDA0003206787730000072
Figure BDA0003206787730000073
wherein the content of the first and second substances,
Figure BDA0003206787730000074
and
Figure BDA0003206787730000075
the loads of the CPU and the storage resource of the physical node m at the time t are respectively represented, the network resource variance of a single physical node cannot measure the load balance of the whole network system, and the load balance of the whole network system can ensure good network performance. Therefore, the CPU, storage load mean value of the whole network system at time t is expressed as follows:
Figure BDA0003206787730000076
Figure BDA0003206787730000077
wherein, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m, the CPU and storage resource variance of the whole network system can be represented as:
Figure BDA0003206787730000078
Figure BDA0003206787730000079
the resource variance of the entire network system can be expressed as:
Figure BDA00032067877300000710
referring to fig. 2, fig. 2 is a VNF migration algorithm for distributed near-end policy optimization according to this embodiment, and the steps are as follows:
step 1): taking an SFC as an agent, placing each agent in different threads to independently and synchronously train a PPO algorithm to obtain different PPO network parameters, wherein for an agent n, the PPO network updating mode is as follows:
the criticic network process of agent n is a loss function for minimizing TD error, and the loss function is expressed as follows:
Figure BDA00032067877300000711
the Actor network loss function of the n-local PPO network of the agent is as follows:
Figure BDA0003206787730000081
wherein s isn(T) represents the operating state of agent n at iteration step T, V(s)n(T +1)) means that the agent n is in the working state snFunction of state value of (T +1), an(t) represents the VNF mapping action taken by agent n at time t, Et[·]Represents the expectation function at time t, sigma represents the hyperparameter of the PPO network,
Figure BDA0003206787730000082
an Actor network parameter representing agent n;
Figure BDA0003206787730000083
for instant awards, gammav∈[0，1]For the discounting factor, T is the iteration step size in a round of training, A(s)n(t),an(t)) is a merit function;
Figure BDA0003206787730000084
for the ratio of old and new mapping strategies, σ is set to 0.2 for the over-parameter.
Step 2): loss function of agent n
Figure BDA0003206787730000085
Critic network gradient for updating global PPO network, global Critic network gradient delta thetacAnd a parameter thetacThe update mode is as follows:
Figure BDA0003206787730000086
θc＝θc+εcΔθc
wherein epsiloncIs the learning rate of the Critic network.
Gradient delta theta of global Actor networkaAnd a parameter thetaaThe update method (2) is as follows:
Figure BDA0003206787730000087
θa＝θa+εaΔθa
wherein epsilonaIs the learning rate of the Actor network.
Step 3): when the gradient is accumulated to the updating frequency times of the global network, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
step 4): and pushing the parameters obtained by the global network to each intelligent agent, wherein the intelligent agent interactively updates the gradient between the obtained parameters and the slice network system, pushes the parameters to the global network to update the global network parameters, and repeats the steps until the energy consumption and the load of the network system are balanced.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.

Claims (7)

1. A VNF migration method based on bidirectional GRU resource demand forecasting of federal learning is characterized by comprising the following steps:
s1: in a network slice scene, a VNF migration problem caused by time-varying network flow and a VNF migration delay problem caused by the fact that the resource demand of the VNF is lack of prediction are considered, and the resource demand of the VNF is predicted by adopting a FedBi-GRU algorithm; the FedBi-GRU represents a bidirectional gating cycle unit network based on federal learning, and the VNF represents a virtual network function;
in step S1, the FedBi-GRU algorithm performs local Bi-GRU resource demand prediction model training on the respective deployed physical nodes for the VNFs on the SFC, performs aggregation averaging of prediction model parameters at the SDN controller, and then issues the prediction model parameters to each VNF for continuing model training, where Bi-GRU represents a bidirectional gated cyclic unit, and SDN represents a software defined network; the method specifically comprises the following steps:
s11: issuing initial global parameters by the SDN controller;
s12: each VNF on the SFC receives the initial parameters to carry out local Bi-GRU resource demand prediction model training;
s13: uploading parameters of a local Bi-GRU resource demand prediction model by each VNF on the SFC;
s14: the SDN controller performs federal averaging on the collected parameters to obtain new global parameters;
s15: the VNF on the SFC receives the new global parameters and continues to train the Bi-GRU resource demand prediction model;
s16: repeating the steps S11-S15 until the Bi-GRU resource demand prediction models of all VNFs on the SFC are converged;
s2: calculating the resource utilization rate of the physical node according to the resource demand prediction result obtained in the step S1, determining the physical node with overloaded resource use or underloaded resource use in the network system, and migrating through the VNF, thereby realizing system energy consumption optimization and load balancing while ensuring network performance;
s3: obtaining an optimal decision of VNF migration by adopting a deep reinforcement learning method of distributed near-end strategy optimization;
in step S3, the method for deep reinforcement learning by distributed near-end policy optimization specifically includes the following steps:
s31: taking an SFC as an agent, and independently and synchronously carrying out PPO algorithm training on each agent in different threads to obtain different PPO network parameters; wherein PPO represents near-end policy optimization;
s32: each agent regularly pushes a loss function gradient to the global PPO network;
s33: when the gradient of each agent is accumulated to the updating frequency times of the global network by collecting the gradient of each agent, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
s34: pushing parameters obtained by the global network to each agent, enabling the agents to interactively update the gradient with the slice network system, pushing the parameters to the global network to update global network parameters, and repeating the steps until the DPPO algorithm is converged; wherein DPPO represents distributed near-end policy optimization;
in step S3, the optimal decision of VNF migration is to find a VNF mapping policy that minimizes system energy consumption and network resource variance by using a DPPO algorithm.
2. The VNF migration method according to claim 1, wherein in step S1, the network slice scenario includes a three-tier architecture: the physical infrastructure layer provides physical resources of the system and consists of a plurality of physical nodes; the virtualization layer abstracts physical resources into virtual resources and provides resource support for the slice layer; the network slice layer is used for processing user network services and is processed by SFC formed by arranging a series of VNFs according to a set sequence; wherein SFC represents a service function chain;
the VNF migration problem is selecting a remapped physical node for a VNF;
the resource requirements of the VNF are CPU, storage and bandwidth resources required by the VNF at a future time.
3. The VNF migration method according to claim 1, wherein in step S2, the resource utilization of the physical node is a ratio of a sum of CPU resource requirements of all VNFs on the physical node to a CPU capacity of the physical node;
the resource utilization rate of the physical node with the resource use overload or the resource use underload is higher than the highest CPU resource use threshold value of the physical node or lower than the lowest CPU resource use threshold value;
the network performance is to satisfy the network service quality of the user.
4. The VNF migration method according to claim 1, wherein in step S2, the system energy consumption is optimized to minimize network system energy consumption after VNF migration, where the network system energy consumption includes an operation state energy consumption and a state switching energy consumption of an underlying physical node; the operation state energy consumption comprises basic energy consumption of physical nodes capable of operating and load energy consumption of the nodes, the load energy consumption is in direct proportion to the utilization rate of CPU resources, and the operation energy consumption of the physical nodes at the moment t is represented as:
Figure FDA0003508818050000021
wherein Z isn(t) binary variable indicating whether physical node n is on, physical node n remaining on as long as VNF is mapped to physical node n, which isTime Zn(t) 1, otherwise the physical node enters the sleep state, Zn(t) ═ 0; is provided with
Figure FDA0003508818050000022
For the fundamental energy consumption of the operation of the physical node n,
Figure FDA0003508818050000023
the energy consumption is the energy consumption that the CPU resource of the physical node is occupied;
Figure FDA0003508818050000024
the CPU resource utilization rate of the physical node n at time t is expressed as:
Figure FDA0003508818050000025
wherein the content of the first and second substances,
Figure FDA0003508818050000026
indicating whether VNF j in the ith SFC at time instant tth maps to a binary variable of physical node n,
Figure FDA0003508818050000027
indicating that VNF j maps onto physical node n; f is a set of SFCs,
Figure FDA0003508818050000028
all VNFs on the ith SFC are collected; cnRepresenting the computing resources owned by node n;
Figure FDA0003508818050000029
representing the computational resource requirements of the jth VNF on the ith SFC at time t;
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) is represented as follows:
Figure FDA0003508818050000031
the switching energy consumption generated by the change of the working state of the node n at the moment t is expressed as
Figure FDA0003508818050000032
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system is expressed as:
Figure FDA0003508818050000033
wherein N isPIs a set of underlying physical nodes.
5. The VNF migration method according to claim 1, wherein in step S2, the load balancing is to minimize a network resource variance, where the network resource variance is a variance of utilization rates of all physical nodes CPU and storage resources of the network system;
assuming that at time t, VNF j on SFC i triggers a migration condition, after VNF j is migrated from physical node n to physical node m, load change of physical node m may be caused, and the load of physical node m at time t is represented as:
Figure FDA0003508818050000034
Figure FDA0003508818050000035
wherein the content of the first and second substances,
Figure FDA0003508818050000036
and
Figure FDA0003508818050000037
respectively representing the CPU and storage resource loads of the physical node m at the moment t;
Figure FDA0003508818050000038
a binary variable indicating whether VNF j in the ith SFC maps to physical node n,
Figure FDA0003508818050000039
indicating that VNF j maps onto physical node n;
Figure FDA00035088180500000310
respectively representing the computing resource requirement and the storage resource requirement of the jth VNF on the ith SFC at time instant tth,
Figure FDA00035088180500000311
all VNFs on the ith SFC are collected; CPU mean value of the entire network system at time t
Figure FDA00035088180500000312
Mean value of storage load
Figure FDA00035088180500000313
Is represented as follows:
Figure FDA00035088180500000314
Figure FDA00035088180500000315
wherein N isPIs a set of underlying physical nodes, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m;
the CPU and storage resource variance of the entire network system is represented as:
Figure FDA00035088180500000316
Figure FDA00035088180500000317
the resource variance of the entire network system is expressed as:
Figure FDA0003508818050000041
6. the VNF migration method according to claim 1, wherein in step S31, for agent n, the PPO network update method is as follows:
the loss function for agent n is:
Figure FDA0003508818050000042
wherein r isn(t) is the instant prize, γv∈[0，1]For the discounting factor, T is the iteration step size in a round of training, sn(T) represents the operating state of agent n at iteration step T, V(s)n(T +1)) means that the agent n is in the working state snA state value function of (T + 1);
the Actor network loss function of the n-local PPO network of the agent is as follows:
Figure FDA0003508818050000043
wherein the content of the first and second substances,
Figure FDA0003508818050000044
for the ratio of old and new mapping strategies, A(s)n(t),an(t)) is the merit function, an(t) represents the VNF mapping action taken by agent n at time t, Et[·]Represents the expectation function at time t, sigma represents the hyperparameter of the PPO network,
Figure FDA0003508818050000045
representing the Actor network parameter of agent n.
7. The VNF migration method according to claim 6, wherein in step S32, a loss function of agent n is used
Figure FDA0003508818050000046
Updating the Critic network gradient of the global PPO network, wherein the global Critic network gradient delta thetacAnd a parameter thetacThe updating method comprises the following steps:
Figure FDA0003508818050000047
θc＝θc+εcΔθc
wherein epsiloncLearning rate of Critic network;
gradient delta theta of global Actor networkaAnd a parameter thetaaThe updating method comprises the following steps:
Figure FDA0003508818050000048
θa＝θa+εaΔθa
wherein epsilonaIs the learning rate of the Actor network.
CN202110919085.3A 2021-08-11 2021-08-11 VNF migration method based on bidirectional GRU resource demand prediction in federal learning Active CN113490254B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110919085.3A CN113490254B (en) 2021-08-11 2021-08-11 VNF migration method based on bidirectional GRU resource demand prediction in federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110919085.3A CN113490254B (en) 2021-08-11 2021-08-11 VNF migration method based on bidirectional GRU resource demand prediction in federal learning

Publications (2)

Publication Number Publication Date
CN113490254A CN113490254A (en) 2021-10-08
CN113490254B true CN113490254B (en) 2022-03-29

Family

ID=77946234

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110919085.3A Active CN113490254B (en) 2021-08-11 2021-08-11 VNF migration method based on bidirectional GRU resource demand prediction in federal learning

Country Status (1)

Country Link
CN (1) CN113490254B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114065951A (en) * 2021-11-17 2022-02-18 黑龙江大学 Semi-supervised federal learning method based on non-IID data
CN114401192B (en) * 2021-12-20 2024-02-06 广西壮族自治区公众信息产业有限公司 Multi-SDN controller cooperative training method
CN114374605B (en) * 2022-01-12 2024-01-05 西安盈科思泰网络技术有限公司 Dynamic adjustment and migration method for service function chain in network slice scene
CN114726743A (en) * 2022-03-04 2022-07-08 重庆邮电大学 Service function chain deployment method based on federal reinforcement learning
CN114827284B (en) * 2022-04-21 2023-10-03 中国电子技术标准化研究院 Service function chain arrangement method and device in industrial Internet of things and federal learning system
CN115130683A (en) * 2022-07-18 2022-09-30 山东大学 Asynchronous federal learning method and system based on multi-agent model
CN115361288B (en) * 2022-08-19 2023-06-23 南京工业大学 Service function chain dynamic reconstruction method under space-air-ground integrated scene
CN117540215A (en) * 2024-01-05 2024-02-09 中国民航大学 Flight delay prediction model training method, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110505099A (en) * 2019-08-28 2019-11-26 重庆邮电大学 A kind of service function chain dispositions method based on migration A-C study
CN111130904A (en) * 2019-12-30 2020-05-08 重庆邮电大学 Virtual network function migration optimization algorithm based on deep certainty strategy gradient
CN111314235A (en) * 2020-02-19 2020-06-19 广东技术师范大学 Network delay optimization method based on virtual network function resource demand prediction
CN111526070A (en) * 2020-04-29 2020-08-11 重庆邮电大学 Service function chain fault detection method based on prediction
CN112631717A (en) * 2020-12-21 2021-04-09 重庆大学 Network service function chain dynamic deployment system and method based on asynchronous reinforcement learning

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11824732B2 (en) * 2018-12-28 2023-11-21 Intel Corporation Techniques for artificial intelligence capabilities at a network switch

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110505099A (en) * 2019-08-28 2019-11-26 重庆邮电大学 A kind of service function chain dispositions method based on migration A-C study
CN111130904A (en) * 2019-12-30 2020-05-08 重庆邮电大学 Virtual network function migration optimization algorithm based on deep certainty strategy gradient
CN111314235A (en) * 2020-02-19 2020-06-19 广东技术师范大学 Network delay optimization method based on virtual network function resource demand prediction
CN111526070A (en) * 2020-04-29 2020-08-11 重庆邮电大学 Service function chain fault detection method based on prediction
CN112631717A (en) * 2020-12-21 2021-04-09 重庆大学 Network service function chain dynamic deployment system and method based on asynchronous reinforcement learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GRU and EdgeQ-Learning based Traffic Prediction and Scaling of SFC;Suman Pandey等;《 2021 IEEE 7th International Conference on Network Softwarization (NetSoft)》;20210726;全文 *
基于深度确定性策略梯度的虚拟网络功能迁移优化算法;唐伦等;《电子与信息学报》;20210215;第43卷(第2期);全文 *

Also Published As

Publication number Publication date
CN113490254A (en) 2021-10-08

Similar Documents

Publication Publication Date Title
CN113490254B (en) VNF migration method based on bidirectional GRU resource demand prediction in federal learning
CN108260169B (en) QoS guarantee-based dynamic service function chain deployment method
CN108829494A (en) Container cloud platform intelligence method for optimizing resources based on load estimation
CN111445111B (en) Electric power Internet of things task allocation method based on edge cooperation
CN111953758B (en) Edge network computing unloading and task migration method and device
CN105610715B (en) A kind of cloud data center multi-dummy machine migration scheduling method of planning based on SDN
CN106411770B (en) A kind of data center network energy-saving routing algorithm based on SDN framework
CN112087509B (en) Task migration method in edge computing platform
CN108965014B (en) QoS-aware service chain backup method and system
CN104636187B (en) Dispatching method of virtual machine in NUMA architecture based on load estimation
CN107748693A (en) Group's virtual machine scheduling policy under cloud computing environment
CN104503832B (en) A kind of scheduling virtual machine system and method for fair and efficiency balance
CN108182105A (en) Local dynamic station moving method and control system based on Docker container techniques
WO2020134133A1 (en) Resource allocation method, substation, and computer-readable storage medium
CN108089914A (en) A kind of cloud computing deploying virtual machine algorithm based on energy consumption
CN114374605A (en) Dynamic adjustment and migration method for service function chain in network slice scene
CN106775949A (en) A kind of Application of composite feature that perceives migrates optimization method online with the virtual machine of the network bandwidth
CN106656555A (en) Dynamic adjustment method of service resources of cloud computing system
CN110597598B (en) Control method for virtual machine migration in cloud environment
Bouzidi et al. Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning
CN104298539B (en) Scheduling virtual machine and dispatching method again based on network aware
CN113676357B (en) Decision method for edge data processing in power internet of things and application thereof
CN111510319A (en) Network slice resource management method based on state perception
CN110958192B (en) Virtual data center resource allocation system and method based on virtual switch
CN105577834B (en) Two layers of bandwidth allocation methods of cloud data center with Predicable performance and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant