CN114827284B

CN114827284B - Service function chain arrangement method and device in industrial Internet of things and federal learning system

Info

Publication number: CN114827284B
Application number: CN202210422270.6A
Authority: CN
Inventors: 张群; 马珊珊; 徐洋; 汪书韵; 芮兰兰; 熊翱; 李娜; 韦磊; 蒋承伶
Original assignee: State Grid Comprehensive Energy Service Group Co ltd; Beijing University of Posts and Telecommunications; State Grid Jiangsu Electric Power Co Ltd; China Electronics Standardization Institute
Current assignee: State Grid Comprehensive Energy Service Group Co ltd; Beijing University of Posts and Telecommunications; State Grid Jiangsu Electric Power Co Ltd; China Electronics Standardization Institute
Priority date: 2022-04-21
Filing date: 2022-04-21
Publication date: 2023-10-03
Anticipated expiration: 2042-04-21
Also published as: CN114827284A

Abstract

The application provides a service function chain arrangement method and device in an industrial Internet of things and a federal learning system, wherein the method comprises the following steps: receiving a service function chain arrangement request aiming at the industrial Internet of things; arranging the service function chain arrangement request according to an arrangement algorithm for balancing the energy consumption and the time delay of the service function chain to obtain an arrangement result of the corresponding service function chain; and distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things. The application can balance the energy consumption and time delay factors of the service function chain, can improve the reliability and the intelligent degree of the service function chain arrangement process in the industrial Internet of things, and can further effectively improve the reliability and the effectiveness of the service function chain.

Description

Service function chain arrangement method and device in industrial Internet of things and federal learning system

Technical Field

The application relates to the technical field of computer network communication, in particular to a service function chain arrangement method and device in the industrial Internet of things and a federal learning system.

Background

Industrial internet of things (Industrial Internet of Things, IIoT for short) is predictive of a new modern surge, requiring more progress in productivity, management, security and flexibility. In order to solve the demands of real-time performance, resource diversity, safety and the like in the industrial Internet of things, various researches introduce network function virtualization and mobile edge calculation, and orderly arrange service function chains SFC (Service Function Chain) in an edge network, so that data forwarding in the industrial Internet of things is more flexible, qoS requirements of users are met, and manageability and flexibility of the industrial Internet of things are greatly improved.

Therefore, in order to ensure the network service quality required by the user, efficient and reasonable arrangement and dynamic optimization of the SFC are required. In combination with the existing research, at least the following problems exist in the service function chain arrangement:

for the service function chain performance problem, part of literature only focuses on optimizing time delay, and optimizing energy consumption expenditure of VNF instantiation or port transmission by adopting different modeling methods, and part of literature only focuses on maximizing the number of service function chains carried in a network and does not focus on service quality, so that during modeling, two constraints of energy consumption and time delay are required to be focused and balanced.

Disclosure of Invention

In view of this, embodiments of the present application provide a method, an apparatus, and a federal learning system for arranging service function chains in industrial internet of things, so as to eliminate or improve one or more drawbacks existing in the prior art.

The application provides a service function chain arrangement method in an industrial Internet of things, which comprises the following steps:

receiving a service function chain arrangement request aiming at the industrial Internet of things;

arranging the service function chain arrangement request according to an arrangement algorithm for balancing the energy consumption and the time delay of the service function chain to obtain an arrangement result of the corresponding service function chain;

And distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things.

In some embodiments of the application, the orchestration algorithm comprises: global programming model for balancing service function chain energy consumption and time delay based on federal learning and deep reinforcement learning training.

In some embodiments of the present application, before the receiving the service function chain orchestration request for the industrial internet of things, the method further comprises:

selecting a plurality of working nodes in the edge cloud layer;

based on a federal learning algorithm, distributing model parameters of a current global programming model for balancing service function chain energy consumption and time delay to each selected working node, so that each working node carries out deep reinforcement learning training on a local programming model corresponding to the model parameters based on respective local data, and outputting model parameters of the local programming model obtained by respective training;

and receiving the model parameters of the local programming model sent by each working node respectively, and carrying out aggregation processing on the model parameters of each local programming model so as to obtain the updated global programming model.

In some embodiments of the application, the global orchestration model and the local orchestration model are both DQN-based deep reinforcement learning, DRL, models;

and obtaining a target Q value and a predicted Q value in the predicted Q-learning by adopting a neural network, wherein the loss function corresponds to the DQN-based deep reinforcement learning DRL model.

In some embodiments of the present application, the selecting a plurality of working nodes in the edge cloud layer includes:

generating comprehensive reputation values corresponding to candidate nodes in the edge cloud layer respectively based on a preset subjective logic model;

and sequencing the candidate nodes according to the sequence of the comprehensive reputation value from high to low, and selecting a plurality of candidate nodes as the working nodes according to the corresponding sequencing result.

In some embodiments of the present application, the generating, based on a preset subjective logic model, the composite reputation value corresponding to each candidate node in the edge cloud layer includes:

sorting the Euclidean distance sum corresponding to each candidate node according to the sequence from big to small according to the Euclidean distance sum between the local arrangement model of each candidate node and the local arrangement model of other candidate nodes, and assigning a value to the sorting result based on a preset score assignment rule to obtain initial scores corresponding to each candidate node;

According to the communication quality of the links between each candidate node and the central cloud and the initial score corresponding to each candidate node, direct reputation scores corresponding to each candidate node are obtained respectively;

and respectively determining the comprehensive reputation value corresponding to each candidate node based on the direct reputation score and the indirect reputation score corresponding to each candidate node, wherein the indirect reputation score of each candidate node is obtained according to the contribution of other candidate nodes in the same network with the candidate node.

Another aspect of the present application provides a service function chain arrangement device in an industrial internet of things, including:

the request receiving module is used for receiving a service function chain arrangement request aiming at the industrial Internet of things;

the balance arrangement module is used for arranging the service function chain arrangement request according to an arrangement algorithm for balancing the service function chain energy consumption and the time delay to obtain an arrangement result of the corresponding service function chain;

and the physical deployment module is used for distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things.

Another aspect of the present application provides a federal learning system for industrial internet of things, comprising: the intelligent equipment layer, the edge cloud layer and the center cloud layer are sequentially connected;

a central node is arranged in the central cloud layer and is used for executing the service function chain arrangement method in the industrial Internet of things;

the intelligent equipment layer is used for receiving the service function chain arrangement request aiming at the industrial Internet of things and forwarding the service function chain arrangement request to the central node through the edge cloud layer;

and each working node is contained in the edge cloud layer.

In another aspect, the application provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the service function chain arrangement method in the industrial internet of things when executing the computer program.

Another aspect of the present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the method of service function chaining in industrial internet of things.

The service function chain arrangement method in the industrial Internet of things provided by the application receives a service function chain arrangement request aiming at the industrial Internet of things; arranging the service function chain arrangement request according to an arrangement algorithm for balancing the energy consumption and the time delay of the service function chain to obtain an arrangement result of the corresponding service function chain; distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things; the service function chain arranging request is arranged by adopting an arranging algorithm for balancing the energy consumption and the time delay of the service function chain, so that the energy consumption expenditure of instantiation or port transmission of the service function chain obtained by arranging in the industrial Internet of things can be effectively improved, the end-to-end time delay of the service function chain can be effectively minimized when the service function chain is initially arranged, the controllability of the end-to-end time delay can be ensured in the working process of the service function chain, namely, the energy consumption and the time delay factor of the service function chain can be balanced simultaneously, the reliability and the intelligent degree of the service function chain arranging process in the industrial Internet of things can be improved, and the reliability and the effectiveness of the service function chain can be further effectively improved.

Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.

It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present application are not limited to the above-described specific ones, and that the above and other objects that can be achieved with the present application will be more clearly understood from the following detailed description.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate and together with the description serve to explain the application. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. Corresponding parts in the drawings may be exaggerated, i.e. made larger relative to other parts in an exemplary device actually manufactured according to the present application, for convenience in showing and describing some parts of the present application. In the drawings:

fig. 1 is a general flow chart of a method for arranging service function chains in an industrial internet of things according to an embodiment of the application.

FIG. 2 is a schematic diagram of a federal learning system according to an embodiment of the present application.

Fig. 3 is a schematic flow chart of a method for arranging service function chains in an industrial internet of things according to an embodiment of the application.

Fig. 4 is a schematic structural diagram of a service function chain arrangement device in an industrial internet of things according to another embodiment of the present application.

Fig. 5 is a schematic diagram of a DRL architecture provided by an application example of the present application.

Fig. 6 is a schematic diagram of an experimental topology.

Fig. 7 is a graph comparing the convergence efficiency of federal DRL and conventional DQN.

FIG. 8 is a graph comparing training curves for different node selection methods.

Fig. 9 is a graph of time delay comparisons for different orchestration methods.

FIG. 10 is a schematic diagram showing a comparison of energy consumption of different orchestration methods.

Detailed Description

The present application will be described in further detail with reference to the following embodiments and the accompanying drawings, in order to make the objects, technical solutions and advantages of the present application more apparent. The exemplary embodiments of the present application and the descriptions thereof are used herein to explain the present application, but are not intended to limit the application.

It should be noted here that, in order to avoid obscuring the present application due to unnecessary details, only structures and/or processing steps closely related to the solution according to the present application are shown in the drawings, while other details not greatly related to the present application are omitted.

It should be emphasized that the term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.

It is also noted that the term "coupled" may refer to not only a direct connection, but also an indirect connection where an intermediate is present, unless otherwise indicated.

Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

Industrial internet of things facilitates interconnection of various intelligent devices in sensor, production line modern industrial production and automation environments. In the IIoT environment, various devices will generate a large number of data streams, and how to handle forwarding of these data streams, reducing network delay becomes a challenge faced by IIoT. Meanwhile, devices in IIoT often move frequently, such as mobile robots, intelligent mechanical arms, unmanned trucks, and mobile communication devices carried by workers, which also puts new demands on flexibility provided by industrial internet of things services. In order to solve the demands of real-time performance, resource diversity, safety and the like in the industrial Internet of things, various researches introduce network function virtualization and mobile edge calculation, and orderly arrange service function chains SFC (Service Function Chain) in an edge network, so that data forwarding in the industrial Internet of things is more flexible, qoS requirements of users are met, and manageability and flexibility of the industrial Internet of things are greatly improved.

The network function virtualization NFV technology uses virtual network functions to replace physical dedicated hardware, and greatly simplifies the complexity of deployment and management. The virtual network function is realized by means of software, so that the cloud server can be used as a resource pool of different network functions, is not limited by geographic factors, and has high flexibility. NFV provides the pooled hardware resources to the user in a more reasonable manner after centralizing the pooled hardware resources, and replaces offline manual repeated configuration with online operation of the network administrator, thereby greatly reducing the difficulty of deployment. In addition, the virtual coverage capability of the NFV, combined with the centralized control and hardware decoupling capability of the software defined network SDN, facilitates the establishment of a network that facilitates expansion and maintenance,

one large field of application of NFV is service function chaining SFC, which functions to put services (e.g., firewall, virus detection, etc.) required by a user into the path of a user session, so that traffic is properly handled by the network service. Before using the virtualization technology, each service node is a special hardware, the system is different from manufacturer to manufacturer, the development and deployment process is complicated, and single-point faults are frequent. The virtualization technology instantiates network functions into virtual machines VM in the server, the realization of SFC is not bound with specific equipment, and the upper layer application is responsible for arranging services and mapping the services into the lower layer physical infrastructure, so that the system has agile deployment and load balancing capabilities. Flow steering can be achieved more flexibly in combination with an SDN controller.

In networks built based on NFV technology, SFC services are typically delay sensitive, with the most important indicator of quality of service being the end-to-end delay of the SFC. Not only is it required to minimize the end-to-end delay during the initial programming of the SFC, but it is also required to ensure the controllability of the end-to-end delay during the SFC operation. Therefore, in order to ensure the network service quality required by the user, efficient and reasonable arrangement and dynamic optimization of the SFC are required. From the perspective of operators, time delay sensing SFC arrangement is to be realized, namely virtual network node (VNF) instances are reasonably placed into a network and orderly linking among the VNs is completed, so that the end-to-end time delay is minimized, meanwhile, the effective use of bottom physical network resources is ensured, and the utilization rate of the resources is improved.

The existing service function chain arrangement method is exemplified as follows:

(one) method one: the invention is applicable to dynamic scene arrangement of service function chains, and has wider application range. The service function chain automatic arrangement architecture provided by the invention has the automatic arrangement capability of closed-loop control, can cope with the scene that the service demand and the network condition are continuously changed, and has stronger adaptability. The corresponding implementation method is provided on the basis of providing the service function chain automatic arrangement architecture, and the method has higher feasibility compared with the existing arrangement architecture research.

(II) method II: a service function chain arrangement method, system and arrangement device for edge computing service comprises: creating an edge computing service descriptor; after receiving an edge computing service request, converting the edge computing service request into a network service request and an edge computing application request based on the edge computing service descriptor description; and orchestrating a service function chain for the edge computing traffic based on the network traffic request and the edge computing application request. The edge computing service processing method provides a specific arrangement mode for the edge computing service, and at least can effectively improve the processing efficiency of the edge computing service.

(III) method III: a service function chain deployment method and system based on group learning belong to the field of communication, and the method comprises the following steps: dividing a physical function area into a plurality of sub-function areas, dividing a service function chain into a plurality of sub-service function chains, wherein the sub-function areas correspond to the sub-service function chains one by one; each sub-functional area carries out deep reinforcement learning training on the local deployment model by utilizing the local data set of the sub-functional area; randomly selecting a sub-functional area to perform parameter aggregation based on group learning on the trained local deployment model to obtain an aggregation model, and updating the local deployment model of each sub-functional area into the aggregation model; and repeating the deep reinforcement learning training, parameter aggregation and updating operation based on group learning until the global loss function converges to obtain a final deployment model, and deploying each sub-service function chain to a corresponding sub-function region according to the final deployment model. The invention can improve network service performance and network security.

The method solves the problem of arranging the service function chain by using different methods and flows respectively. The first method provides a dynamic arrangement structure and an implementation method, which are different from the arrangement algorithm research angle researched by the application; creating an edge computing service descriptor; after receiving an edge computing service request, converting the edge computing service request into a network service request and an edge computing application request based on the edge computing service descriptor description; and arranging a service function chain for the edge computing service based on the network service request and the edge computing application request, wherein the scheme considers the service function chain request under the edge environment, does not consider the optimization targets of time delay and energy consumption, and does not solve the unreliable asking for the service function chain arrangement result. The method three considers the arrangement mode of the group learning service function chain, divides the service function chain into sub-functions for arrangement, and does not consider the joint optimization of QoS and energy consumption of users.

That is, no consideration is given to the optimization index of the service function chain comprehensively in any existing service function chain arrangement method.

Based on this, the embodiment of the application provides a method for arranging service function chains in an industrial internet of things, referring to fig. 1, the method for arranging service function chains in the industrial internet of things specifically includes the following contents:

step 100: a service function chain orchestration request for an industrial internet of things is received.

In one or more embodiments of the application, NFV refers to network function virtualization (Network Function Virtualization); the VNF refers to a virtual network node, which is a virtual network function unit in the NFV architecture; SFC refers to service function chain; VM refers to virtual machine; SDN refers to a software defined network; RL refers to reinforcement learning (Reinforcement Learning); DRL refers to deep reinforcement learning; DQN refers to deep Q-network; BIP (Binary Integer Programming) model refers to binary integer programming model; the working node can be written as a worker; the means for performing the service function chain orchestration method in the industrial internet of things may be a network function virtualization orchestrator NFVO node, a VNF orchestrator, or an NFV coordinator.

Step 200: and arranging the service function chain arrangement request according to an arrangement algorithm for balancing the energy consumption and the time delay of the service function chain to obtain an arrangement result of the corresponding service function chain.

In one or more embodiments of the application, the federal learning system, as shown in FIG. 2, includes a trusted center cloud layer (or: center cloud layer), an edge node layer (or: edge cloud layer), and a device layer (or: smart device layer). The intelligent device layer is composed of intelligent devices for monitoring various public infrastructure environments. Because of the limited resources, they can only filter locally used data and then send it to the edge cloud to request services. The edge cloud layer includes edge nodes bearing the deep reinforcement learning DRL (Deep Reinforcement Learning) local model and nodes VNF1, VNF2, VNF3, and so on bearing the virtual network architecture VNF. The central cloud layer is used for completing the model aggregation.

It can be understood that the network topology of the physical network is represented by g= (V, L). Where L represents a link set in the network, V represents a service node for assuming a network function vnf, and the node has the capability of forwarding and assuming vnf services at the same time. i, j e V denotes a node in the network, and when there is a direct physical link connection between the physical nodes i, j, the link between i, j is denoted as l _i,j . C for total physical computing resources on node v _v And (3) representing. In a network model of a common NFV architecture, a physical network node often also has a certain memory resource and a storage resource. However, because constraint conditions of different resources on the node are similar to those of the computing resources, the computing resources are regarded as node resources without considering the memory, storage resources and the like of the node in order to simplify problem modeling. Each physical link has a certain bandwidth processing capability, which is marked as

Each service function chain SFC consists of an ingress node, an egress node and a VNF request sequence. Can use a directed service function graph G _f ＝(V _f ,L _f ) G represents _f Representing SFC, V _f Representing a set of VNFs. k (k) _f ,l _f Is G _f Two nodes (ingress node, egress node), k _f ,l _f The virtual links between are defined as

Any one SFC request is marked as s, s epsilon SFCs. Each request s has a series of vnfs that need to be deployed _s In order to implement the corresponding network functions, each VNF consumes a certain physical node resource to process data traffic when placed on a physical network. f E vnf _s C for requested physical resource _f A representation; each virtual linkThe requested physical bandwidth resource is then used +.>And (3) representing.

In the present application, the VNF layout problem is modeled as a BIP problem, and the parameters used are shown in table 1:

TABLE 1

Creation of an SFC service can be seen as deployment of SFC requests on a fabric sub-graph of the underlying physical network, this process is noted as:under the constraint of physical resource requirements, deployment mapping and the like, the orchestration process is two mutually associated stages: (1) Placement of VNF. Flexibly placing the VNF in the SFC request onto the appropriate physical node, denoted as f→n': N ^I ,I ^F ). (2) mapping of virtual links. Establishing a mapping relation between virtual links and physical links for ordered VNs, forming a data path by calculating a specific route, and recording as E-L' ^e )。

In order to ensure the service quality of the user, the influence of time delay needs to be considered in the VNF arrangement scheme, and the time delay comprises two parts: (1) processing latency of VNF on physical node; (2) propagation delay of data traffic over the physical link. The definition is shown in formula (6-3):

wherein ,representing the processing delay of VNF numbered f in SFC request s on physical node v. />Representing data traffic on physical link l _i,j Propagation delay on the antenna.

At the same time, in the definition of quality of service, it is also important to reduce the operating cost of the network. Thus, in order to increase the economics of SFC orchestration, a secondary goal of the algorithm is to reduce the total physical resource overhead as much as possible, including the resource overhead on nodes and links, etc. The definition is as follows:

equation (6-4) represents the computational resource overhead consumed by the VNF deployment onto the physical node.

Constraints introduced when SFC orchestration is performed in a particular scenario are discussed next. To guarantee the quality of service, the operator must control the maximum end-to-end delay D when handling the user's network service request _max Within a tolerable range, as shown in equation (6-5). D (D) _max Depending on the type of SFC.

While meeting current and anticipated consumption and demand relationships for physical resources, as well as other deployment constraints, during the SFC orchestration process. The following formula is expressed:

Equation (6-6) indicates that the sum of the computing resources consumed by all VNFs deployed onto v cannot exceed the total amount of computing resources of the node. Equation (6-7) represents mapping to physical link l _i,j The sum of bandwidth resources occupied by the virtual link on the physical link cannot exceed the total amount of bandwidth resources of the physical link.

Equation (6-8) indicates that one VNF in the SFC can only be deployed on one physical node. Equation (6-9) indicates that a virtual link may be mapped onto one or more physical links. Equation (6-10) shows that when the virtual links are mapped onto the physical links, the mapped physical links are guaranteed to be continuous paths, and the traffic conservation theorem is satisfied.

Based on the above discussion, noteω ₁ ，ω ₂ For an adjustable optimization ratio, the optimization objective of the present application can be written as:

OPT：minR(D,C)

step 300: and distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things.

As can be seen from the above description, according to the service function chain arrangement method in the industrial internet of things provided by the embodiment of the application, by arranging the service function chain arrangement request by adopting the arrangement algorithm for balancing the service function chain energy consumption and the time delay, the energy consumption cost for optimizing the instantiation or the port transmission of the service function chain obtained by arrangement in the industrial internet of things can be effectively improved, the end-to-end time delay of the service function chain can be effectively minimized when the service function chain is initially arranged, the controllability of the end-to-end time delay can be ensured in the working process of the service function chain, namely, the service function chain energy consumption and the time delay factor can be balanced at the same time, the reliability and the intelligent degree of the service function chain arrangement process in the industrial internet of things can be further improved, and the reliability and the effectiveness of the service function chain can be further effectively improved.

The method aims at the problems of the existing service function chain arrangement method that: with the increase of network scale and data, the traditional linear programming solving method is slower in speed, and a new algorithm needs to be designed to improve algorithm efficiency; in the method for arranging the service function chains in the industrial internet of things provided by the embodiment of the application, the arrangement algorithm in the method for arranging the service function chains in the industrial internet of things comprises the following steps: global programming model for balancing service function chain energy consumption and time delay based on federal learning and deep reinforcement learning training.

From the above description, it can be seen that, according to the service function chain arrangement method in the industrial internet of things provided by the embodiment of the application, by adopting the global arrangement model obtained by combining the federal learning and the deep reinforcement learning DRL and training, the training efficiency of the arrangement model can be further improved on the basis of ensuring the improvement of the service function chain arrangement reliability.

In order to further improve the training efficiency of the arrangement model, in the method for arranging the service function chain in the industrial internet of things provided by the embodiment of the application, referring to fig. 3, before step 100 of the method for arranging the service function chain in the industrial internet of things, the method specifically further includes the following contents:

Step 010: a plurality of working nodes is selected in the edge cloud layer.

Step 020: based on a federal learning algorithm, distributing model parameters of a current global programming model for balancing service function chain energy consumption and time delay to each selected working node, so that each working node carries out deep reinforcement learning training on a local programming model corresponding to the model parameters based on respective local data, and outputting model parameters of the local programming model obtained by respective training.

Step 030: and receiving the model parameters of the local programming model sent by each working node respectively, and carrying out aggregation processing on the model parameters of each local programming model so as to obtain the updated global programming model.

It can be appreciated that as the network scale increases, the time complexity of the traditional linear programming solution increases dramatically, reinforcement learning is applied in the service orchestration field and achieves good results, but previous studies are all centralized decision-making approaches, and in some studies, the high performance and high efficiency of federal learning are pointed out, so the application combines federal learning with reinforcement learning to train a model to obtain an orchestration scheme.

It can be understood that in order to avoid the falsification of the arrangement result by the untrusted nodes and improve the convergence rate of the model, k nodes carrying the DRL model, called a worker, are deployed in each edge cloud. These nodes may be used for model training. Meanwhile, the present application requires the generation of a global model from the decentralized NFVO nodes. At each aggregation round t=1, 2 …, the central cloud assigns a pre-trained global model to the worker in the edge cloud. Each NFV coordinator then performs a local training step to train its own model. The edge clouds then select a coordinator from each edge cloud to upload its model to perform the joint average in the next round, and after model aggregation, the center cloud will redistribute the joint average model to the workers in the same edge cloud.

As can be seen from the above description, according to the service function chain arrangement method in the industrial internet of things provided by the embodiment of the application, model parameters of the current global arrangement model for balancing the energy consumption and the time delay of the service function chain are distributed to each selected working node based on the federal learning algorithm, so that each working node performs deep reinforcement learning training on the local arrangement model corresponding to the model parameters based on the local data, the reliability and the effectiveness of federal learning can be improved, and the training efficiency of the arrangement model can be further improved.

In order to reduce energy consumption caused by service function chain deployment, in the service function chain arrangement method in the industrial Internet of things provided by the embodiment of the application, the global arrangement model and the local arrangement model in the service function chain arrangement method in the industrial Internet of things are Deep Reinforcement Learning (DRL) models based on DQN;

Specifically, RL is an important branch of machine learning, researching optimization and adaptive decisions of action strategies through frequent interactions between the time-varying environment and smart agents. In Q-learning, the action value function is implemented by a Q table and the optimal strategy is learned by updating. Therefore, Q-learning is suitable for problems with small scale discrete state space and motion space. To accommodate the large scale dynamic environment, DRLs have been proposed that combine DNNs with RL. And establishing a Q table by using the DNN by utilizing nonlinear approximation of the DNN, and converting the update of the Q table into the update of the weight of the neural network. In order to accelerate the training process of the DRL and improve the convergence performance of the DRL, advanced technologies such as experience playback and fixed target network are researched. In the empirical replay technique, a randomly selected conversion history is used to update the DRL model to break the association between successive conversion tuples. In the fixed target network technique, a target Q network is established to predict a target Q value, and a delay update of the target Q network is employed to accelerate and stabilize the training process. Thus, the DRL can be applied to a large scale scene with a continuous value state space. The problem considered in the present application is the discrete action space, and therefore, a deep Q-network (DQN) based DRL framework is used.

From the above description, it can be seen that, by adopting the DRL model based on DQN, the service function chain arranging method in the industrial Internet of things provided by the embodiment of the application can effectively realize dimension reduction by predicting the Q value in Q-learning by using the neural network, and can effectively reduce the energy consumption brought by service function chain deployment on the premise of ensuring the service quality of the service function chain under the scene that SFC dynamically arrives.

In order to improve the arrangement quality of service function chains in the industrial internet of things, in the arrangement method of service function chains in the industrial internet of things provided by the embodiment of the application, step 010 in the arrangement method of service function chains in the industrial internet of things further specifically comprises the following contents:

step 011: generating comprehensive reputation values corresponding to candidate nodes in the edge cloud layer respectively based on a preset subjective logic model;

step 012: and sequencing the candidate nodes according to the sequence of the comprehensive reputation value from high to low, and selecting a plurality of candidate nodes as the working nodes according to the corresponding sequencing result.

As can be seen from the above description, according to the method for arranging the service function chains in the industrial internet of things provided by the embodiment of the application, the subjective logic model is adopted to generate the comprehensive reputation value corresponding to each candidate node in the edge cloud layer, so that the model uploaded to the center can be selected, the malicious node is prevented from tampering with the model data, and the arrangement quality of the service function chains in the industrial internet of things can be further effectively improved.

In order to further improve the arrangement quality of service function chains in the industrial internet of things, in the service function chain arrangement method in the industrial internet of things provided by the embodiment of the application, step 011 in the service function chain arrangement method in the industrial internet of things further specifically comprises the following contents:

step 0111: and sorting the Euclidean distance sum corresponding to each candidate node according to the sequence from big to small according to the Euclidean distance sum between the local arrangement model of each candidate node and the local arrangement model of other candidate nodes, and assigning the sorting result based on a preset score assignment rule to obtain the initial score corresponding to each candidate node.

Step 0112: and respectively obtaining the direct reputation score corresponding to each candidate node according to the communication quality of the link between each candidate node and the central cloud and the initial score corresponding to each candidate node.

Step 0113: and respectively determining the comprehensive reputation value corresponding to each candidate node based on the direct reputation score and the indirect reputation score corresponding to each candidate node, wherein the indirect reputation score of each candidate node is obtained according to the contribution of other candidate nodes in the same network with the candidate node.

As can be seen from the above description, according to the method for arranging the service function chains in the industrial internet of things provided by the embodiment of the application, the comprehensive reputation value corresponding to each candidate node is determined by adopting the direct reputation score and the indirect reputation score corresponding to each candidate node, so that the availability and the reliability for acquiring the comprehensive reputation value can be effectively improved, and the arrangement quality of the service function chains in the industrial internet of things can be further improved.

From the software aspect, the application further provides an apparatus for arranging service function chains in the industrial internet of things for executing all or part of the method for arranging service function chains in the industrial internet of things, referring to fig. 4, the apparatus for arranging service function chains in the industrial internet of things specifically comprises the following contents:

the request receiving module 10 is configured to receive a service function chain arrangement request for the industrial internet of things.

And the balance arrangement module 20 is used for arranging the service function chain arrangement request according to an arrangement algorithm for balancing the service function chain energy consumption and the time delay to obtain an arrangement result of the corresponding service function chain.

And the physical deployment module 30 is used for distributing physical resources to the service function chain based on the arrangement result and deploying the service function chain into the physical network of the industrial Internet of things.

The embodiment of the service function chain arrangement device in the industrial internet of things provided by the application can be particularly used for executing the processing flow of the embodiment of the service function chain arrangement method in the industrial internet of things in the embodiment, and the functions of the embodiment of the service function chain arrangement device in the industrial internet of things are not repeated herein, and can be referred to the detailed description of the embodiment of the service function chain arrangement method in the industrial internet of things.

The part of the service function chain arrangement device in the industrial internet of things for arranging the service function chain in the industrial internet of things can be executed in a server, and in another practical application situation, all operations can be completed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The application is not limited in this regard. If all operations are completed in the client device, the client device may further include a processor for specific processing of service function chain arrangement in the industrial internet of things.

The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server of an intermediate platform, such as a server of a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.

Any suitable network protocol may be used between the server and the client device, including those not yet developed on the filing date of the present application. The network protocols may include, for example, TCP/IP protocol, UDP/IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol may also include, for example, RPC protocol (Remote Procedure Call Protocol ), REST protocol (Representational State Transfer, representational state transfer protocol), etc. used above the above-described protocol.

As can be seen from the above description, the service function chain arrangement device in the industrial internet of things provided by the embodiment of the application can effectively improve and optimize the instantiation of the service function chain or the energy consumption overhead of port transmission obtained by arrangement in the industrial internet of things by arranging the service function chain arrangement request by adopting the arrangement algorithm for balancing the energy consumption and the time delay of the service function chain, can effectively minimize the end-to-end time delay of the service function chain when the service function chain is initially arranged, can also ensure the controllability of the end-to-end time delay in the working process of the service function chain, namely can balance the energy consumption and the time delay factor of the service function chain at the same time, and can further improve the reliability and the intelligent degree of the service function chain arrangement process in the industrial internet of things.

In addition, the application also provides a federal learning system of the industrial internet of things, referring to fig. 2, the federal learning system of the industrial internet of things specifically comprises the following contents:

the intelligent equipment layer, the edge cloud layer and the center cloud layer are sequentially connected;

and each working node is contained in the edge cloud layer.

In order to further explain the scheme, the application also provides a specific application example of the service function chain arrangement method based on federal reinforcement learning for the industrial Internet of things, and the service function chain arrangement has several problems in combination with the existing researches at present: 1) The arrangement of the service function chain needs to consider time delay and energy consumption, and the relation between the time delay and the energy consumption is weighed when a model is established; 2) With the increase of network scale and data, the traditional linear programming solving method is slower in speed, and a new algorithm needs to be designed to improve algorithm efficiency; 3) In environments where different vendors of edge clouds share network resources, service orchestration may be unreliable, requiring design algorithms to exclude malicious nodes.

The purpose of the application example is to comprehensively consider the optimization index of the service function chain and improve the training efficiency and reliability of the programming model, and the specific description is as follows:

(1) For the service function chain performance problem, part of literature only focuses on optimizing time delay, and optimizing energy consumption expenditure of VNF instantiation or port transmission by adopting different modeling methods, and part of literature only focuses on maximizing the number of service function chains carried in a network and does not focus on service quality, so that during modeling, two constraints of energy consumption and time delay are required to be focused and balanced.

(2) With the increase of network scale, the time complexity of the traditional linear programming solving method is greatly increased, reinforcement learning is applied to the field of service arrangement and achieves good effects, but the previous researches are all centralized decision modes, and in some researches, the high performance and high efficiency of federal learning are pointed out, so that the application combines federal learning and reinforcement learning to train a model to acquire an arrangement scheme.

(3) Different heterogeneous networks may have the effect that the result of the orchestration is not trusted, and the federal learning decentralized learning environment can alleviate this problem to some extent, but in order to further improve the quality of the orchestration, an improved federal learning algorithm is proposed, eliminating malicious orchestrators in the network, reducing its impact when model aggregation.

Problem modeling

1. System model

The federal learning system is shown in figure 2.

2. DQN training procedure

Referring to fig. 5, the drl framework is composed of an environment and an agent. Each intelligent DRL agent performs action decisions that interact with the viewing environment. The environment is mainly composed of server nodes in the environment. The present application is presented below with definitions of status, actions and bonus functions.

1) State set: the real-time state of the system is the key to determining the VNF deployment location in the SFC. The system state may be defined as follows depending on the network state and vnf deployment scenario. The application uses s _t Representing the current state of the system, mainly comprising two parts wherein ,/>Indicating the total amount of available resources for each node of the current network, < >> Representing the vnf category to be queued for the current request, < >>

2) Action set: the deployment of VNFs needs to be done according to the current system state. The application regards the combination of physical nodes as one action, if a certain action is selected, it means that the application will deploy vnf on these physical nodes;

for a learning agent, the rewards earned include two aspects: penalty rewards and goal rewards. When there is a behavior that does not meet the constraint, a penalty will be given, and joint rewards refer to OPT-based rewards. Thus, r _i Is defined as

ThenIs defined as +.>Wherein if x is false, f (x) =1; the method comprises the steps of carrying out a first treatment on the surface of the Otherwise Γ (x) =0; omega _p Is a penalty factor omega _p ＜0。

The application mainly considers how to reduce the energy consumption brought by SFC deployment on the premise of ensuring the service quality of SFC under the scene of SFC dynamic arrival. In a real NFV network, SFC requests arrive and leave continuously over time. The arrival and departure of SFC requests are random. When the SFC request arrives, the SFC request is arranged by calling an arrangement algorithm, and physical resources are distributed for the SFC according to an arrangement result and deployed into a physical network.

In model training using DQN, training data is first generated as follows: if the state at the current time t is S, there is a new incoming SFC request S, for each f ε vnf _s The present application performs each action a by placing f in the corresponding network region _i Then transition to the next state S' of the VNF post-placement state to get its prize r _ai . All results include [ S, a _i ,r _ai ,S′]Are recorded as training data. The application then operates optimally in the physical network with the highest return and then continues to optimize the VNF layout for the next time interval. The above process is repeated, and after enough data is sampled, the data is taken out of the database for training the network and updating the network.

Since DQN achieves dimension reduction by predicting Q value in Q-learning with a neural network, a loss function in DQN is defined as follows:

wherein Shows the target Q value, Q (s _j ,a _j The method comprises the steps of carrying out a first treatment on the surface of the θ) is the predicted Q value, s _j+1 A' represents the state and action at the next moment.

3. Federal reinforcement learning service function chain arrangement algorithm

While the DRL algorithm is more suitable for large-scale networks and dynamic environments than traditional heuristic algorithms, the acquisition of training speed and training data presents significant challenges for DRL model training. Furthermore, in a complex edge network environment, the reliability of the training of each edge network model cannot be guaranteed. The DRL engine may tamper with the model and cause performance degradation.

In order to avoid the falsification of the arrangement result by the untrusted nodes and improve the convergence rate of the model, k nodes carrying the DRL model, called a worker, are deployed in each edge cloud. These nodes may be used for model training. Meanwhile, the present application requires the generation of a global model from the decentralized NFVO nodes. At each aggregation round t=1, 2 …, the central cloud assigns a pre-trained global model to the worker in the edge cloud. Each NFV coordinator then performs a local training step to train its own model. The edge clouds then select a coordinator from each edge cloud to upload its model to perform the joint average in the next round, and after model aggregation, the center cloud will redistribute the joint average model to the workers in the same edge cloud. The core part of the joint learning process is joint averaging, which can be expressed as:

Since efficient and accurate reputation calculation is critical to reliable joint learning. In this section, the present application applies a subjective logic model to generate the composite reputation value of the worker candidate. Subjective logic is used to evaluate the trustworthiness or reliability level of different entities. Subjective logic uses the term "opinion" to mean passing positive, negative and uncertain. In the application, in order to obtain a more accurate reputation value of the worker candidate, each VNFO combines its direct reputation value with an indirect reputation value to generate a comprehensive reputation value of the worker candidate. See algorithm 1 shown in table 2.

TABLE 2

In this section, a subjective logic model is used to calculate the total reputation score of the worker node worker. The subjective logic model consists of two parts: direct reputation and indirect reputation. An indirect reputation of a node is contributed by other nodes in the same network as the node; the direct reputation is determined by the behavior of the node. Each action of a node may be considered positive, negative or indeterminate. By aggregation of the direct reputation and the indirect reputation, a more accurate working node reputation value can be obtained.

Consider a worker _i And a central cloud node C _j ，worker _i The node will exchange data with the central cloud node, and the application assumes that the node reputation update time is r. In subjective logic, C _j To worker _i Trust of (a) can be described as opinion vector wherein /> and />Representing trust, distrust and uncertainty, respectively.

It should be noted that reputation opinions can be affected by a number of factors. The following factors are considered as factors in calculating the reputation of a worker.

Given two models, the present application uses Euclidean distance (one of the most common methods of measuring distance in high dimensional space) to measure the difference between them.

In order to force the aggregated global model to deviate from benign models, the bad local model must be different from the benign local model trained by honest workers. Considering that the learning objective of an untrusted worker is quite different from the learning objective of a trusted participant, there may be significant differences between the trusted and untrusted local models.

Thus, for all candidate working nodes, the sum of Euclidean distances of the model and the remaining models is calculated, all models with Euclidean distances greater than a threshold are excluded, the scores are ordered, the scores are assigned according to the rank, for example, 10 nodes in total, the first name is 100, the 10 th name is 10, and the score is recorded as s _sim 。

The application uses s _i→j Representing worker i and central cloud C _j The quality of communication of the links between them, e.g. probability of successful transmission of data packets, determines the local uncertainty vectorIs not deterministic.

In summary, the direct reputation score for each node is

Remembering the indirect reputation of each node asComprehensive reputation as->

Based on the above description, a VNF arrangement algorithm is given. The present application proposes an SFC orchestration method to co-process the placements and links involved in the orchestration. First, given an NFV network g= (N, L), the maximum allowed delay is D _max And the SFC request set is SFCs. In combination with the above constraints, the current VNF set is traversed, algorithm 2 as shown in table 3 is executed, and the most rewarding solution is found for VNF placement in the current request. After obtaining the solution, by passThe physical node sequence is traversed to place the VNF. Since the physical path between two physical nodes is not unique, the Dijkstra algorithm determines the routes of neighboring nodes in the sequence in 8 steps. Step 9 implements the mapping between the virtual link and the physical link. And finally, determining the SFC arrangement result, distributing the required physical resources for the SFC, and realizing the end-to-end creation and management of the network service.

Namely: based on the description, the SFC arrangement algorithm is provided, and the application provides an SFC arrangement method which carries out two-stage cooperative processing on arrangement and linkage. Firstly, giving a NFV network g= (N, L), the maximum tolerant delay is D _max And SFC request set Q. And traversing the current SFC set in combination with the constraint conditions and the like, executing an algorithm 1, searching a solution with the maximum benefit for the VNF placement in the current request, obtaining the solution, traversing a physical node sequence, placing the VNF, because physical nodes are not unique in a physical path between every two physical nodes, and determining a route for adjacent nodes in the sequence in 10 steps by using a Di Jie St algorithm. And 11-13, mapping the virtual link and the physical link. And finally, determining the arrangement result of the SFC, distributing the required physical resources for the SFC, and realizing the end-to-end creation and management of the network service.

TABLE 3 Table 3

In summary, the application example of the present application provides a service function chain arrangement method for industrial internet of things, which has the technical key points that:

1. for the service function chain performance problem, part of the literature only focuses on optimizing time delay, and optimizing the energy consumption expense of VNF instantiation or port transmission by adopting different modeling methods, and part of the literature only focuses on maximizing the number of service function chains carried in a network and does not focus on service quality, so that the application needs to focus on constraint in two aspects of energy consumption and time delay and balance the two aspects of energy consumption and time delay during modeling.

2. In order to improve the training efficiency of the arranging model, a distributed training framework is designed, deep reinforcement learning and federal learning are fused, a time period is divided into time slots, in each training round, the model is firstly trained by a VNF (virtual network function) arranger deployed at the edge, then neural network parameters at the edge are uploaded to a central node, the central node completes model aggregation and then sends the model to the edge node, and the process is executed until the network converges.

3. In order to select a model uploaded to the center and prevent malicious nodes from tampering with model data, the quality of edge nodes needs to be evaluated to obtain direct evaluation of node reputation. The nodes used for model aggregation in each round are selected through the node scores.

Based on the above, the service function chain arrangement method in the industrial Internet of things provided by the application example of the application provides a service migration strategy based on user movement and Markov decision model in the edge network, and has the advantages that:

in order to further improve the arrangement efficiency and the reliability of service arrangement, the application provides a federal reinforcement learning service function chain arrangement algorithm. The algorithm combines federal learning and traditional reinforcement learning, accelerates the convergence of the model, and enhances the robustness of the model. Meanwhile, the reputation theory is introduced, the reliability of different arrangement models is evaluated, the anti-interference capability of federal learning is improved, and the accuracy of arrangement results is improved. The simulation result of the application shows that the proposed algorithm can meet the effects of accelerating convergence and optimizing energy consumption and time delay.

The embodiment of the application also provides a computer device (i.e. electronic device), which may include a processor, a memory, a receiver and a transmitter, where the processor is configured to execute the service function chain arrangement method in the industrial internet of things mentioned in the above embodiment, and the processor and the memory may be connected by a bus or other manners, for example, through a bus connection. The receiver may be connected to the processor, memory, by wire or wirelessly. The computer equipment is in communication connection with a service function chain arrangement device in the industrial Internet of things so as to receive real-time motion data from a sensor in the wireless multimedia sensor network and receive an original video sequence from the video acquisition device.

The processor may be a central processing unit (Central Processing Unit, CPU). The processor may also be any other general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof.

The memory is used as a non-transitory computer readable storage medium and can be used for storing non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions/modules corresponding to the service function chain arrangement method in the industrial internet of things in the embodiment of the application. The processor executes various functional applications and data processing of the processor by running non-transitory software programs, instructions and modules stored in the memory, namely, the service function chain arrangement method in the industrial internet of things in the method embodiment is realized.

The memory may include a memory program area and a memory data area, wherein the memory program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory may optionally include memory located remotely from the processor, the remote memory being connectable to the processor through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory that, when executed by the processor, perform the method of service function chaining in industrial internet of things in an embodiment.

In some embodiments of the present application, a user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter, the processor, the memory, the receiver, and the transmitter may be connected by a bus system, the memory being configured to store computer instructions, the processor being configured to execute the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.

As an implementation manner, the functions of the receiver and the transmitter in the present application may be considered to be implemented by a transceiver circuit or a dedicated chip for transceiver, and the processor may be considered to be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.

As another implementation manner, a manner of using a general-purpose computer may be considered to implement the server provided by the embodiment of the present application. I.e. program code for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general purpose processor implements the functions of the processor, the receiver and the transmitter by executing the code in the memory.

The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, is used for realizing the steps of the service function chain arrangement method in the industrial internet of things. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the art.

In addition, in order to further illustrate the effect of the service function chain arrangement method in the industrial internet of things provided by the embodiment and the application example of the application, the application also provides corresponding experimental data, which is specifically as follows:

in this experiment, two physical topologies obtained in SNDlib were used as experimental topologies. They are german backbone topologies, see fig. 6, with 14 physical nodes and 21 physical links.

The present application sets the computing resources of the physical node to 200. The bandwidth of each physical link is set to 1Gbps and the delay of the physical link is subject to a uniform distribution between 1,3 ms. Assume that there are four different VNFs in the network.

The experiment of the algorithm is realized by using Python language, and the neural network based on the DQN algorithm is realized by using Pytorch language. The neural networks in the local network and the target network adopt three-layer full-connection structures, and the parameters and the structures are the same. The size of the memory pool is 3000, and the number of samples extracted from the memory pool at a time is 64. The specific parameter settings are shown in table 4:

TABLE 4 Table 4

In addition to the SFC orchestration method (FDOA) presented in the present application, several algorithms typical of the current comparison were chosen for comparison. (1) randomly selecting an orchestration algorithm (random). The algorithm randomly selects nodes to which VNFs are to be deployed, and then obtains the shortest path between the nodes as the data forwarding path. (2) Shortest path based orchestration method (SPO), which is the best solution for latency. The algorithm directly calculates the shortest path between user endpoints as the basic path for data flow forwarding, and deploys VNFs on the physical nodes of the path. (3) Deep Q Network (DQN).

Fig. 7 shows the learning process of federal DRL and conventional DQN. As can be seen in fig. 7, the federal DRL and the traditional DNQ final rewards tend to be similar, with federal DRLs slightly higher than traditional DRLs. However, in terms of convergence speed, the federal DRL requires 170 epochs to converge, whereas the conventional DQN requires 250 epochs to converge. This is because the federal DRL trains its global model by averaging the edge local models, excluding the effects of some extreme data, while also taking into account more training data, so that the trained global model can adapt to the dynamic environment.

FIG. 8 shows a comparison of training processes based on both a reputation selection model and a random selection of ignoring reputation. In experiments, the DQN model used in the application is provided with 10 worker nodes, wherein one worker uploads the wrong DQN model at a fixed frequency. It can be seen from the figure that if reputation evaluation is not performed, the training effect is greatly affected.

FIG. 9 shows the end-to-end delay caused by different orchestration methods in processing SFC requests as the number of SFC requests increases. It can be seen that as the number of SFC requests increases, the delay generated by all four algorithms increases. Wherein, the end-to-end time delay obtained by the SPO is shortest and the random time delay is longest. The FDOA method gives delay results close to SPO. This is because SPO forwards data traffic over the shortest path between user endpoints. However, network resources are actually limited. SPO will result in SFC requests being deployed on several shortest paths, which can easily lead to link congestion and degrade overall network performance. FDOA dynamically deploys VNF to near-optimal nodes, and in the process of searching optimal solution, the FDOA approaches to the minimum delay solution as much as possible, and good delay results are obtained. This suggests that FDOA can meet the latency requirements of SFC orchestration.

Fig. 10 shows the power consumption of the different methods as the number of SFC requests increases, as shown. FDOA has the best energy consumption performance, and the energy consumption is lower than SPO and DQN. In the case of 3 SFCs, the delays obtained by the Random, DQN and SPO algorithms were 52, 55 and 72KW, respectively. Since the node is randomly selected to deploy the VNF, while the SPO deploys the VNF on several shortest paths, while it improves node resource utilization, it may ignore reuse of running devices, which would result in high power consumption. While energy minimization is one goal of DQN, its training effect is not as good as FDOA, which may result in higher energy consumption. This suggests that FDOA can meet energy minimization requirements.

Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein may be implemented as hardware, software, or a combination of both. The particular implementation is hardware or software dependent on the specific application of the solution and the design constraints. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave.

It should be understood that the application is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present application.

In this disclosure, features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, and various modifications and variations can be made to the embodiments of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for arranging service function chains in industrial Internet of things is characterized by comprising the following steps:

distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things;

wherein the scheduling the service function chain scheduling request according to the scheduling algorithm for balancing the service function chain energy consumption and the time delay comprises the following steps:

the network topology of the physical network is represented by g= (V, L); wherein L represents a physical link set, V represents a node set used for bearing network functions vnf, and the node has the capability of forwarding and bearing vnf services; i, j e V, represents a node in the network, and when there is a direct physical link connection between the nodes i, j, the physical link between i, j is denoted as l _i,j The method comprises the steps of carrying out a first treatment on the surface of the C for total physical computing resources on physical node v _v A representation; considering the computing resource of the node as node resource without considering the memory and storage resource of the node, each physical link has bandwidth processing capability, and the physical link l _i,j Owned bandwidth resources are noted as

Defining each service function chain SFC composed of an inlet node, an outlet node and a virtual network node VNF request sequence, using a directed service function graph G _f ＝(V _f ,L _f ) G represents _f Representing SFC, V _f Representing a set of VNFs; k (k) _f and l_f G is respectively _f In (a) and (k) are the ingress and egress nodes _f and l_f The virtual links between are defined as

The request of any SFC is recorded as s, s epsilon SFCs, and SFCs represents service workA set of energy chains; each request s has a series of vnfs that need to be deployed _s ，vnf _s Is a virtual service function set, and each VNF consumes physical node resources to process data traffic when placed on a physical network; f E vnf _s C for requested physical resource _f A representation; each virtual linkRequested physical Bandwidth resource usage>A representation;

considering creation of an SFC service as deployment of SFC requests on a fabric sub-graph of the underlying physical network, this process is noted as:the orchestration process is, under the constraints of physical resource requirements and deployment mapping, two phases that are interrelated:

(1) Placement of VNF: placing the VNF in the SFC request on a physical node, and marking the VNF as F;

(2) Mapping of virtual links: in order to establish a mapping relation between a virtual link and a physical link, a data path is formed by calculating a specific route and is marked as E;

wherein ,indicating whether or not VNF of number f is placed on physical node v; />Representing virtual Link->Whether or not to be matched to the physical link l _i,j ；

The impact of latency is considered in the VNF orchestration scheme, which consists of two parts:

(1) Processing delay of VNF on physical node;

(2) Propagation delay of data traffic over the physical link,

the definition is shown in formula (6-3):

wherein ,representing the processing delay of the VNF numbered f in the SFC request s on the physical node v; />Representing data traffic on physical link l _i,j Propagation delay on the upper surface;

the computational resource overhead consumed by the VNF deployment onto the physical node is defined as follows:

wherein ,E_on The starting energy consumption of the server is represented; e (E) _run Representing the running energy consumption of the server;

introducing constraint conditions: (1) The operator controls the maximum end-to-end delay D when processing the network service request of the user _max Within a tolerable range; (2) The sum of the computing resources consumed by all VNFs deployed onto v does not exceed the total computing resources of the node; (3) Mapping toPhysical link l _i,j The sum of bandwidth resources occupied by the virtual links on the physical links does not exceed the sum of bandwidth resources of the physical links; (4) one VNF in the SFC is deployed on only one physical node; (5) One virtual link is mapped to one or more physical links; (6) When the virtual links are mapped onto the physical links, the mapped physical links are continuous paths and meet the flow conservation theorem;

Recording deviceω ₁ and ω₂ For an adjustable optimization ratio, the optimization targets are written as follows: OPT: min R (D, C).

2. The method for arranging service function chains in the industrial internet of things according to claim 1, further comprising, before the receiving the service function chain arrangement request for the industrial internet of things:

selecting a plurality of working nodes in the edge cloud layer;

3. The method for arranging service function chains in the industrial internet of things according to claim 2, wherein the global arrangement model and the local arrangement model are Deep Reinforcement Learning (DRL) models based on DQN;

4. The method for arranging service function chains in the industrial internet of things according to claim 2, wherein selecting a plurality of working nodes in an edge cloud layer comprises:

5. The method for arranging service function chains in the industrial internet of things according to claim 4, wherein the generating the comprehensive reputation value corresponding to each candidate node in the edge cloud layer based on the preset subjective logic model comprises:

6. Service function chain arrangement device in industry thing networking, characterized by comprising:

the physical deployment module is used for distributing physical resources for the service function chain based on the arrangement result and deploying the service function chain into a physical network of the industrial Internet of things;

The request of any SFC is recorded as s, s epsilon SFCs, and the SFCs represent a service function chain set; each request s has a series of vnfs that need to be deployed _s ，vnf _s Is a virtual service function set, and each VNF consumes physical node resources to process data traffic when placed on a physical network; f epsilon unf _s G for requested physical resource _f A representation; each virtual linkRequested physical Bandwidth resource usage>A representation;

(1) Processing delay of VNF on physical node;

(2) Propagation delay of data traffic over the physical link,

the definition is shown in formula (6-3):

wherein ,representing the processing delay of the VNF numbered f in the SFC request s on the physical node v; / >Representing data traffic on physical link l _i,j Propagation delay on the upper surface;

introducing constraint conditions: (1) Operator is processingWhen the network service of the user requests, controlling the maximum time delay D from end to end _max Within a tolerable range; (2) The sum of the computing resources consumed by all VNFs deployed onto v does not exceed the total computing resources of the node; (3) Mapping to physical Link l _i,j The sum of bandwidth resources occupied by the virtual links on the physical links does not exceed the sum of bandwidth resources of the physical links; (4) one VNF in the SFC is deployed on only one physical node; (5) One virtual link is mapped to one or more physical links; (6) When the virtual links are mapped onto the physical links, the mapped physical links are continuous paths and meet the flow conservation theorem;

7. A federal learning system for industrial internet of things, comprising: the intelligent equipment layer, the edge cloud layer and the center cloud layer are sequentially connected;

a central node is arranged in the central cloud layer and is used for executing the service function chain arrangement method in the industrial internet of things according to any one of claims 1 to 5;

and each working node is contained in the edge cloud layer.

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of arranging service function chains in an industrial internet of things as claimed in any one of claims 1 to 5 when the computer program is executed by the processor.

9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a service function chain orchestration method in the industrial internet of things according to any one of claims 1 to 5.