WO2023152547A1

WO2023152547A1 - Deployment of a network service on a cloud infrastructure based on isolation level between service functions

Info

Publication number: WO2023152547A1
Application number: PCT/IB2022/051212
Authority: WO
Inventors: Wubin LI; Carla MOURADIAN; Róbert SZABÓ; Ákos RECSE
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2022-02-10
Filing date: 2022-02-10
Publication date: 2023-08-17

Abstract

A method and electronic device of analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure are described. The method includes selecting, for one or more service functions of the network service, one of a plurality of predefined isolation values based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure. The method further includes generating an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values. The isolation index represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure. The method further comprises facilitating a determination, using the isolation index, whether to use the possible workload placement for the network service.

Description

SPECIFICATION

DEPLOYMENT OF A NETWORK SERVICE ON A CLOUD INFRASTRUCTURE BASED ON ISOLATION LEVEL BETWEEN SERVICE FUNCTIONS

TECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of computing; and more specifically, to the deployment of a network service on a cloud infrastructure based on isolation level between service functions.

BACKGROUND ART

[0002] A network service typically involves multiple service functions and corresponding network connections that couple these service functions. The instantiation of the network service on a cloud infrastructure involves the deployment of the service functions on the compute and network resources of the cloud infrastructure. Deployment and placement of service functions on the cloud infrastructure is a complex problem, and several constraints need to be considered when deploying and placing the service functions. Latency constraints, cost constraints, and capability of the computing resources are some examples of constraints that need to be considered. Complexity of the placement and deployment of the service functions is more pronounced when the underlying cloud infrastructure that hosts the network service is distributed across multiple geographical locations with varying hardware, software, and networking capabilities.

[0003] Within a distributed cloud infrastructure, different sites may be capable of supporting multiple virtualized layers, e.g., OpenStack and/or Kubernetes, and workloads can be hosted in the different virtualized layers. Using the distributed cloud infrastructure, service functions may be provisioned among multiple independent applications and customers (or tenants) while leveraging economies of scale and sharing infrastructure supporting the multiple-layer virtualization. However, the sharing of resources of the distributed cloud infrastructure may lead to possible interference between service functions and/or tenants.

[0004] Further, in some existing service offerings today, the deployment of service functions on distributed computing systems is performed as an ad-hoc manual process. In a manual deployment process, there is no single, end-to-end process to guide the overall deployment of the network service across the distributed cloud infrastructure. SUMMARY

[0005] In some implementations, a method is disclosed of analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure. The cloud infrastructure includes resources and the network service is to comprise service functions and network connections that interconnect the service functions. The method comprises selecting, for each of the service functions, one of a plurality of predefined isolation values based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure. The different ones of the plurality of isolation values represent different levels of isolation provided by respective ones of the plurality of levels of virtualization. The method further comprises generating an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values. The isolation index represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure. The method further comprises facilitating a determination, using the isolation index, whether to use the possible workload placement for the network service.

[0006] Beneficially, the method provides a relatively simple technique for assessing how well different workloads are isolated from each other in the distributed cloud infrastructure.

Decisions may be based on the isolation index to deploy a network service with an increased isolation to provide desired security and/or performance of the network service. The method is also versatile, capable of being performed prior to deployment of the network service or after deployment of the network service. In this way, there is no requirement to collect real-time data of performance of the network service when deployed. The method is also well-suited to support customer (tenant) selection and/or customization of the network service deployment, as the isolation index may be readily assessed by the customer (tenant) when comparing the different workload placements for the deployment.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:

[0008] Figure 1A illustrates a flow diagram of exemplary operations for analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure, according to some embodiments. [0009] Figure IB illustrates a flow diagram of exemplary operations for generating an isolation index for a deployment based on the selected ones of the plurality of predefined isolation values, according to some embodiments.

[0010] Figure 1C illustrates a flow diagram of exemplary operations for determining whether to use the possible workload placement for a deployment, according to some embodiments. [0011] Figure 2A illustrates a flow diagram of exemplary operations for determining a possible workload placement for a deployment of a network service, according to some embodiments.

[0012] Figure 2B illustrates a flow diagram of exemplary operations for selecting the first computing system of the set of the computing systems to be assigned to host the first service function, according to some embodiments.

[0013] Figure 3 illustrates a flow diagram of exemplary operations for generating isolation indexes for one or more possible workload placements, according to some embodiments.

[0014] Figure 4A illustrates a block diagram of an exemplary distributed cloud infrastructure having a VM environment, according to some embodiments.

[0015] Figure 4B illustrates a block diagram of an exemplary distributed cloud infrastructure having a containerized environment, according to some embodiments.

[0016] Figure 4C illustrates a block diagram an exemplary distributed cloud infrastructure having a VM environment and a containerized environment, according to some embodiments. [0017] Figure 5A illustrates a block diagram of an exemplary system for deployment of a network service, in accordance with some embodiments.

[0018] Figure 5B illustrates a block diagram of an exemplary service template that can be used in some embodiments.

[0019] Figure 5C illustrates a block diagram of an exemplary service context than can be used in some embodiments.

[0020] Figure 5D illustrates a block diagram of an exemplary service specification that results from a service template and a service context in some embodiments.

[0021] Figure 5E illustrates a block diagram of an exemplary request that is to be processed for placement of a network service onto a distributed cloud infrastructure, according to some embodiments.

[0022] Figure 5F illustrates an exemplary placement recommendation for a network service, according to some embodiments.

[0023] Figure 5G illustrates a block diagram of exemplary cloud resources selected for a network service, according to some embodiments. [0024] Figure 6 illustrates a block diagram of an exemplary network service deployment system according to some embodiments.

[0025] Figure 7 illustrates an exemplary hierarchy for calculating an isolation index using one or more isolation values for service functions of a network service in a multi-level virtualization cloud infrastructure, according to some embodiments.

[0026] Figure 8A illustrates a block diagram of an exemplary deployment of multiple service functions, according to some embodiments.

[0027] Figure 8B illustrates a block diagram of an exemplary deployment of multiple service functions, according to some embodiments.

[0028] Figure 9A illustrates an exemplary graphical user interface for specifying parameters for possible workload placements, according to some embodiments.

[0029] Figure 9B illustrates an exemplary graphical user interface for assessing possible workload placement(s) for a deployment, according to some embodiments.

[0030] Figure 10A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments.

[0031] Figure 10B illustrates an exemplary way to implement a special-purpose network device according to some embodiments.

[0032] Figure 10C illustrates various exemplary ways in which virtual network elements (VNEs) may be coupled according to some embodiments.

[0033] Figure 10D illustrates a network with a single network element (NE) on each of the NDs, and within this straightforward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments.

[0034] Figure 10E illustrates the simple case of where each of the NDs implements a single NE, but a centralized control plane has abstracted multiple of the NEs in different NDs into (to represent) a single NE in one of the virtual network(s), according to some embodiments.

[0035] Figure 10F illustrates a case where multiple VNEs are implemented on different NDs and are coupled to each other, and where a centralized control plane has abstracted these multiple VNEs such that they appear as a single VNE within one of the virtual networks, according to some embodiments.

DETAILED DESCRIPTION

[0036] The following description describes methods and apparatus for enabling deployment of various service functions of a network service on a distributed cloud infrastructure. In the following description, numerous specific details such as logic implementations, opcodes, means to specify operands, resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

[0037] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0038] Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dotdash, and dots) may be used herein to illustrate optional operations that add additional features to embodiments of the invention. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain embodiments of the invention.

[0039] In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other.

Existing solutions for deployment of network services:

[0040] As used herein, a network service refers to a set of functions and connections between these functions, which, when instantiated on a network and computing infrastructure, offer a service that has a well-defined behavior to one or more customers. As used herein, a service function refers to a function within a service that has a well-defined functional behavior. In some embodiments, the service function may be a virtualized service function. In some embodiments, the service function can be a network service function. Examples of network functions include 3rd Generation Partnership Project (3GPP) network elements (e.g., Mobility Management Entity (MME), Serving Gateway (S-GW), and Packet Data Network Gateway (PDN-GW or P-GW)), Dynamic Host Configuration Protocol (DHCP) servers, firewalls, and other entities that provide network services. As used herein, a service function type refers to a conceptual definition of a service function. As used herein, a service specification/descriptor refers to a set of service function specifications/descriptor of the service functions that form the network service. A service specification/descriptor is used when creating an instance of the service. As used herein, a service function specification/descriptor refers to a detailed description of the characteristics of a service function type as defined, based on a template for the service function and a context for the service function, where the service context is determined for one or more customers of the service. A service function specification/descriptor is used when creating an instance of a given service function type. A service function specification/descriptor may include various information about the service function type that it describes (e.g., default values for attributes, boundaries for attributes, whether an attribute is read-only or read-write (which may depend on the reference point where an instance is accessed), a version of the service function if several versions are available for a given service function type). As used herein, a service function instance refers to an instantiation of a given service function type (e.g., service function instance in Internet Engineering Task Force (IETF) terms, and Virtualized Network Function (VNF) instance in European Telecommunications Standards Institute (ETSI) terms). As used herein, consumer-side refers to the entity that uses the capabilities offered by a service function instance and provider-side refers to the entity that offers access to a service function instance. As used herein, a service context refers to a set of parameters and service policies that can be determined by a customer or a service provider based on the customer’ s request for service, where the service context is for defining the requested service. The service context may be determined based on the service level agreement that the customer has agreed upon with the provider of the service. The service context can be expressed in terms of input parameters and policies that are defined for the service requested by the customer.

[0041] Deployment of a network service involves the selection of the characteristics of the service functions that form the service and the placement of these service functions on computing, storage, and networking resources of the cloud infrastructure to be instantiated to offer the service.

[0042] Some existing approaches for deployment of a network service in a distributed cloud environment rely on a model-driven system that uses a master service orchestrator, which unpacks a cloud services archive file to retrieve and execute a deployment plan. The cloud services archive file describes topological requirements and the cloud computing and networking requirements of the network service. The system extracts from the cloud services archive file a cloud resource configuration template, a cloud network configuration template, and an application dependency graph to determine the exact order and all the execution parameters of deployment steps required to instantiate the multiple functions of the network service. In this approach, the cloud services archive file determines completely the placement sites (distributed cloud computing systems) and service functions characteristics of each one of the network service functions. However, this approach has the significant disadvantage that it only executes an already existing deployment plan as all placement locations and characteristics of the service functions are defined in the cloud services archive file. The burden of designing the service component locations and flavor selection is delegated to the creator (which in many present-day systems is a person) of the cloud services archive file.

[0043] Another approach of deployment of a network service in a distributed cloud environment claims the design of a placement method based on an Integer Linear Program (ILP) which outputs the selected locations of the service functions over an infrastructure of a distributed cloud. The approach further chooses where to route the traffic between the service functions, which can be realized over a software defined network (SDN) architecture. The approach matches the functional and capacity requirements of each service function against the distributed cloud infrastructure. However, this approach does not consider the selection of the characteristics of the service functions when performing the placement. In this approach there is a static set of service functions that are defined prior to being placed. Some other approaches rely on heuristics for enabling deployment of service functions. A greedy backtracking searchbased approach is an example of meta-heuristic that can be used for tackling the service functions placement problem.

[0044] To adapt to today’s offerings in cloud computing environments, network services can be deployed on different types of data centers and cloud computing systems (e.g., edge-clouds, etc.), multiple execution environments, different hardware platforms, and need to support rolling upgrades of service functions, geographical constraints, policy constraints associated with the service, etc. Further, in service virtualization platforms several changes can continuously occur. A network service and the system used for deployment of the service may undergo one or more of hardware upgrades, execution environment upgrades, service component updates, composite service updates, packaging updates, varying co-deployment and interference, updates of deployment locations, etc. In addition, a service may need to be deployed differently for different customers based on the location of the customer, the characteristics of the service functions available at the location of the customer, and the compute and network resources available for deploying the service for that particular customer. Therefore, statically configured service specifications are inadequate for keeping up with the changes and variations that need to be considered when deploying a set of service functions.

[0045] In some deployment approaches of a network service, a Lifecycle Manager (LCM) (e.g., a virtual network functions manager (VNFM)) may be used to select decomposition options, substitutions, software versions and deployment flavors of the service functions that form the network service. Such selection can be referred to as service functions definition and may come before (e.g., indirect mode VNFM) or after (e.g., direct mode VNFM) the placement decision. The placement decision includes the determination of a cloud computing and/or networking resource on which the service function is to be hosted. For example, the placement decision for a service function may include the determination of a computing system that is to host the service function. In some instances, the placement decision further includes the selection of a network resource (such as a Layer 3 Virtual Private Network (L3VPN)) for coupling the service function hosted on the computing resource (e.g., a data center) with another service function that is hosted on another computing resource (e.g., the same or a different data center).

[0046] In some instances the service functions definition is performed before the placement decision that determines where the service function is to be deployed. However, this approach presents a significant disadvantage. When a set of service functions are collectively defined prior to their placement, the placement of these service functions can become challenging as the best placement location for these service functions that satisfy the service requirements (such as latency constraints, location, and topological dependencies) may not support the selected service functions. As a non-limiting example, to satisfy a service requirement of the requested service (such as the latency requirement) a particular edge data center needs to be selected for a given service function of the requested service; however, the selected version of the service function that was identified during the service definition procedure may not be suitable at that edge-cloud computing system. In another non-limiting example, three network functions (A, B, and C) are to be deployed with a maximum latency of 5msec between the service functions A and B and between the service functions B and C requiring the three service functions to be deployed in a given computing system. However, in some instance for reasons, such as lack of resource availability in the computing system, one of the service functions cannot be deployed causing the definition of all of the service functions to be revisited. However, the root-cause of the failure of the placement of the three service functions cannot be indicated to the service function definition component without knowing the placement options. Therefore, the service component definition would need to perform several iterations of service functions definitions, which are tentatively placed onto the available computing resources. The definition of the service functions is repeated as long as the placement is not successful, which is highly inefficient even for simple services.

[0047] In some instances the service function definition is performed after the placement decision. In these instances, a computing system and/or link is selected for hosting a service function prior to the characteristics of the service function being defined. This is performed for all of the service functions that are to be deployed, such that the location (host) and link are selected for all the service functions prior to these functions being defined. The service functions are then defined after the placement decision In this case, some of the selected locations may not support the given set of requirements of the service functions causing a deployment failure. The root cause of the deployment failure may not necessarily be a function definition which cannot be selected, but instead the placement decision that causes the deployment failure. For example, two service functions are placed at a first computing system and then their characteristics are selected, causing a failure of placement as the first computing system could not support deployment of the two service functions as defined. According to these approaches, the system may try to redefine the service functions and perform another deployment attempt based on the same placement decisions, which may result in another failure. However, the solution might have been to modify the placement decision and causing a first of the service functions to be placed on a first computing system and a second of the service functions to be placed at another computing system and resulting in a successful deployment without redefinition of the service functions.

Deployment of a network service on a cloud infrastructure:

[0048] Deployment of a network service that includes multiple service functions typically involves the definition of the service functions and the placement of each one of the service functions that is to be deployed onto computing system(s) (e.g., a data center) of a distributed cloud infrastructure. The deployment of the service functions further includes the selection of links for coupling the different service functions.

[0049] Resources of the distributed cloud infrastructure may be shared between independent applications and customers (or tenants). To assess the possible impact (e.g., interference occurring) between service functions and/or tenants, in some embodiments, one or more service functions of the network service has one of a plurality of predefined isolation values selected based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure.

[0050] An isolation index for the deployment is generated based on the selected ones of the plurality of predefined isolation values. The isolation index represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure. The isolation index may be used to facilitate a determination whether to use the possible workload placement for the deployment.

[0051] Figure 1 A illustrates a flow diagram of exemplary operations for analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure, according to some embodiments. In some embodiments, the operations of the flow diagram are performed by a service orchestrator 222, which is discussed in greater detail below with respect to Figure 5. [0052] At operation 10, the service orchestrator 222 selects, for one or more of the service functions of the network service, one of a plurality of predefined isolation values based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure. The different ones of the plurality of predefined isolation values represent different levels of isolation provided by respective ones of the plurality of levels of virtualization. The predefined isolation values generally reflect different configurations of shared resources by different service functions, which may include one or more service functions of the network service as well as one or more other service functions deployed on the distributed cloud infrastructure. Some non-limiting examples of the predefined isolation values include values for some or all of the following configurations: a shared physical computing system; a shared VM environment; a shared virtual computing system; different virtual computing systems on a shared physical computing system; a shared containerized environment; a same virtual computing unit in the containerized environment; and different virtual computing units on a shared physical computing system in the containerized environment. Some non-limiting examples of the predefined isolation values include values for some or all of the following multi-level virtualization configurations: different virtual computing units of a shared containerized environment on a shared VM environment; different virtual computing units of the shared containerized environment on a same virtual computing system of the shared VM environment; and a shared virtual computing unit of the shared containerized environment on the shared VM environment. An example hierarchy for calculating an isolation index using one or more isolation values is illustrated in Figure 7 and discussed below. In some embodiments, isolation value(s) are not selected for service function(s) that do not share resources with other service function(s).

[0053] At operation 12, the service orchestrator 222 generates an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values. The isolation index represents a level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure. Refer now to Figure IB, which illustrates a flow diagram of exemplary operations for generating an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values. At operation 30, the service orchestrator 222 receives the selected ones of the plurality of predefined isolation values. At operation 32, the service orchestrator 222 applies a predefined function to the selected ones of the plurality of predefined isolation value.

[0054] The predefined function may include any suitable mathematical and/or logical operations. In one example, the predefined function comprises a sum of the isolation values for the possible workload placement. In another example, the predefined function comprises selecting a maximum isolation value of the isolation values for the possible workload placement. In some embodiments, the predefined function may further depend on one or more of the following: a count of the one or more other workloads, a count of tenants associated with the one or more other workloads, and a count of requests associated with the one or more other workloads. For example, the count(s) may be represented in the predefined function as one or more scaling factors that are multiplied with the isolation value(s) when generating the isolation index.

[0055] Returning to Figure 1 A, at operation 14, the service orchestrator 222 facilitates a determination, using the isolation index, whether to use the possible workload placement for the deployment. At operation 16, the service orchestrator 222 determines whether to use the possible workload placement for the deployment. Refer now to Figure 1C, which illustrates a flow diagram of exemplary operations for determining whether to use the possible workload placement for a deployment. At operation 40, the service orchestrator 222 presents the isolation index using a user interface. At operation 42, the service orchestrator 222 determines a cost to deploy the service functions. At operation 44, the service orchestrator 222 presents the cost using the user interface. At operation 46, the service orchestrator 222 receives a user selection for the possible workload placement.

[0056] Returning to Figure 1 A, responsive to determining to use the possible workload placement (“Yes”), the flow of operations proceeds from operation 16 to operation 18, and the service orchestrator 222 optionally deploys the network service according to the possible workload placement. Responsive to determining not to use the possible workload placement (“No”), the flow of operations proceeds from operation 16 to operation 20, and the service orchestrator 222 optionally generates a re-dimensioning recommendation to modify the possible workload placement. At operation 22, the service orchestrator 222 alters one or more criteria of the deployment. At operation 24, the service orchestrator 222 deploys the network service according to another possible workload placement.

[0057] Figure 2A illustrates a flow diagram of exemplary operations for determining a possible workload placement for a deployment of a network service, according to some embodiments. In some embodiments, the operations of the flow diagram are performed by the service orchestrator 222.

[0058] Referring also to Figure 5A, at operation 50 the service orchestrator 222 receives a request 201 to deploy a network service on a cloud infrastructure. The request 201 includes a service specification for the network service. The service specification includes the service function specifications of the service functions that form the network service. For example, the request may include the service specification 228 illustrated in Figures 5D and 5E. In some embodiments, the service specification 228 is an example of service specification 208 that can be determined based on a service context 205 and a service template 202 as described with reference to Figures 5B-5D. One of ordinary skill in the art would understand that the service specification 228 is intended to be exemplary only.

[0059] The flow of operations moves to operation 52. At operation 52, the availability and characteristics of resources of the cloud infrastructure are determined. The availability and characteristics of resources of the cloud infrastructure include one or more attributes that identify the resources that can host an instance of a service function and their corresponding availability and characteristics. For example, the attributes can include an identification of the computing or networking resources (e.g., a data center, virtual or physical network connection between data centers), an indication of the geographical location of the resources, and one or more additional attributes that relate to the capabilities of the resource such as memory capacity, compute power, latency capability, bandwidth, etc. The attributes may include an indication of whether the resource is available or already allocated (e.g., the resource is already allocated to other service function(s) of the network service to be deployed, the resource is already allocated to other services, or both). The attributes may include a list of software and hardware packages that are available at a given location. For example, the attributes may include an indication of whether a computing system supports hardware acceleration or not. In some embodiments, the attributes may indicate the type of the computing system (e.g., edge data center, data center, server, etc.). In some embodiments, the attributes may include an identification of service function types and/or corresponding versions that are pre-loaded onto or supported by the compute resource.

[0060] In some embodiments, one or more of the attributes for a resource can be associated with a cost of selecting the resource. The cost can be a monetary cost that is to be applied for the usage of the service with a corresponding attribute. Additionally or alternatively, the cost can be assigned to the resource in the interest of enabling selection of the resource as it will be described in further detail below. In some embodiments, the cost can be a reward for selecting the resource. [0061] In some embodiments, the availability and characteristics 223 of the cloud infrastructure may be obtained by the service orchestrator 222 from a central inventory component that gathers the information from the multiple elements of the cloud infrastructure (compute and network infrastructure). Additionally or alternatively, the service orchestrator 222 may obtain the information from sub-orchestration systems that are associated with sub-systems of the cloud infrastructure. A sub-orchestration system may be associated with a single one of the computing systems 230A-230N or with two or more of the computing systems 230A-230N. Similarly, a sub-orchestration system can be operative to determine and transmit to the service orchestrator 222 the availability and characteristics 223 of the network infrastructure The availability and characteristics 223 of the cloud infrastructure may be obtained synchronously or asynchronously. In some embodiments, the availability and characteristics 223 of the cloud infrastructure can be regularly updated and made available to the service orchestrator 222. In these embodiments, if neither cached information nor up-to-date inventory of the availability and characteristics 223 is available, the service orchestrator 222 may pull the information from a central inventory and/or one or more sub-orchestrators. In other embodiments, the availability and characteristics 223 of the cloud infrastructure can be pulled by the service orchestrator 222 as needed upon receipt of a request for a new service without the receipt of regular updates.

[0062] In some embodiments, the availability and characteristics 223 include varying characteristics 213 which define potential variability in the resources of the cloud infrastructure. For each computing system of the first subset of the computing systems an associated cost for deployment of the first service function is determined. The associated costs for usage of the computing systems and links may be varying costs that are based on the varying characteristics. [0063] Referring also to the example of Figure 5E, the determination at operation 52 of the availability and characteristics of the cloud infrastructure results in the set of computing systems 230A, 230B, 230C, 230D, and 230N with respective availability and characteristics Av. & Ch. 223A-223N. The determination further results in obtaining the availability and characteristics of the network resources (e.g., link(s) 226A-226D). In some embodiments, two or more links may couple two or more computing systems and the determination of the availability and characteristics results in listing all of these links. Alternatively or additionally, a single link may couple two computing systems and the determination of the availability and characteristics result in the single link. In some embodiments, each available resource (networking (e.g., link) and/or computing resource (e.g., processing and storage)) may be associated with a cost.

[0064] Returning to Figure 2A, the flow of operations then moves to operation 54 where a first service function specification of a first service function of the network service is selected. The first service function is defined based on the first service function specification, which includes a first set of attributes. In some embodiments, the first service function can be an initial function from the multiple service functions that form the network service, where no other service function has yet been assigned to a host in the cloud infrastructure. Alternatively, the first service function can be a subsequent service function that follows one or more service functions that have previously been assigned to one or more hosts in the cloud infrastructure. In some embodiments, the selection of the first service function specification further includes selecting one or more link(s) for the first service function. The link(s) couple the first service function to one or more of the other service functions. In some embodiments, the link(s) may couple the service function to one or more service functions that have been previously assigned to hosts in the cloud infrastructure. In other embodiments, the link(s) may be available link(s) that originates from the first service function and can be used to couple the service function to one or more service functions and their potential hosts that have not yet been assigned.

[0065] The selection of the first service function specification can be performed through different mechanisms. In some embodiments, the selection of the service function specification can be entirely or partially determined according to the chaining of the service functions. Alternatively or additionally, the selection of the first service function specification may be based on ordering one or more attributes of the service functions. For example, different optimization heuristics can be used to order the service functions based on their attributes. The service function can be ordered based on the attributes that indicate the service functions’ requirements and constraints. For example, the service functions can be ordered based on the length of the link/cycle they belong to in the network service, the cloud resources needed (service functions requiring more computing/memory/networking resources can be ordered such that they are selected in priority with respect to service function requiring less resources, service function requiring less computing/memory/networking capabilities can be selected in priority of other service functions, etc.), etc.

[0066] In some embodiments, the selection of the first service function specification further comprises selecting one or more links for the first service function. In some embodiments, the available links associated with the first service function are ordered based on one or several attributes of the links, and one or more links are selected from the links based on the attributes of the links. For example, links with more stringent constraints can be prioritized (e.g., links with higher bandwidth requirement, lower latency requirement, etc.). Alternatively, links with less constraints can be prioritized.

[0067] Referring also to Figure 5E, a service function (SF) 110B is selected as the first service function of the network service. In this example, SF HOB is selected to be assigned to a host in the cloud infrastructure following the assignment of SF 110A to a host. In this example, SF 110B is not an initial service function to be placed on a host, but instead a subsequent service function that is selected following the placement of SF 110A. In some embodiments, the selection of the service function HOB further includes the selection of the links that couple the service function with one or more other service functions. For example, the links may include the link 110B-110A, the link 110B-1 IOC, and the link 110B-110D. In some embodiments, only a subset of these links may be selected, for example, the link 110B-110A may be selected as a result of the service function 110A being already assigned to a host and the links 11 OB- 1 IOC and 110B-110D may not be selected as service functions 1 IOC and 110D may not have been placed yet.

[0068] Returning to Figure 2A, the flow of operations then moves to operation 56, at which a set of computing systems 237 and a set of links 236 from the resources are determined based on availability and characteristics of the computing systems and the links in the cloud infrastructure. The set of the computing systems are selected based on attributes of the computing systems, such as geographic location, policies, ownership of the computing systems, as well availability of the computing systems. Referring to Figure 5E, a set of computing systems 237 that includes the computing systems 130B, 130C, 130D, 130N is determined based on the availabilities and characteristics obtained at operation 52. For example, the characteristics and availability determined at operation 52 may indicate that the computing system 230A does not have capacity to support an additional service function as it is already selected to host the SF 110A. Other criteria can be used to determine the set of computing systems 237 is selected. For example, the availability and characteristics of the computing systems 230A-230N indicate the geographic location of each one of the computing systems and the selection of the set of computing systems 237 is performed based on their respective locations and the geographic location of the first service function. Additionally or alternatively, the selection of the set of computing systems 237 can be performed based on one or more attributes of the first service function HOB and availability and characteristics of the computing systems.

[0069] The flow of operations then moves to operation 58, at which a first computing system of the set of computing systems 237 is assigned to host the first service function and zero or more links that are to be used to couple the first computing system with one or more other ones of the computing systems based on the first service function specification. In some embodiments, the selected computing system can be a computing system that has previously assigned to host another service function. Alternatively, the selected computing system can be a computing system that has not been previously assigned to host. In some embodiments, two or more links can be selected from the set of links to be used to couple the computing system to two or more other computing systems that host other service functions. Alternatively, no link can be selected as the connection with other service functions may have previously been determined by the selection of the links at preceding operations when other service functions were assigned to computing systems. In some embodiments, a single link is selected from the set of links. In the example of Figure 5E, the computing system 230B is selected from the set of computing systems 237 to host the first service function HOB and a link 226A is selected to couple the first service function when instantiated on the computing system 23 OB to service function 110A when hosted on the computing system 230A. In some embodiments, the link 226A is one of multiple links 226A-226K that connect the computing system 230B and the computing system 230A. Each of the links 226A-226K may be associated with the same or a different cost.

[0070] In some embodiments, selecting the first computing system of the set of the computing systems to be assigned to host the first service function and zero or more links of the set of links that couple the first computing system with one or more other computing systems of the computing systems can be performed according to the operations described with respect to Figure 2B, discussed below.

[0071] In some embodiments, the flow may further include operation 60, at which a determination of whether the selection of the first computing system and the zero or more links (to be assigned to the first service function) is successful is performed. In some embodiments, determining that the selection is successful includes a determination that the selected first computing system and/or the zero or more links have attributes that are compatible with the attributes of the first service function. In other words, for the selection to be successful, the first computing system and the zero and more links need to be capable of supporting deployment of the first service function. In some embodiments, the selection may not be successful (“No”), in that case the flow of operations moves to operation 62, discussed in greater detail below. Upon determining that the selection is successful (“Yes”), the flow of operations moves to operation 64.

[0072] At operation 64, a determination of whether all of the service functions of the network service have been assigned to be hosted on one of the computing systems is performed. In response to determining that there are one or more of the service functions that are not yet assigned to be hosted on one of the computing systems (“Yes”), the flow of operations moves to operation 54 and operations 54-64 are repeated until it is determined that all of the service functions that form the network service are assigned to computing systems with links that couple the service functions. For example, upon assignment of the first service function 110B to the computing system 230B with the link 226A, the next service function is selected to be placed on the cloud infrastructure. The next service function is one of the remaining non-assigned service functions 110C-110G. For example, the next service 1 IOC can be selected.

[0073] Alternatively, responsive to determining that all of the service functions have been assigned to be hosted on one of the computing systems in the cloud infrastructure (“No”), a possible workload placement for a deployment of the network service on the cloud infrastructure is output at operation 66. The possible workload placement is to be used for determining a deployment plan 271 for instantiating the service functions of the network service on the cloud infrastructure. The possible workload placement indicates for each one of the service functions a corresponding one of the set of computing systems 237 and the set of links 236 from the one or more links that couple the set of computing systems 237. Figure 5F illustrates an exemplary placement recommendation 270 for the service function, in accordance with some embodiments. The placement recommendation 270 includes an indication of the assignment of each one of the service functions 110A-110G onto one or more computing systems 230A-230N of the cloud infrastructure. In the illustrated example, the service function 110A is assigned to be hosted on the computing system 230A, the service function 110B is assigned to be hosted on the computing system 230B, the service function 110C is assigned to be hosted on the computing system 230B, the service functions 110D, 110E are assigned to be hosted on the computing system 230C, and the service functions 110F, HOG are assigned to be hosted on the computing system 230N. The assignment is performed based on the availability and characteristics of the computing systems 230A, 230B, 230C, and 230N, as well as based on the attributes of the service function specifications of each one of the service functions 110A-110G. The placement recommendation 270 can then be used to determine the deployment plan 271. The deployment plan 271 may be presented using a user interface 274, and responsive to a user input selecting the deployment plan 271, one or more cloud infrastructure managers instantiates the network service on the cloud infrastructure according to the deployment plan 271. The instantiation of the network service may include transmitting detailed steps to each one of the cloud infrastructure managers for instantiating the service functions as defined by their respective service function specifications. In some embodiments, the computing system may already have a version of the service function that is on-boarded (registered) prior to its use and can create a new instance for the current service that is to be deployed. Having versions already on-boarded on the computing system reduces fulfillment times (e.g., fetching images or layers).

Additionally or alternatively, the computing system may not include an image of the service function already loaded, and an image of the service function is transmitted with a request to be instantiated on the computing system. In addition to indicating the placement of the service functions onto computing systems, the placement recommendation 270 further indicates one or more links that are to be used to couple the service functions through the network resources of the cloud infrastructure. The links can be virtual or physical network connections.

[0074] Figure 2B illustrates a flow diagram of exemplary operations that can be performed for selecting the first computing system of the set of the computing systems to be assigned to host the first service function, in accordance with some embodiments. The operations of Figure 2B will be described with reference to Figure 5G. At operation 70, a first subset of computing systems is determined from the first set of the computing systems that have a second set of attributes compatible with the first set of attributes of the first service function. In some embodiments, the second set of attributes of the first subset of computing systems is compatible with the attributes of the first service function when the computing requirements of deployment of the first service function are satisfied by the hardware and/or software resources of the computing systems. For example, the characteristics of the computing system may indicate that one or more versions of the service function are pre-loaded/registered with the computing system (e.g., SF 110A version 4.1 and version 3.3 are registered with the computing system 230A). Alternatively or additionally, the characteristics of the computing system may indicate whether or not some hardware implementations can be supported by the computing system (e.g., hardware acceleration, high performance, graphics processing unit (GPU) implementation, etc.). The flow then moves to operation 72, at which a first subset of a first set of links is determined. The first subset of links has a first set of link attributes compatible with the first set of attributes of the first service function is determined. In some embodiments, the link attributes of the subset of links are compatible with the attributes of the first service function when the networking requirements of deployment of the first service function are satisfied by the subset of the links. For example, the subset of links may satisfy bandwidth, latency and/or geographical location, required for the service function. Additionally or alternatively, the subset of links may satisfy network, communication, and security protocols requirements of the first service function. With reference to Figure 5G, the subset of computing systems includes computing system 230N and computing system 230B. Each of the systems is able to host the service function 11 OB as defined by a service function specification, as each of the two computing systems includes an indication that SF HOB with hardware acceleration is supported. The first subset of links may include the link 226A for the computing system 230B. The first subset of links may include the links 226D for the computing system 230B. In some embodiments, the first subset of links may further include one or more of the links 226C-226G, 226B-226J, and 226A-226K. [0075] At operation 74, for each one of the first subset of computing systems an associated cost for deployment of the first service function is determined. For example, for each one of the subset of computing systems 230B, 230N an associated cost is determined. The cost can be a reward associated with the placement of the service function at a computing system. Alternatively and additionally, the cost can be a fee to be paid by a customer and/or service provider for the placement of the service function at the computing system Cost 224N corresponds to the cost of assigning or placing the service function HOB onto the computing system 230N and cost 224B corresponds to the cost of assigning or placing the service function HOB onto the computing system 230B.

[0076] The flow then moves to operation 76 at which for each of the first subset of links an associated link cost is determined. The link cost is associated with the deployment of the first service function. Link cost 229A is associated with the selection of link 226A and link cost 229D is associated with the selection of link 226D. The flow then moves to operation 78, at which the first computing system and the zero or more links are selected, based on the associated costs 224 and the associated link costs 229. The subset of computing systems 243 and the subset of links 246 are ranked based on the costs (224B, 224N, 229 A, 229D). Different ranking mechanisms can be used without departing from the scope of the present embodiments. For example, the computing systems and links can be ranked based on a cost of using the resources in a decreasing order. Alternatively or additionally, the computing systems and links can be ranked based on a reward that is allocated to the customer and/or service provider for using the resources. Based on this ranking, the computing system 230B and the link 226B are selected for the placement of the first service function 110B. The computing system 230B is compatible with the attribute of the first service function HOB and is operative to host this service function. For example, the computing system 23 OB is operative to support a version of service function 110B with hardware acceleration as well as a version of service function HOB with CPU implementation only. Each of these versions being associated with a respective cost. This is compatible with the attributes of the first service function specification, which indicate the need for hardware acceleration instantiation.

[0077] In some embodiments, responsive to determining at operation 60 that the selection of the first computing system and the zero or more links is not successful (i.e., no computing system may be available and/or compatible with the attributes of the service function that is selected), the flow proceeds to operation 62, where one or more attributes of the first service function specification are altered. In some embodiments, altering the one or more attributes comprises selecting a second set of attributes is selected for the first service function. The second set of attributes includes at least one attribute for the first service function that is different from the first set of attributes determined for the first service function. For example, a first set of attributes for the first service function may determine a first software version for the service function, a vendor of the software version of the service function, hardware requirements (e.g., hardware acceleration), networking requirements such as a first bandwidth, a first latency, a geographic location for the first service function, and one or more additional attributes that define the first service function. When the second set of attributes is determined, at least one of the listed first attributes can be different from the corresponding attribute in the first attributes. For example, a different geographic location for the first service function, a different software version, different hardware implementation requirements, different networking requirements, can be selected as the second attributes for the first service function. These attributes satisfy the requirements and constraints set by the customer and/or the service provider for the network service (e g., these attributes are compatible with a service requester input parameter 206 and the service policies 207).

[0078] In some embodiments, a determination of whether the second set of attributes for the first service function is compatible with third attributes of the first computing system and third link attributes of the one or more links is performed. In some embodiments, the third set of attributes of the first computing system is compatible with the attributes of the first service function when the computing requirements of deployment of the first service function as defined, based on the second attributes, is satisfied by the hardware and/or software resources of the first computing system.

[0079] Responsive to determining that the second set of attributes for the first service function is compatible with the third attributes of the first computing system and the third link attributes of the one or more links, the first service function specification is updated to include the second set of attributes for the first service function and the first computing system and the one or more links of the subset of links are selected for the first service function with the second set of attributes. In this embodiment, instead of cancelling the placement of the first service function onto the first computing system, a new set of attributes may be determined for the first service function without changing the placement of the service function in the cloud infrastructure. This presents a dynamic placement and definition of service functions which, in addition to taking into account characteristics and availability of the cloud infrastructure resources, allows for a modification of attributes of a service function when a resource is selected for placement.

[0080] In some embodiments, responsive to determining that the second set of attributes for the first service function is not compatible with the third attributes, a determination of whether there are any remaining sets of attributes that can be determined for the first service function is performed. Responsive to determining that there is still at least a set of attributes that can be selected, the process of selecting a second set of attributes for the first service function and determining whether the second set of attributes is compatible with the third attributes is repeated until compatibility is determined or no compatibility is found for any of the possible sets of attributes of the service function and the attributes of the first computing system. In some embodiments, the placement of a service function may be backtracked to a previous placement when the placement of the service function is not successful.

[0081] While some of the operations above were described with reference to a selection of a computing system for a first service function not being successful, in some embodiments, the failure of success of the placement of the service function can be due to a lack of compatibility of the networking resources instead of the computing resources. In these embodiments, the link attributes of the links selected for the service function are compared with the attributes of the service function that corresponds to the network resources needed for implementation of the service function. Thus, the various operations described above can be performed based on the link attributes of the selected links from the network resources of the cloud infrastructure.

[0082] The embodiments described herein provide for assessment of possible workload placements for deployment of a network service. The embodiments provide a flexible, and runtime extensible set of service and hardware platform-aware attributes. The solution allows for a dynamic determination of a service function deployment plan which takes into consideration the characteristics and availability of the cloud resources, as well as flexibility in the definition of the service functions to be deployed. In the proposed embodiments, the definition of the service functions to be deployed is performed in conjunction with the placement decision of the service functions resulting in an automated and/or optimal placement and definition mechanisms of a network service.

[0083] Figure 3 illustrates a flow diagram of exemplary operations for generating isolation indexes for one or more possible workload placements, according to some embodiments. In some embodiments, the operations of the flow diagram are performed by the service orchestrator 222.

[0084] At operation 80, the service orchestrator receives one or more possible workload placements for a deployment of a network service on a cloud infrastructure. In some embodiments, the one or more possible workload placements are generated using the operations of Figure 2A. At operation 82, a possible workload placement from the one or more possible workload placements is selected. At operation 84, a virtualization level of the possible workload placement is identified from a plurality of levels of virtualization supported by the cloud infrastructure. [0085] At operation 86, a first service function of a plurality of service functions of the network service is selected. At operation 88, one of a plurality of predefined isolation values for the first service function is selected. As discussed above, the predefined isolation values generally reflect different configurations of shared resources by different service functions, which may include one or more service functions of the network service as well as one or more other service functions deployed on the distributed cloud infrastructure. In some embodiments, the predefined isolation values are stored by the service orchestrator 222.

[0086] At operation 90, the service orchestrator 222 determines whether all of the plurality of service functions have a selected isolation value. If at least one service function does not have a selected isolation value (“No”), the flow of operations moves to operation 86 where a next service function is selected, and operation 88 where one of the plurality of predefined isolation values is selected for the next service function.

[0087] If all of the service functions have selected isolation values (“Yes”), the flow of operations moves to operation 92, where an isolation index for the deployment is generated based on the selected ones of the plurality of predefined isolation values. In some embodiments, a predefined function is applied to some or all of the selected ones of the plurality of predefined isolation values.

[0088] At operation 94, the service orchestrator 222 determines whether all of the one or more possible workload placements have an isolation index. If at least one possible workload placement does not have an isolation index (“No”), the flow of operations moves to operation 82 where a next possible workload placement is selected, and operations 82-92 are repeated for the next possible workload placement.

[0089] Figure 4A illustrates a block diagram of an exemplary VM environment in a distributed cloud infrastructure, according to some embodiments. In some cases, the VM environment may be included in multi-level virtualization supported by the distributed cloud infrastructure.

[0090] In diagram 100, a hardware environment 102 comprises a plurality of physical computing systems 104 A, . .., 104M that are networked with each other. In some embodiments, each of the physical computing systems 104A, ..., 104M comprises a respective electronic device, such as a computing device or a network device.

[0091] A VM environment 106 is implemented using computing resources from the hardware environment 102. In some embodiments, the VM environment 106 comprises an operating system (OS), such as OpenStack, OpenNebula, etc. executing on some or all of the physical computing systems 104A, . .., 104M. The VM environment 106 comprises a plurality of virtual machines (VMs) 108A, 108B, . . ., 108N that are instantiated using computing resources of the hardware environment. In some embodiments, the VMs 108A, 108B, . .., 108N may include OpenStack compute nodes and/or controller nodes.

[0092] A plurality of service functions 110A, HOB, ..., HOP are hosted on the VMs 108A, 108B, . .., 108N. As shown, the SF 110A is hosted on VM 108 A, which operates uses computing resources of the physical computing system 104A. Other hosting and resource arrangements are also contemplated for the VM environment 106, and will be understood by the person of ordinary skill in the art.

[0093] Figure 4B illustrates a block diagram of an exemplary containerized environment in a distributed cloud infrastructure, according to some embodiments In some cases, the containerized environment may be included in multi-level virtualization supported by the distributed cloud infrastructure.

[0094] In diagram 120, the containerized environment 122 is implemented using hardware from the hardware environment 102. For example, within the containerized environment 122 pooled resources 124 from the hardware environment 102 (i.e., one or more of the physical computing systems 104A, . .., 104M) may be designated to form, e.g., a Kubernetes cluster. A plurality of virtual computing units 126A, 126B, ..., 126Q (e.g., Kubernetes nodes) may be designated within the Kubernetes cluster. In some embodiments, the virtual computing units 126A, 126B, ..., 126Q may include physical computing systems 104 and/or virtual machines 108. In this configuration, the service functions 110A, 110B, ... , 110P are hosted as containerized applications using the nodes (e.g., the virtual computing units 126A, 126B, . . ., 126Q) of the cluster. Extending the Kubernetes example, the nodes host one or more pods that are the components of the application workload.

[0095] Figure 4C illustrates a block diagram of an exemplary VM environment and containerized environment in a distributed cloud infrastructure, according to some embodiments. In some cases, the containerized environment may be included in multi-level virtualization supported by the distributed cloud infrastructure.

[0096] In diagram 130, the container environment 122 is depicted above the VM environment 106, and the VM environment 106 is depicted above the hardware environment 102. In this configuration, the service functions 110A, 110B, ..., HOP are hosted as containerized applications on the nodes of the containerized environment 122, and the nodes are virtual machines implemented in the VM environment 106.

[0097] Although Figures 4A-4C depict simple implementations of a multi-level virtualization, alternate implementations are also contemplated. For example, adapting the configuration shown in Figure 4C, one or more of the SFs 110A, 110B, .. . , 11 OP are hosted as containerized applications in the containerized environment 122, one or more other of the SFs 110A, 110B, .. . , 11 OP are hosted on virtual machines of the VM environment 106, and/or one or more other of the SFs 110A, 110B, ... , 11 OP are hosted on a physical machine of the hardware environment.

[0098] Figure 5A illustrates a block diagram of an exemplary system for deployment of a network service, in accordance with some embodiments. The system 200 includes a cloud infrastructure 210 and a service orchestrator 222. In some embodiments, the service orchestrator 222 may be included in a service management and orchestration (SMO) system or component 220. The system 200 is operative to enable deployment of a network service on the cloud infrastructure 210. As discussed above, the network service typically includes a set of service functions that are to be deployed in order to provide the service to a service requester 215. The requested service satisfies a particular business need and adheres to policies that define operational characteristics and access controls/security. The deployment of such a service on the cloud infrastructure 210 involves utilization of one or more service functions that satisfy the service policy and operational requirements.

[0099] The cloud infrastructure 210 provides the hardware and software components which build up the environment in which service functions of the network service can be deployed, managed, and executed. The cloud infrastructure 210 includes hardware resources, which may include, but is not limited to, computing hardware (e.g., processors), storage hardware (e g., Random Access Memory (RAM) and hard disks), and networking hardware (e g., network interface cards). The cloud infrastructure 210 includes a set of computing systems 230A, 230B, .. . , 23 ON. Each of the computing systems is located at a given location and may include a respective set of network devices (e g., NDs 231, 232, 233, 234, 235) on which one or more service functions can be deployed. Each one of the computing systems 230A-230N may have costs associated with each of the hardware and software resources that they offer. For example, computing system 230A may have a first cost of $$$ associated with using the computing resources (CPU usage) of the data center. The edge computing system 230A may have a cost of $$ associated with using the computing memory resources. The edge computing system 230A may further include an indication of characteristics and/or versions of a service function that can be supported by the edge computing system 230A. For example, the edge computing system 230A can include an indication that a first service function of type 142A can be supported if deployed with version v3.3 or version v4.1. In the illustrated example, each of the versions that are supported by the first edge computing system 230A is associated with a different cost.

[00100] In some embodiments, the cloud infrastructure 210 may include computing resources at a single location (e.g., edge computing system 230A). In other embodiments, the cloud infrastructure 210 may include multiple computing systems located at different locations remote from one another.

[00101] The service management and orchestration component (SMO) 220 may include one or more additional components (not illustrated) that may be used in the management and orchestration of the service functions that are deployed on the cloud infrastructure 210. For example, the SMO component 220 may include a virtual function manager, and/or a virtualized infrastructure manager. The virtual function manager is responsible for service function lifecycle management (e.g., instantiation, update, query, scaling, and termination). The virtual function manager can use the recommended deployment plan 271 to instantiate the service functions onto the cloud infrastructure 210. The virtualized infrastructure manager is responsible for controlling and managing the interaction of a service function with computing, storage, and network resources under its authority, as well as the virtualization of these resources. In some embodiments, the virtualized infrastructure manager can be located at the same location as the cloud computing resources (e.g., within the computing systems). The multiple components of the SMO component 220 can be implemented as hardware, software, or firmware, or any combination thereof.

[00102] The service orchestrator 222 is responsible for the orchestration and management of the cloud infrastructure 210 and realizing network services on the cloud infrastructure 210. In some embodiments, the service orchestrator 222 includes a service function-based placement component (SFPC) 219. The service orchestrator 222 is operative to determine a service specification 208 based on a service template 202 and a service context 205. The service orchestrator 222 is operative to determine for each service function identified in the service specification 208, a location in the cloud infrastructure that is to host the service function. The determination of the host takes into account the characteristics of the service functions as defined in the service specification 208. The service orchestrator 222 simultaneously considers the cloud infrastructure availability and characteristics 223 when determining the host on which the service function is to be deployed. The service orchestrator 222 takes as input a service specification 208 and the cloud infrastructure availability and characteristics 223 and determines through an iterative process a host for each one of the service functions in the service specification 208.

[00103] In some embodiments, the service orchestrator 222 selects, for each of the service functions, one of a plurality of predefined isolation values 212 based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure 210. Different ones of the plurality of isolation values 212 represent different levels of isolation provided by respective ones of the plurality of levels of virtualization. The service orchestrator 222 generates an isolation index 214 for the deployment based on the selected ones of the plurality of predefined isolation values 212. The isolation index 214 represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure 210. The service orchestrator 222 facilitates a determination, using the isolation index 214, whether to use the possible workload placement for the network service. Properties of the isolation values 212 and techniques for calculating the isolation index 214 are discussed in greater detail below with respect to Figures 7, 8 A, 8B.

[00104] The service orchestrator 222 determines a placement recommendation 270 for the service functions that form the requested service. The placement recommendation 270 contains an indication of the assignment of each one of the service functions that forms the network service (e g., defined by service function specifications 252A-252G) onto one or more of the computing systems 230A-230N of the cloud infrastructure 210. The placement recommendation 270 further indicates one or more links that are to be used to couple the service functions through the network resources of the cloud infrastructure. The links can be virtual or physical network connections that can be used to couple and forward traffic between the multiple service functions (e.g. L3VPN connecting two zones of virtual infrastructure managers (VIMs) over a WAN). In some embodiments, as will be described in further detail below, the placement recommendation 270 may include a re-dimensioning recommendation 272. The redimensioning recommendation 272 includes possible recommendations to modify capacity of cloud resources (e.g., computing resources and/or networking resources) of the cloud infrastructure.

[00105] The placement recommendation 270 can then be used by one or more components of the SMO component 220 to determine a deployment plan 271, which can be used to deploy the service on the cloud infrastructure 210. The deployment plan 271 may include a sequence of ordered elementary steps, which are understood by the underlying infrastructure managing elements (e.g., VIMs, network function virtualization orchestrators (NFVOs), Wide Area Infrastructure Manager (WIM), and/or network manager, etc.) to deploy the service on the cloud infrastructure and to provision the service to a ready-to-use status.

[00106] Figure 5B illustrates a block diagram of an exemplary service template that can be used in some embodiments. The service template 202 includes a set of service function types 203 and a topology 204 of the network connections (also referred to as links) coupling the service functions. A service function type 203 refers to a conceptual definition of a service function. The topology 204 determines the type of network connections that link the multiple service function types 203. Figure 5B includes an exemplary service template 221 that can be used. The exemplary service template 221 includes a set of service function types (SFT) 142A- 142G that form a service template. The service function types can be types of network service functions such as 3GPP eNodeB (radio node), a serving gateway (GWU), network exposure function (NEF), evolved packet gateway (EPG), a mobility management entity (MME), service aware policy controller (SAPC), or other types of service function such as various Layer 7 application (App) The topology 204 includes a set of connections that link the multiple service functions and associated requirements for these connections. For example, the set of connections can be a set of VPN overlay connections that couple the multiple service function types. In the illustrated exemplary service template 221, a first service function type 142A is coupled with a second service function type 142B. The second service function type 142B is coupled with SFT 142A, SFT 142C, and with SFT 142D. SFT 142D is coupled with SFT 142B, SFT 142E, SFT 142G, and SFT 142F. SFT 142E is coupled with SFT 142D, SFT 142F, and SFT 142G. SFT 142F is coupled with SFT 142E, SFT 142D, and SFT 142G. SFT 142G is coupled with SFT 142F, SFT 142E, and SFT 142D. The topology may include for each connection a set of attributes and parameters that define the connection.

[00107] Figure 5C illustrates a block diagram of an exemplary service context that can be used in some embodiments. The service context 205 includes input parameters 206 for the service requester and service policies 207. The input parameters 206 define the service as requested or desired by the service requester 215. For example, the input parameters 206 may be determined based on a set of features or service options selected by the service requester based on the service offering. In some embodiments, the input parameters 206 may further determine costs associated with each one of the parameter s/attributes requested by the service requester 215 or more generally, a cost associated with the requested service. The service policies 207 may be determined by the provider based on the agreement with the service requester and/or based on regulations defined by regulation and authority agencies. The service context 205 can be expressed as a set of service function specifications/descriptors 252A-252G for the service functions types requested by a particular customer of the service such as service requester 215, based on the input parameter 206 and the service policies 207.

[00108] In some embodiments, the input parameters 206 comprise an isolation parameter 260 specifying a degree of isolation for other workloads (service functions) that are deployed on the cloud infrastructure. In some embodiments, the isolation parameter 260 may be a numerical value corresponding to the isolation index 214 determined by the service orchestrator 222. In other embodiments, the isolation parameter 260 may be an input that specifies or implies a degree of isolation (such as high, medium, or low), and the service orchestrator 222 interprets the input to determine the service context 205. [00109] Each service function specification 252A-252G may include various information about the service function type that it describes defined by a set of attributes in the service function specification. The information may include default values for attributes, boundaries for attributes, whether an attribute is read-only or read-write, etc. In the illustrated example, each one of the service function specifications (SF Spec. 252A-252G) includes a type of the service function that is specified (SFT 142A-142G) and one or more attributes that identify requirements and constraints for the service function. In some embodiments, the requirements can be determined based on the service policies 207 and the input parameters 206. In some embodiments, the attributes may include an attribute for identifying one or more versions of the service function if several versions are available for a given service function type, an attribute for identifying a vendor of the service function, etc. In some embodiments, the service specifications 252A-252G may further include attributes that define other types of requirements that need to be satisfied when the network service is deployed. The attributes can indicate geographical constraints for the deployment of the network service. The geographical constraints can result from one or more of a policy established for the service, administrative considerations defined by the service provider, the location of the customer, and/or the location of the service that is to be provided. In some embodiments, the same geographical constraints may apply to all of the service functions forming the service. Alternatively, different geographical constraints may apply to two or more of the service constraints. The geographical constraints may indicate a geographical region or regions where a service function can be deployed. Alternatively, the geographical constraints may indicate one or more geographical regions where the service function cannot be deployed. Additionally, an attribute can be representative of latency constraints for network traffic between two or more of the service functions. In some embodiments, the same latency constraints may need to be satisfied between the multiple service functions. Alternatively or additionally, different latency constraints may need to be satisfied between different pairs of service functions. The requirements may further include additional optional requirements such as an indication of whether the service function is to be implemented with or without hardware acceleration. In some embodiments, each service function specification can include one or more of the attributes described above that define the service function to deploy. In some embodiments, additional attributes can be used without departing from the scope of the embodiments described herein. In some embodiments for each service function a type of attribute may include a range of possible options for that service function. For example, based on a service agreement of the customer, the service can be provided by selecting two or more versions of a service function for that customer (e g., either one of multiple versions can be instantiated for a service function and would satisfy the service agreement established with the customer). In another example, no version may be selected indicating that any version of the service function can be instantiated for providing the service to the customer.

[00110] Figure 5D illustrates a block diagram of an exemplary service specification that results from a service template and a service context in some embodiments. As used herein, a service specification/descriptor 208 refers to a detailed description of the characteristics of a service. The service specification 208 includes a detailed description of the characteristics of the service functions (service function specifications/descriptors) that form the service. The service specification 208 is determined based on the service template 202 and the service context 205 The determination of the service specification 208 may include, at least in part, assigning values of the attributes identified in the service context 205 to attributes of the service template 202. Alternatively or additionally, the determination of the service specification 208 may include merging values of attributes from the service context 205 with values of attributes from the service template 202. The service specification 208 is then used for determining a deployment plan for the network service and for deploying the service based on the deployment plan.

Architecture

[00111] Figure 6 illustrates a block diagram of an exemplary network service deployment system, according to some embodiments. The system 400 includes a cloud infrastructure 410 and a service management and orchestration component 420. The service management and orchestration component 420 is part of a service provider platform 470. The service provider platform 470 includes several components (that can be implemented on one or more network devices) for offering one or more network services to service requesters. In some embodiments, the service provider platform 470 is a multi-tenant system that provides services to multiple tenants across a distributed cloud infrastructure with various geographically distributed sites. The service management and orchestration component 420 is operative to enable deployment of a network service on the cloud infrastructure 410, which in some embodiments includes analyzing and/or comparing possible workload placemen^ s), receiving user selections, and so forth. The network service typically includes a set of service functions. The requested service satisfies a particular business need and adheres to policies that define operational characteristics and access control s/security. In some embodiments, the service functions may include network functions such as, firewall, load balancer, Deep Packet Inspection (DPI), HTTP header enrichment, Network Address Translation (NAT), etc. These service functions are typically linked to form the service and the linking of these functions can be referred to as service chaining. [00112] The cloud infrastructure 410 includes computing resources (e.g., computing systems 460A-460N) and network resources (e.g., physical transport infrastructure). Each of the computing systems 460A, 460B, ..., 460N includes respective hardware resources 476A, 476B, .. . , 476N as well as or more levels of virtualization. As shown, and similar to the configuration shown in Figure 4A, the computing system 460A comprises a VM layer 474A (e.g., OpenStack) and one or more VM layer resources 472A (e.g., virtual machines). Similar to the configuration shown in Figure 4B, the computing system 460B comprises a containerized layer 475B and one or more containerized layer resources 473B. Similar to the configuration shown in Figure 4C, the computing system 460N comprises a VM layer 474N (with VM layer resources 472N) and a containerized layer 475M (with containerized layer resources 473M). Service functions of the network service may thus be deployed using any of the VM layer resources 472A, . . ., 472N and/or any of the containerized layer resources 473B, .. . , 473M.

[00113] In some embodiments, each one of the computing systems is implemented as a network device as described in further details below. Alternatively, each of the computing systems can be a data center including several network devices. The service orchestrator 222 is operative to perform the embodiments described with reference to Figures 1A-9B. Each of the service orchestrator 222, the service management and orchestration component 420, and the service provider platform 470, can be implemented on one network device or distributed over multiple network devices. Each of the network function virtualization orchestrator (NFVO) 480 and the virtualized infrastructure managers (VIMs) 490A, 490B, ... , 490M can be implemented on one network device or alternatively distributed over multiple network devices.

[00114] Figure 7 illustrates an exemplary hierarchy 500 for calculating an isolation index using one or more isolation values for service functions of a network service in a multi-level virtualization cloud infrastructure, according to some embodiments. In some embodiments, the isolation values (or components used to form the isolation values) may be stored (or otherwise accessed or determined) by the service orchestrator 222. In some embodiments, the isolation values or components thereof are predefined, that is, defined prior to the deployment of the network service. In this way, an isolation index for the deployment of the network service may be determined in advance, without requiring real-time deployment data to be obtained. In some embodiments, the isolation values or components thereof may be updated from the predefined values, e.g., using the real-time deployment data.

[00115] In some embodiments, the isolation values or components thereof may be manually determined (e.g., provided by cloud infrastructure experts). In some embodiments, the isolation values or components thereof may be determined according to one or more rules. For example, a first rule may specify that the isolation values reflect that isolation is greater for shared software-based resources than for shared hardware resources. A second rule may specify that isolation values reflect that isolation is greater for resources in the VM environment (e.g., in OpenStack) than for resources in the containerized environment (e.g., in Kubemetes).

[00116] The hierarchy 500 depicts a plurality of different sharing configurations, which represent various cases of resource sharing by workloads 520-1, 520-2. In some embodiments, the workload 520-1 represents a first service function of the network service to be deployed, and the workload 520-2 represents a workload of another service (e.g., a second service function of a deployed service) that is hosted on the particular resource.

[00117] The sharing by workloads (e g., service functions) of a virtual machine (VM; also “virtual computing system”) 502 (depicted as an unfilled rounded rectangle) corresponds to an isolation value component p. The sharing by workloads of a VM environment 504 (depicted as an unfilled cloud) corresponds to an isolation value component p. The sharing by workloads of a physical machine (PM; also “physical computing system”) 506 (depicted as a rounded rectangle filled with horizontal lines) corresponds to an isolation value component a. The sharing by workloads of a containerized environment 508 (depicted as a cloud filled with vertical lines) corresponds to an isolation value component q. The sharing by workloads of a virtual computing unit (VCU) 510 (e.g., a Kubernetes node; depicted as a rounded rectangle filled with vertical lines) corresponds to an isolation value component y. As one non-limiting example of predefined isolation value components determined according to the first rule and the second rule, a = 2, p = 1.60, y = 1.92, p = 1.25, and q = 1.50.

[00118] In sharing configuration 515, the workloads 520-1, 520-2 are hosted on a shared PM 506. Thus, the isolation index of the sharing configuration 515 equals the corresponding isolation value component (A = a).

[00119] In sharing configuration 525, the workloads 520-1, 520-2 are hosted on a shared VM environment 504 (e.g., a same OpenStack environment or cluster), although the workloads 520- 1, 520-2 are deployed on different VMs 502 and different PMs 506. The isolation index of the sharing configuration 525 equals the corresponding isolation value component (A = p).

[00120] In sharing configuration 530, the workloads 520-1, 520-2 are hosted on different VMs 502 in the VM environment 504 on a shared PM 506. In this case, the isolation index of the sharing configuration 530 equals a multiplicative product of the corresponding isolation value components (A = ap). In other implementations, the isolation index for a particular sharing configuration may be calculated by applying other mathematical and/or logical functions to the isolation value components (e.g., multiplication of some of the isolation value components, addition of some or all of the isolation value components, an average of some or all of the isolation value components, a maximum value of the isolation value components, and so forth). [00121] In sharing configuration 535, the workloads 520-1, 520-2 are hosted on a shared

VM 502 in the VM environment 504 on a shared PM 506. In this case, the isolation index of the sharing configuration 535 equals a multiplicative product of the corresponding isolation value components (A = apP).

[00122] In sharing configuration 540, the workloads 520-1, 520-2 are hosted on a shared containerized environment 508 (e g., a same Kubemetes environment), although the workloads 520-1, 520-2 are deployed on different virtual computing units 510 (e.g., different Kubernetes pods) and different PMs 506. The isolation index of the sharing configuration 540 equals the corresponding isolation value component (A = q).

[00123] In sharing configuration 545, the workloads 520-1, 520-2 are hosted on different virtual computing units 510 (e.g., different Kubernetes pods) in the shared containerized environment 508 on a shared PM 506. In this case, the isolation index of the sharing configuration 545 equals a multiplicative product of the corresponding isolation value components (A = aq).

[00124] In sharing configuration 550, the workloads 520-1, 520-2 are hosted on a same virtual computing unit 510 (e.g., a single Kubernetes pod) in the shared containerized environment 508 on a shared PM 506. In this case, the isolation index of the sharing configuration 550 equals a multiplicative product of the corresponding isolation value components (A = aqy).

[00125] In sharing configuration 555, the workloads 520-1, 520-2 are hosted on different virtual computing units 510 in the shared containerized environment 508 in the shared VM environment 504 on different PMs 506. In this case, the isolation index of the sharing configuration 555 equals a multiplicative product of the corresponding isolation value components (A = qp).

[00126] In sharing configuration 560, the workloads 520-1, 520-2 are hosted on different virtual computing units 510 in the shared containerized environment 508 in the shared VM environment 504 on a shared PM 506. In this case, the isolation index of the sharing configuration 560 equals a multiplicative product of the corresponding isolation value components (A = aPpq).

[00127] In sharing configuration 565, the workloads 520-1, 520-2 are hosted on a shared virtual computing unit 510 in the shared containerized environment 508 in the shared VM environment 504 on a shared PM 506. In this case, the isolation index of the sharing configuration 565 equals a multiplicative product of the corresponding isolation value components (X = aPypq).

[00128] In some embodiments, the service orchestrator 222 selects one of a plurality of predefined isolation values for each of the service functions of the network service. For example, the service orchestrator 222 may iterate through each service function of the network service and may determine whether the possible workload placement would cause the service function to share a computing resource with another workload in the cloud infrastructure. [00129] The hierarchy 500 depicts a set of relatively simple sharing configurations. More specifically, the sharing configurations 515-565 each reflect two workloads 520-1, 520-2 on a particular resource and are agnostic to a count of the workloads, a count of tenants associated with the workloads, a count of requests associated with the workloads, and so forth. In some embodiments, the isolation values for various service functions of the network service may be calculated using one or more amplification values that reflect some or all of a count of workloads, a count of tenants associated with the workloads, and a count of requests associated with the workloads.

[00130] In some embodiments, the isolation value of a particular sharing configuration may be multiplied by the one or more amplification values, which in some cases correspond to one or more predefined functions. Using the sharing configuration 550 as an example, the isolation value may be calculated as X = aqyF(k) where k is a count of the workloads, and F(k) is a first function applied for k > 2 workloads. For example, F(k) may linearly scale with an increasing k, such as F(k) = k 12, although other functions are also contemplated. Returning to the sharing configuration 550, the isolation value may be calculated as X = aqyZ(t) where t is a count of the tenants associated with the workloads and Z(t) is a second function. For example, Z(t) may be calculated as 2 although other functions are also contemplated. Returning to the sharing configuration 550, the isolation value may be calculated as X = aqyV(r) where r is a count of the requests associated with the workloads and V(r) is a third function. For example, V(r) may be calculated as 2^r, although other functions are also contemplated. In some cases, two or more of the predefined functions may be applied when calculating the isolation value.

[00131] The service orchestrator 222 generates an isolation index for the network service based on the selected ones of the plurality of predefined isolation values (which in some cases may include one or more amplification values). In some embodiments, generating the isolation index comprises applying a predefined function to the selected ones of the plurality of predefined isolation values. One example of the predefined function is summing the isolation values. Another example of the predefined function is returning a maximum isolation value of the isolation values. Other mathematical and/or logical functions are also contemplated, e.g., normalizing the isolation values by summing the isolation values and dividing by the number of isolation values.

[00132] Figures 8A and 8B illustrate block diagrams of exemplary deployments of multiple service functions. More specifically, Figures 8A and 8B provide an example of two deployment options (or possible workload placements) for two service requests for different tenants on different hosting candidates within a distributed cloud infrastructure: an access site 600 (generally disposed closer to end users) and a national data center 615 (generally disposed further from end users; often used for centralized applications).

[00133] The first service request for the first tenant includes two service functions: UPF 608-1 and LBO 610-1 The second service request for the second tenant includes two service functions: UPF 608-2 and LBO 610-2. As shown, each of the deployment options includes two Kubernetes clusters 604-1, 604-2. In the first possible workload placement shown in Figure 8A, the service functions UPF 608-1, LBO 610-1, UPF 608-2, LBO 610-2 are assigned as container applications in a respective Kubernetes cluster 604-1, 604-2 on PMs 602-1, 602-2, 602-3. As shown, UPF 608-1 is deployed on a pod 606-2 and LBO 610-1 is deployed on a pod 606-1 of the cluster 604-1. UPF 608-2 and LBO 610-2 are deployed on a pod 606-3 of cluster 604-2. [00134] When analyzing the first possible workload placement, the service orchestrator 222 may determine the isolation value of the first service request as corresponding to the sharing configuration 540 (i.e., UPF 608-1 and LBO 610-1 have a shared VM environment in cluster 604-1; X = q). The service orchestrator 222 may determine the isolation value of the second service request as corresponding to the sharing configuration 550 (i.e., UPF 608-2 and LBO 610-2 have a shared containerized environment in cluster 604-2, a shared virtual computing unit in pod 606-3, and a shared PM 602-3; X= aqy). Using the example of predefined isolation value components discussed above, the first service request has an isolation value of X = 1.50, and the second service request has an isolation value of X = 5.76. Assuming that the predefined function applied by the service orchestrator 222 is selecting a maximum isolation value, the isolation index for the first possible workload placement is calculated as 5.76. There are no resources shared between the different tenants of the first service request and the second service request, so the isolation index does not have a corresponding amplification value applied.

[00135] In the second possible workload placement shown in Figure 8B, the service functions UPF 608-1, LBO 610-1, UPF 608-2, LBO 610-2 are assigned as container applications in VM- based Kubernetes in OpenStack. As shown, UPF 608-1 and LBO 610-1 are deployed on a pod 606-1 of the cluster 604-1, and UPF 608-2 and LBO 610-2 are deployed on a pod 606-2 of cluster 604-2. Each pod 606-1, 606-2 is also assigned to a different OpenStack VM (e.g., controllers 620-1, 620-2) within the same OpenStack environment.

[00136] When analyzing the second possible workload placement, the service orchestrator 222 may determine the isolation value of the first service request as corresponding to the sharing configuration 565 (i.e., UPF 608-1 and LBO 610-1 have a shared virtual computing element in pod 606-1, a shared containerized environment in cluster 604-1, a shared VM environment and a shared VM in controller 620-1, and a shared PM 602-1; X = aPypq). Similarly, the service orchestrator 222 may determine the isolation value of the second service requests as also corresponding to the sharing configuration 565. Using the example of predefined isolation value components discussed above, the first service request and the second service request each have an isolation value of = 11.52.

[00137] Using the maximum of the isolation values, the service orchestrator 222 may calculate the isolation index for the second possible workload placement as 11.52, which indicates that the first possible workload placement provides greater isolation (a lower isolation index).

[00138] Further, if we assume that the first service request and the second service request share the same VM environment, an amplification value of Z(t U 2‘ may be applied to the isolation index. Because the first service request and the second service request correspond to two different tenants, the isolation index for the second possible workload placement may be calculated as 11.52 * 2² = 46.08.

[00139] Figure 9 A illustrates an exemplary graphical user interface (GUI) 700 for specifying parameters for possible workload placements, according to some embodiments. In some embodiments, the GUI 700 is included in the user interface 274 of Figure 5 A.

[00140] The GUI 700 includes an object 702 for a desired isolation index, with an object 704 operable to receive user input for the desired isolation index. The user input for the desired isolation index may have any suitable form, such as a maximum acceptable value of the isolation index, a range of acceptable values of the isolation index, and so forth.

[00141] The GUI 700 further includes an object 706 for one or more desired parameters, with an object 708 operable to receive user input for the one or more desired parameters. Some nonlimiting examples of the one or more desired parameters include cost, resource efficiency, latency, geolocation, affinity/anti-affinity rules, and so forth.

[00142] The GUI 700 further includes an object 710 operable to receive user input (e.g., a finger press or pointer click) to generate and display possible workload placements consistent with the values input for the desired isolation index and/or the desired parameter(s).

[00143] Figure 9B illustrates an exemplary GUI 715 for assessing possible workload placemen^ s) for a deployment, according to some embodiments. In some embodiments, the GUI 715 is included in the user interface 274 of Figure 5 A, and may be generated and displayed responsive to user input at object 710.

[00144] The GUI 715 supports a visual comparison between characteristics of one or more prospective deployments 728, 738 and optionally an existing deployment 726. The prospective deployment 728 corresponds to a possible workload placement 730, and the prospective deployment 738 corresponds to a possible workload placement 740. As shown, the characteristics include a deployment ID 716, an isolation index 718, a ranking 720, a cost 722, and a resource efficiency 724, although other compositions of characteristics are also contemplated.

[00145] The deployment ID 716 may be assigned (e.g., by the service orchestrator 222) to the existing deployment 726 and/or to the prospective deployments 728, 738, and may have any suitable form. Some examples of the deployment ID include descriptions (e.g., “Access Site” or “National Data Center” as in Figures 8A, 8B) as well as alphanumeric codes.

[00146] The isolation index 718 may be the value of the isolation index calculated by the service orchestrator 222, as discussed in greater detail above. The ranking 720 may be provided as a characterization of the isolation index 718, such as the relative ranking of the prospective deployments 728, 738 and the existing deployment 726, a tiering system with threshold values, and so forth.

[00147] The cost 722 may reflect one or more cost components associated with the deployment, such as a service cost, server cost, overhead cost, and so forth. The resource efficiency 724 may be calculated based on the cost 722 or components thereof. In this way, a user may be able to visually assess the prospective deployments 728, 738 relative to each other and to the existing deployment 726. In some cases, the user may consider a trade-off between the degree of isolation and the cost 722 and/or the resource efficiency 724. In other embodiments, a composite index or score may be calculated by the service orchestrator 222 and presented in the GUI 715 that considers both the isolation index 718 and the cost 722 and/or the resource efficiency 724.

[00148] The GUI 715 further includes an object 732 (“View”) operable to receive user input and display additional information about the prospective deployment 728, e g., a map and/or list of the service functions and their possible placement. The GUI 715 further includes an object 734 (“Alter parameter(s)”) operable to receive user input to alter one or more parameters that were used to generate the prospective deployment 728. The altered param eter(s) may then be used to generate one or more additional possible workload placements.

[00149] The GUI 715 further includes an object 736 (“Deploy”) operable to receive user input to deploy the deployment 728. In some embodiments, deploying the deployment 728 alters the existing deployment 726 done according to an existing workload placement to one done according to the possible workload placement 730, which may include reassigning a host for one or more of the service functions and/or using a different network connection to interconnect the service functions. [00150] Corresponding to the prospective deployment 738, the GUI 715 further includes an object 742 (“View”) similar to the object 732, an object 744 (“Alter param eter(s)”) similar to the object 734, and an object 746 (“Deploy”) similar to the object 736.

[00151] An electronic device stores and transmits (internally and/or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors (e g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, other electronic circuitry, a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data. For instance, an electronic device may include non-volatile memory containing the code since the non-volatile memory can persist code/data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower nonvolatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device. Typical electronic devices also include a set or one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and/or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of receiving data from other electronic devices over a wireless connection, and/or sending data out to other devices via a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and/or transceiver s) suitable for radiofrequency communication. The radio circuitry may convert digital data into a radio signal having the appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal may then be transmitted via antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprise network interface controlled s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate in connecting the electronic device to other electronic devices allowing them to communicate via wire through plugging in a cable to a physical port connected to a NIC. One or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and/or hardware.

[00152] A network device (ND) is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and/or subscriber management), and/or provide support for multiple application services (e.g., data, voice, and video).

[00153] Figure 10A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments of the invention. Figure 10A shows NDs 800A-800HH, and their connectivity by way of lines between 800A-800B, 800B-800C, 800C-800D, 800D-800E, 800E-800F, 800F- 800G, and 800A-800G, as well as between 800H and each of 800A, 800C, 800D, and 800G. These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link). An additional line extending from NDs 800A, 800E, and 800F illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs; while the other NDs may be called core NDs).

[00154] Two of the exemplary ND implementations in Figure 9A are: 1) a special-purpose network device 802 that uses custom application-specific integrated-circuits (ASICs) and a special-purpose operating system (OS); and 2) a general-purpose network device 804 that uses common off-the-shelf (COTS) processors and a standard OS.

[00155] The special-purpose network device 802 includes networking hardware 810 comprising a set of one or more processor(s) 812, forwarding resource(s) 814 (which typically include one or more ASICs and/or network processors), and physical network interfaces (NIs) 816 (through which network connections are made, such as those shown by the connectivity between NDs 800A-800H), as well as non-transitory machine-readable storage media 918 having stored therein networking software 820. During operation, the networking software 820 may be executed by the networking hardware 810 to instantiate a set of one or more networking software instance(s) 822. Each of the networking software instance(s) 822, and that part of the networking hardware 810 that executes that network software instance (be it hardware dedicated to that networking software instance and/or time slices of hardware temporally shared by that networking software instance with others of the networking software instance(s) 822), form a separate virtual network element 830A-830R. Each of the virtual network element(s) (VNEs) 830A-830R includes a control communication and configuration module 832A-832R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 834A-R, such that a given virtual network element (e.g., 830A) includes the control communication and configuration module (e.g., 832A), a set of one or more forwarding table(s) (e g , 834A), and that portion of the networking hardware 810 that executes the virtual network element (e.g., 830A).

[00156] The special-purpose network device 802 is often physically and/or logically considered to include: 1) a ND control plane 824 (sometimes referred to as a control plane) comprising the processor(s) 812 that execute the control communication and configuration module(s) 832A-832R; and 2) a ND forwarding plane 826 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 814 that utilize the forwarding table(s) 834A-834R and the physical NIs 816. By way of example, where the ND is a router (or is implementing routing functionality), the ND control plane 824 (the processor(s) 812 executing the control communication and configuration module(s) 832A-832R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 834A-834R, and the ND forwarding plane 826 is responsible for receiving that data on the physical NIs 816 and forwarding that data out to the appropriate ones of the physical NIs 816 based on the forwarding table(s) 834A-834R.

[00157] Figure 10B illustrates an exemplary way to implement the special-purpose network device 802, according to some embodiments of the invention. Figure 10B shows a specialpurpose network device including cards 838 (typically hot pluggable). While in some embodiments, the cards 838 are of two types (one or more that operate as the ND forwarding plane 826 (sometimes called line cards), and one or more that operate to implement the ND control plane 824 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and/or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi-application card). A service card can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway). By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryption algorithms. These cards are coupled together through one or more interconnect mechanisms illustrated as backplane 836 (e.g., a first full mesh coupling the line cards and a second full mesh coupling all of the cards).

[00158] Returning to Figure 10A, the general-purpose network device 804 includes hardware 840 comprising a set of one or more processor(s) 842 (which are often COTS processors) and physical NIs 846, as well as non-transitory machine-readable storage media 848 having stored therein software 850 During operation, the processor(s) 842 execute the software 850 to instantiate one or more sets of one or more applications 814A-814R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment, the virtualized layer 854 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 862A-862R called software containers that may each be used to execute one (or more) of the sets of applications 814A-814R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment the virtualized layer 854 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 814A-814R is run on top of a guest operating system within an instance 862A-862R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or, through para-virtualization the operating system and/or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some or all of the applications are implemented as unikernel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers/libraries of OS services) that provide the particular OS services needed by the application. As a unikernel can be implemented to run directly on hardware 840, directly on a hypervisor (in which case the unikernel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikernels running directly on a hypervisor represented by virtualized layer 854, unikernels running within software containers represented by instances 862A-862R, or as a combination of unikernels and the above-described techniques (e.g., unikemels and virtual machines both run directly on a hypervisor, unikernels and sets of applications that are run in different software containers).

[00159] The instantiation of the one or more sets of one or more applications 814A-R, as well as virtualization, if implemented, are collectively referred to as software instance(s) 852. Each set of applications 814A-814R, corresponding virtualization construct (e.g., instance 862A- 862R) if implemented, and that part of the hardware 840 that executes them (be it hardware dedicated to that execution and/or time slices of hardware temporally shared), forms a separate virtual network element(s) 860A-860R.

[00160] The virtual network element(s) 860A-860R perform similar functionality to the virtual network element(s) 830A-830R - e.g., similar to the control communication and configuration module(s) 832A and forwarding table(s) 834A (this virtualization of the hardware 840 is sometimes referred to as network function virtualization (NFV)). Thus, NFV may be used to consolidate many network equipment types onto industry standard high-volume server hardware, physical switches, and physical storage, which could be located in data centers, NDs, and customer premise equipment (CPE). While embodiments of the invention are illustrated with each instance 862A-862R corresponding to one VNE 860A-860R, alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 862A-862R to VNEs also apply to embodiments where such a finer level of granularity and/or unikemels are used.

[00161] In certain embodiments, the virtualized layer 854 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 862A-862R and the physical NI(s) 846, as well as optionally between the instances 862A-862R; in addition, this virtual switch may enforce network isolation between the VNEs 860A-860R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)).

[00162] The third exemplary ND implementation in Figure 10A is a hybrid network device 806, which includes both custom ASICs/special-purpose OS and COTS processors/ standard OS in a single ND or a single card within an ND. In certain embodiments of such a hybrid network device, a platform VM (i.e., a VM that that implements the functionality of the special-purpose network device 802) could provide for para-virtualization to the networking hardware present in the hybrid network device 806.

[00163] Regardless of the above exemplary implementations of an ND, when a single one of multiple VNEs implemented by an ND is being considered (e.g., only one of the VNEs is part of a given virtual network) or where only a single VNE is currently being implemented by an ND, the shortened term network element (NE) is sometimes used to refer to that VNE. Also, in all of the above exemplary implementations, each of the VNEs (e.g., VNE(s) 830A-830R, VNEs 860A-860R, and those in the hybrid network device 806) receives data on the physical NIs (e.g., 816, 846) and forwards that data out the appropriate ones of the physical NIs (e.g., 816, 846). For example, a VNE implementing IP router functionality forwards IP packets on the basis of some of the IP header information in the IP packet; where IP header information includes source IP address, destination IP address, source port, destination port (where “source port” and “destination port” refer herein to protocol ports, as opposed to physical ports of a ND), transport protocol (e.g., user datagram protocol (UDP), Transmission Control Protocol (TCP), and differentiated services code point (DSCP) values.

[00164] Figure 10C illustrates various exemplary ways in which VNEs may be coupled according to some embodiments of the invention. Figure 10C shows VNEs 870A.1-870A.P (and optionally VNEs 870A.Q-870A.R) implemented in ND 800A and VNE 870H.1 in ND 800H. In Figure 10C, VNEs 870A.1-P are separate from each other in the sense that they can receive packets from outside ND 800A and forward packets outside of ND 800 A;

VNE 870A.1 is coupled with VNE 870H.1, and thus they communicate packets between their respective NDs; VNE 870A.2-870A.3 may optionally forward packets between themselves without forwarding them outside of the ND 800A; and VNE 870A.P may optionally be the first in a chain of VNEs that includes VNE 870A.Q followed by VNE 870A.R (this is sometimes referred to as dynamic network servicing, where each of the VNEs in the series of VNEs provides a different service - e.g., one or more layer 4-7 network services). While Figure 10C illustrates various exemplary relationships between the VNEs, alternative embodiments may support other relationships (e.g., more/fewer VNEs, more/fewer dynamic network services, multiple different dynamic network services with some common VNEs and some different VNEs).

[00165] The NDs of Figure 10A, for example, may form part of the Internet or a private network; and other electronic devices (not shown; such as end user devices, including workstations, laptops, netbooks, tablets, palm tops, mobile phones, smartphones, phablets, multimedia phones, Voice Over Internet Protocol (VOIP) phones, terminals, portable media players, GPS units, wearable devices, gaming systems, set-top boxes, Internet enabled household appliances) may be coupled to the network (directly or through other networks such as access networks) to communicate over the network (e.g., the Internet or virtual private networks (VPNs) overlaid on (e.g., tunneled through) the Internet) with each other (directly or through servers) and/or access content and/or services. Such content and/or services are typically provided by one or more servers (not shown) belonging to a service/content provider or one or more end-user devices (not shown) participating in a peer-to-peer (P2P) service, and may include, for example, public webpages (e.g., free content, store fronts, search services), private webpages (e.g., usemame/password accessed webpages providing email services), and/or corporate networks over VPNs. For instance, end-user devices may be coupled (e.g., through customer premise equipment coupled to an access network (wired or wirelessly)) to edge NDs, which are coupled (e.g., through one or more core NDs) to other edge NDs, which are coupled to electronic devices acting as servers. However, through compute and storage virtualization, one or more of the electronic devices operating as the NDs in Figure 10A, may also host one or more such servers (e.g., in the case of the general purpose network device 804, one or more of the software instances 862A-862R may operate as servers; the same would be true for the hybrid network device 806; in the case of the special-purpose network device 802, one or more such servers could also be run on a virtualized layer executed by the processor(s) 812); in which case the servers are said to be co-located with the VNEs of that ND.

[00166] A virtual network is a logical abstraction of a physical network (such as that in Figure 10A) that provides network services (e.g., L2 and/or L3 services). A virtual network can be implemented as an overlay network (sometimes referred to as a network virtualization overlay) that provides network services (e.g., layer 2 (L2, data link layer) and/or layer 3 (L3, network layer) services) over an underlay network (e.g., an L3 network, such as an Internet Protocol (IP) network that uses tunnels (e.g., generic routing encapsulation (GRE), layer 2 tunneling protocol (L2TP), IPSec) to create the overlay network).

[00167] A network virtualization edge (NVE) sits at the edge of the underlay network and participates in implementing the network virtualization; the network-facing side of the NVE uses the underlay network to tunnel frames to and from other NVEs; the outward-facing side of the NVE sends and receives data to and from systems outside the network. A virtual network instance (VNI) is a specific instance of a virtual network on an NVE (e.g., a NE/VNE on an ND, a part of a NE/VNE on a ND where that NE/VNE is divided into multiple VNEs through emulation); one or more VNIs can be instantiated on an NVE (e.g., as different VNEs on an ND). A virtual access point (VAP) is a logical connection point on the NVE for connecting external systems to a virtual network; a VAP can be physical or virtual ports identified through logical interface identifiers (e.g., a VLAN ID).

[00168] Examples of network services include: 1) an Ethernet LAN emulation service (an Ethernet-based multipoint service similar to an Internet Engineering Task Force (lETF)Multiprotocol Label Switching (MPLS) or Ethernet VPN (EVPN) service) in which external systems are interconnected across the network by a LAN environment over the underlay network (e.g., an NVE provides separate L2 VNIs (virtual switching instances) for different such virtual networks, and L3 (e.g., IP/MPLS) tunneling encapsulation across the underlay network); and 2) a virtualized IP forwarding service (similar to IETF IP VPN (e.g., Border Gateway Protocol (BGP)/MPLS IPVPN) from a service definition perspective) in which external systems are interconnected across the network by an L3 environment over the underlay network (e.g., an NVE provides separate L3 VNIs (forwarding and routing instances) for different such virtual networks, and L3 (e.g., IP/MPLS) tunneling encapsulation across the underlay network)). Network services may also include quality of service capabilities (e.g., traffic classification marking, traffic conditioning and scheduling), security capabilities (e.g., filters to protect customer premises from network - originated attacks, to avoid malformed route announcements), and management capabilities (e.g., full detection and processing).

[00169] Fig. 10D illustrates a network with a single network element on each of the NDs of Figure 10A, and within this straightforward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments. Specifically, Figure 10D illustrates network elements (NEs) 870A-870H with the same connectivity as the NDs 800 A-H of Figure 10A.

[00170] Figure 10D illustrates that the distributed approach 872 distributes responsibility for generating the reachability and forwarding information across the NEs 870A-870H; in other words, the process of neighbor discovery and topology discovery is distributed.

[00171] For example, where the special-purpose network device 802 is used, the control communication and configuration module(s) 832A-832R of the ND control plane 824 typically include a reachability and forwarding information module to implement one or more routing protocols (e.g., an exterior gateway protocol such as Border Gateway Protocol (BGP), Interior Gateway Protocol(s) (IGP) (e.g., Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), Routing Information Protocol (RIP), Label Distribution Protocol (LDP), Resource Reservation Protocol (RSVP) (including RSVP-Traffic Engineering (TE): Extensions to RSVP for LSP Tunnels and Generalized Multi -Protocol Label Switching (GMPLS) Signaling RSVP-TE)) that communicate with other NEs to exchange routes, and then selects those routes based on one or more routing metrics. Thus, the NEs 870A-870H (e.g., the processor(s) 812 executing the control communication and configuration module(s) 832A-R) perform their responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) by distributively determining the reachability within the network and calculating their respective forwarding information. Routes and adjacencies are stored in one or more routing structures (e.g., Routing Information Base (RIB), Label Information Base (LIB), one or more adjacency structures) on the ND control plane 824. The ND control plane 824 programs the ND forwarding plane 826 with information (e.g., adjacency and route information) based on the routing structure(s). For example, the ND control plane 824 programs the adjacency and route information into one or more forwarding table(s) 834A-834R (e.g., Forwarding Information Base (FIB), Label Forwarding Information Base (LFIB), and one or more adjacency structures) on the ND forwarding plane 826. For layer 2 forwarding, the ND can store one or more bridging tables that are used to forward data based on the layer 2 information in that data. While the above example uses the special-purpose network device 802, the same distributed approach 872 can be implemented on the general-purpose network device 804 and the hybrid network device 806.

[00172] Figure 10D illustrates that a centralized approach 874 (also known as software defined networking (SDN)) which decouples the system that makes decisions about where traffic is sent from the underlying systems that forwards traffic to the selected destination. The illustrated centralized approach 874 has the responsibility for the generation of reachability and forwarding information in a centralized control plane 876 (sometimes referred to as an SDN control module, controller, network controller, OpenFlow controller, SDN controller, control plane node, network virtualization authority, or management control entity), and thus the process of neighbor discovery and topology discovery is centralized. The centralized control plane 876 has a south bound interface 882 with a data plane 880 (sometimes referred to the infrastructure layer, network forwarding plane, or forwarding plane (which should not be confused with an ND forwarding plane)) that includes the NEs 870A-870H (sometimes referred to as switches, forwarding elements, data plane elements, or nodes). The centralized control plane 876 includes a network controller 878, which includes a centralized reachability and forwarding information module 879 that determines the reachability within the network and distributes the forwarding information to the NEs 870A-870H of the data plane 880 over the south bound interface 882 (which may use the OpenFlow protocol). Thus, the network intelligence is centralized in the centralized control plane 876 executing on electronic devices that are typically separate from the NDs.

[00173] For example, where the special-purpose network device 802 is used in the data plane 880, each of the control communication and configuration module(s) 832A-832R of the ND control plane 824 typically include a control agent that provides the VNE side of the southbound interface 882. In this case, the ND control plane 824 (the processor(s) 812 executing the control communication and configuration module(s) 832A-832R) performs its responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) through the control agent communicating with the centralized control plane 876 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 879 (it should be understood that in some embodiments of the invention, the control communication and configuration module(s) 832A-832R, in addition to communicating with the centralized control plane 876, may also play some role in determining reachability and/or calculating forwarding information - albeit less so than in the case of a distributed approach; such embodiments are generally considered to fall under the centralized approach 874, but may also be considered a hybrid approach).

[00174] While the above example uses the special-purpose network device 802, the same centralized approach 874 can be implemented with the general purpose network device 804 (e.g., each of the VNE 860A-860R performs its responsibility for controlling how data (e.g., packets) is to be routed (e g., the next hop for the data and the outgoing physical NI for that data) by communicating with the centralized control plane 876 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 879; it should be understood that in some embodiments of the invention, the VNEs 860A-860R, in addition to communicating with the centralized control plane 876, may also play some role in determining reachability and/or calculating forwarding information - albeit less so than in the case of a distributed approach) and the hybrid network device 806. In fact, the use of SDN techniques can enhance the NFV techniques typically used in the general-purpose network device 804 or hybrid network device 806 implementations as NFV is able to support SDN by providing an infrastructure upon which the SDN software can be run, and NFV and SDN both aim to make use of commodity-server hardware and physical switches.

[00175] Figure 10D also shows that the centralized control plane 876 has a north-bound interface 884 to an application layer 886, in which resides application(s) 888. The centralized control plane 876 has the ability to form virtual networks 892 (sometimes referred to as a logical forwarding plane, network services, or overlay networks (with the NEs 870A-870H of the data plane 880 being the underlay network)) for the application(s) 888. Thus, the centralized control plane 876 maintains a global view of all NDs and configured NEs/VNEs, and it maps the virtual networks to the underlying NDs efficiently (including maintaining these mappings as the physical network changes either through hardware (ND, link, or ND component) failure, addition, or removal).

[00176] While Figure 10D shows the distributed approach 872 separate from the centralized approach 874, the effort of network control may be distributed differently or the two combined in certain embodiments of the invention. For example: 1) embodiments may generally use the centralized approach (SDN) 874, but have certain functions delegated to the NEs (e.g., the distributed approach may be used to implement one or more of fault monitoring, performance monitoring, protection switching, and primitives for neighbor and/or topology discovery); or 2) embodiments of the invention may perform neighbor discovery and topology discovery via both the centralized control plane and the distributed protocols, and the results compared to raise exceptions where they do not agree. Such embodiments are generally considered to fall under the centralized approach 874, but may also be considered a hybrid approach.

[00177] While Figure 10D illustrates the simple case where each of the NDs 800A-800H implements a single NE 870A-870H, it should be understood that the network control approaches described with reference to Figure 10D also work for networks where one or more of the NDs 800A-800H implement multiple VNEs (e.g., VNEs 830A-R, VNEs 860A-R, those in the hybrid network device 806). Alternatively or in addition, the network controller 878 may also emulate the implementation of multiple VNEs in a single ND. Specifically, instead of (or in addition to) implementing multiple VNEs in a single ND, the network controller 878 may present the implementation of a VNE/NE in a single ND as multiple VNEs in the virtual networks 892 (all in the same one of the virtual network(s) 892, each in different ones of the virtual network(s) 892, or some combination). For example, the network controller 878 may cause an ND to implement a single VNE (an NE) in the underlay network, and then logically divide up the resources of that NE within the centralized control plane 876 to present different VNEs in the virtual network(s) 879 (where these different VNEs in the overlay networks are sharing the resources of the single VNE/NE implementation on the ND in the underlay network).

[00178] On the other hand, Figures 10E and 10F, respectively, illustrate exemplary abstractions of NEs and VNEs that the network controller 878 may present as part of different ones of the virtual networks 892. Figure 10E illustrates the simple case of where each of the NDs 800A-800H implements a single NE 870A-870H (see Figure 10D), but the centralized control plane 876 has abstracted multiple of the NEs in different NDs (the NEs 870A-870C and 870G-870H) into (to represent) a single NE 8701 in one of the virtual network(s) 892 of Figure 10D, according to some embodiments of the invention. Figure 10E shows that in this virtual network, the NE 8701 is coupled to NE 870D and 870F, which are both still coupled to NE 870E.

[00179] Figure 10F illustrates a case where multiple VNEs (VNE 870A.1 and VNE 870H.1) are implemented on different NDs (ND 800A and ND 800H) and are coupled to each other, and where the centralized control plane 876 has abstracted these multiple VNEs such that they appear as a single VNE 870T within one of the virtual networks 892 of Figure 10D, according to some embodiments of the invention. Thus, the abstraction of an NE or VNE can span multiple NDs.

[00180] While some embodiments of the invention implement the centralized control plane 876 as a single entity (e g., a single instance of software running on a single electronic device), alternative embodiments may spread the functionality across multiple entities for redundancy and/or scalability purposes (e.g., multiple instances of software running on different electronic devices).

[00181] Similar to the network device implementations, the electronic device(s) running the centralized control plane 876, and thus the network controller 878 including the centralized reachability and forwarding information module 879, may be implemented in a variety of ways (e.g., a special-purpose device, a general-purpose (e.g., COTS) device, or hybrid device). These electronic device(s) would similarly include processor(s), a set or one or more physical NIs, and a non-transitory machine-readable storage medium having stored thereon the centralized control plane software.

[00182] A virtual circuit (VC), synonymous with virtual connection and virtual channel, is a connection-oriented communication service that is delivered by means of packet mode communication. Virtual circuit communication resembles circuit switching, since both are connection oriented, meaning that in both cases data is delivered in correct order, and signaling overhead is required during a connection establishment phase. Virtual circuits may exist at different layers. For example, at layer 4, a connection-oriented transport layer datalink protocol such as Transmission Control Protocol (TCP) may rely on a connectionless packet switching network layer protocol such as IP, where different packets may be routed over different paths, and thus be delivered out of order. Where a reliable virtual circuit is established with TCP on top of the underlying unreliable and connectionless IP protocol, the virtual circuit is identified by the source and destination network socket address pair, i.e., the sender and receiver IP address and port number. However, a virtual circuit is possible since TCP includes segment numbering and reordering on the receiver side to prevent out-of-order delivery. Virtual circuits are also possible at Layer 3 (network layer) and Layer 2 (datalink layer); such virtual circuit protocols are based on connection-oriented packet switching, meaning that data is always delivered along the same network path, i.e., through the same NEs/VNEs. In such protocols, the packets are not routed individually and complete addressing information is not provided in the header of each data packet; only a small virtual channel identifier (VCI) is required in each packet; and routing information is transferred to the NEs/VNEs during the connection establishment phase; switching only involves looking up the virtual channel identifier in a table rather than analyzing a complete address. Examples of network layer and datalink-layer virtual circuit protocols, where data always is delivered over the same path: X.25, where the VC is identified by a virtual channel identifier (VCI); Frame relay, where the VC is identified by a VCI; Asynchronous Transfer Mode (ATM), where the circuit is identified by a virtual path identifier (VPI) and virtual channel identifier (VCI) pair; General Packet Radio Service (GPRS); and Multiprotocol label switching (MPLS), which can be used for IP over virtual circuits (Each circuit is identified by a label).

[00183] Each VNE (e g., a virtual router, a virtual bridge (which may act as a virtual switch instance in a Virtual Private LAN Service (VPLS) is typically independently administrable. For example, in the case of multiple virtual routers, each of the virtual routers may share system resources, but is separate from the other virtual routers regarding its management domain, AAA (authentication, authorization, and accounting) name space, IP address, and routing database(s). Multiple VNEs may be employed in an edge ND to provide direct network access and/or different classes of services for subscribers of service and/or content providers.

[00184] While the flow diagrams in the figures show a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).

[00185] While the invention has been described in terms of several embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described, can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting.

Claims

CLAIMS What is claimed is:

1. A method of analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure, wherein the cloud infrastructure includes resources and the network service is to comprise service functions and network connections that interconnect the service functions, the method comprising: selecting (10), for one or more of the service functions, one of a plurality of predefined isolation values based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure, wherein the different ones of the plurality of isolation values represent different levels of isolation provided by respective ones of the plurality of levels of virtualization; generating (12) an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values, wherein the isolation index represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure; and facilitating (14) a determination, using the isolation index, whether to use the possible workload placement for the network service.

2. The method of claim 1, wherein facilitating the determination whether to use the possible workload placement for the network service comprises: presenting (40) the isolation index using a user interface configured to permit user selection of the possible workload placement.

3. The method of claims 1 or 2, wherein facilitating the determination whether to use the possible workload placement for the network service further comprises: determining (42) a cost to deploy the network service according to the possible workload placement; and presenting (44) the cost using the user interface.

4. The method of any of claims 1-3, further comprising: responsive to determining (16) to use the possible workload placement, deploying (18) the network service according to the possible workload placement.

5. The method of claim 4, wherein deploying the network service according to the possible workload placement further comprises: altering an existing deployment done according to an existing workload placement to one done according to the possible workload placement, wherein altering the existing deployment of the network service comprises, for at least one of the service functions, one or both of (i) reassigning which of a plurality of computing systems will host the one of the service functions and (ii) using a different network connection to interconnect the one of the service functions to another of the service functions

6. The method of any of claims 1-3, further comprising: responsive to determining (16) not to use the possible workload placement, performing one or more of the following: generating (20) a re-dimensioning recommendation to modify the possible workload placement; or altering (22) one or more criteria of the deployment.

7. The method of any of claims 1-6, wherein generating the isolation index for the deployment comprises: applying (32) a predefined function to the selected ones of the plurality of predefined isolation values.

8. The method of any of claims 1-7, wherein the plurality of predefined isolation values comprises values for two or more of the following configurations of the possible workload placement and the one or more other workloads: a shared physical computing system; a shared virtual machine (VM) environment; a shared virtual computing system; different virtual computing systems on a shared physical computing system; a shared containerized environment; a same virtual computing unit in the containerized environment; and different virtual computing units on a shared physical computing system in the containerized environment.

9. The method of any of claims 1-8, wherein the plurality of predefined isolation values comprises values for two or more of the following configurations of the possible workload placement and the one or more other workloads: different virtual computing units of a shared containerized environment on a shared virtual machine (VM) environment; different virtual computing units of the shared containerized environment on a same virtual computing system of the shared VM environment; and a shared virtual computing unit of the shared containerized environment on the shared VM environment.

10. The method of any of claims 7-9, wherein the predefined function further depends on one or more of the following: a count of the one or more other workloads; a count of tenants associated with the one or more other workloads; and a count of requests associated with the one or more other workloads.

11. The method of any of claims 1-10, further comprising: determining (80) one or more other possible workload placements for the deployment of the network service on the cloud infrastructure; and generating (92) a respective isolation index for each of the one or more other possible workload placements, wherein facilitating the determination whether to use the possible workload placement for the network service also uses the isolation indexes for the one or more other possible workload placements.

12. A machine-readable medium comprising computer program code which when executed by a computer carries out the method steps of any of claims 1-11.

13. An electronic device for analyzing a possible workload placement for a deployment of a network service on a cloud infrastructure, wherein the cloud infrastructure includes resources and the network service is to comprise service functions and network connections that interconnect the service functions, the electronic device comprising: a set of one or more processors; and a machine-readable medium that provides instructions that, if executed by the set of one or more processors, will cause the electronic device to perform operations comprising: selecting (10), for one or more of the service functions, one of a plurality of predefined isolation values based on an identification of a virtualization level of the possible workload placement from a plurality of levels of virtualization supported by the cloud infrastructure, wherein the different ones of the plurality of isolation values represent different levels of isolation provided by respective ones of the plurality of levels of virtualization; generating (12) an isolation index for the deployment based on the selected ones of the plurality of predefined isolation values, wherein the isolation index represents an expected level of interference between the possible workload placement and one or more other workloads of services deployed on the cloud infrastructure; and facilitating (14) a determination, using the isolation index, whether to use the possible workload placement for the network service.

14. The electronic device of claim 13, wherein facilitating the determination whether to use the possible workload placement for the network service further causes the electronic device to perform operations comprising: presenting (40) the isolation index using a user interface configured to permit user selection of the possible workload placement.

15. The electronic device of claim 13 or 14, wherein facilitating the determination whether to use the possible workload placement for the network service further causes the electronic device to perform operations comprising: determining (42) a cost to deploy the network service according to the possible workload placement; and presenting (44) the cost using the user interface.