US20170264491A1

US20170264491A1 - Intent based controller for provisioning a network

Info

Publication number: US20170264491A1
Application number: US15/437,182
Authority: US
Inventors: Denis DeRuijter
Original assignee: Optum Inc
Current assignee: Optum Inc
Priority date: 2016-03-12
Filing date: 2017-02-20
Publication date: 2017-09-14
Also published as: US10862748B1

Abstract

An intent driven controller (IDC) operates to automatically build and provision networks using specified communication service requirements for one or more computer applications running in the network, optimizing among these requirements given the available networking resources, and then provisioning the forwarding gear according the network requirements.

Description

1. FIELD OF THE INVENTION

The present disclosure relates to managing network infrastructure and to a controller for provisioning the infrastructure in a manner that makes very efficient use of the infrastructure capabilities.

2. BACKGROUND

Existing network provisioning methodologies require stitching a network together by performing a series of distinct tasks such as configuring VLANs, IP subnets, routing protocols and possibly overlays. This approach requires that an administrator manually translate application requirements into network configuration based on knowledge of the deployed equipment and its operational software.

3. BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be best understood by reading the specification with reference to the following figures, in which:

FIG. 1 is a diagram showing a network arrangement having several different sites.

FIG. 2 is a diagram illustrating an abstract representation of a network showing forwarding and processing elements.

FIG. 3 is a diagram illustrating a network having intent driven controller elements and non-intent driven controller elements.

FIG. 4 is a diagram showing functional blocks comprising an intent driven element controller for forwarding and processing elements.

FIG. 5 is a diagram showing the organization of sector and region intent driven controllers in a hierarchical control plane.

FIG. 6 is a diagram showing processing elements linked to one another through communication spheres.

FIG. 7 is a diagram illustrating processing steps in a placement algorithm 700 used to determine optimal forwarding topologies.

FIG. 8 is a logical flow diagram of the processing steps for an ordering algorithm 800.

FIG. 9 is a diagram illustrating the distribution of placement algorithm execution across a plurality of intent driven controllers.

FIG. 10 is a diagram illustrating asymmetrical forwarding behavior between a plurality of endpoints.

FIG. 11 is a diagram illustrating how mission critical software applications can leverage intent to interact with the network infrastructure.

4. DETAILED DESCRIPTION

The current largely manual process for provisioning network resources is both time consuming and does not necessarily result in an optimal flow of network traffic from one point of time to another. I have designed a controller that operates to optimize the utilization of network resources using high-level intent information to express communication requirements for computer applications operating in the network. This intent driven controller (IDC) operates to automatically build and provision networks based on high-level descriptions (intent) of what needs to be achieved, optimizing among these requirements given the available networking resources and then provisioning the forwarding gear. And we do this without taking away operator oversight, but retain control over the outcome similar to the way a pilot uses computerized flight controls to operate an airliner. Generally, intent-driven networks provide the means to create agile networks where the applications or operators can express what they need without having to detail how to make it so.
The fluency with which the communication needs are met brings agility to the network and in turn allows compute and storage endpoints to take full advantage of this flexible network: We can select the best compute or storage endpoints knowing what communication needs must be met. And we can provide compute or storage systems with the network services for access to and synchronization among the constituent compute or storage elements. IDC controlled networks deliver a fluid resource fabric where applications express the logical compute, storage and networking capabilities they need so we can optimally select and provision the physical processors, disk drives and networking resources to provide them.
Infrastructure
Referring now to FIG. 1, this figure shows a number of different sites (datacenters, corporate offices, campus, residences, radio/satellite facilities) with local networks (LANs) that are interconnected by public or private wide area networks (WANs). A site can use wired and wireless networks to connect a variety of devices (computation and storage systems, sensors, actuators, printers, smart phones, tablets, etc.). Such devices provide one or more abstract compute, input, output, storage and communication functions. A system is a clustering of such devices located within, partially within, or across one or more sites. We refer to the communication functions in such devices as forwarding elements (whether they represent individual enclosures or integrated functionality) connected by communication channels as the abstraction for the transmission paths irrespective of ISO layer. Similarly, we refer to the input, output, storage and compute functions as processing elements. In FIG. 1, the forwarding elements relay, mirror or filter communication traffic, and the processing elements produce, consume, retain or transform the data as carried by communication channels and forwarding elements. Note that processing and forwarding elements may be physically combined in one device (for instance a compute server that can relay traffic between Ethernet ports).
Error! Reference source not found. illustrates an abstract network model 200 having processing and forwarding elements linked by communication channels, and having all of the forwarding elements being shown inside a dashed oval 210.
Error! Reference source not found. refers to Intent Driven Controller (IDC) elements as Intent Driven Controller physical or virtual machines or Intent Driven Controller physical or virtual switches, and displays the forwarding and processing elements in four rings, labeled 300, 310, 320, and 330, with Intent Driven Controller elements and communication channels, the latter illustrated as solid lines, and conventional, non-Intent Driven Controller elements and communication channels, the latter being illustrated as dashed lines. It illustrates interconnections of forwarding and processing elements and shows how an Intent Driven Controller network might co-exist with or exchange traffic with a conventional, non-Intent Driven Controller network using inter-domain communication channels, the latter illustrated by dotted lines. Within a given system, one or more Intent Driven Controllers provide for communication among processing elements by operating forwarding elements and communication channels. The Intent Driven Controllers are comprised of software programs that execute on physical or virtual computers and communicate with one or more forwarding elements and communication channels under their control. Hereinafter, an Intent Driven Controller is referred to as an IDC.
Communication Capabilities, Configuration and State
Forwarding elements, processing elements, communication channels and controllers all have attributes—whether implemented in hardware, firmware or software—that we separate into three distinct categories as capabilities, configuration and state. In this regard, capabilities are immutable attributes that describe inherent functions, features or operating aspects which includes, but is not limited to, information across various ISO layers like number of ports, possible line speeds, alternative signal encodings, propagation delay characteristics, internal table capacities, functional abilities and various internal compositions such as the presence of crossbars to forward signal streams, switches to forward frames, routers to forward datagrams, etc. Configuration refers to dynamic attributes that can only be changed through administrative action and includes, but is not limited to, information like enabling and disabling features, protocol options, operating limits. Examples are permitted line speeds, turning auto-negotiation on and off, encryption settings, overclocking a device and manipulating forwarding behavior in crossbars, switches and routers. State refers to dynamic information that is modified as a result of autonomous operation and includes, but is not limited to, statistics counters, learned forwarding behavior, auto-negotiated line rates. While it is noted that some functionality might appear to fit more than one description, this may be an indication that they are different attributes. For instance, line speed could refer to the set of line speeds a device can support (capability), or the set of line speeds a device is permitted to support (configuration) or the actual line speed with which it is currently operating (state).
Intent Driven Controller
Error! Reference source not found. shows functional elements comprising an IDC 400 in communication with one or more forwarding functions 405 and communication channels 406, wherein a forwarding function may represent a forwarding controller or a simpler device such as a serial bus that allows the controller to communicate over a communication channel. The IDC 400 is comprised of a controller function 410, a configuration repository 420, a capability repository 411, a capability retrieve function 413, a configuration retrieve function 414, a configuration retrieve function 414, a configuration modification function 415, a state retrieve function 416, and a state repository 412. The capability retrieve function 413 operates to enable the retrieval of the capabilities of the IDC 400. The capability repository 411 represents the capabilities of the IDC and all of the forwarding functions 405 and communication channels 406 that can be operated by the controller. The configuration retrieve function 414 operates to enable the retrieval of current and possibly past configurations of the IDC and any forwarding functions and communication channels under its operation. The configuration modification function 415 enables the modification of the current or any alternative configuration of the IDC and any forwarding functions and communication channels under its operation. The configuration repository 420 represents the current and possibly past configuration of the IDC and any forwarding functions and communication channels under its operation. The state retrieve function 416 enables the retrieval of the current and possibly past state of the IDC and any forwarding functions and communication channels under its operation. The state repository represents the current and possibly past state of the IDC and any forwarding functions and communication channels under its operation, and the controller function 410 represents the core IDC function that controls the behavior of the forwarding functions and the communication channels under its operation in order to forward communication traffic as indicated by the information in the configuration repository. In addition, it assists in determining the capabilities, state and configuration of the forwarding functions and communication channels.
Intent Driven Controller Organization
Multiple IDCs can cooperate in a hierarchical manner to enable intent-driven networks. The element controller is responsible for one or more forwarding elements and communication channels. As shown in FIG. 5, a sector IDC is responsible for the forwarding behavior of all the element IDCs in its sector, and a region IDC is responsible for the forwarding behavior of all the forwarding elements in its subordinate region and sector IDCs. The forwarding behavior required to render a communication service is orchestrated by the unique region or sector IDC that is lowest in the IDC hierarchy and responsible for all the forwarding elements that are attached to endpoints involved in the communication service. Error! Reference source not found. shows the organization of sector and region IDCs to form a hierarchical control plane 500. Each element, sector and region IDC provides an API to create, modify and retrieve the configuration of the given intent-driven network, to retrieve the state of the given intent-driven network, and to retrieve the capabilities of the given intent-driven network.
IDCs in the controller organization can communicate with each other using forwarding functions and communication channels (illustrated in Error! Reference source not found.), either dedicated to control plane communication (out-of-band control) or shared with the data plane (in-band control).
Communication Requirements
Error! Reference source not found. below shows a plurality of communication facets that can be used to express communication requirements. It should be understood that the facets in this table is not an exhaustive listing, but can include other facets as well.

	TABLE 1

	Forwarding Precedence	Minimum Bandwidth
	Typical Bandwidth	Maximum Bandwidth
	Oversubscription	Latency
	Loss tolerance	Encryption
	Forwarding Confluence

The Forwarding Precedence facets in Table 1 represents the forwarding priority and reservation strategy for the propagation of communication traffic. The Minimum, Typical and Maximum bandwidth needed for communication traffic are three separate facets. Oversubscription is a ratio to reduce the bandwidth requested by a communication service for sharing the more limited bandwidth available on the communication equipment. Latency is the sensitivity of communication traffic to delay. Loss tolerance is the sensitivity of communication traffic to loss of data. Encryption is the security protection of communication traffic, and forwarding confluence is the provisioned redundancy of alternative forwarding paths for communication traffic. A communication facet can be assigned a communication facet value (which can be a scalar or a set of scalars).
Processing elements communicate using communication spheres. As shown in Error! Reference source not found. a communication sphere 600 is a set of endpoints (labeled 601, 602, 603 and 604) that communicate over a given connectivity pattern (communication links illustrated as lines with arrows) with specific communication services where an endpoint can represent processing elements or even external networks. The connectivity pattern in FIG. 6 connects the endpoints in the communication sphere 600 and is constructed of one or more superimposed forks. A fork is a unidirectional conduit that propagates communication traffic from one endpoint (the input segment) to N endpoints (output segments) where N is an integral number greater than zero. Error! Reference source not found. shows three forks labeled 605, 606, and 607. A communication service is a set of service profiles attached to a fork where a service profile is defined as a set of communication facets with communication facet values, and where the service profile is optionally accompanied by a traffic qualifier to narrow the applicable communication traffic. Desired communication services for a given system can be specified by a network administrator, by an application or scripts to a region, sector or element controller which will retain or distribute that information as configuration as needed to control the given intent-driven network.

Placement Algorithm

A placement algorithm 700 operates to build intent-driven networks and uses communication services to determine forwarding topologies. This methodology is referred to here as “placing a communication service”. A topography model is defined as a database that represents the connectivity among forwarding elements, processing elements and communication channels in a given system. The topography model allows us to evaluate the different options to direct communication traffic over the communication channels and forwarding elements. A resource model is defined as a database that represents the capabilities, configuration and state of the forwarding elements, processing elements, communication channels and controllers in a given intent-driven network. The resource model encompasses physical and logical properties including current and historical information. The resource model allows us to evaluate and track the rendering of communication services in the topography model.
Error! Reference source not found. shows processing steps comprising the placement algorithm 700 that can be used to determine the optimal forwarding topologies for communication traffic in the communication spheres in a given intent-driven network. At 701 one or more service tables, placement tables and provisioning tables are initialized. Then at 702 communication channels between forwarding elements are detected in a given system leveraging common detection methods. At 703 a topography model of the forwarding elements, processing elements and communication channels in the given system is composed, and at 704 a resource model of the forwarding elements, processing elements, communication channels and controllers in the given system is composed. At 705 a service table is created containing all permitted communication services in all communication spheres in the given system sorted in order of descending importance as determined by a system administrator, software program or artificial intelligence service (possibly after considering the application, subsystem or other metadata associated with the communication sphere), and at 706 a prioritized placement table is created, that lists communication services in the order they will be placed into the topography from most important to least important, by placing all communication services from the service table using an ordering algorithm described later. Then at 707, a provisioning table is created is created for each subsequent communication service in the placement table, starting with the first, as follows. The set of possible forwarding topologies is computed that fulfill or partially fulfill the communication service given the topography and resource models and select one to yield the provisional forwarding topology, possibly considering other communication services from the placement table to make a prescient choice among alternative topologies, the provisional forwarding topology is appended to the provisioning table, and the resource model is updated to record the resources that are claimed by the provisional forwarding topology. Then in 708, a forwarding topology is configured by reflecting all provisional topologies from the provisioning table into the forwarding elements and communication channels in order to configure their operation in accordance with these provisional topologies.
The above sequence or part thereof is re-evaluated every time the constituent steps of the algorithms incur a change that can affect the outcome (for example a communication channel becoming inoperative can lead to a new provisioned topology). Inversely, certain steps in the algorithm can be skipped if the outcome remains unchanged (for instance the topography model is not affected when the requested bandwidth for a communication sphere is changed).
Ordering Algorithm
Error! Reference source not found. shows the processing steps comprising an ordering algorithm 800 that operates to generate the placement table from the service table. It is primarily driving by communication facets, but other aspects can be considered during ordering, for instance the application, subsystem, endpoint or communication sphere that is involved in the communication traffic.
We define a communication facet clause as a comparison of a communication facet against a given communication facet value where the comparison operator is any one of smaller than (<), smaller than or equal to (≦) equal to (=), not equal to (≠), greater than (>), or greater than or equal to (≧). The comparison operator enables communication facet values to be ordered by value and thus by implied importance. Note that communication facet values may be represented by a set of one or more scalars and that the comparison operator may perform comparisons for each scalar in the set.
We define a placement clause as a simple or compound Boolean expression that resolves to true (or false) and that determines whether a service is to be placed (or not placed) in the placement table.
We define a placement matrix with R rows and C columns where each column represents a communication facet or placement condition and where each row represents a set of communication facet clauses or placement clauses each corresponding to a communication facet or placement condition in a column of the placement matrix. An administrator is allowed to modify the columns and the rows of the placement matrix or change the communication facet clauses or placement clauses they contain.
The ordering algorithm 800 can take entries from the service table and place them into the placement table in an order that is determined by the placement matrix as follows.

- 801. Take each subsequent entry “s” from the service table starting with the first entry.
- 802. Take each subsequent row “r” from the placement matrix starting with the first row.
- 803. Evaluate any communication facet clauses listed in row “r” of the placement matrix using the corresponding communication facet values of entry “s” and if any comparison yields false then the facet evaluation result for row “r” is false.
- 804. If the facet evaluation result for row “r” is false, then we perform the evaluation with the next row in the placement matrix.
- 805. Evaluate any placement clauses listed in row “r” of the placement matrix possibly using metadata of entry “s” and if any Boolean expression resolves to false then the placement evaluation result for row r is false.
- 806. If the facet evaluation result for row “r” is true but the placement evaluation result for row “r” is false, then the placement algorithm proceeds with next entry from the service table.
- 807. If both the facet evaluation result and the placement evaluation result for row “r” are true, then entry “s” is assigned a placement index “p” equal to the index of row “r” of the placement matrix and entry “s” is inserted into the placement table immediately after any previously placed service entries with the same placement index. The placement algorithm then proceeds with the next entry from the service table.
- 808. If the evaluation result is false for all rows in the placement table and a remnant placement index is configured for the given system, then entry “s” is placed into the placement table with an effective placement index equal to the remnant placement index immediately after any previously placed service entries with the remnant placement index. The placement algorithm then proceeds with the next entry in the service table.

The impact of the placement matrix is to allow the administrator to determine the order in which communication services are placed in the topography which in turn can lead to different forwarding topologies. This approach forms the basis for the administrator to affect the behavior of the forwarding elements based on high-level intent directives.
New placement aspects can be added as columns to the placement matrix and rows can be inserted or removed to provide different comparison rules. Service table entries can be expanded to specify these new placement aspects so they can be similarly processed by the placement and ordering algorithms.
Placement Matrix Examples
If the placement matrix is empty, the resulting placement table will have all the communication services listed in the same order as the service table (depending on the configuration of any remnant placement index).
If the placement matrix contains a single row with a communication facet clause “smaller than 10 milliseconds” in the column for the latency communication facet, then all communication services that request a latency smaller than 10 milliseconds will be placed at the top of the placement table—in the same order as encountered in the service table—followed by the rest of the service table entries.
If we prepend a placement matrix row with the same “smaller than 10 milliseconds” communication facet clause and a “very high” communication facet clause for the forwarding precedence, the net effect on the placement table is that all communication services that meet both these requirements will now be listed first.
An example of a new placement aspect is a customer priority field that can be set to “high”, “normal” or “low”. Each service table entry would contain the value of this field (or use a default) while the placement matrix could contain a first row with a clause to match service table entries marked as “high” causing such entries to appear first in the placement table and, as a result, they are afforded better placement and forwarding topologies.
Distributed Computation
The IDCs in the controller hierarchy can cooperate to execute the placement algorithm in a single IDC or using multiple IDCs. The desired communication services in a communication sphere in a given intent-driven network involve endpoints that are attached to forwarding elements under the operational control of one or more IDCs. This set of IDCs is ultimately subordinate to a single IDC (probably a region IDC, but possibly a sector or element IDC) which can delegate the execution of the placement algorithm to its immediate subordinate IDCs as illustrated in Error! Reference source not found. According to FIG. 9, each hexagon represents a sector (numbered 1 a through 6 c) and is assumed to contain forwarding elements (not explicitly shown). Adjacent sectors having the same number form a region (numbered 1 through 6). Four endpoints are also shown (presumably attached to a forwarding element in the sector with a bold edge). If a communication service for endpoints 3 and 4 needs to be placed, it can be resolved by the region controller for the blue sectors (numbered 4 a through 4 c). If a communication service needs to be placed for endpoints 1 and 2 a choice needs to be made whether the associated communication traffic runs through the region 3 (sectors 3 a, 3 b, 3 c) or region 4 (sectors 4 a, 4 b, 4 c) and this decision is made by a superior region IDC (covering regions 2, 3, 4 and 5). If it selects the region 3 (sectors 3 a, 3 b, 3 c), it can delegate the placement for communication services inside these regions to region IDC 2, 3 and 5 which in turn can delegate the placement for communication services inside the respective sectors to the associated sector IDCs.
Asymmetrical Forwarding Example
Error! Reference source not found. shows an example of a communication sphere with seven endpoints (labeled 1-7), a first fork that forwards data from endpoint 1 to the endpoints labeled 2-7 (with multiple output segments), and a set of six forks forwarding data in the opposite direction. The communication service for the first fork could specify a forwarding precedence (communication facet) as “dedicated” (communication facet value) to indicate that a communication path is to be reserved for exclusive use. It could also request a very low setting for forwarding latency (another communication facet). If the forwarding elements in the communication sphere are capable of supporting physical layer switching (for instance by means of an optical or electro-optical crossbar), it could result in a very low-latency forwarding topology by replicating bit patterns instead of having to look up frame headers to perform forwarding. The set of six forks indicate that their communication traffic needs to be merged towards endpoint 1 and this function cannot be performed at the physical layer, so some datalink layer switching or routing function is required in that communication path.
The configuration for the communication sphere in FIG. 10 can be applied to a single forwarding element and operate a properly equipped device to distribute traffic from endpoint 1 to the other endpoints with very low latency while the traffic in the opposite direction can use a conventional layer 2 switch that floods all traffic from endpoints 2 through 7 towards endpoint 1. Such a device exhibits asymmetrical forwarding behavior that benefits fast distribution of traffic streams (multicast traffic being one application).
Intent Driven Networking
FIG. 11 shows how mission critical applications (labeled 1-3) can leverage intent to interact with the network infrastructure (labeled 6-8). The waistline (labeled 4 and 5) in the center is where the two worlds meet at an intent boundary taking expressions of intent (using an Intent Application Programming Interface labeled as 4) and rendering it into instructions to the network (using an Intent Engine labeled as 5). The waistline illustrates how a narrow but universal set of intent definitions—such as communication spheres, facets and profiles can connect the otherwise broad and diverse worlds of software applications and networking equipment.
The forgoing describes how intent-driven networks can be operated to adapt to the performance needs of application workloads and thus lay the foundation for building agile networks. As the demand for network bandwidth grows faster than the capacity gains in the underlying hardware technology can accommodate, we want to rely less on static, one-size-fits-all networking like the current Internet, and rely more on dynamic networks that can differentiate among the service needs of different traffic flows. Instead of requiring human intervention to recalibrate networks when it is too late, intent-driven networks allow applications to express their performance needs—in real time—using the previously described communication spheres, facets and profiles so networks can deliver a better quality of experience to the end user. For example, real-time media flows used in video-conferencing have very stringent network delivery requirements and even automated detection methods, like deep-packet inspection, are increasingly ineffective given the ever-widening adoption of traffic encryption. In one introductory application, intent-driven networks can be used to determine the possible ways these media flows could be routed through the available networks. It can then compute a set of choke points where flow-based policies are applied to mark these traffic flows and thereby allow conventional forwarding equipment to identify and expedite these workloads.
Another benefit of intent-driven networks is in the optimization of computationally intensive applications such as data analytics. In current hierarchical networks, there are limited alternative routes available between clustered compute servers, but advances in hardware integration are embedding networking functions into general purpose processors, allowing servers to be directly interconnected with multiple channels to multiple neighbors. The resulting path diversity allows parallel communication which is not easily be leveraged by today's networking software. But intent-driven networks can compute how to optimize aggregate performance based on intent definitions, and can even consider how to avoid exhaustion of the limited buffering space found in today's network switches that are forced to dedicate silicon resources to increased port density.
In yet another application, virtual networks can be layered over intent-driven networks and leverage computation to avoid hotspots in the physical network that are so common yet hard to remedy with today's networking solutions.
The forgoing description, for purposes of explanation, uses specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practice the invention. Thus, the forgoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed; obviously, many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, they thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.

Claims

I claim:

1. A method for provisioning a network communication domain, comprising:

defining the network communication domain to have a plurality of network forwarding elements;

detecting, by an intent driven controller, one or more capabilities associated with the plurality of the network forwarding elements;

specifying communication service requirements for each one of a plurality of computer applications running in the network, and assigning an importance to the data traffic associated with each application;

selecting the highest importance computer application and provisionally determining a path through the network forwarding elements for the data traffic from the selected application, the provisionally determined network path satisfying the communication service requirements; and

configuring one or more of the network forwarding elements to permit the application data traffic to be forwarded through the network according to the provisionally determined network path.

2. The method of claim 1, wherein the communication facets of the communication service requirements are any one or more of a forwarding precedence, a minimum bandwidth, typical bandwidth, maximum bandwidth, an oversubscription, a latency, a loss tolerance, an encryption, and forwarding confluence.

3. The method of claim 1, further comprising recursively selecting a next highest importance computer application until all computer application are provisionally determined.

4. The method of claim 1, wherein the network forwarding elements operate in ISO layers 1, 2, 3 and 4.

5. The method of claim 1, wherein the network is a physical network, a virtual network or a combination of a physical and a virtual network.

6. The method of claim 1, wherein the intent driven controller comprises a placement algorithm that is initialized by a network administrator or automatically based upon measurable or observable network characteristics.

5. The method of claim 1, where in the specification of communication services can be expressed by a network administrator, a software application or a script representing a class of software applications, and provided to a region, sector or element controller.