CN117424921A - Distributed control system and control method oriented to large-scale deterministic network - Google Patents
Distributed control system and control method oriented to large-scale deterministic network Download PDFInfo
- Publication number
- CN117424921A CN117424921A CN202311256906.5A CN202311256906A CN117424921A CN 117424921 A CN117424921 A CN 117424921A CN 202311256906 A CN202311256906 A CN 202311256906A CN 117424921 A CN117424921 A CN 117424921A
- Authority
- CN
- China
- Prior art keywords
- domain
- deterministic
- network
- det
- forwarding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 61
- 238000007726 management method Methods 0.000 claims abstract description 107
- 238000012384 transportation and delivery Methods 0.000 claims abstract description 6
- 238000013468 resource allocation Methods 0.000 claims abstract description 4
- 238000004364 calculation method Methods 0.000 claims description 43
- 230000008569 process Effects 0.000 claims description 29
- 230000006854 communication Effects 0.000 claims description 28
- 238000004891 communication Methods 0.000 claims description 26
- 230000005540 biological transmission Effects 0.000 claims description 19
- 238000005516 engineering process Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 17
- 230000003993 interaction Effects 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 13
- 230000001934 delay Effects 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 7
- 238000013439 planning Methods 0.000 claims description 3
- 238000007493 shaping process Methods 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 claims description 2
- 230000008859 change Effects 0.000 claims description 2
- 238000005538 encapsulation Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 238000005192 partition Methods 0.000 claims 1
- 238000000638 solvent extraction Methods 0.000 claims 1
- 230000007246 mechanism Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000006855 networking Effects 0.000 description 4
- 230000000737 periodic effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/02—Topology update or discovery
- H04L45/04—Interdomain routing, e.g. hierarchical routing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/12—Shortest path evaluation
- H04L45/121—Shortest path evaluation by minimising delays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/302—Route determination based on requested QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a distributed control system for a large-scale deterministic network, which comprises a Det-AS, domains, a Det-AS controller and a deterministic service manager, wherein the large-scale deterministic network is divided into a plurality of independent Det-AS according to different network requirements and physical characteristics, each domain is abstracted into a node with a domain time delay attribute when the deterministic forwarding is carried out among the domains, route reachability information is interacted with adjacent domain nodes through a route management protocol, and then an optimal path is calculated according to a route selection strategy to carry out deterministic data forwarding so AS to minimize the data forwarding time delay among the domains; each Det-AS is equipped with a controller for path computation, resource allocation, protocol delivery, information collection, etc. in the domain, and the deterministic service manager is configured to receive and collect user requirements of user equipment and requests of application programs of the access network side, take charge of allocation and management of deterministic services, and provide network service allocation requirements between end-to-end application systems.
Description
Technical Field
The invention relates to a distributed control method for a large-scale deterministic network, in particular to a distributed control system for the large-scale deterministic network and a cross-domain forwarding method for end-to-end deterministic service flow, belonging to the technical fields of Internet protocol and Internet.
Background
With the rise of deterministic business such as unmanned, telemedicine and smart grid and the rapid development of industry 4.0, the demand for network communication is further increased. The conventional "best effort" data transmission manner of the ethernet has failed to meet transmission requirements such as bounded time delay, zero jitter, and accurate time synchronization of deterministic traffic. To address the timeliness and accuracy requirements of these emerging real-time and time-sensitive applications, the international standardization organization internet engineering task force (Internet Engineering Task Force, IETF) has proposed some protocols and standards aimed at providing reliable and deterministic guaranteed network transmissions for these applications. However, the prior art is not sufficient to guarantee large scale deterministic network end-to-end low latency data forwarding with time sensitive applications. Meanwhile, in order to implement forwarding of deterministic traffic, some candidate techniques based on periodic forwarding are proposed by the workgroup, such as periodic queuing and forwarding (Cycle Queuing and Forwarding, CQF), periodic specified queuing and forwarding (Cycle Specified Queuing and Forwarding, CSQF), and the like. However, how to implement end-to-end low latency data forwarding in an open large scale deterministic network is an important issue to be addressed.
Currently, as the industry internet advances and deterministic network standards evolve, deterministic network technology transitions from local area networks (Local Area Network, LANs) to wide area networks (Wide Area Network, WANs) to large scale deterministic networks across WANs. For Deterministic traffic with strict latency, jitter, bandwidth, and packet loss requirements, the key technologies of existing Deterministic networks mainly include Time-sensitive network (Time-Sensitive Networking, TSN), deterministic network (Deterministic Networking, detNet), and Deterministic IP technology (DIP).
The TSN was developed by the Audio video bridging (Audio Video Bridging, AVB) task group of the institute of Electrical and electronics Engineers (Institute of Electrical and Electronics Engineers, IEEE) 802.1 standards Committee, and further developed into the TSN task group based thereon. The TSN task group expands the application field from the professional audio and video field to industrial scenes, vehicle-mounted networks, mobile communication and the like, and establishes a series of standards based on specific application requirements. These standards cover time synchronization, high reliability, deterministic latency, and resource management. The TSN establishes a universal time-sensitive mechanism for the ethernet protocol to achieve deterministic transmission of the LAN, suitable for real-time applications within the small network. However, the mechanisms in TSNs have severe limitations on latency and require high precision clock synchronization, which makes these time-sensitive mechanisms inadequate to support deterministic networking scenarios in WANs. With the increasing demand for deterministic networks, the range of applications for deterministic networks is now increasingly beyond the scope of LANs.
DetNet is a new type of networking technology developed by the IETF DetNet working group, which extends the relevant protocol standards set forth by the TSN task force. In order to meet the requirement of the WAN on deterministic performance, the DetNet integrates the Internet technology (Internet Technology, IT) and the operation technology (Operational Technology, OT) and has the capabilities of the layer 2 and the layer 3, and aims to solve the problem of realizing deterministic transmission paths on the layer 2 bridging and the layer 3 routing segments, thereby providing extremely low packet loss rate, end-to-end transmission delay and the like for specific real-time application. The DetNet working group extends deterministic networks over WANs through internet protocol (Internet Protocol, IP) and multiprotocol label switching (Multiprotocol Label Switching, MPLS) technologies to achieve a wider range of deterministic transmissions. However, the work group currently mainly researches IP networks under single or closed management control, which has a disadvantage in that the IP networks under single or closed management control are insufficient to support solving the complex network problems. Thus, there are still many challenges in implementing end-to-end low latency data forwarding in a large scale deterministic network.
DIP technology is a novel 3-layer deterministic network technology architecture focusing on the data forwarding plane and employing frequency synchronization mechanisms to provide low latency, low packet delay variation (jitter) and high reliability boundaries. Deterministic network techniques focus mainly on mechanisms that provide reliability guarantees at layers 2 and 3, without adequately considering the control of forwarding delays. DetNet combines the routing protocol, realizes deterministic forwarding at the routing level, expands deterministic networks to WAN, but also increases the difficulty of large-scale network deployment. Therefore, the DIP technology introduces a periodic scheduling mechanism to carry out innovative breakthrough of the forwarding technology, and realizes large-scale deterministic transmission through period and queue mapping. However, this approach still requires static network topology and flow setup and cannot effectively solve the problems of device and traffic micro-bursts, etc. existing in the emerging network, resulting in difficulty in achieving deterministic transmission. Thus, DIP techniques are still not well-suited for use in large scale deterministic networks without a priori dynamic environmental information. Currently, in large-scale, long-distance networks, research on deterministic transmission is also facing a number of issues that need to be addressed.
For an open network environment such as a large-scale deterministic network, it becomes very difficult to obtain global information of the network, and how to optimize the end-to-end deterministic data forwarding delay is still very challenging. The above-mentioned deterministic network techniques do not take into account the targeted improvements, and mainly have the following problems: 1) In large scale deterministic networks, the occurrence of traffic bursts and congestion, etc., is unavoidable and unpredictable. In the face of network characteristics such as open networks, complex and flexible topologies, and long-distance link transmission, the prior art cannot support stable and reliable data forwarding, which may result in end-to-end network delay and jitter being uncontrollable within a bounded range. Meanwhile, in the process of formulating deterministic network technical standards, the prior art faces a plurality of challenges in ensuring timeliness and certainty of data transmission. Therefore, it is difficult to meet the requirements of new technologies and new application scenarios for deterministic traffic. 2) Currently, existing deterministic network technologies are primarily directed to single or closed managed networks. However, most deterministic traffic requires data forwarding in a cross-domain environment, which has not been able to meet the global deterministic traffic demands of open large-scale networks. In addition, since the conventional open network managed by a single controller has a problem that the controller is overloaded, it is difficult to meet the performance requirements of a large-scale network. Therefore, the manner in which a single controller manages is not applicable to large scale deterministic networks. For how to realize end-to-end low-delay data forwarding of an open large-scale deterministic network, no targeted solution is provided in the prior art. Therefore, a new solution is urgently needed to solve the above technical problems.
Disclosure of Invention
The invention aims at the problems in the prior art, provides a distributed control method for a large-scale deterministic network, and aims to solve the problem of realizing the end-to-end low-delay data forwarding control in the open large-scale deterministic network.
In order to solve the problems, the invention provides a divide-and-conquer idea for managing a large-scale deterministic network, and the current large-scale deterministic network is subjected to distributed processing by designing a distributed control system and a distributed control method for the large-scale deterministic network. Specifically, the invention divides the large-scale deterministic network into a plurality of deterministic autonomous domains (Deterministic Autonomous System, det-AS) for management according to different network demands, physical characteristics and the like, and realizes the cross-domain deterministic service flow forwarding of the open network by designing a routing strategy among the domains. The invention aims to realize the optimal routing in the Det-AS and among domains so AS to minimize the data forwarding delay of the global network from the end to the end of the open network, thereby realizing more efficient and reliable data processing and management.
Each Det-AS includes a plurality of network devices and physical links in its domain, and deploys a controller for traffic management, routing, and forwarding in the domain to minimize latency in the domain. Each Det-AS follows an end-to-end global deterministic service constraint, e.g. latency requirements, the latency in the domain must be smaller than the global latency constraint. In the Det-AS, the network device only communicates with other network devices in the same domain, but only ER with special functions can perform cross-domain communication, and multiple ERs can exist in each domain, so that cross-domain deterministic traffic forwarding can be performed through multiple adjacent Det-AS.
Deterministic forwarding among Det-AS abstracts each Det-AS into a special node with intra-domain delay attribute, and achieves cross-domain deterministic inter-domain communication by adopting a routing management protocol and a routing strategy. First, ER in the domain establishes a neighbor relation with neighboring domain nodes through a route management protocol and exchanges route reachability information. Meanwhile, each domain node may learn route reachability information to other domain nodes through the ER. Then, ER calculates an optimal route forwarding path from the route reachability information according to a certain route selection policy, and performs deterministic forwarding of the data packet according to the optimal path, so as to minimize inter-domain delay.
In the distributed control system, a controller is responsible for arrangement of data forwarding plane resources, maintenance of network topology and state information and the like by controlling a southbound interface; and providing a service northbound interface to the application plane for receiving requests and instructions from the application; the east-west interfaces among the controllers can know the network states and the network congestion degrees of other domains so as to perform cooperation and coordination among the distributed controllers, thereby realizing the cross-domain data forwarding of the whole open large-scale deterministic network.
The technical proposal of the invention is AS follows, a distributed control system facing a large-scale deterministic network, the control system comprises a Det-AS, namely a deterministic autonomous domain (Deterministic Autonomous System, det-AS), inter-domains, a Det-AS controller and a deterministic service manager,
wherein the Det-AS refers to a plurality of independent and deterministic network areas divided in an open wide area deterministic network, each Det-AS is equipped with a controller and several network devices, and at the same time, the controller manages and controls each Det-AS for realizing traffic management and deterministic data forwarding in the domain,
when deterministic forwarding is carried out among domains, abstracting each domain into a node with an intra-domain time delay attribute, interacting route reachability information with adjacent domain nodes through a route management protocol, and then calculating an optimal path according to a route selection strategy to carry out deterministic data forwarding so as to minimize the data forwarding time delay among domains;
Det-AS controllers, wherein each Det-AS is equipped with a controller for path computation, resource allocation, protocol delivery, information collection, etc. within a domain, or cooperates with other controllers, exchange information,
the deterministic service manager is used for collecting the user requirements of the user equipment at the access network side and the requests of the application programs, acquiring the network information at the network side and providing network capability for the user according to the requirements. Is mainly responsible for the configuration and management of deterministic user equipments and network resources,
the interface relation among the components of the control system is as follows:
the network equipment in each domain in the data forwarding plane uploads the information of network state, links, nodes and the like in the domain to the controller of the control plane through controlling the southbound interface, and meanwhile, each controller receives the requirements of various deterministic service flows from the application plane through the service northbound interface and issues network configuration, protocols, routing calculation results and the like through controlling the southbound interface. The interaction between the controllers is correspondingly cooperated and coordinated through the east-west interfaces so as to facilitate the information synchronization and the information interaction of each domain, thereby better making the decision and management of the cross-domain data forwarding.
The control system also comprises ER and FR, wherein the ER is used for connecting adjacent Det-AS and other external networks and is mainly responsible for deterministic data forwarding and routing between domains, and deterministic service flows are transmitted from user equipment at a transmitting end to the domain where the user equipment at a receiving end is located through ER of a plurality of domains by a routing management protocol issued by a controller and a routing strategy adopted between the domains; the FR is a key network device in the route forwarding process, and is used for forwarding deterministic traffic flows entering the domain according to a preset routing rule and a route calculation result of each domain, and finally sending the deterministic traffic flows to the ER of each domain until the terminal device receives the traffic flows.
A network architecture of a distributed control system oriented to a large-scale deterministic network is realized, the network architecture comprises an application plane, a control plane and a data forwarding plane, the planes are mutually separated and interacted through an provided open interface,
the application plane is an upper application layer in the network architecture, and mainly comprises a deterministic service manager and deterministic service applications with different QoS requirements, wherein the deterministic service applications are applications with strict time requirements, such as unmanned, smart home, smart factories and the like. They perform configuration and management of deterministic user equipments through a deterministic traffic manager, which includes acquisition and configuration delivery of the requirements of deterministic traffic, quality of service requirements, etc. The application plane is mainly oriented to deterministic service applications, and requests and instructions from application programs are received through an API protocol. Meanwhile, the service north interface is used for transmitting the requirements of deterministic service flows, routing strategies and the like to the controller of the control plane for interaction, and network states and statistical information are acquired from the control plane so as to allocate network resources.
The control plane mainly comprises one or more controllers, and is different from other network architectures, the invention proposes that each Det-AS corresponds to one controller, meanwhile, the controller has the functions of calculation, management, interaction and the like, is mainly responsible for network configuration, the network configuration comprises configuration and management of basic parameters in a network, such AS a flow table, bandwidth limitation of a router port, link state information and the like, is mainly oriented to network infrastructure in a domain, and configures and manages network equipment in each domain through related protocols such AS a network configuration protocol (Network Configuration Protocol, netconf) and the like. Meanwhile, the controllers transmit network state information such as time delay and the like to the interfaces through the east and west, and perform information interaction with other adjacent controllers so as to meet the requirement of interoperability among the controllers,
the data forwarding plane is connected to the control plane through a control southbound interface and bears the actual forwarding and processing tasks of various deterministic services in a wide area deterministic network, the plane is composed of a plurality of Det-AS, each Det-AS is composed of network equipment elements such AS ER and FR, meanwhile, the plane carries out deterministic forwarding on service flows according to routing rules and strategies which are preset by a controller, and in an open wide area deterministic network environment, the deterministic service flows are forwarded through user equipment at a transmitting end, pass through the plurality of Det-AS and a plurality of forwarding paths and are finally transmitted to the user equipment at a receiving end.
A distributed control method for a large-scale deterministic network, the method comprising the steps of:
step 1: and the user equipment of the transmitting end reports the requirement of the deterministic service to the deterministic service manager through an API protocol for user configuration. The Det-AS controller transmits deterministic service from the user side to the network side through a UNI protocol so AS to carry out network management;
step 2: the network side divides an open large-scale deterministic network into a plurality of independent Det-AS by adopting a distributed control network architecture, abstracts the Det-AS into domain nodes with time delay attributes, and carries out cross-domain data forwarding by constructing the domain nodes;
step 3: in the Det-AS, each domain is configured and scheduled by a controller, the controller carries out route calculation according to network topology information, flow demand, performance index and the like in the domain to obtain an optimal route result in the domain, and the calculation result is issued to the network equipment in the domain for data forwarding so AS to realize the minimization of time delay in the domain;
step 4: when the Det-AS communicates, each domain is regarded AS a node with an intra-domain time delay attribute, the controller transmits a route management protocol to ER in each domain, the ER acquires route reachability information of nodes in other domains through the route management protocol, and performs optimal path selection among domains according to a route selection strategy, and the ER performs deterministic data forwarding according to the optimal inter-domain route path so AS to realize minimization of inter-domain time delay;
Step 5: through the steps, the intra-domain time delay and the inter-domain time delay are calculated respectively, wherein the intra-domain time delay is the delay caused when data forwarding is carried out in the same domain; inter-domain delay is the delay in forwarding data when crossing different domains. The invention sums the intra-domain and inter-domain delays obtained by calculation, and can obtain the limited and smaller large-scale deterministic network end-to-end data forwarding delay.
The scheme is oriented to a distributed control system and a method of a large-scale deterministic network: and carrying out distributed processing on the current large-scale deterministic network, and realizing cross-domain forwarding of deterministic service flows of the open network. The scheme adopts the interface relation of each component part of the distributed control system: the network equipment of the data forwarding plane and the deterministic autonomous domain manager of the control plane interact signaling through controlling the southbound interface, the controllers interact information through the southbound interface, and the deterministic service manager of the application plane interacts deterministic service demands with the control plane through serving the northbound interface. In the scheme, a network architecture of an open wide area deterministic network is oriented: the large-scale deterministic network is divided into a plurality of Det-AS according to different network requirements, and deployment and scheduling are performed by respective controllers. The network architecture includes an application plane, a control plane, and a data forwarding plane, which are separated from each other and interact through an open interface provided. In the above scheme, the deterministic service manager: and collecting user requirements of user equipment and application program requests of the access network side, acquiring network information of the network side, and providing network capability for the user according to the requirements. Deterministic autonomous domain controller: and collecting information such AS network state and issuing configuration and management of network parameters in each Det-AS so AS to realize operations such AS route calculation, deterministic forwarding and the like. Deterministic autonomous domain: the open wide area deterministic network is divided into a plurality of independent and deterministic network areas. Each deterministic autonomous domain is controlled and managed by a controller.
Data forwarding in deterministic autonomous domain: and carrying out deterministic intra-domain data forwarding according to the intra-domain flow table issued by the controller. Including flow management, mapping, scheduling, and deterministic forwarding. Deterministic autonomous inter-domain data forwarding: and obtaining an intra-domain optimal route and corresponding time delay according to operations such as route calculation and the like of the controller, and abstracting the intra-domain time delay into a multi-degree domain node with time delay attributes. And carrying out deterministic data forwarding among the domain nodes through a routing table among the domains.
Compared with the prior art, the invention has the following advantages: (1) The invention divides the current large-scale deterministic network into a plurality of deterministic self-treatment domains according to different network demands, physical characteristics and the like to carry out distributed processing by designing a distributed control system and a method for the large-scale deterministic network, and realizes the forwarding of the end-to-end deterministic service flow of the open network by designing routing strategies in and among domains; (2) The invention provides a novel network architecture for an open wide area deterministic network, which comprises an application plane (deterministic service manager), a control plane (deterministic autonomous domain controller) and a data forwarding plane (the open wide area deterministic network comprises intra-domain forwarding and inter-domain forwarding), wherein the controller and the deterministic autonomous domain have a one-to-one correspondence relationship, the wide area deterministic network is divided into a plurality of deterministic autonomous domains, (3) an interaction mode between planes, the application plane sends deterministic demands to the control plane through a northbound service interface, the control plane carries out deterministic configuration and management on network equipment of the data forwarding plane through a southbound control interface, (4) a topology construction scheme of domain nodes is designed, the invention realizes cross-domain data forwarding through a three-layer topology architecture, the three-layer topology architecture comprises an original open network topology, a deterministic autonomous domain topology and a domain node topology with attributes, (5) each deterministic autonomous domain abstracts out a scheme of domain nodes with time delay attributes, a plurality of time delays of each deterministic autonomous domain are obtained according to different deterministic service demands, an intra-domain network topology structure, routing paths and the like, (6) the data forwarding scheme among domains carries out deterministic service demands in a user domain or a user domain data forwarding function through a user domain controller, then, the requirements of the user side are forwarded to the network side through a deterministic service manager, so that the details of network internal propagation and network configuration are avoided; (8) The function module and function of the deterministic autonomous domain controller. The deterministic autonomous domain controller comprises two functions of scheduling and management, and acts in the deterministic autonomous domain which is responsible for the deterministic autonomous domain. The distributed controllers can solve the problem of overload in a single controller management mode in a large-scale deterministic network.
Drawings
Fig. 1 is a schematic diagram of a network architecture of an open wide area deterministic network according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a topology construction process of a domain node according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of forwarding data requests within a deterministic autonomous domain according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a deterministic self-healing inter-domain data forwarding process according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of end-to-end cross-domain communication setup for a large scale deterministic network according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a deterministic user configuration and network management related operation flow according to an embodiment of the present invention.
Detailed Description
In order to enhance the understanding of the present invention, the present embodiment will be described in detail with reference to the accompanying drawings.
Example 1: referring to fig. 1, a distributed control system for a large scale deterministic network, the control system comprising a Det-AS, inter-domain, det-AS controller and deterministic traffic manager,
wherein the Det-AS refers to a plurality of independent and deterministic network areas divided in an open wide area deterministic network, each Det-AS is equipped with a controller and several network devices, and at the same time, the controller manages and controls each Det-AS for realizing traffic management and deterministic data forwarding in the domain,
When deterministic forwarding is carried out among domains, abstracting each domain into a node with an intra-domain time delay attribute, interacting route reachability information with adjacent domain nodes through a route management protocol, and then calculating an optimal path according to a route selection strategy to carry out deterministic data forwarding so as to minimize the data forwarding time delay among domains;
Det-AS controllers, wherein each Det-AS is equipped with a controller for path computation, resource allocation, protocol delivery, information collection, etc. within a domain, or cooperates with other controllers, exchange information,
the deterministic service manager is used for receiving and collecting the user requirements of the user equipment at the access network side and the requests of the application programs, is responsible for the configuration and management of deterministic services, and provides the network service configuration requirements between the end-to-end application systems,
the interface relation among the components of the control system is as follows:
the network equipment in each domain in the data forwarding plane uploads the information of network state, links, nodes and the like in the domain to the controller of the control plane through controlling the southbound interface, and meanwhile, each controller receives the requirements of various deterministic service flows from the application plane through the service northbound interface and issues network configuration, protocols, routing calculation results and the like through controlling the southbound interface. The interaction between the controllers is correspondingly cooperated and coordinated through the east-west interfaces so as to facilitate the information synchronization and the information interaction of each domain, thereby better making the decision and management of the cross-domain data forwarding.
The control system also comprises ER and FR, wherein the ER is used for connecting adjacent Det-AS and other external networks and is mainly responsible for deterministic data forwarding and routing between domains, and deterministic service flows are transmitted from user equipment at a transmitting end to the domain where the user equipment at a receiving end is located through ER of a plurality of domains by a routing management protocol issued by a controller and a routing strategy adopted between the domains; the FR is a key network device in the route forwarding process, and is used for forwarding deterministic traffic flows entering the domain according to a preset routing rule and a route calculation result of each domain, and finally sending the deterministic traffic flows to the ER of each domain until the terminal device receives the traffic flows.
Example 2: referring to fig. 1, a network architecture of a distributed control system for a large-scale deterministic network is implemented, the network architecture including an application plane, a control plane, and a data forwarding plane, the planes being separated from each other and interacting through an open interface provided,
the application plane is an upper application layer in the network architecture, and mainly comprises a deterministic service manager and deterministic service applications with different QoS, wherein the deterministic service applications are applications with strict time requirements, such as unmanned, smart home, smart factories and the like. They perform configuration and management of deterministic user equipments through a deterministic traffic manager, which includes acquisition and configuration delivery of the requirements of deterministic traffic, quality of service requirements, etc. The application plane is mainly oriented to deterministic service applications, and requests and instructions from application programs are received through an API protocol. Meanwhile, the service north interface is used for transmitting the requirements of deterministic service flows, routing strategies and the like to the controller of the control plane for interaction, and network states and statistical information are acquired from the control plane so as to allocate network resources.
The control plane mainly comprises one or more controllers, and is different from other network architectures, the invention proposes that each Det-AS corresponds to one controller, meanwhile, the controller has the functions of calculation, management, interaction and the like, is mainly responsible for network configuration, the network configuration comprises the configuration and management of basic parameters in a network, such AS a flow table, bandwidth limitation of a router port, link state information and the like, is mainly oriented to network infrastructures in domains, and configures and manages network devices in respective domains through relevant protocols such AS Netconf and the like. Meanwhile, the controllers transmit network state information such as time delay and the like to the interfaces through the east and west, and perform information interaction with other adjacent controllers so as to meet the requirement of interoperability among the controllers,
the data forwarding plane is connected to the control plane through a control southbound interface and bears the actual forwarding and processing tasks of various deterministic services in a wide area deterministic network, the plane is composed of a plurality of Det-AS, each Det-AS is composed of network equipment elements such AS ER and FR, meanwhile, the plane carries out deterministic forwarding on service flows according to routing rules and strategies which are preset by a controller, and in an open wide area deterministic network environment, the deterministic service flows are forwarded through user equipment at a transmitting end, pass through the plurality of Det-AS and a plurality of forwarding paths and are finally transmitted to the user equipment at a receiving end.
Example 3: referring to fig. 1-6, a distributed control method for a large scale deterministic network, the method comprising the steps of:
step 1: and the user equipment of the transmitting end reports the requirement of the deterministic service to the deterministic service manager through an API protocol for user configuration. The Det-AS controller transmits deterministic service from the user side to the network side through a UNI protocol so AS to carry out network management;
step 2: the network side divides an open large-scale deterministic network into a plurality of independent Det-AS by adopting a distributed control network architecture, abstracts the Det-AS into domain nodes with time delay attributes, and carries out cross-domain data forwarding by constructing the domain nodes;
step 3: in the Det-AS, each domain is configured and scheduled by a controller, the controller carries out route calculation according to network topology information, flow demand, performance index and the like in the domain to obtain an optimal route result in the domain, and the calculation result is issued to the network equipment in the domain for data forwarding so AS to realize the minimization of time delay in the domain;
step 4: when the Det-AS communicates, each domain is regarded AS a node with an intra-domain time delay attribute, the controller transmits a route management protocol to ER in each domain, the ER acquires route reachability information of nodes in other domains through the route management protocol, and performs optimal path selection among domains according to a route selection strategy, and the ER performs deterministic data forwarding according to the optimal inter-domain route path so AS to realize minimization of inter-domain time delay;
Step 5: through the steps, the intra-domain time delay and the inter-domain time delay are calculated respectively, wherein the intra-domain time delay is the delay caused when data forwarding is carried out in the same domain; inter-domain delay is the delay in forwarding data when crossing different domains. The invention sums the intra-domain and inter-domain delays obtained by calculation, and can obtain the limited and smaller large-scale deterministic network end-to-end data forwarding delay.
The process implemented by each step is described in detail with reference to the accompanying drawings, and is as follows:
referring to fig. 6, the deterministic traffic manager and Det-AS controller, AS well AS their specific functional modules and collaborative workflow, included in step 1 are described.
The deterministic traffic manager includes a user configuration module. The user configuration module is mainly divided into the requirement uploading of deterministic service and the configuration issuing of deterministic function. The user configuration module is a connection established with the Det-AS controller temporarily when initiating service communication, and is responsible for issuing the acquired deterministic service demand to the controller in the control plane through the UNI protocol.
The functional modules of the Det-AS controller include a network management module within the domain. The network management module in the domain mainly comprises two parts of management and scheduling. The management functions comprise resource management, topology management and traffic flow management, and the scheduling mainly comprises route calculation and data scheduling. The intra-domain network management module is responsible for processing the service flow entering the domain, and the basic information such AS network topology in each Det-AS is stored for a long time, but the service flow is required to be processed according to the requirements of different user equipment. The Det-AS controller issues different configuration instructions to the ER and FR. Specifically, the Det-AS controller issues a forwarding flow table to the FR for data forwarding within the domain. Meanwhile, the Det-AS controller issues a route management protocol and routing results in each Det-AS to the ER for establishing inter-domain links and forwarding deterministic data. In addition, the Det-AS controllers perform cross-domain cooperation and coordination through exchanging route reachability information so AS to ensure the real-time performance and accuracy of service stream transmission.
The specific workflow of user configuration and network management comprises the following steps:
step 11: first, the user equipment of each terminal is interconnected with the deterministic service manager through the API protocol to establish a temporary connection. The deterministic service manager performs function retrieval and user demand acquisition on the user equipment of the terminal through the user configuration module. And the deterministic network characteristics are configured on the terminal user equipment so as to facilitate the deterministic service manager to configure the terminal information, thereby avoiding the details of network internal propagation and network configuration for deterministic services. Meanwhile, communication overhead and time delay in the network are reduced, so that the deterministic service demand can directly influence the data forwarding process in and among domains.
Step 12: after acquiring user requirements and configuring deterministic characteristics from a deterministic service manager, an intra-domain network management module in the Det-AS controller processes QoS requests from end user devices and intra-domain network management. The Det-AS controllers execute the functions of resource management and equipment management through Netconf and other protocols, acquire information such AS network topology in each domain, and collect the capability of each network equipment in each domain to support deterministic network technology. After acquiring the intra-domain network state information, the Det-AS controller collects and manages the information of the user's deterministic service, such AS the demand information, the flow characteristics, and the like, including the source address, the destination address, the service flow size, the QoS, and the like, according to the characteristics of flow management.
Step 13: further, after the flow information, the network resource information and the topology information are collected through each management function module, operations such as route calculation, resource management and the like are performed. After the operations such AS route calculation are completed, the Det-AS controller issues the calculation result to each network device in the domain through protocols such AS Netconf. Specifically, the Det-AS controller issues a forwarding flow table to FR of each domain, and issues a route management protocol and a delay result in each Det-AS to ER, so AS to guide a routing and scheduling process of deterministic traffic flow.
Step 14: further, the Det-AS controller establishes a direct connection relationship with other Det-AS controllers by configuring a route management protocol. And each Det-AS controller performs the interaction of information such AS network topology, routing strategy, network configuration, network state and the like by executing a routing management protocol through east-west interfaces so AS to better forward the cross-domain deterministic service flow. The east-west interface is used for communication and collaboration between the Det-AS controllers, which can exchange network state information of the respective managed domains, such AS link state, topology change, load condition, etc., through which a global network view can be established and perceived.
Step 15: through the steps, the network equipment in each domain carries out data forwarding according to the configuration and the instruction of each Det-AS controller, and the inter-domain carries out the deterministic forwarding of the cross-domain service flow according to the optimal route of the inter-domain routing table by configuring the routing management protocol. The distributed control management mode can ensure that the whole open large-scale network is configured and managed as required, and low-delay data forwarding of the global end-to-end cross-domain network is realized.
Referring to fig. 2, the dividing of deterministic autonomous domain and the construction process of corresponding domain nodes facing to open large-scale network in step 2 are described in detail to realize low-delay forwarding of deterministic traffic flow.
The construction process is represented by a 3-layer architecture, wherein the upper layer represents the network topology formed by domain nodes corresponding to the Det-AS, the middle layer represents the division of the Det-AS, and the lower layer represents the actual topology of the original open network. Because the certainty of the cross-domain time delay is still an important problem in the open network environment, the distributed control method provided by the invention can abstract the open large-scale network into a plurality of domain nodes, the number of domains is not limited, and the access of a plurality of terminals is supported.
The specific topology construction process of the domain node is as follows:
in the original open large-scale network, a plurality of network devices are integrated together to form a complex network topology. When the deterministic service flow is sent from the user equipment of the sending end to the user equipment of the receiving end, the forwarding of the service flow has a plurality of routing paths, and meanwhile, the routing paths are complicated and have a plurality of problems of link redundancy, overlapping and the like. In order to meet different network and management requirements, the data forwarding delay from end to end of the open network is reduced, and the open complex network can be divided into a plurality of independent Det-AS. Each Det-AS has its own network equipment and management structure inside to implement internal route optimization and data transmission control.
Inside each Det-AS, the devices of the original open network are mapped into the corresponding Det-AS, further by re-planning and managing the Det-AS, a limited number of ERs and a number of FR's will be set for efficient management. Wherein ER plays a key role between different autonomous domains and is responsible for cross-domain deterministic traffic forwarding. However, in the forwarding process of the end-to-end deterministic traffic, in order to calculate the optimal delay of the whole open network, first, the optimal delay inside each Det-AS is calculated by re-planning the routing path inside each Det-AS. Then, data forwarding across domains is performed by the ER. The communication between the Det-AS is a complex process, and routing paths among different Det-AS are required to be comprehensively considered.
And regarding the calculated optimal time delay in each Det-AS AS the attribute of the domain node in the domain node topology layer, wherein the domain node with the time delay attribute corresponds to each domain of the Det-AS topology layer one by one. Inter-domain communication takes each domain as a single entity, and the cross-domain deterministic traffic forwarding is realized by executing a route management protocol. First, a neighbor relation is established with other domain nodes, and exchange of route reachability information is performed through ER supporting route management protocol. After receiving the route reachability information sent by the neighbor domain node, the ER processes the route information. The ER may learn multiple route forwarding paths. As shown in fig. 2, there may be multiple route forwarding paths for each deterministic traffic flow between domain nodes. And then, according to the learned route reachability information, the ER takes the route reachability information as a candidate route, and performs optimal route selection according to a route selection strategy adopted among domains so as to obtain globally optimal data forwarding delay. AS can be seen from fig. 2, the routing paths between the domain nodes and the routing paths between the Det-AS correspond to each other. Compared with the routing path of the original open network, the routing path among the domain nodes constructed by the invention is better.
Referring to fig. 3, the intra-domain data request forwarding process of a single domain in step 3, the main entity of intra-domain communication establishment and the data forwarding flow are described in detail,
the main entities required for the communication setup within the Det-AS are AS follows:
within the Det-AS there are two types of routers, ER and FR, respectively. They respectively take on different functions and roles in the network. The ER is positioned at the edge of each domain and used for connecting different Det-AS, and can also interact with other domain nodes through a route management protocol configured by the controller to realize the forwarding of the cross-domain service flow.
The FR is located inside each domain and is responsible for forwarding the data packets in the domain according to a certain rule and path until the service flow reaches the user equipment of the receiving end. The FR of the data forwarding plane may forward deterministic traffic within the domain according to the routing forwarding paths deployed and configured by the controller.
The single Det-AS initiates a communication process and a specific data forwarding flow comprises the following steps:
step 31: when forwarding data in a single domain, one or more deterministic traffic flows are first sent to the ER device of the current domain by the user device of the sender or by the ER device of other domains.
Step 32: after receiving the service flow, ER uploads network state information such AS network topology, link information, node information and the like in the domain to a Det-AS controller in charge of the domain so AS to forward the deterministic data in the domain.
Step 33: each Det-AS controller performs tasks such AS routing path calculation, resource reservation, and flow management in the domain according to the collected user information and network topology information. Specifically, each service flow is processed by a data forwarding processing engine in the routing device, as shown in fig. 3. When deterministic traffic flows enter the processing engine of the router, the traffic flows are first subjected to traffic classification operations. The traffic flow will then map to the corresponding priority queue according to its flow identification, flow characteristics, priority etc. attributes. Each queue is given different priorities for distinguishing deterministic traffic flows of different levels for shaping and scheduling of traffic flows for routing computation, resource reservation, flow management, etc.
Step 34: and the Det-AS controller configures and manages router devices in the domain according to the routing calculation result, and issues a forwarding flow table and a routing management protocol in the domain to each network device. And mapping the deterministic service flow to the corresponding queue for queuing according to the priority queue divided in the flow table. And performing traffic shaping and scheduling operations according to traffic conditions, network states, buffer sizes and the like in each queue so as to perform dequeuing processing. Meanwhile, the Det-AS controller adopts various scheduling algorithms to determine the transmission sequence of the service flow in each queue, and the algorithms can schedule the service flow based on strategies such AS priority, gating list, bandwidth guarantee, minimum delay and the like. Once the transmitted queues are determined, the packets will be pulled from the corresponding queues and sent to the egress port of the router waiting to be forwarded to the next hop router.
Step 35: after the Det-AS controller issues the domain flow table to each network device, each network device configures the flow table to facilitate deterministic forwarding of the traffic flow. The flow table comprises a packet header field, a counter, an action table, clock synchronization and the like. The packet header field information includes a stream ID, a source address, a destination address, an input port, an output port, a priority, and the like. The counter contains statistics of the data packets and traffic flows. The action table refers to actions of forwarding the data packet to the Det-AS controller and the router port, and processing procedures of the traffic flow, etc. The clock synchronization records the clock synchronization mechanism adopted and the forwarding period of the current service flow.
Step 36: after each network device in the Det-AS configures the flow table, the service flow is forwarded. And selecting a next-hop router according to the forwarding flow table configured by the FR to ensure that the service flow correctly reaches the ER in the Det-AS according to the selected path until the forwarding delay of the deterministic service flow in the Det-AS is obtained.
Step 37: according to the data forwarding in the Det-AS, the routing result of each Det-AS can be obtained, so that the intra-domain time delay of each Det-AS and the number of the corresponding domain node are obtained. For example, FIG. 3 has two different routing paths to the ER, which correspond to two different domain delays, each corresponding to an edge of a domain node. According to different time delays and corresponding output numbers, the result is abstracted into a multi-degree domain node with time delay attributes, and the multi-degree domain node can be connected with other domain nodes by a plurality of links.
Step 38: further, the egress of each multi-degree domain node corresponds to a particular egress port of the node through which traffic may be sent from the current domain node to other domain nodes, each egress port being connectable to a different domain node. And finally, according to the designed inter-domain routing forwarding strategy, selecting the optimal time delay in the domain and the corresponding edge outlet to perform deterministic forwarding of the data.
In the whole process, the classification and selective transmission of the traffic are used for meeting the demands of time delay, jitter and the like of deterministic traffic flows and ensuring that the traffic is transmitted according to a preset priority and path. Thus, reliable network service can be provided, and the requirements of deterministic services of different levels are met. Meanwhile, the deterministic service flow is subjected to cross-domain data forwarding by outputting the multi-domain node with the time delay attribute.
Referring to fig. 4, a data forwarding process between Det-AS in step 4 is described, and according to the multi-degree domain nodes with delay attribute corresponding to each Det-AS obtained in step 3, a new network topology is formed by the domain nodes, where the network topology may include a plurality of multi-degree domain nodes. The inter-domain communication setup and data forwarding procedure will now be described in detail. The method specifically comprises the following steps:
Step 41: firstly, according to data forwarding in a domain, a plurality of routing results of each domain, forwarding delay T in the domain corresponding to each routing path and an outlet k corresponding to the optimal delay can be obtained, each Det-AS is abstracted into a multi-degree domain node with delay attribute T, and a connection relationship is established between the domain nodes and adjacent domain nodes through ER so AS to establish inter-domain communication.
Step 42: when the deterministic traffic flows forward data across domains, the ER supporting the route management protocol forwards the deterministic traffic flows across domains. The ER establishes a neighbor relation with other domain nodes by employing a route management protocol. After establishing the neighbor relation, the ER begins exchanging route reachability information. The routing reachability information includes network state information of each Det-AS, requirements of deterministic service, intra-domain delay results, corresponding edge ports and the like.
Step 43: after receiving the route reachability information sent by the neighbor, the ER analyzes the route information and extracts the destination address of the service flow sent by the terminal user equipment and the route reachability information of the neighbor domain node. And storing the analyzed routing information of the domain nodes in a routing table among domains to record the routing path of the domain nodes, the destination nodes, the initial nodes, the domain time delay, the corresponding edge outlet and other information. Meanwhile, the ER may also receive an update of the routing information and add the routing information to the inter-domain routing table and update.
Step 44: further, there are various results (e.g., a, b, and c) of the link states between the domain nodes according to different Det-AS delays t and the outlets k of the multi-degree domain nodes, and the number of links is limited and determined. The number of links between domains is determined by the outlet of each multi-degree domain node, and each time delay result in the domain corresponds to the outlet of one domain node, so that the deterministic forwarding of the service flow is facilitated. Therefore, the ER can receive a plurality of pieces of path information of the domain where the user equipment reaching the receiving end is located, and the ER uses the path information as a candidate route and performs optimal route selection according to a route selection policy (such as a shortest path first algorithm) adopted between the domains.
Step 45: the invention aims at minimizing the end-to-end time delay, so that a routing path with the minimum time delay is selected. In fig. 4, there are three paths a, b and c that can reach the destination domain T4 where the user equipment at the receiving end is located from the start domain T1 where the user equipment at the transmitting end is located, where T1 represents the optimal network delay of domain 1. The ER determines an optimal routing path (e.g., path a) according to a routing policy that minimizes latency and adds the optimal routing path to the inter-domain routing table. Finally, ER performs inter-domain deterministic data forwarding according to the learned routing reachability information and routing policy.
Referring to fig. 5, taking an end-to-end cross-domain data forwarding process as an example, a communication establishment process of the entire open large-scale deterministic network in step 5 and a calculation process of an end-to-end data forwarding delay are described. The process will be described in terms of a 4-layer architecture, with deterministic traffic configuration, deterministic controller management, intra-domain data forwarding, and inter-domain data forwarding in order from top to bottom. The method specifically comprises the following steps:
step 51: first, a deterministic traffic manager gathers deterministic traffic demands from end user devices through an API protocol and then manages these deterministic traffic demands for transmission to the control plane.
Step 52: when the deterministic service manager sends deterministic service flow to the network side through the UNI protocol, the network side manages the current large-scale network through a distributed Det-AS control method. Firstly, the Det-AS controller registers network resource requirements such AS delay, jitter and bandwidth, and application requirements such AS QoS, load balancing and access control.
Step 53: further, after the Det-AS controller obtains the deterministic service requirement and the network state information in the domain, the network device in the domain is configured and managed through relevant protocols such AS Netconf. And the Det-AS controller performs operations such AS route calculation, resource reservation and the like according to the collected network state information and application requirements. After operations such AS route calculation, network devices such AS routers in each Det-AS perform intra-domain deterministic data forwarding according to the optimal path obtained by the route calculation result. Different routing strategies may select different paths for communication, which may result in network delays and corresponding number of outgoing numbers in multiple domains.
Step 54: and transmitting proper route management protocol configuration information to ER in each domain through a Det-AS controller, and abstracting each domain into a multi-domain node with domain delay attribute when carrying out inter-domain deterministic communication according to the calculation result obtained by the steps, including the routing result, the delay and the corresponding output number in the domain, and realizing cross-domain deterministic service flow forwarding through interconnection among the multi-domain nodes.
Step 55: inter-domain data forwarding is performed among the multi-degree domain nodes through an inter-domain routing table, and inter-domain deterministic routing forwarding is performed through a routing strategy, a routing management protocol and the like which are preset in the routing table. AS can be seen from fig. 5, there may be multiple routing paths between domain nodes, but in order to obtain end-to-end global low latency data forwarding, when traffic flows are transmitted between different Det-AS, the traffic flows cross-domain from the initial domain node to the target domain node according to the optimal path.
Step 56: furthermore, the distributed controllers in the control plane interact with network state information such AS time delay, bandwidth and the like through east-west interfaces so AS to adjust routing paths between the Det-AS and the domains in time, and the deterministic service flow can be successfully received by the user equipment of the receiving end through coordination and cooperation between the domains. Meanwhile, deterministic forwarding adopted by the ER across domains may be applicable to various data plane encapsulation manners, including but not limited to IP forwarding, MPLS, and segment routing. Through a deterministic forwarding mechanism, the cross-domain deterministic communication mode can ensure the performance requirements of time delay, jitter and the like of service flows. Meanwhile, the cross-domain communication can support different data surface packaging modes, so that the network can adapt to different application scenes and requirements.
Step 57: through the steps, the intra-domain data forwarding delay and inter-domain data forwarding delay obtained through distributed computation are summed, and the end-to-end data forwarding delay of the large-scale deterministic network can be obtained. Finally, whether the time delay meets the requirement of deterministic service flow can be determined through verification so as to ensure the validity of configuration, and deterministic configuration is carried out on terminal equipment.
Step 58: and obtaining QoS network indexes such as end-to-end delay, jitter and the like of the deterministic service flow through calculation of the deterministic service flow, and reporting a calculation result to a controller of a domain where user equipment of a receiving end is located. If an error or abnormal condition occurs in the index value, the routing is needed to be carried out again. The terminal network equipment reports the messages through the Det-AS controllers one by one through the API and feeds the messages back to the controllers of the domains of the user equipment of the sending end. The above steps are repeated until inter-domain cooperative communication can ensure that the cross-domain end-to-end deterministic latency and jitter are within acceptable ranges.
The distributed control network architecture and the cross-domain end-to-end low-delay data forwarding control scheme for the large-scale deterministic network can realize high-efficiency and stable deterministic data forwarding in open network environments such as a large-scale network, a WAN, a hybrid network and the like. The invention can meet the deterministic communication requirements of different industries and application fields, and provides an effective solution for realizing reliable cross-domain end-to-end low-delay data forwarding control.
It should be noted that the above-mentioned embodiments are not intended to limit the scope of the present invention, and equivalent changes or substitutions made on the basis of the above-mentioned technical solutions fall within the scope of the present invention as defined in the claims.
Claims (9)
1. A distributed control system for large-scale deterministic network is characterized in that the control system comprises a Det-AS, an inter-domain, a Det-AS controller and a deterministic service manager,
wherein the Det-AS refers to a plurality of independent and deterministic network areas divided in an open wide area deterministic network, each Det-AS is equipped with a controller and several network devices, and at the same time, the controller manages and controls each Det-AS for realizing traffic management and deterministic data forwarding in the domain,
when deterministic forwarding is carried out among domains, abstracting each domain into a node with an intra-domain time delay attribute, interacting route reachability information with adjacent domain nodes through a route management protocol, and then calculating an optimal path according to a route selection strategy to carry out deterministic data forwarding so as to minimize the data forwarding time delay among domains;
Det-AS controllers, wherein each Det-AS is equipped with a controller for path computation, resource allocation, protocol delivery, information collection, etc. within a domain, or cooperates with other controllers, exchange information,
The deterministic service manager is used for receiving and collecting the user requirements of the user equipment at the access network side and the requests of the application programs, is responsible for the configuration and management of deterministic services, and provides the network service configuration requirements between the end-to-end application systems,
the interface relation among the components of the control system is as follows:
network equipment in each domain in the data forwarding plane uploads information such as network state, links, nodes and the like in the domain to a controller of the control plane through controlling a southbound interface, meanwhile, each controller receives requirements of various deterministic service flows from an application plane through a service northbound interface, network configuration, protocols and routing calculation results are issued through controlling the southbound interface, interaction among the controllers is correspondingly cooperated and coordinated through the southbound interface, so that information synchronization and information interaction of each domain are facilitated, and decision and management of cross-domain data forwarding are better carried out.
2. The large-scale deterministic network oriented distributed control system according to claim 1, wherein the control system further comprises Edge Router (ER) and forwarding routers (Forwarding Router, FR), wherein ER is used for connecting adjacent Det-AS and other external networks, is mainly responsible for deterministic data forwarding and routing between domains, and transmits deterministic traffic from a user equipment at a transmitting end to a domain where the user equipment at a receiving end is located through ER of several domains by means of routing management protocols issued by the controller and routing policies adopted between domains; the FR is a key network device in the route forwarding process, and is used for forwarding deterministic traffic flows entering the domain according to a preset routing rule and a route calculation result of each domain, and finally sending the deterministic traffic flows to the ER of each domain until the terminal device receives the traffic flows.
3. Network architecture implementing a distributed control system for large scale deterministic networks according to claim 1 or 2, characterized in that said network architecture comprises an application plane, a control plane and a data forwarding plane, which planes are separated from each other and interact through an open interface provided,
wherein the application plane is an upper application layer in the network architecture, and mainly comprises a deterministic service manager and deterministic service applications with different service quality requirements (Quality of Service, qoS),
the control plane is mainly composed of one or more controllers, each Det-AS corresponds to one controller, meanwhile, the controllers have the functions of calculation, management, interaction and the like, are mainly responsible for network configuration, the network configuration comprises configuration and management of basic parameters in a network, the controllers transmit network state information such AS time delay and the like to interfaces through east and west, and interact with other adjacent controllers to meet the requirement of interoperability among the controllers,
the data forwarding plane is connected to the control plane through a control southbound interface and bears the actual forwarding and processing tasks of various deterministic services in a wide area deterministic network, the plane is composed of a plurality of Det-AS, each Det-AS is composed of network equipment elements such AS ER and FR, meanwhile, the plane carries out deterministic forwarding on service flows according to routing rules and strategies which are preset by a controller, and in an open wide area deterministic network environment, the deterministic service flows are forwarded through user equipment at a transmitting end, pass through the plurality of Det-AS and a plurality of forwarding paths and are finally transmitted to the user equipment at a receiving end.
4. A distributed control method for a large-scale deterministic network, the method comprising the steps of:
step 1: user equipment at a transmitting end reports the requirement of deterministic service to a deterministic service manager through an application programming interface (Application Programming Interface, API) protocol for User configuration, and a Det-AS controller transmits the deterministic service from a User side to a network side through a User network information (User/Network Information, UNI) protocol for network management;
step 2: the network side divides an open large-scale deterministic network into a plurality of independent Det-AS by adopting a distributed control network architecture, abstracts the Det-AS into domain nodes with time delay attributes, and carries out cross-domain data forwarding by constructing the domain nodes;
step 3: in the Det-AS, each domain is configured and scheduled by a controller, the controller carries out route calculation according to network topology information, flow demand, performance index and the like in the domain to obtain an optimal route result in the domain, and the calculation result is issued to the network equipment in the domain for data forwarding so AS to realize the minimization of time delay in the domain;
step 4: when the Det-AS communicates, each domain is regarded AS a node with an intra-domain time delay attribute, the controller transmits a route management protocol to ER in each domain, the ER acquires route reachability information of nodes in other domains through the route management protocol, and performs optimal path selection among domains according to a route selection strategy, and the ER performs deterministic data forwarding according to the optimal inter-domain route path so AS to realize minimization of inter-domain time delay;
Step 5: through the steps, the intra-domain time delay and the inter-domain time delay are calculated respectively, wherein the intra-domain time delay is the delay caused when data forwarding is carried out in the same domain; the inter-domain delay is the data forwarding delay caused when different domains are crossed, and the calculated inter-domain delay and inter-domain delay are summed to obtain the limited and smaller large-scale deterministic network end-to-end data forwarding delay.
5. The distributed control method for large-scale deterministic network according to claim 2, wherein the specific process in step 1 is as follows, and the specific workflow of user configuration and network management comprises the steps of:
step 11: firstly, user equipment of each terminal is interconnected with a deterministic service manager through an API protocol to establish temporary connection, the deterministic service manager performs function retrieval and user demand acquisition on the user equipment of the terminal through a user configuration module, and performs configuration of deterministic network characteristics on the terminal user equipment,
step 12: after acquiring user demands and configuring deterministic characteristics from a deterministic service manager, a intra-domain network management module in a Det-AS controller processes QoS requests from end user equipment and intra-domain network management, each Det-AS controller executes resource management and equipment management functions through Netconf and other protocols, acquires information such AS network topology in each domain, collects the capability of each network equipment in each domain to support deterministic network technology, the Det-AS controller collects and manages information such AS demand information, flow characteristics and the like of deterministic services of the user according to the characteristics of flow management after acquiring intra-domain network state information,
Step 13: after collecting flow information, network resource information and topology information through each management function module, carrying out route calculation and resource management operation, after finishing operations such AS route calculation, etc., transmitting calculation results to each network device in the domain through Netconf protocol by the Det-AS controller, specifically transmitting a forwarding flow table to FR of each domain by the Det-AS controller, transmitting delay results in each Det-AS to ER by the route management protocol, thereby guiding the route and dispatch process of deterministic service flow,
step 14: the Det-AS controllers establish direct connection relation with other Det-AS controllers by configuring a route management protocol, the Det-AS controllers perform the route management protocol to perform the interaction of network topology, route strategy, network configuration, network state and other information through east-west interfaces so AS to better perform the cross-domain deterministic service flow forwarding, the east-west interfaces are used for the communication and cooperation among the Det-AS controllers, the Det-AS controllers can exchange the network state information of the respective managed domains, including link state, topology change and load condition, global network views can be established and perceived through the information interaction,
Step 15: through the steps, the network equipment in each domain carries out data forwarding according to the configuration and the instruction of each Det-AS controller, and the inter-domain carries out the deterministic forwarding of the cross-domain service flow according to the optimal route of the inter-domain routing table by configuring the routing management protocol, so that the distributed control management mode ensures that the whole open large-scale network carries out configuration and management AS required, and realizes the low-delay data forwarding from the global end to the end cross-domain network.
6. The distributed control method for large-scale deterministic network according to claim 3, wherein in step 2, the process of partitioning Det-AS and constructing corresponding domain nodes of the open large-scale network is implemented to realize low latency of deterministic traffic flow, the construction process is represented graphically by 3 layers of architecture, the upper layer represents the network topology formed by the domain nodes corresponding to each Det-AS, the middle layer represents the partition of each Det-AS, the lower layer represents the actual topology of the original open network, and the specific topology construction process of the domain nodes is AS follows:
in the original open large-scale network, a plurality of network devices are integrated together to form a complex network topology, when a deterministic service flow is sent from user equipment of a sending end to user equipment of a receiving end, the forwarding of the service flow has a plurality of routing paths, meanwhile, the routing paths are complicated and have a plurality of problems of link redundancy, overlapping and the like, in order to meet different network and management requirements, the data forwarding delay from end to end of the open network is reduced, the open complex network can be divided into a plurality of independent Det-AS, each Det-AS is internally provided with own network devices and management structures so AS to realize the internal route optimization and data transmission control, each device of the original open network can be mapped into a corresponding Det-AS in each Det-AS, the efficient management of each domain is realized by re-planning and managing the Det-AS, and the calculated optimal delay in each Det-AS is regarded AS the attribute of a domain node, and the topology layer with the delay attribute corresponds to each domain of the Det-AS one by one.
7. The distributed control method for large-scale deterministic network according to claim 3 or 4, wherein the single Det-AS initiated communication process and specific data forwarding procedure in step 3 comprises the steps of:
step 31: when data forwarding is performed in a single domain, one or more deterministic traffic flows are first sent by the user equipment at the sender or by the ER equipment of the other domain to the ER equipment of the current domain,
step 32: after receiving the service flow, ER uploads network state information such AS network topology, link information, node information and the like in the domain to a Det-AS controller in charge of the domain to forward deterministic data in the domain,
step 33: each Det-AS controller will perform tasks such AS routing path calculation, resource reservation and flow management in the domain according to the collected user information and network topology and the like,
step 34: the Det-AS controller configures and manages router devices in the domain according to the routing calculation result, issues a forwarding flow table and a routing management protocol in the domain to each network device, maps deterministic service flows into corresponding queues according to priority queues divided in the flow table for queuing, performs flow shaping and scheduling operations according to flow conditions, network states, buffer sizes and the like in each queue for dequeuing processing, and simultaneously adopts various scheduling algorithms to determine the transmission sequence of the service flows in each queue,
Step 35: after the Det-AS controller issues the domain flow table to each network device, each network device configures the flow table to facilitate deterministic forwarding of the traffic flow,
step 36: after each network device in the Det-AS configures the flow table, forwarding the service flow, selecting the next-hop router according to the forwarding flow table configured by the FR to ensure that the service flow correctly reaches the ER in the Det-AS according to the selected path until the forwarding delay of the deterministic service flow in the Det-AS is obtained,
step 37: obtaining a routing result of each Det-AS according to the data forwarding in the Det-AS, thereby obtaining the intra-domain time delay of each Det-AS and the output number of the corresponding domain node, abstracting the result into a multi-domain node with time delay attribute according to different time delays and corresponding output numbers, wherein the multi-domain node can be connected with other domain nodes by a plurality of paths,
step 38: the output degree of each multi-degree domain node corresponds to a specific output edge of the node, the service flow can be sent from the current domain node to other domain nodes through the output edge, each output edge can be connected to different domain nodes, and finally, the optimal time delay in the domain and the corresponding output edge are selected to carry out deterministic forwarding on data according to the designed inter-domain routing forwarding strategy.
8. The distributed control method for large-scale deterministic network according to claim 5, wherein the inter-domain communication establishment and data forwarding in step 4 specifically comprises the steps of:
step 41: firstly, according to the data forwarding in the domains, obtaining a plurality of routing results of each domain, forwarding delay T in the domains corresponding to each routing path and an outlet k corresponding to the optimal delay, abstracting each Det-AS into a multi-degree domain node with delay attribute T, establishing a connection relationship between the domain nodes and adjacent domain nodes through ER to establish communication between domains,
step 42: when the deterministic traffic flows forward data of cross-domain, the ER supporting the route management protocol forwards the deterministic traffic flows of cross-domain, the ER establishes a neighbor relation with other domain nodes by adopting the route management protocol, after the neighbor relation is established, the ER starts exchanging route reachability information, the route reachability information comprises network state information of each Det-AS, the requirement of deterministic traffic, a domain delay result and a corresponding outlet,
step 43: after receiving the route reachability information sent by the neighbor, the ER analyzes the route information, extracts the destination address of the service flow sent by the end user equipment and the route reachability information of the neighbor domain node, stores the analyzed route information of the domain node in a route table among domains to record the route path, the destination node, the initial node, the intra-domain time delay, the corresponding edge outlet and other information of the domain node, meanwhile, the ER can also receive the update of the route information, and adds the route information into the route table among domains for updating,
Step 44: according to different Det-AS time delays t and the outlet k of the multi-degree domain nodes, the link states among the domain nodes have various results, the number of links is limited and determined, the number of links among the domains is determined by the outlet of each multi-degree domain node, and each time delay result in the domain corresponds to the outlet of one domain node so AS to facilitate the deterministic forwarding of the service flow, therefore, ER can receive the path information of the domains where a plurality of pieces of user equipment reaching the receiving end are located, and the ER takes the path information AS candidate routes and performs optimal route selection according to the route selection strategy (such AS shortest path priority algorithm) adopted among the domains,
step 45: the ER determines the optimal routing path according to the routing strategy of the minimized time delay, adds the optimal routing path into the routing table among the domains, and finally, the ER carries out the certainty data forwarding among the domains according to the learned routing reachability information and the routing strategy.
9. The large-scale deterministic network oriented distributed control method according to claim 6, wherein the implementation procedure of step 5 is as follows:
step 51: first, a deterministic traffic manager collects deterministic traffic demands from end user devices through the API protocol, and then manages these deterministic traffic demands in order to send them to the control plane,
Step 52: when the deterministic service manager sends deterministic service flow to the network side through UNI protocol, the network side manages the current large-scale network through distributed Det-AS control method, firstly, the Det-AS controller registers network resource demands such AS time delay, jitter and bandwidth, and application demands such AS QoS, load balancing and access control,
step 53: after the Det-AS controller obtains deterministic service requirements and network state information in the domain, the Det-AS controller configures and manages network devices in the domain through relevant protocols such AS Netconf, and the like, the Det-AS controller performs operations such AS route calculation, resource reservation, and the like according to the collected network state information and application requirements, after the operations such AS route calculation, the network devices such AS routers in each Det-AS perform deterministic data forwarding in the domain according to an optimal path obtained by a route calculation result, different routing strategies may select different paths for communication, different paths cause network delay and corresponding output numbers in a plurality of domains,
step 54: issuing proper route management protocol configuration information to ER in each domain through a Det-AS controller, abstracting each domain into a multi-domain node with domain delay attribute when carrying out inter-domain deterministic communication according to the calculation result obtained by the steps including the routing result, the delay and the corresponding output number in the domain, realizing cross-domain deterministic service flow forwarding through interconnection among the multi-domain nodes,
Step 55: inter-domain data forwarding is carried out among the multi-degree domain nodes through an inter-domain routing table, inter-domain deterministic routing forwarding is carried out through a routing strategy, a routing management protocol and the like which are preset in the routing table,
step 56: the distributed controllers in the control plane interact with network state information such AS time delay, bandwidth and the like through east-west interfaces so AS to adjust the routing paths between the Det-AS and the domains in time, and the deterministic service flow can be successfully received by the user equipment of the receiving end through coordination and cooperation between the domains, meanwhile, the deterministic forwarding adopted by the ER of the cross domains can be suitable for various data plane encapsulation modes including IP forwarding, MPLS and segmented routing,
step 57: through the steps, the intra-domain data forwarding delay and inter-domain data forwarding delay obtained by distributed computation are summed to obtain the data forwarding delay from end to end of the large-scale deterministic network, finally, whether the delay meets the requirements of deterministic service flows is determined through verification so as to ensure the effectiveness of configuration, and the deterministic configuration is carried out on terminal equipment,
step 58: qoS network indexes such AS end-to-end delay and jitter of the deterministic service flow are obtained through calculation of the deterministic service flow, calculation results are reported to a controller of a domain where user equipment of a receiving end is located, if errors or abnormal conditions of the index values are found, routing is needed to be carried out again, the terminal network equipment reports information through an API (application program interface) through a Det-AS controller one by one and feeds the information back to the controller of the domain where the user equipment of a sending end is located, and the steps are repeated until inter-domain cooperative communication can ensure that the deterministic delay and jitter of the end-to-end of a cross-domain are within an acceptable range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311256906.5A CN117424921A (en) | 2023-09-26 | 2023-09-26 | Distributed control system and control method oriented to large-scale deterministic network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311256906.5A CN117424921A (en) | 2023-09-26 | 2023-09-26 | Distributed control system and control method oriented to large-scale deterministic network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117424921A true CN117424921A (en) | 2024-01-19 |
Family
ID=89527399
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311256906.5A Pending CN117424921A (en) | 2023-09-26 | 2023-09-26 | Distributed control system and control method oriented to large-scale deterministic network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117424921A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117978552A (en) * | 2024-03-29 | 2024-05-03 | 昆仑太科(北京)技术股份有限公司 | Business process safety protection system and method for generalized deterministic network |
| CN118018382A (en) * | 2024-04-09 | 2024-05-10 | 南京航空航天大学 | Collaborative management method for distributed deterministic controllers in large-scale wide-area open network |
-
2023
- 2023-09-26 CN CN202311256906.5A patent/CN117424921A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117978552A (en) * | 2024-03-29 | 2024-05-03 | 昆仑太科(北京)技术股份有限公司 | Business process safety protection system and method for generalized deterministic network |
| CN118018382A (en) * | 2024-04-09 | 2024-05-10 | 南京航空航天大学 | Collaborative management method for distributed deterministic controllers in large-scale wide-area open network |
| CN118018382B (en) * | 2024-04-09 | 2024-06-21 | 南京航空航天大学 | Collaborative management method for distributed deterministic controllers in large-scale wide-area open network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2911348B1 (en) | Control device discovery in networks having separate control and forwarding devices | |
| CN117424921A (en) | Distributed control system and control method oriented to large-scale deterministic network | |
| US8493869B2 (en) | Distributed constraints-based inter-domain network traffic management | |
| US20220150159A1 (en) | Control device, switch device and methods | |
| EP3123677B1 (en) | A method to provide elasticity in transport network virtualisation | |
| US20160323144A1 (en) | Traffic-driven network controller placement in software-defined networks | |
| Porxas et al. | QoS-aware virtualization-enabled routing in software-defined networks | |
| CN104780056A (en) | Network management method and equipment | |
| CN111245644A (en) | Method and system for expanding PCEP (physical layer protocol) protocol to automatically create tunnel in SDN (software defined network) scene | |
| Alshaer et al. | Software-defined networking-enabled heterogeneous wireless networks and applications convergence | |
| Paganini et al. | A unified approach to congestion control and node-based multipath routing | |
| Ghalwash et al. | A QoS framework for SDN-based networks | |
| CN110557333A (en) | method and system for controlling and guaranteeing quality of service of software defined network | |
| Nasrallah et al. | Reconfiguration algorithms for high precision communications in time sensitive networks: time-aware shaper configuration with IEEE 802.1 qcc (extended version) | |
| US20140185607A1 (en) | Communication system, communication path establishing method and management server | |
| Hernandez et al. | Comparative performance analysis between software-defined networks and conventional IP Networks | |
| Venmani et al. | Demystifying link congestion in 4G-LTE backhaul using openflow | |
| Babu et al. | A novel framework for resource discovery and self-configuration in software defined wireless mesh networks | |
| Zheng et al. | Mix-flow scheduling for concurrent multipath transmission in time-sensitive networking | |
| Singh | Routing algorithms for time sensitive networks | |
| Rahouti et al. | Latencysmasher: a software-defined networking-based framework for end-to-end latency optimization | |
| Tajiki et al. | Optimal estimation of link delays based on end-to-end active measurements | |
| Dong et al. | Multi-layer and heterogeneous resource management in sdn-based space-terrestrial integrated networks | |
| Tian et al. | Large-Scale Deterministic Networks: Architecture, Enabling Technologies, Case Study and Future Directions | |
| Oginni et al. | Constraint-aware software-defined network for routing real-time multimedia |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |