NL1041873B1

NL1041873B1 - A test device, a testing system, a testing method and a computer program product for testing a network

Info

Publication number: NL1041873B1
Application number: NL1041873A
Authority: NL
Inventors: Johan Alexander Tirion Jeroen
Original assignee: Tirion Networks & Communications
Priority date: 2016-05-18
Filing date: 2016-05-18
Publication date: 2017-11-30
Also published as: NL1041873A

Abstract

The invention relates to a test device, comprising hardware and software, a processing unit, memory and an interface for connecting to a network. The test device is arranged for running a complement of applications and services for performing a testing process on a network. The testing process includes the steps of schedule and perform network performance assessment tests (mode A), perform network performance diagnostic tests (mode B), and/or serve as a target for tests from a single or a multiple number of other test devices on the network for load testing (mode C).

Description

Title: A test device, a testing system, a testing method and a computer program product for testing a network

Field of the invention

The present invention relates generally to computer network management and in particular to the performance and stability of such networks as defined by connectivity, latency and the variation of network latency over time: jitter. The invention defines methods to determine these parameters from multiple points on the network. The invention also defines methods to diagnose performance anomalies from multiple points on the network.

Background A computer network is made up of a plurality of sites (100, 101, 102) and interconnecting links (105, 121), independent of the underlying technology. Network latency, the time it takes for data to flow between computing devices (classified as clients (111) and servers (113)) at different points on the network is affected by many aspects (e.g. physical distance, medium type, number of intermediate network components and their respective internal latencies). These aspects are generally hard to determine objectively, especially in a Wide Area Network which makes use of third-party communications links. Unfortunately, network latency greatly affects the performance of network-related applications and the way this is perceived by users. Temporary variation in network latency (“jitter”) can lead to intermittent communication failures. Temporary loss of connectivity between clients and servers will certainly lead to intermittent communication failures.

Brief summary of the invention

The present invention relates to: • a test device comprising hardware and software, a processing unit (CPU) and memory and interfaces. The device runs a complement of applications and services, and is arranged to: 1. Schedule and perform network performance assessment tests (mode A); 2. Perform network performance diagnostic tests (mode B), and/or; 3. Serve as a target for tests from a single or a multiple number of other test devices on the network for load testing (mode C). A physical or virtualized embodiment of this test device can be connected to the network in a plurality of sites to provide a broad view of the performance of (a subset of) data paths through the network and to provide diagnostic information from multiple positions on the network.

The computing power required for these tests is limited; recent developments in computer technology and miniaturization have produced very small form factor platforms, leading to a low price per test device. This in turn allows large-scale deployment throughout the network. A testing system may include a single or a multiple number of test devices for connecting at different locations of the network or at different network elements.

The present invention also relates to a method of testing a network, including the steps of • a monitoring step (mode A) to periodically assess the performance of a computer network by inserting test traffic from the test device into the network and to measure the round-trip delay time of this traffic; • a diagnostics step (mode B) to assist the diagnosis of performance issues in a computer network by inserting test traffic from the test device into the network and collecting the results, and/or • a load step (mode C) to assess the capabilities of a computer network by generating high volumes of traffic between test devices on the network and to observe the effects of this traffic on other components.

Advantageously, the testing system includes an analysis system combining these elements with automatic periodic analysis of collected test results to determine baseline numbers and trends. The results of the automatic numerical analysis of test results may be used to generate reports, and can be combined with preset thresholds to generate alerts to inform the system user(s) of present or predicted future problems on the network that is being monitored.

The collected test result data can be made available to the system user(s) through a variety of application interfaces for further analysis, reporting and display.

The present invention may employ several well-known techniques to determine the momentary status of (a section of) a network, including e.g. • The ping-utility to determine basic connectivity as well as momentary network response time between the test device and the test target> • The traceroute utility to determine the momentary network path between the test device and the test target • The speedtest.net internet test system to generate traffic on (a section o£> a network, and to determine the available bandwidth between the test device and the speedtest.net test target • The iperf3 utility to generate traffic on (a section of) a network, and to determine the available bandwidth between two test devices.

Further, the invention relates to a computer program product. A computer program product may comprise a set of computer executable instructions stored on a data carrier, such as a flash memory, a CD, a DVD or a cloud storage. The set of computer executable instructions, which allow a programmable computer to carry out the method as defined above, may also be available for downloading from a remote server, for example via the Internet, e.g. as an app. Instead of being run by the operator of the remote infrastructure network, the computer program can be executed in the accessible cloud location and offered back to the operator as a service.

Other advantageous options and embodiments according to the invention are described in the following claims.

Brief description of the drawings

By way of example only, embodiments of the present invention will now be described with reference to the accompanying drawing in which

Fig. 1 depicts the network environment in which the invention is intended to operate.

Fig. 2 depicts the various information flows and repositories required for the invention to operate in monitor mode A.

Fig. 3depicts the various information flows and repositories required for the invention to operate in diagnostics mode B.

Fig. 4 depicts the various information flows required for the invention to operate in load test mode C.

Detailed description of the invention

In a specific embodiment, the test device (112) includes a complement of applications and services to: 1. Periodically retrieve test commands and metadata from a command repository (200); 2. Schedule and perform periodic network performance assessment tests on a network (106) according to the commands (monitor mode A); 3. Store performance assessment test results in a data repository (201); 4. Perform ad-hoc network performance diagnostic tests on a network (106) according to the commands (diagnostice mode B); 5. Return test results to a command repository (200) for delivery to a control application (305); 6. Serve as a target (112c) for tests from other embodiments of the test device connected to the same network, e.g. to perform load tests (mode C).

An embodiment of the test device additionally includes hardware and operating software, a processing unit CPU and memory to run the test applications and services, and an interface to connect to a network. These additional components may be virtualized, allowing for virtualized test devices.

On boot-up, the test device (112) creates (221) a unique command queue (210a) for itself in the command repository (200), and continues to poll this queue at a regular interval (222). The command repository does not have to be directly connected to the network under test (106), as long as the test device can access the repository using a web-safe protocol (HTTPS, 221 & 222).

On boot-up, the test device (112) also creates (220) a record in a device registry {213) in the data repository (201), and continues to update (220) this record at a regular interval. The data repository does not have to be directly connected to the network under test (106), as long as the test device can access the repository using a web-safe protocol (HTTPS, 220). Further actions from the test device are dependent on configuration commands from the control applications (203 & 305). A network engineer creates a test in the control application (203, 305) for a plurality of test devices (112) connected to the network under test (106). Information concerning the status of the test device is retrieved (223) from the device registry (212). Test metadata (224) includes test type, traffic pattern and target end point. Test metadata may include a schedule consisting of start time, end time and test interval, and is placed in the appropriate command queues (210) to be picked up by the test devices after the poll interval expires. The control application (203, 305) can run on a workstation (202a) which does not have to be directly connected to the network under test (106), as long as it has access to the command repository (200) as well as the data repository (201) using a web-safe protocol (HTTPS: 223 & 224, 325).

In monitor mode A, the test device periodically assesses the performance of a network (106) by inserting test traffic (141a) into the network to a plurality of end points (113) and determining the round-trip delay time of this traffic as it returns to the test device (141b), visualized in Fig. 2.

Test traffic consists of various well-known traffic types including ping (ICMP ECHO requests [internet standard RFC 792]), TCP handshake sequences [internet standard RFC 793], and simple HTTP 1.1 HEAD requests [internet standard RFC 2616]. The actual tests of mode A, as well as some basic statistics on the measured values are considered PRIOR ART. Mode A test results are stored (225) in a data repository (201) for time series analysis. Test results can be viewed from a plurality of display applications (204).

In diagnostics mode B, the test device assists in the diagnosis of performance issues in a network by inserting test traffic (341) into the network to a plurality of end points (113) and collecting the results (visualized in Fig. 3).

Mode B test results (324) are transmitted to a command repository (200) for delivery to a control application (305) for near real-time display and human interpretation. The actual tests of mode B may include known test procedures.

In addition to the mode A performance assessment tests, mode B comprises tests such as to determine the local network’s Address Resolution Protocol ARP table [internet standard RFC 892], to verify the local response to a Domain Name Server DNS request [internet standard RFC 1035], to determine the network path using the well-known traceroute utility (crafted UDP datagrams).

In load test mode C, the test device assists in determining the performance capabilities of (a section of) a network, by generating controlled traffic patterns (142) across the network, while the Network Monitoring System (114) keeps track of the status of the intermediate network components (visualized in Fig. 4). The exchange of test instructions and results between the load test control application (405) and the test devices follows the same pattern as mode B. The actual tests of mode C may include known test procedures, including “speedtest.net” and “iperf3”.

The analysis system (203) may combine these elements with automatic analysis of collected test results (227) to determine baseline numbers and trends. The results of the automatic numerical analysis of mode A test result data may be used to generate reports, and can be combined with preset thresholds to generate alerts to inform the system user(s) of present or predicted future problems on the network that is being monitored.

The collected mode A test result data can also be made available to the system user(s) through a variety of application interfaces (226) for further analysis, reporting and display.

Concerning the performance of computer networks

An Enterprise Packet Data Transmission Network (commonly referred to as “computer network” or just “network”) is the prevalent underlying infrastructure which connects a plurality of sites (100, 101, 102) with each other for the purpose of data interchange between various computing devices (ill, 113). Sites are interconnected by means of a plurality of links (105) in a variety of technologies (e.g. fiber optic connections, Multi-Protocol Label Switching MPLS networks, Virtual Private Networks VPN, microwave links, satellite links, etc.), in a variety of network architectures and topologies (e.g. partial or full mesh, hub-and-spoke, etc.). The underlying network technologies may be owned and operated by the enterprise, but may also be rented from a plurality of Managed Network Service Providers. Sites (100, 101, 102) can be geographically distributed giving rise to (potentially considerable) transmission delays due to distance and connection technology. A plurality of servers (113) connected to the network provide a variety of services (e.g. file storage, database storage, mail, telephony, instant messaging, etc.). A plurality of computing entities (111, e.g, desktop computers, laptop computers, network-connected printers, wireless devices) exchange data across the network employing flows of data (140) between a client (111a) and a server (113), sometimes referred to as sender and receiver even though data and control packets will flow in either direction. Network packets flowing between computing entities on a network take a non-zero, finite amount of time known as Network Delay or Latency.

Data (140) on the network is directed to the links by a plurality of active network components (103). The active network components dynamically select the optimal data path for (the flow of) data packets based on a variety of methods and technologies which are beyond the scope of this document. The result is that the time it takes for a data exchange between computing entities to complete is affected by the momentary state of the full path of the flow, including both the active network components as well as the passive network components and links. The variation of Network Delay this causes, is known as Network Jitter.

If the data transferred between the computing entities is considered a request which elicits a response, the time it takes for the last data element of that response to arrive back at the requestor is known as the Response Time, and will be affected by many other aspects beyond the scope of this document. A network with multiple sites and multiple links can quickly become very complex and hard to manage. Network operations personnel often employ a Network Monitoring System (114) to keep a view of the operational status of the active network components as well as the servers. The prevalent technologies are Ping (ICMP ECHO requests and replies between the monitoring system and the monitored component, to determine reachability) and SNMP (to retrieve a plurality of internal operational metrics from the component).

As long as a monitored component is reachable (e.g. ICMP ECHO replies arrive within a predetermined timeout period - e.g. 5 seconds) and the operational metrics (as periodically retrieved by SNMP, 425) are within predetermined acceptable levels, the component is considered “OK” (e.g. the Central Processing Unit of a router is consistently reporting less than 100% load). If all monitored components are “OK”, the network is considered “OK” as well. However, this does not accurately reflect the performance of the network as it pertains to data interchanges between various entities on the network across a variety of paths. Active network components which make up third-party /managed networks, Virtual Private Networks as well as the common Internet (122) are not “visible” to the internal Network Monitoring System - either because these components restrict access to their operational parameters or because the link is configured as a secure tunnel to avoid loss or abuse of the data which flows through it. This commonly includes the use of a plurality of firewalls (120) on the connection (121) to the internet to terminate the secure tunnel, which further reduces visibility of active network components while adding (potentially considerable) latency and jitter to the traffic.

The invention described in this patent application is intended to regularly assess the performance of a network independent of the status or the visibility of the responsible components along the data path, and to do so from the viewpoint of the users - i.e. distributed to a plurality of sites (100, 101, 102) throughout the network instead of from a central monitoring system.

The invention is also intended to assist in the diagnosis of network performance issues by performing a plurality of tests from a plurality of positions throughout the network.

The invention is also intended to determine performance weaknesses and bottlenecks in the network by producing configurable load on the network.

The method of testing a network can be facilitated using dedicated hardware structures, such as computer servers, e.g. a cloud-based server. Otherwise, the method can also at least partially be performed using a computer program product comprising instructions for causing a processor of a computer system to performing the monitoring activities. All (sub)steps can in principle be performed on a single processor. However, it is noted that at least one step can be performed on a separate processor. A processor can be loaded with a specific software module. Dedicated software modules can be provided, e.g. from the Internet.

The invention is not restricted to the embodiments described herein. It will be understood that many variants are possible.

These and other embodiments will be apparent for the person skilled in the art and are considered to fall within the scope of the invention as defined in the following claims. For the purpose of clarity and a concise description features are described herein as part of the same or separate embodiments. However, it will be appreciated that the scope of the invention may include embodiments having combinations of all or some of the features described.

List of references 100 Primary site with network connectivity 101 Secondary site with network connectivity 102 Enterprise Data Center 103 Site access router 104 Switching infrastructure 105 Wide-area network link

106 Enterprise Wide-area network WAN 111 Client workstation 112 Network performance test device 113 Server 114 Network Monitoring System 120 Firewall 121 Uplink to internet 122 Public Internet 130 Internet Servers / Services 140 Production data flow 141 Test data flow 200 Command repository 201 Data repository 202 Remote workstation 203 Control application 204 Display application 210 Command queue 213 Device registry 214 Test results database 220 Test device registration and update 221 Creating test device command queue 222 Retrieving test commands 223 Lookup of test device status 224 Delivering test commands 225 Delivering test results 226 Collecting test results for display 227 Batch generation of reports from test results 305 Diagnostic control application 324 Delivering diagnostic test results 325 Retrieving diagnostic test results 341 Diagnostic test data flow 405 Load test control application 424 Load test instructions and test results 425 Retrieving network component operational metrics

Claims

1. Testapparaat, omvattende hardware en software, a processor, geheugen en een interface voor verbinding met een netwerk, waarbij het testapparaat is ingericht om een combinatie van applicaties en services uit te voeren voor het uitvoeren van een testproces dat de stappen omvat van: - het inplannen en uitvoeren van testen voor netwerk prestatie assessment (mode A); - het uitvoeren van testen voor netwerk prestatie diagnose (mode B), en/of - het dienen als doelwit voor testen vanuit een enkelvoudig of meervoudig aantal andere testapparaten op het netwerk voor het testen van belastingen (mode C).A test device comprising hardware and software, a processor, memory and an interface for connection to a network, the test device being arranged to perform a combination of applications and services for performing a test process comprising the steps of: planning and conducting tests for network performance assessment (mode A); - performing tests for network performance diagnosis (mode B), and / or - serving as a target for testing from a single or multiple number of other test devices on the load testing network (mode C).

2. Testsysteem, omvattende een enkelvoudig of meervoudig aantal testapparaten volgens conclusie 1, voor verbinding aan verschillende netwerkelementen.2. Test system, comprising a single or multiple number of test devices according to claim 1, for connection to different network elements.

3. Testsysteem volgens conclusie 2, voorts omvattende een analyse system (203) dat is ingericht voor het verzamelen van testresultaten die zijn verkregen via het testapparaat volgens conclusie 1, en voor het analyseren van genoemde testresultaten voor het bepalen van basisgetallen en trends.The test system of claim 2, further comprising an analysis system (203) adapted to collect test results obtained through the test device of claim 1, and to analyze said test results for determining base numbers and trends.

4. Testsysteem volgens conclusie 3, waarbij de testresultaten periodiek worden geanalyseerd.The test system of claim 3, wherein the test results are analyzed periodically.

5. Testsysteem volgens één der voorgaande conclusies 3 4, waarbij het analyse systeem is ingericht voor het genereren van rapporten.5. Test system according to one of the preceding claims 3, 4, wherein the analysis system is arranged for generating reports.

6. Testsysteem volgens één der voorgaande conclusies 3*5, waarbij de analyse wordt uitgevoerd met gebruikmaking van vooraf ingestelde drempels voor het genereren van alerts voor het informeren van gebruikers over huidige of voorspelde toekomstige problemen op het netwerk.A test system according to any one of the preceding claims 3 * 5, wherein the analysis is performed using preset thresholds for generating alerts to inform users about current or predicted future problems on the network.

7. Werkwijze voor het testen van een netwerk, gebruik makend van een testapparaat volgens conclusies 1, waarbij de werkwijze de stappen omvat van; - een monitorstap (mode A) voor het periodiek bepalen van de prestatie van een computernetwerk door testverkeer vanaf het testapparaat in te voeren in het network en voor het meten van de round-trip vertraging van dit verkeer! - een diagnosestap (mode B) voor het assisteren van de diagnose van prestatie issues in een computernetwerk door testverkeer van het testapparaat in te voeren in het netwerk en de resultaten t.e verzamelen, en/of - een belastingstap (mode C) voor het bepalen van de capaciteit van een computernetwerk door hoge verkeer volumes tussen testapparaten in het netwerk te genereren en de effecten van dit verkeer op andere componenten te observeren.A method for testing a network, using a testing device according to claim 1, wherein the method comprises the steps of; - a monitoring step (mode A) for periodically determining the performance of a computer network by entering test traffic from the test device into the network and for measuring the round-trip delay of this traffic! - a diagnostic step (mode B) for assisting the diagnosis of performance issues in a computer network by entering test traffic from the test device into the network and collecting the results, and / or - a load step (mode C) for determining the capacity of a computer network by generating high traffic volumes between test devices in the network and observing the effects of this traffic on other components.

8. Computer programma product voor het testen van een netwerk met gebruik making van een testapparaat volgens conclusie 1, waarbij het computer programma product leesbare code omvat dat een processor in staat stelt de stappen uit te voeren van: een monitorstap (mode A) voor het periodiek bepalen van de prestatie van een computernetwerk door testverkeer vanaf het testapparaat in te voeren in het network en voor het meten van de round-trip vertraging van dit verkeer! - een diagnosestap (mode B) voor het assisteren van de diagnose van prestatie issues in een computernetwerk door testverkeer van het testapparaat in te voeren in het netwerk en de resultaten te verzamelen, en/of - een belastingstap (mode C) voor het bepalen van de capaciteit van een computernetwerk door hoge verkeer volumes tussen testapparaten in het netwerk te genereren en de effecten van dit verkeer op andere componenten te observeren.A computer program product for testing a network using a test device according to claim 1, wherein the computer program product comprises readable code that enables a processor to perform the steps of: a monitor step (mode A) for periodically determine the performance of a computer network by entering test traffic from the test device into the network and for measuring the round-trip delay of this traffic! - a diagnostic step (mode B) for assisting the diagnosis of performance issues in a computer network by entering test traffic from the test device into the network and collecting the results, and / or - a load step (mode C) for determining the capacity of a computer network by generating high traffic volumes between test devices in the network and observing the effects of this traffic on other components.