US20190116207A1 - Self-adjusting tiered caching system to optimize traffic performance and origin offload - Google Patents
Self-adjusting tiered caching system to optimize traffic performance and origin offload Download PDFInfo
- Publication number
- US20190116207A1 US20190116207A1 US16/214,895 US201816214895A US2019116207A1 US 20190116207 A1 US20190116207 A1 US 20190116207A1 US 201816214895 A US201816214895 A US 201816214895A US 2019116207 A1 US2019116207 A1 US 2019116207A1
- Authority
- US
- United States
- Prior art keywords
- swr
- server
- value
- cache
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/10—Architectures or entities
- H04L65/102—Gateways
- H04L65/1033—Signalling gateways
- H04L65/104—Signalling gateways in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/28—Flow control; Congestion control in relation to timing considerations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/30—Flow control; Congestion control in combination with information about buffer occupancy at either end or at transit nodes
-
- H04L65/4069—
-
- H04L65/4084—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H04L67/2842—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/2885—Hierarchically arranged intermediate devices, e.g. for hierarchical caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Definitions
- This application relates generally to overlay networking and, in particular, to techniques to optimize traffic performance and origin offload by an overlay network, such as a content delivery network.
- a distributed computer system is a “content delivery network” or “CDN” that is operated and managed by a service provider.
- the service provider typically provides the content delivery service on behalf of third parties (customers) who use the service provider's infrastructure.
- a distributed system of this type typically refers to a collection of autonomous computers linked by a network or networks, together with the software, systems, protocols and techniques designed to facilitate various services, such as content delivery, web application acceleration, or other support of outsourced origin site infrastructure.
- a CDN service provider typically provides service delivery through digital properties (such as a website), which are provisioned in a customer portal and then deployed to the network.
- a digital property typically is bound to one or more edge configurations that allow the service provider to account for traffic and bill its customer.
- SWR Stale-While-Revalidate
- RRC Internet Request for Comment
- the approach herein provides a tiered caching system (e.g., in an overlay network such as CDN) wherein SWR at an edge server therein is automatically set (preferably based on request rates), while SWR at a parent server therein has its SWR set at or near 0.
- a tiered caching system e.g., in an overlay network such as CDN
- SWR at an edge server therein is automatically set (preferably based on request rates), while SWR at a parent server therein has its SWR set at or near 0.
- FIG. 1 is a block diagram illustrating a known distributed computer system configured as a content delivery network (CDN);
- CDN content delivery network
- FIG. 2 is a representative CDN edge machine configuration
- FIG. 3 depicts a Stale-While-Revalidate (SWR) process depicting that a request hitting an edge server during an SWR period immediately returns a stale response while the server revalidates it asynchronously in the background, thereby hiding latency (both in the network and on the server) from clients; and
- SWR Stale-While-Revalidate
- FIG. 4 depicts a self-adjusting tiered caching system to optimize traffic performance and origin offload according to this disclosure.
- FIG. 1 illustrates a known distributed computer system configured as a content delivery network (CDN).
- CDN content delivery network
- the system has a set of machines 102 a - n distributed around the Internet.
- machines 102 Typically, most of the machines are servers located near the edge of the Internet, i.e., at or adjacent end user access networks.
- a network operations command center (NOCC) 104 manages operations of the various machines in the system.
- Third party sites such as web site 106 , offload delivery of content (e.g., HTML, embedded page objects, streaming media, software downloads, and the like) to the distributed computer system 100 and, in particular, to “edge” servers.
- content e.g., HTML, embedded page objects, streaming media, software downloads, and the like
- content providers offload their content delivery by aliasing (e.g., by a DNS CNAME) given content provider domains or sub-domains to domains that are managed by the service provider's authoritative domain name service. End users that desire the content are directed to the distributed computer system to obtain that content more reliably and efficiently.
- the distributed computer system may also include other infrastructure, such as a distributed data collection system 108 that collects usage and other data from the edge servers, aggregates that data across a region or set of regions, and passes that data to other back-end systems 110 , 112 , 114 and 116 to facilitate monitoring, logging, alerts, billing, management and other operational and administrative functions.
- Distributed network agents 118 monitor the network as well as the server loads and provide network, traffic and load data to a DNS query handling mechanism 115 , which is authoritative for content domains being managed by the CDN.
- a distributed data transport mechanism 120 may be used to distribute control information (e.g., metadata to manage content, to facilitate load balancing, and the like) to the edge servers.
- a given machine 200 comprises commodity hardware (e.g., an Intel Pentium processor) 202 running an operating system kernel (such as Linux or variant) 204 that supports one or more applications 206 a - n .
- operating system kernel such as Linux or variant
- given machines typically run a set of applications, such as an HTTP proxy 207 (sometimes referred to as a “global host” or “ghost” process), a name server 208 , a local monitoring process 210 , a distributed data collection process 212 , and the like.
- HTTP proxy 207 sometimes referred to as a “global host” or “ghost” process
- name server 208 a name server 208
- local monitoring process 210 e.g., a local monitoring process
- distributed data collection process e.g., a distributed data collection process
- the machine typically includes one or more media servers, such as a Windows Media Server (WMS) or Flash server, as required by the supported media formats.
- WMS Windows Media Server
- a CDN edge server is configured to provide one or more extended content delivery features, preferably on a domain-specific, customer-specific basis, preferably using configuration files that are distributed to the edge servers using a configuration system.
- a given configuration file preferably is XML-based and includes a set of content handling rules and directives that facilitate one or more advanced content handling features.
- the configuration file may be delivered to the CDN edge server via the data transport mechanism.
- U.S. Pat. No. 7,111,057 illustrates a useful infrastructure for delivering and managing edge server content control information, and this and other edge server control information can be provisioned by the CDN service provider itself, or (via an extranet or the like) the content provider customer who operates the origin server.
- the CDN may include a storage subsystem, such as described in U.S. Pat. No. 7,472,178, the disclosure of which is incorporated herein by reference.
- the CDN may operate a server cache hierarchy to provide intermediate caching of customer content; one such cache hierarchy subsystem is described in U.S. Pat. No. 7,376,716, the disclosure of which is incorporated herein by reference.
- the CDN may provide various technologies and techniques to accelerate traffic flow between an edge server, on the one hand, and a customer origin server, on the other. These technologies provide acceleration for many different types of interactions, e.g., delivery of dynamic content, edge server interactions with back-end origin infrastructures, and the like. Representative examples include, without limitation, the techniques described in U.S. Pat. No. 8,194,438 (overlay path selection optimization), and U.S. Pat. No. 8,477,837 (content pre-fetching). Other IP, TCP, UDP or application-layer optimizations may be implemented as well to facilitate such acceleration.
- the CDN may provide secure content delivery among a client browser, edge server and customer origin server in the manner described in U.S. Publication No. 20040093419. Secure content delivery as described therein enforces SSL-based links between the client and the edge server process, on the one hand, and between the edge server process and an origin server process, on the other hand. This enables an SSL-protected web page and/or components thereof to be delivered via the edge server.
- the CDN resources may be used to facilitate wide area network (WAN) acceleration services between enterprise data centers (which may be privately-managed) and third party software-as-a-service (SaaS) providers.
- WAN wide area network
- SaaS software-as-a-service
- a content provider identifies a content provider domain or sub-domain that it desires to have served by the CDN.
- the CDN service provider associates (e.g., via a canonical name, or CNAME) the content provider domain with an edge network (CDN) hostname, and the CDN provider then provides that edge network hostname to the content provider.
- CDN edge network
- those servers respond by returning the edge network hostname.
- the edge network hostname points to the CDN, and that edge network hostname is then resolved through the CDN name service. To that end, the CDN name service returns one or more IP addresses.
- the requesting client browser then makes a content request (e.g., via HTTP or HTTPS) to an edge server associated with the IP address.
- the request includes a host header that includes the original content provider domain or sub-domain.
- the edge server Upon receipt of the request with the host header, the edge server checks its configuration file to determine whether the content domain or sub-domain requested is actually being handled by the CDN. If so, the edge server applies its content handling rules and directives for that domain or sub-domain as specified in the configuration. These content handling rules and directives may be located within an XML-based “metadata” configuration file.
- a CDN may implement some form of the Stale-While-Revalidate (SWR) HTTP Cache-Control extension.
- SWR Stale-While-Revalidate
- this extension allows a cache to immediately return a stale response while it revalidates it in the background, thereby hiding latency (both in the network and on the server) from clients.
- CDNs want to strictly honor the TTL (time-to-live) of a website resource, they typically trigger this process when a request arrives at the edge server within a certain percentage of the overall TTL. For instance, if the TTL is one minute and the SWR is set to 10%, then a request hitting an edge server 0 to 6 seconds (10% of one minute) before the TTL expires triggers an asynchronous revalidation call to the origin server, while the request is served from the server's cache.
- FIG. 3 illustrates this known SWR process in the context of an edge cache (e.g., one of the edge servers in FIG. 2 ). While use of the SWR process increases overall performance (because the end user does not have to wait for this revalidation to happen), it also lowers off-load to the origin infrastructure, by reducing the actual TTL. In the example above, and for a busy edge machine that gets requests every second, the actual TTL would be around 54 or 55 sec, which lowers the offload (compared to no SWR process) by about 10%.
- the approach herein describes a system that automatically sets the SWR to optimize both the performance and the origin traffic offload.
- an overlay network 400 (such as a CDN comprises a tiered caching system comprising edge server and cache 404 positioned close to a requesting client machine 402 , together with a gateway server and cache 406 (the cache parent) positioned close to the CDN customer's origin server 408 .
- each of the CDN machines 404 and 406 is configured as shown in FIG. 2
- an end user machine 402 is a desktop, laptop, mobile device, set-top box, Smart television, Internet appliance, IoT device, or the like.
- the edge server 404 typically is located very close (e.g., within a few milliseconds of IP packet roundtrip time) to the client 402 , and the cache parent (the gateway server 406 ) may or may not be close to the origin server 408 .
- gateway servers act as “funnels” to edge servers to increase the likelihood that a cacheable resource will be served from the CDN infrastructure and not from the origin infrastructure.
- the SWR preferably is set to 0 at the gateway server 406 while, at the edge 404 , an automatic policy is implemented (preferably) on a per-server basis where the SWR is computed on-the-fly as a function of the request rate.
- the first factor (setting SWR to 0 at the gateway 406 ) is advantageous because if/when the request reaches the gateway, the performance penalty has already happened (to some degree), and therefore it is desired to maximize the origin offload there (by setting SWR to 0 so as to not trigger the asynchronous process).
- edge 404 would generate very high traffic between the edge servers and the gateways, and this would be detrimental to the CDN's ability to offer this system at scale and at a reasonable cost.
- a heuristic which is described below, can be used to provide a best user experience at an optimal CDN infrastructure cost.
- the notion of “best” or “optimal” here, however, is not intended to refer to some absolute value or characteristic but may be relative in nature and/or degree.
- R be the incoming request rate at the edge 404 at a given point in time (in hits/second), and SWR a percentage of the TTL (in seconds).
- SWR a percentage of the TTL (in seconds).
- SWR would be set to 10% if the request rate is one per second, to 5% if the request rate is two per second, and so forth.
- a constraint on the rule set forth in the above equation is that: 0 ⁇ SWR ⁇ 100.
- the system also sets SWR*TTL>Minimum (seconds), where the Minimum is defined either statically (e.g., one second), or as some multiplier of the observed latency to fetch the resource from the edge server.
- the automated policy may be disabled and a static SWR value (e.g., fetched from a configuration file specific to the resource) may be used.
- the approach thus leverages the typical CDN two-tier caching architecture, with SWR at the edge preferably set to 100/(TTL*R), and set to 0 at the gateway.
- SWR at the edge preferably set to 100/(TTL*R), and set to 0 at the gateway.
- the approach achieves an optimum performance and origin offload, while minimizing the CDN internal traffic.
- the notion of “optimum” should be not be taken as some absolute value but may be relative in nature and/or scope.
- a representative machine on which the software executes comprises commodity hardware, an operating system, an application runtime environment, and a set of applications or processes and associated data, that provide the functionality of a given system or subsystem.
- the functionality may be implemented in a standalone machine, or across a distributed set of machines.
- the functionality may be provided as a service, e.g., as a SaaS solution.
- the subject disclosure also relates to apparatus for performing the operations herein.
- This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer.
- a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including an optical disk, a CD-ROM, and a magnetic-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. While given components of the system have been described separately, one of ordinary skill will appreciate that some of the functions may be combined or shared in given instructions, program sequences, code portions, and the like.
- the functionality is implemented in an application layer solution, although this is not a limitation, as portions of the identified functions may be built into an operating system or the like that executes in a given machine.
- the functionality may be implemented with or in association with other application layer protocols besides HTTP or HTTPS, such as SSL VPN or TLS, or any other protocol having similar operating characteristics.
- HTTP or HTTPS such as SSL VPN or TLS, or any other protocol having similar operating characteristics.
- gateway is not intended to be limiting, as typically the forward server (the one closer to the origin infrastructure) is just another edge server in the CDN located in a different location/network.
- the SWR window as described and illustrated herein is an asynchronous (async) refresh window.
- the async refresh window starts before the TTL expires; another equally valid approach is to add the async refresh window after the TTL.
- the technique herein of optimizing the value of the async refresh window by making it dynamic applies equally well to both cases (i.e., whether the window is applied right before or right after the TTL expires).
- computing entity that may implement the client-side or server-side of the connection. Any computing entity (system, machine, device, program, process, utility, or the like) may act as the client or the server.
- CDN content that is delivered by the CDN.
- Typical examples include, without limitation, website objects (e.g., HTML, page objects, etc.), API-driven content, media content or content fragments, or the like.
- the techniques herein provide for improvements to another technology or technical field, namely, content delivery systems, as well as improvements to the functioning of edge servers within such systems.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
- This application relates generally to overlay networking and, in particular, to techniques to optimize traffic performance and origin offload by an overlay network, such as a content delivery network.
- Distributed computer systems are well-known in the prior art. One such distributed computer system is a “content delivery network” or “CDN” that is operated and managed by a service provider. The service provider typically provides the content delivery service on behalf of third parties (customers) who use the service provider's infrastructure. A distributed system of this type typically refers to a collection of autonomous computers linked by a network or networks, together with the software, systems, protocols and techniques designed to facilitate various services, such as content delivery, web application acceleration, or other support of outsourced origin site infrastructure. A CDN service provider typically provides service delivery through digital properties (such as a website), which are provisioned in a customer portal and then deployed to the network. A digital property typically is bound to one or more edge configurations that allow the service provider to account for traffic and bill its customer.
- To maximize end user experience, content delivery networks often implement some form of the Stale-While-Revalidate (“SWR”) HTTP Cache-Control extension. The SWR extension is described in Internet Request for Comment (RFC) 5861, titled “HTTP stale controls.” The functionality allows a cache to immediately return a stale response while it revalidates it in the background, thereby hiding latency (both in the network and on the server) from clients.
- The approach herein provides a tiered caching system (e.g., in an overlay network such as CDN) wherein SWR at an edge server therein is automatically set (preferably based on request rates), while SWR at a parent server therein has its SWR set at or near 0. Collectively, these SWR settings provide for a self-adjusting tiered caching system that optimizes both end user performance and the origin traffic offload.
- The foregoing has outlined some of the more pertinent features of the disclosed subject matter. These features should be construed to be merely illustrative. Many other beneficial results can be attained by applying the disclosed subject matter in a different manner or by modifying the subject matter as will be described.
- For a more complete understanding of the subject disclosure and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
-
FIG. 1 is a block diagram illustrating a known distributed computer system configured as a content delivery network (CDN); -
FIG. 2 is a representative CDN edge machine configuration; -
FIG. 3 depicts a Stale-While-Revalidate (SWR) process depicting that a request hitting an edge server during an SWR period immediately returns a stale response while the server revalidates it asynchronously in the background, thereby hiding latency (both in the network and on the server) from clients; and -
FIG. 4 depicts a self-adjusting tiered caching system to optimize traffic performance and origin offload according to this disclosure. -
FIG. 1 illustrates a known distributed computer system configured as a content delivery network (CDN). The system has a set ofmachines 102 a-n distributed around the Internet. Typically, most of the machines are servers located near the edge of the Internet, i.e., at or adjacent end user access networks. A network operations command center (NOCC) 104 manages operations of the various machines in the system. Third party sites, such asweb site 106, offload delivery of content (e.g., HTML, embedded page objects, streaming media, software downloads, and the like) to thedistributed computer system 100 and, in particular, to “edge” servers. Typically, content providers offload their content delivery by aliasing (e.g., by a DNS CNAME) given content provider domains or sub-domains to domains that are managed by the service provider's authoritative domain name service. End users that desire the content are directed to the distributed computer system to obtain that content more reliably and efficiently. Although not shown in detail, the distributed computer system may also include other infrastructure, such as a distributeddata collection system 108 that collects usage and other data from the edge servers, aggregates that data across a region or set of regions, and passes that data to other back-end systems network agents 118 monitor the network as well as the server loads and provide network, traffic and load data to a DNSquery handling mechanism 115, which is authoritative for content domains being managed by the CDN. A distributeddata transport mechanism 120 may be used to distribute control information (e.g., metadata to manage content, to facilitate load balancing, and the like) to the edge servers. - As illustrated in
FIG. 2 , a givenmachine 200 comprises commodity hardware (e.g., an Intel Pentium processor) 202 running an operating system kernel (such as Linux or variant) 204 that supports one ormore applications 206 a-n. To facilitate content delivery services, for example, given machines typically run a set of applications, such as an HTTP proxy 207 (sometimes referred to as a “global host” or “ghost” process), aname server 208, alocal monitoring process 210, a distributeddata collection process 212, and the like. For streaming media, the machine typically includes one or more media servers, such as a Windows Media Server (WMS) or Flash server, as required by the supported media formats. - A CDN edge server is configured to provide one or more extended content delivery features, preferably on a domain-specific, customer-specific basis, preferably using configuration files that are distributed to the edge servers using a configuration system. A given configuration file preferably is XML-based and includes a set of content handling rules and directives that facilitate one or more advanced content handling features. The configuration file may be delivered to the CDN edge server via the data transport mechanism. U.S. Pat. No. 7,111,057 illustrates a useful infrastructure for delivering and managing edge server content control information, and this and other edge server control information can be provisioned by the CDN service provider itself, or (via an extranet or the like) the content provider customer who operates the origin server.
- The CDN may include a storage subsystem, such as described in U.S. Pat. No. 7,472,178, the disclosure of which is incorporated herein by reference.
- The CDN may operate a server cache hierarchy to provide intermediate caching of customer content; one such cache hierarchy subsystem is described in U.S. Pat. No. 7,376,716, the disclosure of which is incorporated herein by reference.
- The CDN may provide various technologies and techniques to accelerate traffic flow between an edge server, on the one hand, and a customer origin server, on the other. These technologies provide acceleration for many different types of interactions, e.g., delivery of dynamic content, edge server interactions with back-end origin infrastructures, and the like. Representative examples include, without limitation, the techniques described in U.S. Pat. No. 8,194,438 (overlay path selection optimization), and U.S. Pat. No. 8,477,837 (content pre-fetching). Other IP, TCP, UDP or application-layer optimizations may be implemented as well to facilitate such acceleration.
- The CDN may provide secure content delivery among a client browser, edge server and customer origin server in the manner described in U.S. Publication No. 20040093419. Secure content delivery as described therein enforces SSL-based links between the client and the edge server process, on the one hand, and between the edge server process and an origin server process, on the other hand. This enables an SSL-protected web page and/or components thereof to be delivered via the edge server.
- As an overlay, the CDN resources may be used to facilitate wide area network (WAN) acceleration services between enterprise data centers (which may be privately-managed) and third party software-as-a-service (SaaS) providers.
- In a typical operation, a content provider identifies a content provider domain or sub-domain that it desires to have served by the CDN. The CDN service provider associates (e.g., via a canonical name, or CNAME) the content provider domain with an edge network (CDN) hostname, and the CDN provider then provides that edge network hostname to the content provider. When a DNS query to the content provider domain or sub-domain is received at the content provider's domain name servers, those servers respond by returning the edge network hostname. The edge network hostname points to the CDN, and that edge network hostname is then resolved through the CDN name service. To that end, the CDN name service returns one or more IP addresses. The requesting client browser then makes a content request (e.g., via HTTP or HTTPS) to an edge server associated with the IP address. The request includes a host header that includes the original content provider domain or sub-domain. Upon receipt of the request with the host header, the edge server checks its configuration file to determine whether the content domain or sub-domain requested is actually being handled by the CDN. If so, the edge server applies its content handling rules and directives for that domain or sub-domain as specified in the configuration. These content handling rules and directives may be located within an XML-based “metadata” configuration file.
- By way of additional background, to maximize their end user experience, a CDN may implement some form of the Stale-While-Revalidate (SWR) HTTP Cache-Control extension. As noted above, this extension allows a cache to immediately return a stale response while it revalidates it in the background, thereby hiding latency (both in the network and on the server) from clients.
- Because CDNs want to strictly honor the TTL (time-to-live) of a website resource, they typically trigger this process when a request arrives at the edge server within a certain percentage of the overall TTL. For instance, if the TTL is one minute and the SWR is set to 10%, then a request hitting an edge server 0 to 6 seconds (10% of one minute) before the TTL expires triggers an asynchronous revalidation call to the origin server, while the request is served from the server's cache.
-
FIG. 3 illustrates this known SWR process in the context of an edge cache (e.g., one of the edge servers inFIG. 2 ). While use of the SWR process increases overall performance (because the end user does not have to wait for this revalidation to happen), it also lowers off-load to the origin infrastructure, by reducing the actual TTL. In the example above, and for a busy edge machine that gets requests every second, the actual TTL would be around 54 or 55 sec, which lowers the offload (compared to no SWR process) by about 10%. - Thus, there is a tension between, on the one hand, enhancing end user experience (through faster resource download from cache) and, on the other hand, origin traffic offload. This raises the question of how to optimally set the SWR value?
- The approach herein describes a system that automatically sets the SWR to optimize both the performance and the origin traffic offload.
- As will be seen, the technique of this disclosure takes advantage of two factors: (1) the ability of an edge server to compute a rate of incoming requests for a given resource; and (2) the ability of a CDN to leverage a tiered caching architecture, such as depicted in
FIG. 4 (see, also U.S. Pat. No. 7,376,716, referenced above). As depicted inFIG. 4 , an overlay network 400 (such as a CDN comprises a tiered caching system comprising edge server andcache 404 positioned close to a requestingclient machine 402, together with a gateway server and cache 406 (the cache parent) positioned close to the CDN customer'sorigin server 408. Typically, each of theCDN machines FIG. 2 , and anend user machine 402 is a desktop, laptop, mobile device, set-top box, Smart television, Internet appliance, IoT device, or the like. - In a tiered architecture of this type, the
edge server 404 typically is located very close (e.g., within a few milliseconds of IP packet roundtrip time) to theclient 402, and the cache parent (the gateway server 406) may or may not be close to theorigin server 408. With this type of cache hierarchy within the overlay network, gateway servers act as “funnels” to edge servers to increase the likelihood that a cacheable resource will be served from the CDN infrastructure and not from the origin infrastructure. - According to this disclosure, the SWR preferably is set to 0 at the
gateway server 406 while, at theedge 404, an automatic policy is implemented (preferably) on a per-server basis where the SWR is computed on-the-fly as a function of the request rate. The first factor (setting SWR to 0 at the gateway 406) is advantageous because if/when the request reaches the gateway, the performance penalty has already happened (to some degree), and therefore it is desired to maximize the origin offload there (by setting SWR to 0 so as to not trigger the asynchronous process). On the other hand, and for maximum performance, it is desired to set SWR as high as possible at theedge 404, especially because the gateway servers (with the SWR=0 setting) are optimized to ensure optimum origin offload. Yet, applying a simplistic 100% value universally at theedge 404 would generate very high traffic between the edge servers and the gateways, and this would be detrimental to the CDN's ability to offer this system at scale and at a reasonable cost. By taking into account the hit rate at the edge server, however, a heuristic, which is described below, can be used to provide a best user experience at an optimal CDN infrastructure cost. The notion of “best” or “optimal” here, however, is not intended to refer to some absolute value or characteristic but may be relative in nature and/or degree. - The following describes a preferred heuristic. In particular, let R be the incoming request rate at the
edge 404 at a given point in time (in hits/second), and SWR a percentage of the TTL (in seconds). (The edge machine is assumed to be receiving requests from a large number of clients, as typically the machine caches and serves content for a large number of content providers that use the CDN service). Assuming a uniform request temporal distribution, and to guarantee that a last hit before TTL is reached falls within SWR, a preferred heuristic for the edge is then as follows: -
SWR (% of TTL)=100/(TTL*R). - For instance, if the TTL is 10 seconds, SWR would be set to 10% if the request rate is one per second, to 5% if the request rate is two per second, and so forth.
- A constraint on the rule set forth in the above equation is that: 0<SWR<100. To ensure with good confidence that the response to the asynchronous call comes back before the TTL expires, preferably the system also sets SWR*TTL>Minimum (seconds), where the Minimum is defined either statically (e.g., one second), or as some multiplier of the observed latency to fetch the resource from the edge server. In light of the point above, for very short TTLs or other corner use cases (e.g. the server is being placed online and has no request history), the automated policy may be disabled and a static SWR value (e.g., fetched from a configuration file specific to the resource) may be used.
- The approach thus leverages the typical CDN two-tier caching architecture, with SWR at the edge preferably set to 100/(TTL*R), and set to 0 at the gateway. The approach achieves an optimum performance and origin offload, while minimizing the CDN internal traffic. Once again, the notion of “optimum” should be not be taken as some absolute value but may be relative in nature and/or scope.
- More generally, the techniques described herein are provided using a set of one or more computing-related entities (systems, machines, processes, programs, libraries, functions, or the like) that together facilitate or provide the described functionality described above. In a typical implementation, a representative machine on which the software executes comprises commodity hardware, an operating system, an application runtime environment, and a set of applications or processes and associated data, that provide the functionality of a given system or subsystem. As described, the functionality may be implemented in a standalone machine, or across a distributed set of machines. The functionality may be provided as a service, e.g., as a SaaS solution.
- While the above describes a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary, as alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, or the like. References in the specification to a given embodiment indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic.
- While the disclosed subject matter has been described in the context of a method or process, the subject disclosure also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including an optical disk, a CD-ROM, and a magnetic-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic or optical card, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. While given components of the system have been described separately, one of ordinary skill will appreciate that some of the functions may be combined or shared in given instructions, program sequences, code portions, and the like.
- Preferably, the functionality is implemented in an application layer solution, although this is not a limitation, as portions of the identified functions may be built into an operating system or the like that executes in a given machine.
- The functionality may be implemented with or in association with other application layer protocols besides HTTP or HTTPS, such as SSL VPN or TLS, or any other protocol having similar operating characteristics.
- The term “gateway” is not intended to be limiting, as typically the forward server (the one closer to the origin infrastructure) is just another edge server in the CDN located in a different location/network.
- Generalizing, it should be appreciated that the SWR window as described and illustrated herein is an asynchronous (async) refresh window. In the SWR implementation as described, the async refresh window starts before the TTL expires; another equally valid approach is to add the async refresh window after the TTL. Thus, the technique herein of optimizing the value of the async refresh window by making it dynamic (preferably to match the rate of incoming requests) applies equally well to both cases (i.e., whether the window is applied right before or right after the TTL expires).
- The notion of setting the SWR to zero at the gateway does not require that the value be an absolute 0 but also includes a value that is “substantially zero.”
- Generalizing further, there is no limitation on the type of computing entity that may implement the client-side or server-side of the connection. Any computing entity (system, machine, device, program, process, utility, or the like) may act as the client or the server.
- While the technique above has been described in the context of a tiered caching system, it should be appreciated that the approach also provides significant advantages in a single tier system (edge servers only). In this single tier caching context, the SWR is dynamically computed in the same way as described above, with the only difference being that the request from the edge server would go directly to the origin and not thru a parent (gateway) server.
- There is no limitation on the type of content that is delivered by the CDN. Typical examples include, without limitation, website objects (e.g., HTML, page objects, etc.), API-driven content, media content or content fragments, or the like.
- The techniques herein provide for improvements to another technology or technical field, namely, content delivery systems, as well as improvements to the functioning of edge servers within such systems.
- What is claimed is as follows:
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/214,895 US20190116207A1 (en) | 2014-12-30 | 2018-12-10 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201462097630P | 2014-12-30 | 2014-12-30 | |
US14/981,997 US10154068B2 (en) | 2014-12-30 | 2015-12-29 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
US16/214,895 US20190116207A1 (en) | 2014-12-30 | 2018-12-10 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/981,997 Continuation US10154068B2 (en) | 2014-12-30 | 2015-12-29 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
Publications (1)
Publication Number | Publication Date |
---|---|
US20190116207A1 true US20190116207A1 (en) | 2019-04-18 |
Family
ID=56165644
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/981,997 Active 2036-09-15 US10154068B2 (en) | 2014-12-30 | 2015-12-29 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
US16/214,895 Abandoned US20190116207A1 (en) | 2014-12-30 | 2018-12-10 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/981,997 Active 2036-09-15 US10154068B2 (en) | 2014-12-30 | 2015-12-29 | Self-adjusting tiered caching system to optimize traffic performance and origin offload |
Country Status (1)
Country | Link |
---|---|
US (2) | US10154068B2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020263198A1 (en) * | 2019-06-26 | 2020-12-30 | Medianova Internet Hizmetleri Ve Ticaret Anonim Sirketi | Performance enhanced cdn service |
CN112422644A (en) * | 2020-11-02 | 2021-02-26 | 北京邮电大学 | Method and system for unloading computing tasks, electronic device and storage medium |
US11445225B2 (en) * | 2020-10-27 | 2022-09-13 | Akamai Technologies, Inc. | Measuring and improving origin offload and resource utilization in caching systems |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10506027B2 (en) * | 2014-08-27 | 2019-12-10 | Tensera Networks Ltd. | Selecting a content delivery network |
US11134053B2 (en) | 2019-07-17 | 2021-09-28 | International Business Machines Corporation | Determining cache time to live in an object storage system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070192444A1 (en) * | 2002-09-16 | 2007-08-16 | Emmanuel Ackaouy | Apparatus and method for a proxy cache |
US20140095804A1 (en) * | 2012-10-01 | 2014-04-03 | Edgecast Networks, Inc. | Efficient Cache Validation and Content Retrieval in a Content Delivery Network |
US20140207912A1 (en) * | 2013-01-18 | 2014-07-24 | Limelight Networks, Inc. | Selective content pre-warming in content delivery networks based on user actions and content categorizations |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7111057B1 (en) | 2000-10-31 | 2006-09-19 | Akamai Technologies, Inc. | Method and system for purging content from a content delivery network |
US20020143798A1 (en) | 2001-04-02 | 2002-10-03 | Akamai Technologies, Inc. | Highly available distributed storage system for internet content with storage site redirection |
US7133905B2 (en) | 2002-04-09 | 2006-11-07 | Akamai Technologies, Inc. | Method and system for tiered distribution in a content delivery network |
US20040093419A1 (en) | 2002-10-23 | 2004-05-13 | Weihl William E. | Method and system for secure content delivery |
US9158845B1 (en) * | 2004-04-29 | 2015-10-13 | Aol Inc. | Reducing latencies in web page rendering |
US8447837B2 (en) | 2005-12-30 | 2013-05-21 | Akamai Technologies, Inc. | Site acceleration with content prefetching enabled through customer-specific configurations |
US7660296B2 (en) | 2005-12-30 | 2010-02-09 | Akamai Technologies, Inc. | Reliable, high-throughput, high-performance transport and routing mechanism for arbitrary data flows |
US8180720B1 (en) * | 2007-07-19 | 2012-05-15 | Akamai Technologies, Inc. | Content delivery network (CDN) cold content handling |
US7698411B2 (en) * | 2007-08-22 | 2010-04-13 | International Business Machines Corporation | Selectively delivering cached content or processed content to clients based upon a result completed percentage |
WO2009126839A2 (en) * | 2008-04-09 | 2009-10-15 | Level 3 Communications, Llc | Content delivery in a network |
US9742858B2 (en) * | 2011-12-23 | 2017-08-22 | Akamai Technologies Inc. | Assessment of content delivery services using performance measurements from within an end user client application |
-
2015
- 2015-12-29 US US14/981,997 patent/US10154068B2/en active Active
-
2018
- 2018-12-10 US US16/214,895 patent/US20190116207A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070192444A1 (en) * | 2002-09-16 | 2007-08-16 | Emmanuel Ackaouy | Apparatus and method for a proxy cache |
US20140095804A1 (en) * | 2012-10-01 | 2014-04-03 | Edgecast Networks, Inc. | Efficient Cache Validation and Content Retrieval in a Content Delivery Network |
US20140207912A1 (en) * | 2013-01-18 | 2014-07-24 | Limelight Networks, Inc. | Selective content pre-warming in content delivery networks based on user actions and content categorizations |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020263198A1 (en) * | 2019-06-26 | 2020-12-30 | Medianova Internet Hizmetleri Ve Ticaret Anonim Sirketi | Performance enhanced cdn service |
US11445225B2 (en) * | 2020-10-27 | 2022-09-13 | Akamai Technologies, Inc. | Measuring and improving origin offload and resource utilization in caching systems |
US11743513B2 (en) | 2020-10-27 | 2023-08-29 | Akamai Technologies, Inc. | Measuring and improving origin offload and resource utilization in caching systems |
CN112422644A (en) * | 2020-11-02 | 2021-02-26 | 北京邮电大学 | Method and system for unloading computing tasks, electronic device and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US20160191393A1 (en) | 2016-06-30 |
US10154068B2 (en) | 2018-12-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10237363B2 (en) | Content delivery network request handling mechanism with cached control information | |
US20190116207A1 (en) | Self-adjusting tiered caching system to optimize traffic performance and origin offload | |
US11146615B2 (en) | Multi-domain configuration handling in an edge network server | |
US11194719B2 (en) | Cache optimization | |
US10791201B2 (en) | Server initiated multipath content delivery | |
US9418353B2 (en) | Methods and systems for delivering content to differentiated client devices | |
US11483176B2 (en) | Traffic delivery using anycast and end user-based mapping in an overlay network | |
JP5874146B2 (en) | Extension of content distribution network (CDN) to mobile or wired networks | |
US20160269477A1 (en) | Network resource modification for higher network connection concurrence | |
US20120209942A1 (en) | System combining a cdn reverse proxy and an edge forward proxy with secure connections | |
US20140019576A1 (en) | Intelligent edge caching | |
US10230811B2 (en) | System and method for providing an adjunct device in a content distribution network | |
US10440142B2 (en) | Automated TTL adjustment using cache performance and purge data | |
US20230024656A1 (en) | Server-side prediction of media client steady state | |
US20200320154A1 (en) | A webpage loading method, webpage loading system and server |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: AKAMAI TECHNOLOGIES, INC., MASSACHUSETTS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LERMANT, PIERRE;REEL/FRAME:048274/0530 Effective date: 20150115 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |