Transcription
W H I T E PA PE RCloudflare CDN ReferenceArchitecture1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com
INDEXClick to skip to each sectionOverview .3Traditional challenges deploying web applications . 4-5How a CDN tackles web application challenges .6Introducing the Cloudflare CDN .7Cloudflare CDN architecture and design .8-9Argo Tiered Cache . 9Cloudflare Tiered Cache Topologies .10Traffic flow: Argo Tiered Cache, Smart Tiered Cache Topology . 11-12Argo Smart Routing .Traffic Flow: Argo Tiered Cache, Smart Tiered Cache Topology .with Argo Smart Routing1213-14Summary . 151 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com2
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R EOverviewEvery day, users of the Internet enjoy the benefits of performance and reliabilityprovided by content delivery networks (CDNs). CDNs have become a must-haveto combat latency and a requirement for any major company delivering content tousers on the Internet. While providing performance and reliability for customers,CDNs also enable companies to further secure their applications and cut costs.This document discusses the traditional challenges customers face with webapplications, how the Cloudflare CDN resolves these challenges, and CDNarchitecture and design.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com3
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraditional challenges deploying web applicationsThe servers hosting the websites are called originservers. When clients access a website, they make arequest for resources from the server. Navigating toone website can generate hundreds of requests fromthe browser for HTML, CSS, images, videos, etc. Withversions of HTTP prior to HTTP/2, each of these HTTPrequests would also require a new TCP connection.Enhancements in HTTP/2 allow for multiplexingmultiple requests to the same server over a single TCPconnection, thus saving server resources. However,compute and network resources are still consumedas servers respond to these requests. As more clientsaccess the website, the following can result:Over the last several years, especially with the advent ofthe COVID-19 pandemic and the focus on remote work,there has been a significant growth in Internet traffic,further growing the need to efficiently manage networktraffic, cut latency, and increase performance.Companies running their applications in the cloud oron-premise are faced with the challenges of:1. Implementing solutions to increase performance2. As demand grows, scaling out their architecture tomeet availability and redundancy concerns3. Securing their environments and applications fromgrowing Internet threats4. Reining in growing costs related to doing all of theabove The origin server starts to become overloaded withrequests, impacting availability; companies startlooking at scaling out to handle the additional load As each request has to make its way to the originserver, performance and user experience is impacteddue to latency The latency for end users becomes proportional tothe distance between the client and origin server,thus resulting in varying experiences based on clientlocation As origin servers respond to the increasing requests,bandwidth, egress, and compute costs increasedrastically Even as customers scale out to handle the increaseddemand in traffic, they are left exposed to bothinfrastructure-level and application-level distributeddenial-of-service (DDoS) attacksWith companies serving customers across the globe,the above challenges require a significant undertaking.Traditionally, a website/application is deployed centrallyand replicated to another region for availability, or thewebsite/application is deployed across a handful ofservers, sometimes across multiple data centers forresiliency.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com4
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraditional challenges deploying web applications(continued)In Figure 1 below, there is no CDN present and there isan origin server sitting in the US. As clients access thewebsite, the first step is DNS resolution, typically doneby the user’s ISP. The next step is the HTTP request sentdirectly to the origin server. The user experience willvary depending on their location. For example, you cansee the latency is much lower for users in the US, wherethe origin server is located. For users outside the US, thelatency increases, thus resulting in a higher round-triptime (RTT).From a security perspective, the origin server is alsovulnerable to DDoS attacks at both the infrastructureand application layer. A DDoS attack could be initiatedfrom a botnet sending millions of requests to the originserver, consuming resources and preventing it fromserving legitimate clients.Further, in terms of resiliency, if the origin servertemporarily goes offline, all content is inaccessibleto users.As more clients make requests to the origin server, theload on the network and server increases, resultingin higher latency and higher costs for resource andbandwidth use.1. DNS resolution for websiteDNSClient (India)InternetClient (Germany)150 ms2. HTTP request to website/origin-server231 ms2.08 msClient (US)InternetOrigin-server (US)Figure 1: HTTP Request with no CDN1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com5
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R EHow a CDN tackles web application challengesA CDN helps address the challenges customers facearound latency, performance, availability, redundancy,security, and costs. A CDN’s core goal is to decreaselatency and increase performance for websites andapplications by caching content as close as possible toend users or those accessing the content.CDNs decrease latency and increase performance byhaving many data center locations across the globe thatcache the content from the origin. The goal is to havecontent cached as close as possible to users, so contentis cached at the edge of the CDN provider’s network.The impact this has: Improved website load timeInstead of every client making a request to theorigin server, which could be located a considerabledistance away, the request is routed to a local serverthat responds with cached content, thus decreasinglatency and increasing overall performance.Regardless of where the origin server and clients arelocated, performance will be more consistent for allusers, as the CDN will serve locally cached contentwhen possible. Increased content availability and redundancyBecause every client request no longer needs tobe sent to the origin server, CDNs provide not onlyperformance benefits, but also availability andredundancy. Requests are load balanced over localservers with cached content; these servers respondto local requests, significantly decreasing overallload on the origin server. The origin server only iscontacted when needed (when content is not cachedor for dynamic non-cacheable content). Improved website securityA CDN acts as a reverse proxy and sits in front oforigin servers. Thus it can provide enhanced securitysuch as DDoS mitigation, improvements to securitycertificates, and other optimizations. Reduced bandwidth costsBecause CDNs use cached content to respond torequests, the number of requests sent to the originserver is reduced, thus also reducing associatedbandwidth costs.An important difference in some CDN implementations ishow they route traffic to the respective local CDN nodes.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comRouting requests to CDN nodes can be done via twodifferent methods:1. DNS unicast routingIn this method, recursive DNS queries redirectrequests to CDN nodes; the client’s DNS resolverforwards requests to the CDN’s authoritativenameserver. CDNs based on DNS unicast routingare not ideal in that clients may be geographicallydispersed from the DNS resolver. Decisions onclosest-proximity CDN nodes are based on the client’sDNS server instead of client’s IP address.Also, if any changes are needed for the DNSresponse, there is a dependency on DNS time to live(TTL) expiration.Further, since DNS routing uses unicast addresses,traffic is routed directly to a specific node, creatingpossible concerns when there are traffic spikes, as ina DDoS attack.Another challenge with DNS-based CDNs is that DNSis not very graceful upon failover. Typically a newsession or application must be started for the DNSresolver with a different IP address to take over.2. Anycast routingThe Cloudflare CDN, which is discussed in moredetail in the next section, uses Anycast routing.Anycast allows for nodes on a network to have thesame IP address. The same IP address is announcedfrom multiple nodes in different locations, and clientredirection is handled via the Internet’s routingprotocol, BGP.Using an Anycast-based CDN has several advantages: Incoming traffic is routed to the nearest data centerwith the capacity to process the requests efficiently. Availability and redundancy is inherently provided.Since multiple nodes have the same IP address, ifone node were to fail, requests are simply routed toanother node in close proximity. Because Anycast distributes traffic across multipledata centers, it increases the overall surface area,thus preventing any one location from becomingoverwhelmed with requests. For this reason, Anycastnetworks are very resilient to DDoS attacks.6
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R EIntroducing the Cloudflare CDNCloudflare provides a Software as a Service (SaaS) model for CDN. With Cloudflare’s SaaSmodel, customers benefit from the Cloudflare CDN without having to manage or maintain anyinfrastructure or software.The benefits of the Cloudflare CDN can be attributed to the below two points, discussed in moredetail in this section.1. CDNs inherently increase performance by caching content on servers close to the user2. The unique Cloudflare architecture and integrated ecosystemFigure 2 shows a simplified view of the Cloudflare CDN. Clients are receiving their responseback from a server on Cloudflare’s global Anycast edge network closest to where the clients arelocated, thus drastically reducing the latency and RTT. The diagram depicts a consistent end-userexperience regardless of the physical location of the clients and origin.Client (India)Client (Germany)Internet2.28 ms1. DNS resolution via Cloudflare Anycast DNS Resolver2. Request sent to closest Cloudflare data center andserver that will respond with local cached content2.18 msIndiaCacheGermanyDNS,CDN,WAF,etcUS2.08 msClient (US)Origin-server (US)Cloudflare Global AnycastEdge NetworkFigure 2: HTTP request to Cloudflare CDN with Anycast1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com7
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ECloudflare CDN architecture and designFigure 3 is a view of the Cloudflare CDN on the global Anycast edge network. In addition tousing Anycast for network performance and resiliency, the Cloudflare CDN leverages ArgoTiered Cache to deliver optimized results while saving costs for customers. Customers can alsoenable Argo Smart Routing to find the fastest network path to route requests to the origin server.These capabilities are discussed in detail in the remainder of this document.Global Anycast Edge NetworkBGP AnnouncementCloudflare Anycast IPsCustomer IPsData Center 1Lower tierCDoSCDNClient / End UserConsumptionre NetwodflarkuloDNSData Center 4CacheData Center 2Lower tierCustomer Network /Origin ServersUpper tierArgo TieredCacheDoSDNSCDNCacheData Center 3Lower tierData CentersPublic CloudsFigure 3: Cloudflare CDN with Argo Tiered Cache on global Anycast edge networkIn the above diagram, there are a few important keypoints to understand about the Cloudflare CDN andthe global Anycast edge network it resides on: An important differentiator is that Cloudflareutilizes one global network and runs every serviceon every server in every Cloudflare data center,thus providing end users the closest proximityto Cloudflare’s services, with the highest scale,resiliency, and performance. Cloudflare is a reverse proxy, meaning it receivesrequests from clients and proxies the requests backto the customer’s origin servers.Thus, every request traverses through Cloudflare’snetwork before reaching the customer’s network.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comSince Cloudflare has hardened and protectedits infrastructure at the edge (ingress), allcustomers are consequently also protected frominfrastructure-level and volumetric DDoS attacks.Requests and traffic must go through the protectedCloudflare network before reaching the customer’sorigin server. The Cloudflare CDN leverages the Cloudflare globalAnycast edge network. Thus the incoming requestis routed to and answered by the node closest tothe user (eyeball). The inherent benefits of Anycast are decreasedlatency, network resiliency, higher availability,and increased security due to larger surface areafor absorbing both legitimate traffic loads andDDoS attacks.8
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ECloudflare CDN architecture and design (continued)Cloudflare’s global Anycast edge networkspans more than 250 cities across 100 countries, reaching 95% of the world’s Internetconnected population within 50 millisecondswhile providing 100 Tbps of network capacityand DDoS protection capability. Argo uses optimized routing and cachingtechnology across the Cloudflare network todeliver responses to users more quickly, reliably,and securely. Argo includes Smart Routing andTiered Cache. Cloudflare leverages Argo toprovide an enhanced CDN solution. Edge nodes within the Cloudflare networkcache content from the origin server and areable to respond to requests via a cached copy.Cloudflare also provides DNS, DDoS protection,WAF, and other performance, reliability,and security services using the same edgearchitecture.Argo Tiered CacheOnce a site is onboarded, standard caching is configuredby default. With standard caching, each data center actsas a direct reverse proxy for the origin servers. A cachemiss in any data center results in a request being sent tothe origin server from the ingress data center.Although standard caching works, it is not the mostoptimal design — cached content closer to the clientmay already exist in other Cloudflare data centers, andorigin servers are sometimes unnecessarily overloadedas a result. Thus, it is best to enable Argo Tiered Cache,which is included with every Cloudflare plan. With ArgoTiered Cache, certain data centers are reverse proxies tothe origin for other data centers, resulting in more cachehits and faster response times.Argo Tiered Cache leverages the scale of Cloudflare’snetwork to minimize requests to customer origins. Whena request comes into a Cloudflare data center, if therequested content is not locally cached, other Cloudflaredata centers are checked for the cached content.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comCloudflare data centers have shorter distances and fasterpaths between them than the connections betweendata centers and customer origin servers, optimizing theresponse to the client with a significant improvementin cache hit ratio. The Cloudflare CDN leverages ArgoSmart Routing data to determine the best upper tierdata centers to use for Argo Tiered Cache. Argo SmartRouting can also be enabled as an add-on to provide thefastest paths between data centers and origin servers forcache misses and other types of dynamic traffic.The Cloudflare CDN allows customers to configure tieredcaching. Note that depending on the Cloudflare plan,different topologies are available for Argo Tiered Cache.By default, tiered caching is disabled and can be enabledunder the caching tab of the main menu. 9
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R EArgo Tiered Cache TopologiesThe different cache topologies allow customers to control how Cloudflare interacts with originservers to help ensure higher cache hit ratios, fewer origin connections, and reduced latency.Argo Tiered Cache TopologiesSmart Tiered Cache TopologyGeneric Global Tiered TopologyCustom Tiered Cache Topology(All plans)(Enterprise Only)(Enterprise Only) Recommended for mostdeployments. It is the defaultconfiguration once TieredCache is enabled. Ideal for customers who want toleverage CDN for performancebut minimize requests toorigin servers and bandwidthutilization between Cloudflareand origin servers. Recommended for those whohave high traffic that is spreadacross the globe and desire thehighest cache usage and bestperformance possible. Recommended for customerswho have additional dataon their user base and havespecific geographic regionsthey would like to focus on. Generic Global Tiered Topologybalances between cacheefficiency and latency. InstructsCloudflare to use all Tier 1 datacenters as upper tiers. Custom Tiered Cache Topologyallows customers to set acustom topology that fitsspecific needs (ex: upper tiersin specific geographic locationsserving more customers). Cloudflare will dynamically findthe single best upper tier for anorigin using Argo performanceand routing data.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com Engage with a CustomerSuccess Manager (CSM) tobuild a custom topology.10
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraffic flow: Argo Tiered Cache, Smart Tiered Cache TopologyIn Figure 4, Argo Tiered Caching is enabled with Smart Tiered Cache Topology. The diagramdepicts two separate traffic flows, summarized below. The first traffic flow (Client 1 in green) isa request from a client that comes into Data Center 1. The second traffic flow (Client 2 in purple)is a subsequent request for the same resource into a different data center, Data Center 2.Global Anycast Edge NetworkBGP AnnouncementCloudflare Anycast IPsCustomer IPsClient / End UserConsumptionData Center 1Lower tierDoSflare NetwoudorClkDNSCacheData Center 3Upper tierRequestCDNCacheLower tierClient tmlRequestArgo TieredCacheData Center 2RequestCustomer Network /Origin lClient 1CacheCacheDoSDNSCDNCacheData CentersPublic CloudsDoSDNSCDNCacheFigure 4: HTTP requests and traffic flow through Cloudflare CDNClient 1 First request received in Data Center 1 resultsin cache miss, as request had not been madepreviously by any client. No cached content found, so Data Center 1checks with upper tier data center to request acopy of the content. Upper tier data center also does not havecontent cached locally, so it makes a requestto the origin server for content. Upon receivingthe content, the upper tier data centercaches it locally and relays the content tothe requesting lower tier data center. Thelower tier data center caches the content andresponds to the client.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comClient 2 Second request by a different client receivedin Data Center 2 results in cache miss, asrequest had not been made previously by anyclient served by Data Center 2. No cached content found, so Data Center2 checks with the upper tier data center torequest a copy of the content. Cached content found at the upper tier datacenter. Data Center 2 retrieves and caches thiscontent locally and responds to the client.11
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraffic flow: Argo Tiered Cache, Smart Tiered Cache Topology (Continued)In Figure 4, Client 1 traffic flow displays the trafficflow when a client request is received by a data centerclosest to the client, Data Center 1. Since there isnothing locally cached on the ingress data center andtiered caching is enabled, a request is sent to the uppertier data center to request a copy of the content tocache.Because the upper tier data center also does not havethe content cached, it sends the request to the originserver, caches the received content upon response, andresponds to the lower tier data center with the cachedcontent. The lower tier data center caches the contentand responds to the client.Notice that when a new request for the same contentis made to another data center (Client 2 traffic flow),Data Center 2, the content is not locally cached;however, the content is retrieved from the upper tierdata center, where it was cached from the first requestfor the same content.With the upper tier data center returning the cachedcontent for the second request, the trip to the originserver is prevented, resulting in higher cache hit ratios,faster response times, saved bandwidth cost betweenthe Cloudflare network and the origin server, andreduced load on the origin server responding to requests.Argo Smart RoutingArgo Smart Routing is a service that findsoptimized routes across the Cloudflare network todeliver responses to users more quickly. As discussedearlier, Cloudflare CDN leverages Argo Smart Routingto determine the best upper tier data centers forArgo Tiered Cache.In addition, Argo Smart Routing can be enabled toensure the fastest paths over the Cloudflare network aretaken between upper tier data centers and origin serversat all times. Without Argo Smart Routing, communicationbetween upper tier data centers to origin servers are stillintelligently routed around problems on the Internet toensure origin reachability.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comArgo Smart Routing accelerates traffic by taking intoaccount real-time data and network intelligence fromrouting over 28 million HTTP requests per second; itensures the fastest and most reliable network pathsare traversed over the Cloudflare network to the originserver. On average, Argo Smart Routing accounts for30% faster performance on web assets.12
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraffic Flow: Argo Tiered Cache, Smart Tiered Cache Topology withArgo Smart RoutingFigure 5 details the traffic flow when Argo Tiered Cache and Argo Smart Routing are not enabled.The request comes into the closest data center, and, because content is not locally cached andArgo Tiered Cache is not enabled, the request is sent directly to the origin server for the contentAlso, since Argo Smart Routing is not enabled, a reliable, but perhaps not the fastest, path is takenwhen communicating with the origin server.Global Anycast Edge NetworkBGP AnnouncementCloudflare Anycast IPsCustomer IPsNo Argo Tiered CacheNo Argo Smart RoutingData Center 1Lower tierDoSDNSCDNCacheData Center 3Data Center 2Lower tierClient / End UserConsumptionDoSUpper tierCloudflareNetworkCustomer Network /Origin ServersDoSDNSCDNCacheCache miss:Route directly to originRequestCDNData CentersDNSCacheResponseRequestPublic CloudsFigure 5: Cloudflare CDN without Argo Tiered Cache and Argo Smart Routing1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com13
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ETraffic Flow: Argo Tiered Cache, Smart Tiered Cache Topology with Argo Smart Routing(continued)Figure 6 articulates the traffic flow with both Argo TieredCache and Argo Smart Routing enabled.With the Cloudflare CDN, Argo Smart Routing isused when:In Figure 6, when a request is received by Data Center 1and there is a cache miss, the cache of the upper tier datacenter, Data Center 3, is checked. If the cached contentis not found at the upper tier data center, with Argo SmartRouting enabled, the request is sent on the fastest pathfrom the upper tier data center to the origin.1. There is a cache miss and the request needs to besent to the origin server to retrieve the content,2. There is a request for non-cacheable content, suchas dynamic content (ex: APIs), and the request mustgo to the origin server.The fastest path is determined by the Argo networkintelligence capabilities, which take into account real-timenetwork data such as congestion, latency, and RTT.Global Anycast Edge NetworkCBGP AnnouncementCloudflare Anycast IPsCustomer IPsudloe NetworflarkData Center 1Lower tierCustomer Network /Origin ServersData Center 3CacheUpper tierCacheResponseClient / End UserConsumptionRequestDoSDNSDoSCache miss:example.com/index.html Check upper tierCDNCacheArgo TieredCacheDNSCache miss:Use fastest route to mlRequestData Centersexample.com/index.htmlPublic CloudsData Center 2Lower tierFigure 6: Cloudflare CDN with Argo Tiered Cache and Argo Smart Routing enabled1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com14
C LO U D FL A R E C D N R E FE R E N C E A RC H I T ECT U R ESummaryTo summarize, the Cloudflare CDN is SaaS that helps address the challengescustomers face around latency, performance, availability, redundancy, security,and costs. The Cloudflare CDN leverages Cloudflare’s global Anycast edgenetwork and Argo Tiered Cache to deliver optimized results while saving costs forcustomers. Customers can also enable Argo Smart Routing to ensure the fastestnetwork path is used to route requests to the origin server.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.com15
W H I T E PA PE R 2022 Cloudflare Inc. All rights reserved. The Cloudflare logo is a trademarkof Cloudflare. All other company and product names may be trademarks ofthe respective companies with which they are associated.1 888 99 FLARE enterprise@cloudflare.com www.cloudflare.comREV:BDES-2628.2022MAR11
Data Center 2 Lower tier Data Center 3 Lower tier Argo Tiered Cache In the above diagram, there are a few important key points to understand about the Cloudflare CDN and the global Anycast edge network it resides on: An important differentiator is that Cloudflare utilizes one global network and runs every service on every server in every .
