Many companies utilise a content delivery network (CDN) to serve static assets. Using a CDN allows you to offload static assets loading from origin servers, to then serve those assets as close to the end users as possible. With many points of presence across the globe, CDNs help deliver content to end users from a server in their city or country, instead of downloading them from the original location which could be much further away. Still, picking a CDN is not an easy task. How do you measure a CDN’s performance? Which strategy do you apply? Should you use multiple CDNs? When approaching such a broad topic, there are different ways of evaluating and measuring your choices. We should back up each option with an analysis of the current state, the competitors in different regions where we operate and the performance from the perspective of different ISPs. There are negligible variations throughout different browsers and operating systems. We would rule these differences out as statistically insignificant and focus on network performance and the end-user latency as a single metric of a CDN.

"You can't improve what you can't measure."

After evaluating different options, I realised that there is a tool fitting our needs for retrieving scientific results and real-world measurements from many points across the globe. This tool can help us to identify which CDN performs best per country or per region. Our quest for the fastest CDN outgrew the original idea and became a large piece of research. The final result was a lot of wildly coloured maps, including an interactive one that you can use to see the best performing CDN in your country.

Using RIPE Atlas

RIPE Atlas is a global, open and distributed network of probes which actively measure Internet connectivity and reachability. It's the largest Internet measurement infrastructure ever created! Seen in a historical context, people have used electric telegraph since 1835 to broadcast the weather forecast. The first weather station started collecting weather data even before that, in 1781. In a connected and digital-first 21st century world, we still make use of hundreds of thousands of weather stations worldwide to quickly see the current weather conditions on our smartphones and decide on our clothing for the night out.

Such a connected world needs an Internet equivalent of weather stations, that could monitor the Internet itself. And so, that’s where the RIPE Atlas project fits in: it gives everyone the ability to measure the connectivity of any device connected to the Internet (by only having a publicly routable IP address) from different probes. RIPE Atlas gives everyone the ability to use more than 10,000 probes worldwide, thanks to hundreds of volunteers that are hosting them.

It’s important to note that RIPE Atlas is a credit-based system: you can get uptime credits for having the probes online, and you can also get them whenever your probe delivers results for someone else’s measurement. To get started, you can get free credits at RIPE events; then you can also send and receive credits to and from other RIPE Atlas users.

Content Delivery Networks

The image below gives a high-level overview of how a geographically spread Content Delivery Network (CDN) Points of Presence (PoPs) helps get content from servers closer to the end user (image courtesy of Cloudflare).

Figure 1: High-level overview of a CDN

I will not get into the topic of implementing a CDN or how they work "under the hood". Instead, I will point out the benefits that they provide:

• The faster loading time of assets due to decreased latency: Assets (images, JavaScript, CSS files and so on) are fetched once from the origin server, then from that point on they are served from the server closest to the user. This benefit helps to optimise the end user experience, making our customers happy, and boosting our SEO rankings due to a faster page load time.

• Cutting traffic costs: Typically, by serving your static content from popular cloud storages (i.e. AWS S3 & Google Cloud Storage) to your users, you pay for the Internet traffic generated for each download/hit. A CDN helps as a man-in-the-middle: it will fetch the requested content only once from the origin server, store it, and then serve it from the cache. This is a lot cheaper and will result in less outgoing traffic.

• Caching: Using a CDN allows you to specify different dynamic caching policies and increase the cache hit rate. If the content is served from a CDN’s server cache, it does not have to fetch it from your origin server. Cache hit rates of static content requests can often reach 90% and more, which essentially means cutting 90% of traffic costs from your data centre.

• Ensuring readiness for traffic spikes in case of sudden traffic: CDNs have invested a lot of time and knowledge in developing large infrastructures that scale well, from being featured on Reddit and Hacker News to streaming live UEFA Champions League finals.

In the case of assets-heavy websites, the main objective is to serve them to your customers in the fastest way possible. To ensure this, as well as the most efficient caching policy and using a reliable and well-spread CDN, utilise the correct Cache-Control headers with appropriate expirations for different content types; ignoring query strings to avoid cache busting, using immutable flag etc.

Who is using CDNs?

Nowadays, the majority of Internet traffic is passing Internet Exchange Points where traffic is exchanged for free, or for a very low fee. There, big content providers (think Netflix, Facebook, Google/YouTube) and ISPs connect and exchange traffic with the lowest latency and the highest throughput possible. By reading any blog on Medium.com, you have unknowingly accessed Cloudflare, one of the most popular CDNs. Or when you listened to your favourite song on Spotify, your device probably established a connection to Fastly’s servers. 

Your favourite blogs and news portals are served using AWS Cloudfront, Google Cloud or other CDNs. Like the majority of Internet users, you have also generated some traffic to private CDNs by accessing Facebook, Instagram, Youtube, Netflix etc. You may think that only big companies use CDNs, but you would be mistaken. Nowadays, it’s unimaginable to start an online service without thinking about the best way to serve your traffic, so upfront planning to use a CDN makes a lot of sense.

Should you think about a CDN from day one? Absolutely! You want to optimise your costs upfront and achieve the best performance at the same time. To achieve this, you can rely on your gut feeling, a friend’s recommendation, a Google search or you can utilise scientific, statistical data with real numbers. If you’re still interested, keep on reading. Here’s where the research starts!

Creating RIPE Atlas measurements

To create your very first measurement using RIPE Atlas, you can either use a web UI or a nice JSON API. Using the RIPE Atlas Web Wizard is simple; in a few clicks, you can create a measurement with the summary of all the associated costs. The probes available for use, are hosted at various places: on a local router at home in residential areas, racks in workplaces and offices, and inside data centres. They can also be connected to mobile 4G connection or via a satellite uplink on a very remote location. As long as there’s an Ethernet connection, the source of the connectivity doesn’t really matter. The coverage of IPv4 and IPv6 networks in total is pretty much the same: below 10% of worldwide autonomous system numbers (ASNs).

IPv4 ASNs covered: 3,602 (5.627%)
IPv6 ASNs covered: 1,446 (8.617%)

However, in the grand scheme of things, the major worldwide consumer ISPs and hosting companies have a sufficient number of probes hosted with them, and almost all of the world’s countries are connected — 182 (92.857%).

Research methodology

As noted above, using the web UI has its drawbacks in some scenarios. Particularly in my case, as I want to analyse all the countries in the world one by one, selecting the probes and then filtering them by different tags, it would be very cumbersome to repeat this process manually for 182 countries. Luckily, all of this can be done through a very simple REST API. First of all, we need a key ingredient to conduct this research: plenty of RIPE Atlas credits. Luckily, I have had a probe connected for more than five years in which it had collected almost 60 million credits, which was more than enough to conduct this research more than a dozen times. I made some analysis for private use and also for investigations like this. Then, a list of CDN providers was defined, by analysing the current CDN market and favouring companies with a global presence, instead of a regional availability. Here is a breakdown of the CDNs chosen for this research (a total of 7):

1) Akamai: a really old player in the market

2) AWS Cloudfront: a global player with almost 200 PoPs across 30 countries. They have regional edge points of presence to which all the other POPs are connecting to as a pre-optimization step to concentrate hits to a regional edge POP.

3) Microsoft Azure: more than 130 PoPs and a very large network with different tiers.

4) Cloudflare: the most popular choice for small to medium websites with a very generous or almost limitless free tier.

5) Google Cloud CDN: use Google’s global network in conjunction with Cloud Storage or with Compute Engine instances. 

6) Fastly: a popular CDN for different large scale projects (Github, Spotify, etc) that is available in more than 30 PoPs – with plans for expansion.

7) Cachefly: used to be a US-centric CDN, but recently grew to be a global player.

Once the CDN providers list was defined, I decided to write a simple script using the Go programming language, due to its simple concurrency primitives. This small script of fewer than 200 lines, goes through all the ISO2 (ISO 3166) codes of the countries, combines them with all the possible combinations of CDNs that were defined before, and sends a three-packet ping measurement API request to RIPE Atlas’ API.

Cloudflare has its own, very popular, DNS resolver on 1.1.1.1. For Cloudfront and Google Cloud, I had to create my own distribution, but all the others were very easy to test with some of the well-known hostnames of the companies publicly using them (FIFA, etc).

Figure 2: Setting up measurements for a number of CDNs

Using the request option, we would select up to 50 probes and use a couple of tags to filter out all the unavailable or unstable ones that would negatively influence our results set. The probe selection tags I used were system-IPv4-capable, system-IPv4-works and system-resolves-a-correctly to ensure that DNS resolution works correctly.

Parsing the measurements