B-root and IP anycast: the basics

B-root is one of 13 root servers that are authoritative for the DNS root zone. It is one of the core components of the Internet.

IP anycast is a technology that in essence, allows one IP address to be shared among many servers across various locations around the world. It is widely used by CDNs and DNS providers as a way to improve performance and resilience [2].

Figure 1 shows a typical DNS deployment using IP anycast. Consider a recursive DNS resolver (e.g.: 8.8.8.8) that wants to resolve the domain example.nl .In this scenario, a resolver first has to query (in case of cold cache) one of the DNS root servers (a list obtained on its roots.hints file) what are the authoritative DNS servers for the .nl top-level domain. It  needs to know the addresses of the servers responding on behalf of the .nl zone, in this example.

 Figure 1: Root DNS structure, terminology, and mechanisms in use at each level. Source: [2]

So the first step consists in asking one of the DNS root servers "where is .nl"?  Assuming it chooses K-root, as in Figure 1, it will send a DNS query to the IP address of K-root. However, since K-root  is configured with IP anycast, the DNS query can wind up in any of the site (S_n) in the figure. A site, in this case, refers to a different geographical and topological location in which the IP address of K-Root, in this case, is announced (they can be very far apart, as Tokyo and Amsterdam. To give an idea, L-root, the "biggest root server" in terms of sites, has more than 159 sites at the moment, as reported on root-servers.org).

Then, BGP redirects the query to the "closest site" of K-root (in terms of "BGP distance"). At this particular site, let's say S33, there might be multiple servers behind a load-balancer (r1 to rn). Given that, as in the case of K-root and others, the same IP addresses, with anycast, is announced from multiple sites, each site having multiple physical/virtual servers themselves. Overall, this extra redundancy improves the dependability of the entire system.

With regards B-root, before 1 May it was a unicast single site located in Los Angeles (LAX). After 1 May second site (with anycast) was added in Miami (MIA)[1].

Turning on anycast

Turning on anycast on B-root involved announcing its prefixes (IPv4 and IPv6) from multiple locations (MIA and LAX, in this case). The first tool we used to look at the changes was BGPlay. We observed a lot of activity, not on 1 May, but on 2 May around 21:00 GMT, for the /24 IPv4 prefix of B-root.

To double-check that, the best way was to download and process the RIPE Atlas built-in measurements to B-root. Since B-root responds to CHAOS queries, which essentially return a string that allow us to identify which server responds to which query [2], we can count how many queries were answered by each site (after filtering out those probes where DNS queries were hijacked, as described also in [2]).

Effects on the RIPE Atlas Probes

We then download data from 2017-04-30t00:00:00 to 2017-05-03t00:00:00 and created time series for each of the legitimate sites of B-root. Since no queries were answered by MIA on 1 May, we single out 2 May 2017 and see how the probes reacted.

The first criteria we used was reachability: How many RIPE Atlas probes obtain a correct answer from B-root before and after activating anycast. Figure 2 shows that around the same time that BGPlay shows a lot of BGP activity, RIPE Atlas probes started to reach the newly available B-root Miami site (MIA).

  Figure 2: Number of unique RIPE Atlas probes reaching B-root sites every 10min (from CHAOS queries).

After this transition, the number of probes reaching each site remained stable over time. This has been previously reported in [2], as well as in Does Anycast Hang Up on You?. As can be seen, ~1600 probes moved to MIA, and LAX still handles most of the queries.

Which site a probe will reach depends on peering agreements between the AS of the probe and the ASes of B-root. Using a variety of metrics, BGP maps each probe to a site [2,3,4].

Another effect we can observe is how the response time varied after the MIA site of B-root was activated. Figure 3 shows that median RTT for each site of B-root. The probes that moved from LAX to MIA experienced a shorter median RTT than previously.

Figure 3: Median round-trip time (RTT, in ms) for all RIPE Atlas Probes to each B-Root site

Summary

IP anycast is a technology that allows the same prefix to be announced from the same location, and can be used in case of large DDoS attacks [2]. B-Root has activated anycast.

The new site has been announced on 2 May 2017, around 21:00GMT. When it was done, very quickly roughly 17% of the RIPE Atlas probes moved to from the LAX site to MIA. Consequently they experienced better performance than the median of the other probes in terms of response time.

This change to anycast, together with the public datasets of RIPE Atlas, allowed us to see if there were transient effects on the activation of IP anycast on B-root.  Even if we don't have inside knowledge of the operations of B-root, their transition to anycast, from the point of view of RIPE Atlas, has been smooth as possible. Kudo to the operators!


 

References

[1] Root Operators:   B-Root Begins Anycast in May. http://root-servers.org/news/b-root-begins-anycast-in-may.txt . 17 April 2017.

[2]  Giovane C. M. Moura, Ricardo de O. Schmidt, John Heidemann, Wouter B. de Vries, Moritz Müller, Lan Wei and Christian Hesselman. Anycast vs. DDoS: Evaluating the November 2015 Root DNS Event. Proceedings of the ACM Internet Measurement Conference (Nov. 2016). [DOI] [PDF] [alt PDF]

[3] Lan Wei and John Heidemann. Does Anycast Hang up on You? (extended). Technical Report ISI-TR-716. USC/Information Sciences Institute. [PDF] [alt PDF]

[4] Ricardo de O. Schmidt, John Heidemann and Jan Harm Kuipers. Anycast Latency: How Many Sites Are Enough? Proceedings of the Passive and Active Measurement Workshop (Sydney, Australia, Mar. 2017), to appear. [PDF] [Abstract] Details