The techniques used by the L-root DNS server are documented in RFC 7108 "A Summary of Various Mechanisms Deployed at L-Root for the Identification of Anycast Nodes". These techniques can be used by the RIPE Atlas probes to find the instance of L-root they are talking to.
Introduction
RFC 7108 is available online . It documents the various techniques that a user can try to find out which of the instances of the anycast cloud he/she is using. Some are traditional (CH/TXT request of id.server), some more modern (NSID) and some are specific to L-root such as the name identity.l.root-servers.org that you can resolve on your machine and that returns an IP address that is unique per instance. This last technique can be used by ordinary people, who have no access to the dig command or similar programs.
Methodology
Let's first use the NSID queries to examine the many instances of L-root. We already used this technique in another article and the code is still on Github . Here, we will ask 500 RIPE Atlas probes to query the SOA of the root domain to l.root-servers.net. The worldwide measurement is #1071935. It is here analysed by another program .
Among the 452 probes which got a proper NSID response, we find 109 different instances of L-root. According to RFC 7108, there were 273 different nodes (and therefore identifiers) in November 2013. We can see that the selected RIPE Atlas probes are not dispersed enough to find all the nodes. Maybe a test with more probes would help but one measurement is currently limited to 500 probes.
Results
The most popular instances are:
Root Server Instance | Number of Probes | Percentage |
ods01.l.root-servers.org | 39 | 9% |
kbp01.l.root-servers.org | 28 | 6% |
ory01.l.root-servers.org | 19 | 4% |
lba01.l.root-servers.org | 16 | 4% |
ham01.l.root-servers.org | 13 | 3% |
sjc01.l.root-servers.org | 11 | 2% |
ams01.l.root-servers.org | 11 | 2% |
her01.l.root-servers.org | 10 | 2% |
atl01.l.root-servers.org | 10 | 2% |
mmx01.l.root-servers.org | 10 | 2% |
"Most popular" is to be taken with a grain of salt: RIPE Atlas probes are not evenly geographically distributed.
The instances are named after the IATA airport codes . "ods" is Odessa in the Ukraine, "kbp", Kiev, also in the Ukraine, "ory" is Paris in France etc. The popularity of Ukrainian destinations is a bit surprising. It is stable (the same measure two weeks ago or one month after showed the same phenomenon.) Of course, if we measure from a different region, we get a very different picture. Limiting to probes to the "West" (the US) area, we get (measurement #1071936):
Root Server Instance | Number of Probes | Percentage |
sjc01.l.root-servers.org | 46 | 10 % |
atl01.l.root-servers.org | 43 | 9 % |
bos61.l.root-servers.org | 34 | 7 % |
cjr01.l.root-servers.org | 34 | 7 % |
mvd01.l.root-servers.org | 27 | 6 % |
udi01.l.root-servers.org | 22 | 5 % |
yyz01.l.root-servers.org | 21 | 4 % |
phl01.l.root-servers.org | 16 | 3 % |
lax15.l.root-servers.org | 16 | 3 % |
Here, US cities (sjc = San Jose, atl = Atlanta, bos = Boston) get the lion share.
This change of "popularity" appears also when you measure in the South-East region (Asia-Pacific, measurement #1071938): a machine in Noumea (New Caledonia) is the most commonly used by the probes, before a machine in Perth (Australia). In North-Central (Europe, measurement #1071939), the Ukrainian sites are again in front. In North-East (Russia, measurement #1071937), we find of course the same results (but the number of probes is lower, so may be less representative).
All these measurements were done with IPv4. Testing with IPv6 show similar resultats, including the "Ukrainian attractor" (world-wide measurement #1402075).
RFC 7108 proposes another way, besides NSID, to find out to which instance you are talking to. You request the A record for the name identity.l.root-servers.org and then you get an IPv4 address which uniquely identify the instance. Do note that, for a given probe, it may be a different instance than the one obtain by the previous measure. In the previous measure, we used a direct connection from the probe to l.root-servers.net, but now, the probe queries its default resolver, which may be far away, and may use a different instance of L-root. So, let's see the results of measurement #1071473, 500 probes requested:
IP address | Number of occurrences |
93.178.205.146 | 33 |
77.88.206.134 | 26 |
74.80.98.64 | 22 |
74.80.98.65 | 20 |
109.239.109.173 | 17 |
Many of these IP addresses have a PTR record which gives us an indication of the localisation of the instance. For instance, 93.178.205.146 is hostmaster-gw1.odessa.ucomline.net. So, the Ukrainian effect was a real one. There are few RIPE Atlas probes in the Ukraine. Can it be an attraction entirely caused by BGP? Or a misconfiguration? Here is an example seen from Free/Proxad (AS 12322) in Paris (France). NSID returns kbp01.l.root-servers.org and traceroute:
% traceroute l.root-servers.net
traceroute to l.root-servers.net (199.7.83.42), 30 hops max, 60 byte packets
1 192.168.2.254 (192.168.2.254) 0.568 ms 0.426 ms 0.370 ms
2 88.189.152.254 (88.189.152.254) 6.682 ms 6.768 ms 7.904 ms
3 78.254.1.62 (78.254.1.62) 13.724 ms 13.741 ms 13.804 ms
4 rke75-1-v900.intf.nra.proxad.net (78.254.255.42) 10.622 ms 11.369 ms 11.379 ms
5 cev75-1-v902.intf.nra.proxad.net (78.254.255.46) 11.683 ms 12.649 ms 13.141 ms
6 p16-6k-1-po12.intf.nra.proxad.net (78.254.255.50) 13.150 ms 10.321 ms 10.265 ms
7 bzn-crs16-1-be1024.intf.routers.proxad.net (212.27.56.149) 12.052 ms 7.389 ms 11.070 ms
8 th2-6k-3-po21.intf.routers.proxad.net (212.27.56.2) 26.276 ms 26.311 ms *
9 jaguar-pni.routers.proxad.net (212.27.40.122) 10.974 ms 11.412 ms 11.901 ms
10 l.root-servers.net (199.7.83.42) 22.967 ms 23.503 ms 24.502 ms
The route shows only French operators and the TTL seem consistent with a route staying in France (typical TTLs for Ukrainian targets are at least twice as large). At least in this case, we can suspect a misconfiguration.
Conclusion
Being able to find out which instance of an anycast cloud you use is very important for debugging operational issues. As a side effect, it also allows funny statistics. BGP never guaranteed that you get to the "closest" instance, especially if you use "closest" in the geographical sense. The most popular instances are therefore not always the ones you expect. This assume that the labels returned by the DNS server about itself are correct, which is not always easy to achieve in the very dynamic environment of the Internet.
Comments 3
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.
john bond •
Hello Stephan, Firstly, thank you for conducting this analysis. i preformed further investigation on the kbp01.lroot-servers.net. It appears that this node it is in fact very well connected. When I look at the probes that get an NSID of kbp01.lroot-servers.net is see the majority of them have a directed peering with AS 20144 (traceroutes[1]) who is the host of this node. As the network is only one AS hope away it is often preferred. An example of where we see this preference is with AS 33540 where all 27 probes hit kbp01.lroot-servers.net because of direct peering with AS20144 (traceroutes[2], NSIDs[3]). In relation to AS12322 I have been unable to recreate this issue its possible that it no longer exists. I conducted a measurement[4] to be preformed by all AS12322 164 probes. I was allocated 63 and all of them return an NSID of lys01.l.root-servers.org. Including probe 10645[5] which I believe is your probe and therefore has the same path[6] as above. As a final test I preformed the following query. aiw6Bu6g.johnbond.net. IN A Using tcpdump on the lys01 node i was able to see this request come in so i think we can say with confidence that this probe is hitting the lys01 node. Another aspect which influences path choice is AS prepending. Some of our hosts prepend there AS when announcing the l-root prefix. Two examples of this can be seen via RIPEstat looking glass[7]. AS271 is prepending itself an additional 3 times at RRC03 (AMS-IX) and RRC11 (NYIIX). We also see AS35313 prepending an additional path at RRC11. These are only the examples we can see via ripe stat however we do see other examples where this occurs. L-Root hosts are free to choose how they announce our prefix so your conclusion is correct Just because an l-root node is geographically close to you it dose not necessarily mean that is the node you will use. As a side note, I also audited all of the l-root boxes to ensure that there NSID is configured to the correct value and was able to find no issues. To analyse the atlas measurements i used Steaphans script[8] mentioned above for the NSID's and Jan Hugo Prins json2traceroute script[9] for the traceroute's Thanks John ICANN DNS Operations [1]https://atlas.ripe.net/api/v1/measurement/1408582/result/ [2]https://atlas.ripe.net/api/v1/measurement/1407161/result/ [3]https://atlas.ripe.net/api/v1/measurement/1406948/result/ [4]https://atlas.ripe.net/api/v1/measurement/1421320/result/ [5]https://atlas.ripe.net/api/v1/measurement/1421326/result/ [6]https://atlas.ripe.net/api/v1/measurement/1421325/result/ [7]https://stat.ripe.net/widget/looking-glass#w.resource=199.7.83.0%2F24 [8]https://github.com/RIPE-Atlas-Community/ripe-atlas-community-contrib/blob/master/measure-dns-nsid.py [9]https://github.com/RIPE-Atlas-Community/ripe-atlas-community-contrib/blob/master/json2traceroute.py
Robert Kisteleki •
I'd also like to draw your attention to https://atlas.ripe.net/contrib/root_anycast.html?msm_id=8
Dmitry Kohmanyuk •
Pleasantly surprised by those statistics. We (Hostmaster) run three L-root nodes, and they should be well-connected: Kyiv one uses Topnet (AS 21011) and Odessa - Vega Telecom (formerly Ucomline, formerly Farlep - AS41216). The third node is in Kharkiv - ITL, AS 15626.