When we experienced an occasional operational issue with the data returned by one of our name servers, we wanted to verify DNS responses from a number of widely distributed measurement points (vantage points). RIPE Atlas was just the right service to provide us with the necessary raw data.
A couple of weeks ago we learned of a paper to be presented at the ACM SIGCOMM conference : The Collateral Damage of Internet Censorship by DNS Injection (a slightly edited version was also published in the July edition of the Computer Communications Review ). It reported about certain modifications to DNS responses and suggested that a second level domain within the DE top level domain (TLD) could be affected by these modifications. At this stage it was imperative to us to be able to quickly investigate the situation.
The survey described in the paper used open recursive resolvers as vantage points, which add a level of uncertainty to the observation, so we were looking for a more direct approach. We have been using the RIPE NCC's DNSMON service for several years. DNSMON already delivers raw query and response data to subscribers, but the SOA RR (Start of Authority Resource Record) at the DE zone apex was not sufficient to detect potential modifications. RIPE Atlas with its User Defined Measurements (UDMs) came to our rescue: with the help of the RIPE NCC we were able to deploy a small set of DNS queries from all 1,700+ RIPE Atlas probes towards the DE name servers and capture the results as well as gather traceroute data to reproduce both target and path.
The name server that the authors observed responding, a.nic.de, is one of DENIC's two anycast clouds . We wanted to identify which of the RIPE Atlas probes would reach which instance in that cloud and which of the probes, if any, would receive unexpected responses.
Being authoritative for a TLD, our name servers usually respond with a referral down the DNS tree, where the queries look for the address record of a domain name. We let the probes send standard DNS queries (asking for A RRs) towards the authoritative server, similar to what a recursive resolver would do, with and without requesting DNSSEC.
These were the results:
- First and foremost, all DNS responses that involved name server locations other than the Beijing one were unaltered and authentic. These are marked green in Figures 1 and 2. (Five probes, marked orange, got immediately served the final result (the A record) by way of transparent DNS proxies in their local network.)
- Second, several (121 of 1,762) of the probes talking to the Beijing server (marked yellow) were consistently served the correct DNS referral information.
- Another 218 (of 1,762) probes, marked red, either received modified or inconsistent results. The modifications showed either one of several A records unrelated to the query name or a kind of empty response that can only be considered within the far boundaries of the DNS protocol. We verified the A RR responses against a list of IP addresses made available by the authors of the paper and could confirm their presence on that list.
The presence of the DNSSEC OK (DO) bit in the DNS query did not have any influence on the response modifications.
Figure 1: Global view of a.nic.de probe targets as observed in July 2012
Figure 2: a.nic.de probe targets - zoomed in, July 2012
Note that this map represents the situation of July 2012. For technical reasons the probe data that appears when clicking on a probe is the current data as of the time you view the map; so it may or may not be consistent with the state of things in July.
Findings and Conclusions
The basic observation from the SIGCOMM paper is now confirmed: some systems within Europe and the Americas were experiencing altered, non-authentic DNS responses for queries sent to a name server located in Beijing.
However, no evidence was found for altered responses affecting traffic in transit . In other words, traffic from probes outside the PR China going to name servers other than in Beijing was not tampered with.
Due to the ability to submit queries and capture responses directly from RIPE Atlas probes rather than through third party intermediaries, we were able to get deeper insight into the protocol details of the modified DNS response packets.
While the DE TLD has been signed with DNSSEC since May 2011, the zone in question did not yet use DNSSEC. Even if it had, the modifications could only have been detected, but not reverted. This is the case for modifications of any kind, not just those observed in the paper and in this article.
With the results gathered we were able to mitigate the modifications by means beyond the scope of this article. A subsequent re-run of the RIPE Atlas measurements could verify that modified responses are no longer experienced by RIPE Atlas probes located outside the PR China.
We already had decided to become a RIPE Atlas sponsor to enable the integration of RIPE Atlas measurements into our continued monitoring effort to secure the availability and integrity of DNS name resolution within the DE namespace.
The UDMs are already a powerful tool. The following suggestions are offered for consideration and further improvement:
- When using RIPE Atlas probe results it should be double checked whether transparent proxies, DNS or otherwise, are in use and affecting the site's DNS experience.
- The response data collection could benefit from including the IP TTL, because that could assist in distinguishing the authentic from the modified responses.
We thank the RIPE NCC for their support in gathering and visualising the data.