World IPv6 Day - Asymmetric 6to4 Measurements

Introduction

As mentioned in a RIPE Labs article and in a presentation at RIPE 62, the RIPE NCC is leading several measurement activities related to World IPv6 Day; BBN is collaborating with the RIPE NCC on some of these efforts.  One lingering question for World IPv6 Day is how well the 6to4 infrastructure works, and especially how it will work under potentially heavier load.  

Luckily most modern OSes will not attempt to use 6to4 when native IPv4 is available, but the remaining 6to4 traffic might still cause problems.  Studies of the IPv6 capabilities of web users done at the RIPE NCC, APNIC, and elsewhere indicate that a significant fraction of IPv6 users are still connected to IPv6 resources via 6to4 or Teredo — 17% of IPv6-capable web users according to one report .

One way of monitoring the health of the 6to4 infrastructure is to measure the differences between 6to4 routing and native IPv6 routing.  As described below, using specially crafted ICMPv6 pings, it is possible to measure the difference between native and 6to4 paths from a server back to an observation point.  BBN and the RIPE NCC have implemented these tests and begun collecting observations of several IPv6-capable servers.  Although these observations have only been running for a few days, some interesting features are showing up already.

Our test works by having a measurement host that does 2 ping6 measurements to a single target at the same time: One ping using it's native IPv6 source address, and the other ping using a 6to4 IPv6 address.  The 6to4 address is constructed so that the return traffic will end up on our measurement host as well, because the IPv4 address of our measurement host is encoded in the 6to4 IPv6 address. This is visually explained in the Figure below:

By comparing the differences both in round trip time (RTT) and packet loss for the native IPv6 and the 6to4 IPv6 test to a single destination, we get a baseline of 6to4 relay behaviour. This is either going to change or stay the same on World IPv6 Day, which is what we intend to measure.

This experiment is fairly simple to realize on platforms with a version of ping6 that have a command line option to set the source address of the ICMPv6 packet (-S on Mac OS X, -I on Linux).  In such cases, ping6 can be used to generate both pings, and the RTT values and loss rates can be computed from observations of packet captures.  

Currently, measurements are being collected from a single host at BBN, which uses Cogent for IPv6 transit. More information about this network can be found on RIPEstat or REX .  Note that in principle these differential 6to4 measurements should not be very dependent on where they're being observed from, since the 6to4 relay they're measuring is on the server end of the path.  However, differences between the IPv6 and IPv4 paths to the measurement host can make a difference in the results, and for the BBN host in particular, there are a few known peering issues .  

Latency measurements are currently being conducted against all of the IPv6-capable hosts advertised by World IPv6 Day participants (via a list provided by ISOC), plus a list of hosts behind known 6to4 relays that have been volunteered.   If you run a 6to4 relay or IPv6-capable host and want to be included in these tests, please let us know at labs@ripe.net . The information we need from you is the location of the 6to4 relay (what AS, geolocation), plus a ping6able host for which traffic destined for 2002::/16 will end up at your specific relay.

Data collection has been running continuously for around a week as of the publication of this article.  Even over this fairly short time-scale, we can make some observations.  Looking at average, minimum, and maximum, latencies provides an overview of the general behavior of each target, as shown in the image below:

Just based on these summary statistics, we can see a few features:

• The average latency penalty across all hosts is 24.05ms (computed as the difference between native IPv6 and 6to4 averages per host).  Ignoring hosts that where native IPv6 and 6to4 latencies are basically the same (within 10ms), the average penalty is 74.59ms.  Remembering that this measurement only reflects latency in the reverse path, this is consistent with earlier observations that IPv6 users took around a 100-150ms hit in RTT.
• 6to4 latencies have a much higher variability than native latencies.  Some hosts have very stable native paths, while 6to4 latency can vary by as much as 800ms (see 20, 21, 23,24 above).  For hosts with high variability in native latencies, 6to4 latencies were similarly variable (16, 22).
• Some sites have been able to make 6to4 perform roughly as well as native IPv6 (2, 3, 10, 11, 13, 14). One hosts even performs better over 6to4 (5). For these hosts it is likely that they have a 6to4 relay very close to the server under test, likely in the same network, so the return traffic doesn't have to trombone as much.

We can also look at time series for individual hosts, which confirm the conventional wisdom with respect to 6to4: It works much better if there's a relay very close to the server. Without a close relay, several sites display very stable RTTs, but with around a 70ms penalty:

As mentioned above, multiplying by two to account for 6to4 in both directions puts us in the ball park of the ~150ms impact reported elsewhere.  Sites that have a close relay can bring this penalty to within 5ms, at least on the server side:

Actually, one of the sites we're observing seems to have installed a relay since we started measuring RTTs.  The time series for this site shows the impact of local relays very clearly, as the 6to4 penalty drops from around 30ms to around 2ms.  (Presumably the "bands" in the plot are due to load balancing.)

Also visible in this plot as well as several others is the increased variability in 6to4 latencies compared to native latencies.  In the above, native IPv6 RTTs vary within around a millisecond, while with 6to4 this variation increases to five or ten milliseconds.

One thing that is notably absent from almost all of the observed time series is any pattern of diurnal variation, with one exception:

For both hosts in this network (located in South America), RTTs remain fairly stable from around 06:00 UTC to around 12:00 UTC.  This interval corresponds to night time in the time zone where both these hosts and the measurement host are located (UTC-4, UTC-5), so this variation could be attributable to congestion in inter-continental links.

There is also a question as to the impact of 6to4 on packet loss rates. In this initial sample, there was not much observable impact on loss, except for when 6to4 didn't work at all (2 hosts).  There were no hosts for whom 6to4 pings worked and native ones didn't.  There were nine hosts that were unreachable over both native IPv6 and 6to4, but six of these were within major networks known to have connectivity issues with Cogent.

The results already collected will provide a baseline for World IPv6 Day. If the difference between native and 6to4 back-paths RTT and packet loss on World IPv6 Day increases, this would be an indication of problems with 6to4 relays.

In the above results, we've anonymized the targets of the measurements. If you own one of the hosts we're measuring (if you're a World IPv6 Day participant or if you've volunteered), and you would like a copy of your own results, please let us know at labs@ripe.net .

We're planning to keep these measurements running at least through World IPv6 Day, hopefully longer, and we're hoping to add more observation points over time.   If you have an IPv6 host or 6to4 relay that you would like us to measure, or if you would like to run our measurements from your site, please let us know at labs@ripe.net . Otherwise, just keep an eye on RIPE Labs for updated results over the next couple of weeks.