I presented these results at the RIPE 73 meeting last month (slides, video). Please also note the earlier article on this topic: How are you Deploying IPv6? Take this Survey.

These findings include a comparison of how ISPs are deploying IPv6 to residential customers in different countries/ RIR regions, as well as some observations about possible deployment mistakes or misunderstandings about IPv6.

Who responded?

As of 23 October 2016, 1,199 people had responded to the survey, of which 31% used IPv6 to participate in the survey (something I observed by looking at the logs from the apache server hosting the survey).

Figure 1: IP version used by people responding to the survey

Looking at whois and using other tools to correlate the data, I found that around 50% of those who responded are employees of ISPs, most of who responded from their own networks. Customers of ISPs responded typically from their residential network. 

I have verified that almost every response is from an ASN that has both IPv4 and IPv6 allocations. This raises an interesting question: Why are 69% of ISP employees responding using IPv4? This can be explained because in some situations, corporate LANs, even in ISP networks, have not yet deployed IPv6 in all their subnets. This is one of the first points we need to correct because the more users with technical knowledge and experience using IPv6, the faster we will be able to detect issues from broken IPv6 deployments. 

Another interesting observation from the logs is that many users tried to connect to the survey with IPv6, but it reverted to IPv4. Checking the connectivity to those locations, it doesn’t look like IPv6 is much slower than IPv4, so it means that in some cases the Happy-Eyeballs mechanism is failing for other reasons, such as broken Path MTU Discovery. 

This is something that I’ve observed on many occasions among data centres’ deploying websites with IPv6, either because they wrongly filter ICMPv6 or because some other mechanism, such as ECMP, silently breaks it. These problems point to a need for:

1. More experienced IPv6 training to avoid creating the problem, and
2. More complete IPv6 monitoring to discover these kind of deployment mistakes. 

It also begs the question if we should reconsider Happy-Eyeballs because of this problem of it hiding IPv6 deployment errors: we don’t have a similar mechanism in IPv4 and that’s why we discover, almost immediately, any network problem in IPv4! 

Where did responses come from?

The distribution of responses to the survey in different regions looks quite normal considering the degree of IPv6 deployment in each one.

Figure 2: RIR regions of survey respondents

There is a small exception to this distribution, with a high number of responses from LACNIC, in particular Brazil, from where we received 200 responses.

Figure 3: Respondents as per economy

If we analyse the responses comparing regions/countries, some countries in each region have a bigger IPv6 deployment, including the USA in the ARIN region, Belgium in the RIPE NCC service region, Brazil in LACNIC, and Japan in the APNIC region (there is almost no IPv6 deployment in AFRINIC). 

In terms of potential gaps in the data, we are missing responses from two countries with big Internet penetration in the APNIC region: South Korea and India. 

This is one of the reasons why I will continue to run this survey to get a more detailed picture of IPv6 deployment in the APNIC region. 

Please take my survey on how you are deploying IPv6.

What technologies are ISPs using to deploy IPv6?

31% of respondents indicated their IPv6 service is already commercial and another 17% indicated it is still in trial stages. Considering that 52% of respondents answered “no answer”, we can extrapolate that almost 64% of the responders have a commercial service. 

Figure 4: Percentatge of respondents who represent organisation where IPv6 is commercially available

Recent technologies such as FTTH (35%), xDSL (22%) and Cable/DOCSIS (20%) were among the most popular across the different regions. This result confirms that ISPs avoid investing in replacing CPEs, such as older DOCSIS 1 deployment, or first xDSL ones.

Figure 5: Prefered technologies used to deploy IPv6

Looking at the evolution of the responses in previous months, and responses to question on transition mechanisms, I observed an increase in the deployment of CGN. At the same time there is a small increase in the usage of transition technologies such as 464XLAT (and in a minor proportion MAP T/E), in order to provide IPv6-only access to the residential customers, which is very consistent with what has been happening in cellular networks in the past few years.

Figure 6: Transition mechanisms respondents are using

Overall, responses related to the IPv4 technical details or the transition technologies are similar across the regions and countries surveyed. 

Note: I don't have results or any conclusions on whether there is any specific difficulty deploying IPv6 specifically related to access to technologies or vendors. 

WAN

With regards to the technical details of the WAN link, 61% are using /64, 13% are using unnumbered. The rest are using /112, /126, /127, or other choices. However, looking closer at the details of the “other” responses, many of those who answered the survey are non-technical people and may have interpreted the question wrong. Therefore, it is likely that there are more /64 users.

Figure 7: WAN prefix size

When asked about if their WAN prefix is stable (across reboots, etc., for the same customer in the same location), 18% responded "yes", 11% responded "no", and 71% responded “no answers”. From these results, we can extrapolate that the figure is in the region of 62.5% stable and 37.5% non-stable. My view on this question is that non-stability is related mainly to the provisioning systems, not something done on purpose.

Figure 8: Percentage of responsents indicating that their WAN prefix is stable

With regards to what WAN addressing type they use, 63% of respondents indicated GUA, 24% link-local, 11% ULA and 2% other.

Figure 9: WAN addressing types

We also asked if the WAN is from the same pool as the customer prefixes, which 7% responded "yes", 9% "no", and 84% “no answers”. Considering this large proportion of “no answers”, it is interesting that 66% of ISPs are using the first /64 from the customer prefix, while 34% are not.

Figure 10: WAN from same pool as customer prefix

This practice is a very convenient, non-standard way that is supported by almost every vendor. It was described in an Internet Draft in 2006, however, did not progress in the v6ops IETF working group so I let it expire. As a consequence of this survey, I’m intending on reviving it to seek the support of the WG and make it a Best Current Practice document.

Figure 11: WAN /64 from customer prefix

Overall, there were no major differences in the answers to the WAN related questions between regions and countries. 

LAN

Below are the results from a similar set of questions as those above, this time regarding the LAN prefix. 

With regards to what prefix is allocated for customer's LANs, 22% of respondents indicated that they are using /48, 35% indicated they are using a /56, 33% a /64 and 10% other sizes (among them, /60, /62, /57, /127 and /128). Some responses seem unrealistic though (/29, /44).

Figure 12: LAN prefix size

The above results reflect a major allocation problem: the provision of a single /64 for customer LANs. This is a totally “broken” concept, as it implies that customers are not able to do subnetting in their networks, or arrange different VLANs, or different SSIDs (such as a guest one). The reason for this, most likely, stems from the concept that one address is enough because we have NAT, but this is only true in IPv4! 

Again this is highlighting a lack of expertise, or even worse, that people doing IPv6 training are missing key points, which means that people doing those deployments will need to spend time redoing it. 

Looking at the distribution of those responses among different regions/countries, most of the “broken” deployments (/64) are being done in the LACNIC region and in a few APNIC countries. More “advanced” countries (in terms of IPv6 deployment) like those in northern Europe, Japan, Australia, New Zealand and China, are using /48, while the rest of the countries in the RIPE NCC and ARIN regions are using /56. 

With regards to customer LAN prefix stability (same prefix for the same customer in the same location, persistent across reboots, etc.), 17% respondents answered "yes", 11% "no", and 72% “no answer". From this, we can infer an actual proportion globally being 60% "yes". 

This is another example of having an “IPv4 mentality” when deploying IPv6 because we don’t see the need for an end user to have stable addresses. In IPv6, it is clear that if our customers have stable addresses, we are making it easier for our customers, ourselves and third parties, to develop, deploy and provide services, which in turn means increasing our network value and our turnover. 

Again, correlating this data by regions, we can see that mainly organisations in the AFRINIC, RIPE NCC and APNIC regions are doing stable assignments to customers, while ARIN is mainly non-stable, and LACNIC is almost half and half. We also asked if offering the stable prefix is an “extra” or “opt-in” option, and if this option means “extra cost”. As can be seen in the pie charts, the large majority provide IPv4 service as well as the IPv6 one, and still there is an important proportion of ISPs providing a public IPv4 address.

Figures 13 - 16

Figures 17 - 20

DNS

The last set of questions related to the DNS aspects which have provided similar observations as those for prefix stability. It is obvious that addressing devices or services inside the customer network can’t be easily done using IPv6 addresses, so having some kind of DNS “naming” service makes sense. For example, the customer can have IPv6 cameras named as “camera1.customer.ispname.com”, and so on.

Figure 21: Respondents using IPv6 reverse DNS

Figure 22: Is there an NS delegation for a stable IPv6 prefix?

Figure 23: Is there a DNAME for non-stable IPv6 prefixes for PTR records?

Takeaways

The most worrying takeaway for me from this survey so far is that there are still a lot of engineers that try to deploy IPv6 the same way they deploy IPv4, which is plain wrong. 

The main misunderstandings are related to the prefix size assigned to the customer and their stability. One of the most common mistakes, in my experience, is that many ISPs get the default allocation from their RIR (/32 in most cases, /29 in the RIPE NCC service region), without preparing a proper addressing plan. This means that if the ISP is not small, or if they are using a transition mechanism such as 6RD, a /32 can be too small to assign a “correct” prefix to the end-customer. 

Another key issue, which is not directly related to the deployment of IPv6 in residential areas, but can still affect users (because access technologies usually impact in the reduction of the MTU), when deploying IPv6 in web sites or other services, is to either filter ICMPv6 on purpose, misconfigure load-balancers or using technologies such as ECMP without taking measures to avoid breaking PMTUD. Once more this happens because the lack of expert training and the common “IPv4-mind-set”. Moreover, many ISPs aren’t doing the same degree of monitoring for IPv6 as they do for IPv4, so commonly those errors are not advertised. 

This is the first report from the survey. As mentioned I plan to keep it open and report on it over the coming years with a particular goal of targeting countries where we have collected very few or no responses. We also want to study the possible implementation changes when ISPs discover that they are doing something wrong or can improve the way they actually deploy IPv6.