Introduction

OTE Group (AS6799) is the largest ISP/Telco in Greece and one of the leading telecoms groups in Southeastern Europe with 2.6 million fixed-line, 1.8 million Broadband and 8 million mobile customers.

In an effort to mitigate the impeding effects of public IPv4 exhaustion, the engineering team undertook a feasibility study in 2014, which amounted to a plan to move towards an IPv6-only residential service - IPv4 was to be treated as a service over the IPv6 network.

The desired characteristics of an IPv6-only service were: its stateless nature, distributed deployment, the possible complete removal of IPv4 in the future, the use of network functions virtualization (NFV) for scalability and cost efficiency and a simple design.

Having considered the transition mechanisms developed by the IETF, MAP-E (RFC 7597) and Lightweight 4over6 (RFC 7596) were recommended as the appropriate means to mitigate IPv4 address exhaustion. Being a part of Deutsche Telekom Group, lw4o6 was chosen due to DT’s involvement with the technology.

Below is a brief explanation of the technology, the challenges we had with deploying it to the network and some ongoing work.

Lightweight 4over6 overview

Lightweight 4over6 (lw4o6) is an extension to DS-Lite which moves the Network Address and Port Translation (NAPT) function from the centralised DS-Lite AFTR to the tunnel client located in the Customer Premises Equipment (CPE). This removes the requirement for a Carrier Grade NAT function in the tunnel concentrator and reduces the amount of centralised state that must be kept to a per-subscriber level. Port-restricted, shared, public IPv4 addresses are allocated to the CPEs along with necessary parameters, via DHCPv6.

Please also refer to the recent RIPE Labs article by Diego Pino Garcia: Lightweight 4over6: One-step further than dual-stack networks.

IPv6 traffic leaving the network follows the existing native path and all outgoing IPv4 traffic is encapsulated in IPv6 and directed to a tunnel terminator (the lightweight address family transition router or lwAFTR) where decapsulation is performed (see Figure 1).

Figure 1: Traffic is directed to a tunnel terminator (lwAFTR) where decapsulation is performed

Incoming, return IPv4 traffic hits the lwAFTRs where it is encapsulated in IPv6 and forwarded to the end-user.

Figure 2: Return IPv4 traffic goes through the lwAFTRs where it is encapsulated in IPv6 and forwarded to the end-user

lwAFTR VNF

lwAFTR is the softwire termination point and performs only encapsulation/decapsulation of traffic based on binding table rules. Since it performs no NAT function, it is considered a stateless function (strictly speaking, it maintains minimal state by means of a binding table). The lwAFTR maintains a binding table containing the binding between the CPE's (lwB4) IPv6 address, the allocated (public) IPv4 address and the restricted port set.

As part of our deployment, we issued an RFQ process about a lwAFTR VNF running on COTS H/W that could accommodate multi-10Gbps traffic with predictable performance as a deliverable. No solutions were mature at the time (very short after RFC publication) so development was done alongside the RFQ process.

The selected solution uses an open source fast data plane networking toolkit called Snabb. It follows the user-mode networking approach, ethernet I/O with no kernel overhead (‘kernel bypass’ mode). lwAFTR deployment is packaged as a docker container running on Linux commodity servers.

Deployment at production network

With around 32,000 active users in production and 12 Border Network Gateways (BNGs) at the time of writing, the deployment at the production country-wide IP network involved many components, including:

• CPE support (focused on the one with the highest user penetration)
• BNG configuration (we requested several DHCPv6 features from our vendor)
• RADIUS configuration per user (we utilise freeradius open source software, we control all aspects of the deployment)
• Central DHCPv6 in 2 DC locations in Athens (using ISC DHCP)
• lwAFTR in the same DC locations (the stateless nature of lwAFTR, allows for a resilient anycast deployment)
• Monitoring / measurements (we utilise Juniper's open-nti open source software)
• Provisioning / automation scripts (developed internally in Python/Perl)

Challenges

This was a multi-year project with several challenges. We faced issues on the CPE implementations and we still have few cases that remain unresolved.

We also faced lwAFTR-related performance issues, resolved with the help of our vendor - lwAFTR performance improvement is ongoing.

In terms of DHCPv6, our deployment’s scalability remains unknown.

We also faced expected issues with users having services over IPv4 (mostly IP cameras).

On the plus side, no dependency exists between IPv4 and IPv6 addressing, which allows for an incremental deployment approach. No noticeable MTU/fragmentation issues were observed and our anycast lwAFTR deployment provides flexibility on traffic routing.

Finally, capacity increase in the lwAFTR setup can be easily achieved.
 

Future work

A significant amount of work is still pending. We need to globally expand the lw4o6 service in all our BNGs and lighten/offload the current CGN setup.

We need to improve our central DHCPv6 deployment that will also enable ‘static’ IPv6 prefixes to subscribers. Our radar view also includes: the implementation of different classes of service (different-sized port range sets), standardised provisioning via YANG and exposing the IPv4 port ranges of users. These will be described in a future blog post.

We are actively seeking cooperation with other operators in similar service setups.