DNS operators have very few intelligent real-time tools that enable them to monitor their anycast services, for instance during a DDoS attack. In this post, we describe the challenges associated with measuring anycast services and propose a tool called the BGP Tuner. By using our open-source tool, operators can see in advance how changes in their BGP policies may impact the traffic load distribution over the anycast sites.
Anycast services
Anycast is a way of using the Internet’s routing system so that an incoming request can be routed across multiples sites, with the routing system dynamically selecting the one with the smallest number of hops from the client. Anycast can reduce latency, improve network resilience, and defend against DDoS attacks. It is used extensively by root DNS operators, DNS service providers, and content distribution networks (CDNs), including Google, Facebook, and Cloudflare.
Why monitor anycast networks?
Operations teams need to be able to respond to various issues in a real-world setting, for instance, related to:
- Traffic engineering: managing the way the Internet routing system distributes clients across anycast nodes.
- DDoS mitigation: distributed denial-of-service attacks may overload anycast sites; operators can reconfigure the anycast service and shift the load across the available sites.
- User experience: ensuring clients are served by the most-suitable anycast site.
To this end, we should answer the following questions:
- How do clients on the Internet reach my anycasted service?
- How can we change/shift the load of traffic reaching my anycast sites?
How can we monitor an anycast service?
In IP anycast, the BGP protocol routes each network to a particular anycast site, dividing the world into catchments. The behavior of an anycast service is very tightly coupled to BGP. To monitor how BGP routes clients to our anycast service, we have to consider a few approaches. Broadly speaking, there are two possible approaches:
- Using client-side vantage points such as RIPE Atlas to determine which server each client is reaching
- Using the open-source tool Verfploeter
In this investigation, we used the Verfploeter tool, because it has a broader coverage of clients (see details here). To measure the client’s distribution, we carried out real-world measurements using the Tangled Anycast Testbed.
Anticipated effects on anycast services
The client distribution over anycast sites is tied to BGP routing visibility. More than evaluating the changes in routing visibility, it is important to develop a systematic way to measure and map the side effects of each BGP configuration. For example, what are the load distribution implications of implementing an AS-Path prepend?
With this in mind, we developed a systematic way of measuring and mapping the side effects of each BGP configuration performed. The following steps were defined:
- Build baseline: measure and map the distribution of clients across our anycasted sites using the regular BGP configuration.
- Apply changes: change the BGP policy and redo the measurement.
- Traffic shift: redo the measurement and carry out mapping to compute how clients are distributed across our sites after the change (i.e. the deviation from the baseline).
BGP Anycast Tuner
The BGP Tuner is a prototype graphical interface that presents the distribution of clients over the anycast sites using a pre-determined BGP configuration. We used Verfploeter to carry out the measurements on Tangled Anycast Testbed, pre-processed them, and afterward used them as the input for our interface.
This interface assists the operator in an intuitive way, allowing them to "equalise" the traffic volume or catchments on each site in a simple way. An interactive display shows the traffic distribution across the sites when the previously measured BGP policy is applied. The central graph shows the distribution of clients across the respective sites.
The operator can play around with the sliders to increase/decrease the number of clients per site. Note that the sliders do not make linear adjustments; instead, every mark on the slider scale corresponds to a previously measured BGP policy. For some sites, the marks are spread along the full length of the slider scale, while for others the marks are more concentrated. The differences reflect the various sites' ability to shift clients between anycast nodes. Some sites can exert more influence because of their relationships with their neighbors and their traffic agreements.
Our tool enables an operator to find the best available policy for that operator's preferred outcome, i.e. to increase or decrease the number of clients reaching a particular site. Operators can also use more advanced policies. For example, on the left side (drop-down menu) there are options such as "Bring traffic to Europe” and “Reduce traffic to the USA”. Such options will help operators to identify which of the presented policies are most suitable.
The BGP Anycast Tuner is an open-source tool and available here.
Acknowledgments
The findings of this investigation and the tools developed from it form part of the SAND project, a joint effort by SIDN Labs, NLnet Labs, and the University of Twente that started in April 2018 and finished in April 2020.
This post is a short description of our technical report available here.
Comments 0
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.