Border Gateway Protocol (BGP) is the only exterior gateway protocol created to control traffic between Internet Service Providers (ISPs) all over the world. The initial protocol design was so flexible that after 20 years, it has become a de facto standard mechanism for MPLS/VPN and even Interior Gateway Protocol in large-scale data centres.
However, each BGP network - data centres, ISPs or the Internet as a whole - is a network of trust because BGP itself does not provide any mechanisms of data authorisation and validation. And this trust does not scale well. This results in two major BGP anomalies:
- BGP hijacks - when an ISP advertises address space that does not belong to it.
- BGP route leaks - when an ISP advertises prefixes received from one provider or peer to another provider or peer.
Such incidents have already become regular (see list below), with thousands of them happening on a daily basis. The majority of these are the result of misconfiguration, but BGP has also become a popular target for attackers since it provides them with a native mechanism for man-in-the-middle attacks.
Notable incidents that took place during two weeks at the end of April and beginning of May 2018:
- 24 April: ENET, AS10297, US: Malicious BGP and DNS hijack. The attack was successful.
- 26 April: DPSTL, AS267286, Brazil: Mega-hijack. A Brazilian ISP announced 16 /8 prefixes plus several smaller prefixes; ~5% of the entire IPv4 address space in total.
- 4 May: ELCAT, AS8449, Kyrgyzstan: Giant route leak between China and Russia.
If a routing event hits an entire region it may be quickly noticed nowadays thanks to the media (both social and mass media). Nevertheless, most of the BGP anomalies are not detected in time and as such can last a couple of days as providers work to fix faults.
IRR filters: investigation
The problem with BGP hijacks and leaks isn’t a new one and there has already been a significant effort to fix it.
The most common mitigation technique is to use ingress filters that are built using the Internet Routing Registry (IRR). The idea is simple: using route objects and AS-SETs to create ingress filters at customers links.
The underlying problem is that both AS-SETs and route objects are not authorised (vary from one IRR to another), and there may even exist different objects with the same name in different IRRs. The problem is amplified by the fact that the presence of IRR filters themselves are not an obligatory law; they are just a recommendation. In this instance, flexibility does not do the community any favours.
These vast routing events that are propagated globally already provide a hint that some ISPs do not set filters at all, or there are vastly malformed AS-SETs. We decided to measure the number of filters that were already bypassed by routing anomalies. To do so, we checked the way route leaks were propagated: if a route leak is received from a customer link and it does not belong to the customer cone then IRR filters were malformed.
We learned that 7% of transit ISPs in IPv4 and 1% of transit ISPs in IPv6 are accepting leaks outside of their customer cone. One might say that, ”The score is not that big”; but lets take a look at the results sorted by ISP size:
Figure 1: Percentage of violated filters by ISP size
All Tier-1 providers were affected, and more than 50% of the TOP 400 IPv4 ISPs were also affected. IPv6 looks slightly better, but it is still under development, so it has fewer prefixes, ISPs, and routing anomalies. Also, there could be more hidden problems as we were studying filters that were already bypassed.
With the help of the monitoring system, we also checked the route leak distribution for prefixes belonging to Tier-1 providers. Note — you should never accept Tier-1 routes from your customers! The simple presence of such anomalies gives evidence that there are no ingress filters, or they are totally broken.
It was surprising to see the huge transit ISPs from Russia and China in this list, as well as European Internet Exchanges. The presence of Hurricane Electric in the list, which is a significant player in IPv4 and Tier-1 networks in IPv6, should make all of us really anxious as well. These networks have verifiable evidence of the absence of or significantly broken IRR filters because they were accepting and distributing leaks originated by Tier-1 providers.
Check out the presentation I gave at RIPE 76 to see our list of worst affected providers for IPv4 and IPv6.
From further research, we have learned that only DE-CIX and Rostelecom intend to check their IRR filters and work with their customers to study the AS-SETs quality problem.
Tips to survive BGP incidents
The survival guide in the wild world of BGP still remains the same. You should:
- Keep your AS-SETs and route objects up to date.
- Install IRR filters at all your customer links without any exceptions.
- Work with your customers if they add something weird in their AS-SETs.
- Consider applying IRR filters at your peering links.
- Constantly monitor BGP.
At the end of the day, for the end-user who is redirected to a phishing site and loses their money, it doesn’t really matter if you learned about a leaked/hijacked route from your customer link, peer, or uplink. It’s your obligation to avoid such scenarios.
I will provide an overview of next-generation BGP filtering tools, including BGPSec and ROA usage and vulnerabilities in my next article — stay tuned.