The 1M Day Isn't That Far Away

Back in August 2014, we witnessed a series of Internet "hiccups" worldwide when the number of IPv4 prefixes in the global BGP routing table hit the 512K boundary. A nice RIPE Labs article, along with references mentioned in there, explained the reasons behind this and illustrated the effects from the RIPE RIS point of view.

Last year, when the IPv4 prefix count reached 768K, there was a moderate concern that a similar issue could occur. Luckily, it turned to be too much ado about nothing, as explained in another RIPE Labs article. 

However, in two years from now, probably somewhere around June 2022 (+/- 6 months) we'll be facing another Internet milestone - one million IPv4 prefixes in the global routing table, according to the extrapolated BGP stats obtained from bgp.potaroo.net. Look at the graph below, created using the data from 2019. As you can see, the trend line is still valid. And while some people may argue about whether 1M is precisely 1,000,000 or 1,048,576, the difference is not that huge. What is so special about this? And why should you care?

Over the last decade, many vendors performed stress-tests on their routers and switches by generating 1M IPv4 prefixes. Upon successful tests most of them marketed their products with the statements like "Certified for 1M routes". 1M seemed to be enough, even though some routing platforms could process more than that. However, most of those routers were designed to store and process at least 1M prefixes. Of course, many routers nowadays can take much more, the order of magnitude in the industry is currently around 10M, so if your equipment is less than ~7 years old, you should probably forget all of this.

Still, better safe than sorry! And the check is usually simple: create an inventory of all the routing devices in your network running the full BGP global routing table and then ask your vendor to give you a clear answer about the maximum number of IPv4 and IPv6 prefixes those routers can take. You should pay attention to two different figures:

• Control plane prefix capacity (a.k.a. RIB max prefix count): The number of routes that your network processor module (usually called RSP, CPM, Supervisor board, Routing Engine...) can store and process. Routing Information Base (RIB) is a copy of the routing table stored on the network processor module, used by various routing protocols (BGP, OSPF, IS-IS etc.). The limiting factor here is the amount of RAM on your network processor board. This is mostly not an issue nowadays, since modern network processors can easily handle more than 10M prefixes.
  
  
• Forwarding plane prefix capacity (a.k.a. FIB max prefix count) - The number of routes that can be stored in your router's Forwarding Information Base (FIB). The FIB is being used by the router to decide on which egress interfaces to push the received IP packet, based on its destination address. Depending on your router architecture, the FIB may be stored either on the router linecards or in some kind of a centralised switching component. In either case, modern routers use TCAM to store IP prefixes, so the TCAM size determines the prefix count. Most of the times, this is the limiting factor, especially for older devices.

If you have the time and resources, you should definitely test those limits yourself. If you are lucky to have a separate test lab or at least one test device of the same model/type used within your live network, you should generate at least 1.2M BGP routes and see how your box behaves. You don't need expensive testers (e.g. Ixia, Spirent etc....) for that. There was a good article on ExaBGP (also here) on RIPE Labs. NOTE - the code was migrated from Google Code to a GitHub, see ExaNetworks exabgp repository. You can install ExaBGP on a Linux server and use it as a test generator. ExaBGP has a very nice add-on 'super-smash-brogp, allowing you to generate an arbitrary amount of IPv4 and IPv6 prefixes. Just set it to generate 1.2M. Observe the results, check RAM and TCAM utilisations, check router logs to see any signs of rejected prefixes. Contact your vendor if you see serious issues and start making mitigation plans.

Do not wait for bad things to happen. Be proactive!