The IRR Landscape: What RPKI Can Replace

This article is part of a series inspired by discussions on the connect-wg mailing list on a BCOP proposal on fading out non-RIR Internet Route Registries (IRRs) at IXP route servers. The series provides an overview on the IRR landscape, in particular how ASes use IRR databases, how good the data quality in different IRR databases is, and an impact analysis of the BCOP at two major IXPs.

In the previous two parts of this series, we investigated where ASes store their routing security information, followed by an investigation of the quality of the data.

Based on the insights from those articles, we now shift attention to which parts of the Internet Route Registry (IRR) landscape can - or cannot - be replaced safely with information from the Resource Public Key Infrastructure (RPKI) system.

We simulate the impact of deprecating different parts of the IRR system and replacing them with RPKI wherever applicable using Route Server (RS) and traffic data from multiple IXPs operated by AMS-IX and DE-CIX around the globe.

As noted earlier in this series, the IRR ecosystem is not homogeneous. For our analysis, we separate the existing IRR databases into two broad categories:

• Authoritative databases¹ are operated by Regional Internet Registries (RIRs) like RIPE or their official delegates. These IRRs are tied to the organisations responsible for allocating and managing Internet number resources and can therefore validate route objects against resource-holder information.
• As opposed to that, third-party IRR databases² are operated by companies or non-profit organisations outside the RIR system. Naturally, these databases are less tightly validated against authoritative ownership information. Consequently there is significantly more problematic information in third-party IRRs than in authoritative IRRs and we focus on the third-party IRR ecosystem in this study.

Readers who want to revisit the basics of IRR databases, their role in documenting AS routing policy, and the way they are used operationally may find the background section of the first article useful.

BGP filter generation at AMS-IX and DE-CIX

Enforcing the routing policies documented in IRR (and RPKI) databases is the daily job of network operators. As the process is implemented differently across networks, we discuss how AMS-IX and DE-CIX handle the process in particular. To do so, AMS-IX and DE-CIX generate allow-list style prefix filters from IRR data to improve routing security on their RSes. At a high level, this process starts with (1) selecting a routing policy entry-point for each IXP member, usually derived from an AS-SET RPSL object; continues with (2) expanding that AS-SET recursively to derive the ASNs in the member’s customer cone; and ends with (3) resolving those ASNs to the prefixes that are derived from the corresponding ROUTE(6) objects. The result is a per-member list of permitted announcements, for example: AS65536 may announce 192.0.2.0/24, 198.51.100.0/24, 2001:db8::/32 , etc.

Overall, three criteria are taken into account when receiving a BGP announcement from an IXP member:

1. IRR origin validation: is the originating AS inside the announcing IXP member’s customer cone? This only checks AS relationships - not prefixes.³
2. RPKI ROA origin validation: is the originating AS authorised to announce this prefix?
3. IRR customer cone validation: is the announced prefix in the recursively resolved prefix allow-list of the announcing IXP member? This checks whether a prefix can be resolved through the member’s customer cone.

A full table of the validation results for both IXPs can be found in the appendix. For the sake of simplicity, we only discuss the most interesting ones, including the combination where the two IXes differ, as this is helpful for interpreting the traffic measurements later on.

The differences mainly stem from how the two IXPs combine RPKI and IRR data. DE-CIX accepts ROA-covered routes without ROUTE objects if the originating AS is part of the member's IRR-derived customer cone. AMS-IX however requires both the ROA and standard IRR validation to pass.

Further, DE-CIX uses customer-provided AS-SETs including IRR DB preference, supports a broader range of IRR sources,⁴ and prioritises the provided IRR DB during resolution. AMS-IX instead derives AS-SET references from AUT-NUM objects and generally queries a narrower set of IRR databases.⁵

What IRR data can be faded out?

While much community attention has focused on RPKI adoption, comparatively little work has asked how data from the legacy IRR system can be phased out. One option is to wait for an organic takeover by RPKI mechanisms, but this is likely to take a long time: ROAs can replace ROUTE/ROUTE6 objects where coverage exists, but the operational replacement for AS-SET-based customer-cone discovery is still under standardisation with ASPA.

The practical question is whether we can cut the operational headache of maintaining two policy mechanisms in parallel by phasing out the most problematic parts of the IRR ecosystem earlier. We address this question by measuring how many routes and how much traffic at AMS-IX and DE-CIX still depends on different classes of IRR data, and which parts of that dependency can already be removed without significant impact on Internet routing.

We analyse the IXP’s production configuration vs. a scenario where we (a) drop all third-party IRR data and one scenario where we (b) only drop third-party ROUTE(6) objects, but still allow third-party AS-SET objects. We will refer to the former as the "strict scenario", and to the latter as the "loose scenario". In both scenarios, we add pseudo ROUTE(6) objects generated from RPKI ROAs. The following table summarises this as an overview:

Experiment setup

We build an experimental setup to measure the scenarios described above. We aim to keep it as close to production at AMS-IX and DE-CIX as possible by using production tooling and deviate only if necessary. The following figure gives an overview of the different building blocks used:

Both IXPs rely on a custom Route Server toolchain (3) that retrieves data from an IRRd instance (2). This IRRd instance imports static dumps⁶ of IRR and RPKI data (1), the latter of which is used by IRRd to generate pseudo ROUTE(6) objects from ROAs (2).

We use IRRd’s object_class_filter (2) to manipulate object mirroring in IRRd according to the scenario definitions from the previous table. This feature allows operators to define an allow-list of object classes to import per source. Any object class not defined in the object_class_filter definition is ignored upon import and therefore not available when queried by the IXP toolchain. We consider this the safest and most operationally viable approach for filtering unwanted objects from specific IRRd sources. Sample IRRd configurations from our setup can be found in the appendix.

With this IRRd setup, the remaining toolchain runs unmodified and generates configurations (3) for the AMS-IX and DE-CIX RSes, respectively. We then load these configurations into an emulated BIRD2 deployed in a virtual peering LAN (4), acting as an emulated RS. To emulate the IXP member side, we extract information about route announcements and attached action communities from the respective IXP’s production RS which reflect real-world peering relationships. Using this information, we spin off one BIRD2 instance per IXP member which then connects to the emulated RS to announce its routes, including communities. Once the setup has converged, we save the export table for each connected member into separate routing table dumps (5). This approach allows us to determine precisely which member loses which route under which conditions when comparing different scenarios.

We then measure production traffic data based on flow samples that are exported from production PE routers, either in IPFIX (AMS-IX) or Cflowd (DE-CIX) format. The traffic samples are collected in a central database from where they become accessible to the R&D tools. The internal tools are being supplied with the lost prefixes on a per-member basis, in order to classify the traffic correctly.

It is however not possible to distinguish whether traffic destined for a lost prefix was steered via the RS or steered via a private BGP session established directly between two members. Based on prior DE-CIX estimates, we can estimate the share of traffic likely steered over private sessions at 40-60% of total traffic. However, we do not know whether traffic falling into lost prefixes is representative of overall traffic, which would be necessary to apply this factor with confidence. Therefore, all measured traffic numbers presented in this article should be interpreted as a conservative upper-bound worst-case estimate.

Routing table results

We first investigate the changes in the RS RIB and the member's export tables across the different scenarios. We find that, for a given IXP and scenario, virtually all member's export tables are affected in a very similar manner. We therefore select a "representative member", i.e., the member that is representative for the changes seen by the vast majority of all members.

In the following table, we first report production data as a baseline, i.e., the total number of routes in the RS' main RIB (# Routes in RIB) and the number of exported routes (# Exported Routes) for the representative member. We then report - for both scenarios - the impact on the number of exported routes to the representative member (Impact on exported routes) and the fraction of exported routes that see a next hop change (Changed AS paths). Traffic to routes with an updated next hop would be redirected via another AS path through the Internet.

We find that the strict scenario has a significant impact on the routes in the export table for the IXPs’ members. The impact is especially pronounced in DE-CIX DFW (-95.6%) and the APNIC region (-92.3% and -88.3%). Notably, for the RIPE region, almost a tenth of the remaining routes would take a different route through the Internet.

Looking at the loose scenario, the impact is much smaller both in terms of route availability in the representative member’s export table and in terms of their changed next hops. Two of the IXPs even see an increase of prefixes accepted by the RSes as a result of additionally adopting RPKI data in the configuration pipeline as pseudo ROUTE(6) objects.

Traffic results

We now move our attention to the traffic impact by measuring how much traffic is sent into the prefixes that vanish from the representative member's export table in our two scenarios. It is worthwhile noting again that we cannot know if some of the affected prefixes were also exchanged between members bilaterally, i.e., without involvement of the RS. Such bilateral prefix exchange is immune to dropping third-party IRR DBs from the RS toolchain and the respective traffic continues to flow. As we cannot identify these fractions of the traffic, our method is expected to overestimate the actual impact, leading to very conservative numbers being reported.

The results show that the strict scenario consistently has a larger impact than the loose scenario. This indicates that AS-SETs contain relevant information for BGP filter generation at the IXP’s RS. We’ll detail on the traffic impact split by different RIR regions and legacy:

• Impact within RIPE: At DE-CIX FRA, the strict scenario leads to a peak traffic impact of -2.6%, whereas it drops to -0.5% in the loose scenario. The impact at AMS-IX AMS is not as pronounced in both scenarios, -0.6% and -0.4%, respectively, as already fewer third-party IRR DBs are used in production.
• Impact outside RIPE: For the IXPs in the ARIN and APNIC region, the impact can be substantially higher. In the strict scenario, DE-CIX DFW loses up to 7% of peak traffic, AMS-IX HK up to -12.2% and DE-CIX ASEAN even -20.2%. In the loose scenario however, the impact is only -0.9% in DE-CIX DFW and -1.6% in DE-CIX ASEAN, while AMS-IX HK remains almost untouched. The only exception from the overall observation that the strict scenario has a greater impact than the loose one is AMS-IX LAGOS. Here, a single member causes almost all traffic impact at the IXP in both scenarios. This member has no ROAs and stores all its data in a third-party IRR DB leading to the same impact in both scenarios.
• Impact of legacy space: Based on our previous work on legacy address space in the IRR system, we also investigate how much of the lost traffic falls into identifiable legacy space.⁷ The results show similar impacts across regions, but differ by scenario. About half of a percentage point of the traffic impact in the strict scenario belongs to prefixes identified as legacy space. In the loose scenario however, this drops to insignificant levels, at least outside ARIN. At DE-CIX DFW, we see that legacy space plays a larger operational role. In contrast, for example, the APNIC-region shows high overall traffic impact at DE-CIX ASEAN, but only a tiny but unsurprisingly small legacy-space share. This suggests that the ASEAN result is not primarily driven by legacy address space, but by a broader dependence on third-party IRR data. Overall, legacy space contributes to the observed impact, but its share remains comparatively low.

We notice a stark discrepancy when comparing route loss and the traffic impact. The route loss alone looks dramatic, especially outside the RIPE region, but the impact on traffic remains mostly modest overall. This is also in line with the second article of this series, where we showed (with heuristics) that large fractions of third-party ROUTE(6) objects are problematic in terms of data quality. In addition, this result indicates that these objects generate operational and technical stress for a low added value in terms of traffic. Moreover, they lead to BGP filtering logic that is actually a lot more loose in terms of routing security than it could be.

Traffic impact concentration

We now seek to understand how the prefixes that are lost and carry traffic are distributed over the IXP’s members. We were unable to calculate these numbers for the AMS-IX IXPs due to technical reasons and therefore only present the results for the DE-CIX IXPs.

The above plot shows the fraction of peers (Y-axis) that contribute more than X% of the overall traffic loss (X-axis). The concentration of lost traffic to prefixes announced from different members is exceptional. In all scenarios, and across the three different DE-CIX IXPs, virtually all impacted traffic is sent to prefixes announced by a handful of different IXP members. In an extreme case, 96% of all impacted traffic is attributed to lost prefixes announced by a single member (blue line). For about 90% of all members, no traffic is sent to the prefixes they lose, if any. This suggests that the traffic impact would easily be much lower if only a small number of members - or their downstreams - moved their objects to authoritative IRR DBs.

Discussion

Our series of three articles correlates various public and IXP-internal data sources to paint a nuanced picture of the current IRR ecosystem. This allows us to assess to what extent a transition from redundant, inconsistent and potentially insecure third-party IRR data towards cryptographically verifiable information is operationally feasible right now. For ROUTE(6) objects, the situation is encouraging, whereas for AS-SETs, it is not yet.

• ROUTE(6) objects: RPKI ROAs provide broad coverage nowadays. This leaves third-party ROUTE(6) objects to contribute little exclusive information. Our measurements show that network operators can deprioritise or even omit third-party ROUTE(6) objects, given authoritative RPKI ROAs and ROUTE(6) objects from authoritative IRR DBs are consulted first. Relying exclusively on this data shows little impact on the studied IXPs' production traffic across different RIR regions, with legacy space covering ROUTE(6) objects playing an even smaller role. At the same time, a notable number of routes vanish from the RSes, indicating that this route information was redundant or orphaned and therefore not crucial for productive Internet routing.
• AS-SET objects: An AS-SET can be viewed as an entry-point that references a collection of other routing objects, such as ROUTE, ROUTE6, or nested AS-SET objects. If this entry-point becomes unavailable, access to all referenced objects is lost, even when those objects are stored in authoritative IRR databases and remain individually accessible. Our analysis identifies and quantifies this observation which is largely driven by network operators who choose to maintain their AS-SET objects only in third-party IRR databases. As a long-term strategy, the networking community should focus more on ASPA adoption. Due to still ongoing standardisation efforts around ASPA, its deployment and coverage is not yet developed beyond experimental setups. Until ASPA reaches broad deployment and sufficient coverage, AS-SET objects in third-party IRR DBs remain an essential component of BGP filter generation. We consider this dependency unproblematic for the moment, as AS-SETs are generally in a much better shape in terms of data quality compared to ROUTE(6) objects - they are maintained more frequently and barely replicated across multiple IRR databases.

Our findings suggest that the best current practice (BCP) proposal for abandoning third-party IRR databases at IXP RSes presented at RIPE 86 and subsequently discussed on the mailing list of the Connect-WG is totally feasible nowadays, but should accept some refinements in the following key areas:

• The list of supported official IRR Databases shall be extended with a virtual DB “RPKI” containing ROA-derived ROUTE(6) objects. Although the RPKI system is a separate DB, the fact that it contains cryptographically verifiable data and is strictly tightened with the KYC (Know Your Customer) and membership policies of the 5 RIRs, it is an excellent choice for accommodating lookups for prefix-origin combinations.
• Until the standardisation of scoped members is completed, the network operator should be given the option to lookup in a third party DB for AS-SETs that have a unique presence over there. However, authoritative sources and/or the RPKI system should still be prioritised for resolution of prefixes.

Notably, all of this is already doable with common tooling like IRRd (see appendix).

Appendix

The full List of investigated scenarios

In addition to the three presented scenarios ("production", "strict", and "loose"), we investigate a number of additional scenarios. We identify RPKI as a viable ROUTE(6) replacement and that AS-SET information based on third-party IRR DBs remains a crucial source for generating BGP route filters.

We also investigate if the order in which information from the different IRR DBs is queried plays a role. Indeed, IRRd and bgpq4 respect the lookup order and accordingly return different information. This table tracks the different parameters we were tweaking to finally identify which data plays a crucial role at IXPs for routing of Internet production traffic.

IRRd configuration for RPSL object class filtering

We provide snippets from the IRRd configuration that instruct IRRd to return information according to the different scenarios. We used static dumps for both RPKI and IRR data since quite some time passed by during exploration of the 10 different scenarios. Transient effects like temporal information drift in external data sources would’ve made a fair comparison difficult.

We first configure the "RPKI" data source that helps to transform ROA data into fake IRR route(6) objects (the ASPA to AS-SET data source is yet to be implemented):

We configure the IRR data sources to select different RPSL objects to be imported or ignored by IRRd. The following snippet exemplarily configures an authoritative IRR DB (ARIN) from which all relevant objects are imported. We further configure a third-party IRR DB (RADB) from which only AS-SET objects are imported. And lastly, we configure another third-part IRR DB (NTTCOM) from which exemplarily no objects are imported at all.

This can easily be configured using the "object_class_filter", a filter that basically matches what’s listed in the configuration file afterwards with what’s found in the IRR DB’s objects’ class string. If a string is configured to, e.g., "dummy_to_import_nothing", a string that matches none of the standardised RPSL object class strings, none of the objects in the IRR DB are matched and consequently nothing is imported. This allows us to virtually remove IRR DBs from scenarios without touching the IXP toolchain, which would otherwise fail.

Note that the word "authoritative" happens to be also used in the context of IRRd but has a different meaning⁸ than what we refer to in this article.

We finally have to select the configured sources:

Using these snippets to configure IRRd allows us to flexibly construct a wide range of different scenarios without having to modify the IXP's RS toolchains. The other widely used tool bgpq4 does not provide a similar support for filtering certain object classes per source.

BGP filter Generation at DE-CIX and AMS-IX

The intuition that both AMS-IX and DE-CIX RSes apply the same practice of handling announced routes quickly proved misleading. Indeed, the validation checks performed by AMS-IX and DE-CIX differ quite a bit and may lead to different outcomes for similar route announcements with similar action communities attached. Overall, there are three criteria taken into account:

1. IRR origin validation: is the originating AS inside the announcing IXP member’s customer cone?
2. RPKI ROA origin validation: is the originating AS authorised to announce this prefix?
3. IRR customer cone validation: is the announced prefix in the recursively resolved prefix allow-list for the announcing IXP member?

The differences are mainly due to the fact that the DE-CIX RS toolchain implicitly supports ASes that maintain AS-SETs but do not maintain ROUTE(6) objects but rely exclusively on RPKI ROAs. For this purpose, DE-CIX first checks whether these ASes are part of the announcing member’s customer cone, captured by the IRR customer cone validation state. If this check passes, DE-CIX accepts ROA covered routes for these ASes without performing additional IRR checks. In contrast, AMS-IX does not support this use case. Instead, it relies solely on ROAs and standard IRR checks, represented by the second and third columns, and requires both checks to pass for route acceptance by default.

Full routing table results

This is the extended table of the routing table loss. Parts of the numbers are not reported for AMS-IX because they would be misleading for technical reasons. AMS-IX tags routes in the main RIB based on their validity (either IRR or RPKI) and filters invalids at the egress direction. Hence, the number of prefixes in the main RIB always remains the same, while the tags change based on the prefix filters. We mark these cases with a dash ("-") in the table.

Notes:

1. AFRINIC, APNIC, ARIN, IDNIC, JPIRR, LACNIC, NIC.BR, NIC.MX, and the RIPE NCC.
2. RADB, NTTCOM, LEVEL3, RIPE-NONAUTH, TC, ALTDB, BELL, REACH, CANARIE, BBOI, PANIX, and NESTEGG (which ceased operations during the time of writing).
3. Only DE-CIX route servers.
4. DE-CIX search order: RADB, LACNIC, IDNIC, AFRINIC, RIPE, RIPE-NONAUTH, BELL, APNIC, NTTCOM, ALTDB, PANIX, LEVEL3, REACH, ARIN, JPIRR, BBOI, TC, CANARIE, RPKI (italic=third-party; bold=auth; RPKI is not present in production scenarios; per customer preferred IRR is prepended to search order).
5. AMS-IX search oder: RIPE, LACNIC, APNIC, ARIN, AFRINIC, IDNIC, JPIRR, RPKI, RADB, LEVEL3, NTTCOM (italic=third-party; bold=auth; RPKI is not present in production scenarios).
6. Dynamic imports via NRTM are used In production - we use static dumps for the lab setup to ensure reproducibility.
7. Legacy space is notoriously hard to identify as it predates the RIR system. The legacy numbers are to be taken with a grain of salt.
8. sources.{name}.authoritative: a boolean for whether [...] changes are allowed to be submitted to this IRRd instance through e.g. email updates. Source