The Shape of a BGP Update

Introduction

What happens when an IP address prefix gets announced or withdrawn. How does this information propagate through the Internet? And how does it affect the amount of BGP traffic across the Internet when a single prefix is freshly announced or withdrawn from the global routing table? This is not going to be surprising for any BGP expert. The intent of this story is to provide some general intuition about the current state of BGP routing in practice.

Methodology

The prefix 84.205.67.0/24 was not visible in the routing table on 2 February 2011 at 08:00 UTC. At that point it was announced in BGP to all 61 of our local directly connected peers via our Routing Information Service (RIS) Route Collector present at the Amsterdam Internet Exchange AMS-IX.

Two hours later, at 10:00 UTC, the same prefix was actively withdrawn via the same set of peers at AMS-IX.

We measured the BGP update traffic that was generated by these two events from all of the RIS collectors by observing the incoming updates via 90 full table peers across the globe.

The prefix 84.205.67.0/24 is part of the set of RIS beacons that have periodical and pre-determined patterns of announcements and withdrawals. Find more at: http://www.ripe.net/data-tools/stats/ris/current-ris-routing-beacons

Results

For the prefix announcement occurring at 08:00 UTC, Figure 1 below represents the update activity (BGP updates per second) observed over time, as well as the visibility progress of the prefix (the number of peers seeing it).

The 0 seconds at the X axis marks the moment the prefix got announced.

The visibility is calculated by dividing the number of peers already seeing the prefix by all the peers in the set (90).

For the BGP updates we differentiate between (1) the first update ever sent by a peer regarding this event and (2) further updates from a peer that has already seen the event. This allows us to distinguish between updates that represent the first time they are seen by a peer, and updates that most likely represent route selection (AS path convergence, etc).

Figure 1: Prefix Announcement - BGP Updates and Propagation of Visibility

We observe that all of the generated BGP traffic for this announcement occurs in slightly over a minute after the initial prefix announcement, with 90% of the volume occurring in the first 30 seconds. Full visibility (100%) by all peers has been reached after only 38 seconds. A visibility of 40% is accomplished in just a few seconds.

Nearly half of the updates (46%) are further updates from peers, representing the overhead of the BGP convergence.

Figure 2 below shows the BGP update volume for the prefix withdrawal that occurred at 10:00 UTC. Here we distinguish between the update messages per type: announcement or withdrawal. For visual clarity, withdrawals are represented on the negative side of the Y axis. As before, both types are separated between first and further updates as seen by a peer.

Figure 2: Prefix Withdrawal - BGP Updates and Propagation of Visibility

It is interesting to note that the generated volume of traffic for the prefix withdrawal is much higher than for the announcement of the same prefix. Almost 4 times more BGP update messages (503 versus 139) are registered. Also, the activity lasts for nearly 3 minutes as compared to just over a minute for the prefix announcement.

79% of all updates are further updates from a peer. This indicates that on average a route withdrawal will generate about 5 BGP messages sent by a peer, as compared to only 1 or 2 messages for a route announcement.

Also, only 25% of the messages are withdrawal messages, indicating that a peer sends an average of 3 route re-announcements before withdrawing it. Furthermore, most of the withdrawal messages are concentrated at later than 1min after the the initial prefix withdrawal (when visibility is still 80%), with an accented burst at around 90 seconds when visibility drops from 60% to 20%, indicating the moment when the withdrawal of the prefix from the global routing tables starts converging more abruptly.

Note that only 2.6% of the update messages will be concerning a peer that receives the withdrawal for the first time and immediately passes it on to its peers.

The distribution of the BGP traffic per update message type for the whole period is shown below in Figure 3.

Figure 3: Prefix Withdrawal - Update Type Distribution

This phenomenon of the much larger volume of updates, with multiple update messages from each peer can be explained by the way routers interpret a route withdrawal: A router has no way of knowing if a route withdrawal is happening because it was withdrawn at the origin, or because its peer simply dropped the route due to connectivity loss or other malfunction. Therefore, following the resilient nature of the Internet, it will look in its routing table for an alternative path towards the withdrawn prefix and try to save the connection to the prefix in that way. If it succeeds, it will pass the update to its peers, therefore prolonging the life of the route. Only when it runs out of alternative paths, will it finally remove the prefix from its routing table and send the withdrawal message to its peers.

Conclusion

To conclude, we observe that BGP route updates tend to converge globally in just a few minutes. The propagation of newly announced prefixes happens almost instantaneously, reaching 50% visibility in just under 10 seconds, revealing a highly responsive global system. Prefix withdrawals take longer to converge and generate nearly 4 times more BGP traffic, with the visibility dropping below 10% only after approximately 2 minutes.

Keeping all this in mind, we can conclude that it is often easier and quicker to turn on a prefix, than to switch it off.