Is It Really Worth Peering at IXPs? A Comparative Study

Introduction and Motivation

Internet eXchange Points (IXPs) are infrastructures used by Internet Service Providers (ISPs) to exchange traffic between their Autonomous Systems (ASes). An IXP allows ISPs to interconnect their ASes directly, i.e. to establish peerings between them, rather than through third-party networks (upstreams). IXPs play a crucial role in the development of the Internet, encouraging ISPs to create a dense network of interconnections at low cost. Some of them have a throughput of many Tbit/sec and are some of the most important building blocks of today’s Internet.

It is widely known that peering at IXPs is very useful in terms of performance. For example, shorter paths reasonably improve the round-trip delay and the locality of traffic. However, a thorough verification of these properties is missing in the scientific literature of the Internet. We decided to conduct some experiments in order to demonstrate that these benefits are real.

Our study stems from recent news on major ISPs cancelling their peerings at IXPs (de-peering). On several occasions they justified such decisions in terms of more efficient handling of IP traffic and improvement of the quality of service (QoS). As a second topic of interest, we also studied whether IXPs are effective in preserving traffic locality, by checking which countries are traversed to reach frequently visited Italian destinations from Italian sources. This is a security issue of concern for many nations. In particular, given the recent espionage cases of network traffic, a nation might be interested to see if its own citizens, to reach Internet services that are considered critical, have to cross ISPs of different countries or even of different continents.

Our study is focused on Italy, meaning that both the source ASes and the target ASes of our measurements are located in Italy. We concentrated our attention on the two most popular Italian IXPs: The Milan Internet Exchange MIX and the Nautilus Mediterranean Exchange Point NaMeX . Moreover, we collaborated with three medium-size ISPs, in order to actively control their BGP announcements and force the traffic to take specific routes for useful comparison. Namely, we performed experiments in which network paths between two ASes either traverse IXPs or rely only on upstream providers. Such experiments help us determine to which degree IXPs are actually beneficial for involved peers. We performed measurements regarding the following network metrics: round-trip time (RTT), hop count, packet-loss and jitter. For our measuring system we used RIPE Atlas, choosing only probes located in Italy, which currently number about 150.

We performed two types of experiments that we've respectively named Critical Internet Services (CIS) and Selective BGP Announcements (SBA).

In the following paragraphs we will explain the experiments in detail and show the results for each of them.

Critical Internet Services (CIS) - Settings

In our first experiment we engineered network tests aimed at producing a plausible snapshot of the QoS associated to crucial websites commonly accessed by Italian users, with the goal of classifying the results according to the usage/non-usage of IXP peerings. As a first step, we selected two sets of Internet websites that can be considered crucial for Italian users. The first contains services related to critical infrastructures and the second contains popular sites. The first set, called CRITICAL, was selected according to a taxonomy found in literature and contains 50 websites belonging to the following categories: online banking, insurance companies, public administrations, energy companies, legal courts, travel companies, health portals and webmail providers. The second list, called VISITED, contains the 100 Italian websites most visited by Italian users and hosted in Italy, according to the ranking published by Alexa. We scheduled ping and traceroute measurements from all probes locatd in Italy, targeted at each domain.              

Critical Internet Services (CIS) - Results

The following graphs show the RTT for the dataset of CRITICAL sites and the hop count for the dataset of the most VISITED sites. These graphs are Cumulative Distribution Functions (CDF) and they show, for each value of RTT/hop-count, the percentage of probes that have, at most, that value.

Each graph contains two plots: one for all the traceroutes that use paths traversing either MIX or NaMeX (blue) and the other for traceroutes that do not see any of the two IXPs in the traceroute paths (red). The plots show that, in general, probes traversing IXPs have better indicators. For example, about 70% of probes that choose IXPs have an average RTT of 30ms or less, while only 20% of those that do not traverse IXPs have the same performance.

Figure 1: RTT for critical sites (left) and hop count for datasets of most visited sites (right)

Finally, the data collected in the CIS experiment can be analysed in terms of security issues associated with the traffic targeted at crucial Web services. Indeed, recent cases of network traffic espionage solicited governments to check if the traffic generated by their citizens and targeted to critical Web services remains inside the country or not. It is perfectly clear that this type of locality does not give security guarantees on such traffic, but it is equally clear that, nowadays, governments are very sensitive to this issue. Therefore, we measured how frequently using an IXP to reach an Italian target from an Italian source permits the avoidance of transit through a foreign upstream. That is, we quantitatively checked whether MIX and NaMeX are effective in keeping the local traffic local.

In order to determine the country of an AS X, we queried the RIPEstat service asking for the prefixes announced by X, along with the country of each of these prefixes. If the returned prefixes were all associated to the same country, then we assumed that it was the country of X. Otherwise, to determine its country, we manually retrieved metadata on X, looking, for instance, at the website of the corresponding provider. The statistics were computed as follows: First, the ASes traversed by paths in the CIS experiment were labelled as non-Italian and/or non-European if the nation was detected by the aforementioned heuristic. Second, the source-target pairs of the experiment were split into two groups, based on the presence of IXPs in their paths. As an example, abscissa (AS3356) shows that 80% of the source-target pairs not passing through an IXP pass through AS3356 (that is not Italian) while not one of the source-target pairs passing through an IXP traverses it. In most cases, a foreign AS is traversed only when an upstream is used, or in great prevalence when an upstream is used, which confirms that IXPs are effective in preserving the locality of traffic.

Figure 2: % of paths traversing foreign (non Italian LEFT and non European RIGHT) ASes when an IXP is used or not

However, such results must be considered very carefully, because of the following three reasons:

1. The two CDFs refer to disjointed sets of probes, since each probe either passes through an IXP or not. In principle, it is possible that a probe currently passing through an IXP could show better performance when forced, in some way, to pass through an upstream (or vice-versa).
2. RTTs and hop counts refer to the last hop that replied to our measurements, which quite often is not the actual target (40 cases out of 53 for dataset CRITICAL, and 59 cases out of 94 for dataset VISITED).
3. A router in an IXP might answer to a traceroute using an interface that is not on the peering LAN, leading to a misclassification of the corresponding traceroute. Also, the data on hop count could be affected by the presence of tunnels (e.g. transparent MPLS tunnels). On the other hand, this might happen both to packets that traverse IXPs and to packets that do not.

However, these graphs help us draw a first positive impression on the impact of IXPs. Observe that the SBA experiment (see below) does not suffer from the problems pointed out above.

Selective BGP Announcements (SBA) - Settings 

In our second experiment, we focused on analysing routing alternatives involving either IXPs or upstreams used by Italian ISPs to reach Internet services, in order to directly compare their performance. To do so, we partnered with Mc-link, Seeweb and Unidata, three medium-sized Italian ISPs. In preparation for our experiment, we asked each partner ISP to reserve one IP subnet and one server in its data centre. Each server was assigned an IP address falling within the reserved IP subnet. Also, it was configured to correctly handle and answer ICMP echo requests sent with pings and traceroutes. For each ISP, we executed, at different times, the steps described below. We asked the ISP to announce a predetermined sequence of 5 BGP updates involving its reserved IP subnet. Each update had a lifetime of 4 hours, for a total of 20 non-consecutive hours per test. The list of BGP updates was crafted to selectively distribute routes to different subsets of the available peers, adhering to the following scheme:

1. “UPSTREAM”: announce only to transit ASes.
2. “IXPS”: announce only to MIX and NaMeX peers.
3. “MIX”: announce only to MIX peers.
4. “NAMEX”: announce only to NaMeX peers.
5. “ALL”: announce to all peers.

Figure 3: Selective BGP announcements phases

During each interval, we ran traceroutes and pings from all Italian probes to the IP address of the server. For each interval, we started one hour before the BGP announcement and ended one hour after the end of the interval. We performed one ping per minute and one traceroute every 10 minutes. The measurements ran before the first BGP announcement was performed, in order to assess the reachability and the performance stability of each target. The great majority of the intervals (not all of them, due to the technical constraints of one provider) were located during daytime hours, to avoid any time-of-day effect on the measurements. Finally, at the end of each period, we filtered out all ping and traceroute measurements that took place near the five BGP announcements. More precisely, given the exact timestamp logs of BGP announcements, we identified four time intervals spanning one hour and centred at each of the timestamps, and removed each ping and traceroute falling within any of the intervals.

Selective BGP Announcements (SBA) - Results

For experiment SBA, we started by selecting the subset of probes that were able to reach the target during both the “UPSTREAM” interval and during at least one of the “MIX” and the “NAMEX” intervals. These probes allowed us to measure the differences between the connectivity offered by IXPs and that offered by upstream providers.

The following graphs show the round-trip delays measured during the experiment with Seeweb, the hop count measured during the experiment with MC-link and the jitter measured during the experiment with Unidata. As an example, the first graph in Figure 4 contains four plots that show the distribution of the average RTT for the two Seeweb targets, in both cases when the target was reached through an IXP or through an upstream. Consider that, in this and in all the following figures, the plots involving paths through IXPs show the “best” option, i.e. for each performance indicator and for each probe that reached the target during both the “MIX” and “NAMEX” intervals, the smallest value between the two options is shown. Observe that about 50% of multi-choice probes reach both targets with an average RTT of at most 15ms when using IXP connectivity, compared to only about 30% of them in the case of upstream connectivity. The measurements towards the Milan target and the Frosinone target exhibit comparable performance. Additionally, the other graphs suggest that paths traversing IXPs always show performances that are better or at least equal to those traversing upstreams, for each provider among Seeweb, MC-link, and Unidata, and for each considered performance indicator among average RTT, average hop count, jitter and packet loss. The latter is the only metric for which IXPs and upstreams exhibit about the same performance, with negligible differences, and its graphs are omitted for brevity.

Figure 4: RTT delays measured during the experiment with Seeweb (top left), the hop count measured during the experiment with MC-link (top right) and the jitter measured during the experiment with Unidata (bottom left)

During the experiments, we noticed that paths through the upstreams exhibited quite different values of performance indicators, depending on the specific traversed upstream. So, looking at the traceroutes gathered during the “UPSTREAM” interval, we found the upstreams of each of our providers and deepened the analysis for all of them. Here is the list of upstreams that we found:

• MC-link (AS3257, AS12874, AS174, AS3356, AS35612, AS57329)
• Seeweb (AS3257, AS174, AS3549, AS3356)
• Unidata (AS3257, AS12874, AS16004, AS24796, AS20836)

The following graphs show the main outcome of the analysis. As an example, the first figure contains the same data from the graphs related to the experiment performed with Seeweb restricted to the Milan target and to the probes that reached the target through AS174 (COGENT). The IXP plot is restricted to those probes too. Observe that the plot has exactly the same trend as the graph with all the probes. It follows that the paths traversing AS147 are the principal factor that makes the average upstream performance lower than the IXP performance. Similar arguments apply to the other two graphs. In the first one, AS3356 (LEVEL3) is shown as the upstream with the largest difference from the IXP alternative among all the upstreams of MC-link. In the second one, AS3257 (TINET) exhibits a behaviour similar to the plot with all the probes. We argue that the performance obtained from upstream connectivity heavily depends on the choice of a specific upstream.

Figure 5: Main outcome of the analysis described above

As a next step, we considered the classification of ASes using a technique known in literature (A. Dhamdhere and C. Dovrolis, “Ten years in the evolution of the internet ecosystem", in Proceedings of IMC 2008). Our goal was to identify different trends in the improvement of performance indicators based on the type of AS that hosts a probe. ASes were assigned to 3 classes:

1) Customers represent various organisations, universities and companies at the network edge that are mostly users of the network.

2) Transit ISPs provide Internet access and transit services. Transit ISPs aim to maximise their customer base in their geographical area and to reduce their upstream transit costs through selective peering with ISPs.

3) Content/Access/Hosting Providers (CAH) are ISPs that offer Internet access and/or server hosting. Their access customers can be residential users or enterprises with no AS numbers, while their server-hosting customers are content/service providers with no AS numbers.

The next two graphs are respectively based on average RTT and average hop count for the Seeweb Milan target during the SBA experiment. Each graph contains 3 plots, each corresponding to one AS class. A plot is a CDF that shows the distribution of performance gaps derived from the comparison of performance indicators between paths traversing IXPs and paths avoiding them, as seen by each probe. Customer is the class with smaller networks and home users. We can see that they benefit the most from the use of IXPs, therefore IXPs have an important impact on home users.

Figure 6: Average RTT (left) and average hop count (right) for the Seeweb Milan target during the SBA experiment

Conclusions

We investigated the role played by IXPs in the Italian Internet ecosystem. Our experiments put in evidence that peering arrangements that make use of IXPs have a positive effect on key performance indicators such as latency, hop count, packet loss and jitter. They also reduce the number of foreign ISPs traversed by the traffic between users located in Italy and critical Internet services like Banks and Public Administrations. We conclude that it is at least questionable if ISPs motivate their de-peering from an IXP with performance improvements.