Introduction

When we started developing the RIPE Atlas platform in 2010, Internet of Things was looming on the horizon, but at the time we didn't define RIPE Atlas probes as IoT devices. And yet, IoT devices they are. And what's striking is that many of the decisions we made back when we were designing RIPE Atlas turn out to be good examples for the development and roll-out of IoT devices. So, whilst RIPE Atlas is unique in many ways, we believe that certain ethical and design decisions that went into its creation might carry over to the design of IoT devices, providing potential examples for best practice.

Design Principles

We made some initial design decisions that we thought were important for such an infrastructure. Obviously we didn't know exactly how large this network would grow eventually, but we had high hopes that the community and the RIPE NCC membership would like the idea and that RIPE Atlas would produce valuable data and measurement results for Internet operators and researchers.

At the time of designing the system, there were a little over 30,000 ASNs active, and we calculated that if we want to have a device in each (as a naive starting point) with some redundancy, we need to build a system that would scale to over 100,000 devices. We now have 10,000 active RIPE Atlas probes connected worldwide - so there is still some room for growth. However, these probes produce around 450 million measurement results per day, or 5,000 results per second.

Current distribution of RIPE Atlas probes

We also decided that we would only perform active measurements and that we would not observe any traffic or do any network scanning. The initial set of active measurements each RIPE Atlas probe performed were ping, traceroute and DNS lookup. Later TLS (SSL) and NTP measurements were added.

What's more, there was a conscious decision right from the start to build the RIPE Atlas infrastructure by way of distributing hardware devices rather than offering software for people to download (see Daniel Karrenberg's presentation from RIPE 61 for the original reasoning behind this). We may be introducing virtual machines as probes in the future.

It was also important to involve the community from the start. We knew we could not do this without active support from many individuals and volunteers, and in return for that support, we also knew we'd be taking on the responsibility of ensuring that probe hosts would not be put at risk. These volunteers show a great deal of trust in us by hosting our measurement devices in their networks, so it's crucial for us to ensure that the probes behave well; e.g. that they won't disrupt the network of the host or collect data the host would not like to share. (Please also see Ethics of RIPE Atlas Measurements on this topic.)

Security considerations were an essential part of the initial design and were always an important consideration when thinking about new features. In the next section you can read some of the features we incorporated into the design of the system.

RIPE Atlas and Security

From the security point of view, we wanted to make sure that any kind of compromise has limited reach and consequences. For example, even if someone attacks or re-purposes a RIPE Atlas probe, the damage caused should not affect other probes.

Device security

Since RIPE Atlas probes are distributed free of charge, we want to prevent "reuse" or "repurposing" as much as possible. One way of doing this would be to use some kind of Trusted Platform Module (TPM), but in our case that would be prohibitively expensive and would perhaps not be enough even then. Instead, we opted to use cheaper devices, make it non-trivial to do such reuse (there are blog posts out there that describe what to do), and accept the occasional loss. We'll certainly reconsider this position if we observe this kind of event too often.

Version 3 probes (TP-Link based ones) also have a USB stick attached to store firmware and result data. The contents of this stick are encrypted with a per-device key in order to prevent (local) compromises to the code or measurement results.

RIPE Atlas probe initialisation

All probes go through an initialisation procedure at the RIPE NCC before they are distributed to their hosts. During this process we replace the off-the-shelf firmware with the RIPE Atlas firmware. We also generate individual keys and register them. Once finished, we test the probes before they get labeled, packed and shipped.

Potential scenarios

There were a number of scenarios we took into account:

1. A single device is compromised: individuals who have physical access to the probe always have the means of disassembling or otherwise fiddling with it. This is an unavoidable fact of life, therefore our aim is to prevent this from affecting the rest of the network. So, when such events are detected, we separate the device in question from the network. Since each probe has a unique key, we can do this on a per probe granularity.
2. We also need to detect if multiple probes (temporarily) do something other than they are instructed to do by the infrastructure; i.e. unexpected behaviour is observed from multiple probes. In this case we need to identify and fix the cause of the problem, after which the devices need to return to their intended behaviour. 
3. Someone is taking over devices for good; e.g. by disconnecting them from the RIPE Atlas infrastructure. We're aiming at preventing this from happening by having in place all the surrounding security mechanisms.

In order to ensure that the scenarios described above don't cause great damage, we implement the following mechanisms:

• Trust anchors (i.e. the starting points of verification of components from the probes' perspective) are pre-installed on all probes before they are distributed.
• New firmwares are distributed to the probes inside the existing communication infrastructure. 
• All probe firmware updates are signed and each probe has pre-installed public keys to independently verify the firmware signature before upgrading.
• We have mechanisms in place that try to detect unexpected behaviour such as outliers or violations of the internal protocols.
• The probes don't provide direct services to the host or to the world, reducing the network-based attack surface against them considerably.

We also conduct periodic security reviews and improve our infrastructure or code to address potential issues. In addition, we support a "responsible disclosure" approach and will do our best to address any issues brought to our attention by professional researchers or even individual probe hosts.

Conclusions

It's possible to consider RIPE Atlas an IoT network. When designing the network, security was a key element and we were guided by a number of principles that are essential to protect the probe hosts and the wellbeing of the entire infrastructure. We take it to be an encouraging sign that those principles are very much in line with those presented in the IETF's Best Current Practices for Securing Internet of Things (IoT) Devices. Security incidents are inevitable, but it is important to limit the impact as much as possible.

In the next article we will go into more details about the overall architecture of the RIPE Atlas network, the pros and cons of the various probe generations and the communications between the probes and the rest of the system.