RIPE NCC Technical Services 2017 - Part Three: Focus on Tools and Research

RIPEstat

RIPEstat is a web-based service that provides everything you ever wanted to know about IP address space, Autonomous System Numbers (ASNs), and related information for hostnames and countries in one place.

RIPEstat is certainly a success story: on a daily basis we see around 30 million queries and, together with the RIPE Database, this is one of our most used services.

Figure 1: RIPEstat usage over time

RIPEstat is increasingly popular with network operators who use it to debug network issues. At the MENOG 17 meeting in Oman earlier this month, I observed a network operator using RIPEstat extensively to investigate and solve an issue in their rather large network, which happened to occur during the conference. I also hear that many network operators have now integrated RIPEstat in their network management systems.

Improvements in the past two years

While we are committed to constantly improving RIPEstat, we have reduced active development on RIPEstat since last year for two main reasons.

Firstly, we focused more on improving the infrastructure supporting RIPEstat. RIPEstat depends on many internal and external data sources and there are a lot of moving parts, as well as a lot of logic, behind this one-stop shop for Internet resources.

If we want to be able to scale the service and sustainably add new features, we need a robust platform. We have made some major changes in the code and the test environment. We also revamped the deployment system behind RIPEstat in the past two years. Some of the major data provisioning systems behind RIPEstat are being reworked completely (for example RIS, see more below). The system supports scaling up both in terms of capacity and the addition of new features and services.

Secondly, we received requests for more aggregated data. We also want to make sure the service provides useful data to target groups other than network operators. Today RIPEstat can provide aggregated information on countries (in addition to IP addresses, address prefixes and AS numbers you can enter a country code and get aggregated data on that country).

Figure 2: RIPEstat country widget showing resources in Iran over time

In addition to this, we want to provide aggregated data for regions such as ENOG, MENOG, SEE and the RIR service regions, as well as specially defined regions like those devised by organisations like OECD and the World Bank, for instance. By providing these aggregates and adding functionality that allows comparison between datasets and countries or regions, we can provide interested parties with more insight. These groups, include network researchers, regulators and other parties interested in entities larger than a single network. That brought us to a new development, “country reports”, which is a new service built on top of the RIPEstat platform.

Country Reports

This service is a logical extension to RIPEstat. Its goal is to present easily comparable aggregated data on a country or  regional level to end users. A first internal release is already being used to compile country and regional data, which is presented at our regional meetings and member lunches.

We are aiming to populate the system with more data, including some publicly available information, such as the population of a country or Internet penetration numbers.

Being able to compare regions and countries will give a unique view that will be of use in relation for many kinds of research and public policy making scenarios where users can explore the data and/or validate their findings using readily accessible raw data or fast and interactive visualisations.

We believe such a system will help inform public policy discussions and will provide community members with a unique tool that can help them make decisions based on reliable data.

We are putting extra effort on the presentation of and the interface to the data, as a main value of the tool comes from usability and repeatability of work.

OpenIPMap

We are regularly contacted about geolocation issues via RIPEstat and other channels. This has been discussed extensively in our community and the final result, ratified by our board, is documented on the RIPE NCC website.

At the same time, we came to the conclusion that there is a big need for infrastructure geolocation information. This refers to the location of IXPs, routers and other network equipment, which can be used to visualise traceroute data or other related network information, and is very different from end-user geolocation services.

Compiling geolocation data for network infrastructure is tricky. Many of these devices have airport code location names or similar codes assigned in their reverse DNS names, which indicate their rough location. That said, there is no unified format or validation, and to the best of my knowledge, none of the commercial geolocation data providers have such a data set available. Therefore we produced a prototype based on work done by our colleague Emile Aben. It was presented to our community on multiple occasions. Based on the positive feedback we received, we will implement this prototype as a production service. We expect to make the first release available soon, and plan to continue working on useful, requested features throughout the year.

TraceMON

TraceMON is the latest addition to our set of RIPE Atlas data analysis and visualisation tools, in addition to DomainMON and LatencyMON. It’s a web application for monitoring and investigating the reachability and performance of a target in a network. It uses traceroutes for inferring network topology, showing its evolution over time. 

TraceMON has been designed based on interaction with network operators in order to support them with their day to day operations. By aggregating many data sources it provides additional visual information on top of the traceroutes, such as Autonomous System Numbers, IXP detection, geolocation, and reverse DNS lookups for the various IP addresses involved. It also provides a one-click access to resource holder contacts, whois, BGP visibility, IXP detection, and more.

This mode of representing traceroute data, combined with larger numbers of traceroutes stored in RIPE Atlas, provides a unique view on actual network paths and can help make understanding and diagnosing routing issues much easier and faster.

The TraceMON code is available on the RIPE NCC section of GitHub.

Figure 3: TraceMON visualisation

Routing Information Service (RIS)

Even before joining the RIPE NCC, I have always been amazed that RIPE NCC has been providing 5-minute BGP dumps of Internet routing updates since September 1999 from a multitude of locations worldwide with data collected from hundreds of BGP peers. Currently we have 18 RIS collector locations and we expect to add a few more in the coming months.

We are in the process of upgrading the whole RIS infrastructure and changing the RIS collector architecture from a batch-oriented process to a message-based system. By using messaging technologies, each collector pushes a stream of updates to our central repository. From there, we can redistribute the stream (see demo) or add services on top of them (comparable to the RIPE Atlas data streaming service). This will also enable us to develop completely new services, such as real time routing alerts and a real time BGP visibility visualisation with BGPlay.

This new design is already in place for four RIS route collectors (RRC18, 19, 20 and 21). We know that the traditional MRT dump files with five minute (update) and eight hour (full) MRT dumps are used by many operators and researchers, so we have no plans to abandon this data format in the immediate future. Therefore we also generate such MRT dumps from the data streams coming from the new RRCs. So from the external point of view, these are just additional RRCs. This allows us to rebuild our route collectors into the new message-based model, without impacting existing external users of our data and, at the same time, migrate our internal systems to the new data streams.We believe this stream format will be a game changer and, as demonstrated in the recent BGP Hackathons, including the one run by CAIDA, free provisioning of such a global live stream of BGP data at RIS scale will enable many new use cases and services.

Another benefit of the above-described re-architecture work will be that it will allow us to further expand the number of RIS collectors, while also expanding the number of peers again on those collectors that are currently at capacity.

Research collaboration

The RIPE NCC has always been active in the Internet research community. Not only do we maintain a wealth of data that can be useful for researchers, but we can also play a role in connecting people and in helping to analyse the data.

We always try to strike a balance between keeping our research work empirical and factual, but also useful for network operators. Hence, an important goal for us is to help forge a connection between the research community and the operators community, thus providing results for the community at large so as to support the overall good of the Internet.

We always had strong ties with the academic world - after all our roots are in academia - and have always supported academic work through a variety of avenues. Most recently, these efforts are being pursued in the context of the RIPE Academic Cooperation Initiative RACI and our collaboration with universities and various research projects. These efforts are visible in a number of research papers and RIPE Labs articles focusing on a wide range of topics such as RIPE Atlas, RIS, K-root and RIPEstat. Our in-house research centered around subjects that are of operational importance for service and content providers, and our community in general.

We are planning to continue on this track and keep producing useful reports and prototypes for our community. We have increased our efforts in documenting and formalising the way we organise research within the RIPE NCC. Now, our research team has a clearly allocated budget to coordinate with external research activities. Ultimately, it is our intention to make it easier for Internet researchers to collaborate with the RIPE NCC.

Conclusion

This concludes our three-part look at the RIPE NCC’s technical services for 2017. We hope that the information we’ve provided about RIPE Atlas, the RIPE NCC’s DNS related services, and the various other technical services and tools offered by the RIPE NCC has helped illuminate our readers with regards the work we’ve been doing over the past two years, its significance, and its potential benefits.

As well as working to improve and modify our existing services, such as RIPEstat and RIS, we’ve also been busy developing new tools that we hope will provide further support to our users, such as TraceMON. Plans are also underway for enhancing our range of technical services further still, and we strongly recommend that all those interested keep an eye on RIPE Labs to get all the latest news about OpenIPMap, Country Reports, as well as any other new developments that might appear on the horizon as we move forward. In parallel to all this, we are very glad to have been able to take significant steps forward in helping foster better collaboration with regards to Internet related research.  

If you have any comments, queries, or any other form of feedback about this or either of the other articles in this particular series, please feel free to let us know in the comments section. Please also be free to get involved in discussion on these topics on the appropriate mailing lists:

• RIPE Atlas Mailing List, for discussions with active users and RIPE Atlas developers: ripe-atlas [at] ripe [dot] net
• Measurements, Analysis and Tools Working Group Mailing List, for discussions on data, tools and analysis relating to the Internet and its infrastructure: mat-wg [at] ripe [dot] net
• Routing Working Group Mailing List, for discussion on routing architecture for the European Internet: routing-wg [at] ripe [dot] net
• DNS Working Group Mailing List, for discussion on any DNS related questions: dns-wg [at] ripe [dot] net