Vesna Manojlovic

RIPE Atlas DataViz Hackathon Results

Vesna Manojlovic

12 min read

In March 2015, a diverse group of contributors got together to come up with creative ways to visualise the health of the Internet using RIPE Atlas open measurements data. It was the first RIPE Atlas hackathon. Impressive results were hacked together by programmers, designers and operators during an intensive weekend of work and fun in Amsterdam. In this article we celebrate hackathon achievements, document and promote hackathon results, create a memento for participants and report in detail for the benefit of the rest of the community.


Table of Contents:

Laptops Laptop art

Introduction

The first RIPE Atlas hackathon was a chance for developers, designers, computer science students and open data enthusiasts to access the large open datasets provided by RIPE Atlas, a global Internet measurement network that measures Internet connectivity. Hackathon participants were challenged to use this open data to develop useful, creative and stunning visualisations for the benefit of the entire Internet community. They could further develop existing ideas or create entirely new visualisations. The aim was to have completed tools or working prototypes for all-new data visualisations. And there were prizes to win, too!

Numbers

  • We received more than 70 applications for participation
  • Present: 24 participants (& one no-show), 6 jury members and 11 support staff (coincidentally: 42!)
    • Six out of 25 participants were female (24%)
    • Three jury members were female (50%!)
  • Ten final projects were presented
  • Four projects were awarded prizes
  • 14 free & open source software projects were published on GitHub
  • 20 litres of tea were consumed
  • Ten large pizzas were eaten for lunch on Saturday 
  • Two kilograms of chocolate Easter Eggs were munched up
  • 12 Club-Mate bottles were emptied

Links

For reference, please see the original announcement  and additional information for attendees .

Before the storm

More than 70 applications were submitted, and the jury had a difficult time choosing among the very good candidates. The jury payed attention to the complementary skill sets, as well as gender diversity. Most participants came from Europe, particularly Amsterdam, since limited grants for the travel allowance were available. Finally, 25 participants were accepted, due to space constraints and the logistics of "herding so many cats".  

The first day of the hackathon took place at the local hackerspace Technologia Incognita . The next two days took place at the RIPE NCC offices.  

Since this was our first hackathon, we had a lot to learn about organising it, and we did not know what to expect. We experimented with ways of announcing it, contacting appropriate audiences, looking for venues, choosing dates, using different communication channels, preparing material for the participants and studying examples of other hackathons. All this advance planning and attention to organisational details paid off, because we were very much ready for it when we finally dove in :-) 

Starting with a bang

At noon on  Friday, we got together in the hackerspace for introductions to one another and to RIPE Atlas, and to form the teams and brainstorm initial ideas.

techinc At the hackerspace

Well, at least that was the original plan...but then there was a massive power outage in parts of Amsterdam and the airport, which affected flights and public transport, so several people were stranded. Conveniently, this gave us real-time power outage data as captured by RIPE Atlas probes,  documented in a RIPE Labs article

Power Outage Tweet

On Saturday and Sunday, participants worked together in teams to further develop their ideas, culminating in short presentations to the jury on Sunday afternoon and a party to announce the winning teams and celebrate everyone's participation. Food and drinks were provided throughout, allowing for more team-building and brainstorming during mealtimes.

Brainstorms Brainstorms


Hackathon results

Below we briefly present the final results of all teams and individuals in random order.

1. Traceroute Consistency Check (by Valentino Di Donato)

Valentino This tool draws a traceroute path and colour codes each node based on its consistency over time. The path is updated through the new RIPE Atlas streaming API. The tool can be found on GitHub: Traceroute-consistency-check.

ScreenShot20150401at11.17.35.png

Traceroute path change visualisation

2. Spotting Anomalies Using R (by Luuk Hendriks, Barbera van Schaik and Larisa Blazic)

By using statistical tools and a data analytics approach, this team visualised the asymmetry in IPv4 vs. IPv6 measurements towards RIPE Atlas anchors, and experimented with the RIPE Atlas streaming API in order to detect interesting events.

ScreenShot20150401at19.57.13.png Lara, Barbera & Luuk presenting their work

3. Spatial Bucketing of RIPE Atlas Probes on Map Projection (by Julian Hammer)

withoutsbucket.png

Julian wanted to solve the following problem: when scheduling measurements from lots of probes, requesting a "random worldwide selection" currently results in a choice that is very biased towards Western/Northern Europe and the USA (see image without Sbucket on the left).

withsbucket.png

Julian's solution makes a more equal selection, by using the grid to divide the globe an d choose probes based on that division (see image on the right after applying Sbucket). It also offers alternatives to  the Mercator projection .

The code can be found on GitHub: RIPE Atlas Sbucket

4. AnyAtlas (by "The Martins", a.k.a. Tom Levine, Caleb James DeLisle, Martin Levy, Shane Kerr and Evan Thompson)

This team used RIPE Atlas to measure latency and the responding node name from an anycast DNS deployment. They then compared it with the speed of light in fibre, in order to determine the difference between measured and theoretical minimum latency. There was also a musical interpretation of data, but not enough time to show it. The code can be found on GitHub: RIPE Atlas anycast  and RIPE Atlas anycast work .

ScreenShot20150401at21.15.52.png ScreenShot20150401at21.15.19.png Any Atlas (please also see the video )

5. RIPE Map (by Michael Kreil and Katja Dittrich)

IMG_8899.JPG This team made two suggestions on how to improve the grouping of RIPE Atlas probes on a map: one with fractals, the other one by placing the probes on the map and adding the probe's connection and disconnection data from the RIPE Atlas streaming API to indicate events. The small dots indicate RIPE Atlas probes. The pie-charts indicate countries.

The code can be found on GitHub: RIPE Map .

6. Disco - Appetite for Disconnection (by Merlijn Tishauser, Nana Klok-Manojlovic, Anna-Kaisa Pietilainen and Igor Rinkovec)

disco_556605939649845855_n.jpg

This team combined multiple data sources: live RIPE Atlas data showing disconnection events with a Twitter feed and weather information, specific to a zoomed-in area of interest. On the left you can see a heat map visualisation of disconnected probes.

The code can be found on GitHub: RIPE hackathon disco .

7. BGP+traceroute (by Jelle Herold)

This work was a combination of RIS & RIPE Atlas data that maps traceroute measurements towards anchors and correlates this data with ASN information. There are similarities with Valentino's work, as these participants collaborated in the first phase of their creations. The code can be found on GitHub: BGP Traceroutes. 

ScreenShot20150401at21.49.25.png Jelle presents the results of the BGP+traceroute project

8. Zeerover DNS (by Matt Calder and Ruwaifa Anwar)

This project answers a need of the research community: to measure "the whole Internet". That means multiple destinations in the same measurement (while RIPE Atlas is more suitable for periodic measurements towards the same destination from multiple vantage points). The hack converts IP addresses of those multiple destinations into the unique hostname, schedules the "normal" measurement towards that hostname, and configures a "zeerover" (Dutch for "pirate") DNS server to point probes to different IP addresses. This was inspired by, and loosely based on,  scapy-dns-ninja . The code can be found on on GitHub: Zeerover.

ScreenShot20150401at21.49.33.png Zeerover DNS presentation

9. BGP Atlas Monitor "BAM!" (by Guillaume Valadon, Francois Contat, Mathias Handsche  and Thomas Holterbach)

The goal of this project was to display - in real time - information that is useful for network operators, such as the visibility of prefixes as seen from RIS, or ping delays as seen from RIPE Atlas probes. The code can be found on GitHub: BAM!

IMG_8932.JPG bam

10. Probe Streams (by Vinayak Hegde)

IMG_8938.JPG

The probe streams visualisation by Vinayak Hegde

This is a beautiful graphical display of the RIPE Atlas probes landscape over time. It shows the distribution of the probes per country or per continent. The code can be found on GitHub: Probe Streams .

In order to present the results of this project, we had to "hack" the recording/presentation setup, because we were missing an HDMI converter. What a wonderful ending to the hackathon!

Prizes

All participants receive RIPE Atlas t-shirts, but when it came to awarding the big prizes, the jury had a hard decision to make. There were three monetary prizes awarded, courtesy of Comcast.

Jury   The jury watches the final presentations

After a short deliberation, the jury agreed on the winners and announced them at the closing party:

  • Third place was shared between two projects working on traceroute visualisations: Valentino & Jelle 
  • Second place was given to sBucket by Julian
  • First place went to Team Disco 

Congratulations to all the winners!

DISCLAIMER: The monetary part of the first place was shared between three team members, because Nana Manojlovic is a family member of one of the RIPE NCC staff members and was therefore not eligible.

The winning team               

Conclusion

The hackathon was a great success. The dynamics among the participants were really positive and everyone had a wonderful time. On top of that, each of the 10 teams produced impressive and highly useful results. Some of these results can quite easily be integrated into RIPE Atlas with little further development.

On behalf of the community, we are grateful to Comcast for sponsoring prizes and travel expenses. We also thank the hackerspace Technologia Incognita for the inspiring location on Friday.

On behalf of the organisers, we are grateful to everyone who took part and contributed, as well as to their family and employers, who let them get away for the weekend to combine work with fun.

Parties Celebrating

Lessons Learned

  1. Hackathons work!
    • Connecting a group of people who have never met before and introducing them to a tool that they might have not seen before, creates amazing results.
  2. Invest less in preparation of logistics, more in educating participants (for instance via a webinar).
  3. The motivation for most people is a sense of accomplishment, having fun, making connections and learning.
  4. Idea for the next hackathon: monetary prizes might not be necessary to accentuate our intention on fostering cooperation rather than competition.
  5. Longer hacking days (and nights) are needed.
  6. Set aside a shorter lead-time between the announcement and the event itself.
  7. Invest energy in reaching out to a wider audience - more designers, students and scientists as well as operators.

Future Plans

  • RIPE NCC: use the feedback, feature requests and the resulting code to improve RIPE Atlas
  • Participants: improve the software produced
  • Promote the winning teams at the RIPE 70 Meeting in Amsterdam in May 2015
  • Publish more in-depth articles about the hackathon contributions on RIPE Labs
  • Have another hackathon! 

Keep in Touch

You may also like

View more

About the author

Vesna Manojlovic is Community Builder at RIPE NCC. Vesna joined the RIPE NCC as a Trainer in 1999. In 2003, she took responsibility for developing and delivering advanced courses, such as RPSL, Routing Registry, DNSSEC and IPv6. In 2008, she lead efforts to establish IPv6 RIPEness as a measure of IPv6 deployment among LIRs. In 2011, she joined the Science Division as Manager of the Measurements Community Building team; in 2015 she moved to Communications Department as Senior Community Builder, with a focus on organising hackathons. Vesna gives presentations at many technical conferences and workshops, and enjoys visiting hackerspaces. Vesna received a Batchelor of Sciences Degree in Computer Science and Informatics from the School of Electrical Engineering, University of Belgrade. She has three children.

Comments 0