We are releasing the code we use to analyse RIPE Atlas measurements and produce summary statistics. This will make processing these measurement results easier for the community. We're also inviting you to participate in improving this further.
RIPE Atlas is producing millions of measurement results every day. Our users can easily download these, but making sense of the data is more difficult: parsing, aggregating and analysing the results of a particular measurement on your own computer takes some effort.
We'd like to help the community with this task by releasing scripts and library code we created recently. As development at the RIPE NCC continues, we'd like to invite all users of RIPE Atlas data to provide feedback on the functionality provided by the code. Users who enhance the code further are encouraged to share the results with other users by contributing back.
The script we hereby release creates "reports" from raw results. With the aid of library code it tries to capture the significant contents of all the data that appear on input and print this in a human readable format. Ultimately, we intend to use the reporting script inside the RIPE Atlas system itself, to provide reports on ongoing (or one-off) measurements to our users as part of the built-in functionality. In addition, since the library code will group the collective knowledge on parsing the data and extracting basic per-probe statistics, we envisage using that in other more specialised tools too.
Usage and examples
The code is available from the RIPE Atlas Community section on github. Follow the instructions in the README file to download and install it on your system.
The entry point for the user is the atlasreport.py script. It takes as input the results from a RIPE Atlas measurement, parses and aggregates the data, collects statistics and at the end outputs a report summarizing the measurement results. Note that the script expects each line of input to hold the complete result of one sample of the measurement. When downloading RIPE Atlas data from the User Defined Measurements section of the website, be sure to select the ' fragments ' format rather than ' JSON '; the latter would return results in one large array where each element holds the result of one sample of the measurement.
The reports generated differ per measurement type, as shown in the following examples.
DNS report
To illustrate the DNS reporting, we use the DNS anycast instance measurement which Stéphane Bortzmeyer described in his recent
RIPE Labs article.
Downloading the results of this p
ublic user defined measurement from atlas.ripe.net (using format option '
fragments
') gives us a file named
RIPE-Atlas-measurement.txt
which we can input to the reporting tool.
$ atlasreport.py /tmp/RIPE-Atlas-measurement.txt
Report for measurement 1008591
------------------------------
Measurement type: dns
Number of destination IP addresses: 1
Destination: 194.0.9.1
Reverse DNS: d.nic.fr.
Timeinterval: 2013-04-27T15:50Z - 2013-04-27T15:50Z
Number of samples: 500
Processed a total of 500 measurements for measurement ID 1008591
Time period is from 1367077838 to 1367077848 or from 2013-04-27T15:50:38 to 2013-04-27T15:50:48
Saw a total of 500 different probes, 485 probes could reach destination
----------------------------------------------------------------
We had 485 probes doing 1 successful measurements
We had 15 probes doing less than 1 successful measurements
----------------------------------------------------------------
count cumulative percentile with an rt value of
1 0 Percentile with rt <= 14.921
12 2.5 Percentile with rt <= 18.239
121 25 Percentile with rt <= 36.779
242 50 Percentile with rt <= 47.437
363 75 Percentile with rt <= 72.514
388 80 Percentile with rt <= 92.558
412 85 Percentile with rt <= 113.428
436 90 Percentile with rt <= 148.497
460 95 Percentile with rt <= 286.299
472 97.5 Percentile with rt <= 359.400
485 100 Percentile with rt <= 878.900
460 2.5-97.5 Percentile with 18.239 < rt < 359.400 with average rt 65.499
----------------------------------------------------------------
Total responses with MNAME nsmaster.nic.fr. 484 96.80% 99.79%
Total count for NSID dns.bru.nic.fr 19 3.80% 3.93%
Total count for NSID dns.fra.nic.fr 173 34.60% 35.74%
Total count for NSID dns.ix1.nic.fr 18 3.60% 3.72%
Total count for NSID dns.lon.nic.fr 47 9.40% 9.71%
Total count for NSID dns.lyn1.nic.fr 25 5.00% 5.17%
Total count for NSID dns.lyn2.nic.fr 29 5.80% 5.99%
Total count for NSID dns.th2.nic.fr 173 34.60% 35.74%
Total count for all NSIDs 484 96.80% 100.00%
Total responses with a ReturnCode==NOERROR but no NSID: 1
From which:
with header fields AA=0 RA=1: 1
----------------------------------------------------------------
Correct (expected) responses (header fields AA=1 RA=0): 484 96.80% 99.79%
Intercepted responses (header fields AA=0 RA=1): 1 0.20% 0.21%
^ ^
Percentage of ALL measurement records ======================================+ |
Percentage of all measurements with 0 < rt <5000 ===================================+
Apart from the results on NSID, the different instances of the anycast server reached by the query, we also report on observed round trip times of the queries and the observed combinations of AA and RA header fields.
Ping report
For the ping report we take as example a measurement to www.bbc.co.uk over IPv4 where the DNS lookup for the IP address was performed locally on each of the participating probes. Collectively, the probes found a total of 19 unique IP addresses which were pinged. Although the analysis code is all set to collect and report the statistics per unique destination IP, this would result in too much output for human consumption. We therefore aggregate the statistics and report on the combined set of destination IPs.
Report for measurement 1008887
------------------------------
Measurement type: ping
Destination hostname: www.bbc.co.uk
Number of destination IP addresses: 19
Total samples: 24284
Destination: All IPs combined
Timeinterval: 2013-05-07T01:29Z - 2013-05-07T03:30Z
Packets sent: 61041
Packets received: 60602
Overall loss rate: 0.72%
Total probes measuring: 1011
Probes with 100% errors: 158
Total errors on probes: 175
Most common error:"dns resolution failed: non-recoverable failure in name resolution (168x)"
Probes with no packets lost: 810
Probes with 0%-5% loss: 28
Probes with 5%-20% loss: 7
Probes with 20%-40% loss: 4
Probes with 40%-60% loss: 1
Probes with 60%-80% loss: 0
Probes with 80%-100% loss: 0
Probes with 100% loss: 3
Probes not sending any packets: 0
RTT distributions:
-----------------
2.5 percentile ("Minimum")
lowest 2.5 percentile RTT in all probes: 1.29ms
2.5% of probes had 2.5 percentile <= 5.23ms
25% of probes had 2.5 percentile <= 24.43ms
50% of probes had 2.5 percentile <= 37.64ms
75% of probes had 2.5 percentile <= 56.37ms
97.5% of probes had 2.5 percentile <= 314.53ms
highest 2.5 percentile in all probes 404.98ms
Median
lowest median RTT in all probes: 2.25ms
2.5% of probes had median RTT <= 6.51ms
25% of probes had median RTT <= 26.03ms
50% of probes had median RTT <= 40.15ms
75% of probes had median RTT <= 60.92ms
97.5% of probes had median RTT <= 323.25ms
highest median RTT in all probes 428.17ms
97.5 percentile ("Maximum")
lowest 97.5 percentile RTT in all probes: 3.67ms
2.5% of probes had 97.5 percentile <= 12.20ms
25% of probes had 97.5 percentile <= 37.56ms
50% of probes had 97.5 percentile <= 51.88ms
75% of probes had 97.5 percentile <= 89.14ms
97.5% of probes had 97.5 percentile <= 426.32ms
highest 97.5 percentile in all probes 1757.13ms
Instead of the better known minimum, average and maximum values, the ping report uses percentiles to characterize the distribution of observed round trip times. The reason for this is that with a large number of measurements, incidental outliers skew the average and don't provide a good indication of typically observed round trip times.
Traceroute report
The traceroute reports are in a less advanced state. Currently we collect statistics on average hop count, traces reaching and not reaching the target IP address and the number of unique traceroute vectors observed. Reply packets which are received out of order (too late or too early compared to the outgoing packet's TTL) make this last step of data parsing a bit more complex compared to the looking at output from standard command line traceroute tools. The example below lists results for one of the ongoing measurements to 128.0.168.1
Report for measurement 1004651
------------------------------
Measurement type: traceroute
Number of destination IP addresses: 1
Destination: 128.0.168.1
Timeinterval: 2013-05-02T00:09Z - 2013-05-02T23:59Z
Number of traces: 69485
Number of unique routing vectors: 3595
Traces reaching target IP address: 66905
Average hop count: 14.02
Traces not ending at target IP 2580
Future directions
At the RIPE NCC we will continue to further develop this reporting functionality for RIPE Atlas data. A first, mostly technical step, is to migrate the DNS reporting code which is mostly done in seperate Perl code, to the general Python based framework. This will improve performance for large sets of DNS data. Next we will look at expanding the functionality. For that we are also interested in your feedback. Do you find the reports useful? Do you have suggestions for additional parameters to be reported? Your input is welcome at atlas [at] ripe [dot] net
Comments 3
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.
Anonymous •
These tools require measurement results in 'Fragments' format - is there a way to request that format be returned when using the API to retrieve results?
Hide replies
Bert Wijnen •
If you look at this measurement: https://atlas.ripe.net/atlas/udm.html You will see a "DOWNLOAD" button (in blue on my screen). Just above that button, there is a "Format" pulldown menu. From that you can choose "fragments". Hope this answers your question Bert
Antony Antony •
append format=txt here is an example https://udm01.atlas.ripe.net/api/v1/measurement/1002368/result/?format=txt is this what you are looking for? full json is https://udm01.atlas.ripe.net/api/v1/measurement/1002368/result/