RIPE Atlas Code for Analysis and Statistics Reporting

Rene Wilhelm — May 13, 2013 03:40 PM
Contributors: Bert Wijnen
Filed under:
We are releasing the code we use to analyse RIPE Atlas measurements and produce summary statistics. This will make processing these measurement results easier for the community. We're also inviting you to participate in improving this further.

RIPE Atlas is producing millions of measurement results every day. Our users can easily download these, but making sense of the data is more difficult: parsing, aggregating and analysing the results of a particular measurement on your own computer takes some effort.

We'd like to help the community with this task by releasing scripts and library code we created recently. As development at the RIPE NCC continues, we'd like to invite all users of RIPE Atlas data to provide feedback on the functionality provided by the code. Users who enhance the code further are encouraged to share the results with other users by contributing back.

The script we hereby release creates "reports" from raw results. With the aid of library code it tries to capture the significant contents of all the data that appear on input and print this in a human readable format. Ultimately, we intend to use the reporting script inside the RIPE Atlas system itself, to provide reports on ongoing (or one-off) measurements to our users as part of the built-in functionality. In addition, since the library code will group the collective knowledge on parsing the data and extracting basic per-probe statistics, we envisage using that in other more specialised tools too.

 

Usage and examples

The code is available from the RIPE Atlas Community section on github. Follow the instructions in the README file to download and install it on your system.

The entry point for the user is the atlasreport.py script. It takes as input the results from a RIPE Atlas measurement, parses and aggregates the data, collects statistics and at the end outputs a report summarizing the measurement results. Note that the script expects each line of input to hold the complete result of one sample of the measurement. When downloading RIPE Atlas data from the User Defined Measurements section of the website, be sure to select the 'fragments' format rather than 'JSON'; the latter would return results in one large array where each element holds the result of one sample of the measurement.

The reports generated differ per measurement type, as shown in the following examples.

DNS report

To illustrate the DNS reporting, we use the DNS anycast instance measurement which Stéphane Bortzmeyer described in his recent RIPE Labs article. Downloading the results of this public user defined measurement from atlas.ripe.net (using format option 'fragments') gives us a file named RIPE-Atlas-measurement.txt which we can input to the reporting tool.

$ atlasreport.py /tmp/RIPE-Atlas-measurement.txt 


Report for measurement 1008591
------------------------------

Measurement type: dns
Number of destination IP addresses:  1
Destination: 194.0.9.1
Reverse DNS: d.nic.fr.
Timeinterval: 2013-04-27T15:50Z  -  2013-04-27T15:50Z
Number of samples: 500

Processed a total of 500 measurements for measurement ID 1008591
Time period is from 1367077838 to 1367077848 or from 2013-04-27T15:50:38 to 2013-04-27T15:50:48
Saw a total of 500 different probes, 485 probes could reach destination
----------------------------------------------------------------
We had    485 probes doing 1 successful measurements
We had     15 probes doing less than 1 successful measurements
----------------------------------------------------------------
   count  cumulative percentile with       an rt value of
       1       0   Percentile with            rt <=   14.921
      12       2.5 Percentile with            rt <=   18.239
     121      25   Percentile with            rt <=   36.779
     242      50   Percentile with            rt <=   47.437
     363      75   Percentile with            rt <=   72.514
     388      80   Percentile with            rt <=   92.558
     412      85   Percentile with            rt <=  113.428
     436      90   Percentile with            rt <=  148.497
     460      95   Percentile with            rt <=  286.299
     472      97.5 Percentile with            rt <=  359.400
     485     100   Percentile with            rt <=  878.900
     460  2.5-97.5 Percentile with   18.239 < rt <   359.400    with average rt   65.499
----------------------------------------------------------------
Total responses with MNAME               nsmaster.nic.fr.            484  96.80%  99.79%
Total count for NSID                       dns.bru.nic.fr             19   3.80%   3.93%
Total count for NSID                       dns.fra.nic.fr            173  34.60%  35.74%
Total count for NSID                       dns.ix1.nic.fr             18   3.60%   3.72%
Total count for NSID                       dns.lon.nic.fr             47   9.40%   9.71%
Total count for NSID                      dns.lyn1.nic.fr             25   5.00%   5.17%
Total count for NSID                      dns.lyn2.nic.fr             29   5.80%   5.99%
Total count for NSID                       dns.th2.nic.fr            173  34.60%  35.74%
Total count for all NSIDs                                            484  96.80% 100.00%
Total responses with a ReturnCode==NOERROR but no NSID: 1
From which:
  with header fields AA=0 RA=1:      1
----------------------------------------------------------------
Correct (expected) responses       (header fields AA=1 RA=0):        484  96.80%  99.79%
Intercepted responses              (header fields AA=0 RA=1):          1   0.20%   0.21%
                                                                            ^       ^
Percentage of ALL measurement records ======================================+       |
Percentage of all measurements with 0 < rt <5000 ===================================+

Apart from the results on NSID, the different instances of the anycast server reached by the query, we also report on observed round trip times of the queries and the observed combinations of  AA and RA header fields.

Ping report

For the ping report we take as example a measurement to www.bbc.co.uk over IPv4 where the DNS lookup for the IP address was performed locally on each of the participating probes. Collectively, the probes found a total of 19 unique IP addresses which were pinged. Although the analysis code is all set to collect and report the statistics per unique destination IP, this would result in too much output for human consumption. We therefore aggregate the statistics and report on the combined set of destination IPs.

Report for measurement 1008887
------------------------------

Measurement type: ping
Destination hostname: www.bbc.co.uk
Number of destination IP addresses:  19
Total samples:                       24284
Destination: All IPs combined
Timeinterval: 2013-05-07T01:29Z  -  2013-05-07T03:30Z
Packets sent: 61041
Packets received: 60602
Overall loss rate: 0.72%

Total probes measuring:   1011
Probes with 100% errors:   158
Total errors on probes:    175
Most common error:"dns resolution failed: non-recoverable failure in name resolution (168x)"

Probes with no packets lost:      810
Probes with 0%-5% loss:            28
Probes with 5%-20% loss:            7
Probes with 20%-40% loss:           4
Probes with 40%-60% loss:           1
Probes with 60%-80% loss:           0
Probes with 80%-100% loss:          0
Probes with 100% loss:              3
Probes not sending any packets:     0

RTT distributions:
-----------------

2.5 percentile ("Minimum")
lowest 2.5 percentile RTT in all probes:    1.29ms
2.5% of probes had 2.5 percentile    <=     5.23ms
25% of probes had 2.5 percentile   <=    24.43ms
50% of probes had 2.5 percentile     <=    37.64ms
75% of probes had 2.5 percentile     <=    56.37ms
97.5% of probes had 2.5 percentile   <=   314.53ms
highest 2.5 percentile in all probes      404.98ms

Median
lowest median RTT in all probes:         2.25ms
2.5% of probes had median RTT    <=      6.51ms
25% of probes had median RTT    <=     26.03ms
50% of probes had median RTT     <=     40.15ms
75% of probes had median RTT     <=     60.92ms
97.5% of probes had median RTT  <=    323.25ms
highest median RTT in all probes       428.17ms

97.5 percentile ("Maximum")
lowest 97.5 percentile RTT in all probes:    3.67ms
2.5% of probes had 97.5 percentile    <=    12.20ms
25% of probes had 97.5 percentile    <=    37.56ms
50% of probes had 97.5 percentile     <=    51.88ms
75% of probes had 97.5 percentile     <=    89.14ms
97.5% of probes had 97.5 percentile   <=   426.32ms
highest 97.5 percentile in all probes     1757.13ms

Instead of the better known minimum, average and maximum values, the ping report uses percentiles to characterize the distribution of observed round trip times. The reason for this is that with a large number of measurements, incidental outliers skew the average and don't provide a good indication of typically observed round trip times.

Traceroute report

The traceroute reports are in a less advanced state. Currently we collect statistics on average hop count, traces reaching and not reaching the target IP address and the number of unique traceroute vectors observed. Reply packets which are received out of order (too late or too early compared to the outgoing packet's TTL) make this last step of data parsing a bit more complex compared to the looking at output from standard command line traceroute tools. The example below lists results for one of the ongoing measurements to 128.0.168.1

Report for measurement 1004651
------------------------------

Measurement type: traceroute
Number of destination IP addresses:  1
Destination: 128.0.168.1
Timeinterval: 2013-05-02T00:09Z  -  2013-05-02T23:59Z
Number of traces: 69485
Number of unique routing vectors: 3595
Traces reaching target IP address:  66905
Average hop count: 14.02
Traces not ending at target IP 2580

Future directions

At the RIPE NCC we will continue to further develop this reporting functionality for RIPE Atlas data. A first, mostly technical step, is to migrate the DNS reporting code which is mostly done in seperate Perl code, to the general Python based framework. This will improve performance for large sets of DNS data. Next we will look at expanding the functionality. For that we are also interested in your feedback.  Do you find the reports useful? Do you have suggestions for additional parameters to be reported? Your input is welcome at atlas [at] ripe [dot] net

3 Comments

Anonymous says:
May 29, 2013 07:26 PM
These tools require measurement results in 'Fragments' format - is there a way to request that format be returned when using the API to retrieve results?
Bert Wijnen
Bert Wijnen says:
May 29, 2013 11:23 PM
If you look at this measurement:
   https://atlas.ripe.net/atlas/udm.html
You will see a "DOWNLOAD" button (in blue on my screen).
Just above that button, there is a "Format" pulldown menu.
From that you can choose "fragments".

Hope this answers your question
Bert
Antony Antony
Antony Antony says:
May 30, 2013 08:52 AM
append format=txt here is an example
https://udm01.atlas.ripe.net/[…]/?format=txt

is this what you are looking for?

full json is https://udm01.atlas.ripe.net/[…]/
Add comment

You can add a comment by filling out the form below. Only plain text is possible. Web and email addresses will be transformed into clickable links. Comments are moderated so they won't appear immediately.