The RIPE Global Resource Service

Denis Walker — Jul 15, 2013 09:10 AM
Filed under:
The RIPE NCC has changed the way data is imported for this service. This allows us to present a more complete Global Resource Service (GRS) for all Internet resources. This article outlines the GRS service including the recent changes. None of these changes have any impact on the standard RIPE Database query service.

Introduction

The RIPE Global Resource Service is a service offered by the RIPE NCC to provide a single point of access to information about all Internet resources administered by the five Regional Internet Registries (RIR). To comply with European Data Protection legislation, personal data has been obfuscated. This does not include abuse contact details.

What this service offers

The primary goal of this service is to provide information on global Internet resources from all the RIRs and some routing registries (listed below). This involves inetnum and inet6num objects, representing IPv4 and IPv6 address space, and aut-num objects, representing Autonomous System Numbers (ASN). Other operational objects like route and route6 and additional supporting objects, such as person and role, are also included for some sources.

The raw data is imported from each registry as a nightly dump, so it can be up to one day behind the authoritative source of the data sets. The RIPE NCC is only authoritative for the data contained in the database for the 'ripe-grs' source. For the other source databases, the RIPE NCC has agreements with the authoritative sources to provide access to this data. The "source:" attribute of each object clearly shows who is authoritative for that data object.

inetnum:        193.0.18.0 - 193.0.21.255
netname: RIPE-NCC
descr: RIPE Network Coordination Centre
descr: Amsterdam, Netherlands
remarks: Used for RIPE NCC infrastructure.
country: NL
admin-c: DUMY-RIPE
tech-c: DUMY-RIPE
status: ASSIGNED PI
mnt-by: RIPE-NCC-MNT
mnt-lower: RIPE-NCC-MNT
source: RIPE-GRS
changed: unread@ripe.net 20000101

All objects are converted to, stored and displayed in RIPE RPSL format. When we display any data we can obfuscate the data depending on the requirements of the data set owner and the European Data Protection directives. The data can be queried by any of the RIPE Database query methods:

  • The Web Query form
  • The RESTful API
  • Command line queries.

With the data being in RPSL format and stored locally in a RIPE Database structure, RIPE Database features can be applied to this data. For example it is possible to do hierarchical queries on any source.

Because the data is dummified, there is no access control applied to queries on the GRS service. This means no user will be blocked by any queries to the GRS service. However, if you are blocked after excessive querying of the RIPE Database for personal data, then you will not be able to access the GRS service either during the period you are blocked. (This is something that may need to be reviewed and was raised on the DB WG mailing list last year.)

Integrity of data

As the Internet resource data is imported, it is checked against published lists from each of the RIRs. These lists declare what resources each RIR is administratively responsible for. This ensures that only the data they are authoritative for is entered into the GRS database. Any place holder objects, that each of the databases contain for administrative purposes, are eliminated from the final data sets. By combining these data sets into one logical database, the RIPE GRS provides complete coverage of these Internet resources without any gaps or overlaps. With the right query (described below) the RIPE GRS will return single responses for any global resource.

The data is currently quite heavily dummified when presented in response to a query. The RIPE NCC proposes to relax this strict dummification in the same way as proposed for bulk data downloads. The aim is to have one standard mechanism for dummifying personal data wherever this process is needed. The details of how it is presented now and the proposed changes are available in another RIPE Labs article.

Querying the GRS

Selecting sources

There are a number of ways to query the GRS service. For most queries the result is the same no matter which method was used. There is one exception with a command line query to maintain backwards compatibility (explained below). The currently available sources are:

$ whois -q sources
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf

RIPE:3:N:0-0
AFRINIC-GRS:3:N:0-0
APNIC-GRS:3:N:0-0
ARIN-GRS:3:N:0-0
JPIRR-GRS:3:N:0-0
LACNIC-GRS:3:N:0-0
RADB-GRS:3:N:0-0
RIPE-GRS:3:N:0-0

% This query was served by the RIPE Database Query Service version 1.66.3 (WHOIS1)

This can also be done with Web queries by entering '-q sources' in the search box. Individual sources can be selected using the '-s source' flag on the command line, or checking the 'Sources' boxes on Web queries. Multiple sources can be selected. You can also specify the sources with the RESTful API.

Note that there are two source options for 'ripe' data. If you query the source 'ripe' with '-s ripe' the main production RIPE Database service will be queried. If you query the source 'ripe-grs' with '-s ripe-grs' the GRS version of the RIPE Database will be queried. This is 'dummified on the fly' data from the RIPE Database to match the dummification process applied to the other GRS data sources. The 'ripe-grs' source is added to provide a complete GRS service without any access control query limits. This distinction applies with all the query interfaces.

The '-a' query flag is still available for a command line query for backwards compatibility. It's function is mixed. It returns data from the 'ripe' source and all the GRS sources, except for 'ripe-grs'. So queries using '-a' are still subject to access control query limits if personal data is returned. This is different for other query interfaces. Selecting 'All' on the Web query for GRS sources will query the ripe-grs' source. This will not be subject to any access control. You cannot mix source 'ripe' with any of the GRS sources in the Web queries. With the API there is no 'all' option. You can specify each individual source. If you specify source 'ripe' then access control will apply to your query results. If you choose source 'ripe-grs', no access control will apply.

These queries where one or more, or all, sources are selected accept any object type as the query string.

Queries for Internet Resources

There is a new query option '--resource' that only accepts Internet resource objects as the query string. These are inetnum, inet6num and aut-num objects. With this option the query is always made against the set of GRS sources as a single logical database. This logical database contains the full set of global Internet resources without any gaps or duplicates and no place holder or dummy objects.

The response you get is a single view of the requested resource object(s). No referenced objects are returned. No hierarchical query options are allowed with this option. A benefit of this query is its simplicity. Here is a resource, tell me something about it. You do not need to know which region it is in, RIPE GRS knows that. All the basic information is returned. It is not followed by lots of other objects with confusing details.

--resource 193.0.21.0

% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf

% Information related to '193.0.18.0 - 193.0.21.255'

inetnum:        193.0.18.0 - 193.0.21.255
netname:        RIPE-NCC
descr:          RIPE Network Coordination Centre
descr:          Amsterdam, Netherlands
remarks:        Used for RIPE NCC infrastructure.
country:        NL
admin-c:        DUMY-RIPE
tech-c:         DUMY-RIPE
status:         ASSIGNED PI
mnt-by:         RIPE-NCC-MNT
mnt-lower:      RIPE-NCC-MNT
source:         RIPE-GRS
changed:        unread@ripe.net 20000101

% Information related to '193.0.20.0/23AS3333'

route:          193.0.20.0/23
descr:          RIPE-NCC
origin:         AS3333
mnt-by:         RIPE-NCC-MNT
changed:        unread@ripe.net 20000101
source:         RIPE-GRS

% This query was served by the RIPE Database Query Service version 1.66.3 (WHOIS4)

This query option can be used with the RESTful API, Web queries (by adding it to the search string) and command line queries.

Next step?

We plan to look at how we can further develop this '--resource' query option and extend the basic information, but still keeping it simple.

One useful extension could be abuse contact details. We have this information for the RIPE resources. We will investigate how we can add this for data from other RIRs. One of the advantages of agile development, used by the RIPE Database development team, is we can offer the basic information now. Then over time add more features to it. So we can work towards a global abuse contact database for Internet resources. We don't have to wait until everything is in place to deliver the final product.

Another possible extension is to provide some basic details of the organisation that holds the resource.

The RIPE NCC would very much like your comments and feedback on this service and in particular the '--resource' feature. Please direct your comments to the Anti-Abuse Working Group mailing list.

1 Comment

Michael
Michael says:
Jul 18, 2013 11:28 AM
This is cool, nice work.
Add comment

You can add a comment by filling out the form below. Only plain text is possible. Web and email addresses will be transformed into clickable links. Comments are moderated so they won't appear immediately.

Related Items
A New RIPE Database Prototype

The RIPE NCC is improving the RIPE Database functionality and usability by introducing a trial of ...

New and Improved RIPE Registry Global Resource Service

We have redesigned and improved the way we mirror other databases. We now have a method of ...

Abuse Handling in the RIPE Database

This article describes a technical design for the introduction of abuse contact details in the RIPE ...

Updated Heuristics for the Abuse Finder Service

This article outlines some of the refinements made to the Abuse Finder tool based on community ...

Updates to the RIPE Database Query API and Search Clients

Since the first announcement of the RIPE Database Query API, some changes and additions have been ...

more ...