You are here: Home > Publications > RIPE Labs > RIPE Database > Database API > Introduction to the RIPE Database API

Introduction to the RIPE Database API

Paul Palse — Apr 2010 expired

 

 

Note that this article is not up to date anymore. Please refer to the most up-to-date RIPE Database documentation .

 

Introduction

I am sure it won’t come as a surprise to hear that the developers of the RIPE Database are also ‘power users’ of the database. We pull out information for statistics, maintenance and support and we run projects where we work with bulk data.

The Database Group as well as several other departments within the RIPE NCC has written applications that interface with the RIPE Database.

A recurring task for any project that needs bulk or programmatic access to the RIPE Database is parsing the returned RPSL output. There are several RPSL parser libraries out there for different programming languages, including some written by the RIPE NCC. They all have their own strong and weak points. Anyone who has written code that interacts with the RIPE Database will know that working with RPSL can be tricky.

This time we used a different approach to solving some of these issues, which has resulted in this new prototype service.

A Different Approach

Before going into the details of this prototype service, we would like to take the opportunity to give some background information about this new approach.

Scrum has been more widely accepted within the RIPE NCC in recent years. This is an Agile software development practice that streamlines project management and encourages iterative development cycles. It basically means that large projects are split up into small deliverables, produced within a fixed, predetermined time box called a ‘Sprint’. In the Database Group, Sprints last for two weeks. The most important element of a Sprint is “working software”. Creating working versions of the software in an iterative environment allows us to adapt and react to changing requirements more easily and more frequently.

When the RIPE NCC Database Group, together with other RIPE NCC departments, identified a need for an improved, well-defined software interface, we dedicated some of our Sprints to deliver exactly that.

Some basic requirements were:

  1. Develop a well-defined software interface (API) for RIPE Database queries.
  2. Make the query service available over a protocol that maximizes accessibility and is widely supported by routers and firewalls.
  3. A language agnostic API.
  4. Output should be parsable using standard solutions.

With this set of requirements, we chose to implement the API in the form of a RESTful Web Service.

REST stands for “REpresentational State Transfer” and is based on the existing design of the HTTP protocol. A RESTful web service over HTTP works with the same set of request and response codes (vocabulary) as plain HTTP. It also enhances and extends the existing HTTP URIs, request and response headers, Internet media types and response styles.

The RIPE NCC is not the only RIR looking into exposing whois data via RESTful Web Services. ARIN is also running a demo service of their Whois Web Services at http://whoisrws-demo.arin.net/ui .

We started with a service that answers a simple RIPE Database question:

Give me the object of type [type] with primary key [primary key]

The way such a request is made is by using a URI, just like requesting a webpage (HTTP GET for techies).

For the “ripe-dbm-mnt” maintainer the query to answer this question would translate into:

http://apps.db.ripe.net/whois/lookup/ripe/mntner/ripe-dbm-mnt.xml

The difference between this new service and current RIPE Database clients is the output. This service returns data in XML. XML is the best choice for client services, web applications and desktop tools that can take advantage of the powerful parsing and binding API which development platforms like Java, C#, Python provide these days.

One powerful aspect of XML is that it can be easily transformed and filtered via business transformations, for example when connecting applications that use different schema languages. XML can also be converted into other non-XML representation styles like JSON or HTML by using technologies like XSLT, XQuery or client-side templating solutions that are well known to web 2.0 developers.

Later in this article we provide an XSL template that converts the XML output into RPSL.

JSON is the best choice for lightweight client components that don't need to reconstruct a full-fledged object model and don't want to depend on third party parsing or binding libraries that may affect portability across different browsers. For example JavaScript, flash clients and other types of client side components may favour a JSON response to render their own views using some JSON manipulation or templating solution. For this reason the new services also supply JSON content negotiation. Clients can get a JSON representation of our data by just setting the right content type in the Accept header of their HTTP requests. It is important to note that JSON is just one data representation style. The standard representation we expose via these web services is XML.

To illustrate how the new service works, we are also releasing a sample look-up form that generates object view using the RESTful Query Web Services (RDQWS) here:

userfiles-image-RDB-lookupform.png

http://apps.db.ripe.net/portal/lookup.htm

The form builds the HTML views of the database objects by a simple DOM (Document Object Model) inspection of the XML response that the RDQWS provides. To see the XML response you can click the following button on the response in the look-up form:

userfiles-image-RDB-ico_xml.jpg

We encourage you to try this new look-up form, but be aware that the power of it all lies in the underlying service and its output.

This article will now continue with some of the technical details of the web service, aimed mainly at software developers and script writers.

Direct look-up API

Please find the documentation for the RESTful Query Web Services on RIPE Labs.

We will continue with some background information and some (code) examples.

The output schema

Submitting the query from the example earlier in this article returns the XML below:

http://apps.db.ripe.net/whois/lookup/ripe/mntner/ripe-dbm-mnt.xml

 <whois-resources service="lookup" xmlns:xlink="http://www.w3.org/1999/xlink">
  <link xlink:type="locator" xlink:href="/whois/objects/ripe/mntner/RIPE-DBM-MNT"/>
    <objects>
      <object type="mntner">
        <link xlink:type="locator" xlink:href="/whois/objects/ripe/mntner/RIPE-DBM-MNT"/>
        <source name="RIPE NCC" id="ripe"/>
        <primary-key>
          <attribute name="mntner" value="RIPE-DBM-MNT"/>
        </primary-key>
        <attributes>
          <attribute name="mntner" value="RIPE-DBM-MNT"/>
          <attribute name="descr" value="Mntner for RIPE DBM objects."/>
          <attribute name="admin-c" value="AMR68-RIPE" referenced-type="person-role">
          <link xlink:type="locator" xlink:href="/whois/objects/ripe/person-role/AMR68-RIPE"/>
        </attribute>
        <attribute name="tech-c" value="RD132-RIPE" referenced-type="person-role">
          <link xlink:type="locator" xlink:href="/whois/objects/ripe/person-role/RD132-RIPE"/>
        </attribute>
        <attribute name="org" value="ORG-NCC1-RIPE" referenced-type="organisation">
          <link xlink:type="locator" xlink:href="/whois/objects/ripe/organisation/ORG-NCC1-RIPE"/>
        </attribute>
        <attribute name="auth" value="PGPKEY-1290F9D2"/>
        <attribute name="auth" value="MD5-PW $1$zjDctbLS$YrqbSsejQwLUzTtwqmhE7/"/>
        <attribute name="mnt-by" value="RIPE-DBM-MNT" referenced-type="mntner">
          <link xlink:type="locator" xlink:href="/whois/objects/ripe/mntner/RIPE-DBM-MNT"/>
        </attribute>
        <attribute name="referral-by" value="RIPE-DBM-MNT" referenced-type="mntner">
          <link xlink:type="locator" xlink:href="/whois/objects/ripe/mntner/RIPE-DBM-MNT"/>
        </attribute>
        <attribute name="source" value="RIPE # Filtered"/>
      </attributes>
    </object>
  </objects>
</whois-resources> 

Although the response schema is still subject to change, there are some notable features that are useful now and for future service enhancements:

The first thing to note is the bookmark link to the object itself.

<link xlink:type="locator" xlink:href="/whois/objects/ripe/mntner/RIPE-DBM-MNT"/>

Each object has a type.

 

 <object type="mntner">

By looking at the type attribute in the object node, it is possible to determine the type of the object that is returned.

This is useful for several reasons:

There is a feature in the look-up service that makes it possible to look up a contact without knowing if it references a person or role object. For example, the query “http://apps.db.ripe.net/whois/lookup/ripe/person-role/pp-ripe.xml” will return a response containing a person object. It’s useful to know the type of the object the service returned.

We will be extending our web services with interfaces that will return multiple objects. Knowing the object types is useful, for example to filter for (un)desired results or if you want to build object binding solutions.

The XML also clearly identifies the primary key of the object instance. This can be seen in the above example bound by the <primary-key> tags.

Attributes may reference other objects and in this case the XML will contain a referenced-type element, and a link pointing to the child object:

 

 <attribute name="org" value="ORG-NCC1-RIPE" referenced-type="organisation"> 
  <link xlink:type="locator" xlink:href="/whois/objects/ripe/organisation/ORG-NCC1-RIPE"/> 
</attribute>

 

XML and XSL transformations

As we said earlier, it is possible to transform the XML responses into RPSL objects quite easily. We cannot guarantee that the style and formatting will be 100% equal to the original RPSL object, but it should be very close. The content will be exactly the same with regard to the attributes and values.

We used XSLT to transform XML to HTML in the RDQWS when a HTML response is requested. XSLT is easy to use and most programming languages support various ways to perform these transforms. The XSL template we use internally for some projects can be downloaded from:

http://labs.ripe.net/ripe-database/database-api/xml2rpsl-xslt.zip (MD5: 8194d94cd5b966cec4d2d6e38141db22)

What’s next?

We are working on a similar web service that covers search related use-cases. With the search service we aim to offer a RESTful Web service interface that covers the current RIPE Database search functionality. You can expect this service to hit RIPE Labs over the coming weeks.

We are looking forward to your feedback and feature requests. We’re also offering the possibility for anyone to share their new client code or projects with the community via RIPE Labs.

Writing client code

This part of the article is targeted at an audience with a software development background. If you are interested in writing a client for this RESTful interface, this may give you some ideas on where to start. The examples will be in Java, but are also of interest to developers using any other language.

We published a first version of the API documentation at:

http://labs.ripe.net/content/ripe-database-api-documentation

The look-up form we publish with this service uses a client for the RDQWS written in Java. To give an idea of how simple it is to write a client for the RDQWS we will provide two Java examples:

The first example is the “SimplePersonLookupClient” which just looks up a person using the Jakarta commons-http open source library. In practice, you can use any generic HTTP client implementation to invoke a RESTful service.

 
  import java.io.IOException; import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.HttpException; import org.apache.commons.httpclient.HttpStatus; import org.apache.commons.httpclient.methods.GetMethod; public class SimplePersonLookupClient { private static String url = "http://apps.db.ripe.net/whois/lookup/ripe/person/"; public String lookupPerson(String nicHandle) throws HttpException,IOException{ HttpClient client = new HttpClient(); GetMethod method = new GetMethod(url+nicHandle); try { int statusCode = client.executeMethod(method); if (statusCode != HttpStatus.SC_OK) { throw new HttpException("Method failed: " + method.getStatusLine()); } String responseBody = method.getResponseBodyAsString(); return responseBody } finally { method.releaseConnection(); } } public static void main(String[] args){ try{ SimplePersonLookupClient client=new SimplePersonLookupClient(); String xmlResponse=client.lookupPerson("pp-ripe"); System.out.println(xmlResponse); }catch(Exception e){ e.printStackTrace(); } } }
 

The following “LookupClient” uses the JBoss RESTEasy client framework to generate a fully-fledged dynamic proxy of our service. Other JAX-RS frameworks will offer similar facilities for Java. On other platforms like .NET there are similar frameworks that offer this functionality.

 

 import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import org.jboss.resteasy.client.ProxyFactory;
interface LookupClientInterface { 
  @GET
  @Path("whois/lookup/{source}/{objectType}/{pKey}")
  @Produces("text/xml")
  public String lookup(@PathParam("source") String source,
                       @PathParam("objectType") String obejctType,
                       @PathParam("pKey") String pKey);
}

public class LookupClient {

  public String executeLookup(String source, String objectType,String primaryKey){
    LookupClientInterface client=ProxyFactory.create(LookupClientInterface.class, 
                                                     "http://apps.db.ripe.net");
    return client.lookup(source,objectType,primaryKey);
  }
  public static void main(String [] args){
    LookupClient lookup=new LookupClient();
    String xmlResponse=lookup.executeLookup("ripe","person","pp-ripe");
    System.out.println(xmlResponse);
  }
}

 

There are some other equally interesting ways one could use the RDQWS.

For instance, Apple’s OSX and Microsoft Windows support widgets that can be developed using standard JavaScript.

There are also several browser plug-ins that extend the browser functionality through scripting and code injection. We suggest you look at Greasemonkey ( http://www.greasespot.net ), which allows you to customize the way a webpage is displayed using small bits of JavaScript. Mix Greasemonkey with the RDQWS and we’re sure you can come up with some very useful extensions.

0 Comments

Add comment

You can add a comment by filling out the form below. Comments are moderated so they won't appear immediately. If you have a RIPE NCC Access account, we would like you to log in.