Over the last few months, since the last release of our RESTful Query Web Service, we have been adding functionality and fixed some issues that we found while using the underlying API. So, with this update we are releasing all these refinements and fixes to you. Most of the improvements address parsing complexities of the RPSL format.
Complex Reference Lookup
By complex reference we mean an attribute whose value can point to different types of objects or can even be just a string value.
Resolving a complex reference means identifying the type of object referenced without actually first querying for it.
This can obviously be done by analysing the value of the attribute by prefix or regular expression matching. Also, for certain RPSL attributes the type of the objects that holds a certain attribute determines the reference type. For example a "members" attribute of an AS-SET object can only point to an AS-SET or an AUT-NUM object, however a "member" attribute in a RS-SET can only point to a RS-SET, an INET-RTR or just hold an IP prefix string.
The xml responses now contain "xlink" pointers for all possible types of references, so clients don't not need to do any RPSL parsing anymore for these attributes.
The RIPE Database Web Services can be found here:
http://lab.db.ripe.net/portal/
Comma Separated Lookup references
For some types of attributes the value can be a list of comma separated lookup keys. Since some of these attributes can point to different types of objects, comma separated list can contain a mix of lookup keys for objects of different types.
For example, the "member" attribute just described before can contain multiple comma separated lookup keys and these keys may point to objects of different types.
With this new version of the Query API all comma separated list containing lookup keys will be normalised, so that any lookup key will have its own attribute element and its own "xlink" reference pointer. Clients will not need to do any parsing anymore, everything will already be normalised and strongly typed.
If we do a port 43 query to a test database for an AS-SET object with primary key "as-33334" it returns:
as-set: as-33334
descr: description
members: as3333,as3334
mbrs-by-ref: mnt10
tech-c: pe1-TEST
admin-c: pe1-TEST
mnt-by: mnt10
changed: a@a.com 20100804
source: TEST
In the previous version of the RESTful query service the XML response for the "members" attribute in the same object would have been:
...
<attribute name= "members" value= "as3333,as3334" /> ...
The new version will generate the response for the "members" attribute as follows:
...
<attribute name= "members" value= "as3333" referenced-type= "aut-num" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/aut-num/as3333" /> </attribute> <attribute name= "members" value= "as3334" referenced-type= "aut-num" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/aut-num/as3334" /> </attribute> ...
This is obviously much easier handle and the links are strongly typed, which is especially handy when lists reference a mix of object types.
An indication of what attributes will be handled this way:
"mnt-by" pointing to MNTNER objects
"mnt-domain" pointing to MNTNER objects
"mnt-lower" pointing to MNTNER objects
"mnt-ref" pointing to MNTNER objects
"referral-by" pointing to MNTNER objects
"mbrs-by-ref" pointing to multiple maintainers or ANY
"members" within an AS-SET pointing to AS-SET and AUT-NUM objects
"members" within a RS-SET pointing to RS-SET objects and IPv4 address prefixes
"members" within a RTRS-SET pointing to an RTR-SET object, INET-RTR objects and IP addresses
"mp-members" within an AS-SET pointing to AS-SET and AUT-NUM objects
"mp-members" within RS-SET pointing to RS-SET objects and IPv4 address prefixes
"mp-members" within RTRS-SET pointing to RTR-SET objects, INET-RTR objects and IP addresses
"member-of" within within an AUT-NUM objects
"member-of" within within an ROUTE(6) object pointing to RS-SET objects
"member-of" within within an INET-RTR object pointing to RTR-SET objects
As an example of the last scenario of the list above, we do a port 43 query to a test database for an RTR-SET object with primary key "RTRS-1":
rtr-set: RTRS-1
descr: description
members: Amsterdam.ripe.net,RTRS-1,192.87.45.190
mp-members: Amsterdam.ripe.net,RTRS-1,192.87.45.190
mbrs-by-ref: ANY,mnt10
tech-c: pe1-TEST
admin-c: pe1-TEST
mnt-by: mnt10
changed: a@a.com 20100805
source: TEST
The XML response for this RTR-SET from the old Query API used to be:
...
<attribute name= "members" value= "Amsterdam.ripe.net,RTRS-1,192.87.45.190" /> <attribute name= "mp-members" value= "Amsterdam.ripe.net,RTRS-1,192.87.45.190" /> <attribute name= "mbrs-by-ref" value= "ANY,mnt10" /> ...
This version will now return:
...
<attribute name= "members" value= "Amsterdam.ripe.net" referenced-type= "inet-rtr" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/inet-rtr/Amsterdam.ripe.net" /> </attribute> <attribute name= "members" value= "RTRS-1" referenced-type= "rtr-set" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/rtr-set/RTRS-1" /> </attribute> <attribute name= "members" value= "192.87.45.190" /> <attribute name= "mp-members" value= "Amsterdam.ripe.net" referenced-type= "inet-rtr" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/inet-rtr/Amsterdam.ripe.net" /> </attribute> <attribute name= "mp-members" value= "RTRS-1" referenced-type= "rtr-set" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/rtr-set/RTRS-1" /> </attribute> <attribute name= "mp-members" value= "192.87.45.190" /> <attribute name= "mbrs-by-ref" value= "ANY" /> <attribute name= "mbrs-by-ref" value= "mnt10" referenced-type= "mntner" > <link xlink:type= "locator"
xlink:href= "http://dev.db.ripe.net/whois/lookup/test/mntner/mnt10" /> </attribute> ...
Continuation Lines and EOL comments
Continuation lines and "End Of Line" comments allow users to create very complex (human readable) RPSL. It becomes nearly impossible to parse these values without complex handling of edge cases, especially because it can be used in conjunction with comma separated lookup keys.
For example
Below is a valid attribute that holds 1 string value ("ci cc c io") split over multiple continuation lines and with end of line comments that also span over multiple lines:
descr: # end of line comment
+ # more eolc
ci
cc c # more eolc
io # more eolc #
+ #even more!
Next is a valid attribute that holds comma separated lookup keys on multiple continuations, all intertwined with end of line comments:
mnt-by: #comment before keys
mnt1, # eol comment 1
mnt2,mnt3 # eol comment 2
+ ,mnt4 #eol comment 3
#eol comm # ent4
Parsing these kinds of values can be very complex, especially considering the fact that there is no clear specification of what the correct use of these features is.
Splitting one reference key over multiple continuation lines is not valid in RPSL and any RIPE Database update interface will reject it. On the other hand, splitting multiple comma separated keys over multiple continuation lines exactly around the commas is considered valid RPSL by all update interfaces.
This makes parsing very complex and every client has to reinvent the wheel when trying to create the correct parser.
For this reason the Query API will now normalize these values, separating them from end of line comments, concatenating continuations into valid single values and handling comma separated keys split on continuations.
In the new version of the RESTful Web Services the XML for the "ci cc c io" example will look like this:
...
<attribute name= "descr" value= "ci cc c io"
comment= "end of line comment more eolc more eolc more eolc # even more!" /> ...
The xml for the "mnt-by" example will look like this:
...
<attribute name= "mnt-by"
value= "mnt1"
comment= "comment before keys eol comment 1 eol comment 2 eol comment 3 eol comm # ent4"
referenced-type= "mntner" >
<link xlink:type= "locator" xlink:href= "http://dev.db.ripe.net/whois/lookup/test/mntner/mnt1" />
</attribute>
<attribute name= "mnt-by"
value= "mnt2"
comment= "comment before keys eol comment 1 eol comment 2 eol comment 3 eol comm # ent4"
referenced-type= "mntner" >
<link xlink:type= "locator" xlink:href= "http://dev.db.ripe.net/whois/lookup/test/mntner/mnt2" />
</attribute>
<attribute name= "mnt-by"
value= "mnt3"
comment= "comment before keys eol comment 1 eol comment 2 eol comment 3 eol comm # ent4"
referenced-type= "mntner" > <link xlink:type= "locator" xlink:href= "http://dev.db.ripe.net/whois/lookup/test/mntner/mnt3" /> </attribute> <attribute name= "mnt-by"
value= "mnt4"
comment= "comment before keys eol comment 1 eol comment 2 eol comment 3 eol comm # ent4"
referenced-type= "mntner" > <link xlink:type= "locator" xlink:href= "http://dev.db.ripe.net/whois/lookup/test/mntner/mnt4" /> </attribute> ...
Fixes
Aside from some little bugfixes we also fixed:
- The HTML responses from the forms are now stateless, reference links to other objects are no longer form commands but are links (href) to resources pointing to the lookup service and as such are now bookmarkable.
Comments 0
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.