You are here: Home > Publications > RIPE Labs > Paul Palse > A New RIPE Database Prototype

A New RIPE Database Prototype

Paul Palse — 04 Nov 2010
The RIPE NCC is improving the RIPE Database functionality and usability by introducing a trial of an improved service. With this new service it is easier for any user to see which of the data objects they’re looking at are maintained by the RIPE NCC or an Internet resource Holder.

A New RIPE Database Prototype 

The RIPE NCC is improving the RIPE Database functionality and usability by introducing a trial of an improved service. With this new service it is easier for any user to see which of the data objects they’re looking at are maintained by the RIPE NCC or an Internet resource Holder. 

Currently, querying the RIPE Database to find out information about a prefix would return data that looks like this:

RDR image1

 

Figure 1: Current RIPE Database output

 

These query results might make perfect sense to a Local Internet Registry (LIR), but not to a casual user of the database. Since the RIPE Database is public, the data it holds needs to be easily accessed and easily understood.

A top level registration (TLR) is made by the RIPE NCC to an LIR or End User (resource Holder). Responsibility for maintaining the TLR data in the RIPE Database is shared between the RIPE NCC and the resource Holder. With this new service, the TLR data has been split into two parts. One part contains the fixed registration data. This is the part of the TLR information, supplied by the resource Holder to the RIPE NCC, which is then maintained in the RIPE Database by the RIPE NCC.  The second part contains the top level user data (TLU) which is that part of the TLR information which can be changed by the resource Holder at any time.

The principle aim of this new service is to make this separation well defined and clearly visible to anyone looking at the data. The RIPE NCC can then assert a level of accuracy and trust in the fixed data. The RIPE NCC provides tools and services for people who maintain the resource Holder’s data (Maintainer) to keep their data up to date and encourages them to do so. The RIPE NCC has no direct control over resource Holders data, which includes the TLU data and all more specific assignments made by the resource Holder. The final responsibility for resource Holder’s data remains with the resource Holder.

RDR image2

Figure 2: RIPE Database prototype output

 

Benefits of the New RIPE Database Service 

Enhanced Reliability  - It’s now easier to identify the fixed part of the TLR data and users will be able to trust the accuracy of this data. Any errors found in resource Holder’s data in the RIPE Database should not detract from the value of the fixed data maintained by the RIPE NCC.

Easier Reporting - It’s now easier to identify who is the Maintainer of outdated or incorrect data in the RIPE Database. 

Easier Maintainance – Maintainers of data can update all the data they are responsible for using one standard interface.

Guiding Principles 

The new service was designed taking into account the following principles: 

  1. Show a clear distinction between fixed data and resource Holder’s data for top level registrations
  2. Ensure that RIPE Database up-time and response times are maintained
  3. Provide data Maintainers with a common mechanism to change all the data they are responsible for maintaining in the RIPE Database
  4. Changes to the RIPE Database should have minimal impact on data Maintainers and other users 

Maintaining a strong registry is of the upmost importance to the RIPE NCC, and we are continually working with the Internet community to improve our services. 

How we can do it 

To make the clear distinction between the fixed data maintained by the RIPE NCC and the data maintained by or for the resource Holders, we divided the TLR data into two separate, physical layers at the RPSL object level in the RIPE Database. 

The new and modified business, syntax and authentication rules that we built around this data separation will ensure a consistent database model. The relationship between the separated objects and their contents enforces the goals of this trial, while at the same time maintaining the principles of the RPSL and RPSS structure of the RIPE Database.

This separation will result in two types of data:

  • Fixed data 
  • Data maintained by or for the resource Holder 

These two types will fall into four categories:

  • Fixed data for the RIPE NCC to manage resources in the RIPE service region
  • Fixed data with matching resource Holder data
  • Resource Holder data with matching fixed data (TLU)
  • Resource Holder data 

Different business and authentication rules apply to each of the four categories.

RDR image3

Figure 3: RIPE Database will contain four categories of data separating fixed data from resource holder data

 

There are currently four object types that fall into the separation layers:

  • INETNUM
  • INET6NUM
  • AUT-NUM
  • ORGANISATION

For a fifth object, AS-BLOCK, the functionality and contents will be merged into the new structure and it will then be deprecated.

Four new object types will be added to the database schema which are designed such that they form pairs with the four existing object types above:

  • INET-REG,
  • INET6-REG
  • AS-REG
  • ORG-REG

These four new and five current objects define the interface between the fixed data and the resource Holders data. The basic principle is that the -REG objects contain only fixed data and the other objects contain only resource Holder data. At the interface there is a matching pair of objects with the same primary key, but different object types. (There is a slight difference in the primary key for AS Numbers. This will be described later.)

All other object types are unchanged. All objects more specific to the TLU objects are also unchanged.

Definition of new object types and matching TLU objects

TLU objects

For all separated object types, the TLU object contains those parts of the original object that the resource Holder was able to change. No changes were made to the structure of the objects. However, for some attributes the mandatory or optional flag was changed. In some cases business rules require optional attributes to be included.

-REG objects

The matching -REG object to a TLU object contains those parts of the original object that the resource Holder should not change. Currently, in case the resource Holder does make changes to parts of the fixed data, the RIPE NCC maintains monitoring software that detects such changes. When detected, the resource Holder is asked to undo the changes. With the introduction of this new service, there will be no need for this monitoring anymore. The fixed data cannot be changed by users.

The fixed data in the -REG objects will look slightly different than the existing equivalent objects:

  • There will be no need for maintainer attributes, because the fixed data is by default maintained by the RIPE NCC. All changes to this data will be done by new internal database processes linked to the Registration Services processes. Our new database update service will not allow any user to change -REG object data.
  • The "changed:" attribute will be replaced by "created:" and "last-changed:" attributes. These will contain only time stamp values. Because this is fixed data maintained by the RIPE NCC, these time stamps will be auto-generated to provide accurate audit information.
  • The -REG objects will only include an optional "admin-c:" attribute if the object represents un-allocated or un-assigned Internet resources. In this case, the RIPE NCC is the administrator of these resources while in the free pool. For allocated or assigned Internet resources, the resource Holder will be the administrator for the resource. The TLU object will contain this information.

Management of the separated data

When the RIPE NCC allocates or assigns an Internet resource to a resource Holder, it will create the -REG object and the matching TLU object in the RIPE Database. The pair of objects will contain all the data that was previously in the single resource object, plus the new data for the -REG object. Only the RIPE NCC can create these objects.

Modifications or deletions of any –REG object can only be done by the RIPE NCC. The -REG object cannot be deleted while the matching TLU object exists.

The TLU object will be maintained by the resource Holder’s maintainer object. This means that all the normal update processes can be used to change any attributes in these objects by the resource Holder. There is no need for special object editors in the LIR Portal. Some currently mandatory attributes will be optional in this new service. But business rules will not allow some of these attributes to be included in a TLU object if they are now included in the matching –REG object.

For example the "status:" attribute will be optional in INETNUM objects. 

For the TLU INETNUM objects, the status of this range will be in the fixed INET-REG object. Resource Holders will not be allowed to add a "status:" attribute to the TLU INETNUM object. However, when a resource Holder assigns address space to an End User, "status:" will be a required attribute for the assignment INETNUM objects. 

As the TLU object relates to an allocated or assigned resource, it is unlikely that the resource Holder will need to delete this object. It will only be deleted if the resource is returned to or reclaimed by the RIPE NCC or if the organisation holding the resource is closed. Therefore, only the RIPE NCC can delete a TLU object.

References to organisations for separated data will be done at the fixed data level. This means a –REG object for a resource will reference the ORG-REG with a new “org-ref:” attribute. The matching TLU resource object will not be allowed to reference the matching TLU ORGANISATION object or any other ORGANISATION object. This ensures that the resource is always linked to the organisation details known to the RIPE NCC at the time the resource was made available to the resource Holder.

Queries for -REG and TLU objects

We have not yet fully implemented all the use cases for expected results to a default query involving separated data. In general when a queried primary key occurs in both a -REG and a TLU object, both objects will be returned. When recursive queries are made within the database to find all related secondary objects, any ORGANISATION referenced which has a -REG and a TLU object will return both objects.

Issues already identified

From internal presentations to RIPE NCC staff we have already identified some issues with this prototype service.

  • For this first iteration we have made assumptions about which parts of the data should be fixed. This of course needs further discussion.
  • Having no maintainer attributes on -REG objects might give the impression that the object is not maintained at all
  • Queries using filters to only return TLU objects look incomplete without the matching -REG data
  • Direct end user resources often reference ORGANISATION objects with type 'OTHER'. These ORGANISATION objects only contain user data. We need to have separated organisation details so there is a fixed ORG–REG object to reference from the fixed –REG object for the resource
  • There may be a use case for a query that returns the most specific, less specific fixed –REG object data for any given resource
  • The RIPE Routing Registry is often used to document routing of RIPE address space with non RIPE AS Numbers, and vice versa. This is currently done by allowing copies of non RIPE AS Numbers to be created in the RIPE Database. This does not fit well with this new service. We need to investigate alternative solutions to achieve the same result.

Impacts of data separation

Any implementation of a new service has impacts on users and their work practices. This is a list of some impacts we have identified.

  • Holders of the data in the RIPE Database can be clearly identified
  • Reliability and quality level of the fixed data will increase
  • No impact on resource Holder maintained data more specific to the TLU data
  • Users may need to make some change to their internal processes and software to accommodate the new database schema and updates of TLU objects using normal update methods instead of LIR Portal
  • Users may also need to handle new ERROR messages on updates resulting from the new business rules
  • Users may need to change any scripts used to parse query results where separated data is returned
  • Tighter implementation of address policy rules in the RIPE Database through revised business rules
  • Conversion of existing data for the deployment of such a service will need careful planning
  • Need for new toolset for the RIPE NCC Registration Services to manage the fixed and TLU data in the RIPE Database

Prototype service

A prototype service has been developed and is available now.

To query this service:

 whois -h whois-four.db.ripe.net <query>

To update this database you can send mail updates to:

auto-dbm-four _at_ ripe _dot_ net

or use the syncupdates service:

https://syncupdates-four.db.ripe.net

You will need to include your current credentials to authorise any update to the copy of your own data.

This service has a converted, partial copy of the RIPE Database data. It includes a full copy of the (separated) data objects:

  • INET-REG
  • INET6-REG
  • AS-REG
  • ORG-REG
  • INETNUM
  • INET6NUM
  • AUT-NUM
  • ORGANISATION

It also includes the ROUTE and ROUTE6 objects even though they have not changed. This is to give a more complete view when doing queries. All the MNTNER and KEY-CERT objects are also included to preserve resource Holders authentication for updating the data in this prototype.

DOMAIN, IRT and all the SET objects have not been included, but none of this data has changed. Also all personal data has been replaced by two dummy PERSON objects, one for resource Holder data and one for fixed data objects.

The data conversion for the separation may not be perfect in this prototype, but it is a very close approximation.

Object templates

 inet-reg:       [mandatory]  [single]     [primary/look-up key]
netname:        [mandatory]  [single]     [lookup key]
descr:          [mandatory]  [multiple]   [ ]
country:        [mandatory]  [multiple]   [ ]
org-ref:        [optional]   [single]     [inverse key]
admin-c:        [optional]   [multiple]   [inverse key]
status:         [mandatory]  [single]     [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
created:        [generated]  [single]     [ ]
last-changed:   [generated]  [single]     [ ]
source:         [mandatory]  [single]     [ ]
 inet6-reg:      [mandatory]  [single]     [primary/look-up key]
netname:        [mandatory]  [single]     [lookup key]
descr:          [mandatory]  [multiple]   [ ]
country:        [mandatory]  [multiple]   [ ]
org-ref:        [optional]   [single]     [inverse key]
admin-c:        [optional]   [multiple]   [inverse key]
status:         [mandatory]  [single]     [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
created:        [generated]  [single]     [ ]
last-changed:   [generated]  [single]     [ ]
source:         [mandatory]  [single]     [ ]
 as-reg:         [mandatory]  [single]     [primary/look-up key]
as-name:        [mandatory]  [single]     [ ]
descr:          [mandatory]  [multiple]   [ ]
org-ref:        [optional]   [single]     [inverse key]
admin-c:        [optional]   [multiple]   [inverse key]
status:         [mandatory]  [single]     [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
created:        [generated]  [single]     [ ]
last-changed:   [generated]  [single]     [ ]
source:         [mandatory]  [single]     [ ]
 org-reg:        [mandatory]  [single]     [primary/look-up key]
org-name:       [mandatory]  [single]     [lookup key]
org-type:       [mandatory]  [single]     [ ]
descr:          [mandatory]  [multiple]   [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
created:        [generated]  [single]     [ ]
last-changed:   [generated]  [single]     [ ]
source:         [mandatory]  [single]     [ ]
 inetnum:        [mandatory]  [single]     [primary/look-up key]
netname:        [optional]   [single]     [lookup key]
descr:          [optional]   [multiple]   [ ]
country:        [mandatory]  [multiple]   [ ]
org:            [optional]   [single]     [inverse key]
admin-c:        [mandatory]  [multiple]   [inverse key]
tech-c:         [mandatory]  [multiple]   [inverse key]
status:         [optional]   [single]     [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
mnt-by:         [mandatory]  [multiple]   [inverse key]
mnt-lower:      [optional]   [multiple]   [inverse key]
mnt-domains:    [optional]   [multiple]   [inverse key]
mnt-routes:     [optional]   [multiple]   [inverse key]
mnt-irt:        [optional]   [multiple]   [inverse key]
changed:        [mandatory]  [multiple]   [ ]
source:         [mandatory]  [single]     [ ]
 inet6num:       [mandatory]  [single]     [primary/look-up key]
netname:        [optional]   [single]     [lookup key]
descr:          [optional]   [multiple]   [ ]
country:        [mandatory]  [multiple]   [ ]
org:            [optional]   [single]     [inverse key]
admin-c:        [mandatory]  [multiple]   [inverse key]
tech-c:         [mandatory]  [multiple]   [inverse key]
status:         [optional]   [single]     [ ]
remarks:        [optional]   [multiple]   [ ]
notify:         [optional]   [multiple]   [inverse key]
mnt-by:         [mandatory]  [multiple]   [inverse key]
mnt-lower:      [optional]   [multiple]   [inverse key]
mnt-routes:     [optional]   [multiple]   [inverse key]
mnt-domains:    [optional]   [multiple]   [inverse key]
mnt-irt:        [optional]   [multiple]   [inverse key]
changed:        [mandatory]  [multiple]   [ ]
source:         [mandatory]  [single]     [ ]
 aut-num:        [mandatory]  [single]     [primary/look-up key]
as-name:        [optional]   [single]     [ ]
descr:          [optional]   [multiple]   [ ]
member-of:      [optional]   [multiple]   [ ]
import:         [optional]   [multiple]   [ ]
mp-import:      [optional]   [multiple]   [ ]
export:         [optional]   [multiple]   [ ]
mp-export:      [optional]   [multiple]   [ ]
default:        [optional]   [multiple]   [ ]
mp-default:     [optional]   [multiple]   [ ]
remarks:        [optional]   [multiple]   [ ]
org:            [optional]   [single]     [inverse key]
admin-c:        [mandatory]  [multiple]   [inverse key]
tech-c:         [mandatory]  [multiple]   [inverse key]
notify:         [optional]   [multiple]   [inverse key]
mnt-lower:      [optional]   [multiple]   [inverse key]
mnt-routes:     [optional]   [multiple]   [inverse key]
mnt-by:         [mandatory]  [multiple]   [inverse key]
changed:        [mandatory]  [multiple]   [ ]
source:         [mandatory]  [single]     [ ]
 organisation:   [mandatory]  [single]     [primary/look-up key]
org-name:       [optional]   [single]     [lookup key]
org-type:       [optional]   [single]     [ ]
descr:          [optional]   [multiple]   [ ]
remarks:        [optional]   [multiple]   [ ]
address:        [mandatory]  [multiple]   [ ]
phone:          [optional]   [multiple]   [ ]
fax-no:         [optional]   [multiple]   [ ]
e-mail:         [mandatory]  [multiple]   [lookup key]
org:            [optional]   [multiple]   [inverse key]
admin-c:        [optional]   [multiple]   [inverse key]
tech-c:         [optional]   [multiple]   [inverse key]
ref-nfy:        [optional]   [multiple]   [inverse key]
mnt-ref:        [mandatory]  [multiple]   [inverse key]
notify:         [optional]   [multiple]   [inverse key]
abuse-mailbox:  [optional]   [multiple]   [inverse key]
mnt-by:         [mandatory]  [multiple]   [inverse key]
changed:        [mandatory]  [multiple]   [ ]
source:         [mandatory]  [single]     [ ]

Examples

Unchanged resource Holder data
 mntner:       LIR-MNT
descr:        Test maintainer for ipv6 status
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
auth:         MD5-PW $1$12345678$ctXe/33SiImj97lXBf4Mw0
mnt-by:       LIR-MNT
referral-by:  TEST-DBM-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20020101
source:       RIPE
Unchanged resource Holder data
 mntner:       END-USER-MNT
descr:        Test maintainer for ipv6 status
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
auth:         MD5-PW $1$12345678$mvVg2qg9mfB9tlQoFo2W//
mnt-by:       END-USER-MNT
referral-by:  TEST-DBM-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20020101
source:       RIPE

Fixed data with no matching TLU resource Holder data

 org-reg:       ORG-Ro1-RIPE
org-name:      RIR organisation
org-type:      RIR
descr:         Registry
created:       20020101
last-changed:  20020101
source:        RIPE

Fixed data with matching TLU resource Holder data

 org-reg:       ORG-Lo1-RIPE
org-name:      LIR organisation
org-type:      LIR
descr:         business
created:       20020101
last-changed:  20020101
source:        RIPE

TLU resource Holder data with matching –REG data

 organisation:  ORG-Lo1-RIPE
address:       Address
e-mail:
 
  dbtest _at_ ripe _dot_ net
 
 admin-c:       TP1-RIPE
tech-c:        TP1-RIPE
mnt-ref:       LIR-MNT
mnt-by:        LIR-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20030102
source:        RIPE

Resource Holder data with no matching –REG data

 organisation:  ORG-Eo2-RIPE
org-name:      EXAMPLE organisation
org-type:      OTHER
descr:         end user
address:       Address
e-mail:
 
  dbtest _at_ ripe _dot_ net
 
 admin-c:       TP1-RIPE
tech-c:        TP1-RIPE
mnt-ref:       END-USER-MNT
mnt-by:        END-USER-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20030102
source:        RIPE

Fixed data with no matching TLU resource Holder data

 inet-reg:       10.0.0.0 - 10.255.255.255
netname:        EU-ZZ
descr:          European Regional Registry
admin-c:        RP1-RIPE
country:        EU
status:         ALLOCATED UNSPECIFIED
org-ref:        ORG-Ro1-RIPE
created:        20020101
last-changed:   20020101
source:         RIPE

Fixed data with matching TLU resource Holder data

 inet-reg:       10.16.0.0 - 10.16.255.255
netname:        LIR_alloc
descr:          LIR allocation
country:        NL
status:         ALLOCATED PA
org-ref:        ORG-Lo1-RIPE
created:        20020101
last-changed:   20020101
source:         RIPE

TLU resource Holder data with matching –REG data

 inetnum:      10.16.0.0 - 10.16.255.255
country:      NL
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
mnt-by:       LIR-MNT
notify:
 
  dbtest _at_ ripe _dot_ net
 
 changed:
 
  dbtest _at_ ripe _dot_ net
 
 20030101
source:       RIPE

Resource Holder data with no matching –REG data

 inetnum:      10.16.151.0 - 10.16.151.255
netname:      TEST-NET-NAME
descr:        TEST network
status:       ASSIGNED PA
org:          ORG-Eo2-RIPE
country:      NL
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
mnt-by:       LIR-MNT
notify:
 
  dbtest _at_ ripe _dot_ net
 
 changed:
 
  dbtest _at_ ripe _dot_ net
 
 20030101
source:       RIPE
Fixed data with no matching TLU resource Holder data
 inet6-reg:      2001:0600::/23
netname:        EU-ZZ-2001-0600
descr:          European Regional Registry
country:        EU
admin-c:        RP1-RIPE
status:         ALLOCATED-BY-RIR
org-ref:        ORG-Ro1-RIPE
created:        20020101
last-changed:   20020101
source:         RIPE

Fixed data with matching TLU resource Holder data

 inet6-reg:      2001:0600::/25
netname:        LIR_alloc
descr:          LIR allocation
country:        EU
admin-c:        TP1-RIPE
status:         ALLOCATED-BY-RIR
org-ref:        ORG-Lo1-RIPE
created:        20020101
last-changed:   20020101
source:         RIPE

TLU resource Holder data with matching –REG data

 inet6num:     2001:0600::/25
country:      EU
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
mnt-by:       LIR-MNT
mnt-lower:    LIR-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20020101
source:       RIPE
 
 
Resource Holder data with no matching –REG data
 inet6num:     2001:0600::/27
netname:      UsersIP
descr:        business
status:       ASSIGNED
org:          ORG-Eo2-RIPE
country:      EU
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
mnt-by:       LIR-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20020101
source:       RIPE

Fixed data with matching TLU resource Holder data

 as-reg:       AS3333 - AS3333
as-name:      my_autnum
descr:        aut-num for user
status:       ASSIGNED
created:      20020101
last-changed: 20020101
source:       RIPE

TLU resource Holder data with matching –REG data

 aut-num:      AS3333
import:       from AS1 accept {1.2.3.4/24}
export:       to AS0 announce {1.2.3.4/24}
admin-c:      TP1-RIPE
tech-c:       TP1-RIPE
notify:
 
  dbtest _at_ ripe _dot_ net
 
 mnt-by:       LIR-MNT
changed:
 
  dbtest _at_ ripe _dot_ net
 
 20030101
source:       RIPE

Fixed data with no matching TLU resource Holder data

 as-reg:       AS65536 - AS69999
as-name:      free_pool
descr:        aut-num free pool
status:       NOT ASSIGNED
admin-c:      RP1-RIPE
org-ref:      ORG-Ro1-RIPE
created:      20020101
last-changed: 20020101
source:       RIPE
 

0 Comments

Add comment

You can add a comment by filling out the form below. Comments are moderated so they won't appear immediately. If you have a RIPE NCC Access account, we would like you to log in.