The RIPE NCC is proposing to deprecate DBConstat, a service to check consistency of data in the RIPE Database and produce statistics. It was written over ten years ago and has changed little over that period.
Introduction
In the early days of version 3 of the RIPE Database there were not as many business rules and referential integrity checks in the software as there are now. This allowed several types of inconsistencies to build up in the data. Also syntax changes have been made many times over the years. The early policy was not to 'fix' users data after a change to the syntax. This left many inconsistencies in the data.
The idea behind DBConstat was to monitor and measure a set of inconsistencies. Graphs are published on the RIPE website showing total counts over time. Users can also request a consistency check of their own data based on a MNTNER object.
The DBConstat service was provided late in 2001. It reports on inconsistencies that were known at that time. It has not received much development effort since then. Some of those early inconsistencies no longer apply, while there may be others that are not checked. The service is written in perl and C, using an old, customised graphics package to produce the graphs and runs on an old legacy server.
The current software is unmaintainable. If there is still a need for such a service the RIPE NCC should design and write a new service to meet the needs of today's data. As part of the RIPE NCC's infrastructure improvements, this legacy server is end of life. The effort required to migrate this service to a new server running a different operating system and further modify the old, customised graphics libraries would be considerable. It may not even be possible to adapt such old software packages to run on a modern server.
Documentation of the service is available on the RIPE website. Graphs of current inconsistencies are also available.
Proposal
The following sections provide a detailed review of the current service. After looking closely at this, the RIPE NCC's impression is that the graphs provide little, if any, useful information. The per MNTNER reports are only requested by a handful of people each year. Looking at a few random requested reports we see little, if any, inconsistencies being fixed after receiving the reports. The RIPE NCC believes this is not the way to address inconsistencies in the RIPE Database.
The RIPE NCC therefore proposes to deprecate this service and will investigate outstanding inconsistencies and set up small projects where necessary to fix them. If the community feels some form of checking is needed for any specific issue we can look at providing customised tools for it.
Graphical data
Most of the inconsistencies measured by DBConstat are shown on the web site as graphs . Most of these graphs are flat lines within relatively small error margins. That generally shows it is static information that does not need to be constantly monitored. The RIPE NCC can generate lists of current inconsistencies and can re-generate these lists at any time from the database without needing DBConstat. In this article these graphs have been divided into groups based on how the inconsistencies should be handled or viewed.
Counts that no longer have any meaning
As this service is over 10 years old there are some checks that no longer have any meaning. The RIPE Database has evolved with new business rules and syntax checks, but DBConstat has not evolved at the same rate.
Inetnum Objects with Wrong Status:
Currently 49 with a yearly average of 42
This check does not take into account all current status values. Some objects with a valid status are now counted as wrong by DBConstat. Hence the increasing number. Even if the check was modified, business rules no longer allow the wrong status to be applied to objects within RIPE allocated space. The remaining objects within RIPE allocated space will be fixed, so there is no need for this check.
Inet6num Objects with Wrong Status:
Currently 528 with a yearly average of 334
This check does not take into account all current status values. Some objects with a valid status are now counted as wrong by DBConstat. Hence the increasing number. Even if the check was modified, business rules no longer allow the wrong status to be applied to objects within RIPE allocated space. The remaining objects within RIPE allocated space will be fixed, so there is no need for this check.
Inetnum Objects with Suspicious Boundaries:
Currently 26k with a yearly average of 24k
This check was based on the assumption that all networks should map to CIDR boundaries. So if an object range mapped to more than one CIDR address block it was assumed to be wrong. From a registration point of view, there is no requirement for this check.
Duplicate Person/Role Objects:
Currently 729k with a yearly average of 544k
Even if this check could accurately identify 'duplicate' PERSON/ROLE objects that relate to the same person or organisation, there is no policy or database rule that disallows this. Although it is preferable to minimise PERSON/ROLE objects, it is at the discretion of the user. Therefore there is no requirement for this check.
Invalid Contact References:
Currently 0 with a yearly average of 0
Syntax checks no longer allow these creations or modifications. As there are none, there is no requirement for this check.
Weak Maintainer Auth:
Currently 585 (no average available)
This check applied to the early days of the RIPE Database when we had NONE and MAIL FROM as valid authentication checks as well as CRYPT-PW. Now this check produces a meaningless number. The authentication is now based on PGP and MD5 with hidden hashes, so there is no requirement for this check.
No mnt-lower:
Currently 1931 (no average available)
This is another old check from the days when authentication for creating sub assignments defaulted to OK if there was no "mnt-lower:" attribute. Now the default action passes to the "mnt-by:" attribute if there is no "mnt-lower:" attribute. Therefore there is no requirement for this check.
Automated cleanup processes
This set of inconsistencies are already, or will be, subject to automated cleanup processes. The automated cleanup process could go even further than what was checked here.
Unreferenced Person/Role Objects:
Currently 272k with a yearly average of 109k
These are already subject to an automated cleanup process. Any PERSON/ROLE object that remains unreferenced for 90 days will be scheduled for deletion. There is no need for this check.
Unreferenced Maintainers:
Currently 813 with a yearly average of 768
These are not currently cleaned up by any automated process. The PERSON/ROLE cleanup process will be extended to cover MNTNER objects as well. So there is no need for this check.
Unreferenced Organisation Objects:
Currently 22k with a yearly average of 21k
These are not currently cleaned up by any automated process. The PERSON/ROLE cleanup process will be extended to cover ORGANISATION objects as well. So there is no need for this check.
RIPE Database data management cleanup
These inconsistencies will be cleaned up by the RIPE NCC as part of an ongoing database management process. Such cleanups will also be applied to the whole data set whenever there is a syntax change.
References to Non-Existing Person/Role Objects:
Currently 21 with a yearly average of 23
Business rules no longer allow these references. The remaining objects will be fixed, so there is no need for this check.
Legacy space policy exceptions
Legacy and ERX space is currently not subject to all policies and business rules that apply to RIPE allocated address space registered in the RIPE Database. Registrations with certain status values are currently allowed to bypass some of the business rules.
Inetnum Assignments from Assignments:
Currently 37k with a yearly average of 37k
Legacy and ERX space is currently allowed to create more specific objects that bypass some of the status business rules in some situations. This is not an inconsistency and therefore we don't need to check for it.
Inetnum Objects with Wrong PA/PI:
Currently 956 (no average available)
Legacy and ERX space is currently allowed to create more specific objects that bypass some of the status business rules in some situations. This is not an inconsistency and therefore we don't need to check for it.
Inetnum Assignments with no Allocations:
Currently 107 with a yearly average of 117
Legacy and ERX space is currently allowed to create more specific objects that bypass some of the status business rules in some situations. This is not an inconsistency and therefore we don't need to check for it.
RIPE NCC driven projects
This set of inconsistencies will be investigated by the RIPE NCC as a project and processes will be put in place to fix the data. Some of these inconsistencies are in very old data and the software no longer allows such objects to be created.
Overlapping Inetnum Objects:
Currently 7 objects with a yearly average of 8.8
Business rules should no longer allow these creations. However there is a bug that allows the creation under certain conditions. This bug is scheduled for fixing and the remaining objects to be fixed, so there is no need for this check.
Inetnum Objects with No Status:
Currently 32 with a yearly average of 33
Syntax checks no longer allow these creations or modifications. The remaining objects will be fixed, so there is no need for this check.
Inetnum Objects with Wrong PA/PI:
Currently 956 (no average available)
Business rules no longer allow these creations or modifications within RIPE allocated space. Any remaining objects within RIPE allocated space will be fixed, so there is no need for this check.
Inetnum Assignments with no Allocations:
Currently 107 with a yearly average of 117
Business rules no longer allow these creations or modifications within RIPE allocated space. Any remaining objects within RIPE allocated space will be fixed, so there is no need for this check.
Inetnum Assignments from Assignments:
Currently 38k with a yearly average of 37k
Registration policy does not permit these creations or modifications within RIPE allocated space. A business rule was never implemented in the software as Legacy and ERX space is currently allowed to do this. We have a clear picture of what is RIPE allocated space and what is Legacy or ERX space so we will implement a business rule that currently only applies to RIPE allocated space. Any remaining objects within RIPE allocated space will be fixed, so there is no need for this check.
Inet6num Objects with No Status:
Currently 8 with a yearly average of 10
Business rules no longer allow these creations or modifications. The remaining objects will be fixed, so there is no need for this check.
Per MNTNER Reports
Users can request a report of inconsistencies for any data maintained by a specified MNTNER object. The report contains the same data as that shown in the graphs, but in more detail for each resource concerned.
Looking at the logs for the usage of this service we see 21 requests so far in 2012. These requests came from only 9 unique MNTNER objects. This is a drop of about 50% on previous years. Taking some of these MNTNER objects and re-generating the DBConstat reports for them we get reports still showing a number of inconsistencies.
So basically only a handful of users actually use this service and it seems few of them (if any) fix any of the inconsistencies reported. Clearly this is not the way to fix these inconsistencies. Considering the numbers (shown in the graphical report above) it might be better to set up some small RIPE NCC driven projects to fix each inconsistency.
Comments
Please submit any comments to the RIPE Database Working Group mailing list .
Comments 0
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.