In addition to the actual data presented in a RIPE Database object, there is additional data, not directly related to the information contained within an object but still valuable from an operational point of view. The RIPE NCC is proposing a method to optionally display this metadata alongside the actual object. This will make maintenance of objects easier and will improve the overall quality of data in the RIPE Database.
Background
The RIPE Database contains millions of objects. There are several object types, each representing a defined set of data. In many cases, there is additional information related to an object. This information is not part of the defined data set, but it contains valuable operational data.
At the moment, RIPE Database users have no access to most of this data. An example is the key-cert object. When the key in a key-cert object has expired, a warning is only shown when a user wants to use the key to update their data. Unreferenced person and role objects are other examples. They are deleted automatically if unreferenced for 90 days. Although the maintainer is notified of the deletion, there is no way for the maintainer to know how long an object has been unreferenced and when it is going to be deleted. There are also objects which are syntactically incorrect. They were created or updated a long time ago with an old set of rules.
A maintainer usually does not realise an object is incorrect until they try to update it. The RIPE NCC is proposing a method to present this object metadata alongside the usual RIPE Database output. This way, there will be an informal way to communicate operational metadata to interested parties.
We are naming this new method object tags .
How does it work?
On port 43 queries, the new feature will be disabled by default and users will be able to enable it with a special option:
$whois –h whois.ripe.net -- “--show-tag-info 10/8”
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf
% Note: this output has been filtered.
% To receive output for a database update, use the "-B" flag.
% Information related to '10.0.0.0 - 10.255.255.255'
inetnum: 10.0.0.0 - 10.255.255.255
netname: IANA-ABLK-RESERVED1
descr: Class A address space for private internets
country: EU # Country is really world wide
org: ORG-IANA1-RIPE
admin-c: RFC1918-RIPE
tech-c: RFC1918-RIPE
status: ALLOCATED UNSPECIFIED
remarks: See RFC1918 for further information
mnt-by: RIPE-NCC-HM-MNT
mnt-lower: RIPE-NCC-HM-MNT
mnt-routes: RIPE-NCC-RPSL-MNT
source: RIPE # Filtered
% Tags relating to ‘10.0.0.0 - 10.255.255.255'
% PLACEHOLDER #This is a placeholder object and does not represent an allocation
Each tag will have two parts: the tag name and a description of the reason for the tag being added to the object. The tag name will have no spaces and the tag description will be separated from the tag name by a # sign, which is standard for line-ending comments in our port 43 output.
In the example above, the tag name is "PLACEHOLDER" and the tag description is "This is a placeholder object and does not represent an allocation".
Users will be able to filter any query output based on the presence of tag names. This will be a final stage filter on any output generated by the RIPE Database. There are two methods for this filtering. One method will only show objects that have the specified tag(s), very similar to “grep”ing the output from the query.
$whois –h whois.ripe.net -- “--filter-taginclude PLACEHOLDER 10/8”
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf
% Note: this output has been filtered.
% To receive output for a database update, use the "-B" flag.
% Note: Tags filtering is enabled,
% Only showing objects WITH tag(s): PLACEHOLDER
% Information related to '10.0.0.0 - 10.255.255.255'
inetnum: 10.0.0.0 - 10.255.255.255
netname: IANA-ABLK-RESERVED1
descr: Class A address space for private internets
country: EU # Country is really world wide
org: ORG-IANA1-RIPE
admin-c: RFC1918-RIPE
tech-c: RFC1918-RIPE
status: ALLOCATED UNSPECIFIED
remarks: See RFC1918 for further information
mnt-by: RIPE-NCC-HM-MNT
mnt-lower: RIPE-NCC-HM-MNT
mnt-routes: RIPE-NCC-RPSL-MNT
source: RIPE # Filtered
% Tags relating to ‘10.0.0.0 - 10.255.255.255'
% PLACEHOLDER #This is a placeholder object and does not represent an allocation
% This query was served by the RIPE Database Query Service version 1.57.1 (WHOIS3)
With the other method it will be possible to exclude the tags for a result set:
$whois –h whois.ripe.net -- “--filter-tagexclude PLACEHOLDER 10/8”
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf
% Note: this output has been filtered.
% To receive output for a database update, use the "-B" flag.
% Note: Tags filtering is enabled,
% Only showing objects WITHOUT tag(s): PLACEHOLDER
%ERROR:101: no entries found
%
% No entries found in source RIPE.
% This query was served by the RIPE Database Query Service version 1.57.1 (WHOIS3)
The “include” and “exclude” filter options will accept a comma-separated list of tag names. They can also be combined for more fine-tuned searching.
This tag information will also be included in query outputs via the RESTful API. The only difference will be that the metadata will always be included by default in the result set. But since the data is formatted in JSON or XML outputs, clients will be able to filter them out based on any rule set they define. Alternatively, they will be able to set a “--no-tag-info” option, in which case no tag information will be included in the output.
Basis for the service
The main aim for providing object metadata or tags is to help users improve data quality by informing them about potential maintenance issues with an object.
A secondary benefit is structuring and filtering output data. For example, there are a lot of placeholder objects in the RIPE Database, such as
as-block
objects or 10/8
inetnum
objects. By tagging these objects as “PLACEHOLDERS”, users will be able to exclude them from any query output and only see operational data.
There are many other secondary benefits of long-term or short-term object tagging to highlight transient changes to data in the database.
Restrictions
At the moment, we are planning to implement only system-generated tags. This means all tags will be added automatically by the RIPE Database system based on rules that will be defined in the software. At this stage, it will be not be possible for any RIPE Database object maintainers or RIPE NCC staff to manually add tags to an object. If needed, this feature can be added later.
Tags will be informal in nature. We are not planning to restrict the tag name or description in a pre-defined format. Obviously, as they will be added automatically by our system, each tag name will be fixed once it is introduced. The tag description is free text, but may follow a style for each tag and include specific information. For example, a tag to show unreferenced objects flagged for deletion will include a count down of number of days before it is deleted.
The main idea is to provide a free platform for providing metadata about objects, which is what will separate object tags from the formal definition of RIPE RPSL for RIPE Database objects. That said, every system operation that will add tags will be defined clearly and the criteria for adding these tags will be published.
The way forward
We propose to start the project in Beta status with just an initial, basic set of tags in order to measure the usefulness of this service. After that, if the community finds this addition a useful feature, we can add new tags based on community input.
Later, we can add new features such as maintainer-added tags, allowing users to tag their own data in some way. We will perhaps add more search and filtering options based on feedback received from the community.
If you have any ideas or comments about this new feature, please participate in the discussion on the
RIPE Database Working Group mailing list
.
Comments 0
Comments are disabled on articles published more than a year ago. If you'd like to inform us of any issues, please reach out to us via the contact form here.