You are here: Home > Publications > RIPE Labs > Angela Dall'Ara > Abuse-c Validation: Update on Progress and Some Numbers

Abuse-c Validation: Update on Progress and Some Numbers

Angela Dall'Ara — 15 May 2019
Contributors: Antony Gollan
With RIPE 78 around the corner, we want to update you on our work to validate abuse contacts in the RIPE Database.

A Quick Recap

As you’ll know from our previous updates (here and here), this is part of our work to implement a policy proposal that was accepted by the RIPE community in June 2018.

You can find more details in our previous articles, but to summarise our approach briefly: we started by splitting abuse contacts into three categories according to the objects in which they are referenced. We use a third-party tool to check whether an abuse contact was able to receive emails. Contacts that are marked as potentially invalid by the tool are sent a series of automated emails and reminders. Resource holders can then confirm that the email is working by clicking on a validation link that is sent to their abuse mailbox. If the contact is not working, they will need to update this (and we will then run the validation process on the new mail).

LIRs have three weeks to act on the automated emails we send before our staff start to contact them manually – first by using contacts in the LIR Portal before broadening their search if necessary. We can therefore think of this as a two-step process: an initial automated approach, followed by intervention by our staff if the first step is unsuccessful. 

This figure illustrates how abuse contacts are referenced in the RIPE Database

Current Status

While we had originally expected to complete our initial validation of all abuse contacts by the end of the year (December 2019), the project is actually running ahead of schedule. It now looks like we might be finished around the end of the summer. This is mostly due to a higher-than-expected response rate to our automated emails, with around 60% of cases being resolved without our staff needing to intervene. This number is actually somewhat higher, but if someone updates their abuse contact and replies: “Thank you – updated!” this re-opens the ticket and is counted as requiring staff intervention (to close the ticket manually).

LIR organisation abuse contacts [DONE]

  • Abuse contacts in total: ~18,200
  • Tickets created: ~3,500
    • Resolved via automated process: ~ 2,100
    • Resolved after staff intervention: ~1,400

LIR resource abuse contacts [DONE]

  • Abuse contacts in total: ~3,200
  • Tickets created: ~200
    • Resolved via automated process: ~160
    • Required staff intervention: ~40

End User abuse contacts [IN PROGRESS]

  • Abuse contacts in total: ~13,500
  • Tickets created: ~1,200
    • Resolved via automated process: ~650 so far
    • Resolved after staff intervention: ~300 so far

Some Observations and Lessons Learned

We started with LIR organisation objects first, which took around three months to finish. One thing to note especially was the lack of relevant technical knowledge that we encountered on the part of some members, particularly newer ones. This is not surprising, as over the past few years many companies have joined the RIPE NCC to secure an IPv4 /22 allocation for their business (often on the recommendation of their upstream provider).

This meant that we often had to explain quite foundational concepts and offer more guidance than usual. This has implications when we think about how we should communicate similar efforts in the future. For this project, we consciously tried to make our emails simple and direct, but at a certain point we have to assume at least some basic understanding on the part of our members.

In a few cases, it took quite some time and effort to finally get a response from a member. Their contacts in the LIR Portal were out-of-date or unresponsive, and we had to do some digging to finally get in touch with them. It’s important to keep this information up-to-date, as members can eventually be closed for unresponsiveness. Of course, this comes at the end of a very long road and, so far, no members have been closed for being uncooperative or unresponsive during our implementation.

An LIR that is not making assignments to third parties will typically use the same abuse contact for both its organisation and any resource objects. Typically, a difference here will indicate that a resource has been sub-allocated or assigned to a third-party. The response to our initial emails for this category was much better, which indicates a higher level of technical knowledge on both the part of these members (more likely to be a traditional ISP) and their customers.

A positive side-effect of this implementation was that it helped us to clean-up some outdated data from the RIPE Database. The validation emails alerted some LIRs about obsolete assignments that they wanted to remove, and others have also reviewed and updated objects related to End User assignments as well.

As a result of this implementation, we saw ~9,500 abuse contacts updated, out of the ~67,000 we checked (note that this includes duplicates, so the number of unique emails is much lower). We also saw ~50 independent resources change their sponsorship status.

Throughout this process, we received valuable feedback from our members – especially in some cases where our tool was returning false positives. Some also highlighted what seemed like an inflated sense of urgency in our reminder emails, which is something that we will take on board for the future.

We are grateful for the help and cooperation that we received from our members – in responding to our emails, getting their database information updated and offering suggestions for improvements to our process.

Ongoing Validation

We have just opened the last tickets for the initial validation of End User abuse contacts. Because we have been sending our initial emails in batches, and because our staff only become involved three weeks after sending the initial email, it is still too soon to say for sure what the response rate will be (and how long this will take). But we are happy with the progress so far.

Our implementation of the policy doesn’t end with this initial validation: we then have to make sure that the abuse contact information stays valid. So, while we have completed our initial validation of LIR organisation and resource abuse contacts, we are also validating the abuse contacts of new members, and any previously-validated contacts that have been updated since. This is a much lower workload (around ten new tickets per day) and should be manageable as an ongoing activity.

3 Comments

Jordi Palet Martinez says:
15 May, 2019 12:33 PM
Of course, the problem of this system is that *ANY* valid email will pass the validation, even if actually is from someone else and not just a real abuse contact ...

For example, a bad guy can just find the RIPE NCC contact email, and use that one.

Nobody will notice, the validation will pass.

We need something else. Stay tuned!
garbage says:
16 May, 2019 01:57 AM
Exactly, it does not establish that an abuse email address is valid or monitored, it simply establishes that an email address exists.

They could set their email address to hostmaster at ripe dot net and it would pass!
Angela Dall'Ara says:
16 May, 2019 09:55 AM
Hi Jordi,

You're correct that this wasn't in the scope of the policy proposal/our implementation. We were only asked to check that there was a working abuse contact that could receive mail. Of course, as you say, the policy can always be improved.

It's worth noting that we have the report form on our website. People can use this to alert us to incorrect information in the RIPE Database, and If a member consistently refused to maintain correct data, it could eventually be closed.

Ps. An email to hostmaster@ripe.net opens a ticket - so we are reasonably confident that we would figure this out pretty quickly ;)
Add comment

You can add a comment by filling out the form below. Comments are moderated so they won't appear immediately. If you have a RIPE NCC Access account, we would like you to log in.