ASN Ranking Correlations Between Spam Blocklists

CBL and PSBL

In addition to CBL ( Composite Blocking List ), we also have volume data from PSBL ( Passive Spam Block List ), which we can compare, as in Table 1 below. 

As Table 1 shows, 6 out of 10 ASNs are the same in this comparison of CBL and PSBL data for all ASNs worldwide for one month from 8 Sep 2010 to 7 Oct 2010. PSBL's spam traps and detection algorithms are different from CBL's, yet there is quite a bit of agreement.

CBL and UTCS

Table 2 shows 4 out of 10 ASNs are the same CBL and UTCS for these 30 days. That seems like a small number, but universities get more spam from universities, for example from FIVE-COLLEGES-AS and the University of Utah in this example.

Nonetheless, not only are four ASNs the same between CBL and UTCS, three of them are in the PSBL top 10 as well: VNPT-AS-VN, BSNL-NIB, and SAUDINETSTC-AS. Further, two of them, VNPT-AS-VN and BSNL-NIB, are in the top 4 from all three data sources. This amount of correlation may be one basis for not just rankings but also for certification classes.

ARIN in CBL and PSBL

Table 3 shows 8 out of 10 ASNs are the same for ARIN between CBL and PSBL in this period. Comcast places 4 of the top 10 ASNs in rankings from both data sources. Charter, Shaw, AT&T, and Windstream also place in both. Verizon only shows up in the top 10 for CBL. With this much agreement in rankings from such disparate data sources, we have some confidence in the results.

These rankings raise all sorts of interesting questions, such as:

• What about rankings by organization, grouping ASNs by owners first?
• What about rankings normalized somehow, for example by size of address space?

Country Comparisons

BE in CBL and PSBL

Table 4 shows 9 out of 10 ASNs are the same for BE between CBL and PSBL for October 2010. The top 2 are even in the same order.

NL in CBL and PSBL

Table 5 shows 8 out of 10 ASNs are the same for the Netherlands, and the top 3 are in the same order.

RIPE NCC Oct 2010

Table 6 shows 6 ASNs registered by RIPE NCC rank in the top 10 for spam volume by both CBL and PSBL for October 2010.

Summary

For countries like Belgium and the Netherlands, or for the RIPE NCC or ARIN regions, few ASNs cannot be the reason for such close correlations. ASNs that show up at the top in rankings from both sources probably really are sending more spam than other ASNs.

We would be very puzzled if rankings from the CBL and PSBL data did not agree at all. If they agreed 100%, we would also be surprised in the sense that there would be no information added by a second list. Our observations show that they overlap enough to establish confidence in the rankings. They disagree some, which means that there might be some value in looking into multiple lists. At this time we cannot interpret this, but if we watch them over time, we may be able to build some intuition regarding this situation.

As we mentioned in the talk at RIPE 61, Transparency as Incentive for Internet Security: Organizational Layers for Reputation , extreme precision is not the point of the rankings. Reasonable accuracy and confidence is sufficient. This is especially true for building certifications out of rankings over time.

In the other direction, we have daily data, and can show relative changes in addition to total volumes, as in Figure 1 below.

Figure 1: Most Spam Volume from ASNs registered by RIPE NCC, October 2010
Volume (spam message counts) derived by the IIAR project from custom CBL blocklist data.

Other rankings may draw on how quickly rankings of a given ASN change, or on how much spam from a given botnet the ASN produces.

Meanwhile, the correlations between these spam volume sources provide sufficient confidence to start prototyping rankings.

Acknowledgments

We also gratefully acknowledge custom data from CBL, PSBL, Fletcher Mattox and the University of Texas Computer Science Department, Quarterman Creations, Gretchen Phillips and GP Enterprise, and especially Team Cymru. None of them are responsible for anything we do, either.

John S. Quarterman for the IIAR project, Andrew B. Whinston PI, Serpil Sayin, Eshwaran Vijaya Kumar, Jouni Reinikainen, Joni Ahlroth, and other previous personnel.
antispam@quarterman.com

Previous RIPE Labs Articles:

• Economic Incentives for Cooperation to Fight Spam
• Internet Cloud Layers for Economic Incentives for Internet Security
• Internet Reputation Experiments for Better Security
• RIPE 61 talk, Transparency as Incentive for Internet Security: Organizational Layers for Reputation