The Big Medical Drop in SpamRankings.net

Preliminary confirmation for outbound spam reputational rankings.

"When are you going to deploy the rankings?" People asked that after our previous proposal for Internet Reputation experiments for Better Security and our presentation last November at at RIPE 61 in Rome . We deployed rankings in May 2011 , as SpamRankings.net and have been adding new rankings since then, plus a new month's worth of data every month. See Table 1 and Figure 1 for example rankings. Surprisingly, we have already gotten initial tentative confirmation that these reputational rankings have effects.

Medical Rankings to Illustrate Peer Pressure


1	AS 9208 WIN	BE
2	AS 22328 CSHS	US
3	AS 38668 KONKUKHOSPITAL-AS-KR	KR
4	AS 26199 NKCHA	US
5	AS 23235 BOSMED-88-ENEWTON	US
6	AS 20252 JSIWMC	US
Table 1: March 2011 World Medical from CBL volume

The theory behind SpamRankings.net is that comparing similar organizations produces peer pressure, because organizations, like people, care how they look compared to the competition. We draw upon fifty years of social comparison theory, thirty years of commons governance theory, and a decade of online individual and organizational comparisons, which indicate that publicly comparing similar organizations can change their behavior through peer (and customer) pressure. See Internet Reputation Experiments for Better Security for references and discussion.

Previous rankings of other kinds indicate the theory is sound. But does it work for organizational reputational rankings using outbound spam?

Figure 1: March 2011 U.S. Medical from CBL volume

To illustrate and test the theory, we chose medical organizations among our first rankings, because they had enough spam volume to make a good initial example.

The more selective the organizational category, the less spam volume to be seen. Most countries don't show enough spam from medical organizations to rank; there's little point in a ranking of zero or one organizations. The two geographical categories that do show enough volume are the entire world, as in Table 1, and the U.S., as in Figure 1.

Changes in these rankings are interesting to follow from month to month and day to day. There is

Figure 2: Mid-2011 World Medical from cbl volume

quite a bit of stability in the top of the larger rankings, and even in the medical rankings for the top three or so. However, there is also quite a bit of change over time, some attributable at least tentatively to known events; some for unknown reasons.

Even in the first of published rankings, for March 2011, we saw some interesting permutations in the rankings on a daily basis, as in Figure 3. We thought little of them at the time. It turns out medical organizations were already watching the rankings shortly after they were published.

The Big Drop

Figure 4 shows that Bastille Day, 14 July, was the Big Drop for

medical rankings.

Figure 3: March 2011 World Medical from CBL volume

medical rankings all went to zero. Between 17 and 24 July,

World medical rankings went from hundreds and thousands to near zero.

That's in rankings from CBL volume data. PSBL shows much less volume data for medical organizations, yet nonetheless the same effect. U.S. medical rankings went to near zero by 14 July, and World medical rankings went to near zero a week or so later.

There was no such effect in any rankings other than medical.

Why the Big Drop?

	CBL	PSBL
U.S
World
Figure 4: July 2011 Big Medical Drop from CBL and PSBL volume

When we discovered this sudden drop to near zero in all medical rankings, we wondered:

Did medical organizations actually clean up their act? Or did they just manage to whitelist their netblocks at CBL and PSBL?
Either way, it looks like they noticed SpamRankings.net.

The blocklists have not given any indication of any sudden influx of delisting requests from medical organizations.

It turns out our first hypothesis was correct.

We have since received correspondence from a large medical organization which chooses to remain anonymous, confirming that they were indeed watching the rankings. even back in June, shortly after we first published the March, April, and May rankings. The small flurry of news articles about the rankings in June may have helped draw attention to SpamRankings.net.

In any case, our correspondent notes:

“...we had shutdown the compromised accounts before the rankings came out, since we received complaints via [redacted] and always take those seriously. The listing on your site added additional impetus to make sure we "stay clean" so in that regard, you are successful.”

The rankings did not cause that medical organization to fix the problem, but the rankings do provide additional incentives, which is the point of the rankings.

Someone had previously speculated in comments on a news article by Brian Krebs that:

“I don't know enough about spam quantities to give any real meaning to 11,000 messages a month, but isn't that the kind of quantity that could come from a single machine?”

That commenter was correct in this case.

However, the problems was not, as it tends to be for larger volumes, a machine infected by a botnet. Instead our anonymous correspondant tells us it was was phishing:

“...as in a recent message, directed the recipient to a compromised website belonging to a local towing company. At any rate, with the compromised credentials, an attacker then crafts an [email web server] session that logs into the compromised account and generates thousands of spam messages.”

Another commenter on the Krebs article wanted to know:

“...how much of this badness is coming from the guest wireless and how much is coming from staff/medical networks?”

The answer appears to be that it was not coming from the guest wireless network.

How widespread is this problem? Our correspondant indicated at least one other medical institution had the exact same problem:

“poor security in terms of user awareness education”

Just about every organization on the Internet has that problem, so that's not surprising.

Apparently the medical organizations are looking into further technical solutions that should catch this problem even if there remain some users who didn't get the educational message, don't care, or just slipped. Remember, it can happen to anyone , even you, no matter how smart or knowledgable you are. IT security people probably can think of several possibilities for such infosec improvements. It would be inappropriate to reveal here which ones the medical organizations seem to be contemplating.

Summary

Frequent, regular, and comprehensive reputational rankings are deployed, with some specific categories and geographies, as SpamRankings.net. In addition to numerous other interesting features which may be addressed in other articles, after only a few months of publication, they have resulted in at least one tentative confirmation that they have the intended effect.

There are numerous potential extensions and public policy implications, but we will leave those to other articles.

We are continually interested in correspondance with other ranked organizations, and other interested parties. As you can see, we are careful about confidentiality.

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grant No. 0831338. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

We also gratefully acknowledge custom data from CBL, PSBL, Fletcher Mattox and the University of Texas Computer Science Department, Quarterman Creations, Gretchen Phillips and GP Enterprise, and especially Team Cymru. None of them are responsible for anything we do, either.

John S. Quarterman for the IIAR project, Andrew B. Whinston PI, Serpil Sayin, Jouni Reinikainen, and other previous personnel.
antispam@quarterman.com

The Big Medical Drop in SpamRankings.net

John S. Quarterman