Preliminary confirmation for outbound spam reputational rankings.
"When are you going to deploy the rankings?" People asked that after our previous proposal for Internet Reputation experiments for Better Security and our presentation last November at at RIPE 61 in Rome . We deployed rankings in May 2011 , as SpamRankings.net and have been adding new rankings since then, plus a new month's worth of data every month. See Table 1 and Figure 1 for example rankings. Surprisingly, we have already gotten initial tentative confirmation that these reputational rankings have effects.
Medical Rankings to Illustrate Peer Pressure
March 2011 World Medical
from CBL volume
Previous rankings of other kinds indicate the theory is sound. But does it work for organizational reputational rankings using outbound spam?
March 2011 U.S. Medical
from CBL volume
To illustrate and test the theory, we chose medical organizations among our first rankings, because they had enough spam volume to make a good initial example.
The more selective the organizational category, the less spam volume to be seen. Most countries don't show enough spam from medical organizations to rank; there's little point in a ranking of zero or one organizations. The two geographical categories that do show enough volume are the entire world, as in Table 1, and the U.S., as in Figure 1.
Changes in these rankings are interesting to follow from month to month and day to day. There is
Mid-2011 World Medical
from cbl volume
quite a bit of stability in the top of the larger rankings, and even in the medical rankings for the top three or so. However, there is also quite a bit of change over time, some attributable at least tentatively to known events; some for unknown reasons.
Even in the first of published rankings, for March 2011, we saw some interesting permutations in the rankings on a daily basis, as in Figure 3. We thought little of them at the time. It turns out medical organizations were already watching the rankings shortly after they were published.
The Big DropFigure 4 shows that Bastille Day, 14 July, was the Big Drop for medical rankings. US
March 2011 World Medical
from CBL volume
That's in rankings from CBL volume data. PSBL shows much less volume data for medical organizations, yet nonetheless the same effect. U.S. medical rankings went to near zero by 14 July, and World medical rankings went to near zero a week or so later.
There was no such effect in any rankings other than medical.
Why the Big Drop?
Figure 4: July 2011 Big Medical Drop
from CBL and PSBL volume
Did medical organizations actually clean up their act? Or did they just manage to whitelist their netblocks at CBL and PSBL?The blocklists have not given any indication of any sudden influx of delisting requests from medical organizations.
Either way, it looks like they noticed SpamRankings.net.
It turns out our first hypothesis was correct.
We have since received correspondence from a large medical organization which chooses to remain anonymous, confirming that they were indeed watching the rankings. even back in June, shortly after we first published the March, April, and May rankings. The small flurry of news articles about the rankings in June may have helped draw attention to SpamRankings.net.
In any case, our correspondent notes:
“...we had shutdown the compromised accounts before the rankings came out, since we received complaints via [redacted] and always take those seriously. The listing on your site added additional impetus to make sure we "stay clean" so in that regard, you are successful.”The rankings did not cause that medical organization to fix the problem, but the rankings do provide additional incentives, which is the point of the rankings.
Someone had previously speculated in comments on a news article by Brian Krebs that:
“I don't know enough about spam quantities to give any real meaning to 11,000 messages a month, but isn't that the kind of quantity that could come from a single machine?”That commenter was correct in this case.
However, the problems was not, as it tends to be for larger volumes, a machine infected by a botnet. Instead our anonymous correspondant tells us it was was phishing:
“...as in a recent message, directed the recipient to a compromised website belonging to a local towing company. At any rate, with the compromised credentials, an attacker then crafts an [email web server] session that logs into the compromised account and generates thousands of spam messages.”
Another commenter on the Krebs article wanted to know:
“...how much of this badness is coming from the guest wireless and how much is coming from staff/medical networks?”The answer appears to be that it was not coming from the guest wireless network.
How widespread is this problem? Our correspondant indicated at least one other medical institution had the exact same problem:
Apparently the medical organizations are looking into further technical solutions that should catch this problem even if there remain some users who didn't get the educational message, don't care, or just slipped. Remember, it can happen to anyone , even you, no matter how smart or knowledgable you are. IT security people probably can think of several possibilities for such infosec improvements. It would be inappropriate to reveal here which ones the medical organizations seem to be contemplating.
SummaryFrequent, regular, and comprehensive reputational rankings are deployed, with some specific categories and geographies, as SpamRankings.net. In addition to numerous other interesting features which may be addressed in other articles, after only a few months of publication, they have resulted in at least one tentative confirmation that they have the intended effect.
There are numerous potential extensions and public policy implications, but we will leave those to other articles.
We are continually interested in correspondance with other ranked organizations, and other interested parties. As you can see, we are careful about confidentiality.
AcknowledgmentsThis material is based upon work supported by the National Science Foundation under Grant No. 0831338. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
We also gratefully acknowledge custom data from CBL, PSBL, Fletcher Mattox and the University of Texas Computer Science Department, Quarterman Creations, Gretchen Phillips and GP Enterprise, and especially Team Cymru. None of them are responsible for anything we do, either.
John S. Quarterman for the
Andrew B. Whinston PI,
and other previous personnel.