You're kidding, right? You're asking me to demonstrate to you that the ocean is wet. But presuming this is a serious question:
If you run a mail server of any size/volume/scope/etc., then all you have to do to answer that question is to look at your own logs. (By which I mean not just your SMTP logs, but everything else as well.)
If you don't, and not everyone does of course, then all you have to do is to read the relevant traffic on nanog, mailop, spam-l, full-disclosure, bugtraq, spamtools, and other related lists for the last ten years or so.
Either way, what you need to be paying attention to is not so much what's going into Hotmail (although that's certainly interesting in its own right) but what's coming out of it.
...should be required to pass an examination in mathematics, including geometry, trigonometry, algebra, basic calculus and statistics. We certainly would not (and should not) allow someone who is illiterate in the relevant language(s) to hold office; neither should we permit anyone who's mathematically illiterate to do so.
(Yes, I'm well aware that this is not a panacea, but wouldn't it be nice to actually have a senator who could multiply? Or a DA who knew what the bell curve is? And how refreshing would it be to have ANY elected official who understood the proper way to extrapolate an exponential?
"The greatest shortcoming of the human race is man's inability to understand the exponential function."--- Albert A. Bartlett
Of course. It is quite, quite impossible to secure Windows systems for any meaningful value of "secure". Witness Microsoft itself, which has (a) the source code (b) massive in-house expertise (c) essentially-infinite personnel (d) essentially-infinite money and yet still cannot manage to run a mere email service (Hotmail) securely. Anyone who is actually paying attention to their own mail servers knows this; their own log files prove it repeatedly, all day, every day, and have been doing so for years.
But I wouldn't start with Linux for this purpose. Oh, it's certainly an enormous step up from Windows, but then again a steaming pile of cow manure would be the same. I'd start with OpenBSD, which is considerably smaller and much more focused on security.
None of this will happen, of course. Instead, those responsible for this will be rewarded and promoted, there will be more of the same epic failure, and even the poorest countries out there (the ones without a beer and an airline, thanks FZ) will be able to penetrate this operation whenever they want, merely by hiring a bored college student with a laptop.
Well, yes, it's obvious and pathetic...and it works.
See for example: http://techcrunch.com/2009/07/19/the-anatomy-of-the-twitter-attack/ for a case study. Approaches like this are frequently succesful because nearly all organizations presume that their users are secure...and they're not.
So it didn't work with you: are you certain it won't work on the idiot three offices down, you know, the one who clicks on everything shiny?
You make some (partially) good points, but unfortunately you paint a far better picture than reality and decades of spam history support. Let me -- briefly -- address each of these.
1. Windows is still Windows -- a markedly inferior operating system both in design and implementation. (As are the associated applications.) Windows 7 may be an improvement, and it appears to be so in some aspects, but it is still miserably insecure and routinely breached. It is worth noting that not even Microsoft, who wrote it and has essentially infinite personnel and financial resources, has been able to do so.
2. Anti-virus software is worthless pablum. It's a greedy attempt to exploit the people who purchase and operate inferior operating systems. Those who choose their operating systems properly do not NEED anti-virus because they run operating systems which are of sufficient quality. Those who choose their operating systems poorly are deluding themselves if they think anti-virus will save them.
3. The shift to webmail has diminished the use of some propagation vectors; it's brought new ones. Surely you are aware that some major webmail providers, such as Yahoo and Hotmail, are riddled with security holes, are utterly incompetent at stopping inbound or outbound spam, phishing, and exploits?
4. Yes, some browers are better, although IE is still a worthless piece of shit that nobody should use. What has also helped is the deployment of browser add-ons such as NoScript. If we were really serious about security those wouldn't be add-ons but would be built-in to every browser. Of course the idiot web designers who cook up sites whose home pages won't even display without scripting would be appalled, but they are disposable.
5. The shutdown of a botnet is of no importance whatsoever. It merely provides an opportunity for Microsoft et.al. to beat their chest and lie about what a great victory they've achieved...when in fact they've accomplished nothing. Of course all the zombies that are part of that botnet are still fully-compromised, they still have the security holes, they still have the same careless, clueless users, they still have the same inferior operating system, they still have the same crappy applications. And so, soon enough, they will be part of another botnet.
As to the connections into my mail servers, if you will review the archives of spam-l from the better part of a decade ago, you'll find that I was one of the (several) people using not only connecting DNS/rDNS information, but passive OS fingerprinting to identify their OS. I'm not only familiar with these techniques, I invented, or co-invented, some of them. I am (painfully) well aware of what the landscape out there looks like, and thus equally-painfully well aware that the situation continues to get worse.
And therefore I dismiss, with prejudice, any suggestion that the number of zombies is getting smaller. Not only is there absolutely no reason for that to happen, but (close to) a decade's worth of data says precisely the opposite.
Typo in the last paragraph: s/1995/2005. Mea culpa.
I'll sharply disagree with you on the number of zombies worldwide, but this is of course just a question of your estimate vs. mine vs. every else's. The true number is not only unknown, but unknowable, since (a) any zombie that does nothing to make its presence known will likely remain undetected indefinitely (b) even if it DOES do something, if that something isn't noticed, it'll still remain undetected indefinitely (c) the "somethings" are getting more subtle and harder to recognize all the time and (d) we've known for years that large numbers of them are kept in reserve.
That said, three of us with significant experience in the field came up with a consensus estimate several years ago based on very large-scale observations; at that point, we concurred that 100M was in the ballpark. In the intervening time, nothing has happened to indicate any decrease in that number, and a lot of things have happened to indicate a significant increase. (Consider: it is now fairly routine to hear reports of busts of botnets with 10M members. Surely those are not unique, and surely they're not the largest, and surely they're not the most competently-operated...since the latter won't be busted.)
Note that shortly after we made our estimate, Vint Cerf of Google made his: http://arstechnica.com/old/content/2007/01/8707.ars cites him as estimating that 150M out of 600M connected systems were members of botnets. That's 4.5 years ago, and in the interim, we've connected a LOT of new systems to the 'net, including smartphones, tablets, etc. So my view (as of summer 2011) is that any estimate under 100M is ludicrous and may be instantly discarded. Higher estimates vary, of course, but I think 200M is certainly "plausible", but would not reject (let's say) 150M or 300M, both of which are also possible.
And I think -- whatever the number is -- it's important to note that it's monotonically increasing, and that it will apparently continue to monotonically increase because nothing has been done to make it do something else. So if there are -- for the sake of argument -- 183M zombies today, then a year from now there will be 184M or 201M or 193M -- not 170M.
All that said, it's not clear to me who, in a civil case, is liable. For example, I would say "Microsoft, for shipping an operating system that's pre-compromised at the factory and unfit for any purpose". Others would blame the lack of anti-virus software, others would blame poor computer skills/habits, etc. I think there's a lot of debate here, both on a technical level and on a legal level. But I do think that plaintiff should be required to establish at least a reasonable likelihood that defendant did X, Y and Z -- and I think that in the present environment, even clinching, forensically-verified proof that the defendant's computer did X, Y and Z doesn't mean the defendant did it.
(Let me toss in an aside that many forensic examinations are awful. Note that last week here on TD we were reading about how the FBI was baffled by a dual-boot system. And remember the Julie Amero case, also discussed here repeatedly? That poor woman had her life destroyed by malicious prosecutors, ignorant police, and incompetent IT people.)
Where I will agree with you is that the statistical distribution of zombies has changed over the years. Botnet operators of course prefer control over systems with substantial CPU/memory/disk, and with adequate bandwidth. As the Internet has become more prevalent around the world, and as computer costs have dropped, there are more systems with more bandwidth available in more places. So whereas a 1995 botnet might have had most of its members in the US, Australia, and western Europe, today's botnets may have members in Vietnam, Peru, Romania, and Egypt. This may well overlap with bootlegged/unpatched software, but there's no way (that I know of) to measure that correlation based on what's in the packets.
To add to that: many times the ISPs themselves supply bloatware (or install on users' computers) as part of the ISP provisioning process. That software represents a significant downgrade of the security of the user's system.
So who should be held responsible for that? The user, because they signed up with Comcast for service, or Comcast, because they dropped a load of crap on the user's system, opening it up to all kinds of exploits that weren't previously available?
As I've noted here repeatedly over the years, there are a couple hundred million zombies plugged into the Internet. (Others have cited higher estimates, and they may well be right, but I think 200M is a reasonable minimum number.)
NOTHING that those systems do can be definitively laid at the feet of their (former) (putative) owners, because those computers really do not belong to the people on whose desks or laps they rest. Not any more. Those computers belong to their new owners, who are the botmasters controlling them.
And the new owners use them to phish, to harvest email addresses, to spam, to conduct DoS attacks, to host illicit content, to crack passwords, to do anything they want.
And there is absolutely no way for an external observer, watching the traffic in/out of that system, to discern which of it is associated with the old owner and which with the new. Astute guesses can be made: rolex spam is probably from the botmaster, POP requests to the relevant ISPs mail server the former owner. But these are still just guesses, and thus should not be admitted into evidence. And -- as botnet operators have become more crafty, they've also got much better at hiding their handiwork in "normal" traffic.
The only way to associate traffic with a user is to watch them type it/click it. (And even that is starting to become suspect as a methodology -- like I said, botnet operators have gotten crafty.)
TL;DR: there is a profound disconnect between "what X's computer did" and "what X did".
That last sentence is precisely right: anything, anywhere, anytime on the Internet that requires "opt-out" is abusive.
And those responsible know this is true, which is why they're sneakily forcing it down users' throats: if what they had was truly good and truly desired, they wouldn't have to do that. They KNOW that what they're doing (which is monetizing NXDOMAIN, an inherently dishonest and fraudulent act) is wrong, they KNOW that people don't want it, they KNOW that it breaks things...but because they're greedy assholes, they're going to try to do it anyway.
...whereas some of us have been running them for decades.
There is no doubt that an excess of messages is abusive, but "abusive" is not the same as "hacking". "Abusive" (in the context of email) encompasses the sometimes-overlapping categories of mailbombing, forgery, spam, DoS attacks against SMTP, rapid-retry, etc. None of these qualify as hacking. Oh, they're all reprehensible, like many other things that aren't hacking either, but that doesn't make them what the court imagines them to be.
Moreover, a read through this indicates that the company's own profound incompetence is largely responsible for its troubles. It is a trivial matter for any minimally-clueful mail system administrator to deal with issues like this -- many of us deal with them on a routine basis. Sometimes they're the result of malicious action; sometimes they're the result of somebody else's screwup; sometimes they're the result of a well-meaning but poorly conceived campaign, as appears to be the case here. But whatever the cause, dealing with the results is very easy, so much so that I think it reasonable to presume any competent mail system administrator would be ready for this and would only need to flick the switch, so to speak, to deal with the issue.
Of course one of the obvious, fundamental errors made by the mail system administrators shows up as early as page 3 of this ruling, where it states that the mail system "limits the number of emails in an inbox". In a time when 2T drives cost much less than an hour of system admin time, that's not just stupid, it's set-yourself-on-fire stupid. While there is a reasonable argument to be made that the very largest email providers (e.g., gmail) may need count/size quotas, there is no such argument to made for the overwhelming majority of mail operations. (Yes, I'm well aware there are outliers. I've run some of them.) It is vastly more efficient, cost-effective, secure (mail quotas facilitate DoS attacks), and simple to add storage in almost every case.
The correct response from Pulte Homes is not to pursue this in court, but to fire their mail system admin(s) on the spot and replace them with individuals who possess at least minimal competence in the field.
...is that there's no such thing as an "ex-spammer". Zero recorded cases, ever. Certainly there are many *claims* of such cases, almost all of which come from (surprise) spammers or their supporters/enablers/partners/suppliers. But those claims evaporate when examined closely enough. (Or when we wait a few months or years -- sometimes they do take a break.)
Oh, and incidentally, Internet "old-timers" find Spamford and his ilk a relatively new problem.
Check the precise bid amounts. That should be a clue that Google was making a snide comment about the bogosity of this.
...might be the best contribution Google could make. But, as you observe, it would be a massive uphill fight against the stupid, the myopic, and the greedy. I'm sure they know that; but I'm sure they also know that they are among the few with the resources to tackle the issue.
(This was in response to Violet Blue's column on the topic.)
Everything you said is true. But it's worse than that: they're building
a target. Or, rather, many targets. Let me explain.
Personal information has value: to advertisers, to marketers, to spammers,
to phishers, to actual real live pedophiles, to disgruntled ex-boyfriends,
to insurance companies, to extortionists, to all kinds of people.
And let me pause to interject: it's no comfort at all to hear that
it's "not personally identifiable". As recent research has shown us,
pointedly, when enough disparate data sources are combined, that becomes
wishful thinking.
So there are people who have uses for this data...and are willing to pay
for it. Therefore there WILL be a market for it, just like there are
markets for everything else illicit on the 'net, e.g., custom spamming
software.
And since there will be a market for it, there will be buyers...and
sellers.
Some of the sellers will be crooked ISPs who are willing to sell their
own users out to anyone with cash-in-hand. (My bet: Comcast and
Verizon will fall all over themselves to do this.) But some of them
will be ISP employees, who will have access to it and will be more than
happy to exchange a USB stick or ten, stuffed with compressed log files,
for an envelope of tax-free income.
Then there will be a secondary market: crafty people who are willing
to buy data from a few dozen sources and combine, correlate, reduce,
filter, enhance it -- and then sell that composite product. (If I have
the logs that indicate what DNS queries you've run, then I can make
good guesses at what web sites you visit...or perhaps have logins on...
your email provider, your social network, your IM accounts, etc. I can
then search those, one at a time or via Google. The more I know about
you, the more I *can* know.) And of course these same crafty people
know all about credit cards -- so they'll be able to produce individual
dossiers that make it very easy to perform competent identity theft.
And since (putatively) we're talking about pedophiles here: think of
the possibilities for them.
This idiotic bill puts ISPs in the position of building targets:
big stationary highly attractive targets that everyone will *know*
they have.
And let me interject once again: there's no reason at all to be reassured
by ANYONE'S claim that they'll be kept "securely". LulzSec/AntiSec have
been pulling the shorts of one government security contractor after
another over their heads for months, and they're not even trying hard.
Determined adversaries will go right through whatever inept "security"
is put in place around this.
So here's what'll happen: the information will be collected. Some of
it will be collected incorrectly, people will get doors kicked down
because a network monitoring script mangled an IP address. Some of
it will be sold by ISPs, some by ISP employees. Insurance companies
will cancel policies, abused wives will be stalked by crazy ex-husbands,
pedophiles will select targets, etc. Big chunks of data will be bought
and sold at places like the Russian Business Network (which, by the way,
is not as gone as people wish it were). The end result will be a privacy
and security nightmare for everyone...and it will increase, not decrease,
the risks to the children that it supposedly protects.
Oh, and the politicians responsible will pat themselves on the back
and take credit for it. And when it all goes wrong...they will use
that time-honored phrase of spokesliars everywhere:
"...and nobody could have foreseen..."
...so Big Content gives them drivel.
For every one of these intelligence-insulting feature movies, there are 10 low-budget films that blow it away. For every insanely vapid TV show, there's 10 more interesting things online. For every shallow untalented hack of a pop star, there are 10 wonderful musicians playing in your town or posting their work online.
The trick is to wean yourself from mass-produced "culture" and seek out the individual efforts: the visions and craft and creativity of people who are making things not because they think they'll make $500M, but because they're artists and THAT'S WHAT THEY DO. Thanks to them, there are small bits of genius to be found almost everywhere -- and once in a while, a masterpiece.
Hollywood is obsolete. The TV networks are obsolete. The record companies are obsolete. We don't need them any more.
(And of course this is why they're frightened out of their minds: they KNOW this is true.)
...I do hope he publishes it on toilet paper.
The benefit is profit -- in some cases, lots of it, far more than what any phone company makes from a single voice line. Abusers, because they are frequently detected and blocked, constantly need new domains, new DNS, new routing, new web hosting, new email service, etc.; this makes them repeat customers and means that they provide a steady flow of income. [Some] ISPs are reluctant to give that up and will argue that what they're doing isn't illegal...and in some cases they may be right, despite being abusive, it's NOT illegal.
One of the things we've learned over the past few decades of fighting spam, and more generally, abuse, is that legal methods DO NOT WORK.
That is not to advocate illegal methods, of course; it's to point out that the legal system is the wrong place to address the problem, because it's clueless, outdated, slow, inept, local (whereas the problem is global) and in some cases, effectively owned by the abusers.
But beyond all that: we already *have* quite effective means at our disposal for applying pressure on ISPs who, let's say, host gangs of spammers. The problem is not the lack of these methods or their effectiveness; the problem is our unwillingness to use them, particularly our unwillingness to use them when they cause (or appear to cause) issues for our own operations. This problem persists despite the escalating seriousness of the issue -- which, as I've said elsewhere, I fault *us* for. Had we acted more effectively much sooner, there wouldn't be an entire ecosystem of abuse to contend with now.
Re: Re: Re: Re: Re: *facepalm!*
There are many things wrong with your statements, some of which are obvious and some of which are subtle. I'm just going to cover the major points, and refer you to the resources already outlined for more extensive coverage.
First: what comes out of any site has everything to do with whether or not it's secure. This is a first principle of network security, albeit one that is often overlooked. (Haven't you noticed that the most serious security issues don't involve someone breaking in...they involve someone breaking out?)
Second: one of the other fundamental principles is that outbound abuse is a surface-level indicator of underlying problems. Spam is of course not the only form of abuse -- it's merely one of many that uses the SMTP protocol -- but it does provide a highly reliable measure of internal security. Secure sites do not emit spam on a systemic and persistent basis. (Nor do they emit other forms of abuse on a systemic and persistent basis.)
Third: everyone who knows how to read email headers and/or evaluate their own logs is quite aware of what's really coming from Hotmail and what's not. Attempts to forge Hotmail's domain have decreased steadily, in part because even spammers, dull as some of them can be, have figured out that it's not a worthy target for forgery.
Fourth: spam is far from the only security problem at Hotmail. Again, read the references I cited, or, if you don't want to plow through the historical record, just subscribe to them...and wait. You probably won't have to wait long.