Australia Sets New Overblocking Record: Aims For One Site, Takes Down 250,000
from the whoops dept
Overblocking is not a new problem — over two years ago, Techdirt wrote about an instance where Homeland Security took down 84,000 innocent sites at a stroke — so you might have thought that those employing this blunt instrument would take a little more care these days. However, things seem to be getting worse, not better. In Australia, the Australian Securities & Investments Commission (ASIC) has just scored a whopper:
The largest number of sites censored when attempting to block one particular site ASIC believed was defrauding Australians was 250,000. Of these, ASIC said about 1000, or 0.4 per cent, were active sites. It said the 249,000 other sites hosted “no substantive content” or offered their domain name up for sale, rather than hosting a fully-fledged active site.
I wonder how the ASIC established that 249,000 had “no substantive content”. I can’t believe it really checked all of them. And that’s the big problem with overblocking: when huge numbers of sites get taken down by mistake, there’s no way of telling what just fell off the Web, and what obscure but possibly important information is no longer available.
At least ASIC realizes that its current ham-fisted approach isn’t acceptable:
ASIC told senate estimates in its opening statement that it was now examining how it could ensure only a site’s specific domain name was blocked and ways it could alert the public to a site being blocked via a pop up page. It was also examining ways such a page could indicate why access was blocked and to whom queries could be made to dispute a block.
That’s good, but maybe it would have been better if it had explored those options before shutting down nearly a quarter of a million innocuous sites by mistake.
Follow me @glynmoody on Twitter or identi.ca, and on Google+
Filed Under: australia, copyright, overblocking, takedowns
Comments on “Australia Sets New Overblocking Record: Aims For One Site, Takes Down 250,000”
The 249K number reminds me of the number in some ads I saw about how one supermarket claimed 1.5 million customers where happy with them, when all they did was to take a sample of 200K and extrapolate the rest, that ad got banned in the UK for being misleading.
Overblocking
Preventing access to even a single website is overblocking. The proper way to shut down a website is to go through whatever legal process is due at the location in which it’s hosted.
Re: Overblocking
What if the legal process involves blocking it?
Re: Re: Overblocking
What if the legal process involved blocking anything within a 50 mile blast radius?
Re: Re: Re: Overblocking
They better make it 100 just to be sure.
Re: Re: Re:2 Overblocking
-From orbit-
Re: Re: Overblocking
Then the legal process needs fixing.
Several points
> I wonder how the ASIC established that 249,000 had “no substantive content”.
Forget substantive content. It was all just an anomaly right? Or would this be collateral damage? You can expect some collateral damage in a city if you use a 45 megaton nuclear weapon to kill an ant.
Whether content is substantive or not, people have a right to say it and not have it taken down by private interests that cannot be bothered to exercise even the smallest bit of care in their use of nuclear weapons.
From the article . . .
> This meant thousands of other sites were blocked in the process,
> as many sites are often hosted on one shared IP address.
But wait. I thought an IP address was equal to one person, not a quarter million websites!
Re: Several points
“I thought an IP address was equal to one person…”
Yes, and Mr. Prenda would like to know who that one person is. He’d like them to send him money.
Re: Several points ASIC not private interests
Just clarifying, ASIC is a government organisation that is supposed to keep businesses honest in Australia. So sites that have a purpose of defrauding the public are within their purview.
However, ASIC doesn’t have a perfect record for managing this part of their portfolio. They are not the brightest “Cube” on the block. Though that can be applied to many parts of the Federal Government here in Australia.
As one national newspaper commentator said many years ago (paraphrased). The fundamental thinking in any government department at any level of government in Australia makes the same basic assumption, “all members of the general public are convicts and must be treated as such.”
Hence, if taking down one site means that multiple other sites (100’s of 1000’s) are also affected then it really doesn’t matter as they are all problematic as well.
Duh
ASIC told senate estimates in its opening statement that it was now examining how it could ensure only a site’s specific domain name was blocked
Just, y’know, block the site’s specific domain.
Seriously, are Joeys in charge of this? Because that would at least be cute.
Re: Duh
“ASIC told senate estimates in its opening statement that it was now examining how it could ensure only a site’s specific domain name was blocked”
…and I’m just spit-balling here…but wouldn’t it be prudent to know how to do this BEFORE you start doing it?
Re: Duh
Re: Re: Duh
Yeah, we’ll probably have to teach the Joeys what a “site specific domain” means.
Re: Re: Re: Duh
Wouldn’t more specific mean further to the right. So, for example, dot-com would be more specific than, say, blogspot-dot-com.
Great job guys!
It’s so comforting that people like this who CLEARLY know what they’re doing are the ones making the decisions regarding what websites are online and what ones aren’t.
/sarcasm
Well, Techdirt usually has "no substantive content"...
“I can’t believe it really checked all of them.” — Never heard of automation, eh? Tip for you: computers can be programmed to download and analyze the text on sites. One corporation, named “Google” does this surely millions of times an hour. Any half-decent programmer could knock out an adequate script in under an hour.
But I admire the chutzpah of assuming on zero direct evidence that you’re right and the “ASIC” is wrong. You have grokked the Techdirt way. Gold star for you.
SO, with that out of the way, you’re left with a 1000 sites that MAY need to be switched or otherwise adjusted, also by a script.
But in ANY case, the cause is not from ASIC, it’s from alleged criminals, which Techdirt always tries to protect.
BUT WAIT A SEC… OMG! The one site out of hundreds of millions with THE ANSWER has been taken down! Humanity is DOOMED!
Re: Well, Techdirt usually has "no substantive content"...
“Well, Techdirt usually has “no substantive content””
You do realise that your ramblings are not the only thing on this site, right?
Re: Well, Techdirt usually has "no substantive content"...
Any half-decent programmer could knock out an adequate script in under an hour.
To crawl the sites, maybe. But to decide what is “substantive”, let alone to decide what qualifies as “substantive” is hardly so trivial.
Re: Re: Well, Techdirt usually has "no substantive content"...
You’ve got to remember, he’s paided by Hollywood, so ‘substantive’ is defined as bulk volume of number and/or size. A $200 Million dollar movie with no plot is better than a $20 Million dollar movie that wins awards, for example.
Re: Well, Techdirt usually has "no substantive content"...
“… the cause is not from ASIC, it’s from alleged criminals…”
Ahh, our old friend, “alleged”. What if it were possible to, say, suspend someone’s ISP account… not saying who… for being an “alleged asshat”?
Guess you’re right… “alleged” can be useful after all.
Re: Well, Techdirt usually has "no substantive content"...
Who gives a shit about automation? What the hell does THAT have to do with the issue?
Why not focus on the REAL problem – one site needed to come down and they instead took down 250,000!
Great job missing the point AGAIN, retard. Funny – I don’t think you’d see it so trivially if it was YOUR site.
Re: Re: Well, Techdirt usually has "no substantive content"...
Re: Well, Techdirt usually has "no substantive content"...
So what about the 1,000 or so that DID have substantive content?
Tough shit?
And the cause IS from ASIC, moron. They initiated the order that took them down (or did you miss that when you “read” the article?).
I just don’t understand your way of “thinking” blue. How exactly can you defend gross incompetence of this magnitude over and over again? Or is 250,000 sites just an anomaly?
Re: Well, Techdirt usually has "no substantive content"...
A two line site saying child protection, phone 12345678910 is substantive. A site that collected all your whining and ad-homs would not be, even if it had 10,000 pages crammed full of text.
Re: Re: Well, Techdirt usually has "no substantive content"...
(Hit return too soon).
I should also add that blocking your site, if you had one, would still be objectionable as censorship. You have the right to free speech, even if you add nothing to discussions.
Re: Well, Techdirt usually has "no substantive content"...
This actually made me laugh out loud.
The best programmer in the world couldn’t create a system to do this in an adequate manner in a month, let alone an hour, unless you’re talking about downloading a small number of sites and your idea of “analysis” consists of nothing more complicated than scanning for keywords.
Re: Well, Techdirt usually has "no substantive content"...
If Techdirt really has no substantive content, Blue, why are you here every day, rubbing your ass all over it like a dog with worms?
Read this: http://en.wikipedia.org/wiki/The_Trial
Then this: http://en.wikipedia.org/wiki/Due_process
250,000 ≠ 1. Go back to school, troll.
Wait, if you can’t even count to one, how in the world are you going to be able to read those linked articles? Does your mommy type your posts? :S
Re: Well, Techdirt usually has "no substantive content"...
One word about OOTB – laughable. What’s it like to be a laughing stock, Blue?
ASIC
Why the surprise?
ASIC have always been synonymous with single-mindedness and applying that inflexibility quickly and efficiently to large numbers in parallel!
Pop up page?
What about when the website uses https, with the modern certificate pining stuff? The user would get an unskippable certificate error.
And with SNI, which is already usable for everyone except very old browsers, you can have thousands of https sites on a single IP.
With the IPv4 address crunch, many https sites on a single IP will become more common, and https is becoming more and more popular. So this pop up page idea gets even harder as time passes.
This particular task isn't that hard
I wonder how the ASIC established that 249,000 had “no substantive content”. I can’t believe it really checked all of them.
Why not? If you know how to use curl or wget combined with a little perl or python it’s fairly easy to discern which web sites aren’t really web site, but single-page placeholders being used by domaineers. All those are cookie-cuttered: they have the same markup, the same layout, (nearly) the same content, (nearly) the same links, etc. The only reason they exist is that these scammers are hoping someone will typo a URL and land there, generating a fraction of a cent’s worth of income.
The computing resources needed to do this are minimal: with a decent laptop and a modestly-fast connection, this can be done overnight.
Re: This particular task isn't that hard
The problem with that is that you can’t tell by the web site.
For example, I have a personal domain name that I use for email and other private services. There is no website attached to it at all — if you browse to my domain name, you’ll get the placeholder page you’re talking about.
This domain is heavily used every day and critical to a reasonable number of people. It would have been blocked as having “no substantive content” anyway.
The web != the internet.
Re: Re: This particular task isn't that hard
Yes we understand that John. However, what is the chance of all 250,000 domains on a single IP being exactly the same, having absolute no content? Powerball odds come to mind here.
Your argument would only hold water in a single or very small number of domains. 250,000? Better chance that you get hit by lightning while partying with the Yankees – in a subway station.
Re: Re: Re: This particular task isn't that hard
I’m not sure how that’s relevant.
I wonder how the ASIC established that 249,000 had "no substantive content"
I wonder how the ASIC established that 249,000 had “no substantive content”
Oh, i don’t know… maybe by looking at access logs? Maybe by looking at how little traffic was actually coming in?
Actually, in this case it looks like a parking page site, because all 250,000 were apparently on a single IP address or had DNS from the domain in question.
Perhaps rather than just repeating what the original writer put up, you might want to actually put some effort into understanding. Your blind outrage is amusing, by sort of stupid.
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
horse with no brain wrote:
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
So, if 250,000 Music albumns or movies were removed from the major studios when attempting to remove 1, then that would be fine?
250,000 books removed from a publisher when attempting to remove 1, that would be fine?
Collateral damage is fine when it is not your industry on the receiving end?
Wanker
Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
Way to miss the point. 250,000 blank websites lost is not a loss, except perhaps if your thrill in life is surfing parking pages and “coming soon” websites.
Is there any true collateral damage here?
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
What was the standard used to determine if there was substantive content on the site?
Who detrmines substantive content? the media industry?
If it is not content from the legacy entertainment industry then it does not count as substantive content?
You would think that it is collateral damage if one of those 250,000 sites was yours.
This is the definition of ‘Intellectual Property THEFT’, not Intellectual property infringement that copyright maximalists like to conflate with theft
Re: Re: Re:2 I wonder how the ASIC established that 249,000 had "no substantive content"
What was the standard used to determine if there was substantive content on the site?
I don’t know – maybe they were all parking pages with the same code? maybe they were all single pages? Maybe they were all identical except the name?
Do we know if it was 250,000 domain names, or just 250,000 third level domains of the same site?
I think you might want to cool your jets until you know more, because you are certainly going off the handle with limited working material here.
Re: Re: Re:3 I wonder how the ASIC established that 249,000 had "no substantive content"
so you were just making up facts when you stated that it is 250,000 blank websites?
Way to miss the point. 250,000 blank websites lost is not a loss
How can you state that it is not a loss if you do not even know what was removed?
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
You know damn well you’d call it collateral damage if it was your site being wrongly taken down.
Re: Re: Re: I wonder how the ASIC established that 249,000 had "no substantive content"
See this post about a domain being used for email.
Re: I wonder how the ASIC established that 249,000 had "no substantive content"
Having worked with these type of people before, I would guess that the whole process was initiated by some non-technical person who didn’t bother to get the necessary facts before issuing the order.
In the aftermath of the problem, there would have been an analysis of the pages taken down by personal view (looking at maybe five or six sites). This analysis would then be used to justify what has happened and to colour the view.
Anybody with any technical expertise would very likely not have got their input past along – technical people are just too freaking dangerous to allow their opinions or views to be passed up the chain of reporting. They tend to highlight problems with the various approaches being considered.
Robert Peasley, the ASIC intern assigned to visit each and every one of those 249,000 sites, reported “they contained nothing but cat pictures.” Peasley is currently “resting comfortably in a cat-free institution.”
But for real humor, there’s this:
This quote is from an article abstract. The punchline? The article was written in 2003. How long ago was that in internet years?
Re: Re:
Shared servers. My website is hosted on one of these. That is not and never will be a good enough excuse to take a website down.
And when you’re in the process of building a website, i.e. adding substantive content, it’s common to put up a Coming Soon page.
Go Daddy provides a revenue stream for people who buy up domains by letting them use them for advertising. And Go Daddy offers a shared hosting service.
NOW can you see what the problems are, Horse’s Ass with no brain?
i read where all of this sort of thing has to be OK’d by Gillard first. it seems as though she’s not a lot of good at a lot of things, and sorting out the ‘net’ is one of them
Re: Prime Minister Gillard to you
Please have some respect for the function of Prime Minister. It is a hallowed position in the governance of Australia.
She is a very intelligent woman. She was able to shaft the previous Prime Minister quite easily without having suffered the same consequences herself. She is leading Australia back to the greatness that was called “The Dark Ages” with her various policies for national development.
Security is a high priority for her and her government that they will use whatever means possible to keep all of the people safe by watching their every action.
She follows in the footsteps of those great previous Prime Ministers Robert Hawke and Paul Keating. Men of such renown that the first declared that there would be no children living in poverty in Australia by the year 2000 and the second declared that Australia was a “banana republic”.
So please respect her and the position she holds by addressing her correctly as Prime Minister Gillard.
horse with no name loathes it when due process is enforced.