Australian Census Data Released Under CC License, But Official Site Tries To Make It Hard To Download
from the once-free,-always-free dept
The whole point about adopting Creative Commons licenses is to make it easier for people to share and use works released under them. Sometimes, though, you get the impression that certain organizations adopting these licenses would rather that didn’t happen, as in the following case from Australia, reported by IT News:
The Australian Bureau of Statistics has released the latest census data for free under a Creative Commons license but appears to be steering people towards a $250 mailed out DVD rather than making it easy to download the information directly over the internet.
Programmer and freelance journalist Grahame Bowland who first noticed it, said the government agency is going to great lengths to discourage people from downloading the files directly by dint of a convoluted site layout and Javascript functions that obfuscate file paths.
The post then goes on to describe in detail some of the attempts to make it difficult to download all of the census data, including a hard-to-find registration page, a complex matrix of download options, and Javascript code that does stuff like this:
// Function: guidGenerator
// Description:returns a pseudo-random GUID
//This is appended to a url for 2 reasons
//1. to make the URL unique, so that the browser always gets it and doesn’t use a cached version
//2. to make a URL look like its got a unique key, in a naive attempt to fool a not-so-wily hacker
//into thinking they can’t download a datapack directly if they know the URL pattern, because they
//need a unique key.
Notice how anyone who might want to download datapacks directly is branded a hacker. That’s a worrying attitude, since it seems to equate people who want to take advantage of the CC license to explore the census without jumping through the site’s hoops as shady subversives (I doubt the comment used the term “hacker” in its more positive sense).
As the IT News story suggests, the motivation for this obfuscation seems to be to encourage people to pay AU $250 (about US $257) for the DVD version instead. To save others from having to deal with the unhelpful Web site, Bowland generously stumped up the $250 himself, and made the full census database freely available as a torrent, as is perfectly legal under the CC-BY license. This shows perfectly why it is pointless trying to make it hard for people to download content that is CC licensed: once anyone has obtained a copy, they can then make it available in a more convenient form, neatly by-passing forlorn attempts to control something that has been set free forever.
Follow me @glynmoody on Twitter or identi.ca, and on Google+
Filed Under: australia, creative commons, open data
Comments on “Australian Census Data Released Under CC License, But Official Site Tries To Make It Hard To Download”
census
ahh dont be so mad, there just trying to make some money back, 🙂
But really, they probably should have gone about it a different way, you know instead of frustrating the free option why not enhance the payed for version.
On a side note, anyone who wonders on the accuracy of the data, I worked on this census as an information collector, im one of the guys that walked around house to house delivering and collection the forms.
Such a great experience it was, I got to see a wide array of people in just one suburb, I met elderly people, young families, insane people (real wacko’s), shady criminal people, and a lot of nice people just trying to get along in life.
And I can say the organiser in my area really did try to get the most accurate info he could, he instructed all of the walkers to to the best job they could and I believe we did.
What rubbish.
It took me all of two minutes to find what I was looking for (the data packs), another two for registration (email confirmation), and I’m at the “Data Packs – Download” page.
At no stage did any page suggest I shell out $250 for a DVD.
I can only suggest that Bowland is more journalist than programmer 🙂
Convoluted site layouts are par for the course for government departments anyway, I would have liked a big button on the front page that said “Download 2011 census data here” but to say the agency is going to great lengths to discourage downloading seems bizarre.
Re: What rubbish.
“Convoluted everything is par for the course for goverment”
Fixed that for you
Re: What rubbish.
The included javascript segment would suggest otherwise:
// Function: guidGenerator
// Description:returns a pseudo-random GUID
//This is appended to a url for 2 reasons
//1. to make the URL unique, so that the browser always gets it and doesn’t use a cached version
//2. to make a URL look like its got a unique key, in a naive attempt to fool a not-so-wily hacker
//into thinking they can’t download a datapack directly if they know the URL pattern, because they
//need a unique key.
Re: Re: What rubbish.
And yet, with all that super-sekrit-sauce encryption :-), I still managed to get to the download area in a few minutes. I didn’t even have to “view source” in my browser of choice to hack my way in.
Re: Re: Re: What rubbish.
Same here.. I wish I knew about this before that guy paid $250 though. 🙁
Oz has crown copyright
Oz has crown copyright — which has turned out , tends to make it difficult for government agencies to share information with other government agencies. Thus the increased interest in CC-licensing of stuff like census data.
(As I recall, based on a talk by Brian Fitzgerald and Anne Fitzgerald.)
Re: Oz has crown copyright
Which is a good example of why “copyright all the things” is a bad idea. Some things should just be free
Re: Oz has crown copyright
True, though the ABS data and that includes the Business data they collect as well, not just the major census data has ALWAYS been free (I was using it way back in 90’s at zero cost) to use by anyone just as long as attribution is correctly done.
The physical product be it tape, floppy disc, CD, DVD – yep I remember all 😉 – was always an added price like anything that requires a bit more labour from governmental departments, though compared to other databases $250 is a minute amount.
Re: Oz has crown copyright
“Oz has crown copyright — which has turned out , tends to make it difficult for government agencies to share information with other government agencies.”
They DELIBERATELY make it difficult for agencies to get data and info from other agencies?
And copyfools support this sort of thing?
Well, looks like you're jumping the gun a little, read the FAQ?
FAQ http://www.abs.gov.au/websitedbs/censushome.nsf/home/datapackshelpansodp?opendocument&navpos=250
Can I get a DVD DataPack with just one DataPack on it ? and pay less?
DataPacks for download become available about three weeks after the DVDs become available
How much does a DataPack cost to download?
DataPack downloads are free.
Well, looks like you're jumping the gun a little, read the FAQ?
FAQ http://www.abs.gov.au/websitedbs/censushome.nsf/home/datapackshelpansodp?opendocument&navpos=250
Can I get a DVD DataPack with just one DataPack on it ? and pay less?
DataPacks for download become available about three weeks after the DVDs become available
How much does a DataPack cost to download?
DataPack downloads are free.
The only reason I can see, besides them being money grubbing asshats, for not favoring the download would be they don’t want to pay for the bandwidth. If that’s the case they should’ve just posted a torrent file themselves. This is what torrents are made for. Despite the popular belief that they’re only used by “hackers” doing such nefarious things as “hacking the system” and toppling record companies by stealing all their product.
CC
I can only add that media released under public domain and creative commons highlights their folly in trying to destroy BT sites which link to a true sharing medium.
Census data is extremely valuable for a vast array of reasons and Creative Commons is a wonderful medium for keeping this data free while preventing others from exploiting it for profit.
Re: CC
I disagree! People are entirely free to exploit the census data for profit, and they should do so. Everyone can. But they have to manage this trick without having any exclusive rights.
Re: CC
Lol wat? cc allows for profit use if it’s just cc by-(sa)-(nd)
I just want to re-iterate Glyn’s point:
This case shows the strength of the CC license. Though the site operators may instinctively wish to view Aaron-Swartz-style mass downloading from the site and/or distributing the $250 CD as a violation of something or other, it actually isn’t — not because of a a fervently held opposite wish on the part of the anarchic poster, but due to a legal instrument, the CC license, like a little metal walnut embedded there, that now works against the site operator.
Thus putting the site operator inadvertently in the position of a volunteer, like so many others around on the net, who build great things all the time.
Wisecracking by the developer
Notice how anyone who might want to download datapacks directly is branded a hacker
To be fair, to me the “naive attempt” line reads less as a statement of policy by the higher-ups in the Burearu of Statistics, and more a wisecrack by the poor guy in IT who got stuck with the job of writing the Javascript.
Less “everyone who wants to download the data is a hacker” and more “I’m annoyed I had to write this stupid function, but the people who sign my paychecks have demanded it”.