Last month, the BBC announced its intention to kill off a bunch of websites, including Douglas Adams' old stomping grounds, H2G2. Apparently some of the sites were going to be archived, and others weren't -- but as you may know, the BBC does not have the greatest of reputations when it comes to archiving old material. However, this is the internet. If someone announces they are going to get rid of something that other people would like saved, there are tools to save it. Glyn Moody points us to a note from Ben Goldacre about how some "anonymous nerd" archived all of the sites the BBC is set to take down (it cost him a whopping $3.99) and he's now set that archive free, so you, too, can help make sure this content lives on. The whole thing is available via a torrent file at http://178.63.252.42/, which also has a description of the project, so have at it.

Recently, we pointed out that various film archives were disintegrating, and noting that perverse copyright laws were partly to blame. Now, Copycense points us to the news that experts are also quite worried about audio recordings degrading and disappearing -- including recent recordings, such as from 9/11 and the 2008 election. This is part of a study commissioned by the Library of Congress -- so it's not just some random researchers. Thankfully (unlike the article on films) the study and the reporting about it highlights the copyright issue:

A hodgepodge of 20th century state anti-piracy laws also has kept most sound files out of the public domain before U.S. copyright law was extended to sound recordings in 1972. The study found only 14 percent of commercially released recordings are available from rights holders. That limits how much preservation can be accomplished, Brylawski said.

The full study (The State of Recorded Sound Preservation in the United States: A National Legacy at Risk in the Digital Age (pdf)) suggests that copyright laws be changed to deal with the archival problem. The study mentions the famous (and mostly ignored) UK Gowers study that found that extending copyright was a mistake (Gowers later admitted, separate from the report, that the economic evidence clearly indicated copyright terms should be shortened). The study also quotes many researchers, academics and archivists who point out how copyright law has made it quite difficult for them to do basic research and other useful things without violating the law.

While fair use is frequently invoked, there is little case law interpreting its application to specific situations. Some archivists believe the lack of applicable case law discourages institutional general counsels from authorizing preservation and access programs on the basis of fair use. Fair-use provisions in the copyright law (Section 107) permit use of excerpts of copyrighted materials, including recordings, for educational and scholarly purposes, for citation in news reporting and criticism, and for nonprofit activities. However, the courts have interpreted fair use to be an affirmative defense. In other words, the burden is on the defendant to prove fair use; the plaintiff does not need to establish that the use falls outside fair use. Besek further notes that "favored uses are not automatically deemed fair, and other uses are not automatically unfair. The determination depends on the facts of a particular case."

Most witnesses at the hearings conducted in support of this study who were affiliated with educational institutions expressed a belief that copyright law as it applies to sound recordings is too complex to interpret easily, too restrictive, or both. Section 108, for example, which limits access to digital copies made available under that section to library premises, is too narrow to address all educational needs. One witness called for "premises" to be expanded to include network domain, in order to accommodate "the manner in which students and scholars use information in the current academic and scholarly environment. Increasingly, learning is occurring off-site, that is, at home, in the dorm or just anywhere on campus, in addition to the classroom and library."

Separately, the report notes that DRM and the DMCA's anti-circumvention provision is a real problem for researchers and archivists.

In the end, the study says that copyright law needs to change to deal with this very real challenge:

Creation of new copyright laws or licensing procedures that acknowledge best practices in audio preservation and assure access to audio heritage is essential to ensure the preservation of that heritage and its understanding and appreciation by generations to come.

It then repeats five suggestions made by The Association for Recorded Sound Collections (ARSC), such as changing the fact that pre-1972 recordings are all limited under ancient state laws, rather than federal copyright law (meaning they won't be in the public domain for much, much, much longer than more modern recordings). They also suggest moving copyright term back (just slightly) to 50 or 75 years, supporting orphan works legislation, enabling compulsory licensing to copyright holders for anyone wishing to reissue "abandoned" recordings and allowing non-profits to backup and archive sound recordings without violating copyright. The recommendations are, frankly, a bit weak, but in an era where the government only seems to want to strengthen copyright, it's nice to see at least some effort to push back in some areas. Of course, the likelihood of the Library of Congress actually following through on any of these recommendations seems slim at best.

While UK politicians are arguing over a ridiculous and unnecessary change to copyright law as part of the Digital Economy Bill, it appears there's a much bigger problem with UK copyright law that isn't getting very much attention at all. Slashdot points us to the news that due to the way copyright law currently works in the UK, archiving websites without permission is illegal. Yes, even for the British Library and other institutions who are designated by law to keep a copy of every printed publication. But when it comes to the web, the Library needs to get permission from every website that it wishes to archive. Obviously, that greatly limits the archival activity that the Library can be involved in -- and, as a result, the public suffers greatly. This is a clear case where fair use should cover the issues, but current law does not adequately handle this. Making fair use work better should be a priority -- but instead we have politicians trying to prop up Hollywood's business model by pushing copyright law in the other direction.

A bunch of folks have sent in the news that the Economist appears to be putting up something of a paywall, locking up all archival content older than 90 days, while also locking up one version of the magazine (the one that is made to look just like the physical paper layout). I have to be honest: I don't see how this makes any sense at all. In our experience, somewhere between 25% to 30% of our daily traffic is to archival content, usually in the form of search engine traffic -- or occasionally other sites picking up on an older story. Archival content is perfect Google fodder, driving traffic (and ad views) to pages that otherwise would get no traffic at all. In many ways, that's a big part of the value of having widespread archives -- to bring in such traffic for those who care about it. The chances of such a "drive by" viewer paying up for a subscription to view that content seems incredibly slim -- and it seems quite likely that the decline in traffic (and ad dollars) would almost certainly outweigh the number of new subscribers added. This doesn't seem to make any sense at all. Does The Economist have any information economists on staff?

Back at the end of March, we were surprised that the NY Times, in consolidating its regular site with the site of the International Herald Tribune (which it owned) had broken all the links to IHT.com. Rather than taking them to the article in question on the NY Times site, it simply took them to a landing page. This was just a bad idea all around. It appears that a former reporter for IHT, Thomas Crampton, discovered this over the weekend and has brought renewed attention to the issue by issuing an open letter to the NY Times asking why it "deleted" his career -- in that all of his early work that appeared in the IHT is now gone (some, but not all, of it remains in the NY Times). Additionally, he pointed out that this is also causing problems for Wikipedia, notably with any article that relied on evidence from an IHT article. While we've seen others erase old articles as well (and the Associated Press is famous for forcing all its partners to take down AP articles after just a short time period), it still is amazing in this day and age that anyone thinks it's a good idea to break links to news stories -- especially when the value of archives found via search engines is so high.

Plenty of folks have pointed this out for years, but newspapers that try to lock up their back archives and charge for viewing those articles are very likely hurting their bottom line more than helping it. That's because those archives are a treasure trove of info that people would be interested in finding via a search engine -- but they almost never want to pay for it. For many years, the NY Times tried locking up its archives and charging to read stories, but eventually did the math and realized it made a lot more sense to put all its archives online for free, and make money off the ads. Since removing the barriers, the NY Times has seen its traffic spike significantly, and its archives have become a significant portion of the overall site's traffic.

However, some newspapers still can't see the forest for the trees, and think that the answer is to charge high prices to view old articles. That most likely just gets people to look elsewhere, and diminishes ad revenue as well. Parker Mason has written an open letter to the Toronto Globe & Mail decrying its continued practice of charging $5 for access to a single archived article (for just 30 days of access). It's a good read, and I'm guessing that folks like Mathew Ingram, who works at The Globe, have been pushing for changes to the paper's policy, but until then, the company seems to be hurting itself. Mason's letter is well worth reading, but here's a snippet:

But then you go and do something like trying to charge me $4.95 for a newspaper article that I've already paid for and read, and this hurts me (telling me that this content will only be available for 30 days only adds insult to injury).

Your greatest asset is the thousands and thousands of pages of information and news stories that you have in your archives. People want to view this content, and just as they have endured advertising in your print publications, they'll endure the same kind of advertising on your website.

I understand your thinking when it comes to locking up this content behind a pay wall: it is valuable information, so people will pay to see it.

The problem is, you are only half-right. It is valuable information, but only when it is easy to access. In the age of Google, people will quickly move on and find the information elsewhere, somewhere where it easier to get at.

Copyright was clearly designed for a different age: when not everyone was a "publisher." And while we've spent years pointing out many of the different problems that has caused, here's another one: how is a library or some other institution charged with "archiving" written works for posterity supposed to deal with copyright laws that can often make such archival activities against the law? Well, the Library of Congress and a bunch of other organizations have a suggestion: let them all ignore copyright law for the sake of archiving. Basically, the report recommends that certain organizations be designated as "preservation institutions," which are then more or less allowed to ignore copyright law and copy-at-will for the sake of preservation. Of course, this is clearly going to lead to many questions, including just who would get designated as such. Many people can probably agree on public libraries and such -- but what about Google? After all, Google is already one of the largest players in "preserving" what's online and also, with its book scanning project, what's in books. Yet it's a private, for-profit company. Should it qualify? I would argue that it makes sense to allow it, given how beneficial the archival activities of Google have already been. Even if it is for profit, the public benefit has been tremendous as well. But then what's to stop any other company from arguing that it to deserves an exemption for preservation purposes? Wouldn't a better solution be to start rethinking copyright law altogether, since what has become clear from this is that copyright doesn't quite fit today's world any more?

In the wake of the ridiculous dustup between bloggers and the AP, where the AP threatened bloggers who help promote AP articles, some are starting to point out that the AP's problem goes well beyond a seriously distorted view of copyright law. The reason this came up at all was because the AP's business model is pretty screwed up in a web world. This was clear from the fact that the more involved the AP gets online the more it ends up competing with all its member newspapers. Almost every action it takes seems to help the AP's business model, while hurting its members. With the latest skirmish, the AP is ensuring that those partner sites get fewer links in and less traffic. Back when the AP signed a deal with Google News, we pointed out that its member newspapers should be pissed off. Basically, the AP and Google had worked out a deal to keep traffic away from the member papers. That's no way to survive. Eventually, if this keeps up, those members freak out and stop supporting the AP.

That leads to great suggestions from Danny Sullivan on how it's time for the AP to totally rethink its business model, recognizing what the web does for its business:

Well, wake up call. You need a new model. Really. Or you're going to die.

The AP should have a news portal. You should take in content from your members, put it up in an easy-to-find way and generate the ad dollars to be redistributed back to your members. Like do it now, before since the entire licensing thing ain't going to live that long.

It's worth reading the whole thing, as it's right on point. However, there is one additional thing that's worth mentioning: the AP really needs to learn to keep its news online. Back when I would link to AP articles, one of the most annoying things was that they would disappear after a couple weeks. We used to get complaints all the time from readers who would find an older post and the underlying AP article would be gone. I had thought that maybe its deal with Google would change this, and started pointing to the Google versions of AP articles... but, nope, those disappeared after a few weeks as well.

In the narrowminded world of an AP exec, they probably think this leads to more licensing revenue, since it will make people search out and license the article after it can't be found any more. Nope. It just makes people pissed off. Many newspapers have realized that there's tremendous value found in freeing up the archives, and monetizing that long tail of traffic via advertisements. That's a lot more effective than pissing off large groups of folks (including the people who promote your articles) and hoping it leads to a little more licensing revenue. So, while I agree with Danny's recommendations for the AP to join the internet era, I'd also add a recommendation that it open up its archives and recognize that URLs should be permanent rather than fleeting.

Here at Techdirt we have over ten years worth of content, all available for anyone to read, and as we certainly get a fair amount of traffic to those back archives. While we don't pay that much attention to ad revenues (our business isn't advertising), access to those archives (mainly from Google searches or links from other sites into a specific older story) represent a fair chunk of our page views and ad revenue. With that in mind, it's been quite surprising to see so many publications try to lock up their archives -- either (worst of all!) taking down old stories completely or trying to lock them up behind a pay wall. Luckily, it looks like more and more publications are recognizing that this is a bad business strategy. The article is in the NY Times, which only recognized this very issue a few months ago. Prior to that, it charged for access to its archives, but since opening it up has seen traffic shoot up and ad revenues appear to be following. The article also mentions how Newsweek has had a lot of success opening up its archive, and Sports Illustrated is getting set to make its own archive available later this week. For all of those publishers who worry that there isn't enough ad revenue online, it makes little sense to sit on so much inventory. These days, you need to work on using Google to help drive more traffic, not suing it to stop sending traffic. What better way to make money off your archive than getting a lot more people to look at it?

Is it really any kind of surprise that Hollywood is worried about the wrong thing? The NY Times ran an interesting article this past weekend about how Hollywood is starting to freak out over the potential costs of digitally archiving movies. Currently, film archives are simply stored in cool places, like salt mines -- but Hollywood doesn't quite know what to do with digital archives, and a new report has them freaking out about just how expensive it will be to store digital content. There are many reasons why this worry is misplaced -- starting with the simple fact that whatever it costs today is only getting cheaper, and that trend is only going to continue for the foreseeable future. However, we've talked about the risks of digital archiving and "digital extinction" before, and the threat is completely overblown and often misplaced.

The problem isn't with what it costs to store content. Storage is cheap and getting cheaper all the time. The real problem is that those doing the archiving keep wanting to put their content into proprietary formats which will rapidly go extinct. If, instead, Hollywood focused on storing (and making many, many copies) of the content in more open, easily accessible formats, this wouldn't be a problem at all. Hell, I'm sure the experts over at the Internet Archive, Google or Amazon would all be thrilled to help Hollywood preserve its digital films. However, since Hollywood is so freaked out by technology these days, the chances of them letting any of those organizations help out (even a not-for-profit one like the Internet Archive) seems slim to none.

In the meantime, why not get creative? How hard would it be to create a system that would build a p2p storage system for Hollywood archives, where lots of folks could store bits and pieces of movies for the studios in exchange for... say... a free sneak preview of an upcoming blockbuster? It's the sort of thing that the community would love to take part in... but, of course, in MPAA land anything P2P must be evil.