Internet Archive Fire Shows Vulnerability Of The World's Online Memory

from the taking-things-for-granted dept

The Internet Archive is a jewel of the digital world:

The Internet Archive is a 501(c)(3) non-profit that was founded to build an Internet library. Its purposes include offering permanent access for researchers, historians, scholars, people with disabilities, and the general public to historical collections that exist in digital format.

Founded in 1996 and located in San Francisco, the Archive has been receiving data donations from Alexa Internet and others. In late 1999, the organization started to grow to include more well-rounded collections. Now the Internet Archive includes: texts, audio, moving images, and software as well as archived web pages in our collections, and provides specialized services for adaptive reading and information access for the blind and other persons with disabilities.

Here’s the amazing scale of the project today:

The Internet Archive Wayback Machine contains almost 2 petabytes of data and is currently growing at a rate of 20 terabytes per month. This eclipses the amount of text contained in the world’s largest libraries, including the Library of Congress.

The Internet Archive is the world’s online memory, holding the only copies of many historic (and not-so-historic) Web pages that have long disappeared from the Web itself.

Bad news:

This morning at about 3:30 a.m. a fire started at the Internet Archive’s San Francisco scanning center.

Good news:

no one was hurt and no data was lost. Our main building was not affected except for damage to one electrical run. This power issue caused us to lose power to some servers for a while.

Bad news:

Some physical materials were in the scanning center because they were being digitized, but most were in a separate locked room or in our physical archive and were not lost. Of those materials we did unfortunately lose, about half had already been digitized. We are working with our library partners now to assess.

That loss is unfortunate, but imagine if the fire had been in the main server room holding the Internet Archive’s 2 petabytes of data. Wisely, the project has placed copies at other locations:

We have copies of the data in the Internet Archive in multiple locations, so even if our main building had been involved in the fire we still would not have lost the amazing content we have all worked so hard to collect.

That’s good to know, but it seems rather foolish for the world to depend on the Internet Archive always being able to keep all its copies up to date, especially as the quantity of data that it stores continues to rise. This digital library is so important in historical and cultural terms: surely it’s time to start mirroring the Internet Archive around the world in many locations, with direct and sustained support from multiple governments. They can also help provide the Internet Archive with a wider, more international range of content, to make an even more representative store of the world’s digital activity.

Unfortunately, that’s not likely to happen anytime soon, as people seem happy to take for granted the amazing work of Brewster Kahle and his team. The next best thing would be to donate so that they can continue with their indispensable project — and perhaps create a few more backup copies.

Follow me @glynmoody on Twitter or, and +glynmoody on Google+

Filed Under: , , , ,
Companies: internet archive

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Internet Archive Fire Shows Vulnerability Of The World's Online Memory”

Subscribe: RSS Leave a comment
Lawrence D'Oliveiro says:

One Of Their Redundant Sites Is In Alexandria, Egypt

The same Alexandria that was the site of the Great Library back in Classical times. The same Library where Hypatia, daughter of Theon, was in charge. The same Hypatia that was put to a horrible death by the Christians who didn?t hold with all that pagan learning. Even more unnatural if it came from a woman.

Some things haven?t changed that much…

PRMan (profile) says:

Re: One Of Their Redundant Sites Is In Alexandria, Egypt

Neglected a relevant fact or 2 did we?

“Upon hearing of this, Cyril threatened the Jews of Alexandria with “the utmost severities” if harassment of Christians was not ceased at once. In response, the Jews of Alexandria grew only more furious over Cyril’s threat, and in their anger they eventually resorted to violence against the Christians. They plotted to flush the Christians out at night by running through the streets, claiming that the Church of Alexander was on fire. When the Christians responded to what they were led to believe was the burning down of their church, “the Jews immediately fell upon and slew them”, using rings to recognize one another in the dark, while killing everyone else in sight. When the morning came, the Jews of Alexandria could not hide their guilt, and Cyril, along with many of his followers, took to the city?s synagogues in search of the perpetrators of the night’s massacre.”

I think it was maybe her defense and counsel of this slaughter, and not her learning, that caused her to be killed. But way to spread the hatred.

Pragmatic says:

Re: Re: Re: One Of Their Redundant Sites Is In Alexandria, Egypt

It’s not a specific group that’s being targeted in statements like this; PRMan was taking a pop at authoritarianism. It just so happens that certain Jewish activists were attacking Christians in the same way and for the same reasons as the Roman Catholic Church attacked “heretics.”

It was the same idea: Christians were a heretic group, to their minds, and they hit them for that.

What I’m saying is, nobody is hating on anybody, Lawrence D’Oliveiro. We’re just stating facts and letting them speak for themselves.

kizilbash says:

Re: One Of Their Redundant Sites Is In Alexandria, Egypt

There is no historical basis for the idea that the library at Alexandria was burned, by Christians or anyone else. Hypatia was indeed killed by a Christian mob, at the tail end of decades of back and forth violence that saw many killed on both sides. Contrary to popular belief, Hypatia was not a significant figure in the history of thought. Doesnt mean she deserved to die, but rather that history doesn’t need to be abused and rewritten just because you personally dislike Christians in the 21st century.

Anonymous Coward says:

“Internet Archive Fire Shows Vulnerability Of The World’s Online Memory”

No, it shows that people that actually care about preserving history are smart enough to have multiple backups. It shows that the Internet archive is in good hands.

These aren’t the guys that wanted to “preserve” IsoHunt just so they could open a few clones of the site the very next week.

Anonymous Coward says:

this is the modern day equivalent of what happened to some many records of bygone ages. but they is far more concern with copyright than with preserving anything. i’m trying to figure out how having a copyright on something is going to preserve it and enable it to be reproduced if necessary. i think that once it’s lost, it’s lost. look what happened to Atlantis.

Anonymous Coward says:

But the Internet Archive sucks balls now

Seriously, have you used it at all these past few years? It sucks, its usefulness has passed. Do to its overzealous enthusiasm to please the ever powerful Copyright Gods, and the fact that the coders are all lazy good-for-nothings, the wayback machine and search functions have been crippled with retroactive content removal. They allow current owners of sites and content to use bots to retroactively block websites from back in time to when they didn’t even own said content. Half of all things they have archived is inaccessible, blocked by domain squatters who don’t own the old versions of the sites they block. If TechDirt goes bankrupt today, and I buy this domain, I can flag all the old sites in the archive with a simple bot and nobody will be able to view them.

Anonymous Coward says:

Re: But the Internet Archive sucks balls now

The Internet Archive definitely as it’s problems and should not be depended upon to always being able to keep all its copies up to date as Glyn Moody said or to keep every single copy functional , especially as the quantity of data that it stores continues to rise. However it is far more exhaustive then anything else out there at the moment.

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...