How Important Is It To Preserve Our Digital Heritage?
from the the-archive-team-at-work dept
Having recently been part of an effort to preserve videos on Google Video, in light of its announced shutdown, it’s made me more aware how brittle our current culture is, with many many artifacts available only in digital form. There are conservation efforts such as the aforementioned one, part of a larger group called Archive Team. Google changed its mind and promised to keep Google Video online and try to move as much as possible to YouTube where it has permission from the original uploader. This is, obviously, a win for the archivists, who kept the effort going just the same and are still in the process of uploading all of Google Video to archive.org.
Contrast this with Friendster, where the Archive Team project was unable to save everything before the end of May, when everything was set to be deleted. Much has been downloaded and it will still give a good picture of what this early social network was like. Similarly, the Archive Team was able to rescue much, but not all of GeoCities, before Yahoo shut that down last year, releasing its collection as a massive 1 TB bittorrent file.
Now you may ask yourself is Friendster (or even GeoCities) worth saving? Answering that question purely myself, I’d say no, but I have no connection with those sites. Looking at the question however from the perspective of people who have spent many hours building these profiles and interacting with each other, I can see there’s clearly value there to those who used them. Answering instead as someone with a deep interest in history, it’s not for us to say what will eventually prove worth saving. Instead, that’s something for the historians, archeologists, sociologists and other interested parties in the future to decide. But they can’t do that if the information isn’t even there.
How often have we not wished more of a particular point in our history had been preserved? Ironically many of the analog writings of our past are in a better state than some of our digital ones, even when it’s often pointed out how easy it is to make bit-perfect copies of something in this day and age.
All of this, leads me to ask the following questions:
- Do you agree the digital aspects of our culture should be better preserved?
- Should we have, maybe even one on each continent or in each country, a modern Library of Alexandria? (identical copies in different places to prevent ‘a fire’ from destroying it)
- What would such a world-wide archival effort look like, technologically?
- What should we do about copyrights and patents (and DRM) that might get in the way of preserving our heritage?
- How do we go about archiving software for a computer that’s available only in a museum, to preserve interactive access?
- What about the proliferation of file formats? Do we transform everything into a canonical form where we can, or do we store the original along with software to interpret?
I think we owe it to ourselves that this generation doesn’t look like a black hole when viewed from the future, and by extension, I’d like to make it easier on future historians to learn about us than it has been for us to learn about previous societies (e.g. Sumerians).
I’ll start off by saying that though little personal correspondence has been preserved of earlier times, we should recognize the right to privacy and any such posited archive would be allowed to include private communications only if you explicitly opted in. Valuable though these are for historians, if we preserve as much as we can of the digital domain, enough can be inferred from context that this would be an unwarranted intrusion.
But what about the wider issue of preservation of public content that can be wiped out by shutting off a power switch?
Filed Under: archives, culture, preserving
Comments on “How Important Is It To Preserve Our Digital Heritage?”
Another interesting issue is personal privacy. Because of how quickly a site like Friendster could shut down, these archival efforts are going to take place while the people who posted their work online are still very much alive. But say Facebook is scheduled to close; how would we tackle private Facebook pages? It could be argued that only archiving publicly-facing pages would be incomplete and give a false image of Facebook, but at the same time a user has set his or her profile to private.
I honestly don’t know what should be done about this seeming mismatch of priorities. Perhaps a clause in the terms and conditions that allows archival by a specific third party like the Archive Team?
Re: Re:
It’s it supposed to be the Library of Congresses job to archive historical memorabilia?
Re: Re: Re:
That’s a bit of a disingenuous question. For one, in 1800, there was no internet. Secondly, they already do, to a certain extent, archive historical memorabilia. You might even say that’s their modus operandi; there is no accounting for taste.
Re: Re: Re: Re:
Scratch that. The typo in the original comment led me to misread initially as “Is it supposed to be”, rather than “It is supposed to be?” My apologies.
Re: Re: Re:2 Re:
Yeah, meant to say “Isn’t it”, sorry for the typo.
Re: Re: Re:
Addressing then the question, typo notwithstanding, you appear to have intended to ask. The Library of Congresses of the world would together make up the Library of Alexandria Mk. 2, as posited in the article.
I do however have sympathy for the position held by Jacob Cooper, below, questioning whether it’s something a government can be trusted to do. A third party might have a mandate to preserve as much as possible, regardless of PoV or source, whereas a government entity might be tempted to archive predominantly artifacts showing them in a favourable or neutral light.
A government might also decide cultural artifacts not from their own country, as far as that’s at all easy to tell on the internet, are not their problem to look after. How much would be left by the wayside if all such Library of Congresses came to that conclusion?
Re: Re: Re:
The primary purpose of the Library of Congress is to serve the research needs of Congress, the President, and associated government agencies in pursuing their work. Secondarily, they are the “library of record” and publishers of books and other materials are supposed to send two copies of each title to LC as part of the whole copyright aspect of the Library. They don’t collect ephemera, normally.
This whole issue has been hashed out by librarians and archivists over and over and over again for many decades and over many formats. We haven’t the money, staff or technical resources to do more than well defined, usually grant funded, projects.
Re: Re: Re: Re:
Which is a damn shame, to be honest. Culture makes society makes culture.
Not that it’s a slight against LoC. It’s not they who set their budget. I’m certain they’d love being able to do a more thorough job, even if still not of the scope outlined.
Nor, for the record was I suggesting LoC picks and chooses artifacts adding up to a biased image; it would defeat their stated purpose of being able to inform the research needs of Congress. That’s not to say, as a governmental institution, they’re not subject to the possibility of having that mandate hijacked in the future.
I can’t count the number of times I’ve gone looking for an unofficial patch for a game, or instructions on how to do something obscure, only to be redirected to a web site that no longer exists.
Just the other day, I learned that people had created extra tracks for use with the game Need For Speed 2. Of course the only web site to host those tracks was gone. It’s not even on the Internet Archive.
I really don’t care how much is preserved as long as it’s done by private organizations as opposed to government mandate.
Re: Re:
“We have always been at war with …”
Re: Re:
boy, there’s a conundrum. do you trust the government or do you trust a corporation?
i wish my dog knew how to do it. at least i can trust him. well, except when that damn poodle comes around.
“Do you agree the digital aspects of our culture should be better preserved?”
Absolutely, especially when today digital IS our culture. Consider that digital degrades a lot less gracefully than analog. Where old film reels are managing to last almost a century, a DVD lasts a decade before the data on its starts becoming corrupted. A lot of old computer games stored on floppies or magnetic tape have already been lost, and that trend will continue unless something is done soon.
“Should we have, maybe even one on each continent or in each country, a modern Library of Alexandria? (identical copies in different places to prevent ‘a fire’ from destroying it)”
That’s the idea. There needs to be international cooperation to achieve that and copyright restrictions on archiving need to be removed. In fact, that may be impossible to achieve without taking away “automatic” copyrights and requiring that before any copyright is granted, the creator is required by law to submit to the copyright office a high-quality copy of the work in a standard format without DRM.
“What would such a world-wide archival effort look like, technologically?”
Simplest case, your typical big datacenters. Problem is, whatever we come up with should be able to survive the test of time, even if left in disrepair. Arrays of harddisks will fail fairly quickly if not copied every few months, as will optical media.
A more ambitious proposal would look to find a way to make copies of digital data that will last forever without maintenance. I don’t know if such technology is available, so it may need to be developed. I think I read in Scientific American there have been moderately successful attempts at creating read-only holocubes with about a TB of data imprinted in them…
“What should we do about copyrights and patents (and DRM) that might get in the way of preserving our heritage?”
They don’t matter. They are the opposite of preserving our heritage.
“How do we go about archiving software for a computer that’s available only in a museum, to preserve interactive access?”
Create a unified emulation platform that contains the specifications of various system architectures and runs on a fully documented, open source virtual machine.
Bonus points if hardware and software developers are required by law to maintain the system and make sure their products are compatible with it.
Further bonus if all software developers are required by law to submit source code for archiving (but not necessarily public disclosure).
“What about the proliferation of file formats? Do we transform everything into a canonical form where we can, or do we store the original along with software to interpret?”
Open standards are the future.
Re: Re:
“Should we have, maybe even one on each continent or in each country, a modern Library of Alexandria? (identical copies in different places to prevent ‘a fire’ from destroying it)”
No, we do not. Each country will always serve it’s own interest.
“Create a unified emulation platform that contains the specifications of various system architectures and runs on a fully documented, open source virtual machine. “
Good luck with that 🙂 Do you already have open source cars? How about planes? Guns? I see.
“Further bonus if all software developers are required by law to submit source code for archiving (but not necessarily public disclosure).”
Do you have it for cars already? Ah, I see. People overestimate importance of this “software” thing.
“A more ambitious proposal would look to find a way to make copies of digital data that will last forever without maintenance”
“Forever” is very long time. Nothing, absolutely nothing stay forever. However, writings from Dead See scrolls survived, so Google’s datacenter may too.
Re: Re: Re:
“No, we do not. Each country will always serve it’s own interest.”
Makes no difference.
“Good luck with that 🙂 Do you already have open source cars? How about planes? Guns? I see.”
Just because you don’t see a way of doing something doesn’t mean it’s impossible. We’ll get open source cars and planes some day, just like we’ll get ways to preserve digital culture.
“Do you have it for cars already? Ah, I see. People overestimate importance of this “software” thing.”
What do cars have to do with this? Software is information, cars are cars. The way you preserve one thing is totally different from the other.
“”Forever” is very long time. Nothing, absolutely nothing stay forever. However, writings from Dead See scrolls survived, so Google’s datacenter may too.”
You know perfectly well what I meant by “forever”. If you have nothing to add, I’d rather you didn’t reply to my comments.
Ephemera = Culture
Wow, great question.
And while I understand that it’s not in LoC’s purview nor budget to catalog and archive what IS our ever-evolving digital culture at any given moment, it’s a great point to be brought up.
I actually HAVE my 1995 original GeoCities website (Soho/8499) backed up on a data disc. It’s hilarious, somewhat sad, and accurate to look back on the images and text included as a viable-to-be-cataloged moment in time of digital culture.
That said, how the heck would one go about that? I have no idea; other than to guess it would take a helluva lot of data storage and money.
Even worse (and slightly off-topic) is the number of old film & television series we are going to loose, that will never get to be public-domained because the studios and networks *literally* can’t find the right paperwork that allows them to create digital copies. There’s a LOT of this media, that is going to disappear.
This is Already Happening in our Personal Lives
I have family photos and documents over 100 years old that are easily accessible and viewable.
I have photos and documents on a diskette 10 years old that I can no longer access beacuse my only computer with a diskette drive has died.
Re: This is Already Happening in our Personal Lives
Many fairly recent computers still have a floppy connector on the motherboard, even if there’s no floppy drive installed. You can probably just pull the drive out of your old system and plug it into your current one.
Failing that, you can probably find a system at the dump or in the trash (if you area has bulk trash pickup) that has a floppy drive.
Re: Re: This is Already Happening in our Personal Lives
Or, you can buy a USB floppy drive.
Re: Re: Re: This is Already Happening in our Personal Lives
Or, you can buy a USB floppy drive.
True. Being a tinkerer, I was thinking of solutions that wouldn’t cost anything. I mean, people are literally throwing out old floppy drives and computers with floppy drives in them. Even CD-ROM drives aren’t considered worth saving any more. Not long ago I saw a computer stripped down to the motherboard (cards, memory, fans, etc), but they left the CD & floppy drives.
Art will last as long as the fans make it last.
Look at Jagged Alliance 2 (PC game) with it’s still-bustling mod community, based around a game released in ’99.
Then look at how fast bad movies drop out of circulation. Heck, the worst are released straight to DVD because they know the fans WON’T support it.
As long it’s driven by the quality of the work, or at least the dedication of the fans, we’ll be fine.
Re: Art will last as long as the fans make it last.
I agree with you. Even if governments or companies won’t be backing everything up, luckily for us and the future historians, the fans are the ones who will try to keep the rest of digital history alive, irregardless of the increasingly draconian copyright laws and other surmountable roadblocks placed in our way.
A terrific example of someone taking it upon themselves to preserve an important part of our early online presence can be found at: http://www.textfiles.com/ , covering the history of BBS text files from the 80’s and 90’s. Many of these files may be familiar to others out there, and you might even have them still stored on some type of medium. But with this site preserving the files, there is no need to pull out the old PC in the garage with the floppy drive and see if that disk is even still readable.
The Internet never forgets
The Internet never forgets. Ever. Even when we really want it to.
It just may be a bit harder to find the older stuff.
Digital lifetimes
Consider images, plain old jpg or even older bmp formats. These digital images exist only as data (unless someone prints it out). With the advent of the digital camera, people are free to choose what to keep and what to trash, once trashed they are gone forever. Burn the images to DVD is a usual way to ‘preserve’ these images.
Now, store this DVD in an old biscuit tin, keep in the attic for 30 years and then dig it out to show the grand kids what life was like way back then… ah, now we have a problem. The DVD may be in perfect condition but no DVD readers are in existence any more.
However those old film negatives are still there, and can still be ‘seen’ as photos using nothing more hi-tech than a lamp and a lens. As a bonus, those shots that a digital user would have deleted are still there, and give another view of the world at that time.
There is something being lost with this digital revolution, despite the huge amount of data we produce, how much will be available to the generations to come?
I remember reading about the digital ‘Doomsday project’ this was suppose to be a digital version of the original, sadly the 12 inch video disks used to store the data became obsolete and only by the intervention of a few dedicated people was the data retrieved and put on the web (and how long will that last? what happens when the hosting goes bust?)
I lament the loss of plain old film cameras. And vinyl, cassette tapes, and paper books etc. etc.
RevJim
Re: Digital lifetimes
Consider images, plain old jpg or even older bmp formats. These digital images exist only as data (unless someone prints it out). With the advent of the digital camera, people are free to choose what to keep and what to trash, once trashed they are gone forever.
On the flip-side of that, digital cameras are much more economical to use. In the past you had to buy film, then, if you weren’t a photography buff, you had to pay to have the film developed and printed. Even if you were an expert with your own darkroom, developing the images took expensive chemicals and time. With a digital camera, people can shoot hundreds of images that are ready for use as soon as they’re downloaded to a computer. They can then be printed out, emailed or archived. It’s true, that people only keep the shots that they like, but they’re taking more photos to begin with. I hardly ever used a film camera except on special occasions. My mother let about a dozen rolls of film from various trips, sit and rot because she kept putting off having them developed. If we’d had a digital camera back then, I’d have all those photos today.
There is something being lost with this digital revolution, despite the huge amount of data we produce, how much will be available to the generations to come?
You’re missing one very important point; Today’s digital files, such a jpegs, AVIs, etc, can easily be transferred to any future technology. You could argue that some will be lost because nobody bothers to do it, but that’s not the fault of the technology.
For example, I have hundreds of images saved on Amiga floppy disks. While I still have an Amiga, ways to directly read the disks on current computers don’t exist (I used a custom format to get more on the disks). However, there ARE ways to transfer those files. I can use MS-DOS formatted disks, although that would be slow. A better solution would be to copy them to an MS-DOS formatted Zip disk, then read that on my current system. True, it’s “only” 100MB, but that will hold quite a few floppies. I can also use a null modem cable and transfer the files that way. Once transferred, the files can be burned to DVD, Blu-Ray or stored on a 2TB drive. And if some new storage format comes along in the future, they can be transferred to that as well, with no loss in quality.
I lament the loss of plain old film cameras. And vinyl, cassette tapes, and paper books etc. etc.
I have video tapes that I recorded in the 1980s which are now unwatchable due to the degradation of the magnetic fields on the tape. Some of my paper books are literally falling apart. While I do agree that reading an actual book is nicer than reading text on a screen, books can’t easily be searched for certain passages, which I have often wanted to do. Not to mention that every single book I own could fit on a single DVD with room to spare.
Preserving history
Important questions.
I think it is important to realize that a huge part of the culture of the western world was considered “unimportant” in the middle ages, and was preserved, and eventually returned, to us by Islamic scholars.
We owe them an enormous debt of gratitude.
P2P - solution
Maybe P2P-like solutions could help preserving a digital heritage?
The information need to be stored somewhere. What about storing it everywhere. Not sure exactly how it would work, but lets say everyone holds a copy of the info, stored on local devices, like P2P networks. As long as one device functions, the info can be restored.
Lets look forward in time, let say 5 years, 10, 50..
What will storage capazity be like at that time? What will network transfer speed be like? Would it be possible with the kind of solution im thinking of?
Re: P2P - solution
Yes, a decentralized storage.
Kind of like the digital currency “Bitcoins” seem to function?
Re: P2P - solution
I have pretty good idea on how that could be accomplished.
Look at Omemo and see how it works or Osiris.
Everybody could help save part of their heritage storing bits and pieces of it.
I believe it has its importance, maybe not all the data, but as said in the article, it is not for me to decide what will be useful or not, it is a thing for those who love that kind of stuff.
On a practical note one could clean some information on how certain years fared based on the language used, or see trends developing slowly.
I am trying to find a justification for saving millions of Tweets about parking spots and bowel movements. I am unable to find a reason, explaining why this isn’t a black and white issue. Saving it all would be about as useless as saving none of it.
All digital input should be archived somewhere.
even Techdirt
just for the sake of argument i’m going to try the other side of the topic.
part of what makes the past “the past” is that it has faded from our collective memory. we should let these things die. with 100% recall we would never be able to escape our past. right now institutions like the smithsonian keep and display the best of the best and the worst of the worst. this works pretty well. if we keep everything then there is no nostalgia or mystery. instead of keeping everything we should make an effort to randomly delete three-fourths of all archives out there. if certain facts and accounts are important they will survive because they are better documented.
Research project on personal digital archiving
This was a very timely article. I’ve explored the questions raised in this article as a research project while at the University of Central Florida. This resulted in a very comprehensive discussion of these issues, coupled with specific recommendations. My findings were published in my honors thesis, which may be viewed at:
http://explorer.cyberstreet.com/CET4970H-Peterson-Thesis.pdf
To anyone who is interested in looking at these issues in depth, please feel free to download and read.
An in the present temporal comment.(I’ll get over it)
Saved private pages. Hahahaha. Did you ever think you (you yourself, personally, sitting in a chair looking at the screen.) were making history? It is likely that your saved locally Facebook/other private non trollable web page in your computers temp cash is the last remembrance from your dead friend (Condolences, it happens). Will you be able to access it? (love the pages that don’t require a refresh.)