The Stasi's Tiny Torn-Up Analog Files Defeat Modern Digital Technology's Attempts To Re-Assemble East Germany's Surveillance Records

from the too-hard-for-today's-hardware dept

It is nearly 30 years since the wall separating East and West Berlin came down, and yet work is still going on to deal with the toxic political legacy of East Germany. As Techdirt readers are well aware, one of the defining characteristics of the regime in East Germany was the unprecedented — for the time, at least — level of surveillance inflicted on citizens by the Stasi (short for Staatssicherheitsdienst, or State Security Service). This led to the creation of huge archives holding dossiers about millions of people.

As it became clear that East Germany’s government would fall, and that its long-suffering citizens would demand to know who had been spying on them over the years, Stasi officers began to destroy the most incriminating documents. But there were so many files — a 2008 Wired article about them says they occupied 100 miles of shelving — that the shredding machines they used started to burn out. Eventually, Stasi agents were reduced to tearing pages by hand — some 45 million of them, ripping them into around 600 million scraps of paper.

After thousands of bags holding the torn sheets were recovered, a team working for the Stasi records agency, the body responsible for handling the mountain of paper left behind by the secret police, began assembling the pages manually. It was hoped that the re-assembled documents would shed further light on the Stasi and its deeper secrets. But it was calculated that it would take 700 years to deal with all the scraps of paper by hand. A computerized approach was devised by the Fraunhofer Institute, best-known for devising the MP3 format, and implemented following a pilot project. After some initial successes, the program has run into problems, as the Guardian reports:

A so-called ePuzzler, working with an algorithm developed by the Fraunhofer Institute and costing about €8m of [German] federal funds, has managed to digitally reassemble about 91,000 pages since 2013. However, it has recently run into trouble.

For the last two years, the Stasi records agency has been waiting for engineers to develop more advanced hardware that can scan in smaller snippets, some of which are only the size of a fingernail.

The ePuzzler works by matching up types of paper stock, typewriter fonts, or the outline of the torn-up page. It has struggled with hand-written files that were folded before being torn, leaving several snippets with near-identical outlines.

While the hardware engineers try to come up with a suitable scanner that can handle these tiny fragments, a small team continues to match up the more crudely ripped pages manually. Inevitably, some people will be thinking: “If only the Stasi had used blockchain, all these problems could have been avoided…”

Follow me @glynmoody on Twitter or, and +glynmoody on Google+

Filed Under: , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “The Stasi's Tiny Torn-Up Analog Files Defeat Modern Digital Technology's Attempts To Re-Assemble East Germany's Surveillance Records”

Subscribe: RSS Leave a comment
Anonymous Coward says:

"unprecedented -- for the time, at least --" but nothing compared to Google and Facebook.

Both Google and Facebook have BILLIONS OF PERSONS INDEXED. And you think it’s good, merely "commercial", and that gov’ts are able to resist detailed data. — You are the nebbishes in this sci-fi dystopia.

"Inevitably, some people will be thinking: "If only the Stasi had used blockchain, all these problems could have been avoided…" — ??? One of your wackiest attempts at "color". Or whatever.

steell (profile) says:

Re: "unprecedented -- for the time, at least --" but nothing compared to Google and Facebook.

Dude, generally I refrain from replying to your posts, I just flag them and move on. But this one is posted from the depths of ignorance so great the bottom may never be found.

So tell me oh ignorant one, does Google make people disappear? Does Google slip a note in your pocket saying if you don’t cooperate all your transgressions will be revealed to Command? Does Google threaten your family with harm if you don’t comply?

To compare Google to the Stasi is the stupidest thing I’ve ever read.

Richard (profile) says:

Re: Re: "unprecedented -- for the time, at least --" but nothing compared to Google and Facebook.

To compare Google to the Stasi is the stupidest thing I’ve ever read.

It is not true that comparing Google to the Stasi is stupid.

(even though his comparison was)

Both collect(ed) large amounts of personal information.

Google uses it for purposes that are not necessarily evil, and may sometimes be beneficial to the subject.

The risk with Google is a 3rd (stasi -like) party getting access to the information and using it in an evil way.

Anonymous Coward says:

Re: Re: "unprecedented -- for the time, at least --" but nothing compared to Google and Facebook.

“Does Google slip a note in your pocket saying if you don’t cooperate all your transgressions will be revealed to Command?”

I would categorize ad-tracking as being within the same wheel house. Perhaps, not to “command”, but to any advertiser willing to pay.

Targeted advertising supplied by Google has probably outed more homosexuals than all the angry ex-girlfriends on the planet. Not to mention outing peoples medical conditions, and just about anything else a person might be embarrassed about.

The Stassi at least charged a serious bribe. Google outs people for fractions of a cent. Notably TD contributes to this since they use Googles analytics services.

Or do you not consider the causality of the TV advertising you see in other peoples homes?

That One Guy (profile) says:

Re: Re:

I was thinking that same thing. Did they not realize that ‘fire’ was a thing? A bonfire or ‘accidental’ building fire would have cleared the whole thing up real quick, yet they spent I don’t even want to think how much time manually ripping up pages of highly flammable paper.

The Stasi: Great at collecting evidence, idiots when it comes to destroying it.

Still, I suppose people should be glad they were stupid in that way, as there’s at least a chance to reconstruct ripped up pages, a possibility that doesn’t work so well when it comes to ashes.

Anonymous Coward says:

Re: Re: Re:

Take a look at Nazi practices: they murdered millions in remote and isolate places. But not all places were remote, some even really close to villages, for example the psychiatry in Hadamar. The fetidness from the included crematory was very well discernible by villagers (they all played along anyway).

Likewise, a bonfire destroying the files would have sent a very clear signal, while Stasi was all about secrecy. Maybe Stasi officers weren’t motivated anymore, which leads to worse performance at work. Or maybe they just couldn’t afford the gas and/or the matches.

Anonymous Coward says:

Re: Re:

And at no point did any of them think ‘this would be a lot easier with a couple gallons of petrol and a pack of matches’?

To quote the Wired article linked by Glyn, "Burning all those files would tip off angry Berliners that something was up."

They were outnumbered, and worried about what would happen if the public found out what they had been doing. They wanted the most incrimining stuff destroyed before anyone found out they were destroying it.

Ben (profile) says:

Book scanning writ large

In Vernor Vinge’s Rainbow’s End there was a (somewhat) horrifying mechanism for scanning books: toss the books in a shredder and while the shredded pieces are floating down an air-filled channel they are photographed and then digitally reassembled. It certainly gets away from having to carefully scan each piece; I just wonder if the technique would require the re-shredding (further shredding?) of the Stasi papers to deal with the less than uniform existing pieces.

Once a workable solution is found, however, this will have very big implications for the SEC and FBI trying to recover documents after corporate shredding parties.

Anonymous Coward says:

Re: Book scanning writ large

To address that last point: maybe. Files are increasingly kept in digital form, which makes them much easier to disappear, provided those doing so have a baseline level of competence.

For example, as part of my work, I sometimes decommission servers and thus the disks inside them. My modus operandi is to hook those disks up, one at a time, via SATA-to-USB hardware, to a system that runs quite a random write passes on the entire drive. Then I open them up and expose the platters to a degausser. Then I remove the platters and feed them to a shredder. Then I separate the the shreds and dispose of them in multiple different places.

Good luck recovering anything from that, even with the resources of an intelligence agency. (Of course, if they were interested, they likely snagged all the data while the systems in question were still live.) The point is, though, that anyone who’s trying to destroy data thoroughly has the means to do so, and it’s much easier than dealing with miles of (paper) files.

Ben (profile) says:

Re: Re: Book scanning writ large

In the modern era, yes, all the data/memos would be on servers, so getting rid of it is much easier than in the past (and easier to explain away — oh! that project ended so we decommissioned the server with a sledgehammer…)

The issue is the so-called "paperless office." The reality is people still print things out — it is just that they can re-print it at need. That is what they have shredding parties for.

The Wanderer (profile) says:

Re: Re: Book scanning writ large

Even that isn’t theoretically 100% guaranteed to render the data unrecoverable. It’s enough for the limits of modern technology and the resources of pretty much any organization that exists, but in theory, a sufficiently-powerful organization could collect the scraps and shuffle through them and reassemble them (or scan them and reassemble the scanned data objects) and read the result.

If you really want to guarantee that no one will ever be able to read what used to be on a given hard drive, you need to pull out the platters and melt them, then stir the result together till it’s unrecognizable.

Any entity capable of recovering the data from that would be so far into "indistinguishable from magic" that it could just as readily recover the data without access to the hard drive in the first place.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...