Snowden Docs Show GCHQ, MI5 To Be All Haystack, No Needle

from the collections-so-bulky-they're-practically-immobile dept

Collect it all,” they said. “You can’t find needles without haystacks,” they proclaimed. “The more you know,” they rainbowed. All well and good, except the NSA, GCHQ, et al. appear to have far more in common with the protagonists of “Hoarding: Buried Alive” than with effective, finely-tuned terrorism-fighting machines.

New documents from the Snowden stash show the UK’s intelligence agencies love piling data on top of data, but seemingly have no idea how to utilize this massive haul.

MI5 “can currently collect (whether itself or through partners …) significantly more than it is able to exploit fully,” the report warned. “This creates a real risk of ‘intelligence failure’ i.e. from the Service being unable to access potentially life-saving intelligence from data that it has already collected.”

A followup report from a month later echoed these concerns:

“There is an imbalance between collection and exploitation capabilities, resulting in a failure to make effective use of some of the intelligence collected today,” the report noted. “With the exception of the highest priority investigations, a lack of staff and tools means that investigators are presented with raw and unfiltered DIGINT data. Frequently, this material is not fully assessed because of the significant time required to review it.”

This isn’t just an MI5 problem. And it’s not just a bulk surveillance problem. GCHQ uses the same “data broker” — a program called PRESTON, run by the National Technical Assistance Center, which is supposed to act as a go-between for intelligence agencies in order to prevent the siloing of data. But it doesn’t work. It has prevented agencies from walling each other off, but the info firehose is still too much for agencies to handle — even with more-targeted surveillance.

Targeted collections fare little better than the bulk collections, in terms of needle location. The following chart shows how much data goes unutilized in cases where suspects are known and targeted with individualized warrants.

From the 2009 report detailing these problems:

[I]n one six-month period, the PRESTON program had intercepted more than 5 million communications. Remarkably, 97 percent of the calls, messages, and data it had collected were found to have been “not viewed” by the authorities.

Despite this failure to fully use the collections they already have, UK intelligence agencies are asking the government to give them more, supposedly to capture the “growing range of services available to internet users.” But they’ve already shown they can’t handle the data they already collect. Piling more hay on the stacks is only going to allow more “needles” to escape the attention of analysts. One solution would be to throw more people at the problem, but even this might cause its own issues, as every intelligence agency is increasingly wary of becoming the temporary home of the next Snowden. Vetting procedures have ramped up and security clearances have been scaled back.

More judicious collections are a better answer, but intelligence agencies — just like many of the internet users they track — apparently suffer from Fear of Missing Out. But an examination of their current collections/analysis shows they’re “missing out” already, and expanded powers/collections would do nothing to better secure the nation. In fact, these reports show the more they collect, the less they know.

Filed Under: , , , , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Snowden Docs Show GCHQ, MI5 To Be All Haystack, No Needle”

Subscribe: RSS Leave a comment
Anonymous Coward says:

Re: Re:

This, this, this! It was never about anything else other than this!

Throughout history the old ploy of taking your liberty to fight a foreign enemy has been used again and again to GREAT effect!

The law can never be a mechanism of prevention. Only retribution, which a lot of people mistake as justice. Law enforcement has no reason or requirement to do a damn thing for you when you are fucked over by a criminal.

Real justice is a criminal being shot the fuck down by a 2nd Amendment citizen the moment they try to molest someone or something!

Anonymous Coward says:

Re: Re: Re:

The FBI’s infamous ‘director for life’ J. Edgar Hoover kept files on all his potential adversaries, mainly for the purpose of blackmail, but also prosecution if and when the threat of blackmail failed to keep them in line.

That’s why the people who run the nation’s spy agencies have the tools on hand to become the most powerful in the country, whether they choose to sit as president (like Russia’s Putin) or prefer to remain as head of the agency (like Hoover).

Uriel-238 (profile) says:

Re: Put in Harlequin Romance Novel Terms...

Brad the Handsome High-ranking US official fancies you, but you’re married to a good man, Chaz. So Brad digs up some NSA dirt on Chaz to get him arrested. Conviction comes swift.

Chaz goes to the nice jail. There he will stay so long as you play nice with Brad. You’ll be his guest. You’ll accept his gifts and entertainment, you’ll let him prove that he’s ten times the lover Chaz is.

Don’t cooperate and Chaz goes to the secret cells in Supermax facilities where people cease to exist. (Yes, we have those. It’s where we’re sending Camp Delta guys.)

You have to be strong for Chaz, but if you resist Brad too much, you may lose Chaz. Oh the sturm und drang!

Anonymous Coward says:

These programs were never about terrorism.

NSA Surveillance Is about Power, Not “Safety”
– by Edward S.

“American Senators tell us that [we] should not worry, because this is not “surveillance,” it’s “data collection.” They say it is done to keep you safe. They’re wrong. There is a huge difference between legal programs, legitimate spying, legitimate law enforcement — where individuals are targeted based on a reasonable, individualized suspicion — and these programs of dragnet mass surveillance that put entire populations under an all-seeing eye and save copies forever.

These programs were never about terrorism: they’re about economic spying, social control, and diplomatic manipulation. They’re about power.

Wyrm (profile) says:

Re: These programs were never about terrorism.

That reminds me of France, some years back: they passed a law for the sole purpose of changing in all existing laws any instance of the word “videosurveillance” into “videoprotection”.
Not changing anything to make it fit to a “protection” purpose, just changing the word so they can pretend that any new law on videosurveillance is about “protecting the public”.

That is not anything new, just the most obvious case of “changing the words, not the actions” I’ve ever seen. Subtle as a 30-ton truck driving through your garden.

Chuck says:

Math, bitches

It should be noted that the fact that they haven’t viewed 97% of 5 million records also indicates they HAVE viewed 3% of them, which is 150,000 records.

The thing is, let’s say they’re all text messages. We’re talking no more than 140 characters each. Even reading at first grade level, most people can read a single text in what, like 10 seconds?

Now, there’s 28,800 seconds in an 8 hour work day (because you know government types never work more than 8 hours. Like, ever.) That’s roughly 2,500 messages per day, given several minutes for government mandated smoke breaks (because you know government types never pass up a smoke break, even if they don’t, yanno, smoke.)

At this rate ONE SINGLE FIRST GRADER could read all of the messages they collected in a 6 month period in 60 days.

How in the hell, exactly, can multiple agencies, full of full grown adults, most if not all with a college degree, not manage to achieve better than what a lone first grade student could do?

Math, bitches. –Chuck

Anonymous Coward says:

Re: Math, bitches

Actually the thing is the authorisation: They are not allowed by law to look at the rest.
That is also why they squirm at more laws to hold them back: Laws are not stopping mass collection, it is encouraging mass selection. That some high rankers dont get it and are asking for greater piles only makes the logical issue even worse.

The databases are also favourable for ministries. As long as the people committing heinous acts are buried there, the president can claim that the intelligence agencies had the person in their data and thus can avoid the brunt of the witch-hunt from the warhawks.

So “collect it all” is an easy way to protect politicians and the agencies from difficult targeting and prioritizing. The front is easy for the agency as it gets an easy excuse to ask for more money and less supervision. Something even politicians can formulate in the 30 second pitch test.

ECA (profile) says:

I SAID it long ago

The Amounts of DATA..
The number of CHATS
The number of Forums..
The number of GAMES with chats..
The ways and means to HIDE information/scrambled/encoded/what ever..

The CIA tried this ONCE…and in 1 day, filled a 10×10 foot room 4 feet high..
Even with an inter-relational data base and search engine..ITS ALLOT OF DATA..

You cant search a HAYSTACK until you get a GOOD MAGNET..

FM Hilton (profile) says:

Lack of understanding

This is why the NSA built a huge processing/data center out in Utah-so they can collect all the information they think they need and keep it ‘in case of’, which is something they haven’t figured out when to apply it.

Of course it’s about power-but the data is not going to be good forever, you it’s useless ultimately to even dare to collect it.

Sure, knowledge is power, but in this case it leads to total insanity of not knowing exactly why.

Kind of like those hoarder-type people, with their stacks of newspaper all over the house-they think they’re important for some reason, even though the data is old, outdated and buried under tons of the same thing.

It just doesn’t make sense anymore. So using outdated and senseless data makes them happy? Go for it.

Whatever says:

Had to log out again and find another IP address because Masnick keeps moderating my posts. Now the thing is, even if you did disagree with the government, which I do, you would still need to punish Snowden for endangering national interests. He chose to dump all the data for everyone to see, so all that evidence should be considered tainted. Once all of the “gotcha!” moments have been removed you can see that any “threat” this surveillance might have posed was negligible at best. The government is not going to care which Hollywood movie you’re downloading, but I understand if Techdirties are a little scared that their pirate protections aren’t helping them.

Sorry, Masnick, but your savior Wyden can’t help you now.

Uriel-238 (profile) says:

"punish Snowden for endangering national interests..."

We’ve had this conversation?

You mean the interests of the people, which is to stop agencies from engaging in surveillance of the masses? I think that’s exactly what Snowden was trying to prevent.

Or do you mean agency interests? Interests of elected government officials? The rare monied interests that put them there? Or do you believe that what’s good for General Motors is good for America?

I think Snowden served American interests just fine. Just not the American interests that you prefer were served.

PS: Masnick isn’t moderating you. We do. But have a flag for Techdirties.

BernardoVerda says:

Mullah Nasruddin and the Lost Key

Mullah Nasruddin and the Lost Keys

Late one evening, the famous folk-tale hero and “wise fool”, Sufi master Mullah Nasruddin, was on his hands and knees searching for something under a streetlamp. A passer-by saw him and asked, “What are you looking for?” “My house key,” Nasruddin replied. “I lost it.”

The man joined him in looking for the key, and after some time had passed without any success, the helpful fellow asked, “Are you sure you lost it around here?” Nasruddin replied, “Ummm, no… I lost it over there, by my house.”

“Then why in God’s name,” the man asked, “would you ever be looking for it over here?” Nasruddin promptly explained, “because the light is so much better over here, of course.”

Whatever says:

Re: Re: Re:

I log in all the time. Sadly, Techdirt is totally into censorship, holding all of my comments for moderation and making them entirely worthless.

Techdirt uses a form of prior restraint to keep my comments out of the public eye, only adding them to the site once the article is no longer in the first couple of pages. It’s funny as hell to watch them do it, it’s so insanely two faced.

Anonymous Coward says:

Re: Re: Re:2 Re:

You trolls don’t seem to get that if you’ve had a proven history of spamming and/or trolling the site, anything you say is passed through algorithms to react in kind. And now you’re mad because people aren’t reading your sad conspiracy theories over Wyden?

In the same article, you switch IP addresses and spam. Just who are you kidding?

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...