by Mike Masnick

Filed Under:
bill binney, nsa, nsa surveillance

Former NSA Whistleblower Bill Binney: The NSA Is Making Itself Dysfunctional With Too Much Data

from the analysis-paralysis dept

Former NSA whistleblower Bill Binney has been making the interview rounds in the wake of the NSA surveillance leaks, adding some useful context and history to some of the allegations. In one interview, with the Daily Caller, almost as an aside, he makes a really good point about how in the desire to collect all these haystacks of information, the NSA is missing out on a lot of needles. There's a discussion of how law enforcement apparently has a recording of a phone call between one of the Boston Marathon bombers, Tamerlan Tsarnaev and his wife, but that nothing was done with it. Binney points out that, when you have that much data, important stuff gets lost.
They're making themselves dysfunctional by collecting all of this data. They've got so much collection capability but they can't do everything. They're probably getting something on the order of 80 percent of what goes up on the network. So they're going into the telecoms who have recorded all of the material that has gone across the network. And the telecoms keep a record of it for I think about a year. They're asking the telecoms for all the data so they can fill in the gaps. So between the two sources of what they've collected, they get the whole picture.

They can do textual processing at a rate of about 10 gigabits a second. What that means is about a million and a quarter 1,000-character emails a second. They've got something like 10 to 20 sites for this around the United States. So you can really see why they need to build something like Utah to store all of this stuff. But the basic problem is they can't figure out what they have, so they store it all in the hope that down the road they might figure something out and they can go back and figure out what's happening and what people did. It's retroactive analysis.
Yes, it may be useful in hindsight (not that useful automatically makes it legal), but what would be even more useful is if they stopped focusing so much on collecting data, and went back to doing traditional investigative work that focuses on real targets. Piling on to the haystack doesn't help anyone. It comes from the faulty belief that piling more data on top -- even if it's useless data -- must be a good thing.

Reader Comments (rss)

(Flattened / Threaded)

  1. identicon
    Anonymous Coward, Jun 14th, 2013 @ 2:09pm


    They need to be on an episode of hoarders, they've obviously got a problem.

    reply to this | link to this | view in thread ]

  2. icon
    jupiterkansas (profile), Jun 14th, 2013 @ 2:16pm

    Re: hoarders

    The NSA is the ultimate hoarder. I imagine their building's full of cats, too.

    reply to this | link to this | view in thread ]

  3. identicon
    Anonymous Coward, Jun 14th, 2013 @ 2:19pm

    Biden in 2006 debates Obama in 2013

    reply to this | link to this | view in thread ]

  4. icon
    pixelpusher220 (profile), Jun 14th, 2013 @ 2:40pm

    Haystacks and other missions

    The ever increasing haystacks *may* cause needles to be missed - ff they're trying to process the haystacks in real-time. I'm not as convinced they are trying to do that.

    The 'haystacks' purpose is to be a time machine so they can go back to last year and figure out who you were talking to then, once they decide 'now' that you're suspicious.

    Different purposes. And frankly you can do both at the same time as long as you aren't trying to monitor the haystacks in real time, but use more traditional methods for ongoing investigations.

    reply to this | link to this | view in thread ]

  5. identicon
    Anonymous Coward, Jun 14th, 2013 @ 2:47pm

    They claim all this data has foiled "at least one" up to "several dozen" terrorist plots, but they have yet to produce any evidence that it was unequivocally impossible to foil those plots (however many of them there may or may not have been) without this information So far, the vast majority of the plots I've seen stopped, are the one's they themselves created.

    Furthermore, it's been well established they had all the information they needed to stop the 9/11 attacks, but nobody shared enough or connected enough dots. Nor did this massive mound of data do anything to stop the Boston bombing. I fail to see how adding billions of random and useless dots is going to make connecting the relevant dots easier when they couldn't do it in the first place.

    Sorry, but trading 50% of one's freedom for maybe a 1% increase on safety is simply not a practical or prudent business decision.

    reply to this | link to this | view in thread ]

  6. This comment has been flagged by the community. Click here to show it
    Anonymous Coward, Jun 14th, 2013 @ 2:49pm

    Nobody milks this NSA story more than Mikey Milkman Masnick!!

    Run away! Run away! Never discuss! Never build up! Always tear down! Always hate! Always FUD!

    reply to this | link to this | view in thread ]

  7. identicon
    AC Unknown, Jun 14th, 2013 @ 2:52pm


    You're pathetic enough to be a spammer.

    reply to this | link to this | view in thread ]

  8. identicon
    Anonymous Coward, Jun 14th, 2013 @ 2:55pm

    Preserve and expand the organization

    Like any large bureaucratic organization, the main function of the NSA is to maintain and expand the NSA. Effective and efficient intelligence gathering is one *mechanism* for doing so, not a goal. Other mechanisms might work better -- and a bigger budget will *always* appeal.

    reply to this | link to this | view in thread ]

  9. identicon
    Anonymous Coward, Jun 14th, 2013 @ 3:46pm

    Metadata calculations

    The math involved in analysing these huge piles of data can get pretty big pretty quickly. For a fun example of a very basic calculation on a relatively tiny data set, see:

    (Don't bother if you're scared of math or think matrix multiplication is witchcraft.)

    reply to this | link to this | view in thread ]

  10. identicon
    Anonymous Coward, Jun 14th, 2013 @ 4:34pm


    Lets puts the risk into perspective. In general, you are more likely to die in an accident in your own bath than in a terrorist attack. While this does not hold true for the year of 9/11, more people drowned that year than were killed in the attack.

    reply to this | link to this | view in thread ]

  11. identicon
    The loophole NSA uses when spying on US citizens, Jun 14th, 2013 @ 4:37pm

    [...] any assurance to US citizens that the NSA will not gather and archive their data is suspect. The “Five Eyes” alliance between the intelligence agencies of the US, Australia, Canada, New Zealand and the UK effectively permits those governments to circumvent the prohibition against gathering data on their own citizens by sharing information across the Five Eyes intelligence community. The UK for example can spy on Americans and make that information available to the US government on its massive spy cloud – one that the NSA operates and the Five Eyes share. [...]

    reply to this | link to this | view in thread ]

  12. identicon
    Anonymous Coward, Jun 14th, 2013 @ 5:57pm

    The boston suspects were framed. Once one gets targeted, a large machine comes into play that will kill or frame anyone who has made the target list. The reason the Boston plot wasn't stopped is because it was sponsored by the Feds themselves.

    There was a DHS "training exercise" planned to be put into effect that was supposed to be the EXACT same scenario, but somehow these brothers beat them to the punch. Funny how that is, isn't it?

    I have some expertise in this matter. I am supposed to be framed because of something innocent that happened to me about 20 years ago. I am not the first. I will not be the last.

    reply to this | link to this | view in thread ]

  13. identicon
    Anonymous Coward, Jun 14th, 2013 @ 6:00pm


    DHS exercise

    Operation Urban Shield

    reply to this | link to this | view in thread ]

  14. icon
    BearGriz72 (profile), Jun 15th, 2013 @ 6:59am


    Spam in a can?

    reply to this | link to this | view in thread ]

  15. identicon
    Anonymous Coward, Jun 15th, 2013 @ 9:54am

    Traffic Analysis

    As the reports clarify what the NSA is doing it sounds like traffic analysis. Basically in traffic analysis one follows the flow of communications between nodes. This gives very good ideas of structure and the relationships of the nodes to each other. Very good to know even if you do not know the content of the information.

    Backing up about 25 - 30 years, relatively few people communicated with regularly with someone in a foreign country. Problems with cost, time zones, and the lack of any foreign contacts were the primary reasons. So traffic analysis of international phone calls was relatively easy to do. The networks were simpler and only a few had to be monitored. In this era the vast majority of the traffic was no interest to the spooks.

    Today with social media especially it is quite possible for people to communicate with others worldwide. Compounding this is how easy it is for one to set an informational website to post instructions about doing whatever. The spooks in many ways appear to operating with a 1980 mentality. Thus, they assume they must monitor all the communication links. This will lead to them drowning in data because even a higher percentage of the traffic is inconsequential - as Jay Leno quipped "90% was ordering pizzas".

    The better procedure requires more work initially. That is to identify the websites and primary "command" nodes involved and monitor them. The identification will be harder but once done it can be assumed that a very high percentage of the traffic is from the terrorist or criminals.

    reply to this | link to this | view in thread ]

  16. icon
    Mega1987 (profile), Jun 15th, 2013 @ 12:04pm

    Brainwave overload

    now, well see if NSA can give it's own system a self-inflicted pernament BSOD due to memory shortage... by gigabits of it...

    Karma's a bitch!

    reply to this | link to this | view in thread ]

  17. icon
    RyanNerd (profile), Jun 16th, 2013 @ 11:59am

    Re: hoarders

    I wonder how many dead cats we would find in the giant horde of data at the NSA?
    I belive the record holder for the number of dead cats found is 13...

    reply to this | link to this | view in thread ]

  18. icon
    Tim Griffiths (profile), Jun 17th, 2013 @ 2:11am


    Wouldn't it be just all that much more comforting if that was the case? That it would have to be a government who had such power and control to pull this off rather than two random guys on the street? The idea that a small number of individuals with next to no resources can bring such horrific acts to fruition in the face of overwhelming surveillance is horrifying and as such as I can understand your need to find a 'true' actor behind it but life doesn't work like that. Horrible, random, things happen and trying to find order and patterns and reasons in them above and beyond the obvious "these men chose to kill people and found a way to do it" is an attempt to rationalise that fear but in doing so you are blinding yourself to the reality. The truth is government conspiracy exists but it's of the dull, obvious, type, banking, surveillance and we are aware of them. The NSA program was tripped up a contractor and you expect me to believe they could pull off something like this in total secrecy? Really?

    reply to this | link to this | view in thread ]

  19. identicon
    Anonymous Coward, Jun 18th, 2013 @ 2:42pm


    reply to this | link to this | view in thread ]

  20. icon
    AmericanGulagAuthor (profile), Oct 20th, 2013 @ 4:36pm

    Tips on terrorism by concerned Americans are Ignored

    If the NSA were really concerned about the safety of Americans, they would take reports made to the authorities by concerned Americans who have evidence into consideration. For example, many Americans warned the NYPD about terrorist plots to create a massive fire in NYC. If the NYPD had followed up on such reports, the twin towers would most likely be standing today. A pilot school in Florida also made reports that were ignored.

    It does not make sense that so much data is collected if it is not utilized to prevent, such as the recent attack in Boston's marathon. It concerns me because I am aware of the increasing pattern of abductions and kidnappings in the US, and I would not be surprised if there are thugs in the NSA who utilize the data for human trafficking purposes.

    There was a terrorist in Freddie Prinze, Sr.'s soup and there is a terrorist in the soup of all Americans. "Ms. Quixote Goes Country - Raised on the Marxist Frontier", a novel written by a concerned American.

    reply to this | link to this | view in thread ]

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Insider Shop - Show Your Support!

Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Report this ad  |  Hide Techdirt ads
Recent Stories
Report this ad  |  Hide Techdirt ads


Email This

This feature is only available to registered users. Register or sign in to use it.