The NSA Reveals That It Does 20 Million Database Queries Per Month

from the that's-a-lot dept

As we noted earlier today, the NSA's two key "defenses" of the thousands of abuses and violations of the law that recently came out thanks to a leaked document are that there wasn't "intent" to abuse the system (we had no idea that made illegal things legal...) and, second, that it was such a small percentage of the activity that it's really no big deal. Glenn Greenwald quickly noted that the NSA is actually saying "we collect billion of emails and calls every day, so what's a few thousand privacy violations?" hoping that everyone focuses on the second half of the sentence. But the key point is actually the first half of that sentence. In fact, as we noted in that last post, the NSA's top compliance guy actually revealed a startling fact in his attempt to push the meaningless "ratio" of violations to queries:
The official, John DeLong, the N.S.A. director of compliance, said that the number of mistakes by the agency was extremely low compared with its overall activities. The report showed about 100 errors by analysts in making queries of databases of already-collected communications data; by comparison, he said, the agency performs about 20 million such queries each month.
Again, the ratio is a meaningless number. You're not declared innocent of murder because you didn't happen to murder someone every other day of your life. But, perhaps more important in this is the revelation of the 20 million queries every single month. Or, approximately 600,000 queries every day. How about 25,000 queries every hour? Or 417 queries every minute? Seven queries every single second. Holy crap, that's a lot of queries.

Remember, too, that the NSA has insisted that it doesn't datamine its data collection, which is clearly hogwash. That many queries means they're trolling through that database all the time. Remember how the NSA was trying to play down how often it did queries by saying that only 300 phone numbers had been used to "initiate" a query? Yeah, well, once again, it would appear that the NSA was not being fully forthcoming about these sorts of things. Shocking, I know, but I'd imagine they'd claim it was the "least untruthful" answer they could come up with after having a good week or so to answer the question.


Reader Comments (rss)

(Flattened / Threaded)

  1.  
    icon
    Ninja (profile), Aug 19th, 2013 @ 9:47am

    Well, if you consider Snowden came from the NSA and he's letting all this in the wild then the NSA is indirectly telling the truth. O.o

    And some of the basic infra-structure of the web is in American soil. Sure this mass surveillance on Americans is worrisome for the Americans themselves but shouldn't the rest of the world be worried too?

     

    reply to this | link to this | view in thread ]

  2.  
    icon
    Zakida Paul (profile), Aug 19th, 2013 @ 10:20am

    Well, to be honest that's not a lot considering the US has over 300 million terrorists inside it's borders.

     

    reply to this | link to this | view in thread ]

  3.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:31am

    SELECT * FROM Public_Data_All
    --WHERE IsTerrorist = 1

     

    reply to this | link to this | view in thread ]

  4. This comment has been flagged by the community. Click here to show it
     
    identicon
    out_of_the_blue, Aug 19th, 2013 @ 10:33am

    How many of those use "direct access" to Google's servers?

    Which is worse: NSA or the collaborating corporations?

    And which spying corporation is the worst? Google, Facebook, Microsoft, Yahoo, Apple...

     

    reply to this | link to this | view in thread ]

  5.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:35am

    'only 300 phone numbers had been used to "initiate" a query'

    Well, if those 300 people each called 41 people, and those all called 41 people, and those all called 41 people, we're already over 20 million.

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:36am

    Re:

    Where is the mapreduce of that one?

     

    reply to this | link to this | view in thread ]

  7.  
    icon
    Zakida Paul (profile), Aug 19th, 2013 @ 10:39am

    Re: How many of those use "direct access" to Google's servers?

    "Which is worse: NSA or the collaborating corporations?"

    Two sides of the same coin. The NSA uses the private corporations to harvest data on the population.

    Corporate spying alone is way down the list because people have the choice as to whether or not they want to use that company's services.

    I don't know why I bothered because you will inevitably ignore what I said and continue with your anti corporation rants.

     

    reply to this | link to this | view in thread ]

  8.  
    icon
    RyanNerd (profile), Aug 19th, 2013 @ 10:39am

    Holy Crap that's quite a bit of data mining

    I worked for a health insurance company. And on occasion we would have to query the raw data for a patient's prescription history. We only did this when an issue came up or we were performing an audit. Even then there were strict guidelines for how we were to perform a search. This was to protect the privacy of everyone.
    For example: If we had a first and last name but no social security number. We were forbidden from doing a search unless we also had their zip code of residence. The searches were to be exact and targeted as possible; which if you care about privacy is how things should be.
    This broad untargeted data-mining by the NSA is astounding. Makes me sick.

     

    reply to this | link to this | view in thread ]

  9.  
    identicon
    Michael, Aug 19th, 2013 @ 10:42am

    Not to nit pick, but the number of queries does not necessarily tell you much. Many people are thinking "select * from phonecalls where country <> 'US'", but that's not how all databases work.

    Some data analytics systems will run thousands of queries to satisfy a single request.

    They certainly should not be using the total volume of queries to try to minimize the number of abuses, but at this point, they are simply throwing out numbers that are meaningless without context.


    We should all be focused on the simple number of times they have abused their power or broken the law. Trust me, the number of times we were not speeding does not matter one bit when it comes to defending the time the radar gun got us.

     

    reply to this | link to this | view in thread ]

  10.  
    icon
    RyanNerd (profile), Aug 19th, 2013 @ 10:42am

    Re:

    SELECT * FROM Public_Data_All
    WHERE last_name like 'Snowde%'

    FTFY

     

    reply to this | link to this | view in thread ]

  11.  
    identicon
    NSA, Aug 19th, 2013 @ 10:44am

    don't crossreference our statements!

    thats our job, we do millions every month and are damn good at it.

     

    reply to this | link to this | view in thread ]

  12.  
    identicon
    Michael, Aug 19th, 2013 @ 10:45am

    Re: Holy Crap that's quite a bit of data mining

    It's a bit more like data-strip-mining.

     

    reply to this | link to this | view in thread ]

  13.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:45am

    Re: How many of those use "direct access" to Google's servers?

    You forgot the companies of the industrial military complex.

    Corporations --------> $$$$$$$$ ---------> Government Officials

    Government Officials --------> Beneficial Laws ---------> Corporations


    Google, Facebook, Microsoft, Yahoo, Apple etc... are pretty much victims and are just complying with these new "Laws".
    It's the Corporations that are developing/supplying Tech and services for the NSA that are the ones making the laws via bribery.

     

    reply to this | link to this | view in thread ]

  14.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:50am

    The worrying problem is that they have the data available to allow them to make mistakes. This can only increase the pressure to allow other law and security services to access the data increasing the opportunities for mistakes and abuse.

     

    reply to this | link to this | view in thread ]

  15.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 10:51am

    Re:

    SELECT * FROM Public_Data_All
    --WHERE IsTerrorist = 1


    Commenting out the where clause explains why they think every American is a terrorist.

     

    reply to this | link to this | view in thread ]

  16.  
    icon
    Uriel-238 (profile), Aug 19th, 2013 @ 10:57am

    If the Bacon numbers are a good indicator.

    That vastly improves the chances that they're reading my personal email.

    And yours too.

    I suspect everyone is few enough degrees from a terrorist suspect to be worthy of inclusion in search queries.

     

    reply to this | link to this | view in thread ]

  17.  
    identicon
    Michael, Aug 19th, 2013 @ 10:57am

    Re:

    The NSA does not make mistakes. They simply expand their searches to other relevant data.

     

    reply to this | link to this | view in thread ]

  18.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 11:00am

    Corporate money out of politics

    Being anti-corporate is not exactly irrational. Partially misdirected blame but even if a person is completely "anti-corporation", is it really that bad of an ideology ?

    OBLIGATION of business to make profit disregarding any societal cost.
    Without regulation and laws then growing people to be slaves would be legitimate pro-corporation ideology.
    It's the laws and regulations that need to be the focus.

    By law, google are not allowed to grow slaves.
    By law, google are required to hand over all access to NSA.
    Who is funding the American Laws ?
    Corporations.

    So being anti-corporation is actually legitimate.
    Being anti corporate funding of politics is smarter.


    Until the money is out of politics, there will be a valid reason for people to be anti-corporatist.

     

    reply to this | link to this | view in thread ]

  19.  
    identicon
    Michael, Aug 19th, 2013 @ 11:02am

    Re: If the Bacon numbers are a good indicator.

    Are you calling Kevin Bacon a terrorist?

    Well, I suppose all of that anti-establishment dancing could raise some red flags these days.

     

    reply to this | link to this | view in thread ]

  20.  
    icon
    RyanNerd (profile), Aug 19th, 2013 @ 11:05am

    Re:

    By no stretch of the imagination does 20 million queries per month == targeted information gathering. Plain and simple this is indiscriminate data-mining. There is no legitimate argument to the contrary.

     

    reply to this | link to this | view in thread ]

  21.  
    icon
    That Anonymous Coward (profile), Aug 19th, 2013 @ 11:15am

    Re: Re: If the Bacon numbers are a good indicator.

    HE IS!
    There was a law against dancing, and he violated it and lead others to violate it as well!

     

    reply to this | link to this | view in thread ]

  22.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 11:17am

    So I'm getting between 7 and 8 queries per second, there. Assuming around-the-clock querying.

    Between 34 and 35 per second if the querying is only during business hours, where 20 business days defines a "month".

     

    reply to this | link to this | view in thread ]

  23.  
    identicon
    RD, Aug 19th, 2013 @ 11:17am

    Re: How many of those use "direct access" to Google's servers?

    "Which is worse: NSA or the collaborating corporations?"

    Fucking idiot. The fact that you even have to ask that shows your bias and insane focus on Google to the exclusion of all else.

    The NSA is FAR worse than any corporate collaborator, because the NSA has the full weight of the government and law behind it.

    A corporation can't arrest you or put a gun to your head or order you murdered via drone strike.

     

    reply to this | link to this | view in thread ]

  24.  
    icon
    ArkieGuy (profile), Aug 19th, 2013 @ 11:25am

    One query for 3 out of 4 Americans every year

    Another way to think of this is the NSA is doing 3 queries for every 4 people in the US.

     

    reply to this | link to this | view in thread ]

  25.  
    icon
    Ninja (profile), Aug 19th, 2013 @ 11:44am

    Re:

    We should replace some commonly used word for terrorist for the lulz. Ie: reader

    Techdirt terrorists often discuss about tech stuff. According to these terrorists technology can make carrying their daily terrorism (reading) tasks easier. Filter gone haywire!

     

    reply to this | link to this | view in thread ]

  26.  
    icon
    Ninja (profile), Aug 19th, 2013 @ 11:47am

    Re:

    This is a matter of semantics. Are they using queries in the technical sense (and then you would be right while not making their abuse any better) or in the common sense of the word where they do search 20 million times (thus the number of technical queries is way higher) which makes their abuse sound even worse... Now, which definition is the NSA playing by?

     

    reply to this | link to this | view in thread ]

  27.  
    icon
    GoldHoarder (profile), Aug 19th, 2013 @ 11:51am

    Re: Re: How many of those use "direct access" to Google's servers?

    Corporations --------> $$ ---------> Government Officials

    Government Officials --------> Beneficial Laws ---------> Corporations------> Hire Government Officials upon leaving office or positions for $$$$$$$$$$$$$$

    Fixed

     

    reply to this | link to this | view in thread ]

  28.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 12:22pm

    Re: Re:

    Some other definition that neither common folk nor database administrators would ever dream of.

     

    reply to this | link to this | view in thread ]

  29.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 12:26pm

    Re:

    And how long before ICE is using to assist their funders, sorry clients, sorry (what are they?) in the music and movie industries in sending out shakedown, sorry settlement letters?

     

    reply to this | link to this | view in thread ]

  30.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 12:51pm

    Re:

    SELECT * FROM PublicData
    WHERE Exists = 1

     

    reply to this | link to this | view in thread ]

  31.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 1:01pm

    How is this not illegal searching through illegally seized information.

    That's not a question, it's a statement. If anyone does have any questions, I refer you to the Constitution, Amendment 4.

     

    reply to this | link to this | view in thread ]

  32.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 1:06pm

    Why break the local laws? Should just share.

    What I don't understand is why all the government allies don't just work together and save themselves some huge embarrassment and political ire. If I had put this together I would have our spies force US companies to hand over our info to the UK spies. The UK spies have their companies giving us their info. We could then both LEGALLY spy on the others' citizens and share all their findings back with the respective governments legally. Seems so simple.

    Everything I needed to know I learned in kindergarten.

     

    reply to this | link to this | view in thread ]

  33.  
    identicon
    Anonymous, Aug 19th, 2013 @ 3:53pm

    In the words of Bonnie Raitt...

    Let's give 'em something to talk about.

     

    reply to this | link to this | view in thread ]

  34.  
    identicon
    Anonymous Coward, Aug 19th, 2013 @ 4:41pm

    The abuses come from LOGGED data, the biggest abuses are in the UNLOGGED queries.


    Do you really think there is no skunkworks version of that app that doesn't log queries? Because I think that's naive.

     

    reply to this | link to this | view in thread ]

  35.  
    identicon
    Drew, Aug 19th, 2013 @ 8:17pm

    Re:

    That's about .002% of the population?

    Er.. I need to stop posting stupid shit like this before the NSA decides to recruit me as an numbers analyst.

     

    reply to this | link to this | view in thread ]

  36.  
    identicon
    Anonymous Coward, Aug 20th, 2013 @ 7:59am

    So Much Open Resistance

    It seems incredible at the amount of open resistance to tactics of Security agencies here at Techdirt, given the fact that we are all still speaking English and not some ancient Chinese or Mesopotamian dialect..

     

    reply to this | link to this | view in thread ]

  37.  
    icon
    nasch (profile), Aug 20th, 2013 @ 12:51pm

    Re:

    Do you really think there is no skunkworks version of that app that doesn't log queries? Because I think that's naive.

    I don't know, the NSA seems to have a compulsive need to keep track of everything. I'm not sure they would be willing to just not log something.

     

    reply to this | link to this | view in thread ]

  38.  
    identicon
    Anonymous Coward, Aug 20th, 2013 @ 2:51pm

    Re: Re: Holy Crap that's quite a bit of data mining

    Is data-strip-mining a form of strip search?

     

    reply to this | link to this | view in thread ]

  39.  
    icon
    Uriel-238 (profile), Aug 20th, 2013 @ 3:49pm

    The NSA compulsion to keep track

    I don't know, the NSA seems to have a compulsive need to keep track of everything. I'm not sure they would be willing to just not log something.

    I hope you're right. Thorough records will be convenient when rounding up those responsible for the program.

     

    reply to this | link to this | view in thread ]

  40.  
    identicon
    bb, Aug 20th, 2013 @ 10:09pm

    Of course they're obviously not lying

    LOL.

    Of course they're obviously not lying, and it's like 10 times that amount...or more...they never lie.

    fool me three times...derrr

     

    reply to this | link to this | view in thread ]

  41.  
    identicon
    gmathol, Aug 21st, 2013 @ 12:32am

    I remember the data ware house we build 10 years ago had to endure more queries than 20 million a month and those were build for business reporting.

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
A word from our sponsors...
Essential Reading
Techdirt Reading List
Techdirt Insider Chat
A word from our sponsors...
Recent Stories
A word from our sponsors...

Close

Email This