GAO Report: TSA Has No Idea How Effective Its Suspicionless Surveillance Program Is

from the results-to-be-reported-the-moment-there-are-actually-any-results dept

The TSA's "Quiet Skies" program continues and it doesn't appear to be making flying any safer. The program first exposed last year by the Boston Globe involved the surveillance of travelers for doing things like looking in shop windows or changing direction while walking through airports.

None of the people surveilled were on any terrorist watchlists. According to the TSA, it was hoping to find "unknown terrorists" by using a broad list of "suspicious" behavior to subject a greater number of travelers to additional screening and the apparent company of a flying air marshal (FAM).

The TSA thought it was great. The air marshals tasked with surveilling random people thought it was a waste of time and resources, if not an unconstitutional use of their powers. Backlash from the public and the air marshals themselves led the TSA to curtail the program. It promised not to surveill people for engaging in normal behavior the TSA had unilaterally deemed suspicious. There was also evidence the program was completely useless, as none of the 5,000 people targeted by "Quiet Skies" over a 6-month period in 2018 had gone on to do anything that air marshals deemed suspicious or worthy of further scrutiny.

The program lives on, unfortunately. The TSA may have scaled back its long list of "suspicious" behaviors, but it's still subjecting an unknown number of travelers to additional screening and surveillance, even if they're not tied to known terrorists or anyone on the government's multiple terrorist databases.

And it still doesn't work. It's still operational, I suppose, but the Government Accountability Office says the TSA doesn't know whether the program is effective. The program -- which has been running for nearly a decade now -- still hasn't been examined by the TSA to see whether it's actually doing anything to improve air security. From the report [PDF]:

We found that TSA coordinates reviews of Silent Partner and Quiet Skies through quarterly meetings and notifies an expanded set of DHS and TSA stakeholders—including DHS Traveler Redress Inquiry Program and the Federal Air Marshal Service—of rule changes as required under the Act. We also found that TSA has not identified a means to comprehensively measure rule effectiveness.

The TSA's reason for not following up on the program's effectiveness? It's just too much work.

TSA officials explained that they had not yet fully assessed the rules’ effectiveness because it was difficult to measure.

Difficult? Maybe. Impossible? No, says the GAO.

TSA has access to data—such as the outcomes of enhanced screening of Silent Partner and Quiet Skies passengers at airport checkpoints—that could be explored to better assess rule effectiveness. Exploring additional data sources could help TSA refine and supplement the agency’s existing efforts to measure program effectiveness.

The TSA may move forward with the GAO's recommendation it start actually measuring effectiveness. Then again, it's spent nine years not measuring it, so any progress will be on the same timescale.

TSA is currently reviewing the draft report and is scheduled to provide any comments by early November 2019.

At which point the TSA is free to reject the GAO's suggestion and continue to perform presumably useless screening and surveillance for the rest of forever.

That's not the only thing the TSA isn't following up on. The agency likes spending tax dollars on new security gear, whether it's needed or not.

Our review of TSA acquisition documents found that TSA considers risk at the beginning of the screening technologies acquisition process. However, TSA officials could not provide an example of when risk information for specific airports had directly influenced decisions about where and in what order to deploy screening technologies to airports in the recent past.

Then the agency deploys the tech and promptly forgets about it.

We also found that TSA does not ensure that screening technologies continue to meet detection requirements after they have been deployed to airports, when performance can degrade over time.

The TSA's process involves certifying the equipment prior to deployment and periodic calibration to ensure the tech is in "minimally operational." The GAO says this misses the point: the tech should be tested more frequently and, more importantly, assessed periodically to see if the deployed equipment still addresses the perceived risks present at that location as those may also change over time.

The TSA is mostly a reactive agency, one that responds to each attempted attack it's failed to thwart by subjecting travelers to another layer of screening annoyance. Its proactive efforts -- "Quiet Skies" and behavioral detection teams -- are somehow even worse. The TSA doesn't know what it's looking for because it hasn't prevented a terrorist attack yet. Its shotgun approach to screening and surveillance isn't likely to head off future attacks but the TSA seems content to settle for collateral damage.

Filed Under: airport security, gao, quiet skies, surveillance, tsa


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Gary (profile), 5 Nov 2019 @ 10:10am

    Still Trying

    Who needs Evidence if you have Faith?

    reply to this | link to this | view in chronology ]

    • identicon
      Rocky, 5 Nov 2019 @ 11:39am

      Re: Still Trying

      Who needs Evidence if you have Faith?

      What is truly mind-boggling is how large a percentage of the population that actually reasons that way.

      reply to this | link to this | view in chronology ]

      • icon
        rangda (profile), 5 Nov 2019 @ 11:54am

        Re: Re: Still Trying

        What is truly mind-boggling is how large a percentage of the population that actually reasons that way.

        The only thing mind boggling is that more people don't do it. Faith is the lazy person's way out. If you are in a situation you don't like, faith lets you do nothing and expect it to change. If things don't work out, it gives you an out to avoid looking at your own personal faults. It lets you justify actions that deep down you know are morally wrong.

        It also allows you to be easily manipulated and controlled by those associated with the object of your faith, but that's more of a benefit for people who manipulate the faithful rather than the faithful themselves.

        reply to this | link to this | view in chronology ]

    • icon
      bhull242 (profile), 6 Nov 2019 @ 8:37am

      Re: Still Trying

      Faith is apparently enough to provoke suspicion from the TSA.

      reply to this | link to this | view in chronology ]

  • icon
    aerinai (profile), 5 Nov 2019 @ 10:20am

    Security Theater is subjective by nature

    This makes people safer as much as they believe it makes them safer... So for the GAO to ask for hard numbers must be really appalling to the directors of the play!

    All the world's a stage, and all the TSA men and women merely players: they scan the luggage and the liquids; and one man in his time frisks many parts, his actions lawful and lauded.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 5 Nov 2019 @ 10:56am

    I used to walk through airports very conscious that I was being monitored. I'd only glance in the stores if I was looking for something, and only go in if I actually planned to buy (or at least seriously look at) something. I practiced never doubling back, but always keeping up the "policeman's stroll" when wandering a terminal waiting for my plane to arrive.

    Then at some point I decided it would be more interesting to see what actually happens if I do the suspicious stuff, as they wouldn't actually have enough to grab me off my flight and interrogate me / stick me on some long-term watch list. So on a few flights where I wasn't in a hurry, I started doubling back, loitering near garbage cans, putting my bag down and then picking it up again, looking directly at video cameras, etc.

    End result? On one of those flights, I picked up an Air Marshall tail, at which point I calmly wandered back to my gate and sat down for the rest of the wait. For the rest -- nothing.

    And it seems to me that if I could do both of these things, so could people with malicious intent: just set someone up as the target, bumbling about the terminal. Then all eyes are on them, leaving the actual terrorist/smuggler/etc. to calmly walk in to the terminal, do what they came to do, and carry on as normal.

    Maybe now that facial recognition is implemented and the background video processing is much more automated they could actually get some wins from this sort of surveillance -- but I doubt it; it would still be just too much work.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 5 Nov 2019 @ 12:09pm

      Re:

      leaving the actual terrorist/smuggler/etc. to calmly walk in to the terminal, do what they came to do, and carry on as normal.

      It's not the TSA's job to watch for smugglers—only for safety-related concerns like terrorists, who basically don't exist at airports anyway.

      reply to this | link to this | view in chronology ]

    • icon
      Scary Devil Monastery (profile), 7 Nov 2019 @ 7:13am

      Re:

      "Maybe now that facial recognition is implemented and the background video processing is much more automated they could actually get some wins from this sort of surveillance -- but I doubt it; it would still be just too much work."

      Assuming it works which is unlikely. The false positives ensure that with every thousand individuals who get scanned at least one, probably more, will suddenly get linked to a "Kill on Sight" warning issued by some agency or other. At random.

      That, essentially, is why EVERY tech expert not actively vested in trying to sell this type of tech is adamant that this type of tech brings no good and much harm.

      reply to this | link to this | view in chronology ]

  • identicon
    bob, 5 Nov 2019 @ 10:58am

    thanks a lot Osama :(

    Just look at all the time and money wasted on the TSA.

    We dont need any more attacks in the United States, we hurt ourselves more than any terrorist ever could.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 5 Nov 2019 @ 12:22pm

      Re: thanks a lot Osama :(

      Osama is a red herring here, merely the excuse to tighten the government grip on US citizens. Much like any bill pushed with some flavor of "but think of the children!". All bullshit.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 5 Nov 2019 @ 11:01am

    Hmmm I think just about everything about the TSA is supicious.

    Can we create an federally funded agency to harrass TSA agents as they enter the airport, detain then when ever they feel like it, and stalk them? It would be called TTSA, TSA security Agency. Of course it's activities wouldn't be limited to that, that's just all I could be bothered to type right now (an attitude that should permeate the new agency as well).

    reply to this | link to this | view in chronology ]

  • icon
    tom (profile), 5 Nov 2019 @ 11:09am

    Not sure anything the Terminally Stupid Agency had done has made flying safer.

    reply to this | link to this | view in chronology ]

  • identicon
    Michael, 5 Nov 2019 @ 11:33am

    TSA officials explained that they had not yet fully assessed the rules’ effectiveness because it was difficult to get additional funding after admitting the program was useless.

    There, fixed that for you.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 5 Nov 2019 @ 5:04pm

      Re:

      because it was difficult to get additional funding after admitting the program was useless.

      I say we should replace them with a Ministry of Silly Walks. It was said to cost £348,000,000/year in 1970, roughly equivalent to £5,500,000,000/year or $7,100,000,000/year today. The TSA, by contrast, takes $7,780,000,000/year. Neither accomplishes any important task, but we'd save money and make people happier—safer too, because several thousand people (one 9/11) per year die avoiding the TSA.

      reply to this | link to this | view in chronology ]

  • icon
    Norahc (profile), 5 Nov 2019 @ 3:45pm

    TSA = The Stoopid Agency

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 5 Nov 2019 @ 7:16pm

    It's reasonable they don't know how effective the program is. They need a sufficiently large sample size in order to get a valid result.

    reply to this | link to this | view in chronology ]

    • icon
      bhull242 (profile), 6 Nov 2019 @ 8:54am

      Re:

      Considering the number of agents and the number of people who use airplanes, don’t they have a pretty large sample size to work from?

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 6 Nov 2019 @ 9:15am

        Re: Re:

        They need terrorists, drug dealers, smugglers, ect as the samples. Normal people who aren't known before or after the fact to be one of those things is throw away data.

        reply to this | link to this | view in chronology ]

        • icon
          Scary Devil Monastery (profile), 7 Nov 2019 @ 7:18am

          Re: Re: Re:

          "They need terrorists, drug dealers, smugglers, ect as the samples. Normal people who aren't known before or after the fact to be one of those things is throw away data."

          The reason the program was evaluated as useless is because any algorithm with ANY percentage of error has a scary tendency to identify those normal people as terrorists and other types of incredibly dangerous people.

          That's how false positives work - a 99% accuracy rate doesn't just mean that the program identifies 99 out of a 100 terrorists. It also means 1% of all the NORMAL people to get scanned ALSO identifies as terrorists.

          I'm pretty sure the TSA knew full well the program was a bust when their shiny new tech insisted they had a hundred horrible criminals on their airport on an average jam-packed day.

          reply to this | link to this | view in chronology ]

  • identicon
    Dr evil, 6 Nov 2019 @ 8:40am

    Did any think to ask

    If the TSA actually CARES what the results are or what the efficiency is? LOL

    reply to this | link to this | view in chronology ]

  • icon
    bhull242 (profile), 6 Nov 2019 @ 8:51am

    A reminder of how BS the TSA’s “behavioral detection” is

    Here is the TSA’s list of indicators for female suicide bombers:

    Single, married, mothers or grandmothers

    i.e. all women

    High school and college students
    Working professionals such as lawyers, journalists, or medical
    practitioners

    So, being a young adult or having a profession is suspicious. That basically just eliminates infants, children, the unemployed, and nonprofessional workers. Also, we have journalists being singled out, which isn’t really consistent with a free press.

    Devout or non-devout in religious beliefs

    So basically anyone with religious beliefs, which makes this religious discrimination. And while I can maybe see religious devotion as an indicator, why would not being devout be suspicious?

    Intelligent, charming, and attractive

    So anyone you want to marry or fuck is suspicious?

    Very active in their cause

    This is honestly the only one (other than being devout in religious, though that has its own problems) that I could see as a potential indicator. But it’s incredibly vague about what causes would be included. What if someone’s cause is opposition to war and violence?

    reply to this | link to this | view in chronology ]

    • icon
      Wyrm (profile), 6 Nov 2019 @ 9:58am

      Re: A reminder of how BS the TSA’s “behavioral detection”

      What if someone’s cause is opposition to war and violence?

      Given the atmosphere in the US, that might be the most suspicious behavior in the whole list.

      (To /s or not to /s. I'm not sure.)

      reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Techdirt Gear
Show Now: Takedown
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.