Oversight Says FBI's Facial Recognition System Has Gotten Bigger, But Not Better

from the it's-not-like-the-FBI-is-some-tiny-underfunded-agency... dept

It appears the FBI's facial recognition program will never live up to the minimal expectations its oversight has placed on it. The FBI's database went live in 2014, far preceding the Privacy Impact Assessment that was supposed to be delivered in 2012.

Two years after its debut, the Government Accountability Office found the FBI's database -- which went live with a 20% failure rate -- was still a mess. The FBI showed little interest in improving the accuracy of its searches. It also showed little interest in periodically testing the system to see if it was improving or, quite possibly, getting worse.

The FBI's hands-off approach to facial recognition only applies to its oversight of the program. Otherwise, it's an enthusiastic participant. At the time of the GAO's examination, the FBI's database contained 411 million photos, drawn from both criminal and non-criminal databases. Indicative of the FBI's lackadaisical approach to facial recognition was a bank robbery case in Colorado, where the feds pitched in to help arrest the wrong person twice.

A year later, the House Oversight Committee noted nothing had improved since the GAO's 2016 recommendations. Input and output remained flawed, and the FBI still showed little interest in fixing the problems reported by the GAO.

Two years later, it's deja vu all over again. The GAO's latest report [PDF] says the only thing that's really changed is the size of the database. Since it's last assessment, the FBI has added 230 million photos, bringing the total to 641 million face shots. But otherwise, there's been little improvement. The GAO made six recommendations in 2016. To date, the FBI has only fully implemented one, and has taken no action at all on three of them.

As for the Privacy Impact Assessment the FBI was supposed to deliver in 2012? It's still in the works seven years later.

In its May 2016 report, GAO found that DOJ did not complete or publish key privacy documents for FBI’s face recognition systems in a timely manner and made two recommendations to DOJ regarding its processes for developing these documents. These included privacy impact assessments (PIA), which analyze how personal information is collected, stored, shared, and managed in federal systems, and system of records notices, which inform the public about, among other things, the existence of the systems and the types of data collected. DOJ has taken actions to expedite the development process of the PIA.

As for the system's accuracy, little forward progress has been made. The FBI is at least engaging in limited audits of the system, but only to ensure face searches are done according to policy. The problem with accuracy remains virtually untested. The FBI's testimony claims its vendor delivers a 99% accuracy rate, but as the GAO points out, this number comes from limited testing of batch sizes that may not be representative of those most commonly seen by the system's users.

GAO found that the FBI conducted limited assessments of the accuracy of face recognition searches prior to accepting and deploying its face recognition system. The face recognition system automatically generates a list of photos containing the requested number of best matched photos. The FBI assessed accuracy when users requested a list of 50 possible matches, but did not test other list sizes. GAO recommended accuracy testing on different list sizes.

On top of that, the FBI has no idea how accurate outside systems it utilizes are. It's own vendor might be delivering 99% accuracy, but the FBI makes use of databases and software used by other federal and state agencies. Despite being notified of this issue in 2016, the FBI has yet to assess the accuracy of these external systems.

This refusal to better police its system explains why the House Oversight Committee was less than impressed with the FBI's performance since it last took a look at the agency's facial recognition tech. The FBI's testimony was constantly undercut by the GAO's report, and this resulted in plenty of criticism from members of Congress.

During a hearing, members of the House Oversight Committee questioned witnesses on the steps being taken to ensure the facial recognition tools used by their agencies aren’t infringing on individuals’ privacy and civil liberties. By and large, lawmakers on both sides of the aisle seemed unsatisfied with their answers.


Lawmakers criticized Kimberly Del Greco, deputy assistant director of the FBI’s Criminal Justice Information Services division, over the bureau’s failure to correct multiple flaws in the way it evaluates its primary facial recognition tool.

Maybe this will finally prompt the FBI to follow up on the issues found in the GAO's latest assessment. But I wouldn't count on it. This same cycle of events played out in 2016 and 2017 -- a GAO report followed by Congressional tongue-lashing -- and the FBI still chose to completely ignore three of the GAO's recommendations. Maybe Congress should just tell the FBI it can't use the tech until it fixes the problems and see if that finally motivates the agency. Nothing else has worked so far. All the FBI has proven is that it can't be trusted with facial recognition tech.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: doj, error rates, facial recognition, fbi

Reader Comments

Subscribe: RSS

View by: Time | Thread

  1. icon
    That Anonymous Coward (profile), 12 Jun 2019 @ 9:10pm

    They saw it in a movie, so they know its possible.
    Scientists dream things up all the time that seem crazy or impossible, and then its a box in our kitchens that makes popcorn in a bag in minutes.

    It doesn't matter if it works or not, we've suspended our common sense the moment you invoke the sacred 'but terrorism'.
    We have a POTUS who has faithful that even in the face of him lying to them, knowing he lied, they still want to vote for him b/c he sold the idea only he can save them.
    Well sure the Congressman might have diddled some kiddies, but we have to vote for him b/c the other guy is a crazy man who wants to bring sharia here!! (And of course his own party won't even suggest someone else to run lest they be accused of being an arm of Isis.)

    For all of these things we've allowed them to funnel huge sums of our money into to keep us 'safe'... please provide a list of every plot stopped (that was not put in motion by the FBI) and then look at the list that is 3 times as long of domestic terrorism attacks that were never spotted or considered.
    We are terrified someone might wear a pair of flipflops made out of TNT into a plane, but a guy posts 100's of messages about killing teh Jews & then shoots up a museum... and we bitch about guns (which is something we are incapable of discussing like adults) rather than asking a very pointed question of how can someone in the 100 mile inland no rights area who made these threats for years & then explained his entire plan was never visited by anyone to protect the 'Homeland' to make sure he wasn't more than a crank?? Maybe because he was white & white guys never ever do the terrorism (b/c its not terrorism unless old rich lilly white WASP's are killed) & we should spend another 200 Million on tracking those dangerous Grannies against War.

    Welcome to the shitshow.
    We all drank the kool-aid & have allowed ourselves to be pushed to the extreme ends of the spectrum & anyone trying to remind us that compromise makes things work is to be destroyed. Nothing gets better, we slip further into chaos, and we keep spending huge sums of money to 'win' eternal wars at the expense of the homeland slipping further into the history books as a nation that once was great.

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.