Oversight Says FBI's Facial Recognition System Has Gotten Bigger, But Not Better

from the it's-not-like-the-FBI-is-some-tiny-underfunded-agency... dept

It appears the FBI's facial recognition program will never live up to the minimal expectations its oversight has placed on it. The FBI's database went live in 2014, far preceding the Privacy Impact Assessment that was supposed to be delivered in 2012.

Two years after its debut, the Government Accountability Office found the FBI's database -- which went live with a 20% failure rate -- was still a mess. The FBI showed little interest in improving the accuracy of its searches. It also showed little interest in periodically testing the system to see if it was improving or, quite possibly, getting worse.

The FBI's hands-off approach to facial recognition only applies to its oversight of the program. Otherwise, it's an enthusiastic participant. At the time of the GAO's examination, the FBI's database contained 411 million photos, drawn from both criminal and non-criminal databases. Indicative of the FBI's lackadaisical approach to facial recognition was a bank robbery case in Colorado, where the feds pitched in to help arrest the wrong person twice.

A year later, the House Oversight Committee noted nothing had improved since the GAO's 2016 recommendations. Input and output remained flawed, and the FBI still showed little interest in fixing the problems reported by the GAO.

Two years later, it's deja vu all over again. The GAO's latest report [PDF] says the only thing that's really changed is the size of the database. Since it's last assessment, the FBI has added 230 million photos, bringing the total to 641 million face shots. But otherwise, there's been little improvement. The GAO made six recommendations in 2016. To date, the FBI has only fully implemented one, and has taken no action at all on three of them.

As for the Privacy Impact Assessment the FBI was supposed to deliver in 2012? It's still in the works seven years later.

In its May 2016 report, GAO found that DOJ did not complete or publish key privacy documents for FBI’s face recognition systems in a timely manner and made two recommendations to DOJ regarding its processes for developing these documents. These included privacy impact assessments (PIA), which analyze how personal information is collected, stored, shared, and managed in federal systems, and system of records notices, which inform the public about, among other things, the existence of the systems and the types of data collected. DOJ has taken actions to expedite the development process of the PIA.

As for the system's accuracy, little forward progress has been made. The FBI is at least engaging in limited audits of the system, but only to ensure face searches are done according to policy. The problem with accuracy remains virtually untested. The FBI's testimony claims its vendor delivers a 99% accuracy rate, but as the GAO points out, this number comes from limited testing of batch sizes that may not be representative of those most commonly seen by the system's users.

GAO found that the FBI conducted limited assessments of the accuracy of face recognition searches prior to accepting and deploying its face recognition system. The face recognition system automatically generates a list of photos containing the requested number of best matched photos. The FBI assessed accuracy when users requested a list of 50 possible matches, but did not test other list sizes. GAO recommended accuracy testing on different list sizes.

On top of that, the FBI has no idea how accurate outside systems it utilizes are. It's own vendor might be delivering 99% accuracy, but the FBI makes use of databases and software used by other federal and state agencies. Despite being notified of this issue in 2016, the FBI has yet to assess the accuracy of these external systems.

This refusal to better police its system explains why the House Oversight Committee was less than impressed with the FBI's performance since it last took a look at the agency's facial recognition tech. The FBI's testimony was constantly undercut by the GAO's report, and this resulted in plenty of criticism from members of Congress.

During a hearing, members of the House Oversight Committee questioned witnesses on the steps being taken to ensure the facial recognition tools used by their agencies aren’t infringing on individuals’ privacy and civil liberties. By and large, lawmakers on both sides of the aisle seemed unsatisfied with their answers.


Lawmakers criticized Kimberly Del Greco, deputy assistant director of the FBI’s Criminal Justice Information Services division, over the bureau’s failure to correct multiple flaws in the way it evaluates its primary facial recognition tool.

Maybe this will finally prompt the FBI to follow up on the issues found in the GAO's latest assessment. But I wouldn't count on it. This same cycle of events played out in 2016 and 2017 -- a GAO report followed by Congressional tongue-lashing -- and the FBI still chose to completely ignore three of the GAO's recommendations. Maybe Congress should just tell the FBI it can't use the tech until it fixes the problems and see if that finally motivates the agency. Nothing else has worked so far. All the FBI has proven is that it can't be trusted with facial recognition tech.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: doj, error rates, facial recognition, fbi

Reader Comments

Subscribe: RSS

View by: Time | Thread

  1. icon
    That One Guy (profile), 13 Jun 2019 @ 6:00am

    Got to collect ALL the hay before hunting for needles you know

    Ah but you see vastly increasing the size of the database is improving it as far as the FBI is concerned, or at least it's the only improvement they care about.

    Once they've grabbed everything they can then they might consider putting together a plan to look into the feasibility of scheduling a meeting to talk about accuracy, and/or discuss how the privacy of people that don't matter(that is, aren't members of their agency) might be impacted by such a database.

    As for the House Oversight Committee, I'd say it's pretty clear that's nothing more than a farce and everyone involved knows it. The members of the committee grumble and whine about how the FBI isn't doing what it should be/is doing what it shouldn't in order to show that they are providing some Real Oversight and Doing Something, whatever stooge the FBI sent makes some vague, placating noises about how they will definitely get right on to solving the issues raised, and then both parties completely forget about the whole thing until the next song and dance.

    The Oversight Committee has no interest in providing any actual oversight, as that would require them to do some gorram work and might require them to step on the toes of powerful people, and the FBI has neither the need or desire to care what the committee says because they know said committee is basically the political version of an extremely fat and lazy, two-paws-in-the-grave dog: good for some grumbles and foul smells, but harmless and toothless beyond that.

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.