The Failures Of Facial Recognition Software: Drivers Losing Licenses For Looking Like Terrorists

from the you-know-you're-a-terrorist,-when... dept

Over the years, we’ve discussed the technological failings of facial recognition software for law enforcement… but they just keep on trying. RichS was the first of a bunch of you to send in the story of the Massachusetts Registry of Motor Vehicles suspending licenses due to a match on facial recognition software designed to “detect” terrorists:

After frantic calls and a hearing with Registry officials, Gass learned the problem: An antiterrorism computerized facial recognition system that scans a database of millions of state driver?s license images had picked his as a possible fraud.

It turned out Gass was flagged because he looks like another driver, not because his image was being used to create a fake identity. His driving privileges were returned but, he alleges in a lawsuit, only after 10 days of bureaucratic wrangling to prove he is who he says he is.

Massachusetts bureaucrats seem positively thrilled with the system, claiming that they’re sending out 1,500 suspension notices a day based on such reports. To be honest, I can’t believe that they really mean per day, seeing as the article also notes that the facial recognition system only called out 1,000 such matches last year (and then later claims 1,860 licenses were revoked last year because of the software, so the numbers are all over the place). But, still, it sounds like a lot of folks in Massachusetts have to re-prove their identity every day because some computer falsely thinks they’re someone else.

Either way, the bureaucrats don’t seem at all concerned about relying on a highly questionable system to declare people guilty:

Kaprielian said the Registry gives drivers enough time to respond to the suspension letters and that it is the individual?s ?burden?? to clear up any confusion. She added that protecting the public far outweighs any inconvenience Gass or anyone else might experience.

Ah, the logic of clueless bureaucrats.

Filed Under: , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “The Failures Of Facial Recognition Software: Drivers Losing Licenses For Looking Like Terrorists”

Subscribe: RSS Leave a comment
Donnicton says:

Think about it for a second, if they’re identifying 1,500 terrorists a day in Massachusetts, then if all we do is funnel 10 Billion dollars per month into that state, then they’d be able to get it up to 3,000 per day easily!

With that many terrorists identified and getting their licenses suspended, they’ll be so effective at crippling the mobility of the jihadists that we can finally bring our troops back from the middle east, thereby saving countless lives without firing a shot! Massachusetts legislators are geniuses!

No doubt without licenses, Al Qaeda will find it quite difficult when they realize that they’ll have to start carrying their WMDs around by foot when their cars don’t start because their licenses aren’t valid. …..wait…

That Anonymous Coward says:

Re: There are 1500 terrorists living in Mass

Logic… from elected officials? *falls out of the chair laughing*

They must have never encountered an 18 yr old with a fake ID saying they were 21.

They know it is flawed and not working, but it generates a number they can cite in the media.

If your against this program being a waste of money funneled to someone who is paying us a nice kickback and hired my nephew well then your a fing Terrorist!

One wonders how much of the money used to fund this was grabbed as their share of antiterrorism funding.

Anonymous Coward says:

Re: Re: driving

Yeah, odds are the next president’s going to be exactly the same as the last two; a popular figurehead who pretends to be in charge while businesses bribe laws into effect that increase their profits.
I wonder how much lobbying L-1 Identity Solutions had to do to get paid $1.5 million from Homeland Security for software that doesn’t actually work?

FuzzyDuck says:

Fighting terrorism the American way

“Sir you look like a known terrorist, we have to revoke your driving license in 10 days.”

What if it really was a terrorist, shouldn’t you be arresting them instead? I guess the fact they don’t, proves that even the bureautards don’t really think they are terrorists.

PaulT (profile) says:

Re: Fighting terrorism the American way

Reminds me of the whole “piracy” thing a la 3 strikes, etc…

“Sir, we suspect you of a crime. We’re not going to go through due process or actually prove you have committed the crime in question because that’s too hard/expensive/impossible to prove. But, we’re so scared of the crime you might have committed, we’ll just go ahead and remove your rights and punish you anyway. If you complain in any way, that just proves your guilt.”

sehlat (profile) says:

Justice has been there before

There was a case many years ago where an innocent man was arrested, tried, and almost convicted for armed robbery, kidnapping and rape. He underwent an ordeal at the hands of police and judges that ruined his marriage and darn near landed him in prison (and remember, rapists do NOT do easy time in any prison) for crimes he had not committed.

The real culprit? A completely unrelated man who almost looked as if he were the innocent man’s twin brother. There was even a movie based on this: Blind Justice.

Rekrul says:

Re: Justice has been there before

There was a case many years ago where an innocent man was arrested, tried, and almost convicted for armed robbery, kidnapping and rape. He underwent an ordeal at the hands of police and judges that ruined his marriage and darn near landed him in prison (and remember, rapists do NOT do easy time in any prison) for crimes he had not committed.

That’s because the police don’t spend their time looking for the truth. They pick the easiest/most likely suspect and then devote 100% of their time and efforts to finding a way to prove that he/she is guilty.

Thomas (profile) says:

Oh dear...

they are just lucky the police didn’t show up with a swat team and drag the poor guy out and give him a 1 way ticket to Gitmo.

Nowdays, states prefer “guilty until proven innocent” carries much more weight than any silly constitutional protections.

I would have thought Mass had better sense.

I live in Mass, so if I don’t post again, you will know the found me out and I’m being waterboarded to find out the next location of whatever.

However, how in the heck would I know about it? No one has asked for my DL for years and years. Maybe I should just watch for a black SUV following me around.

Andrew D. Todd (user link) says:

Previous Cases.

It gets worse, of course. There was the well-known Brandon Mayfield case.

The FBI’s computer artificial intelligence system fed their racist beliefs back at them, and caused the agency to become more than usually delusional. The federal government eventually settled for a couple of million dollars. When the federal government pays that much money, it is a tacit confession of guilt to charges severe enough that a private individual would probably be sentenced to life imprisonment. FBI agents, of course, are “teflon.” Nothing sticks to them.

Of course, there was an even worse case, back in 1979, in France. There was a database whose existence the French Government had officially denied. It did exist, however , and it was full of errors, and a policeman, relying on the database, shot a young man in the head. The young man nearly died. The policeman had thought him to be an automobile thief, but he had in fact purchased the automobile from its legal owner.

See: Jacques Vallee, “Problem Scenario,” pp. 122-23, in: Gunther R. Geiss and Narayan Viswanathan eds, _The Human Edge: Information Technology and Helping People_, 1986, orig. pub. in Vallee’s _The Network Revolution: Confessions of a Computer Scientist_, 1982

Artifical Intelligence systems tend to have a lack of “articulable basis.” Going around and shooting people “because they look Jewish” is obviously a criminal act, and probably a terrorist act as well, but AI systems tend to unconsciously embed similar reasoning. Of course, faces move. There are muscles which move the skin around, and consequently, there is a considerable region of unmeasurability. Effectively, computerized facial recognition is pseudo-science.

Anonymous Coward says:

So… so far, if you travel to the US, you have to:
– Expect to be groped – and your kids – and grandma (even if by a random mobile unit)
– Expect to be falsely accused of being a terrorist
– Expect your laptop seized at the border for up to 3 years
– Expect to get arrested if you film police
– Expect to get sued (for pretty much any (im)plausible reason)
– Go to jail for having a garden.. ok maybe not for travelers, but I couldn’t leave it out.

Why aren’t we all rushing to plan our vacations there? Screw that, I’m heading off to Sweden to get my metal financial help.

Danny says:

Let me guess

Kaprielian said the Registry gives drivers enough time to respond to the suspension letters and that it is the individual?s ?burden?? to clear up any confusion. She added that protecting the public far outweighs any inconvenience Gass or anyone else might experience.
High ranking state politicians are except from it right?

Anonymous Coward says:

So, a computer system identifies possible fraud, and then suspends your license without any actual evidence – because you look like one of the other 6.5 million people in the state? If they actually think fraud is happening, why not actually arrest you, or at least conduct an investigation (especially since the reason for doing this is antiterrorism?) They really think the best course of action is to send a letter telling them that they’re been found out and the license is suspended?

And whatever happened to due process?

I’d hate to be an identical twin in Boston.

6 says:

“Kaprielian said the Registry gives drivers enough time to respond to the suspension letters and that it is the individual?s ?burden?? to clear up any confusion. She added that protecting the public far outweighs any inconvenience Gass or anyone else might experience. “

This is yet another reason why we need due process to be involved in the sending out of notices of drivers license suspension.

aldestrawk says:

Faulty numbers

My guess is that 1500/day number was the result of a misunderstanding of a question by the reporter or the reporter’s misunderstanding of the statement by the registrar. It probably is the number of suspensions for any reason, not just a picture conflict.

When the system flags a 1000 (obviously a rounded number) matches, that represents at least 2000 licenses. They will be revoking both licenses if they figure it is fraud and a single person has multiple licenses.

aldestrawk says:

Is this about terrorism?

One of the reasons given to support the existence of this system is prevention of terrorism. That reason would seem to be the primary one behind the $1.5 million grant from the DHS to Massachusetts . A terrorist wants the legitimacy granted by the possession of a driver’s license credential. A terrorist may use some sort of fraud to be issued a license or to make a counterfeit license. This facial recognition process is limited to registering for a driver’s license and will not uncover counterfeit licensing. If a terrorist already has a license, why would they need another one? That would be an unnecessary and silly risk. Registration requires you to show up, in person, and get your picture taken. What sorts of fraud will be caught by a facial recognition check, outside of duplicate licenses to the same person? An identity theft situation, where the terrorist does not resemble the person with an existing license, could be caught by the registrar, who is facing you, checking the database for the same name. This leaves identity theft where the terrorist does resemble an existing license holder. First, the terrorist has to find a license holder they resemble and use a change of address or gender to justify issuing a new license. Normally, the terrorist would then have a window of opportunity of up to a year (average of 6 months) before the real person has to renew car registration and uncover the theft because of the changes now affecting them. Applying the facial recognition system reduces that window to however long it takes someone in the DMV to get around to looking over the flagged license by hand and then issue a notice to validate identity. In this situation, why wouldn’t the terrorist wait until the couple of days before their terrorist action and steal that person’s physical license rather than their identity?
It doesn’t make sense to justify vetting licenses via facial recognition software as a defense against terrorism. It seems more likely that some bureaucrat(s) blindly advocated facial recognition license vetting as a weapon against terrorism because it fit into their “one person, one license” mantra and didn’t think it through.

aldestrawk says:

Lack of numbers, FAR and FRR.

The article states:
“Neither the Registry nor State Police keep tabs on the number of people wrongly tagged by the system.”

They do have the total number of people who were tagged by the system and they do have the number of licenses revoked. If anyone there can subtract, they would immediately have the number of people wrongly tagged. I can see that they would rather not keep track of this problematic number, however, I will attempt to make an estimate.
Massachusetts is using L-1 Identity Solutions’ facial recognition system. L1 claims in their website that one of their facial recognition systems (all their products may be based on the same algorithms) came out as “best all around performer” in a National Institute of Standards (NIST) evaluation of facial recognition algorithms in 2006. Unfortunately, it is not clear from the names which one of these is L1’s algorithm. Keep in mind that the algorithms may have been improved since 2006. I don’t know. This was NIST’s most recent facial recognition shootout. Here is the link:

The two most important characteristics of biometric identification algorithms are False Acceptance Rate (FAR) and False Rejection Rate (FRR). In this case, FAR covers those cases where two photos of the same person are not identified as identical. FRR applies to cases where the photos from two different people are wrongly identified as identical. It is usually the case that tweaking the algorithm to favor on rate makes the other worse. NIST reports that the best algorithms had an FRR of .01 while holding the FAR at .001. The FAR is a percentage of the total multiple license fraud cases. .001 X (>1000 cases) means there was on average 1 or 2 fraud cases that went undetected. They might have decided to improve this by tweaking the algorithm with the side effect of making the FRR worse. This is actually reasonable considering they further vet the flagged cases with human evaluation. It is interesting to note that the NIST evaluation shows that humans are worse than the algorithms at recognizing that two pictures belong to the same person when that person is not familiar to them. The further vetting that the RMV does probably involves other information than looking at the pictures. The FRR rate given was .01 for the best algorithms. FRR would be a percentage of the total number of licenses issued. It is independent of the fraud rate. The number of licensed drivers in Massachusetts is 4,645,705 ( Licenses are renewed every 5 years. Doing the math: .01 X (4,645,705 / 5) = 9291. This is a low number as Massachusetts is also vetting the entire licensed driver database during the next 3 years. It is not clear how much human evaluation reduces this number. There is a potential that thousands of innocent licensed drivers are inconvenienced by having to verify their identity again.
I would suggest, at the very least, that Massachusetts, and any other state doing such facial recognition, be required to officially serve notice (i.e. process serving) when a license is to be revoked

A side note: The NIST 2006 evaluation also covered Iris Challenge Evaluation (ICE). This involves using the iris of the eye for biometric identification. What is interesting is that this report, from a government agency, hints that patents discourage innovation.

“Because of the Flom and Safir patent and the lack of a publicly accessible dataset of images, there was limited research in iris recognition for most of the decade following the publication of Daugman?s algorithm. With the expiration of the Flom and Safir patent, and the availability of the CASIA dataset and the ICE 2005 challenge problem and dataset, research activity in iris recognition has greatly increased in recent years.”

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...