San Francisco Is The First City In The World To Restrict Government Use Of Facial Recognition Technology. Hopefully It's Not The Last.

(Mis)Uses of Technology

from the keep-it-going dept

We welcome the city of San Francisco’s decision to ban the use of AI-enabled facial recognition technology by police and other municipal agencies. Facial recognition technology is prone to misidentification and biased targeting, particularly among members of vulnerable communities. It also opens the door to intrusive surveillance — beyond the scope of existing technology. Until the technology improves to avoid such harms, its use should be heavily restricted. And even with improvements, the public must confront tough questions about how closely governments should monitor their behavior.

Regrettably, global trends are headed in a reverse direction from San Francisco’s decision. From Azerbaijan to Singapore, governments are enthusiastically embracing facial recognition technology. Led by China, such systems are becoming increasingly ubiquitous. Facial recognition has been incorporated into smart city platforms. Cities are placing them on public lampposts and integrating them into security operations centers. While there is greater public awareness of China’s surveillance strategy – particularly its establishment of artificial intelligence-powered facial recognition repression in Xinjiang — few realize that numerous other countries are developing facial recognition capabilities as well.

This is particularly problematic in countries that lack basic rule of law protections and are already committing major human rights abuses. In countries such as Kazakhstan, Turkey, and Zimbabwe, facial recognition is a dangerous tool that further augments those governments’ arsenals of digital repression by giving them an enhanced capacity to deploy targeted monitoring in public places, to track and harass dissidents and opposition figures at will, and to suppress democratic political activity.

But even in democracies like the United States, government use of facial recognition technology, in its current form, corrodes civil rights and civil liberties because its errors disproportionately impact vulnerable communities. Astudy published earlier this year by MIT’s Media Lab, for instance, found that Amazon’s facial recognition software, which the company has peddled to police departments and Immigration and Customs Enforcement, more frequently misidentified an individual’s gender if they were female or dark-skinned.

Concerned about such disparities, San Francisco’s decision prohibits municipal agencies from using facial recognition technology except at federal facilities like the airport. Taken under the Stop Secret Surveillance ordinance, municipal agencies must also seek approval for any new surveillance technology they want to acquire. The decision passed 8-1 with overwhelming public support.

Even for a state like California that prides itself on trailblazing legislation (among many ‘firsts’, California led the way in requiring that corporate boards include women and enacting a greenhouse gas cap-and-trade program), San Francisco’s decision is extraordinary. Most of the current commentary points out that San Francisco is the first American city to ban government use of facial recognition technology. However to our knowledge, it’s not just the first American city to do so– it’s the first city in the world; a feat made only more remarkable by the city’s position as the seat of technological innovation — a rare instance of Silicon Valley putting values ahead of profits.

San Francisco’s decision also has important international implications. Facial recognition surveillance remains relatively nascent in most countries. While governments are interested in expanding their use of it, few states are deploying facial recognition at scale…yet. The world still has an opportunity to institute global safeguards and norms to shape how governments use facial recognition.

San Francisco’s ban is receiving widespread attention. We sincerely hope that this attention will lead to technological innovations that fix facial recognition’s bias problem and, notwithstanding such improvements, a healthy public discourse about facial recognition’s repressive potential.

Charlotte Stanton is a fellow in the technology and international affairs program and the director of the Silicon Valley office of the Carnegie Endowment for International Peace. Steven Feldstein is a non-resident fellow in Carnegie’s democracy, conflict, and governance program and the Frank and Bethine Church Chair of Public Affairs and Associate Professor in the School of Public Service at Boise State University.

Filed Under: bans, facial recognition, police, privacy, san francisco, searches