Another Study Confirms NYC’s Shotspotter Deployment Was A Waste Of Money

(Mis)Uses of Technology

from the throwing-bad-money-after-bad dept

ShotSpotter spent a few years feeling really sure its tech was capable of detecting gunshots. It felt so confident it didn’t mind (allegedly) fudging detection data to help cops secure criminal charges against people who might not have been actual criminals.

Then it all started falling apart. Lots of cop shops and the cities that oversaw them assumed the system worked because SpotShotter’s (now known as “SoundThinking”) salespeople and PR people assured them that the system worked. But when cities decided to look into ShotSpotter tech themselves, they came to the opposite conclusion: that they were wasting money on tech that not only didn’t reliable spot (gun)shots, but also tied up limited law enforcement resources chasing down loud noises that rarely resulted in the discovery of evidence, much less successful prosecutions.

As cities began to terminate their contracts with ShotSpotter, the company fought back, claiming (without facts in evidence) that the real value was alerting EMS units of possible gunshots, thereby saving lives. But there’s no evidence the tech is saving more lives, much less putting a dent in gun-related crime.

The biggest blow to ShotSpotter arrived earlier this year. In June, the New York City Comptroller released its audit of the system, arriving at the conclusion the system was doing little more than wasting a lot of time and money:

During the month of June 2023, for example, out of the 940 ShotSpotter alerts that NYPD responded to 771 could not be confirmed as shootings upon arrival at the scene (82%), 47 were determined to be unfounded (5%), and 122 were confirmed as shootings (13%). NYPD officers spent 426.9 hours investigating alerts that were not confirmed as shootings. If only one officer responded, this equates to almost 36 twelve-hour shifts; if two officers responded, this number doubles.

It also rebuffed ShotSpotter’s latest PR spin — the thing about decreasing response time for EMS — by pointing out that even if the city had been tracking this data solely for the purpose of proving ShotSpotter right, the available data shows response times to reported shots (via ShotSpotter tech) delivered only a 90-second decrease in response times. As the Comptroller’s report points out, this is far less than the five minute improvement claimed by SpotShotter and its alleged analysis of NYC EMS response data.

So, we’ve already been made aware that ShotSpotter doesn’t reduce gun crime. Nor does it contribute regularly to prosecutions involving gun violence. On top of that, it doesn’t even perform that much better when using the company’s latest preferred metric: massive decreases in 911 response time.

A new report confirms the futility detailed in the city’s report. This one, performed by Brooklyn Defender Services, gathered a bunch of publicly-available data, combined with documents secured via public records requests, to further pound the nails into ShotSpotter’s PR coffin.

The report analyzed nearly 62,000 ShotSpotter alerts logged over nine years. The data reveals that only 16% of these alerts led to confirmed incidents of gunfire. This means that over 80% of deployments prompted by ShotSpotter yielded no evidence of gunfire at the reported locations.

During high-noise holidays like New Year’s Eve and July 4th, the system’s inaccuracies become even more pronounced. Alerts surge by 175% and 200%, respectively, as fireworks and other celebratory noises are misclassified as gunshots. 

It gets worse the noisier it is, which definitely conflicts with the promises made by the company. When law enforcement is better served by better acoustic detection, ShotSpotter actually gets worse, scrambling cops to harmless acts of celebration, rather than towards actual criminal activity. And this doesn’t do any favors for ShotSpotter/SoundThinking’s latest PR spin either because this means EMS units are rushing off to save gunshot victims who simply don’t exist because the tech is far too often incapable of separating fireworks from gunshots.

But heading back to the original claims made by ShotSpotter and its initial customers — that the tech is an invaluable contributor to the reduction of violent crime — the data says otherwise. From the Brooklyn Defenders’ report [PDF]:

According to the NYPD’s own data, less than 0.9% of responses to ShotSpotter alerts resulted in NYPD recovering a firearm, and only 0.7% of responses resulted in NYPD making an arrest for alleged illegal activity of any kind.

Being only 0.7-0.9% better than doing nothing certainly isn’t worth the $45 million the city has spent on this tech over the past few years. You could achieve the same “improvements” by hiring a psychic to peruse gunshot reports and suggest the next steps investigators should take to solve these crimes.

This isn’t making the city any safer. And, with 62,000 false positives over the past nine years, the only thing the tech has accomplished is wasting the time of hundreds of officers. If $45 million over 9 years seems like a drop in the bucket in terms of city finances, it really isn’t. The real cost is the (at minimum!) 62,000 hours of payroll wasted over the past nine years. Every reported shot cops respond to generates at least one hour of payroll time. Even if cops are quick to report that the alert was a false alarm, I can almost guarantee you no fewer than two officers are sent to each alert area and that they spend at least a half-hour trying to determine whether or not there’s any criminal activity to be pursued.

Will ShotSpotter continue to improve as its AI receives more training and feedback from all of these failures? Who knows? It’s been trying and failing for most a decade in New York City alone with no discernible uptick in accuracy rate. But as long as it has enough true believers located in police departments or city councils, it will always be able to find some suckers willing to spend other’s people money on tech that has repeatedly shown it’s incapable of accurately doing the one thing it’s asked to do: spot gunshots.

Filed Under: acoustic detection, gunshot detection, nyc, surveillance
Companies: shotspotter, soundthinking