NYC Comptroller Report Says ShotSpotter Is Just Wasted Money
from the setting-money-on-fire dept
Well, we’ll see how long ShotSpotter/SoundThinking will keep making that New York money. The outlook is not good. A lot of this will depend on how well the NYPD can defend the useless product it’s spending millions on, but at the end of the day, the city still holds the purse strings and it has the power to terminate contracts that simply aren’t worth paying for.
The NYC comptroller performed an audit of the NYPD and ShotSpotter not with the intent of burying them, but simply to determine whether or not the NYPD was paying its bills on time and whether or not ShotSpotter was fulfilling the obligations of its contract.
The answer to both questions is “No.”
First, the comptroller takes on ShotSpotter and its guarantees of certain amount of law enforcement success:
When measured against the contractual performance standards set by NYPD, ShotSpotter met its 90% target for avoiding missed incidents in almost all boroughs except Manhattan, but when measured against the number of confirmed shootings, performance is far lower. During the sampled months of review in 2022 and 2023, ShotSpotter alerts only resulted in confirmed shootings between 8% and 20% of the time.
That’s pretty terrible, especially by the standards ShotSpotter claims to hold itself. When you’re wrong that often, you start costing cities real money while providing very little value in exchange.
During the month of June 2023, for example, out of the 940 ShotSpotter alerts that NYPD responded to 771 could not be confirmed as shootings upon arrival at the scene (82%), 47 were determined to be unfounded (5%), and 122 were confirmed as shootings (13%). NYPD officers spent 426.9 hours investigating alerts that were not confirmed as shootings. If only one officer responded, this equates to almost 36 twelve-hour shifts; if two officers responded, this number doubles.
More than 427 hours of wasted payroll in a single month. That’s pretty fucking terrible. But it could actually be much worse. There’s no way to know how much payroll is being blown by officers responding to ShotSpotter alerts because (surprise surprise!) the NYPD does not “track the amount of time — or associated staff costs — spent responding to such instances.”
I’ll try not to read too much into the NYPD’s disagreements with the comptroller’s conclusions, but they sound eerily like the new wave of PR emanating from SoundThinking HQ: the real value of ShotSpotter isn’t measured in arrests and prosecutions but rather lives saved. According to its new set of talking points, ShotSpotter brings EMS units to shooting victims faster, resulting in fewer lives lost to gun violence.
Maybe it would have a point if anyone was tracking this particular factor. But no one is, not even the NYPD which uses the same exact argument to excuse ShotSpotter’s underperformance:
NYPD does not agree that confirmed shootings should be used to measure ShotSpotter’s performance. It asserts that ShotSpotter improves the response time to possible shots fired which in turn increases the ability to provide assistance to victims, increases officers’ safety, and provides a more accurate location of the possible shooting than a 911 call alone.
However, NYPD does not measure ShotSpotter alert response times in comparison to 911 call response times to shots fired outside, the metric most closely aligned to ShotSpotter alerts.
Since the NYPD can’t or won’t do it, the comptroller’s office did what it could with available data. And that data contradicts the claims made by the NYPD (and, indeed, claims made by ShotSpotter itself).
The audit analyzed the two data sets, NYPD’s internal OCD ShotSpotter Tracking report and the Open Data NYPD Call for Service report, and found that during the month of June 2023 average response times to ShotSpotter alerts were 1 minute and 38 seconds faster than response times to reports to 911 of outdoor shots fired (3 minutes 50 seconds for ShotSpotter versus 5 minutes 28 seconds for 911), far less than the difference of 5 minutes claimed in publicly-available data.
So far, the NYPD has spent $45 million on ShotSpotter since 2014. Another $9 million will have to be spent before the contract expires in December of this year. That’s not much compared to the PD’s multi-billion dollar budget, but it’s still money that could be spent elsewhere on more useful things or, if legislators are trying to do a bit of budget balancing — not spent at all.
The comptroller has suggestions, most of which the NYPD disagrees with. The NYPD should track and publish data on ShotSpotter alerts, including the percentage of false or unconfirmed alerts. It already has the capacity to compile this data, but the comptroller wants the PD to pass that info on to the general public.
It also says the NYPD needs to start tracking how many officer hours are wasted responding to ShotSpotter alerts. And, if it really wants to use the argument that ShotSpotter is there to save shooting victims, rather than contribute to shooting investigations, the NYPD should start tracking this information too.
It also recommends the contract with ShotSpotter be allowed to expire at the end of the year. And, until that point, the NYPD should start paying its invoices within 30 days of receiving them as is mandated by the city’s procurement policy. (lol)
The NYPD has responded by saying pretty much everything asked of it is impossible or difficult or something it just doesn’t feel like doing, starting with the recommendation it allow its ShotSpotter contract expire:
NYPD disagreed with this recommendation, stating that “…non-renewal of ShotSpotter services may endanger the public and not renewing the ShotSpotter contract until the Department conducts further analysis would be a premature measure…”
First off, the data the comptroller compiled shows that, in most city boroughs, ShotSpotter is far better at wasting officers’ time than finding shooters or shooting victims. Second, the department should have been conducting “further analysis” ever since the inception of this contract a decade ago. It has no right to be asking for more time to conduct analysis, especially when it’s clear it’s just a stalling tactic meant to help the NYPD slide by the contract termination date and into another (potentially) long-term contract with the company.
Law enforcement agencies and ShotSpotter itself (although more often the latter) continue to defend this questionable tech with vague statements about safety and even more vague representations about its usefulness. But pretty much any city that’s actually dug into the data has come to the same conclusions: ShotSpotter may be an innovative use of acoustic detection tech, but it’s really not worth paying for.
Filed Under: gunshot detection, new york city, nypd, surveillance
Companies: shotspotter, soundthinking
Comments on “NYC Comptroller Report Says ShotSpotter Is Just Wasted Money”
So… taking longer to get to a 911 call where things actually happened… isnot putting lives in danger.
Also, would like to see how shotspotter increases officer safety. “It’s just one of those things we say.” Of course, if you’re actually responding to nothing, you’re probably pretty safe. While other actual things are happening that people call 911 for, but you’re busy doing nothing. Good for you.
Finally, how is EMS expedited by shotspotter? They can’t even go in until a scene is cleared, so you’re telling me they are, in fact, deployed, and merely waiting around a corner somewhere? For nothing? So how much delay involving illegitimately tied-up ambulances is “saving lives” elsewhere?
Shotspotter is probably killing people. Maybe not entirely true if their stories and claims aren’t entirely true.
Re:
“Finally, how is EMS expedited by shotspotter? They can’t even go in until a scene is cleared, so you’re telling me they are, in fact, deployed, and merely waiting around a corner somewhere? For nothing? So how much delay involving illegitimately tied-up ambulances is “saving lives” elsewhere?”
EXACTLY.
There are multiple ways to time emergency medical response. Here’s an example set:
T1: time from incident to time EMR is notified
T2: time from EMR notification to vehicle roll
T3: time from vehicle roll to on-scene
T4: time from on-scene to patient contact
T5: time from patient contact to transport-ready (i.e. loaded in ambulance)
T6: time from transport-ready to ER/trauma center arrival
Of course not all of these apply every time: if the patient refuses transport, then T5 and T6 don’t happen.
IF something like ShotSpotter worked then it could shorten T1. But T2 is mostly fixed (and is as small as possible, of course), while T3 varies based on distance and traffic. And T4 depends, as you noted, on whether the scene’s cleared. Assuming transport is indicated (e.g., gunshot wound to the thigh) T5 depends on prep and load, and again, T6 depends on distance and traffic.
So again, IF something like ShotSpotter worked, it could shorten T1 — maybe. But it’ll do nothing for the rest, and those tend to be much longer, e.g. cutting the total from 59:30 to 57:45 really isn’t going to do much good.
AND: if there are too many false alarms, something ShotSpotter is well-known for, then T3 may go up — because the nearest available vehicle may already be en route to a (false alarm) scene and thus the second-nearest vehicle needs to roll.
Re:
We’re no strangers to love
You know the rules and so do I
A full commitment’s what I’m thinking of
You wouldn’t get this from any other guy
I just wanna tell you how I’m feeling
Gotta make you understand
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
We’ve known each other for so long
Your heart’s been aching, but
You’re too shy to say it
Inside, we both know what’s been going on
We know the game and we’re gonna play it
And if you ask me how I’m feeling
Don’t tell me you’re too blind to see
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
(Ooh, give you up)
(Ooh, give you up)
Never gonna give, never gonna give
(Give you up)
Never gonna give, never gonna give
(Give you up)
We’ve known each other for so long
Your heart’s been aching, but
You’re too shy to say it
Inside, we both know what’s been going on
We know the game and we’re gonna play it
I just wanna tell you how I’m feeling
Gotta make you understand
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Re: Re:
Enough with the Rick rolling, dude.
NYPD disagrees with itself?
“NYPD does not agree that confirmed shootings should be used to measure ShotSpotter’s performance” but “When measured against the contractual performance standards set by NYPD…”
Re:
What it really gives them is pretext to harass random minorities at a given location. For the time being, it’s still a bad idea for them to put that in writing though.
Re: Re:
Which makes it even more of a waste of money since cops don’t given 2 entire shits about pretexts anyway. The Cop meta is “What color are they?” and “Do I have my hand cannon handy?”
Most often, when a cop shoots a man in a wheelchair because “he feared for his life”, he gets qualified immunity and the death certificate is marked with “Excited delirium” as the cause of death.
Confusion
“ShotSpotter met its 90% target for avoiding missed incidents“
I really can’t understand what this means. Anyone?
This comment has been flagged by the community. Click here to show it.
Re:
It means We’re no strangers to love
You know the rules and so do I
A full commitment’s what I’m thinking of
You wouldn’t get this from any other guy
I just wanna tell you how I’m feeling
Gotta make you understand
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
We’ve known each other for so long
Your heart’s been aching, but
You’re too shy to say it
Inside, we both know what’s been going on
We know the game and we’re gonna play it
And if you ask me how I’m feeling
Don’t tell me you’re too blind to see
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
(Ooh, give you up)
(Ooh, give you up)
Never gonna give, never gonna give
(Give you up)
Never gonna give, never gonna give
(Give you up)
We’ve known each other for so long
Your heart’s been aching, but
You’re too shy to say it
Inside, we both know what’s been going on
We know the game and we’re gonna play it
I just wanna tell you how I’m feeling
Gotta make you understand
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Never gonna give you up
Never gonna let you down
Never gonna run around and desert you
Never gonna make you cry
Never gonna say goodbye
Never gonna tell a lie and hurt you
Re:
To my understanding, if there is a gunshot in the area they cover, they need to report it at least 90% of the time. In other words, they need to have a <10% false negative rate.
They appear to be achieving this <10% false negative rate by having an >90% false positive rate, under the theory that if they report everything as a gunshot, they won’t miss any gunshots.
Free performance improvement for shotspotter?
python -c "import random;print(random.choice(['shot spotted','all quiet']))"… bumps up from 20% to 50% … improvement???
Shotspotter Works
Yes, the price is alot, I bet a bunch of kids with Rasberry Pi’s could get it done for 10% of the cost. But the problem is that science is science and physics is physics. When shotspotter goes off, a firearm was discharged and a bullet flew through the air. So what if by the time the police get there, the shooters have left. That is how it is with 99% of shootings anyway. Who sticks around after popping caps? What is worse than shotspotter, eyewitnesses. SHotspotter places the shot within a few feet. People that hear it and call 911 get it within a few blocks and rarely the right direction. Simple triangulation vs one source.