EU Parliament Told Predictive Policing Software Relies On Dirty Data Generated By Corrupt Cops

from the junk-data,-garbage-outcomes dept

Predictive policing efforts continue to expand around the world. Fortunately, so has the criticism. The data witchcraft that expects a bunch of crap data created by biased policing to coalesce into actionable info continues to be touted by those least likely to be negatively affected by it: namely, law enforcement agencies and the government officials that love them.

The theory that people should be treated like criminals because someone else did some crimes in the area in the past is pretty specious, but as long as it results in temporary drops in criminal activity, fans of unreasonable suspicion will continue to use these tools that still have no long-term proven track record.

It's not just a US problem. It's a problem everywhere. The European Parliament has been asking for feedback on predictive policing efforts, which is more than most agencies in the US are willing to do. The Executive Director of the AI Now Institute, Andrea Nill Sanchez, recently testified during a public hearing on the issue, using the Institute's 2019 report on predictive policing to highlight everything that's wrong with turning law enforcement over to modeling algorithms. (via The Next Web)

The point of Sanchez's testimony [PDF] is this: we can't trust the data being fed to these systems. Therefore, we definitely can't trust the predictions being produced by them. Dirty cops create dirty data.

Despite what the term may suggest, predictive policing is neither magic nor a precise science that allows us to see into the future. Instead, predictive policing refers to fallible systems that use algorithms to analyze available data and aim to produce a forecasted probability of where a crime may occur, who might commit it, or who could be a victim.

Left unchecked, the proliferation of predictive policing risks replicating and amplifying patterns of corrupt, illegal, and unethical conduct linked to legacies of discrimination that plague law enforcement agencies across the globe.

This is drawn from AI Now's report, which pointed out some of the most corrupt police forces in the nation were feeding data from biased policing into systems that were now destined to generate nothing but more of the same corruption.

(1) Chicago, an example of where dirty data was ingested directly into the city’s predictive system; (2) New Orleans, an example where the extensive evidence of dirty policing practices and recent litigation suggests an extremely high risk that dirty data was or could be used in predictive policing; and (3) Maricopa County, where despite extensive evidence of dirty policing practices, a lack of public transparency about the details of various predictive policing systems restricts a proper assessment of the risks.

This is only one of several problems. First, almost every system used by law enforcement agencies is a closed system, unable to be inspected by anyone outside the companies that sell them and the agencies that use them. There's nearly no way for outsiders to vet data or outcomes. They can only judge the systems by the results. And those results are withheld or released pretty much at the sole discretion of the agencies using predictive policing software. If it's not producing decreases in crime rates, citizens will be the last to know.

Second, the data used is both tainted and selective, skewing the output towards the selective enforcement agencies already engage in.

[P]redictive policing primarily relies on inherently subjective police data, which reflects police practices and policies—not actual crime rates. Law enforcement exercises enormous discretion in how it carries out its work and collects data, including the crimes and criminals it overlooks.

Finally, there's no way to cleanse the systems of dirty data. Police departments are unable to recognize that their own biases might taint inputs. And citizens are powerless to challenge the data that's being used to target them for increased law enforcement scrutiny. With pushback effectively neutralized -- both by police practices and the secrecy surrounding proprietary algorithms -- agencies are just going to continue engaging in the same biased policing, only with the justification that the software told them to do it.

Since life and liberty is on the line for citizens put at the mercy of software being fed junk data, Sanchez suggests the following should be instituted (at a bare minimum) before predictive policing software is deployed:

As a first step, agencies considering using predictive policing tools should undertake Algorithmic Impact Assessments that include the following: (1) a self-assessment evaluating the system’s potential impacts on fairness, justice, and bias; (2) a meaningful external review process; (3) public notice and comment; and (4) enhanced due process mechanisms to challenge unfair, biased, or other harmful effects.

But that's just the start. The whole system has failed for years and reform from the ground up is needed before this software can be trusted in the hands of the police. Sanchez calls for the entire criminal justice system to be reformed, since it's the source of institutional racism and the enabler of some of the worst law enforcement behavior.

The negatives far outweigh the positives when it comes to predictive policing. The EU is taking the right step by asking for public input. Here in the US, public input is still an afterthought -- something that only happens after enough damage has been done. Turning law enforcement over to tainted data and proprietary algorithms poses a genuine threat to life and liberty. Hopefully, someone in the US will take a cue from the EU Parliament's actions and start asking law enforcement agencies tough questions about their predictive policing programs.

Filed Under: eu, junk data, law enforcement, predictive policing


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. identicon
    ROGS, 7 Apr 2020 @ 9:55am

    Language is not merely a binary excercise

    OK, sometimes I forget that TDs community is peopled by techish, binary I/O thinkers, whereas I approach language as a multi-tool, with non-binary, and relative, situational meanings and statements.

    So, in human language, the term in that statement functions differently than in computer code( or racist speech):

    deny
    [dəˈnī]
    VERB

    *state that one refuses to admit the truth or existence of*.

    In this context, my phrase simply did not rule it out; and has has a deeper and more colloquial meaning, cued by para-linguistic cues, whereas your interpretation of that statement is binary, and non-human to some extent.

    As in either/or statements:

    A disjunction is a set of two statements joined by the word “or”, so that both statements could be true, or only one statement could be true.

    That said, So if you do not deny something, then you accept it is stated as a binary, whereas what I actually said has colloquial context cues, and I used disjunctively to a minor extent, because it was a minor point. SO, either/or, if, and, etc.

    In linguistics, a disjunct is a type of adverbial adjunct that expresses information that is not considered essential to the sentence it appears in, but which is considered to be the speaker's or writer's attitude towards, or descriptive statement of, the propositional content of the sentence

    So,mine does not express a mere binary option, but rather my acknowledgement of the existence of white supremacy as a concept, and feature of American law, culture and politics; and of Russian law, European law, and I might even throw in that Maximilianos in Tlaquepaque, MX probably get better service at upscale restaurants too.

    Even your implied version of white supremacy is open to interpretation beyond binaries, unless the goal of the binary speaker is to divide the I from the 0, in which case, even binary conversation cannot occur.

    SO we see a problem emerge in the argument here: it's possible that we are speaking entirely different languages.


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.