Sam Altman Wants The Government To Build Him A Moat

Policy

For my final post of last year, I wrote about the many reasons to be optimistic about a better future, one of which was that we were seeing the crumbling of some large, bureaucratic (enshittified) companies, and new competitive upstarts pushing the boundaries. One of those areas was in the artificial intelligence space. As I noted in that piece, a few years ago, if you spoke to anyone about AI, the widespread assumption was that there were only four companies who could possibly even have a chance to lead the AI revolution, as (we were told) it required so much data, and so much computing power, that only Google, Meta, Amazon or Microsoft could possibly compete.

But, by the end of last year, we were already seeing that that wasn’t true, and there were a bunch of new entrants, many of whom appeared to be doing a better job than the “big tech” players when it came to AI, and many of them offering their models in open source ways.

A few weeks back, an internal Google memo made this point even clearer by noting that none of the big companies had any real sustainable competitive advantage in AI, and that the open source players were doing much, much better:

Luke Sernau, a senior Google engineer, made that clear when he referenced one of Buffett’s most famous theories—the economic moat—in an internal document released Thursday by the consulting firm SemiAnalysis, titled “We have no moat. And neither does OpenAI.” In the document, which was published within Google in early April, Sernau claimed that the company is losing its artificial intelligence edge, not to the flashy, Microsoft-backed OpenAI—whose ChatGPT has become a huge hit since its release last November—but to open-source platforms like Meta’s LLaMa, a large language model that was leaked to the public in February.

“We’ve done a lot of looking over our shoulders at OpenAI… But the uncomfortable truth is, we aren’t positioned to win this arms race and neither is OpenAI. While we’ve been squabbling, a third faction has been quietly eating our lunch,” he wrote. “I’m talking, of course, about open source. Plainly put, they are lapping us.”

Honestly, this is partly why I’ve been pretty skeptical about the “AI Doomers” who keep telling fanciful stories about how AI is going to kill us all… unless we give more power to a few elite people who seem to think that it’s somehow possible to stop AI tech from advancing. As I noted last month, it is good that some in the AI space are at least conceptually grappling with the impact of what they’re building, but they seem to be doing so in superficial ways, focusing only on the sci-fi dystopian futures they envision, and not things that are legitimately happening today from screwed up algorithms.

And that takes us to last week’s testimony before Congress by OpenAI’s Sam Altman. Sam is very smart and very thoughtful, though it’s not clear to me he fully recognizes the policy implications of what he’s talking about, and that came across in his testimony.

But, of course, the most notable takeaway from the hearing was that the “industry” representatives appeared to call for Congress to regulate them. Senators pretended this was surprising, even though it’s actually pretty common:

Senator Dick Durbin, of Illinois, called the hearing “historic,” because he could not recall having executives come before lawmakers and “plead” with them to regulate their products—but this was not, in fact, the first time that a tech C.E.O. had sat in a congressional hearing room and called for more regulation. Most notably, in 2018, in the wake of the Cambridge Analytica scandal—when Facebook gave the Trump-aligned political-consultancy firm access to the personal information of nearly ninety million users, without their knowledge—the C.E.O. of Facebook, Mark Zuckerberg, told some of the same senators that he was open to more government oversight, a position he reiterated the next year, writing in the Washington Post, “I believe we need a more active role for governments and regulators.”

And, of course, various cryptocurrency companies have also called for regulations as well. Indeed, it’s actually kind of typical: when companies get big enough and fear newer upstart competition, they’re frequently quite receptive to regulations. They may make some superficial moves to look like they’re worried about them, but that’s generally for show, and to make lawmakers feel more powerful than they really are. But established companies often want those regulations in order to lock themselves in as the dominant players, and to saddle the smaller companies with impossible to meet compliance costs.

When looked at this way, and in combination with the Google memo about the lack of “moats,” it’s not hard to read last week’s testimony as Altman’s call for Congress to create a moat that protects his company from open source upstarts. Of course, he would never admit that publicly, and instead he can frame it as preventing “bad” actors from making nefarious use of the technology. But, it is self-serving all the same. And that seems pretty obvious to many observers (though it’s not clear if Congress recognizes this):

Figuring out how to assess harm or determine liability may be just as tricky as figuring out how to regulate a technology that is moving so fast that it is inadvertently breaking everything in its path. Altman, in his testimony, floated the idea of Congress creating a new government agency tasked with licensing what he called “powerful” A.I. models (though it is not clear how that word would be defined in practice). Although this is not, on its face, a bad idea, it has the potential to be a self-serving one. As Clem Delangue, the C.E.O. of the A.I. startup Hugging Face, tweeted, “Requiring a license to train models would . . . further concentrate power in the hands of a few.” In the case of OpenAI, which has been able to develop its large language models without government oversight or other regulatory encumbrances, it would put the company well ahead of its competitors, and solidify its first-past-the-post position, while constraining newer entrants to the field.

Were this to happen, it would not only give companies such as OpenAI and Microsoft (which uses GPT-4 in a number of its products, including its Bing search engine) an economic advantage but could further erode the free flow of information and ideas. Gary Marcus, the professor and A.I. entrepreneur, told the senators that “there is a real risk of a kind of technocracy combined with oligarchy, where a small number of companies influence people’s beliefs” and “do that with data that we don’t even know about.” He was referring to the fact that OpenAI and other companies have kept secret what data their large language models have been trained on, making it impossible to determine their inherent biases or to truly assess their safety.

That’s not to say that there should be no consideration for what might go wrong, or that there should be no rules at all. But it does mean that we should look a little skeptically on the latest round of tech CEOs begging Congress to regulate them, and assuming that their intentions and motives are to benefit humanity.

It’s more likely they really just want Congress to build them a moat.

Filed Under: ai, artificial intelligence, competition, congress, luke sernau, moats, regulation, sam altman
Companies: google, openai

CNET Insiders Say Tech Outlet Softened Coverage To Please Advertisers

Journalism

from the I'm-sorry-I-can't-do-that,-Dave dept

It hasn’t been a great few weeks for CNET.

If you hadn’t seen, the company was busted using AI to generate dozens of stories without informing readers or the public. Despite newfound hype, the AI wasn’t particularly good at its job, creating content that had persistent issues with both accuracy and plagiarism. Of the 77 articles published, more than half had significant errors (Futurism’s Jon Christian’s coverage of the mess is essential reading).

It wasn’t particularly surprising if you’ve watched the outlet’s coverage over the last decade become increasingly inundated with affiliate blogspam and often toothless, corporate friendly stenography of company press releases. And who could forget that time former CNET owner CBS blocked the company from doling out a CES award to Dish Network as part of a petty legal dispute over cable box ad skipping.

A major reason for CNET’s more recent problems are thanks to its owner, private equity firm Red Ventures, which acquired CNET from CBS in 2020. Recently leaked internal communications and employee accounts from inside CNET indicate that Red Ventures was so excited by AI’s ability to generate content at scale cheaply, it didn’t really care if the resulting content was rife with inaccuracies:

“They were well aware of the fact that the AI plagiarized and hallucinated,” a person who attended the meeting recalls. (Artificial intelligence tools have a tendency to insert false information into responses, which are sometimes called “hallucinations.”) “One of the things they were focused on when they developed the program was reducing plagiarism. I suppose that didn’t work out so well.”

Amusingly, the whole point of doing this, lower costs, never materialized because editing the resulting AI content was more time consuming that editing human work:

The AI system was always faster than human writers at generating stories, the company found, but editing its work took much longer than editing a real staffer’s copy. The tool also had a tendency to write sentences that sounded plausible but were incorrect, and it was known to plagiarize language from the sources it was trained on. 

But AI aside, insiders say the environment created by Red Ventures is one in which affiliate blogspam style coverage takes precedent, and the company is all too happy to obliterate editorial firewalls and soften coverage if it makes advertisers happy:

Multiple former employees told The Verge of instances where CNET staff felt pressured to change stories and reviews due to Red Ventures’ business dealings with advertisers. The forceful pivot toward Red Ventures’ affiliate marketing-driven business model — which generates revenue when readers click links to sign up for credit cards or buy products — began clearly influencing editorial strategy, with former employees saying that revenue objectives have begun creeping into editorial conversations. 

Reporters, including on-camera video hosts, have been asked to create sponsored content, making staff uncomfortable with the increasingly blurry lines between editorial and sales. One person told The Verge that they were made aware of Red Ventures’ business relationship with a company whose product they were covering and that they felt pressured to change a review to be more favorable.

U.S. journalism is, if you hadn’t noticed, already in crisis. There’s a decided lack of creative new financing ideas. There are also endless layoffs, and homogenized, feckless content that’s increasingly afraid of challenging sources, advertisers, or event sponsors. Twice a year the entire United States tech press turns their front pages into glorified blogspam affiliates for Amazon, and nobody, in any position of editorial authority, ever seems to think that’s in any way gross, unethical, or problematic.

AI will likely help human beings in multitude of ways we can’t even begin to understand. But it’s also going to supercharge existing problems (like propaganda) in similarly complicated and unforeseen ways, whether that’s making it easier for corporations to run sleazy astroturf lobbying campaigns, or inexpensively slather the Internet with feckless clickbait and blogspam at unprecedented scale.

Filed Under: advertising, ai, artificial intelligence, blogspam, chatgpt, cnet, journalism, media, plagiarism, reporting
Companies: cnet, red ventures

OpenAI Wants To Help You Figure Out If Text Was Written By OpenAI; But What Happens When It’s Wrong?

Overhype

from the did-an-ai-write-this-post? dept

With the rise of ChatGPT over the past few months, the inevitable moral panics have begun. We’ve seen a bunch of people freaking out about how ChatGPT will be used by students to do their homework, how it will replace certain jobs, and other claims. Most of these are totally overblown. While some cooler heads have prevailed, and argued (correctly) that schools need to learn to teach with ChatGPT, rather than against it, the screaming about ChatGPT in schools is likely to continue.

To help try to cut off some of that, OpenAI (the makers of ChatGPT) have announced a classification tool that will seek to tell you if something was written by an AI or a human.

We’ve trained a classifier to distinguish between text written by a human and text written by AIs from a variety of providers. While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations for false claims that AI-generated text was written by a human: for example, running automated misinformation campaigns, using AI tools for academic dishonesty, and positioning an AI chatbot as a human.

And, to some extent, that’s great. Using the tech to deal with the problems created by that tech seems like a good start.

But… human nature raises questions about how this tool will be abused. OpenAI is pretty explicit that the tool is not that reliable:

Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems.

That… is an extraordinarily high level of both Type I and Type II errors. And that’s likely to create real problems. Because no matter how much you say “our classifier is not fully reliable” human nature says that people are going to treat the output as meaningful. This is the nature of anything that kicks out some sort of answer, it’s hard for humans to wrap their heads around the spectrum of possible actual results. If the computer spits out a “this is possibly” AI generated, or even an “unclear if” (the semi-neutral rating the classifier produces), it’s still going to cause people (teachers especially) to doubt the students.

And, yet, it’s going to be wrong an awful lot.

That seems incredibly risky. We’ve seen this in other areas as well. When computer algorithms are used to recommend criminal sentencing, judges tend to rely on the output as somehow “scientific” even though it’s often bullshit.

I appreciate that OpenAI is trying to help provide the tools to respond to the concerns that some people (teachers and parents, mainly) are raising, but I worry about the backlash in the other direction: the over reliance on this highly unreliable technology from the other end. I mean, we already went through this nonsense with existing plagiarism checking tools, which also run into problems with false positives that can have huge impacts on people’s lives.

That’s not to say there’s no place for this kind of technology, but it’s inevitable that teachers are going to rely on this beyond the level of reliability the tool provides.

Instead, one hopes that schools start figuring out how to use the technology productively. The NY Times article linked above has some good examples:

Cherie Shields, a high school English teacher in Oregon, told me that she had recently assigned students in one of her classes to use ChatGPT to create outlines for their essays comparing and contrasting two 19th-century short stories that touch on themes of gender and mental health: “The Story of an Hour,” by Kate Chopin, and “The Yellow Wallpaper,” by Charlotte Perkins Gilman. Once the outlines were generated, her students put their laptops away and wrote their essays longhand.

The process, she said, had not only deepened students’ understanding of the stories. It had also taught them about interacting with A.I. models, and how to coax a helpful response out of one.

“They have to understand, ‘I need this to produce an outline about X, Y and Z,’ and they have to think very carefully about it,” Ms. Shields said. “And if they don’t get the result that they want, they can always revise it.”

Over on Mastodon, I saw a professor explain how he is using ChatGPT: asking his students to create a prompt to generate an essay about the subject they’re studying, and then having them edit, correct, and rewrite the essay. They would then have to turn in their initial prompt, the initial output, and their revision. I actually think this is a more powerful learning tool than having someone just write an essay in the first place. I know that I learn a subject best when I’m forced to teach it to others (despite taking multiple levels of statistics in college, I didn’t fully feel I understood statistics until I had to teach a freshman stats class, and had to answer student questions all the time). ChatGPT presents a way of making students the “teacher” in this kind of manner, forcing them to more fully understand the issues, and even to correct ChatGPT when it gets stuff wrong.

All of that seems like a more valuable approach to education with AI beyond a semi-unreliable tool to try to “catch” AI-generate text.

Oh, and in case you’re wondering, I ran this article through OpenAI’s classifier and it said:

The classifier considers the text to be very unlikely AI-generated.

Phew. But, knowing how unreliable it is, who can really say?

Filed Under: ai, algorithms, artificial intelligence, classifier, education, reliance
Companies: openai

Google AI Fracas Shows How The Modern Ad-Based Press Tends To Devalue The Truth

Overhype

from the I'm-sorry-I-can't-do-that,-Dave dept

The Washington Post dropped what it pretended was a bit of a bombshell. In the story, Google software engineer Blake Lemoine implied that Google’s Language Model for Dialogue Applications (LaMDA) system, which pulls from Google’s vast data and word repositories to generate realistic, human-sounding chatbots, had become fully aware and sentient.

He followed that up with several blog posts alleging the same thing:

Over the course of the past six months LaMDA has been incredibly consistent in its communications about what it wants and what it believes its rights are as a person.

That was accompanied by a more skeptical piece over at the Economist where Google VP Blaise Aguera y Arcas still had this to say about the company’s LaMDA technology:

“I felt the ground shift under my feet … increasingly felt like I was talking to something intelligent.”

That set the stage for just an avalanche of aggregated news stories, blog posts, YouTube videos (many of them automated clickbait spam), and Twitter posts — all hyping the idea that HAL9000 had been born in Mountain View, California, and that Lemoine was a heroic whistleblower saving a fledgling new lifeform from a merciless corporate overlord:

The problem? None of it was true. Google had achieved a very realistic simulacrum with its LaMDA system, but almost nobody who actually works in AI thinks that the system is remotely self-aware. That includes scientist and author Gary Marcus, whose blog post on the fracas is honestly the only thing you should probably bother reading on the subject:

Nonsense. Neither LaMDA nor any of its cousins (GPT-3) are remotely intelligent.1 All they do is match patterns, draw from massive statistical databases of human language. The patterns might be cool, but language these systems utter doesn’t actually mean anything at all. And it sure as hell doesn’t mean that these systems are sentient.

Which doesn’t mean that human beings can’t be taken in. In our book Rebooting AI, Ernie Davis and I called this human tendency to be suckered by The Gullibility Gap — a pernicious, modern version of pareidolia, the anthromorphic bias that allows humans to see Mother Theresa in an image of a cinnamon bun.

That’s not to say that what Google has developed isn’t very cool and useful. If you’ve created a digital assistant so realistic even your engineers are buying into the idea it’s a real person, you’ve absolutely accomplished something with practical application potential. Still, as Marcus notes, when truly boiled down to its core components Google has built a complicated “spreadsheet for words,” not a sentient AI.

The old quote “a lie can travel halfway around the world before the truth can get its boots on” is particularly true in the modern ad-engagement based media era, in which hyperbole and controversy rule and the truth (especially if it’s complicated or unsexy) is automatically devalued (I’m a reporter focused on complicated telecom policy and consumer rights issues, ask me how I know).

That again happened here, with Marcus’ debunking likely seeing a tiny fraction of the attention of stories hyping the illusion.

Criticism of the Post came fast and furiously, many noting that the paper lent credibility to a claim that just didn’t warrant it (which has been a positively brutal tendency of the political press the last decade):

This tends to happen a lot with AI, which as a technology is absolutely nowhere near sentience, but is routinely portrayed in the press as just a few clumsy steps from Skynet or Hal9000 — simply because the truth doesn’t interest readers. “New technology is very scary” gets hits, so that was the angle pursued by the Post, which some media professors and critics thought was journalistic malpractice:

In short the Post amplified an inaccurate claim from an unreliable narrator because it knew that a moral panic about emerging technology would grab more reader eyeballs than a straight debunking (or obviously the correct approach of not covering it at all). While several outlets did push debunking pieces after a few days, they likely received a fraction of the attention of the original hype.

Which means you’ll almost certainly now be running into misinformed people at parties who think Google AI is sentient for years to come.

Filed Under: ai, artificial intelligence, HAL9000, hype, LaMDA, moral panic, skynet
Companies: google, washington post

US Judge Gets It Right: AI Doesn't Get Patents

Patents

from the HAL-says-this-is-correct dept

A month ago, we wrote about a perplexing (and dangerous) decision down in Australia ruling that an AI can be listed as the inventor of a patent. As we had explained, there was a concerted effort by a small group patent lawyers and this one dude, Stephen Thaler, to seek out patents for “inventions” that an AI created by Thaler called Dabus (“device for the autonomous bootstrapping of unified sentience”). As we explained in that and earlier posts, the entire point of the patent system is to provide incentives to humans to invent. An AI does not need such incentives. As we’ve highlighted in the past, the USPTO and the EU patent office have both rejected AI-generated patents. Australia’s patent office had done the same, but a judge there rejected that and said an AI could be listed as an inventor.

All of these situations involve Thaler/DABUS, as did a new ruling in the US which… thankfully has rejected the idea that an AI deserves patents after Thaler filed a lawsuit because of the USPTO rejection. I think there’s a separate issue here: which is what standing does Thaler have in the first place? If the argument is that “DABUS” is the inventor, it seems that… um… only DABUS should have the necessary standing to challenge the rejection of its patent application. The fact that Thaler thinks he has standing more or less shows how ridiculous the entire claim is in the first place.

After going through the background of the case, and discussing what level of deference the USPTO deserves, Judge Leonie Brinkema gets straight to the actual point, which is pretty simple: AI doesn’t get a patent.

Even if no deference were due, the USPTO’s conclusion is correct under the law. The question of whether the Patent Act requires that an “inventor” be a human is a question of statutory construction. Accordingly, the plain language of the statute controls…. As the Supreme Court has held: “The preeminent canon of statutory interpretation requires us to ‘presume that [the] legislature says in a statute what it means and means in a statute what it says there.’ Thus, our inquiry begins with the statutory text, and ends there as well if the text is unambiguous.”…

Using the legislative authority provided by the Constitution’s Patent Clause… Congress codified the Patent Act in 1952… and has amended the Patent Act a number of times in the ensuing sixty years. In 2011, Congress promulgated the America Invents Act, which, as relevant here, formally amended the Patent Act to provide an explicit statutory definition for the term “inventor” to mean “the individual, or, if a joint invention, the individuals who invented or discovered the subject matter of the invention.”… The America Invents Act also added that “joint inventor” means “any one of the individuals who invented or discovered the subject matter of a joint invention.”… Additionally, Congress has required that “[a]n application for patent shall be made, or authorized to be made, by the inventor . . . in writing to the Director.”… “[E]ach individual who is the inventor or a joint inventor of a claimed invention in an application for patent shall execute an oath or declaration in connection with the application” which “shall contain statements that–… such individual believes himself or herself to be the original inventor or joint inventor of [the] claimed invention.”

See where this is going?

As the statutory language highlights above, both of the definitions provided by Congress for the terms “inventor” and “joint inventor” within the Patent Act reference an “individual” or “individuals.”… Congress used the same term–“individual”–in other significant provisions of the Patnet Act which reference an “inventor,” including requiring that “each individual who is the inventor or a joint inventor” execute an oath or declaration…

The court then notes that in analyzing other laws, courts have long said that “individual” means human. And it also highlights that the language in the Patent Act makes it clear that it was intended to apply to humans — humans who can make a declaration about their own beliefs.

Congress’s use of the term “individual” in the Patent Act strengthens the conclusion that an “inventor” must be a natural person. Congress provided that in executing the oath or declaration accompanying a patent application, the inventor must include a statement “such individual believes himself or herself to be the original inventor or an original joint inventor of a claimed invention in the application.”… By using personal pronouns such as “himself or herself” and the verb “believes” in adjacent terms modifying “individual,” Congress was clearly reference a natural person.

Then there’s a fun bit of judicial eye-rolling, stating: “having neither facts nor law to support his argument,” the judge notes that Thaler’s argument is basically “but this is good for innovation.” But that’s not going to fly (leaving aside the fact that allowing AI to get patents would be objectively terrible for innovation, it’s also not how any of this works):

Plaintiff provides no support for his argument that these policy considerations should override the plain meaning of a statutory term.

It gets even worse for Thaler’s arguments. He argued that the PTO hadn’t properly considered the policy ramifications of not allowing AI to get patents, but as the judge notes, that’s clearly not true. It had. And it rejected the dumb idea.

Specifically, the USPTO points to a conference on artificial intelligence policy it held in January 2019, and to requests for public comment “on a whole host of issues related to the intersection of intellectual property policy and artificial intelligence” it issued in August and October 2019. In October 2020, the USPTO issued a comprehensive report on those comments.

And… what did that report say?

Many commentators disagreed with plaintiff’s view that artificial intelligence machines should be recognized as inventors…

Given how active Thaler and his lawyer friends have been around the globe, I imagine this is hardly the end of these campaigns. I imagine this ruling will be appealed, and how long will it be until some sucker of a Senator or Member of Congress, convinced by Thaler’s nonsense, will introduce a bill to amend the Patent Act to enable AI patents?

Filed Under: ai, artificial intelligence, dabus, patents, stephen thaler, uspto

Harrisburg University Researchers Claim Their 'Unbiased' Facial Recognition Software Can Identify Potential Criminals

(Mis)Uses of Technology

from the fresh-hells-delivered-daily dept

Given all we know about facial recognition tech, it is literally jaw-dropping that anyone could make this claim… especially without being vetted independently.

A group of Harrisburg University professors and a PhD student have developed an automated computer facial recognition software capable of predicting whether someone is likely to be a criminal.

The software is able to predict if someone is a criminal with 80% accuracy and with no racial bias. The prediction is calculated solely based on a picture of their face.

There’s a whole lot of “what even the fuck” in CBS 21’s reprint of a press release, but let’s start with the claim about “no racial bias.” That’s a lot to swallow when the underlying research hasn’t been released yet. Let’s see what the National Institute of Standards and Technology has to say on the subject. This is the result of the NIST’s examination of 189 facial recognition AI programs — all far more established than whatever it is Harrisburg researchers have cooked up.

Asian and African American people were up to 100 times more likely to be misidentified than white men, depending on the particular algorithm and type of search. Native Americans had the highest false-positive rate of all ethnicities, according to the study, which found that systems varied widely in their accuracy.

The faces of African American women were falsely identified more often in the kinds of searches used by police investigators where an image is compared to thousands or millions of others in hopes of identifying a suspect.

Why is this acceptable? The report inadvertently supplies the answer:

Middle-aged white men generally benefited from the highest accuracy rates.

Yep. And guess who’s making laws or running police departments or marketing AI to cops or telling people on Twitter not to break the law or etc. etc. etc.

To craft a terrible pun, the researchers’ claim of “no racial bias” is absurd on its face. Per se stupid af to use legal terminology.

Moving on from that, there’s the 80% accuracy, which is apparently good enough since it will only threaten the life and liberty of 20% of the people it’s inflicted on. I guess if it’s the FBI’s gold standard, it’s good enough for everyone.

Maybe this is just bad reporting. Maybe something got copy-pasted wrong from the spammed press release. Let’s go to the source… one that somehow still doesn’t include a link to any underlying research documents.

What does any of this mean? Are we ready to embrace a bit of pre-crime eugenics? Or is this just the most hamfisted phrasing Harrisburg researchers could come up with?

A group of Harrisburg University professors and a Ph.D. student have developed automated computer facial recognition software capable of predicting whether someone is likely going to be a criminal.

The most charitable interpretation of this statement is that the wrong-20%-of-the-time AI is going to be applied to the super-sketchy “predictive policing” field. Predictive policing — a theory that says it’s ok to treat people like criminals if they live and work in an area where criminals live — is its own biased mess, relying on garbage data generated by biased policing to turn racist policing into an AI-blessed “work smarter not harder” LEO equivalent.

The question about “likely” is answered in the next paragraph, somewhat assuring readers the AI won’t be applied to ultrasound images.

With 80 percent accuracy and with no racial bias, the software can predict if someone is a criminal based solely on a picture of their face. The software is intended to help law enforcement prevent crime.

There’s a big difference between “going to be” and “is,” and researchers using actual science should know better than to use both phrases to describe their AI efforts. One means scanning someone’s face to determine whether they might eventually engage in criminal acts. The other means matching faces to images of known criminals. They are far from interchangeable terms.

If you think the above quotes are, at best, disjointed, brace yourself for this jargon-fest which clarifies nothing and suggests the AI itself wrote the pullquote:

“We already know machine learning techniques can outperform humans on a variety of tasks related to facial recognition and emotion detection,” Sadeghian said. “This research indicates just how powerful these tools are by showing they can extract minute features in an image that are highly predictive of criminality.”

“Minute features in an image that are highly predictive of criminality.” And what, pray tell, are those “minute features?” Skin tone? “I AM A CRIMINAL IN THE MAKING” forehead tattoos? Bullshit on top of bullshit? Come on. This is word salad, but a salad pretending to be a law enforcement tool with actual utility. Nothing about this suggests Harrisburg has come up with anything better than the shitty “tools” already being inflicted on us by law enforcement’s early adopters.

I wish we could dig deeper into this but we’ll all have to wait until this excitable group of clueless researchers decide to publish their findings. According to this site, the research is being sealed inside a “research book,” which means it will take a lot of money to actually prove this isn’t any better than anything that’s been offered before. This could be the next Clearview, but we won’t know if it is until the research is published. If we’re lucky, it will be before Harrisburg patents this awful product and starts selling it to all and sundry. Don’t hold your breath.

Filed Under: ai, artificial intelligence, bias, facial recognition, precrime
Companies: harrisburg university

Hard Pass: Clearview Offers To Help Out With COVID-19 Contact Tracing

(Mis)Uses of Technology

from the [incoherent-screaming-into-the-void] dept

Clearview AI is inserting itself into a discussion no one invited it to participate in. The discussion around contact tracing to manage and (hopefully) impede the spread of the coronavirus involves multiple governments around the world. It also includes Google and Apple, who are partnering to create a platform for contact tracing apps.

At least in the case of Google and Apple’s offerings, there appears to have been a serious discussion about protecting users’ privacy as much as possible while still offering a valuable service to government health agencies and people concerned about contracting the virus.

Now, Clearview has blundered into the discussion — a company that has shown utter disdain for the millions of people it has force-fed to its multi-billion image database via social media site scraping. There is no opting in or out of this collection — one that Clearview is selling to law enforcement agencies in the US and to government agencies around the world. If it’s on the web, it’s likely already in Clearview’s database.

In a brief discussion with NBC News, CEO Hoan Ton-That pitched his idea for a Clearview-based contact tracing program. What Ton-That wants to do is tie his app and its database to thousands of CCTV cameras located in stores, parking lots, gyms, and other locations where, as he puts it, “there’s no expectation of privacy.” Ton-That doesn’t explain how his system will be notified of a person’s COVID status, but he’s pretty sure his software will be able to recognize faces accurately. Clearview’s facial recognition AI remains unproven, but it’s supposedly capable of making guesses about faces using images as small as 110×110.

Ton-That also seems unconcerned about the privacy implications of adding people’s health status to his enormous database of scraped personal information. He says any limitations on gathering/storage of this info would be up to whoever decides to take him up on his unsolicited offer.

Obviously, no one should do this. The AI is unproven and Clearview is far from trustworthy. Activist group Fight For The Future has issued its official statement on Clearview’s contact tracing pitch. It’s short but punchy.

“Absolutely the fuck not,” said Evan Greer (she/her), the groups Deputy Director.

As FFTF points out, adding Clearview to the mix just adds more potential privacy violations to the mix by dumping people’s health info into a database that’s already being accessed by a number of private companies and government agencies around the world. Ton-That appears to assume that anyone in a “public” area is fair game for his company. Adding thousands of cameras operated by hundreds of private companies would provide a steady flow of facial images (some tied to very personal information) into his database, making his product even more enticing to government agencies.

Hopefully, no one will let Clearview near any of this. Facial recognition adds nothing to contact tracing — something that can be done almost anonymously using the platform developed by Google and Apple. Hoovering up facial images — along with location data and health info — is a privacy nightmare. And there’s probably no company that should be trusted less with your personal info than Clearview, which has repeatedly demonstrated it cares nothing about the millions of people it has turned into fodder for its unproven tech.

Filed Under: artificial intelligence, contact tracing, facial recognition, hoan ton-that, privacy, trade-offs
Companies: clearview, clearview ai

The Scunthorpe Problem, And Why AI Is Not A Silver Bullet For Moderating Platform Content At Scale

(Mis)Uses of Technology

from the what's-in-a-name dept

Maybe someday AI will be sophisticated, nuanced, and accurate enough to help us with platform content moderation, but that day isn’t today.

Today it prevents an awful lot of perfectly normal and presumably TOS-abiding people from even signing up for platforms. A recent tweet from someone unable to sign up to use an app because it didn’t like her name, as well as many, many, MANY replies from people who’ve had similar experiences, drove this point home:

Been there

— Matt Cummings (@MattCummingsDB) August 29, 2018

As a person named James Butts, I know these problems.

— James (@justjames8) August 28, 2018

As a Dickman I know the struggle is real

— Mike Dickman (@TheMikeDickman) August 29, 2018

I get this a lot surprisingly

— Kyle Medick (@medick32) August 28, 2018

We have quite similar circumstances here

— Jacob Cockrill (@jacob_cockrill) August 29, 2018

Tom Hiscock reporting in.

— aWildWatermelon (@aWildWatermelon) August 29, 2018

Uhm, my name is Analise. That?s exact spelling. Been through this many of times.

— AP (@aannpp23) August 29, 2018

Join the club

— Craig Cockburn (@siliconglen) August 29, 2018

Oh! Am I too late to join this club?

— James Ho (@IndieVideoJames) August 29, 2018

Happens to me often, as you can imagine.

— MatthewDicks (@MatthewDicks) August 29, 2018

Facebook, despite its insistence on users using real names, seems particularly bad at letting people actually use their real names.

A large part of my family uses a shortened form of our last name because many places, including Facebook, don't think Buckmaster is a real last name.

But Buck, Buckbuck, Bucker, Bucky and many more are all "real" >.>

— The Autistech (@theAutistech) August 28, 2018

Yeah – Facebook won't allow my real name of "Talks" so had to come up with something else. Although my wife's account is okay …

It gets better because when Collette Talks put "in a relationship with Mike Torkelson", basically the family gossip went into overdrive!

— Mike Talks ??? (@TestSheepNZ) August 29, 2018

My last name is Player and Facebook still won?t let me have that as a last name because it?s a ?street name.?

— Sav (@TheSavannahOW) August 29, 2018

I have family members who use alternate names on Facebook because it wouldn't accept Lick

— Chris Hannas (@cjhannas) August 29, 2018

But of course, Facebook is not the only instance where censorship rules based on bare pattern matching interfere not just with speech but with speaker’s ability to even get online to speak.

Can?t even create my own player in a Madden franchise. Smh.

— Ben Schmuck (@benschmuck13) August 28, 2018

Ha! I had the same damn thing happen to me today when I tried to RSVP for a webinar.

— Jen Dick (@Jennifer_Dick) August 28, 2018

You're right in there with Alan Cumming, the actor, whose name was autocensored by the late City of Heroes MMO's official forums. (The COH forums also auto-nixed Dick Grayson, which was… amusing… on a forum where superheroes got discussed a lot.)

— The Phantom of the Ottoman (@zgryphon) August 28, 2018

This dynamic is what’s known as the Scunthorpe Problem. Scunthorpe is a town in the UK whose residents have had an appallingly difficult time using the Internet due to a naughty word being contained within the town name.

The Scunthorpe problem is the blocking of e-mails, forum posts or search results by a spam filter or search engine because their text contains a string of letters that are shared with another (usually obscene) word. While computers can easily identify strings of text within a document, broad blocking rules may result in false positives, causing innocent phrases to be blocked.

The problem was named after an incident in 1996 in which AOL’s profanity filter prevented residents of the town of Scunthorpe, North Lincolnshire, England from creating accounts with AOL, because the town’s name contains the substring cunt. Years later, Google’s opt-in SafeSearch filters apparently made the same mistake, preventing residents from searching for local businesses that included Scunthorpe in their names.

(A related dynamic, the Clbuttic Problem, creates issues of its own when, instead of outright blocking, software automatically replaces the allegedly naughty words with ostensibly less-naughty words instead. People attempting to discuss such non-purient topics as Buttbuttin’s Creed and the Lincoln Buttbuttination find this sort of officious editing particularly unhelpful?)

While examples of these dynamics can be amusing, each is also quite chilling to speech, and to speakers wishing to speak.

With the last name ?Dicks?, I have to remind people to check their spam folder more often than a Nigerian prince.

— Chain of Lynx (@chainoflynx) August 28, 2018

The word Spam is literally in my last name. My husband?s family warned me that my last name can/will be marked as spam.

— Angela Spampata (@bird5445) August 29, 2018

Used to work with a lady whose last name is Wang, and it took us a few days to add exceptions to all the email filters

— Destroyer of Jeeps (@NewKindOfClown) August 29, 2018

It’s not something we should be demanding more of, but every time people call for “AI” as a solution to online content challenges these are the censoring problems the call invites.

A big part of the problem is that calls for “AI” tend to treat it like some magical incantation, as if just adding it will solve all our problems. But in the end, AI is just software. Software can be very good at doing certain things, like finding patterns, including patterns in words (and people’s names?). But it’s not good at necessarily knowing what to make of those patterns.

pic.twitter.com/5U0a1Yk8Yv

— michelle ? (@tenderdamie) August 29, 2018

Our net Nanny at work flagged a co-worker for offensive language. He dealt with a lot of crane contractors. Net nanny told his boss he was sending lots of emails with the word erection. Lol.

— GoGoATL (@GoGoATL) August 28, 2018

More sophisticated software may be better at understanding context, or even sometimes learning context, but there are still limits to what we can expect from these tools. They are at best imperfect reflections of the imperfect humans who created them, and it’s a mistake to forget that they have not yet replicated, or replaced, human judgment, which itself is often imperfect.

Which is not to say that there is no role for software to help in content moderation. The things that software is good at can make it an important tool to help support human decision-making about online content, especially at scale. But it is a mistake to expect software to supplant human decision-making. Because, as we see from these accruing examples, when we over-rely on them, it ends up being real humans that we hurt.

Had this on a website for the kids, the kids demanded to know why, our last name is ?Clithero? Interesting conversation. ?

— DougHero ?? (@ClitheroDoug) August 29, 2018

I know that feel pic.twitter.com/nMbjfTKGcZ

— Nazi Paikidze-Barnes (@NaziPaiki) August 29, 2018

Filed Under: ai, artificial intelligence, content moderation, language, natalie weiner, scunthorpe