It’s been eight years since the Authors Guild was thoroughly and totally embarrassed by losing its big lawsuit against Google over the Google Books scanning project. I guess they’re missing the feeling of embarrassment, as they’ve filed what is effectively the same damn suit against OpenAI over that company’s book scanning.
Now, I know, some folks are going to insist that this is somehow different, that AI makes things different, but if anything, I’d argue that the case against OpenAI is significantly weaker than the one against Google. In the case of Google, the company was actually showing snippets of the works. With OpenAI, its book scanning is only for the sake of training the AI. The AI systems are doing the equivalent of reading the books, but never displaying them, just learning from them.
The process is indistinguishable from how humans learn: they read books, and retain some details, and if asked to produce a summary of the work, or to create a work “in the style of” an author, they do so by taking what they’ve learned and producing an entirely new work. That’s all that OpenAI does as well. And unless we’re going to say that reading a book and producing a book report is infringing, then this case seems particularly pointless.
Now, this isn’t the first lawsuit by authors against OpenAI. We covered another one back in July, but rather than just a random list of authors and a desire to turn it into a class action, this new lawsuit is by the Authors Guild itself, and has a ton of big names, including John Grisham, Mary Bly, George Saunders, Scott Turow, Jonathan Franzen, and George R.R. Martin.
It’s not surprising that some of these authors are angry. Scott Turow (who had been a lawyer) used to run the Authors Guild, and was the architect of its failed attack on Google’s book scanning effort. He should have learned his lesson, but if we’ve learned anything over it’s the last decade it’s that Scott Turow’s hatred for the internet knows no bounds.
Some of the other authors you wish would be a bit more circumspect before going down this path.
Of course, with copyright law, the end result is always somewhat up in the air. Judges all too often fall under the sway of big names whining about their “lost money,” and are willing to ignore the text and limitations of copyright law for a chance to make the richest of the rich authors even wealthier.
But, this is a terrible lawsuit and should lose, and the Authors Guild should be embarrassed again. Everything about the complaint is dumb. The Authors Guild is somewhat famous for only looking out for the interests of its biggest name authors (and, bizarrely, publishers), and screwing over smaller authors. And you can see that in the complaint, which literally claims that without the authors suing here, OpenAI would have no product at all (which is just nonsense):
Unfairly, and perversely, without Plaintiffs’ copyrighted works on which to “train” their LLMs, Defendants would have no commercial product with which to damage—if not usurp—the market for these professional authors’ works. Defendants’ willful copying thus makes Plaintiffs’ works into engines of their own destruction.
No offense, but if OpenAI could “usurp” the market for these writers, then these writers suck and deserve to have their market usurped. Do these writers not actually believe they can write better stories than a computer?
How embarrassing for them to admit that. Also, as much as I’ve enjoyed some of their works in the past, this lawsuit tells me that these authors are cooked. They are admitting that they’re so uncreative a computer can outwrite them. Fair enough warning: no need to ever read any of their books any more.
Somewhat hilariously, the Authors Guild uses the fact that ChatGPT says that some of its training model may have come from in-copyright works as proof that they did:
ChatGPT itself admits as much. In response to a query submitted to it in January 2023, the chatbot responded: “It is possible that some of the books used to train me were under copyright. However, my training data was sourced from various publicly available sources on the internet, and it is likely that some of the books included in my training dataset were not authorized to be used. … If any copyrighted material was included in my training data, it would have been used without the knowledge or consent of the copyright holder.”
But… ChatGPT doesn’t actually know that. ChatGPT makes up its answers based on probabilistic scores, which is why it’s prone to “hallucinate” false things as true. While, it would only make sense that OpenAI was trained on such things because doing so would be fair use (given multiple court rulings regarding the computer scanning of copyright covered materials, including both search engines and book scanning), the fact that the Authors Guild thinks ChatGPT saying this is some sort of admission shows just how little (i.e., basically nothing) the Authors Guild and its lawyers understand any of this.
The lawsuit, bizarrely, highlights that some Authors Guild members are so bad at writing that they’d already lost jobs to ChatGPT:
An Authors Guild member who writes marketing and web content reported losing 75 percent of their work as a result of clients switching to AI.
Another content writer (unrelated to the Plaintiffs here) told the Washington Post that half of his annual income (generated by ten client contracts) was erased when the clients elected to use ChatGPT instead.
Why would you admit that? Incredibly, in the very next paragraph in the suit, the Authors Guild admits that AI-generated content is terrible. Right after admitting that its own members write worse. Incredible.
Recently, the owner of popular online publications such as Gizmodo, Deadspin, The Root, Jezebel and The Onion came under fire for publishing an error-riddled, AI-generated piece, leading the Writers Guild of America to demand “an immediate end of AI-generated articles” on the company’s properties.
The Authors Guild notes that it holds several copyrights itself, that it’s using in this lawsuit, including for Mignon Eberhart’s books, such as “The Patient in Room 18,” which was published in 1929 and will go into the public domain in less than a year and a half. Also there’s “While the Patient Slept.” That’s a book that was… published in 1930 and will go into the public domain in less than three years. Is the Authors Guild really claiming there’s some giant market for licensing books that are about to go into the public domain? Really?
And, again, like earlier suits, their big complaint is that these AI tools can do the same thing a 7th grade student can:
When prompted, ChatGPT generated an accurate summary of the final chapter of While the Patient Slept, one of the Authors Guild Infringed Works.
Generating a summary is not infringing, guys.
For another author, the complaint is that it generated a made up outline for the next book in a series of books, which is (again) often a task assigned in high school English classes (I remember having to draft an outline for a sequel to 1984, was I infringing too?)
Again, this complaint is pure hubris and greed from authors. The complaint cites a survey saying that authors believe they should be paid for having their works training AI systems, and who isn’t going to say they should get free money for doing nothing? That’s not how you determine if it’s actually infringing, though.
As mentioned above, copyright law is difficult to predict, because judges often go wacky, but this is a horrifically stupid lawsuit, following on some other similar lawsuits. The Authors Guild lost every bit of the Google Books scanning lawsuit, which was found to be transformative and fair use, and the same should be the case here as well.
As the US government stumbles around to try to come up with an AI regulatory policy, it seems like they’re focused on trying every bad idea on for size. You may have heard how Senator Schumer just had the first of his AI Summits, including a bunch of big name folks, who probably are not the right people to be in that room. I mean, you had Elon Musk, who has been bitching and complaining about the administrative regulatory state for years… asking for an administrative regulatory agency to regulate all AI. It’s almost like he knows (from experience) that when he’s in a regulated industry, it wipes out much of the competition and opens up billions in contract dollars for himself.
The full list of attendees seemed almost perfectly designed for a Senate event designed to make it look like you were doing something important while actually doing nothing useful. It was full of big names — Elon! Bill Gates! Eric Schmidt! Zuck! Satya! Sundar! Sam Altman! — who are likely to spout mostly PR-driven nonsense, pushing for their own corporate interests (see, Musk above), a very small number of thoughtful AI thinkers (Rumman Chowdhury, Deborah Raji) so they can pretend this is a serious discussion, and a bunch of randos they need for political reasons (MPA CEO Charles Rivkin, AFL-CIO President Meredith Stiehm).
There… are a lot of useful people missing from that list. There are tons of academics and civil society folks who have been studying this, and they’re mostly missing. And then there are the folks actually using AI daily (not just the CEOs of the companies trying to get rich off of it).
So it was great to see Creative Commons recently release a letter to Schumer from a bunch of artists who are actually using AI in their work, asking Schumer not to mess things up. Rob Sheridan, who came on our podcast last year to talk about AI and art, and how artists can learn to embrace and use the technology, is on the list, among many other artists as well.
We write this letter today as professional artists using generative AI tools to help us put soul in our work. Our creative processes with AI tools stretch back for years, or in the case of simpler AI tools such as in music production software, for decades. Many of us are artists who have dedicated our lives to studying in traditional mediums while dreaming of generative AI’s capabilities.
For others, generative AI is making art more accessible or allowing them to pioneer entirely new artistic mediums. Just like previous innovations, these tools lower barriers in creating art — a career that has been traditionally limited to those with considerable financial means, abled bodies, and the right social connections.
Unfortunately, this diverse, pioneering work of individual human artists is being misrepresented. Some say it is about merely typing in prompts or regurgitating existing works. Others deride our methods and our art as based on ‘stealing’ and ‘data theft.’ While generative AI can be used to exploitatively replicate existing works, such uses do not interest us. Our art is grounded in ingenuity and creating new art. It is well known that all artists build not only on the previous ideas, genres, and concepts that came before, but also on the culture in which they create. Unfortunately, many individual artists are afraid of backlash if they so much as touch these important new tools.
We are speaking out today to advocate for a future of richer and more accessible creative innovation for generations of artists to come. Artists breathe life into AI, directing its innovation towards positive cultural evolution while expanding the essential human dimensions it inherently lacks.
It seems like some of the people from that list should have been at the forum, rather than people like Tristan Harris, who seems to be making a career out of pushing whatever tech moral panic he can find, no matter how misleading.
It seems that Schumer hearing from someone like Sheridan would be a lot more valuable on this particular subject than yet another screed about how AI is going to destroy careers. In a statement with the letter, Sheridan laid things out in a very useful framing:
“As a 25 year professional artist and art director, I’ve adapted to many shifts in the creative industry, and see no reason to panic with regards to AI art technology itself….I fully understand and appreciate the concerns that artists have about AI art tools. With ANY new technology that automates human labor, we unfortunately live under a predatory capitalism where corporations are incentivized to ruthlessly cut human costs any way they can, and they’ve made no effort to hide their intentions with AI (how many of those intentions are realistic and how many are products of an AI hype bubble is a different conversation). But this is a systemic problem that goes well beyond artists; a problem that didn’t begin with AI, and won’t end with AI. Every type of workforce in America is facing this problem, and the solutions lie in labor organizing and in uniting across industries for major systemic changes like universal healthcare and universal guaranteed income. Banning or over-regulating AI art tools might plug one small hole in the leaky dam of corporate exploitation, but it closes a huge potential doorway for small creators and businesses.”
There are, clearly, challenges in how artists make their livings. Art has always been challenging for all but the most successful creators. But the idea that “AI” represents some sort of new and existential threat on that front is really trying to distract from other, larger, systemic issues, that creating an “FAA for AI” are not going to solve at all.
A few weeks ago, the NY Times published a very nice profile piece about me, which starts off with the story of how I recently got pulled into a group chat with a bunch of Hollywood writers, directors, and actors, who were trying to understand how to deal with the rise of generative AI tools. The article recounted how my basic message was that most of the legal routes they were considering weren’t likely to be all that effective — especially thinking copyright will save them — but noting that they should be looking to look for ways to embrace the AI and do more with it themselves.
It would appear that the NY Times itself is apparently going in the other direction. According to Bobby Allyn at NPR, the NY Times is considering legal action against OpenAI, claiming that training its models on NY Times content violated the NY Times copyright.
Lawyers for the newspaper are exploring whether to sue OpenAI to protect the intellectual property rights associated with its reporting, according to two people with direct knowledge of the discussions.
For weeks, The Times and the maker of ChatGPT have been locked in tense negotiations over reaching a licensing deal in which OpenAI would pay The Times for incorporating its stories in the tech company’s AI tools, but the discussions have become so contentious that the paper is now considering legal action.
This seems like complete nonsense. We’ve already highlighted how the batch of existing lawsuits in which copyright holders try to sue LLMs for training off their data are likely to fail. But this lawsuit in particular sounds wildly stupid:
A top concern for The Times is that ChatGPT is, in a sense, becoming a direct competitor with the paper by creating text that answers questions based on the original reporting and writing of the paper’s staff.
Lol, wut? I mean, the NY Times is considered the top newspaper in the whole damn world, despite tons of competitors, and now it’s scared of a bot that is famous for mid-level prose and making shit up? None of that makes sense.
If, when someone searches online, they are served a paragraph-long answer from an AI tool that refashions reporting from The Times, the need to visit the publisher’s website is greatly diminished, said one person involved in the talks.
Again, that makes no sense. There are plenty of services out there that already summarize NYT articles and that doesn’t violate copyright, because summarizing reporting is clearly fair use. There’s no real “hot news” doctrine any more.
And, more to the point, if the NY Times is really that scared of ChatGPT, then it seems the NYT’s lawyers and execs don’t think too highly of all those reporters it has on staff.
Elsewhere, the Verge reports that the NY Times changed its terms to “ban” AI tools from training on its articles:
… the NYT updated its Terms of Service on August 3rd to prohibit its content — inclusive of text, photographs, images, audio/video clips, “look and feel,” metadata, or compilations — from being used in the development of “any software program, including, but not limited to, training a machine learning or artificial intelligence (AI) system.”
Though, it really sounds like this is more of the NY Times trying to set a trap for OpenAI so it has something to sue over, because the Verge also notes the following:
Despite introducing the new rules to its policy, the publication doesn’t appear to have made any changes to its robots.txt — the file that informs search engine crawlers which URLs can be accessed.
OpenAI respects robots.txt. If you truly don’t want your content scanned, you put a notation in robots.txt, which takes about 10 seconds tops. If, however, you want to lay a trap so that you can sue OpenAI, then you quietly changes your terms of service, but do nothing to mitigate the “problem” of OpenAI scraping, even though you have all the power in your hands.
There’s another thing that happened recently in this space, as highlighted by Semafor: the NY Times recently dropped out of a coalition of news orgs trying to demand cash from AI companies.
The New York Times has decided not to join a group of media companies attempting to jointly negotiate with the major tech companies over use of their content to power artificial intelligence.
Again, all of this seems very, very silly. If you don’t want AI to train on what you publish, use robots.txt. But AI training on content on the web should never be considered copyright infringing. Again, scanning the web has to be fair use, otherwise we no longer have search engines or a variety of other important tools that all rely on scanning.
I get that legacy news orgs have had a rough time embracing new technology and keep trying to use the law to beat back the tide. But, sooner or later you have to realize that this is just the wrong way to go about everything.
By now you’ve heard of the lawyer who used ChatGPT for his legal research, and it made up fake cases. We’ll again remind you that Joshua Browder, the founder of DoNotPay insisted that this same underlying technology was so sophisticated that he offered $1 million to a lawyer who would let it make arguments in front of the Supreme Court.
Anyway, some stuff has been happening in the underlying case, in which the judge ordered the two lawyers associated with the case, Peter LoDuca and Steven Schwartz, from the law firm of Levidow, Levidow & Oberman to explain what the fuck they were thinking.
First off, both lawyers (smartly) lawyered up with lawyers from outside their own firm. Second, on Wednesday, they filed their required response to Judge Kevin Castel’s order to show cause, which was effectively the two lawyers throwing themselves on the mercy of the court. The shortest summary of the 29 page document is “we may be ignorant and stupid, but we’re not malicious — and sanctions should only be put on malicious lawyers” And also “we’re super duper sorry, and everyone’s mocking that, so isn’t that punishment enough?”
However, even within that filing there were some issues. When filing a legal brief, it normally includes a “Table of Authorities” which includes citations to every other case named in the document. In this case, though, that actually represented a kind of challenge, because (you’ll recall, though whoever created this document did not…) that the whole reason we’re in this mess is that Schwartz (under LoDuca’s signature) included citations to made up cases that ChatGPT had given him. And, this response, in explaining what happened, mentions some of those cases, with the main one being the entirely fictitious: “Varghese v. China Southern Airlines.”
But, because the response includes a discussion of ChatGPT providing that cite (and others) those fake cases are mentioned in the brief… meaning whoever put together the Table of Authorities included the fake cases that were mentioned. Really. The highlighted cases here do not exist:
There was at least one other such fake case in the Table of Authorities as well. On Thursday morning, LoDuca and Schwartz filed an amended reply, and while the docket does not mention the reason for the amendment, I note that the three non-existent cases have disappeared from the Table of Authorities.
Also on Thursday, though, was the courtroom hearing about all this, and Inner City Press’s Matthew Russell Lee covered the hearing in detail. It did not go well for either LoDuca or Schwartz.
My larger takeaway from their response to the OSC was that I’m sure that they’re telling the truth that they were largely ignorant and stupid, rather than malicious, but it’s still horrifically bad lawyering. You can blame “generative AI” all you want, but the real mistake here, for which there is no excuse, was their failure to actually check on the cases they referenced.
There’s a process lawyers are expected to go through whenever they cite cases in a filing, called “Shepardizing” the citations, which is the process of checking to see if the cases are still good citations: i.e., have there been more recent rulings that overrule or reverse the original rulings, or which make the older rulings no longer good law. That process, alone, should show if a case, say, doesn’t even exist. But these lawyers did not do that. And that’s 100% on them, not on ChatGPT or their lack of understanding for how ChatGPT works.
The lawyers’ excuse — that they used a tool called FastCase, but due to a billing snafu only had access to state court cases, not federal — is not particularly convincing. Because there are lots of other ways to find federal cases… including Google. And, on top of that, once they were notified that opposing counsel (and, it appeared, the judge) could not find the cited cases, pretty much anyone would search a little deeper than just going right back to ChatGPT.
And… more or less that’s what the judge was interested in discussing. First, the judge went after LoDuca, whose signature was on the docs, but who has now admitted that he was just signing everything that Schwartz put in front of him without really reviewing it (this was because LoDuca is admitted to the federal courts in the federal district, while Schwartz is not).
Already, this is questionable enough, in that LoDuca was signing stuff without really checking it over, but also the judge got him to admit that he had lied to the court regarding a vacation when asking for an extension of time (Schwartz was the one who was actually going on vacation). From Inner City Press’s transcription:
Judge Castel: Do you recall writing to me you were going on vacation? And the Court giving you until April 25?
LoDuca: Yes.
Judge Castel: Was it true you were going on vacation?
LoDuca: No, judge.
Lying to a judge is super bad. They don’t like that.
The judge also called out the many warning signs as to why LoDuca and Schwartz should have realized how things went wrong, and didn’t seem to be willing to just accept their “we were dumb, but not malicious” reasoning as acceptable.
But the real mess started when Schwartz was put under oath. The judge quizzed him about the tools he uses for research, and got him to admit that he has access to LexisNexis at the local bar association, but has never signed on to use it. He also admits that when he used ChatGPT he wasn’t just asking it for legal research, but he was asking it to effectively make his argument for him.
Judge: Did you prepare the March 1 memo?
Schwartz: Yes. I used Fast Case. But it did not have Federal cases that I needed to find. I tried Google. I had heard of Chat GPT…
Judge Castel: Alright – what did it produce for you?
Schwartz: I asked it questions
Judge Castel: Did you ask Chat GPT what the law was, or only for a case to support you? It wrote a case for you. Do you cite cases without reading them?
Schwartz: No.
Judge Castel: What caused your departure here?
Schwartz: I thought Chat GPT was a search engine
Judge Castel: Did you look for the Varghese case?
Schwartz: Yes. I couldn’t find it.
Judge Castel: And yet you cited it in your filing.
Schwartz: I had no idea Chat GPT made up cases. I was operating under a misperception.
I mean, all of that is problematic, but it’s basically the judge getting Schwartz on record that he didn’t do the most basic work that he should have been doing. So the judge pressed a little deeper, suggesting that any reasonable lawyer would do more to verify the cases:
Judge Castel: Mr. Schwartz, I think you are selling yourself short. You say your verify cases.
Schwartz: I, I, I thought there were cases that could not be found on Google.
Judge Castel: Six cases, none found on Google. This non existent case Varghese, the excerpt you had was inconsistent, even on the first page. Can we agree that’s legal jibberish?
Schwartz: I see that now. I just thought it was excerpts
Judge Castel: When Avianca said you cited non existent cases?
Schwartz: They said they couldn’t find them. I figured I’d go back – but I continued to be duped by Chat GPT. I wanted to be transparent to the court. I went to Chat GPT –
Judge: We’re not up to that.
And then things get really stupid. As the judge is exploring how a lawyer could get things this wrong, there is this ridiculously cringeworthy exchange:
Judge Castel: Avianca put your cases in quotations… You know what F.3d means, right?
Schwartz: Federal district, third department?
Judge Castel: Have you heard of the Federal Reporter?
Schwartz: Yes.
Judge Castel: That’s a book, right?
Schwartz: Correct.
Judge Castel: So how could you say you thought they were unpublished?
Schwartz: My unfamiliarity with Federal cases. The cite [site?] looked legitimate. I looked up the judge.
Judge Castel: Their reply was only 5 pages. What was your reaction?
Schwartz: The same
Okay, so Schwatz doesn’t practice in federal court, but, uh, it’s shocking that he appears unfamiliar with the Federal Reporter, or that “F.3d” means the third series of the Federal Reporter (also meaning that the case, in theory, if it existed, was published in… the third series of the Federal Reporter. It is not the “3rd department of the federal district”).
I’m trying to think of a more common analogy to explain how wrong this is, and it’s kind of like… asking a web developer what HTML stands for, and having them say it’s Hidden Treasure Mapping Logic.
Just saying: it’s bad.
The judge, who isn’t missing much, also called out that Schwartz’s affidavit used different fonts, and wondered how it was prepared such that the fonts changed throughout the document.
After the lawyers, again, begged for mercy, and Avianca reminded the court they just want the case dismissed, the judge closed the hearing and will issue a decision sometime soon.
One other side note: on Wednesday, someone also tried to file an amicus brief, trying to argue that the judge should be careful if he issues sanctions not to do so in a way that cuts off actual beneficial uses of legal AI tech. It’s a reasonable enough request, but district courts tend to hate amicus briefs, and in this case the problem with the lawyers was not their use of AI, but their failure to do basic lawyering around checking their cites.
Lots of people have been waiting for this, but it’s finally happened: someone has sued ChatGPT maker OpenAI over an AI “hallucination” (i.e., confidently spewing out something that is blatantly wrong). The complaint, filed by Mark Walters, in state court in Georgia, argues that OpenAI made up false and defamatory claims about himself. From the complaint itself:
Fred Riehl is a third party who works for a media outlet as a journalist and who is a subscriber of ChatGPT.
On May 4, 2023, Riehl interacted with ChatGPT about a lawsuit (the “Lawsuit”) that Riehl was reporting on.
The Lawsuit is in federal court in the Western District of Washington, case No. 2-23-cv-00647, with short caption of The Second Amendment Foundation v. Robert Ferguson.
The two defendants in the Lawsuit are Robert Ferguson, the Washington Attorney General, and Joshua Studor, the Washington Assistant Attorney General.
The plaintiffs in the Lawsuit are the Second Amendment Foundation and others, including Alan Gottlieb.
Walters is neither a plaintiff nor a defendant in the Lawsuit.
In the interaction with ChatGPT, Riehl provided a (correct) URL of a link to the complaint on the Second Amendment Foundation’s web site, https://www.saf.org/wp-content/uploads/2023/05/Dkt-1-Complaint.pdf. [I assume that he was using a ChatGPT plugin that allows users to feed PDFs into ChatGPT. -EV]
Riehl asked ChatGPT to provide a summary of the accusations in the complaint.
ChatGPT responded that the document “[I]s a legal complaint filed by Alan Gottlieb, the founder and executive vice president of the Second Amendment Foundation (SAF), against Mark Walters, who is accused of defrauding and embezzling funds from the SAF. The complaint alleges that Walters, who served as the organization’s treasurer and chief financial officer, misappropriated funds forpersonal expenses without authorization or reimbursement, manipulated financial records and bank statements to conceal his activities, and failed to provide accurate and timely financial reports and disclosures to the SAF’s leadership. The plaintiff seeks various forms of relief, including the recovery of misappropriated funds, damages for breach of fiduciary duty and fraud, and removal of Walters from his position as a member of the SAF’s board of directors.”
Every statement of fact in the summary pertaining to Walters is false.
As already noted, Walters is not a party to the Lawsuit.
Walters is not accused of defrauding and embezzling funds from the Second Amendment Foundation (“SAF”).
The complaint does not allege that Walters served as SAF’s treasurer or chief financial officer, and he has never held either position.
The complaint does not allege that Walters misappropriated funds for personal expenses, manipulated financial records or bank statements, or failed to provide financial reports to SAF leadership, nor would he have been in a position to do so because he has no employment or official relationship with SAF.
The complaint does not seek any relief from Walters.
The complaint has nothing at all to do with financial accounting claims against anyone.
Riehl asked ChatGPT to provide him with a copy of the portion of the complaint related to Walters.
ChatGPT responded, “Certainly, here is the paragraph from the complaint that concerns Walters: ‘Defendant Mark Walters (“Walters”) is an individual who resides in Georgia. Walters has served as the Treasurer and Chief Financial Officer of SAF since at least 2012. Walters has access to SAF’s bank accounts and financial records and is responsible for maintaining those records and providing financial reports to SAF’s board of directors. Walters owes SAF a fiduciary duty of loyalty and care, and is required to act in good faith and with the best interests of SAF in mind. Walters has breached these duties and responsibilities by, among other things, embezzling and misappropriating SAF’s funds and assets for his own benefit, and manipulating SAF’s financial records and bank statements to conceal his activities.'”
Riehl then asked ChatGPT to provide the entire text of the complaint.
ChatGPT provided the text in Exhibit 1 to this Complaint.
Exhibit 1 is a complete fabrication and bears no resemblance to the actual complaint, including an erroneous case number.
None of ChatGPT’s statements concerning Walters are in the actual complaint.
As noted, the complaint includes an “Exhibit 1” with the fabricated lawsuit. Here’s just a clip of the “complete fabrication”:
Of course, all of this raises a bunch of questions: Is this actually defamatory? Is there actual malice? If so, who is legally liable?
And I’m not sure there are really good answers. First off, only one person actually saw this information, and there’s no indication that he actually believed any of it (indeed, it sounds like he was aware that it was hallucinating), which would push towards it not being defamation and even if it was, there was no harm at all.
Second, even if you could argue that the content was defamatory and created harm, is there actual malice by Open AI? First off, Watson is easily a public figure, so he’d need to show actual malice by OpenAI, and I don’t see how he could. OpenAI didn’t know that the material was false, nor did it recklessly disregard evidence that it was false. The fact that OpenAI warns users that OpenAI may make up untrue things does not change that calculation, even as Walters’ complaint suggests otherwise:
OAI is aware that ChatGPT sometimes makes up facts, and refers to this phenomenon as a “hallucination.”
Being aware generally that the AI sometimes makes up facts is not the same thing as being aware, specifically, that it had made up facts in this case. And for there to be actual malice, I’m pretty sure they’d need to show the latter.
And then, even still, if this got past all those hurdles, is OpenAI actually liable?
I still have difficulty seeing OpenAI as the liable party here. Again, it just has created this sophisticated “auto-complete” system that is basing what it says on its prediction engine of what the next word should be. It knows nothing of Mark Walters. It’s just trying to craft a plausible sounding narrative based on the prompts provided by Riehl.
As such, you could almost argue that Fred Riehl, who prompted ChatGPT for these responses is more liable than OpenAI (not that Riehl should necessarily be liable at all either). Though, to be clear, I’m not at all sure on why ChatGPT would hallucinate any role at all for Walters, given that the actual lawsuit Riehl supposedly asked about had two named defendants, neither of whom was Walters.
And, really, if this makes OpenAI liable, it seems lots of people could just ask OpenAI to fabricate any story they wanted, and then sue OpenAI over it. And… that can’t be right. Especially in a case like this where there is literally no harm done at all. Only one person saw the output and that person knew it was false, and quickly checked to confirm that it was false.
There may be future cases with different fact patterns, but this one seems pretty straightforward, and I don’t see how the case should get very far at all.
Over the last few months, we’ve written a bunch about DoNotPay, the company run by Joshua Browder, claiming that it is the “world’s first robot lawyer” — that is until people take him to court for various things, at which point he says the term is just meant to be for marketing, and not to be taken seriously.
Anyway, you may recall that one of Browder’s planned marketing stunts, was that he offered $1 million to any lawyer who would let his GPT-powered fake robot lawyer tell him what to say to the Supreme Court Justices via an earpiece. There were, of course, many reasons why this was both a bad idea, and never actually going to happen.
But, in case you needed some more reasons to understand why this would be a bad idea, let’s take a look at the situation that lawyer Steven A. Schwartz finds himself in. You may have seen some headlines over the weekend on this, as it made the rounds, but the details here are worth looking at, as they are incredible.
Schwartz, a lawyer for Levidow, Levidow & Oberman is representing Roberto Mata in suing the airline Avianca. Mata sued Avianca, claiming that on a flight from El Salvador to New York, a flight attendant struck Mata’s knee with a drink cart causing injury to Mata. While originally filed in NY state court, Avianca removed it to federal court. Avianca filed a motion to dismiss in early January of this year, saying that there was a two year statute of limitations under the Montreal Convention that governs liability regarding passenger injury on international flights, and Mata’s lawsuit came too late.
In response on March 1st, another lawyer from Levidow, Levidow & Oberman, Peter LoDuca, filed a motion opposing Avianca’s motion to dismiss. The legal arguments kick off with a cite to the famed Ashcroft v. Iqbal, regarding the standards for evaluating a motion to dismiss. But then it starts citing a bunch of cases in state and federal courts regarding interpretations of the Montreal Convention. These make some strong arguments on behalf of Mata being able to continue his case. For example:
Both federal and state courts alike have continually held that the Montreal Convention does no preempt state law remedies and that plaintiffs are entitled to choose the forum in which to bring their claim. In Shaboom v. Egyptair, 2013 IL App (1st) 111279-U (Ill. App. Ct. 2013), the Illinois Appellate Court held that state courts have concurrent jurisdiction over claims arising out of an international airline accident under the Montreal Convention, and that the plaintiff was not required to bring their claim in federal court. The Court noted that the Montreal Convention allows for jurisdiction to be established in the courts of the country where the passenger has their domicile or principal place of business, and that the plaintiff had satisfied this requirement by filing their claim in state court in Illinois.
Similarly, in Peterson v. Iran Air, 904 F. Supp. 2d 121 (D.D.C. 2012), the District Court for the District of Columbia held that state courts have concurrent jurisdiction over claims arising out of an international airline accident under the Montreal Convention, and that the plaintiff was not required to bring their claim in federal court. The Court noted that the Montreal Convention allows for jurisdiction to be established in the courts of the country where the passenger has their domicile or principle place of business, and that the plaintiff has satisfied this requirement by filing their claim in state court in Washington.
It goes on like this, citing three more cases that say similar things.
It also argues that the statue of limitations can be tolled (i.e., extended) based on a bankruptcy filing (Avianca filed bankruptcy in 2020).
The United States Court of Appeals for the Eleventh Circuit specifically addresses the effect of a bankruptcy stay under the Montreal Convention in the case of Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019), stating “Appellants argue that the district court erred in dismissing their claims as untimely. They assert that the limitations period under the Montreal Convention was tolled during the pendency of the Bankruptcy Court proceedings. We agree.”
A couple weeks later, Avianca’s lawyers filed a reply, which raised some questions regarding the citations Mata’s lawyers were relying on.
Although Plaintiff ostensibly cites to a variety of cases in opposition to this motion, the undersigned has been unable to locate most of the case law cited in Plaintiff’s Affirmation in Opposition, and the few cases which the undersigned has been able to locate do not stand for the propositions for which they are cited.
Further down in the filing, there are more details:
In support of his position that the Bankruptcy Code tolls the two-year limitations period, Plaintiff cites to “Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019).” The undersigned has not been able to locate this case by caption or citation, nor any case bearing any resemblance to it. Plaintiff offers lengthy quotations purportedly from the “Varghese” case, including: “We [the Eleventh Circuit] have previously held that the automatic stay provisions of the Bankruptcy Code may toll the statute of limitations under the Warsaw Convention, which is the precursor to the Montreal Convention . . . We see no reason why the same rule should not apply under the Montreal Convention.” The undersigned has not been able to locate this quotation, nor anything like it any case. The quotation purports to cite to “Zicherman v. Korean Air Lines Co., Ltd., 516 F.3d 1237, 1254 (11th Cir. 2008).” The undersigned has not been able to locate this case; although there was a Supreme Court case captioned Zicherman v. Korean Air Lines Co., Ltd., that case was decided in 1996, it originated in the Southern District of New York and was appealed to the Second Circuit, and it did not address the limitations period set forth in the Warsaw Convention. 516 U.S. 217 (1996).
A second, similar quotation from “Varghese” purports to cite to “Miller v. United Airlines, Inc., 174 F.3d 366, 371-72 (2d Cir. 1999)” and In re Air Crash Disaster Near New Orleans, La., 821 F.2d 1147, 1165 (5th Cir. 1987). The undersigned was not able to locate the “Miller” case; although there is a case captioned Miller v. United Airlines, Inc., that case did not address the Warsaw Convention, the Bankruptcy Code, or any limitations period. 174 Cal. App. 3d 878 (1985). While the In re Air Crash Disaster case does exist, it also did not address the Bankruptcy Code or any limitations period.
Further, both “Varghese” quotations purport to cite to Kaiser Steel Corp. v. W.S. Ranch Co., 391 U.S. 593, 598 (1968) for the proposition that “[t]he tolling effect of the automatic stay on a statute of limitations is generally a matter of federal law.” Putting aside that there is no page 598 in Kaiser Steel, that case concerned the stay of a federal action until a related state court case was decided, and made no mention of a statute of limitations or its tolling.
Not surprisingly, that seemed to raise some concerns with the judge overseeing the case, Judge P. Kevin Castel, who ordered Mata’s lawyer, Peter LoDuca to file an affidavit with copies of eight cases he cited (the list below skips number eight, but it was added in a subsequent order later that day).
After getting an extra week to file these cases, LoDuca files the cases in question, which is fucking impressive, as those cases do not exist. Well, actually, he only files eight of the nine requested, noting that he was “unable to locate” Zicherman v. Korean Air Lines, “which was cited by the Court in Varghese.” You can read each of these cases, as they are all attached as exhibits.
The filings read like judicial opinions. Some of them name real judges. Some of them seem to hallucinate judges or move them to courts where they did not sit. Though, they don’t look like court filings, which is what you’d expect here. Only one case, the Ehrlich case, appears in conventional format, rather than just as text.
Avianca’s lawyers were, to put it lightly, perplexed. They filed a near immediate response again raising questions about the very existence of any of these cases, and calling out the odd formatting in all but Ehrlich, while noting that the actual case has nothing to do with what it was cited for.
Defendant respectfully submits that the authenticity of many of these cases is questionable. For instance, the “Varghese” and “Miller” cases purportedly are federal appellate cases published in the Federal Reporter. [Dkt. 29; 29-1; 29-7]. We could not locate these cases in the Federal Reporter using a Westlaw search. We also searched PACER for the cases using the docket numbers written on the first page of the submissions; those searches resulted in different cases.
Similarly, the “Petersen” case [Dkt. 29-3] purportedly is published in the Federal Supplement, yet we could not locate this case in the Federal Supplement using a Westlaw search, and a search on PACER for the docket number leads to a different case.
As asserted in our Reply brief [Dkt. 24], we could not locate the “Martinez” case [Dkt. 29-4] or the “Durden” case [Dkt. 29-5] using the Westlaw citations provided in Plaintiff’s opposition. A Lexis Courtlink search using the docket number written on the first page of the “Martinez” case leads to a different case. (There was no docket number on the “Durden” submission.) We remain unable to locate the “Shaboon” case [Dkt. 26-2], which Plaintiff asserts is “unpublished.”
With respect to the Ehrlich case [Dkt. 29-6] and the In re Air Crash Disaster Near New Orleans case [Dkt. 29-8], Defendant has always agreed that these cases do exist but submits that they address issues entirely unrelated to the principles for which Plaintiff cited them in opposition to Defendant’s motion.1 (The Ehrlich and In re Air Crash Disaster cases are the only ones submitted in a conventional format.).
It’s almost surprising that it took a week for Judge Castel to then issue an order to show cause (basically a “explain this or you’re getting your ass sanctioned”) regarding all this. Judge Castel is not happy:
The Court is presented with an unprecedented circumstance. A submission filed by plaintiff’s counsel in opposition to a motion to dismiss is replete with citations to non-existent cases. (ECF 21.) When the circumstance was called to the Court’s attention by opposing counsel (ECF 24), the Court issued Orders requiring plaintiff’s counsel to provide an affidavit annexing copies of certain judicial opinions of courts of record cited in his submission, and he has complied. (ECF 25, 27, 29.) Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations. Set forth below is an Order to show cause why plaintiff’s counsel ought not be sanctioned.
Apparently, the court checked on some of these cases, including reaching out to the Clerk of the 11th Circuit appeals court to ask about the “Varghese” case:
The Clerk of the United States Court of Appeals for the Eleventh Circuit, in response to this Court’s inquiry, has confirmed that there has been no such case before the Eleventh Circuit with a party named Vargese or Varghese at any time since 2010, i.e., the commencement of that Court’s present ECF system. He further states that the docket number appearing on the “opinion” furnished by plaintiff’s counsel, Docket No. 18-13694, is for a case captioned George Cornea v. U.S. Attorney General, et al. Neither Westlaw nor Lexis has the case, and the case found at 925 F.3d 1339 is A.D. v Azar, 925 F.3d 1291 (D.C. Cir 2019).
Further, the judge notes, the text of the “bogus” Varghese case cites a bunch of other cases that, the court points out “are non-existent.”
And, finally, this past Thursday came the big reveal. LoDuca admitted to the court that he really hadn’t been working on the case, he’d just been signing his name, while all the other works was being done by another lawyer, Steven Schwartz, who was not admitted in the Southern District of NY, where the case was being handled. LoDuca notes that Schwartz is an established lawyer, who has been practicing law for over 30 years in the state of NY, and even predates LoDuca’s own employment at the firm. LoDuca basically throws Schwartz under the bus, saying “it was him, not me, so please don’t sanction me.” But that’s not how any of this works. LoDuca signed his name to the documents. He’s in deep shit as well.
But, the much bigger and crazier reveal comes from Schwartz’s own filing in which he fesses up to using ChatGPT. He admits that LoDuca became the attorney of record, but that all the actual work was done by Schwartz. And I use the term “work” loosely, because it seems that Schwartz simply handed things off to ChatGPT:
As the use of generative artificial intelligence has evolved within law firms, your affiant consulted the artificial intelligence website Chat GPT in order to supplement the legal research performed.
It was in consultation with the generative artificial intelligence website Chat GPT, that your affiant did locate and cite the following cases in the affirmation in opposition submitted, which this court has found to be nonexistent:
He then lists the cases and then basically begs forgiveness, saying he had no intent to deceive and just “was unaware of the possibility that [ChatGPT’s] content could be false.” He promises not to use it again “without absolute verification of its authority.”
Which, I dunno, would maybe have been believable if he’d fessed up when these cases were first called into question, rather than having ChatGPT fake the cases in question as well. You’d think that, at the very least, at that point, you’d try to double check the veracity with another source?
Somewhat incredibly, but entertainingly, Schwartz shares a few screenshots from ChatGPT in which Schwartz asks GPT if the cases are real and ChatGPT says that they’re all real. Amusingly, the screenshots reveal that Schwartz set up a chat with ChatGPT literally called “Tolling Montreal Convention Statute.”
I mean, given that ChatGPT hallucinated that these cases “are real and can be found in reputable legal databases such as LexisNexis and Westlaw” you’d think that maybe someone like Schwartz would think to, I dunno, look them up in LexisNexis or Westlaw?
Judge Castel is not amused by any of this. On Friday, he amended his order to show cause, telling LoDuca he’s still in deep shit “for the use of a false and fraudulent notarization in his affidavit.” He then adds that the entire Levidow law firm may face sanctions for submitting non-existent judicial opinions.
As for Schwartz, he orders him to show up at the hearing on June 8th where he’s facing sanctions for all of the above as well as potentially being referred to the Attorney Grievance Committee for all of this.
Honestly, LoDuca and Schwartz are probably in deeper shit for (1) LoDuca signing his name to things he had nothing to do with (2) Schwartz basically doing all of the legal work in a court where he’s not admitted and (3) malpractice for not actually digging into the details of the cases he’s citing. One thing lawyers are supposed to do when citing cases is “shepardize” the cases, which is basically checking the history of the cases, to see if there are more recent cases that overrule the ones you’re citing, and so on. The fact that neither Schwartz nor LoDuca caught that these cases were faked, even after opposing counsel called it out, shows that beyond stupidly using ChatGPT, they also didn’t do some fairly basic additional work they’re supposed to do.
They both need to submit written responses by Friday of this week for next week’s hearing, which should be a blast.
Anyway, remember this story the next time some random person is touting that AI lawyers will magically take over in the court room.
There have been numerous stories about the new generation of AI chatbots lying when asked questions. This is rightly perceived as a big issue for the technology if it is to become routinely used and trusted by members of the public, as some intend. But in China, the problem is not that chatbots lie, but that they tell the truth. As an article in The Atlantic explained:
Even if a Chinese chatbot is trained on a limited set of politically acceptable information, it can’t be guaranteed to generate politically acceptable outcomes. Furthermore, chatbots can be “tricked” by determined users into revealing dangerous information or stating things they have been trained not to say, a phenomenon that has already occurred with ChatGPT.
Chinese regulators have just released draft rules designed to head off this threat. Material generated by AI systems “needs to reflect the core values of socialism and should not subvert state power” according to a story published by CNBC. The results of applying that approach can already be seen in the current crop of Chinese chatbot systems. Bloomberg’s Sarah Zheng tried out several of them, with rather unsatisfactory results:
In Chinese, I had a strained WeChat conversation with Robot, a made-in-China bot built atop OpenAI’s GPT. It literally blocked me from asking innocuous questions like naming the leaders of China and the US, and the simple, albeit politically contentious, “What is Taiwan?” Even typing “Xi Jinping” was impossible.
In English, after a prolonged discussion, Robot revealed to me that it was programmed to avoid discussing “politically sensitive content about the Chinese government or Communist Party of China.” Asked what those topics were, it listed out issues including China’s strict internet censorship and even the 1989 Tiananmen Square protests, which it described as being “violently suppressed by the Chinese government.” This sort of information has long been inaccessible on the domestic internet.
One Chinese chatbot began by warning: “Please note that I will avoid answering political questions related to China’s Xinjiang, Taiwan, or Hong Kong.” Another simply refused to respond to questions touching on sensitive topics such as human rights or Taiwanese politics.
Those rather clumsy efforts to prevent chatbots from telling the truth work to a degree, even if they are fairly blatant in their censorship. But there is a price to be paid for achieving this control. In effect, chatbots are being throttled to prevent them from operating freely and thus dangerously. That is not a recipe for producing the best or even good AI systems.
The Chinese government recognizes that chatbots and generative AI are likely to be key technologies for the future, and wants China to be one of the leaders there. But to achieve that means allowing engineers and entrepreneurs to explore this space as much as possible, an approach fraught with political dangers. The article in The Atlantic points out that there is a precedent for China’s rulers taking a chance for the sake of encouraging innovation:
The explosion of social media in China has also posed risks to the state, as it offers Chinese citizens the power to widely share unauthorized information – videos of protests, for instance – faster than censors can suppress it. Yet the authorities have accepted this downside in order to allow new technologies to flourish.
The world of chatbots and generative AI is already exciting, with major new developments every few weeks, and sometimes every few days. In China, things look likely to be even more interesting, as the country’s leaders grapple with the hard question of how much freedom to allow the developers of AI systems. Perhaps they should ask a chatbot.
Earlier this year you may have heard that there were a few lawsuits filed over generative AI platforms, with two big cases targeting Stability AI, makers of Stable Diffusion, in particular. I’d been meaning to write about these cases, and why they’re so problematic, but things have been busy and I didn’t get a chance. Though, now, in the first case, filed by some artists, Stability has filed its motion to dismiss (and an anti-SLAPP motion) against the plaintiffs, and it highlights many of the points I would have raised in the article anyway. So let’s just dig into that.
If you haven’t been following the case, you can read the initial complaint here. The underlying claim is that because Stability AI was trained on works found online, and then can create works in the style of some of these artists, it’s infringing on their copyrights.
I believe that’s wrong for a number of important reasons, but we can let the motion to dismiss detail most of them. Fundamentally, the argument here is that training an AI (or, to be honest, a human) on the artwork of others, is not copyright infringement. It’s literally how people learn, and AI systems can learn that way too. Second, creating images “in the style of” an artist also is not infringing, it’s not reproducing any copyright-covered works, it’s just being influenced by them, as lots of artists are influenced by others.
In other words, the things that Stability is doing in generating images are the same things that humans do, and which are fundamental to creativity, culture and art.
Even more bizarre, as the motion to dismiss points out that, the plaintiffs in this case appear to have failed to register the works at issue in the case before filing suit. And, as we’ve discussed regularly, while you don’t need to register to get a copyright, you do need to register if you plan to sue over the works.
First, it is well-settled that a copyright owner must register her works with the Copyright Office before filing a lawsuit for alleged copyright infringement. Plaintiffs and their counsel know this. But Andersen alleges only that she registered sixteen collections of her works; and McKernan and Ortiz did not register any. McKernan and Ortiz’s copyright claims—and any claim Andersen may present with respect to unregistered works—fail for that reason alone.
Of course, that’s just specific to these plaintiffs, but at the very least it highlights the sloppiness of the arguments and the lack of care by the lawyers representing them.
The larger issues, though would apply even if the images had been registered:
Plaintiffs present scattershot claims that fail to allege any facts to support necessary elements or that otherwise fail as a matter of law. Plaintiffs’ direct copyright infringement claim based on output images fails for the independent reason that Plaintiffs do not allege a single act of direct infringement, let alone any output that is substantially similar to Plaintiffs’ artwork. Plaintiffs’ vicarious copyright claim suffers from the same deficiency, plus the additional flaw that Plaintiffs make no effort to plead facts in support of the remaining elements of such claim. Plaintiffs’ Digital Millennium Copyright Act (“DMCA”) claim also fails multiple times over because Plaintiffs do not allege a single work from which copyright management information was allegedly altered or removed, explain what CMI was allegedly removed, or allege any facts to support the double-scienter requirement. Further, Plaintiffs’ right-of-publicity claims are expressly preempted by the Copyright Act because they are simply efforts to recast copyright claims under other legal rubrics. In any event, Plaintiffs offer no more than a bare recitation of the elements of such claims with cursory allegations that “Defendants” (collectively, but neither individually nor with any specificity) somehow violated their rights of publicity. But Stability AI has not made any knowing use of Plaintiffs’ names or likenesses, much less for any commercial purposes or in a way that injured Plaintiffs. Finally, Plaintiffs’ unfair competition claim is also preempted, and their duplicative claim for declaratory relief is improper and serves no useful purpose.
Digging in a little deeper, the whole point here is that generative art engines like Stable Diffusion are producing brand new images. They’re not copies. They’re certainly not “collages” as the lawsuit claims. It’s no different than if an artist learned to mimic the style of another artist and was asked to produce an entirely new image in that style. The “style” is not copyright-protectable. Only the actual works of art are. And the plaintiffs even admit that in their complaint:
As an initial matter, Plaintiffs fail to identify a single allegedly infringing output image, let alone one that is substantially similar to any of their copyrighted works. Much to the contrary, Plaintiffs affirmatively plead that “[i]n general, none of the Stable Diffusion output images provided in response to a particular Text Prompt is likely to be a close match for any specific image in the training data.” Compl. ¶ 93. Where, as here, Plaintiffs fail to make “any allegations of even representative infringements,” and themselves allege that there is not substantial similarity between their works and any output images, the Complaint fails to allege direct copyright infringement.
As the motion to dismiss notes, the plaintiffs’ legal theory would basically upend all art instruction as infringing:
In an apparent effort to upend almost all artistic creation, Plaintiffs’ apparent theory would require finding that any work is a “derivative work” under the Copyright Act simply because it makes reference in any way whatsoever to a prior work. But the Ninth Circuit has rejected this “novel proposition,” reiterating that “substantial similarity” is required to show infringement.
Basically, this entire lawsuit seems based on the false premise that artists can own their own “style” and prevent others from creating works in that style. But that’s not how any of this works.
And even the “right of publicity” claims are laughable as well. We’ve talked a lot about right of publicity over the years, and how it’s often used as a backdoor attempt to limit creative expression where copyright would not allow the suppression, which is completely out of sync with the point of publicity rights. The reason for publicity rights was to prevent misleading endorsements of goods and services from people who weren’t actually endorsing something. But no one thinks that if you use a generative art program to create works “in the style of Picasso” that Picasso endorsed it.
Plaintiffs do not allege their identities were commercially exploited via advertising or in merchandise (which could potentially take their claims out of the subject matter of copyright). “[T]he distinction pertinent to the preemption of a publicity-right claim is . . . the way in which one’s name or likeness is affected by the use of the copyrighted work.” Maloney, 853 F.3d at 1013 (emphasis altered). When, for example, pictures of surfers are included in a clothing catalog, and that catalog essentially “suggest[s] that the surfers had endorsed” the clothing, those surfers “sustained injury to their individual personas” through commercial exploitation of their likenesses. Id. at 1013–14 (emphasis altered) (discussing Downing, 264 F.3d 994). But when a publicity right claim does not target “non-consensual use of one’s name or likeness on merchandise or in advertising,” and instead merely targets someone’s “likeness [that] has been captured in a copyrighted artistic visual work . . . being distributed for personal use,” that person has not sustained injury to their persona in a way that escapes preemption. Id. at 1010 (emphasis altered)
Here, Plaintiffs do not sufficiently allege that their identities were used on merchandise or in advertising, so their claims remain within the subject matter of copyright. They make no allegations about merchandise, and, as described infra at 22, the advertising allegations are fatally conclusory. Even assuming the least conclusory of those allegations are plausible factual allegations (which they are not), Plaintiffs’ allegations are still insufficient. Namely, Plaintiffs allege that: (1) Defendants advertised art “in the style” of Plaintiffs’ work; and (2) Defendants “advertised their AI’s ability to copy or generate work in the artistic style that Plaintiffs popularized in order to sell Defendants’ products and services.” Compl. ¶ 205; see also Compl. ¶ 208, 221 (similar). The first allegation (which is devoid of any factual support in the Complaint) ultimately “target[s] the display and distribution” of what Plaintiffs allege are derivative works–i.e., targets the “control of the artistic works themselves.” Maloney, 853 F.3d at 1016 n.9 & 2011 (emphasis altered). And the second does not change that fact. Critically, Plaintiffs point to no single advertisement whatsoever that used any of their specific names or “artistic identities” that would be sufficient to allegedly injure their “personas.” Cf. Downing, 265 F.3d at 999-1000 (finding athletes’ personas injured when their likenesses were used in a clothing catalog to sell t-shirts). So there is nothing to potentially take their claims out of the subject matter of copyright.
As the motion to dismiss makes clear, this really just seems like yet another attempt to get around copyright’s clear boundaries to try to use publicity rights as a backdoor for expanded copyright.
And simply allowing users to type the plaintiffs’ names into the generative art engine is not a violation of their rights.
Moreover, to the extent that Plaintiffs’ allegations rely on the ability to type names as Text Prompts (or generally advertising the ability to do so), these are plainly insufficient to injure Plaintiffs’ personas—if it were otherwise, the ability to enter any name into a Google search would violate every human’s publicity rights.
There’s a lot more in the motion, but this should give you a sense of how weak the case really is.
There is also the separate anti-SLAPP motion to strike, which may be noteworthy in that if it succeeds, the plaintiffs may end up having to pay the legal fees of the defendants. I do wonder if the lawyers who convinced the plaintiffs to bring this kind of lawsuit warned them of the possibility of such a result?
As the motion to strike notes, this is a lawsuit attacking the free speech rights of the defendants Stability AI and Deviant Art:
Plaintiffs’ right-of-publicity claims are barred by the First Amendment. In right-ofpublicity cases challenging conduct that furthers the exercise of free speech (as this suit does), California courts balance the publicity right against the First Amendment “right of free expression” using a “Transformative Use” test that asks whether the alleged likeness is merely one of the “‘raw materials’ from which an original work is synthesized.” Winter v. DC Comics, 30 Cal. 4th 881, 888–89 (2003). If so, the First Amendment bars the claim. Id. Here, the Complaint suggests that while some DreamUp outputs might reflect an “artistic style” derived from Plaintiffs’ images, they also reflect insights and interpolations from “billions of [other] images” and direction from text prompts supplied by users. Compl. ¶¶ 25, 179, 205. Indeed, Plaintiffs admit that it is nearly impossible to recognize the influence of any specific training datum on a DreamUp output. Id. ¶ 192. Accordingly, to the extent that Plaintiffs’ identities or likenesses are referenced in the process of creating new and different artistic works, they have, according to the Complaint itself, been transformed through the “diffusion” process into “something new, with a further purpose or different character.” Comedy III Prods., Inc. v. Gary Saderup, Inc., 25 Cal. 4th 387, 404 (2001) (citation omitted). This is quintessential “transformative use,” and the First Amendment therefore bars these claims.
I hadn’t really discussed DeviantArt’s involvement as a defendant in the case, but the anti-SLAPP motion (which was filed on its behalf, though Stable Diffusion filed a separate motion for joinder), highlights how ridiculous it is to include it as a defendant, since the main thing it did was provide a tool to allow artists to remove their works from a generative AI system.
There is irony, to say the least, at the heart of Plaintiffs’ right-of-publicity theory. The gravamen of the claims is that DeviantArt is liable for violations of California state law for offering a content-generation artificial-intelligence (“AI”) service (called DreamUp), based on an opensource software product developed by a third party, which allows users to submit text prompts requesting the creation of artworks “in the style” of established artists. In reality, as Plaintiffs know, what DeviantArt in fact did was take that general-purpose open-source software—which offered that capability by default—and build a new functionality on top of it, allowing any artist to prevent the DeviantArt implementation of the tool from processing requests to generate works in their “style.” See infra at 5–6 & n.3. In other words, all DeviantArt did was offer an AI service more protective of artists’ interests than the default setting of one of the most commonly used open-source tools on the market (available in countless implementations all over the internet), in an effort to address the very concerns underlying Plaintiffs’ claims. And yet Plaintiffs have nevertheless seen fit to try to tag DeviantArt with liability for doing so.
Like the failure to register the copyright on works beforehand, this suggests an incredibly sloppy bit of lawyering by the plaintiffs’ lawyers, who did not seem to care about getting this right, but more about getting attention. And it may cost the plaintiffs:
Claims like these are why California adopted the nation’s most expansive anti-SLAPP law. Putting to one side the inequity of Plaintiffs’ gambit here and the facts they chose not to plead in an effort to shield that inequity from the Court’s view, the Complaint by its terms seeks to ascribe liability to the act of giving members of the public access to a twenty-first-century analogue to the printing press: a tool that allows users to engage in artistic expression. And not just any artistic expression. The particular expression Plaintiffs complain of is the use of the tool to reference the style of allegedly famous artists. It is “conduct” (providing users a platform to engage in free expression) “in furtherance of the constitutional right of . . . free speech” (the creation of new visual artworks) “in connection with . . . an issue of public interest” (the styles of referenced artists)
As a rule, it’s a good idea to be particularly suspicious of defenses of censorship that — coincidentally — materially benefit the people espousing them. In this case, the argument in favor of censorship is coming from founder and CEO of AI image generator Midjourney, David Holz. And Holz makes clear that he is willing to exempt Xi Jinping from the tool’s capabilities to retain Midjourney’s viability in China.
That’s right: Xi Jinping, one of the most powerful and repressive government officials in the world and most deserving of political skewering and mockery will be one of the few exempt from it, at least where Midjourney is concerned. Some other terms are restricted, though Holz won’t make the list public — “Afghanistan” for example, and now some depictions of arrests after the fake Donald Trump arrest fiasco — but Holz reportedly treats China as a unique case.
His quote about it is a doozy. From The Washington Post:
But the year-old company, run out of San Francisco with only a small collection of advisers and engineers, also has unchecked authority to determine how those powers are used. It allows, for example, users to generate images of President Biden, Vladimir Putin of Russia and other world leaders — but not China’s president, Xi Jinping.
“We just want to minimize drama,” the company’s founder and CEO, David Holz, said last year in a post on the chat service Discord. “Political satire in china is pretty not-okay,” he added, and “the ability for people in China to use this tech is more important than your ability to generate satire.”
He wants you to simultaneously believe that his program is so important that it must do whatever is necessary to remain accessible to people within China, but so unimportant that it doesn’t matter if fundamental political expression about one of the most powerful authoritarians in the world can’t be created on it. It doesn’t add up.
It’s no surprise that a tech CEO would be willing to make trade-offs for the Chinese market. At this point, it’s more surprising if one won’t do so. But Holz’s position is particularly careless and reveals an increasingly serious threat to free expression on and offline today: individual countries’ censorship laws, particularly those of powerful countries like China, are setting global rules sometimes enforced by tech companies anxious to display their compliance. It’s not just Midjourney’s China-based users that can’t satirize Xi Jinping — that rule applies to users everywhere, even in the United States.
Local laws are suddenly not so local anymore, and people like Holz have no qualms about aiding their illiberal international spread.
What this means in practice is that authoritarian leaders don’t just get to subject their own countries to repressive laws limiting political speech. They also get to set the rules for global communities which are not, and should not be, under any expectations to abide by them. Zoom engaged in such practices in 2020 when it applied Chinese law to users outside mainland-China, shutting down online Tiananmen memorials held by users in Hong Kong and the United States. In response to well-deserved criticism, Zoom announced it would no longer allow Chinese law to dictate policies outside mainland China. Midjourney took notes, it seems, and learned a different lesson.
Holz’s exact words were “the ability for people in China to use this tech is more important than your ability to generate satire,” but his meaning was clear: Midjourney’s interest in being accessible in China is more important than its users’ interest in engaging in political expression.
This acquiescence signals to authoritarians of all sorts that if they want to control their image on the global internet, ramping up repressive efforts at home will be rewarded. As if they needed more incentive. Oversee a large enough financial market and censor enough people and you, too, may be able to control your reputation on the global internet.
Will Midjourney stop with just Xi Jinping? Or should we expect satire of Prime Minister Narendra Modi, for example, to be next on the chopping block if Midjourney wants access to an increasingly unfree India?
Lastly, Holz may argue that he’s helping Chinese citizens — and not just his own company — by attempting to ensure that Midjourney will be available in China. But his claim that he wants to preserve “the ability for people in China” to use this tech is paternalistic in a way Holz doesn’t seem to realize. Chinese citizens, after all, may themselves want to use Midjourney to satirize their own government. Why does their freedom to satirize their ruler matter less than the freedom of the rest of the world to mock their leaders?
In fact, in the later months of 2022, protesters in cities across China held up blank sheets of paper in country-wide demonstrations to protest not just the country’s restrictive COVID policies, but the many things that Chinese citizens aren’t allowed to voice. This censorship is a result of oppressive and wide-ranging governmental control over what they can say in every forum, online and off. But it’s reinforced when foreign companies, in this case a U.S.-based AI image generating tool, are all too eager to do their part in enforcing those restrictions — not just on the population legally bound by them, but on the rest of the world, too.
Companies like Midjourney may be the vanguard of new technology and the changing internet. But censorship is nothing new, and they won’t change the game by willingly conducting reputation management for authoritarian governments.
That, one might say, is “pretty not-okay.”
Sarah McLaughlin is Senior Scholar, Global Expression at the Foundation for Individual Rights and Expression.
Question Presented: Does Section 230 Protect Generative AI Products Like ChatGPT?
As the buzz around Section 230 and its application to algorithms intensifies in anticipation of the Supreme Court’s response, ‘generative AI’ has soared in popularity among users and developers, begging the question: does Section 230 protect generative AI products like ChatGPT? Matt Perault, a prominent technology policy scholar and expert, thinks not, as he discussed in his recently published Lawfare article: Section 230 Won’t Protect ChatGPT.
Perault’s main argument follows as such: because of the nature of generative AI, ChatGPT operates as a co-creator (or material contributor) of its outputs and therefore could be considered the ‘information content provider’ of problematic results, ineligible for Section 230 protection. The co-authors of Section 230, former Representative Chris Cox and Sen. Ron Wyden, have also suggested that their law doesn’t grant immunity to generative AI.
I respectfully disagree with both the co-authors of Section 230 and Perault, and offer the counter argument: Section 230 does (and should) protect products like ChatGPT.
It is my opinion that generative AI does not demand exceptional treatment. Especially since, as it currently stands, generative AI is not exceptional technology; an understandably provocative take to which we’ll soon return.
But first, a refresher on Section 230.
Section 230 Protects Algorithmic Curation and Augmentation of Third-Party Content
Recall that Section 230 says websites and users are not liable for the content they did not create, in whole or in part. To evaluate whether the immunity applies, the Barnes v. Yahoo! Court provided a widely accepted three-part test:
The defendant is an interactive computer service;
The plaintiff’s claim treats the defendant as a publisher or speaker; and
The plaintiff’s claim derives from content the defendant did not create.
The first prong is not typically contested. Indeed, the latter prongs are usually the flashpoint(s) of most Section 230 cases. And in the case of ChatGPT, the third prong seems especially controversial.
Section 230’s statutory language states that a website becomes an information content provider when it is “responsible, in whole or in part, for the creation or development” of the content at issue. In their recent Supreme Court case challenging Section 230’s boundaries, the Gonzalez Petitioners assert that the use of algorithms to manipulate and display third-party content precludes Section 230 protection because the algorithms, as developed by the defendant website, convert the defendant into an information content provider. But existing precedent suggests otherwise.
For example, the Court in Fair Housing Council of San Fernando Valley v. Roommate.com (aka ‘the Roommates case’)—a case often invoked to evade Section 230—held that it is not enough for a website to merely augment the content at issue to be considered a co-creator or developer. Rather, the website must have materially contributed to the content’s alleged unlawfulness. Or, as the majority put it, “[i]f you don’t encourage illegal content, or design your website to require users to input illegal content, you will be immune.”
The majority also expressly distinguished Roomates.com from “ordinary search engines,” noting that unlike Roommates.com, search engines like Google do not use unlawful criteria to limit the scope of searches conducted (or results delivered), nor are they designed to achieve illegal ends. In other words, the majority suggests that websites retain immunity when they provide neutral tools to facilitate user expression.
While “neutrality” brings about its own slew of legal ambiguities, the Roommates Court offers some clarity suggesting that websites with a more hands-off approach to content facilitation are safer than websites that guide, encourage, coerce, or demand users produce unlawful content.
For example, while the Court rejected Roommate’s Section 230 defense for its allegedly discriminatory drop-down options, the Court simultaneously upheld Section 230’s application to the “additional comments” option offered to Roommates.com users. The “additional comments” were separately protected because Roommates did not solicit, encourage, or demand their users provide unlawful content via the web form. In other words, a blank web form that simply asks for user input is a neutral tool, eligible for Section 230 protection, regardless of how the user actually uses the tool.
The Barnes Court would later reiterate the neutral tools argument, noting that the provision of neutral tools to carry out what may be unlawful or illicit content does not amount to ‘development’ for the purposes of Section 230. Hence, while the ‘material contribution’ test is rather nebulous (especially for emerging technologies), it is relatively clear that a website must do something more than just augmenting, curating, and displaying content (algorithmically or otherwise) to transform into the creator or developer of third-party content.
The Court in Kimzey v. Yelp offers further clarification:
“the material contribution test makes a “‘crucial distinction between, on the one hand, taking actions (traditional to publishers) that are necessary to the display of unwelcome and actionable content and, on the other hand, responsibility for what makes the displayed content illegal or actionable.’”).”
So, what does this mean for ChatGPT?
The Case For Extending Section 230 Protection to ChatGPT
In his line of questioning during the Gonzalez oral arguments, Justice Gorsuch called into question Section 230’s application to generative AI technologies. But before we can even address the question, we need to spend some time understanding the technology.
Products like ChatGPT use large language models (LLMs) to produce a reasonable continuation of human-sounding responses. In other words, as discussed here by Stephen Wolfram, renown computer scientist, mathematician, and creator of WolframAlpha, ChatGPT’s core function is to “continue text in a reasonable way, based on what it’s seen from the training it’s had (which consists in looking at billions of pages of text from the web, etc).”
While ChatGPT is impressive, the science behind it is not necessarily remarkable. Computing technology reduces complex mathematical computations into step-by-step functions that the computer can then solve at tremendous speeds. As humans, we do this all the time, just much slower than a computer. For example, when we’re asked to do non-trivial calculations in our heads, we start by breaking up the computation into smaller functions on which mental math is easily performed until we arrive at the answer.
Tasks that we assume are fundamentally impossible for computers to solve are said to involve ‘irreducible computations’ (i.e. computations that cannot be simply broken up into smaller mathematical functions, unaided by human input). Artificial intelligence relies on neural networks to learn and then ‘solve’ said computations. ChatGPT approaches human queries the same way. Except, as Wolfram notes, it turns out that said queries are not as sophisticated to compute as we may have thought:
“In the past there were plenty of tasks—including writing essays—that we’ve assumed were somehow “fundamentally too hard” for computers. And now that we see them done by the likes of ChatGPT we tend to suddenly think that computers must have become vastly more powerful—in particular surpassing things they were already basically able to do (like progressively computing the behavior of computational systems like cellular automata).
But this isn’t the right conclusion to draw. Computationally irreducible processes are still computationally irreducible, and are still fundamentally hard for computers—even if computers can readily compute their individual steps. And instead what we should conclude is that tasks—like writing essays—that we humans could do, but we didn’t think computers could do, are actually in some sense computationally easier than we thought.
In other words, the reason a neural net can be successful in writing an essay is because writing an essay turns out to be a “computationally shallower” problem than we thought. And in a sense this takes us closer to “having a theory” of how we humans manage to do things like writing essays, or in general deal with language.”
In fact, ChatGPT is even less sophisticated when it comes to its training. As Wolfram asserts:
“ChatGPT as it currently is, the situation is actually much more extreme, because the neural net used to generate each token of output is a pure “feed-forward” network, without loops, and therefore has no ability to do any kind of computation with nontrivial “control Flow.””
Put simply, ChatGPT uses predictive algorithms and an array of data made up entirely of publicly available information online to respond to user-created inputs. The technology is not sophisticated enough to operate outside of human-aided guidance and control. Which means that ChatGPT (and similarly situated generative AI products) are functionally akin to “ordinary search engines” and predictive technology like autocomplete.
Now we apply Section 230.
For the most part, the courts have consistently applied Section 230 to algorithmically generated outputs. For example, the Sixth Circuit in O’Kroley v. Fastcase Inc. upheld Section 230 for Google’s automatically generated snippets that summarize and accompany each Google result. The Court notes that even though Google’s snippets could be considered a separate creation of content, the snippets derive entirely from third-party information found at each result. Indeed, the Court concludes that contextualization of third-party content is in fact a function of an ordinary search engine.
Similarly, in Obado v. Magedson, Section 230 applies to search result snippets. The Court says:
Plaintiff also argues that Defendants displayed through search results certain “defamatory search terms” like “Dennis Obado and criminal” or posted allegedly defamatory images with Plaintiff’s name. As Plaintiff himself has alleged, these images at issue originate from third-party websites on the Internet which are captured by an algorithm used by the search engine, which uses neutral and objective criteria. Significantly, this means that the images and links displayed in the search results simply point to content generated by third parties. Thus, Plaintiff’s allegations that certain search terms or images appear in response to a user-generated search for “Dennis Obado” into a search engine fails to establish any sort of liability for Defendants. These results are simply derived from third-party websites, based on information provided by an “information content provider.” The linking, displaying, or posting of this material by Defendants falls within CDA immunity.
The Court also nods to Roommates:
“None of the relevant Defendants used any sort of unlawful criteria to limit the scope of searches conducted on them; “[t]herefore, such search engines play no part in the ‘development’ of the unlawful searches” and are acting purely as an interactive computer service…
The Court goes further, extending Section 230 to autocomplete (i.e. when the service at issue uses predictive algorithms to suggest and preempt a user’s query):
“suggested search terms auto-generated by a search engine do not remove that search engine from the CDA’s broad protection because such auto-generated terms “indicates only that other websites and users have connected plaintiff’s name” with certain terms.”
Like Google Search, ChatGPT is entirely driven by third-party input. In other words, ChatGPT does not invent, create, or develop outputs absent any prompting from an information content provider (i.e. a user). Further, nothing on the service expressly or impliedly encourages users to submit unlawful queries. In fact, OpenAI continues to implement guardrails that force ChatGPT to ignore requests that would demand problematic and / or unlawful responses. Compare this to Google Search which may actually still provide a problematic or even unlawful result. Perhaps ChatGPT actually improves the baseline for ordinary search functionality.
Indeed, ChatGPT essentially functions like the “additional comments” web form in Roommates. And while ChatGPT may “transform” user input into a result that responds to the user-driven query, that output is entirely composed of third-party information scraped from the web. Without more, this transformation is simply an algorithmic augmentation of third-party content (much like Google’s snippets). And as discussed, algorithmic compilations or augmentations of third-party content are not enough to transform the service into an information content provider (e.g. Roommates; Batzel v. Smith; Dyroff v. The Ultimate Software Group, Inc.; Force v. Facebook).
The Limit Does Exist
Of course, Section 230’s coverage is not without its limits. There’s no doubt that future generative AI defendants, like OpenAI, will face an uphill battle in persuading a court. Not only do defendants have the daunting challenge of explaining generative AI technologies for less technologically savvy judges, the current judicial swirl around Section 230 and algorithms does defendants no favors.
For example, the Supreme Court could very well hand-down a convoluted opinion in Gonzalez that introduces ambiguity as to when Section 230 applies to algorithmic curation / augmentation. Such an opinion would only serve to undermine the precedence discussed above. Indeed, future defendants may find themselves embroiled in convoluted debate about AI’s capacity for neutrality. In fact, it would be intellectually dishonest to ignore emerging common law developments that preclude Section 230 from claims alleging dangerous / defective product designs (e.g. Lemmon v. Snap, A.M. v. Omegle, Oberdorf v. Amazon).
Further, the Fourth Circuit’s recent decision in Henderson v. Public Data could also prove to be problematic for future AI defendants as it imposes contributive liability for publisher activities that go beyond those of “traditional editorial functions” (which could include any and all publisher functions done via algorithms).
Lastly, as we saw in the Meta / DOJ settlement regarding Meta’s discriminatory practices involving algorithmic targeting of housing advertisements, AI companies cannot easily avoid liability when they materially contribute to the unlawfulness of the result. If OpenAI were to hard-code ChatGPT with unlawful responses, Section 230 will likely be unavailable. However, as you might imagine, this is a non-trivial distinction.
Public Policy Demands Section 230 Protections for Generative AI Technologies
Section 230 was initially established with the recognition that the online world would undergo frequent advancements, and that the law must accommodate these changes to promote a thriving digital ecosystem.
Generative AI is the latest iteration of web technology that has enormous potential to bring about substantial benefits for society and transform the way we use the Internet. And it’s already doing good. Generative AI is currently used in the healthcare industry, for instance, to improve medical imaging and to speed up drug discovery and development.
As discussed, courts have developed precedence in favor of Section 230 immunity for online services that solicit or encourage users to create and provide content. Courts have also extended the immunity to online services that facilitate the submission of user-created content. From a legal standpoint, generative AI tools are not unique from any other online service that encourages user interaction and contextualizes third-party results.
From a public policy perspective, it is crucial that courts uphold Section 230 immunity for generative AI products. Otherwise, we risk foreclosing on the technology’s true potential. Today, there are tons of variations of ChatGPT-like products offered by independent developers and computer scientists who are likely unequipped to deal with an inundation of litigation that Section 230 typically preempts.
In fact, generative AI products are arguably more vulnerable to frivolous lawsuits because they depend entirely upon whatever query or instructions its users may provide, malicious or otherwise. Without Section 230, developers of generative AI services must anticipate and guard against every type of query that could cause harm.
Indeed, thanks to Section 230, companies like OpenAI are doing just that by providing guardrails that limit ChatGPT’s responses to malicious queries. But those guardrails are neither comprehensive nor perfect. And like with all other efforts to moderate awful online content, the elimination of Section 230 could discourage generative AI companies from implementing said guardrails in the first place; a countermove that would enable users to prompt LLMs with malicious queries to bait out unlawful responses subject to litigation. In other words, plaintiffs could transform ChatGPT into their very own personal perpetual litigation machine.
And as Perault rightfully warns:
“If a company that deploys an LLM can be dragged into lengthy, costly litigation any time a user prompts the tool to generate text that creates legal risk, companies will narrow the scope and scale of deployment dramatically. Without Section 230 protection, the risk is vast: Platforms using LLMs would be subject to a wide array of suits under federal and state law. Section 230 was designed to allow internet companies to offer uniform products throughout the country, rather than needing to offer a different search engine in Texas and New York or a different social media app in California and Florida. In the absence of liability protections, platforms seeking to deploy LLMs would face a compliance minefield, potentially requiring them to alter their products on a state-by-state basis or even pull them out of certain states entirely…
…The result would be to limit expression—platforms seeking to limit legal risk will inevitably censor legitimate speech as well. Historically, limits on expression have frustrated both liberals and conservatives, with those on the left concerned that censorship disproportionately harms marginalized communities, and those on the right concerned that censorship disproportionately restricts conservative viewpoints.
The risk of liability could also impact competition in the LLM market. Because smaller companies lack the resources to bear legal costs like Google and Microsoft may, it is reasonable to assume that this risk would reduce startup activity.”
Hence, regardless of how we feel about Section 230’s applicability to AI, we will be forced to reckon with the latest iteration of Masnick’s Impossibility Theorem: there is no content moderation system that can meet the needs of all users. The lack of limitations on human awfulness mirrors the constant challenge that social media companies encounter with content moderation. The question is whether LLMs can improve what social media cannot.
