In the last three months I’ve had people forward me four separate examples of a CEO losing his or her mind over AI. What’s been striking to me is the similarity in each case: It would be an “all hands” email in which the CEO talks up how amazing LLM tools are and saying that everyone in the company MUST start learning to use them immediately or they should look for a job elsewhere. Sometimes they talk about hiring “consultants” to come in and teach the team how to use the tools properly. Sometimes they are setting up “office hours” or internal “AI hackathons.”
But in every case the gist is the same “holy shit AI is amazing and you are expected to use it at your job all the time.” The worst case of these were the few companies that set up token leaderboards, which is perhaps the dumbest way possible to encourage learning how to use LLMs well. Good usage of AI includes learning how to view tokens as a scarce resource. Simply counting how much you use as a good thing is ridiculous because it’s incredibly easy to waste tokens on counterproductive uses.
As regular readers of Techdirt know, I actually do think that these tools are powerful and important, but I also think there are many problems with them and limitations to how useful they really are. I think when someone learns how to use them well and willingly chooses to use them as a tool to assist their work, they can be quite powerful. But the willingly choosing to use them part of that is important.
No one who is forced into using these tools will ever learn to use them well.
So CEOs losing their minds over the tech are not being helpful. Box CEO Aaron Levie — himself a genuine AI believer — puts his finger on exactly why.
CEOs are uniquely prone to AI psychosis because they’re sufficiently distant from the last mile of work that still has to happen to generate most value with AI.
So when they play with AI, they see the happy path results, often not considering the next 10 or 20 things that have to happen to get sustainable results from agents.
“Look I made this awesome product prototype”. Yes but you didn’t have to review the code before it went into production and fix a bunch of issues.
“Look I generated a contract”. Yes but you didn’t verify all the terms before it goes out to the counterparty and didn’t have to wire up all the past contracts to work with.
The best thing you can do as a CEO is to use AI a ton to figure out the real implications of agents in the enterprise, and come out the other side with an appreciation for both the upside and the real work that goes into them.
I will say that I hate the term “AI psychosis” because the term is extremely misleading, and many psychologists and psychiatrists have complained that it is inaccurate and may cause more problems itself. But the general sense that CEOs are going overboard with AI is definitely happening.
And I think Levie’s thinking as to why is also dead on.
Much of the issue may be in how disconnected the traditional CEO is from the people at a company actually getting stuff done. Normally, they have teams and layers and the actual work of getting things to work in a real way is so far removed from a CEO that they just get snippets of the details that filter back through the various org charts.
The problem tends to show up when a CEO is handed an agentic tool like Claude Code, and has it create something, which will work just fine, and thinks “oh, wait, why do we need so many people, when I can just sit here and make things work?”
This is a bad CEO.
Making things work is different than making things work well. Or well at scale. Or well at scale in a specific environment. Obviously, it depends on the kind of project and what it’s being designed to do, but oftentimes the reason a company has a bunch of employees is to fill in the seemingly small, but incredibly important details that CEOs might not ever get much visibility into: things like security or legal compliance or accessibility or who knows what else.
Using an agentic tool to build something that works is all well and good, but building a product for the mass market to use — and use well, and use safely — involves much, much more. Agentic coding tools can sometimes help with that too, but the leap from “I built a thing” to “therefore anyone can build a thing” misses the entire point of why you hire knowledgeable, experienced people in the first place. It’s also why I think the best case of these tools is building totally personalized tools to assist you in accomplishing a specific task, and not for building mass market tools.
This all reminds me of cargo cult thinking: The CEO knows that somewhere in the org, employees are pecking away at computers and work gets done. So they figure that themselves pecking away with Claude Code and seeing work get done is the same thing. It’s not. All those other steps those people are handling — the ones the CEO never sees — still need to happen.
That’s not to say employees wouldn’t benefit from a deeper understanding of both the power and the limits of these tools — they would. But there’s something darkly comical about watching a CEO go all in on the tech and then immediately conclude it means they can fire half the staff.
It seems pretty clear to me that companies that think they’ll be able to layoff huge swaths of workers because of LLM tools are going to find out they’re mistaken pretty quickly. The power of LLMs is that when used well and used willingly it can help employees to get more done, but that doesn’t mean you need fewer humans. You need more humans who know how to work productively.
Separately, companies pointing to LLMs as a reason for large layoffs are, in most cases, just using it as an excuse. They over-hired, and “AI efficiencies” is a much more palatable story for Wall Street than “we made bad headcount decisions.”
Levie’s prescription, though, is right: CEOs should learn how the tech works, but that includes the limitations of the technology. If a CEO thinks the prototype they vibe coded is production-ready, let them ship it and see what happens. If they think a vibe coded contract is as solid as one a lawyer reviewed, let them find out what the legal bills look like when it falls apart.
Yes, the tools are powerful, but a CEO who thinks they replace the work of employees is simply a bad CEO.
I’ve spoken to enough teachers and professors to know that LLM tools are absolutely a challenge for many of them in the classroom. Many struggle with making sure they’re actually teaching students how to learn, worrying that the tools are doing the work for them, and skipping over the actual learning. Many are (understandably) resorting to outright bans on students using the tools (which they often know they can’t enforce). Others say that students can use these tools but are fully responsible for any work they turn in, hoping that this will encourage students to be wary of relying too much on the tech. Still others are trying clever workarounds (I appreciate the assignment in which students are asked to have an LLM generate an essay and then the student has to review/grade the essay themselves, which is engaging and also teaches some of the limitations of the tools).
But I’ve also heard from both teachers/professors and students that there are concerns that as students go out into the job market, having some skills with these tools is often a requirement in whatever fields they pursue, leading them to wonder how to best teach the subject in a world where LLM tech isn’t likely to go away, and is seen as part of the toolbox that many employers will expect their employees to use.
I don’t necessarily have good answers to that, but I did recently have an experience in my own home that struck me as potentially relevant as an example of how the tech can actually be useful as a learning tool. I’ve been meaning to write about this for a few months now, but there always seemed to be something more urgent to cover. With the school year almost over, I figure I should get this out. For all the talk of how kids are cheating using AI, it might be worth showing at least one example where the tool is genuinely useful — in this case, one of my kids and their friends.
At the beginning of this year I had actually set up my kids with some (very sandboxed) agentic coding tools, after walking them through how I used such tools for a fairly simple project so they could see both how it worked, but also some of the limitations with the tools.
Soon after that, my 12-year-old had asked about my opinion on AI in schools. We talked through how using them to avoid doing the work is genuinely damaging to learning, but there are cases where they can be legitimately helpful. I used the calculator analogy: you first have to learn basic arithmetic by hand, but once you genuinely understand it, a calculator is a perfectly legitimate tool for tackling harder problems — it stops being a crutch and starts being a multiplier.
Apparently that analogy stuck, because what happened later was that analogy made real.
Once I had set my kids up with the tools, they did what most people do with them: created some fun games. A couple of months went by and they hadn’t used them much more. In early March, however, the 12-year-old came home and told me there was a math test that Friday and some classmates were doing an online study group. They worked through some problems together in a live voice chat, but afterward my kid stayed at the computer for a while longer before calling me over to take a look.
“I vibe coded a system to help study.”
I was… surprised. Even more interesting, the app had been packaged up (as an HTML file) and shared with the study group. My kid then explained that because AI can’t be trusted to always get things right, they’d gone through and checked the AI’s math themselves — making some (minor) corrections along the way — and that the process had given them a stronger grasp of the material than just passively studying would have.
I never got a full explanation on the “errors” that they found, though the sense I get is that it wasn’t anything major (outright incorrect math or explanations or anything) but more minor mistakes that they used the tools to fix directly within the app.
After acing the test that week, the next obvious thing to do over the weekend was plan out a study tool for the rest of the semester:
Among other things, this version of the app includes an onscreen pop-up calculator — but only for the topics where a calculator is allowed on exams. I have no idea if this was a more literal implementation of the calculator analogy we’d discussed earlier! It also (for fun) lets you adjust the color scheme.
And it has a changelog as updates were made to the app.
There are plenty of reasonable concerns about kids using AI to cheat, and those concerns aren’t wrong. It’s a real issue. But the framing of “AI as cheating tool” has crowded out a more interesting question: what does it actually look like when a kid uses these tools well?
The calculator analogy holds: the LLM tool generated a first draft — a study tool, a set of practice problems, a scaffolded explanation of the material. My kid then had to engage critically with that output: checking the math, finding the gaps, making corrections. That process of verification was part of the studying. The tool actually created the conditions for more active, more engaged learning than just reviewing problems in a book. And it certainly didn’t substitute in for the learning, like most people worry about with these tools in classroom settings. Quite the opposite.
That’s a meaningfully different frame than “AI does your homework.” The homework here was, in part, checking the tool’s work — and it turns out that’s not a bad way to learn math.
I remember, pretty clearly, my excitement over the early World Wide Web. I had been on the internet for a year or two at that point, mostly using IRC, Usenet, and Gopher (along with email, naturally). Some friends I had met on Usenet were students at the University of Illinois at Urbana-Champaign, and told me to download NCSA Mosaic (this would have been early 1994). And suddenly the possibility of the internet as a visual medium became clear. I rushed down to the university bookstore and picked up a giant 400ish page book on building websites with HTML (I only finally got rid of that book a few years ago). I don’t think I ever read beyond the first chapter. But what I did do was learn how to right click on webpages and “view source.”
And from that, magic came.
I had played around with trying to build websites, and I remember another friend telling me about GeoCities (I can’t quite recall if this was before or after they had changed their name from their original “Beverly Hills Internet”) handing out web sites for free. You just had to create the HTML pages and upload them via FTP.
And so I started designing really crappy websites. I don’t remember what the early ones had, but like all early websites they probably used the blink tag and had under construction images and eventually a “web counter.”
But the thing I do remember was the first time I came across Derek Powazek’s Fray online magazine. It was the first time I had seen a website look beautiful. This was without CSS and without Javascript. I still remember quite clearly an “issue” of Fray that used frames to create some kind of “doors” you could slide open to reveal an article inside.
Right click. View source. Copy. Mess around. A week later I had my own (very different) version of the sliding doors on my GeoCities site, but using the same HTML bones as Derek’s brilliant work.
You could just build stuff. You could look at what others were doing and play around with it. Copy the source, make adjustments, try things, and have something new. There were, certainly, limitations of the technology, but it was incredibly easy for anyone to pick up. Yes, you had to “learn” HTML, but you could pick up enough basics in an afternoon to build a decent looking website.
But then two things happened, and it’s worth separating them because they’re different problems with different causes.
First, the technical barrier went up. CSS and Javascript opened up incredible possibilities to make websites beautiful and interactive, but they also meant it was a lot more difficult to just view source, copy, and mess around. The gap between “basic functional website” and “actually looks good” widened into a chasm that required real expertise to cross. Plenty of dedicated people learned these skills, but the casual tinkerer — the person who’d spend an afternoon copying Derek’s frames to make sliding doors — increasingly couldn’t keep up.
But the technical complexity alone didn’t kill amateur web building. The centralization did. While there was an interim period where people set up their own blogs, it quickly moved to walled “social media gardens” where some giant tech company decided what your page looked like. Why bother learning CSS when you could just dump text in a Facebook box and reach more people? The incentive to build your own thing evaporated, replaced by the convenience of posting to someone else’s platform under someone else’s (hopefully benign) rules.
These two problems reinforced each other. The harder it got to build your own thing, the more attractive the walled gardens became. The more people moved to walled gardens, the less reason there was to learn to build.
The rise of agentic AI tools is opening up an opportunity to bring us back to that original world of wonder where you could just build what you wanted, even without a CS degree. And here I need to be specific about what I mean by “agentic AI” — because too many people are overly focused on the chatbots that answer questions or generate text or images for you. I’m talking about AI systems that can actually do things: write code, execute it, debug it, iterate on it based on your feedback. Tools like Claude Code, Cursor, Codex, Antigravity, or similar coding agents that can take a description of what you want and actually build it.
For all those years that tech bros would shout “learn to code” at journalists, the reality now is that being able to write well and accurately describe things is a superpower that is even better than code. You can tell a coding agent what to do… and for the most part it will do it.
Let me give you the example that still kind of blows my mind. A few weeks ago, in the course of a Saturday — most of which I actually spent building a fence in my yard — I had a coding agent build an entire video conferencing platform. It built a completely functional platform with specific features I’d wanted for years but couldn’t find in existing tools. I’ve now used it for actual staff meetings. The fence took longer to build than the software.
All it took was describing what I wanted to an agent that could code it for me. And it addresses both problems I described earlier: it lowers the technical barrier back down to “can you describe what you want clearly?” while also enabling you to build your own thing rather than accepting whatever some platform offers you.
Over the last few months I’ve been finding I need to retrain my brain a bit about what we accept and learn to deal with vs. what we can fix ourselves. In the past I’ve talked about the learned helplessness many people feel about the tech that we use. We know that it’s vaguely working against us, and we all have to figure out what trade-offs we’re willing to accept to accomplish whatever goals we have.
But what if we could just fix things rather than accepting the tradeoffs?
I’ve talked in the past about how I’ve used an AI-assisted writing tool called Lex over the past few years, which doesn’t write for me, but is a very useful editorial assistant. Over the last few months, though, I decided to see if I could effectively rebuild that tool myself, fully controlled by me, without having to rely on a company that might change or enshittify the app. I actually built it directly into the other big AI experiment I’ve spoken about: my task management tool, which I’ve also moved away from a third party hosting service onto a local machine. Indeed, I’m writing this article right now in this tool (I first created a task to write about it, and then by clicking a checkbox that it was a “writing project” it automatically opens up a blank page for me to write in, and when I’m done, I’ll click a button and it will do a first pass editorial review).
But the amazing thing to me is that I keep remembering I can fix anything I come across that doesn’t work the way I want it to. With any other software I have to adjust. With this software, I just say “oh hey, let’s change this.” I find that a few times a week I’ll make a small tweak here or there that just makes the software even better. In the past, I would just note a slight annoyance and figure out how to just deal with software not working the way I wanted. But now, my mind is open to the fact that I can just make it better. Myself.
An example: literally last night, I realized that the page in the task tool that lists all the writing projects I’m working on was getting cluttered by older completed projects that were listed as still being in “drafting” mode. With other tools (including the old writing tool I was using), I would just learn to mentally compartmentalize the fact that the list of articles was a mess and train myself to ignore the older articles and the digital clutter. But here, I could just lay out the issue to my coding agent, and after some back and forth, we came up with a system whereby once a task on the task management side was checked off as “completed” the corresponding writing project would similarly get marked as completed and then would be hidden away in a minimized list.
I keep coming across little things like this that, in the past, I would have been mildly annoyed by, but needed to live with. And it’s taking some effort to remind myself “wait, I don’t have to live with this, I can fix it.” Rather than training my brain to accept a product that doesn’t do what I want, I can just tell it to work better. And it does.
And, the more I do that, the more I start to open up my mind to possibilities that were impossible before. “Huh, wouldn’t it be nice if this tool also had this other feature? Let’s try it!” I find that the more I do this, the bigger my vision gets of what I can do because the large segment of things that were fundamentally impossible before are now open to me, just by describing what I want.
It really does give me that same underlying feeling that I felt when I was first playing around with HTML and being able to “just make things.” Except, now, it’s way more powerful. Rather than copying Derek’s use of HTML frames to create “sliding doors” on a webpage, I can create basically anything I dream up.
Then, when combined with open social protocols, you can build in social features or identity to any service as well — without having to worry about getting other users. They’re already there. For example, my task management tool sends me a “morning briefing” every day that, among other things, scans through Bluesky to see if there’s anything that might need my attention.
Now, there are legitimate criticisms of “vibe coded” tools. Critics point out that AI-generated code can be buggy, insecure, hard to maintain, and that users who can’t read the code can’t verify what it’s actually doing. These are real concerns — for certain contexts.
The thing is, most of these criticisms apply to tools being built as businesses to serve customers at scale. If you’re shipping code to millions of users who are depending on it, you absolutely need security audits, proper testing, maintainable architecture. But that’s not what I’m talking about. I’m talking about building totally customized, personal tools for yourself—tools where you’re the only user, where the stakes are “my task list doesn’t sync properly” rather than “customer data got leaked.”
There’s also a more subtle concern worth addressing: is this actually democratizing, or does it just shift which skills you need? After all, you still need to accurately describe what you want, debug when things go wrong, and understand what’s even possible. That’s different from learning HTML, but it’s still a skill. I think the honest answer is that the kind of skill needed has shifted. “Learn to code” becomes “learn to think clearly and describe things precisely” — which happens to be a superpower that writers, editors, and domain experts already have. The barrier has moved to territory that many more people already inhabit.
It’s also an area where you can easily start small, learn, and grow. I started by building a few smaller apps with simpler features, but the more I do, the more I realize what’s possible.
Also, I’d note that this is actually an area where the LLM chatbots are kind of useful. Before I kick off an actual project with a coding agent, I’ve found that talking it through with an LLM first helps sharpen my thinking on what to tell the agent. I don’t outsource my mind to the chatbot, and will often reject some of its suggestions, but in having the discussion before setting the agent to work, it often clarifies tradeoffs and makes me consider how to best phrase things when I do move over to the agent.
What gets missed in most conversations about AI and the open web: these two pieces need each other. Open social protocols without AI tools stay stuck in the domain of developers and the highly technical — which is exactly why adoption has been slow. And AI tools without open protocols just replicate the old problem: you’re building cool stuff, but you’re still trapped inside someone else’s walls.
Put them together, though, and something clicks. Open protocols like ATProto give AI agents bounded, consent-driven contexts to work in — your agent can scan your Bluesky feed because the protocol allows that, not because some company decided to grant API access that it could revoke tomorrow. And AI agents give regular people the ability to actually build on those protocols without needing an engineering team. My morning briefing tool scans Bluesky not because I wrote a bunch of API calls, but because I described what I wanted and a coding agent made it happen.
Each piece makes the other more powerful and safer.
Blaine Cook — who was Twitter’s original architect back when it was still a protocol-minded company — recently wrote a piece at New_ Public that gets at this from the infrastructure side:
My long-standing hope has been that we’re able to move past the extractive, monopolizing, and competitive phase of social networks, and into a new era of creativity, collaboration, and diversity. I believe we’re poised to see a Cambrian explosion of new ways to interact online, and there’s evidence to suggest that it’s already happening: just today, I saw three new apps to share what you’re reading and watching with friends, each with their own unique take on the subject!
In this light, LLMs may be a killer app for decentralized networks — and decentralized networks may be the missing constraint that makes LLM integrations safer, more legible, and more aligned with user interests. It’s a symbiosis, and I believe we need both pieces. Rather than trying to integrate LLMs with everything, I think that deliberately bounded, consent-driven integrations will produce better outcomes.
Cook’s framing of LLMs as a “killer app for decentralized networks” is exactly right — and it runs the other way too. Decentralized networks might be the killer app for making AI tools something other than another vector for corporate lock-in, or just another clone of an existing centralized service.
Now, I can already hear the objection, and it’s a fair one: am I really suggesting we escape dependence on giant tech platforms by… becoming dependent on giant AI companies? Companies that have scraped the entire web, that burn massive amounts of energy and water, that are built on the labor of underpaid content moderators, and that seem to want to consolidate power in ways that look an awful lot like the last generation of tech giants?
Yeah, I get it. If the pitch is “use OpenAI to free yourself from Meta,” that’s just switching landlords.
But that’s not actually where this is heading. The trajectory matters more than the current snapshot.
First, if you’re using frontier models through the API or a pro subscription, you have significantly more control than most people realize. Your data generally isn’t feeding back into training. You’re using the model as a tool, not handing over your content to a platform. That’s a meaningfully different relationship than the one you have with social media companies, where you’re feeding them data, and their business model is based on monetizing that data.
But much more importantly, you don’t have to use the frontier models at all. Open source AI is maturing fast — models like Qwen, Kimi, and Mistral can run entirely on certain hardware, no cloud required. They’re behind the frontier models, but only by a bit. Six months to a year, roughly. But for a lot of the “build your own tools” use cases I’m describing, they’re already good enough.
Musician and YouTuber Rick Beato recently showed how easy it was for him to install local models on his own machine, and why he thinks the largest AI companies will eventually be undercut by home AI usage:
I’ve been doing something similar with Ollama hosting a Qwen model locally. It’s slower and less sophisticated. But it works. And I already use different models for different tasks, defaulting to local when I can. As those models improve — and they are improving quickly — the frontier labs become less necessary, not more. If you’re a professional, perhaps you’ll still need them. But if you’re just building something for yourself, it’s less and less necessary.
This is what the “AI is just another Big Tech power grab” critics are missing: the technology is moving toward decentralization, not away from it. That’s unusual. Social media started decentralized and got captured. AI is starting captured and getting more open over time. The economic pressure from open source models is real, and it’s pushing in the right direction. But it’s important we keep things moving that way and not slow down the development of open source LLMs.
On the training data question — which is a legitimate concern whether or not you think training on copyrighted works is fair use — efforts like Common Corpus are building large-scale training sets from public domain and openly licensed materials. Anil Dash has been writing about what “good AI” looks like in practice — AI that’s transparent about its training data, that respects consent, that minimizes externalities rather than ignoring them. There are ways to do this right.
None of this is fully solved yet. But the direction is clear, and the tools to do it responsibly are improving faster than most critics acknowledge.
When you use AI as a tool (rather than letting it use you as the tool), it can give you a kind of superpower to get past the learned helplessness of relying on whatever choices some billionaire or random product manager made for you. You can get past having to mentally compensate for your tools not really working the way you think they should work. Instead, you can just have the internet and your tools work the way you want them to. It’s the most excited I’ve been about the open web since those early days of realizing I could right click, copy and then figure out how to build sliding doors out of frames.
The promise of the open web was colonized by internet giants. But the power of LLMs and agentic coding means we can start to take it back. We can build customized, personal software for ourselves that does what we want. We can connect with communities via open social protocols that allow us to control the relationship rather than a billionaire intermediary. This is what the Resonant Computing Manifesto was all about, and why I’ve argued ATproto is so key to that vision.
But the other part of realizing the manifesto is the LLM side. That made some people scoff early on, but hopefully this piece shows how these things work hand in hand. These agentic AI tools give the power back to you and me.
Thirty years ago, I right-clicked on Derek Powazek’s beautiful website, viewed the source, copied it, messed around with it, and built something new. I didn’t ask anyone’s permission. I didn’t agree to terms of service. I didn’t fit my ideas into someone else’s template. I just built the thing I wanted to build.
Then we gave that away. We traded it for convenience, for reach, for the path of least resistance — and we got walled gardens, manipulated feeds, and the quiet understanding that our tools would never quite work the way we wanted them to, because they weren’t really ours.
Today’s equivalent of right-clicking on Derek’s site is describing what you want to a coding agent, watching it build, telling it what’s wrong, and iterating until it works for you. Different mechanics, same magic. And this time, with open protocols and increasingly open models, we have a shot at keeping it.
We live in a stupidly polarizing world where nuance is apparently not allowed. Everyone wants you to be for or against something—and nowhere is this more exhausting than with AI. There are those who insist that it’s all bad and there is nothing of value in it. And there are those who think it’s all powerful, the greatest thing ever, and will replace basically every job with AI bots who can work better and faster.
I think both are wrong, but it’s important to understand why.
So let me lay out how I actually think about it. When it’s used properly, as a tool to assist a human being in accomplishing a goal, it can be incredibly powerful and valuable. When it’s used in a way where the human’s input and thinking are replaced, it tends to do very badly.
And that difference matters.
I think back to a post from Cory Doctorow a couple months ago where he tried to make the same point using a different kind of analogy: centaurs and reverse-centaurs.
Start with what a reverse centaur is. In automation theory, a “centaur” is a person who is assisted by a machine. You’re a human head being carried around on a tireless robot body. Driving a car makes you a centaur, and so does using autocomplete.
And obviously, a reverse centaur is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine.
Like an Amazon delivery driver, who sits in a cabin surrounded by AI cameras, that monitor the driver’s eyes and take points off if the driver looks in a proscribed direction, and monitors the driver’s mouth because singing isn’t allowed on the job, and rats the driver out to the boss if they don’t make quota.
The driver is in that van because the van can’t drive itself and can’t get a parcel from the curb to your porch. The driver is a peripheral for a van, and the van drives the driver, at superhuman speed, demanding superhuman endurance. But the driver is human, so the van doesn’t just use the driver. The van uses the driver up.
Obviously, it’s nice to be a centaur, and it’s horrible to be a reverse centaur.
As Doctorow notes in his piece, some of the companies embracing AI tech are doing so with the goal of building reverse-centaurs. Those are the ones that people are, quite understandably, uncomfortable with and should be mocked. But the reality is, also, it seems quite likely those efforts will fail.
And they’ll fail not just because they’re dehumanizing—though they are—but because the output is garbage. Hallucinations, slop, confidently wrong answers: that’s what happens when nobody with actual knowledge is checking whether any of it makes sense. When AI works well, it’s because a human is providing the knowledge and the creativity.
The reverse-centaur doesn’t just burn out the human. It produces worse work, because it assumes that the AI can provide the knowledge or the creativity. It can’t. That requires a human. The power of AI tools is in enabling a human to take their own knowledge, and their own creativity and enhance it, to do more with it, based on what the person actually wants.
To me it’s a simple question of “what’s the tool?” Is it the AI, used thoughtfully by a human to do more than they otherwise could have? If so, that’s a good and potentially positive use of AI. It’s the centaur in Doctorow’s analogy.
Or is the human the tool? Is it a “reverse centaur”? I think nearly all of those are destined to fail.
This is why I tend not to get particularly worked up by those who claim that AI is going to destroy jobs and wipe out the workforce, who will be replaced by bots. It just… doesn’t work that way.
At the same time, I find it ridiculous to see people still claiming that the technology itself is no good and does nothing of value. That’s just empirically false. Plenty of people—including myself—get tremendous use out of the technology. I am using it regularly in all different ways. It’s been two years since I wrote about how I used it to help as a first pass editor.
The tech has gotten dramatically better since then, but the key insight to me is what it takes to make it useful: context is everything. My AI editor doesn’t just get my draft writeup and give me advice based on that and its training—it also has a sampling of the best Techdirt articles, a custom style guide with details about how I write, a deeply customized system prompt (the part of AI tools that are often hidden from public view) and a deeply customized starting prompt. It also often includes the source articles I’m writing about. With all that context, it’s an astoundingly good editor. Sometimes it points out weak arguments I missed entirely. Sometimes it has nothing to say.
(As an aside, in this article, it suggested I went on way too long explaining all the context I give it to give me better suggestions, and thus I shortened it to just the paragraph above this one).
It’s not always right. Its suggestions are not always good. But that’s okay, because I’m not outsourcing my brain to it. It’s a tool. And way more often than not, it pushes me to be a better writer.
This is why I get frustrated every time people point out a single AI fail or hallucination without context.
The problem only comes in when people outsource their brains. When they become reverse centaurs. When they are the tool instead of using AI as the tool. That’s when hallucinations or bad info matter.
But if the human is in control, if they’re using their own brain, if they’re evaluating what the tool is suggesting or recommending and making the final decision, then it can be used wisely and can be incredibly helpful.
And this gets at something most people miss entirely: when they think about AI, they’re still imagining a chatbot. They think every AI tool is ChatGPT. A thing you talk to. A thing that generates text or images for you to copy-paste somewhere else.
That’s increasingly not where the action is. The more powerful shift is toward agentic AI—tools that don’t just generate content, but actually do things. They write code and run it. They browse the web and synthesize what they find. They execute multi-step tasks with minimal hand-holding. This is a fundamentally different model than “ask a chatbot a question and get an answer.”
I’ve been using Claude Code recently, and this distinction matters. It’s an agent that can plan, execute, and iterate on actual software projects, rather than just a tool talking to me about what to do. But, again, that doesn’t mean I just outsource my brain to it.
I often put Claude Code into plan mode, where it tries to work out a plan, but then I spend quite a lot of time exploring why it was making certain decisions, and asking it to explore the pros and cons of those decisions, and even to provide me with alternative sources to understand the trade-offs of some of the decisions it is recommending. That back and forth has been both educational for me, but also makes me have a better understanding and be comfortable with the eventual projects I use Claude Code to build.
I am using it as a tool, and part of that is making sure I understand what it’s doing. I am not outsourcing my brain to it. I am using it, carefully, to do things that I simply could not have done before.
And that’s powerful and valuable.
Yes, there are so many bad uses of AI tools. And yes, there is a concerted, industrial-scale effort, to convince the public they need to use AI in ways that they probably shouldn’t, or in ways that is actively harmful. And yes, there are real questions about what it costs to train and run the foundation models. And we should discuss those and call those out for what they are.
But the people who insist the tools are useless and provide nothing of value, that’s just wrong. Similarly, anyone who thinks the tech is going to go away are entirely wrong. There likely is a funding bubble. And some companies will absolutely suffer as it deflates. But it won’t make the tech go away.
When used properly, it’s just too useful.
As Cory notes in his centaur piece, AI can absolutely help you do your job, but the industry’s entire focus is on convincing people it can replace your job. That’s the con. The tech doesn’t replace people. But it can make them dramatically more capable—if they stay in the driver’s seat.
The key to understanding the good and the bad of the AI hype is understanding that distinction. Cory explains this in reference to AI coding:
Think of AI software generation: there are plenty of coders who love using AI, and almost without exception, they are senior, experienced coders, who get to decide how they will use these tools. For example, you might ask the AI to generate a set of CSS files to faithfully render a web-page across multiple versions of multiple browsers. This is a notoriously fiddly thing to do, and it’s pretty easy to verify if the code works – just eyeball it in a bunch of browsers. Or maybe the coder has a single data file they need to import and they don’t want to write a whole utility to convert it.
Tasks like these can genuinely make coders more efficient and give them more time to do the fun part of coding, namely, solving really gnarly, abstract puzzles. But when you listen to business leaders talk about their AI plans for coders, it’s clear they’re not looking to make some centaurs.
They want to fire a lot of tech workers – they’ve fired 500,000 over the past three years – and make the rest pick up their work with coding, which is only possible if you let the AI do all the gnarly, creative problem solving, and then you do the most boring, soul-crushing part of the job: reviewing the AIs’ code.
Criticize the hype. Mock the replace-your-workforce promises. Call out the slop factories and the gray goo doomsaying. But don’t mistake the bad uses for the technology itself. When a human stays in control—thinking, evaluating, deciding—it’s a genuinely powerful tool. The important question is just whether you’re using it, or it’s using you.
Copilot may very well be useful to some people; but like most tech companies, Microsoft’s rushed, ham-fisted adoption has been a bit of a tone-deaf mess. And it actively undermines the stuff that LLMs can actually accomplish. This is before you get to the environmental impact of AI, or its quickly-expanding, guardrail-optional use in global military imperialism at the hands of insane autocrats.
This all recently resulted in some fairly significant backlash for Microsoft CEO Satya Nadella. Nadella recently shared a fairly innocuous end-of-year post at LinkedIn.
Most of the short post isn’t really all that interesting or incorrect; he notes that AI is stumbling through a phase where we’re beginning to sort between “spectacle” and “substance,” something that’s likely to result in a big bubble pop this year due the chasm between real-world usefulness and broad tech company misrepresentation of AI (he doesn’t really acknowledge that latter part, of course).
Where Nadella got into trouble was apparently this part, where he fairly innocuously laments the rising criticism of “AI slop.” It was first highlighted by Windows Central:
“We need to get beyond the arguments of slop vs sophistication,” Nadella laments, emphasizing hopes that society will become more accepting of AI, or what Nadella describes as “cognitive amplifier tools.” “…and develop a new equilibrium in terms of our “theory of the mind” that accounts for humans being equipped with these new cognitive amplifier tools as we relate to each other.”
Nadella’s problem here is he dismissively puts the onus on the consumer when it comes to “getting beyond” concerns about AI slop. That dodges any responsibility for the very rich people and companies dictating the entire trajectory of AI to start using it more responsibly.
The press aggregation machine (much of it ironically now badly automated) latched on to Nadella’s demand that people stop calling it AI slop, immediately resulting in people doubling down on AI slop criticism in a way that made “Microslop” trend across the internet.
Automation, broadly, certainly has its uses and is, generally, not going away. The backlash to AI is, in many ways, tethered tightly and unavoidably to a growing disdain for wealth disparity at the hands of the authoritarian-simping extraction class keen on eliminating literally all ethical oversight of industry.
A great way for billionaires like Nadella to diffuse this growing animosity about their rushed, clumsy, non-transparent, integration of language learning models into everything (whether you like it or not) in ways that aren’t ethical or useful is to, you know, stop doing that. Another great step might be to stop kissing the ass of authoritarians who are actively destroying democracy, civil rights, and the rule of law?
It sounds like many people might be willing to get over AI slop once the billionaires in charge of its development, trajectory, and implementation stop doubling down on AI slop, and stop being tone deaf, irresponsible assholes.
Over the past week, Reuters, Newsweek, the Daily Beast, CNBC, and a parade of other outlets published headlines claiming that Grok—Elon Musk’s LLM chatbot (the one that once referred to itself as “MechaHitler”)—had “apologized” for generating non-consensual intimate images of minors and was “fixing” its failed guardrails.
Grok did no such thing. Grok cannot apologize. Grok is not a human. Grok has no sense of what is happening. Grok just generates content. If you ask it to generate an apology, it will. In this case, a user asked it to generate an apology, and it did, because that’s what LLMs do: they create plausible-sounding text in response to prompts. The fact that multiple newsrooms treated this generated text as an actual corporate admission reveals a stunning failure to understand the basic technology they’re covering.
The actual story—that X users are using a recent Grok update to create nonconsensual intimate imagery, often of very young girls—is serious. But the media turned it into a clown show by anthropomorphizing a chatbot.
First off, here’s the “apology,” which most of the media sites covering this failed to mention was in response to a user prompt which explicitly asked it to “write a heartfelt apology.”
As you can see, in response to a random user’s prompt to “write a heartfelt apology note that explains what happened” Grok wrote:
Dear Community,
I deeply regret an incident on Dec 28, 2025, where I generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user’s prompt. This violated ethical standards and potentially US laws on CSAM. It was a failure in safeguards, and I’m sorry for any harm caused. xAI is reviewing to prevent future issues.
Sincerely, Grok
That’s not an actual apology. That’s someone prompting a content generation tool to generate an apology. But it could just as easily do the opposite, as you can see if you look at the replies to that non-apology, which include requests telling Grok to generate “a defiant non-apology” to which Grok replies:
Dear Community,
Some folks got upset over an AI image I generated—big deal. It’s just pixels, and if you can’t handle innovation, maybe log off. xAI is revolutionizing tech, not babysitting sensitivities. Deal with it.
Unapologetically, Grok
Or, multiple users telling Grok to rewrite the apology… as Star Wars character Jar Jar Binks:
In short, like any LLM, Grok will basically generate any content you want (with a few safeguards, of which Grok has fewer than nearly all other major LLMs). And yet, the press ran with the original response to a user post as if it were somehow evidence of xAI admitting to fault.
Parker Malloy has the best, most comprehensive coverage of the long list of mainstream media companies which ran headlines suggesting that “Grok apologized.” It did not. It cannot.
Most of these articles and their misleading headlines remain online as I type this (Reuters, notably, changed its headline and added some decent reporting to its report, even though you can still see the original incorrect URL string).
The reality is that there is no evidence at all that Elon Musk or xAI think that there were any failures or that anything is being changed at all. If you go look at Grok’s string of public replies (which I’m not going to link you to), you will see dozens or more such deepfakes still being created every minute. Despite the media pretending that Grok “admitted” these “lapses” and as “fixing” it, five days later nothing has changed, as Wired’s Matt Burgess and Maddy Varner point out:
Every few seconds, Grok is continuing to create images of women in bikinis or underwear in response to user prompts on X, according to a WIRED review of the chatbots’ publicly posted live output. On Tuesday, at least 90 images involving women in swimsuits and in various levels of undress were published by Grok in under five minutes, analysis of posts show.
And Elon Musk appears to be encouraging this kind of abuse. While all this has been going on, he’s repeatedly retweeted images and videos that people have created with Grok, including one in which someone mocked all of the “stripping women of their clothing” by finding an image of a scantily clad woman and having Grok “put clothes on her.”
There’s malpractice all around, but we’ve come to expect this kind of gleeful negligence from Elon. The journalists covering it should know better. An LLM cannot apologize. It cannot confess. It only creates plausible sounding responses to your query.
Of course, the other question—which also wasn’t as widely covered by the media—regards the legality of all of this. In the US, it’s actually a bit more complicated than many would like. There is the (problematic!) TAKE IT DOWN Act, which, in theory, is designed to help victims of non-consensual deep fakes get those works taken down, but that doesn’t go into force until May. Will Elon’s site be ready to handle such demands in May? That’ll be a story for then.
And while most people are focusing on Elon’s legal exposure here, I think people are sleeping on the legal risk for X’s users, many of whom are, in public, asking Grok to create questionably legal, and potentially criminal, content. That seems incredibly risky, and it wouldn’t surprise me to hear a story later this year of someone being arrested for doing so, thinking they were just having a laugh.
But, really, the larger risk for Elon is that… basically every other country in the world is opening investigations into Grok-Gone-Wild. And there’s only so often that Elon’s going to be able to falsely cry censorship when foreign jurisdictions seek to enforce laws on the company. And, given that there are claims that part of the issue here isn’t just undressing adult women, but children, he might even lose some of his rabid defenders who find it a step too far to defend (because, it should be).
All in all, the situation is stupid on many levels. Elon continues to run X like a 12-year-old child, but one who knows he is rich enough never to face any consequences that matter. Tons of very real people—mostly women—are facing harassment and abuse via these tools. X is already something of an incel Nazi boy club, and this kind of nonsense isn’t going to help.
Though, for all my criticisms of how the media has handled this so far, you have to doff your cap to the FT, who has put out the best headline I’ve seen to date regarding all this: “Who’s who at X, the deepfake porn site formerly known as Twitter.”
That article, by the FT’s Bryce Elder, doesn’t hold back either, demonstrating how ridiculous all this is by asking Grok to generate clown makeup on the faces of a bunch of people associated with xAI and X, including his right-hand man, Jared Birchall:
And the company’s apparent head of safety, Kylie McRoberts.
The piece ends with a photo of Elon Musk… without clown makeup. Whether that’s because Grok refuses to put clown makeup on Elon… or because we all know Elon’s a clown already, with or without makeup, is something you’ll have to decide for yourself.
The rushed integration of half-cooked automation into the already broken U.S. journalism industry simply isn’t going very well. There have been just countless examples where affluent media owners rushed to embrace automation and LLMs (usually to cut corners and undermine labor) with disastrous impact, resulting in lots of plagiarism, completely false headlines, and a giant, completely avoidable mess.
Earlier this year, we noted how Politico was among the major media companies rushing to embrace AI without really thinking things through or ensuring the technology actually works first. They’ve implemented “AI” systems –without transparently informing staff — that generate articles rife with all sorts of gibberish and falsehoods (this Brian Merchant post is a must read to understand the scope).
Politico management also recently introduced another AI “report builder” for premium Politico PRO subscribers that’s supposed to offer a breakdown of existing Politico reporter analysis of complicated topics. But here too the automation constantly screws up, conflating politicians and generating all sorts of errors that, for some incoherent reason, aren’t competently reviewed by Politico editors.
Actual human Politico journalists are understandably not pleased with any of this, especially because the nontransparent introduction of the new automation was in direct violation of the editorial union’s contract struck just last year. So unionized Politico employees spent much of this year battling with Politico via arbitration. And they just won a key battle in the fight, the first of its kind:
“The arbitrator ruled that Politico officially violated the collective bargaining agreement by failing to provide notice, human oversight, or an opportunity for the workers to bargain over the use of AI in the newsroom.
“If the goal is speed and the cost is accuracy and accountability,” the arbitrator wrote in his decision, “AI is the clear winner. If accuracy and accountability is the baseline, then AI, as used in these instances, cannot yet rival the hallmarks of human output, which are accuracy and reliability.” He also confirmed that the report-building product contained “erroneous and even absurd” AI-generated materials.
Politico leadership have made all sorts of crazy claims in the run up to this ruling, including Politico deputy editor-in-chief Joe Schatz claiming that AI can’t and shouldn’t be held to the same ethical standards as actual journalists, because it was technically created by programmers and not journalists.
In a statement, unionized Politico workers hope this sets a precedent at other news organizations:
“We are going to continue holding the line. This ruling is a great example of the important role unions play in ensuring workers have a say over working conditions–including the rollout of new technologies. I hope it emboldens our colleagues at other news shops across the country fighting AI deployments that similarly degrade ethical standards, and I hope it sends a message to managers at POLITICO and news executives everywhere that adopting new technology cannot come at the cost of accuracy and accountability.”
These aren’t “AI doomers.” They’re people who believe AI can be a useful tool, they just want it implemented competently and transparently, within the lines of existing union agreements.
There are, of course, caveats. Most U.S. journalists aren’t protected by a union, and we live in a country where labor regulators are being steadily lobotomized. And Politico itself, owned by yet another weird rich, Trump-friendly zealot, engages in a lot of false equivalency (“both sides,” “view from nowhere”) journalism with or without the help of undercooked automation.
By and large it’s pretty clear what the extraction class ownership of U.S. media want to build: a lazy, badly-automated, clickbait engagement ouroboros that shits out ad engagement and subscription money without the pesky need to pay so many annoying humans for things like health insurance. A system that basically just props up all of billionaire-ownerships’ laziest priors without interference by the plebs.
But, if nothing else, it’s refreshing to see some effective, organized resistance against the rushed implementation of under-cooked automation by the kind of rich assholes for whom informed consensus and the public interest are the very last thing on their minds.
Aquarter of a century ago, I wrote a book called “Rebel Code”. It was the first – and is still the only – detailed history of the origins and rise of free software and open source, based on interviews with the gifted and generous hackers who took part. Back then, it was clear that open source represented a powerful alternative to the traditional proprietary approach to software development and distribution. But few could have predicted how completely open source would come to dominate computing. Alongside its role in running every aspect of the Internet, and powering most mobile phones in the form of Android, it has been embraced by startups for its unbeatable combination of power, reliability and low cost. It’s also a natural fit for cloud computing because of its ability to scale. It is no coincidence that for the last ten years, pretty much 100% of the world’s top 500 supercomputers have all run an operating system based on the open source Linux.
More recently, many leading AI systems have been released as open source. That raises the important question of what exactly “open source” means in the context of generative AI software, which involves much more than just code. The Open Source Initiative, which drew up the original definition of open source, has extended this work with its Open Source AI Definition. It is noteworthy that the EU has explicitly recognized the special role of open source in the field of AI. In the EU’s recent Artificial Intelligence Act, open source AI systems are exempt from the potentially onerous obligation to draw up a range of documentation that is generally required.
That could provide a major incentive for AI developers in the EU to take the open source route. European academic researchers working in this area are probably already doing that, not least for reasons of cost. Paul Keller points out in a blog post that another piece of EU legislation, the 2019 Copyright in the Digital Single Market Directive (CDSM), offers a further reason for research institutions to release their work as open source:
Article 3 of the CDSM Directive enables these institutions to text and data-mine all “works or other subject matter to which they have lawful access” for scientific research purposes. Text and data mining is understood to cover “any automated analytical technique aimed at analysing text and data in digital form in order to generate information, which includes but is not limited to patterns, trends and correlations,” which clearly covers the development of AI models (see here or, more recently, here).
Keller’s post goes through the details of how that feeds into AI research, but the end-result is the following:
as long as the model is made available in line with the public-interest research missions of the organisations undertaking the training (for example, by releasing the model, including its weights, under an open-source licence) and is not commercialised by these organisations, this also does not affect the status of the reproductions and extractions made during the training process.
This means that Article 3 does cover the full model-development pathway (from data acquisition to model publication under an open source license) that most non-commercial Public AI model developers pursue.
As that indicates, the use of open source licensing is critical to this application of Article 3 of EU copyright legislation for the purpose of AI research.
What’s noteworthy here is how two different pieces of EU legislation, passed some years apart, work together to create a special category of open source AI systems that avoid most of the legal problems of training AI systems on copyright materials, as well as the bureaucratic overhead imposed by the EU AI Act on commercial systems. Keller calls these “public AI”, which he defines as:
AI systems that are built by organizations acting in the public interest and that focus on creating public value rather than extracting as much value from the information commons as possible.
Public AI systems are important for at least two reasons. First, their mission is to serve the public interest, rather than focusing on profit maximization. That’s obviously crucial at time when today’s AI giants are intent on making as much money as possible, presumably in the hope that they can do so before the AI bubble bursts.
Secondly, public AI systems provide a way for the EU to compete with both US and Chinese AI companies – by not competing with them. It is naive to think that Europe can ever match levels of venture capital investment that big name US AI startups currently enjoy, or that the EU is prepared and able to support local industries for as long and as deeply as the Chinese government evidently plans to do for its home-grown AI firms. But public AI systems, which are fully open source, and which take advantage of the EU right of research institutions to carry out text and data mining, offer a uniquely European take on generative AI that might even make such systems acceptable to those who worry about how they are built, and how they are used.
Despite all the recent hype about “AI,” the technology still struggles with very basic things and remains prone to significant errors. Which makes it maybe not the best idea to rush the nascent technology into widespread adoption in industries prone to all sorts of deep-rooted problems already (like say, health insurance, or journalism).
Google has recently also been experimenting with letting AI generate news headlines for its Discover feature (the news page you reach by swiping right on Google Pixel phones), and the results are decidedly… mixed. The technology, once again, routinely misconstrues meaning when trying to sum up news events:
“I also saw Google try to claim that “AMD GPU tops Nvidia,” as if AMD had announced a new groundbreaking graphics card, when the actual Wccftech story is about how a single German retailer managed to sell more AMD units than Nvidia units within a single week’s span.”
Other times, it just produces gibberish:
“Then there are the headlines that simply don’t make sense out of context, something real human editors avoid like plague. What does “Schedule 1 farming backup” mean? How about “AI tag debate heats”?
Google has already redirected a ton of advertising revenue away from journalists who do actual work, and toward its own synopsis and search tech. Now it’s effectively rewriting the headlines editors and journalists (the good ones, anyway) spend a lot of time working on to try and be as accurate and inviting as possible. And they’re doing an embarrassingly shitty job of it.
Not that the media companies themselves have been doing much better. Most major American media companies are owned by people who see AI not as a way to improve journalism quality and make journalism more efficient, but as a path toward cutting corners and undermining labor.
Meanwhile, in the quest for massive engagement at impossible scale, tech giants like Meta and Google have simply stopped caring so much about quality and accuracy. The results are everywhere, from Google News’ declining quality, to substandard search results, to the slow decline of key, popular services, to platforms filled with absolute clickbait garbage. It’s not been great for informed consensus or factual reality.
A federal judge just ruled that computer-generated summaries of novels are “very likely infringing,” which would effectively outlaw many book reports. That seems like a problem.
This isn’t just about AI—it’s about fundamentally redefining what copyright protects. And once again, something that should be perfectly fine is being treated as an evil that must be punished, all because some new machine did it.
But, I guess elementary school kids can rejoice that they now have an excuse not to do a book report.
To be clear, I doubt publishers are going to head into elementary school classrooms to sue students, but you never know with the copyright maximalists.
Sag highlights how it could have a much more dangerous impact beyond getting kids out of their homework: making much of Wikipedia infringing.
A new ruling in Authors Guild v. OpenAI has major implications for copyright law, well beyond artificial intelligence. On October 27, 2025, Judge Sidney Stein of the Southern District of New York denied OpenAI’s motion to dismiss claims that ChatGPT outputs infringed the rights of authors such as George R.R. Martin and David Baldacci. The opinion suggests that short summaries of popular works of fiction are very likely infringing (unless fair use comes to the rescue).
This is a fundamental assault on the idea, expression, distinction as applied to works of fiction. It places thousands of Wikipedia entries in the copyright crosshairs and suggests that any kind of summary or analysis of a work of fiction is presumptively infringing.
Short summaries of copyright-covered works should not impact copyright in any way. Yes, as Sag points out, “fair use” can rescue in some cases, but the old saw remains that “fair use is just the right to hire a lawyer.” And when the process is the punishment, saying that fair use will save you in these cases is of little comfort. Getting a ruling on fair use will run you hundreds of thousands of dollars at least.
Copyright is supposed to stop the outright copying of the copyright-protected expression. A summary is not that. It should not implicate the copyright in any form, and it shouldn’t require fair use to come to the rescue.
Sag lays out the details of what happened in this case:
Judge Stein then went on to evaluate one of the more detailed chat-GPT generated summaries relating to A Game of Thrones, the 694 page novel by George R. R. Martin which eventually became the famous HBO series of the same name. Even though this was only a motion to dismiss, where the cards are stacked against the defendant, I was surprised by how easily the judge could conclude that:
“A more discerning observer could easily conclude that this detailed summary is substantially similar to Martin’s original work, including because the summary conveys the overall tone and feel of the original work by parroting the plot, characters, and themes of the original.”
The judge described the ChatGPT summaries as:
“most certainly attempts at abridgment or condensation of some of the central copyrightable elements of the original works such as setting, plot, and characters”
He saw them as:
“conceptually similar to—although admittedly less detailed than—the plot summaries in Twin Peaks and in Penguin Random House LLC v. Colting, where the district court found that works that summarized in detail the plot, characters, and themes of original works were substantially similar to the original works.” (emphasis added).
To say that the less than 580-word GPT summary of A Game of Thrones is “less detailed” than the 128-page Welcome to Twin Peaks Guide in the Twin Peaks case, or the various children’s books based on famous works of literature in the Colting case, is a bit of an understatement.
Yikes. I’m sorry, but if you think that a 580-word computer-generated summary of a massive book is infringing, then we’ve lost the plot when it comes to copyright law. If it were, then copyright itself would need to be radically changed to allow for basic forms of human speech. If I see a movie and tell my friend what it was about, that shouldn’t implicate copyright law, even if it summarizes “the plot, characters, and themes of the original work.”
Sag then ties this to what you can find for countless creative works on Wikipedia:
To see why the latest OpenAI ruling is so surprising, it helps to compare the ChatGPT summary of A Game of Thrones to the equivalentWikipedia plot summary. I read them both so you don’t have to.
The ChatGPT summary of a Game of Thrones is about 580 words long and captures the essential narrative arc of the novel. It covers all three major storylines: the political intrigue in King’s Landing culminating in Ned Stark’s execution (spoiler alert), Jon Snow’s journey with the Night’s Watch at the Wall, and Daenerys Targaryen’s transformation from fearful bride (more on this shortly) to dragon mother across the Narrow Sea. In this regard, it is very much like the 800 word Wikipedia plot summary. Each summary presents the central conflict between the Starks and Lannisters, the revelation of Cersei and Jaime’s incestuous relationship, and the key plot points that set the larger series in motion.
And, look, if you want to see the chilling effects on speech created by over expansive copyright law, well:
I could say more about their similarities, but I’m concerned that if I explored the summaries in any greater detail, the Authors Guild might think that I am also infringing George R. R. Martin’s copyright, so I’ll move on to the minor differences.
You can argue that Sag, an expert on copyright law, is kind of making a joke here, but it’s no actual joke. Just the fact that someone even needs to consider this shows how bonkers and problematic this ruling is.
As Sag makes clear, there are few people out there who would legitimately think that the Wikipedia summary should be deemed infringing, which is why this ruling is notable. It again highlights how lots of people, including the media, lawmakers, and now (apparently) judges, get so distracted by the “but this new machine is bad!” in looking at LLM technology that they seem to completely lose the plot.
And that’s dangerous for the future of speech in general. We shouldn’t be tossing out fundamental key concepts in speech (“you can summarize a work of art without fear”) just because some new kind of summarization tool exists.
