AI Fabricated Quotes In A Book About AI Undermining Truth. The Author Says This Proves His Point.
from the it-undermines-something-alright dept
Last week I read this excerpt of Steven Rosenbaum’s new book in Wired. His book is titled “The Future of Truth” and the Wired article has the attention grabbing headline: “Gen Z Is Pioneering a New Understanding of Truth.” I debated writing about the article, because it read to me like pretty typical “older generation whines about the kids these days and how their newfangled tech is melting their brains.”
We’ve seen this moral panic before.
And nothing has yet convinced me that “the kids these days” are any worse off than any previous generation. Yes, the technology is new, but like every generation, they tend to actually figure out the pros and cons of new technology way before their parents do.
To be fair, the Wired excerpt presents Rosenbaum’s argument as somewhat more nuanced than a standard “kids are dumb now” panic — he’s describing how Gen Z has developed different epistemic habits, relying on emotional cues and communal verification through social networks rather than traditional institutional gatekeepers. Some of that uncovers a real phenomenon that might be worth discussing. And yet, something about the framing still felt off to me — as if “different” was being dressed up as “broken.”
Also, if you tend to read a lot of AI-generated content, the Wired excerpt has a large number of tells. Plenty of “that’s not x, it’s y” and a bunch of words that AI loves to use (to be clear: that’s not definitive, as a reason why AI tools use those types of words and phrases is because they’re so common in human written communications, and it frustrates me to no end that I now feel the need to consciously limit my own use of certain rhetorical devices I actually liked to use in my past writing).
Speaking of which… a week later, the NY Times reports that Rosenbaum’s book appears to be stuffed with quotes that were made up entirely by whatever AI tool he used to write it. Oops!
The author of a nonfiction book about the effects of artificial intelligence on truth acknowledged on Monday that he had included numerous made-up or misattributed quotes concocted by A.I.
The author, Steven Rosenbaum, whose book “The Future of Truth” was released this month to great fanfare, incorporated more than a half-dozen misattributed or fake quotes in sections of the book reviewed by The New York Times.
Yes, but the truly astounding bit is buried all the way at the end of the the NY Times article, in which Rosenbaum seeks to judo this total editorial failure into evidence supporting the premise of his book. I only wish I were kidding:
In his statement, Mr. Rosenbaum said that if the episode “serves as a warning about the risks of A.I.-assisted research and verification, that is why I wrote the book.”
“These A.I. errors do not, in fact, diminish the larger questions that the book raises about truth, trust and A.I. and its impact on society, democracy and editorial,” he added.
Dude. No. Just… no.
You don’t get to write an entire book fretting about how the kids these days don’t understand truth because of AI… and then when its exposed that you didn’t even check the quotes AI gave you, claim that this just proves your point.
That’s not how any of this works.
This is a book called “The Future of Truth.” It seems like you should at least grapple with the fact that part of the “future of truth” is that your own book is spreading false information because you didn’t… actually write parts of it.
If anything, it seems likely that kids are learning whatever lesson there is to be learned here way better than the adults. The widespread disdain many kids have for AI is, in part, a direct response to all the bullshit ways adults are using it. I will continue to argue that LLMs are a tool that, when properly used, can be quite empowering. But the absolute worst way to use these tools is to let them do your primary work for you. They can help assist you, but anyone who is relying on them as a lazy way of doing the deeper work you need to do will run into problems.
But I don’t think any of that has to do with how “the kids these days” “understand truth.” A lot of it has to do with how adults are rushing around looking for shortcuts and schemes to get away from doing the actual work. But apparently there’s no book deal or Wired feature story in “the kids these days are probably figuring it out just fine.”
Filed Under: ai, generation z, moral panic, steven rosenbaum, truth
Comments on “AI Fabricated Quotes In A Book About AI Undermining Truth. The Author Says This Proves His Point.”
It’s also due to the many ways adults are shoving AI into everything and the way AI evangelists keep hyping up AI as the effective end of the job market in multiple industries (especially in the arts). When young people are being told “AI is going to take over everything” (with a dash of consent-manufacturing “it’ll happen whether you like it or not” bullshit that sounds like something you’d hear from a rapist), they’re hearing “your future is fucked”—and rightfully enough, they’re not happy about that.
Re:
Can you explain how your list of bullshit terrible ways AI can be used isn’t covered under the phrase “all the bullshit ways adults are using [AI]”.
Im unsure how you construct your problems to be over the top of the category of ‘all the bullshit’.
Re: Re:
I think you’re misreading it: “using” and “shoving” are two different things, and the people shoving this technology into other technology aren’t necessarily using it. Mike was referring only to the bullshit uses.
Calling it “A.I.” is also bullshit, of course, and it’s an elementary debating mistake to let one’s opponent define the terminology. In this case, using the term creates some implicit support for the opposing view: if it’s “intelligent”, of course it’s gonna be able to do all that stuff; and, speaking of “manufacturing consent”, that makes it sound like a done deal (there are ethical concerns with destroying intelligent life after creating it).
Re: Re: Re:
It is the commonly accepted verbiage in discussions like these. Until that changes, I’ll use “AI” as a shorthand for LLMs and chatbots and other such tech.
Re: Re: Re:2
Sure, and you can also talk about the “illegals” coming across the borders if you want. Nobody’s stopping you. Language has the power to subtly influence people’s views, and it’s every individual’s choice whether to fight that power or to go along with it.
Re: Re: Re:3
I could, but I don’t—not using that specific terminology, anyway.
The thing is, I do use “LLM” and “chatbot” and other such terminology already. But I also use “AI” because it is the commonly accepted terminology regardless of whether you or anyone else likes that fact. I’ll use it that way when- and wherever I see fit, your concerns notwithstanding.
Re: Re: Re:3
You’re a linguistic prescriptivist, we get it.
Meanwhile everyone else here are linguistic descriptivists, so you can just deal with it.
Gee, we just don’t get enough AI pundiitry and hand wringing these days ….
Humans mastered the art of deceiving others with phony words and images … thousands of years before AI
The kids are not alright... but they're not terrible either
From everything I have heard about kids these days… while I wouldn’t say they’re alright, I would say they make me proud and are finding ways to navigate the shit they are inheriting.
Moving to RedNote when TikTok was sold was the first big thing that made me proud. I hope they are making a lot of good Chinese friends.
Speedrunning Scientology is admittedly concerning, but even in this there is something to be proud of, as they are actively organizing and coordinating a large scale project.
Plus, if I am being totally honest, “relying on emotional cues and communal verification through social networks rather than traditional institutional gatekeepers” is probably not a bad response to the traditional gatekeepers being caught up in obviously untrustworthy behaviors. It’s not perfect, but it’s certainly better than giving up!
It almost begs the question, did he do it as a publicity stunt!
Re:
Even if he did, he ruined his credibility as an author to do it.
Re: Re:
The commercial success of being confidently wrong, should not be understated.
Destroying your credibility with the mainstream, quite often seems to just enhance it for the cringe fringe.
Re: I can promise you not
No one spends 4 years writing and publishing a book to make it any way less trustworthy or relevant. It’s a massive bummer, and i’ve lost lots of sleep asking ‘what could I have done differently.’ The sad truth is that AI made the book in a myriad of ways much better, and in a handful of ways much worse. I’m hardly the only writer facing that daily AI battle.
Re: Re:
If you’re making a list, “not using AI” should be the first ten spots on it.
Re: Re:
As a published author struggling as the vast majority of us are to just get eyes and reviews on our work, much less sales, the only AI battle is the very black and white need to resist cheating ourselves and our audience by trying to take a shortcut to real meaning.
Look at Stephen Glass. Before LLMs, you could just make shit up yourself and hope not to get caught. This isn’t a new issue, just a different mechanism for shortcuts and ethical compromises. And if you’re serious about having something to convey, you wouldn’t want it to be anything other than your own input.
book
Mike,
How about this – why don’t I send you the book. You can read it, and then we can have a conversation about it, you’re criticisms of it, and where I got it wrong. The book has 380 citations, and 4 mistakes – so clearly it was proof read (4x times) and if yes, I got fooled by AI, so shame on me – but that doesn’t undermine the argument- which isnt about GenZ. The Wired article was one chapter excerpt. What’s the best way to get you a book pdf or hard copy – let me know.
Re:
If you had even one false quote due to AI, it’d still be bad. Relying on an AI chatbot or Google’s AI summaries or whatever to find quotes is a bad idea because you don’t know if it’s giving you the real deal. Your failures may not undermine your argument in a vacuum, but they do undermine your credibility.
Re:
You are suspiciously em dashy today, sir.
Re:
I understand the inclination to try to dig yourself out after all the hope and effort you’ve invested, but there’s no way out of this and digging will just look desperate and flailing and like you haven’t learned anything from the episode. Start writing your next book (without the help of an LLM) and include an anecdote about how you got this wake up call and come to Jeebus moment regarding the use of an LLM to even “help” writing a book. Anything else will just come off as a defensive attempt to deflect.
You’re drunk driver who got caught but is telling everyone you just had a few drinks and you got home safe and you didn’t kill anyone, so it’s okay. Nothing other than “I fucked up and I’m looking into changing my behavior” is going to redeem this.
That you think the rest of the book holds up, but also you’re going to find people don’t want to bother to see if the rest of the book holds up, only serves to say that this oversight betrayed any possibly valuable meaning in the rest of the book and you shouldn’t do that to the ideas you’re trying to convey to others.
Re:
If the book were about anything else, you might have a point.
The book is about AI and truth. The one thing that people need to internalize about AI, with respect to truth, is that AI does not know what is true and everything it spits out needs to be checked for accuracy.
The fact that there are any AI-generated false citations in your work tells me that you haven’t, actually, internalized this fact. Otherwise, you would have rigorously checked every citation your LLM gave you. And you didn’t, given the fact that, as you say, four of them made it into your final published work.
If you have been proven to not actually know the single most salient point about AI and truth, why should anyone want to listen to anything you have to say on the subject?
Good points all
But how about Learning about a Thing/subject First, Rather then Listen to the AI or other idiots, and the Idiom of knowing Both sides/All sides before opening mouth and inserting foot, Because Someone ELSE read the manual.
Is this really any different from what we did before? It’s not like any of the current institutional gatekeepers came into existence without a priori obtaining buy-in from people’s social networks.
Nor is it possible to trace any of them back to a blank slate, there was another institutional gatekeeper before each of these ones which lost the ability to gatekeep and was supplanted. That evidently didn’t happen because the gatekeeper agreed to become irrelevant, so it ipso facto must have happened because the community stopped using it despite that.
I think you’ve reversed cause and effect. It’s the communal verification and social networks which produce and sustain gatekeepers, not the gatekeepers who produce the community. The epistimic habits here are the same, the eisting gatekeepers have just outsmarted themselves in believing they do the gatekeeping rather than merely reflect what already exists in the community, and so are going the way of their predecessors.
Neh. AI doesn’t even work well enough to undermine truth. The AI doomers are just as deluded as the people who think AI is the best thing since sliced bread.
It’s a useless tool being shoved into things where it doesn’t work, and the sooner the bubble pops and takes the economy with it, the sooner I can get to work on buying up the dip.
The kids may be fine in the sense that they’ll find ways to cope with the bullshit and life goes on, but one could still argue that kids shouldn’t have to deal with all that nonsense in the first place. I am aware of the seemingly ever-present “[thing kids like] is very bad for them” moral panics, but do kids actually like dealing with AI-generated bullshit? I don’t think so, and neither do adults.
Some technologies really are harmful to society, kids included, and even beneficial technologies can have negative effects. The important thing is what to do about it, and what not to do about it. We probably agree on what not to do about most of these things, but I think you sometimes lean too far in the direction of reducing legitimate concerns to a “moral panic.”