Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Study: Amazon Alters Publishing Rules To Deter Kindle Unlimited Scammers (April 2016)

from the it's-always-the-scammers dept

Summary: In July 2014, Amazon announced its “Netflix, but for ebooks” service, Kindle Unlimited. Kindle Unlimited allowed readers access to hundreds of thousands of ebooks for a flat rate of $9.99/month.

Amazon paid authors from a subscriber fee pool. Authors were paid per page read by readers — a system that was meant to reward more popular writers with a larger share of the Kindle Unlimited payment pool.

This system was abused by scammers once it became clear Amazon wasn’t spying on Kindle Users to ensure books were actually being read — i.e., keeping track of time spent on pages of text by readers or total amount of time spent reading. Since Amazon had no way to verify if readers were actually reading the content, scammers deployed a variety of tricks to increase their unearned earnings.

Part of the scam relied on Amazon’s willingness to pay authors for partially-read books. If only 100 pages of a 500-page book were read, the author still got credit for the 100 pages read by an Unlimited user. Scammers inflated “pages read” counts by moving the table of contents to the end of the book or offering dozens of different languages in the same ebook, relying on readers skipping hundreds of pages into the ebook to access the most popular translation. Other scammers offered readers chances to win free products and gift cards via hyperlinks that brought readers to the end of the scammers’ ebooks — books that sometimes contained thousands of pages.

The other part of the scam equation was Amazon’s hands-off approach to self-publishing. Amazon has opened its platform and appears to do very little to police the content of ebooks, other than requiring authors to follow certain formatting rules. Amazon is neither a publisher nor an editor, which has created a market for algorithmically-generated content as well as a home for writers seeking a distribution outlet for their bigoted and hateful writing.

Once Amazon realized the payout system was being gamed, it altered the way Kindle Unlimited operated. It began removing scammers, notifying authors and customers that it was doing this in response to Unlimited readers’ complaints.

Some in the community have contacted us about the activities of a small minority of publishers who may attempt to inflate sales or pages read through the use of various techniques, such as adding unnecessary or confusing hyperlinks, misplacing the TOC [table of contents] or adding distracting content.

Unfortunately, Amazon’s moderation efforts did affect a very small number of legitimate authors. Writer Walter Jon Williams was blocked from selling his ebook because his table of contents was located near the end of his book. Williams pointed out he had done this to maximize the amount of content prospective readers/purchasers could access using Amazon’s “Look Inside” feature. After some back-and-forth, Williams’ book and buy button were restored by Amazon.

Amazon continues to work to minimize abuse of the Kindle Unlimited system. The most noticeable and major change has been to cap earnings at 3,000 pages per ebook per reader. This limits the amount of money scammers can pull from the Unlimited payout pool. It also limits the number of times Kindle Unlimited readers will find themselves scrolling through ebooks solely designed to inflate page counts.

Decisions to be made by Amazon:

  • Can automated moderation alone determine whether an uploaded ebook is a legitimate offering?
  • Does altering the payout rules for Kindle Unlimited negatively affect legitimate authors?
  • Does the ongoing abuse of various Amazon ebook programs justify more data collection on customers and their reading habits?
  • Should authors be notified ahead of changes to Amazon services or would more transparency result in more abuse by scammers?

Questions and policy implications to consider:

  • Does the flat rate subscriber fee cover the costs of policing an ebook publishing ecosystem of this size?
  • Who deserves more protection? Sellers/writers or customers? How do you strike the correct balance that provides more value to both sides of the transaction?
  • Is more vetting needed on the front end (ID verification, etc.) to prevent further abuse?

Resolution: Amazon reacted to abuse of its Unlimited system by clarifying rules for content placement and removing ebooks that violated the company’s publishing guidelines. It also changed the way the pool of Kindle Unlimited funds were paid out, limiting the amount of pool money scammers could remove from the system by artificially inflating “read pages” counts.

Filed Under: , , ,
Companies: amazon

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation Case Study: Amazon Alters Publishing Rules To Deter Kindle Unlimited Scammers (April 2016)”

Subscribe: RSS Leave a comment
renato (profile) says:

Is the amazon payment model really relevant to this case?
It seems that whatever method to allocate the money would be gamed by scammer (like Goodhart’s law) and maybe even shape how honest authors shape their published work to squeeze a bit more money.
For example, if the payment was from book read (whatever counts as read), both scammer and authors would offer more and shorter books to inflate the variable used to distribute the money.

At least, they tried to keep it simple at the beginning and then started dealing with bad behavior after it became rampant, instead of just trying to make complex rules and spy harder on users’ data.

TKnarr (profile) says:

Bayesian filtering

I’d hope Amazon was applying this already, but Bayesian filtering worked pretty well (still works pretty well, in fact) for separating spam from non-spam email. The Kindle store should be able to provide good-quality large samples of both actual books (pull from known authors and books which have been published on dead trees) and generated content, I’d honestly start by taking those samples and using them to initialize bogofilter, then feed it a selection of test books and see how accurate it’s classification was.

After classification, there’s some heuristics that can be applied. If an author account is long-standing and doesn’t have any scam-content flags, it’s probably safe to just list any new books regardless of what the filter says. If they’re uploading a lot of works over a short time-frame, check whether that author’s got hardcopy-published works. If they do (or they don’t and the filter says they’re mostly or all scam content) it’s probably safe to just go with the filter results, otherwise flag the lot for manual review because it’s anomalous behavior.

Anonymous Coward says:

The spammer/troll’s motive is relevant. In this case, it seems to be greed. Where it’s hatred or one of the other notoriously-deadly motives, some other approach to disencentivizationalizing the contemptible behavior might be appropriate.

That’s why responding to a troll is generally the worst thing you can do (volunteers for porcine mud-wrestling, anyone?) Or harassing a sociopath with a victim-complex (come see the violence inherent in the system!)

This may seem strange, but it’s almost as if you had to look on each one as a person ….

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Older Stuff
15:43 Content Moderation Case Study: Facebook Struggles To Correctly Moderate The Word 'Hoe' (2021) (21)
15:32 Content Moderation Case Study: Linkedin Blocks Access To Journalist Profiles In China (2021) (1)
16:12 Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018) (11)
15:30 Content Moderation Case Study: Tumblr's Approach To Adult Content (2013) (5)
15:41 Content Moderation Case Study: Twitter's Self-Deleting Tweets Feature Creates New Moderation Problems (2)
15:47 Content Moderation Case Studies: Coca Cola Realizes Custom Bottle Labels Involve Moderation Issues (2021) (14)
15:28 Content Moderation Case Study: Bing Search Results Erases Images Of 'Tank Man' On Anniversary Of Tiananmen Square Crackdown (2021) (33)
15:32 Content Moderation Case Study: Twitter Removes 'Verified' Badge In Response To Policy Violations (2017) (8)
15:36 Content Moderation Case Study: Spam "Hacks" in Among Us (2020) (4)
15:37 Content Moderation Case Study: YouTube Deals With Disturbing Content Disguised As Videos For Kids (2017) (11)
15:48 Content Moderation Case Study: Twitter Temporarily Locks Account Of Indian Technology Minister For Copyright Violations (2021) (8)
15:45 Content Moderation Case Study: Spotify Comes Under Fire For Hosting Joe Rogan's Podcast (2020) (64)
15:48 Content Moderation Case Study: Twitter Experiences Problems Moderating Audio Tweets (2020) (6)
15:48 Content Moderation Case Study: Dealing With 'Cheap Fake' Modified Political Videos (2020) (9)
15:35 Content Moderation Case Study: Facebook Removes Image Of Two Men Kissing (2011) (13)
15:23 Content Moderation Case Study: Instagram Takes Down Instagram Account Of Book About Instagram (2020) (90)
15:49 Content Moderation Case Study: YouTube Relocates Video Accused Of Inflated Views (2014) (2)
15:34 Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015) (23)
16:03 Content Moderation Case Study: Roblox Tries To Deal With Adult Content On A Platform Used By Many Kids (2020) (0)
15:43 Content Moderation Case Study: Twitter Suspends Users Who Tweet The Word 'Memphis' (2021) (10)
15:35 Content Moderation Case Study: Time Warner Cable Doesn't Want Anyone To See Critical Parody (2013) (14)
15:38 Content Moderation Case Studies: Twitter Clarifies Hacked Material Policy After Hunter Biden Controversy (2020) (9)
15:42 Content Moderation Case Study: Kik Tries To Get Abuse Under Control (2017) (1)
15:31 Content Moderation Case Study: Newsletter Platform Substack Lets Users Make Most Of The Moderation Calls (2020) (8)
15:40 Content Moderation Case Study: Knitting Community Ravelry Bans All Talk Supporting President Trump (2019) (29)
15:50 Content Moderation Case Study: YouTube's New Policy On Nazi Content Results In Removal Of Historical And Education Videos (2019) (5)
15:36 Content Moderation Case Study: Google Removes Popular App That Removed Chinese Apps From Users' Phones (2020) (28)
15:42 Content Moderation Case Studies: How To Moderate World Leaders Justifying Violence (2020) (5)
15:47 Content Moderation Case Study: Apple Blocks WordPress Updates In Dispute Over Non-Existent In-app Purchase (2020) (18)
15:47 Content Moderation Case Study: Google Refuses To Honor Questionable Requests For Removal Of 'Defamatory' Content (2019) (25)
More arrow