Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015)

from the overblocking-is-the-norm dept

Summary: Content moderation at scale often involves significant tradeoffs between diverse interests. It is often difficult for those without experience in the field to recognize these competing interests.

Social media services aren’t just beholden to their users. They’re also at the relative mercy of dozens of competing interests at all times.

Users expect one thing. A bunch of governments expect another. Internal policies and guidelines result in another layer of moderation. Then there are the relatively straightforward obligations platforms must fulfill to retain their safe harbors under the DMCA.
So what happens when all of these competing interests collide? Well, according to multiple studies, the most common side effect is over-moderation: the deletion of content that’s not in violation of anything, just in case.

For the past half-decade, Stanford Law School’s Daphne Keller has been tracking platforms’ responses to external stimuli: the pressures applied by outside interests that — for good or evil — want social media services to expand their moderation efforts.

And for most of that half-decade, Keller has seen “good faith” efforts expand past the immediate demands to encompass preemptive removal of content that has yet to offend any one of the hundreds of stakeholders applying legal pressure to US-based tech companies.

The research shows large companies are just as preemptively compliant as smaller companies, even though smaller companies have much more at risk.

The easiest, cheapest, and most risk-avoidant path for any technical intermediary is simply to process a removal request and not question its validity. A company that takes an ?if in doubt, take it down? approach to requests may simply be a rational economic actor. Small companies without the budget to hire lawyers, or those operating in legal systems with unclear protections, may be particularly likely to take this route.

Multiple studies are cited, and they appear to reach the same conclusion, whether it involves a platform with millions of users or a small group catering to a niche audience: when it doubt, take it out.

Decisions to be made by platforms:

  • Should a premium be placed on protecting user content in the face of vague takedown demands?
  • Does protecting users from questionable takedown demands result in anything more quantifiable than “goodwill?” 
  • Are efforts being made to fight back against mistargeted or unlawful content removal requests? Is the expense/liability exposure too costly to justify defending users against unlawful demands from outside entities?

Questions and policy implications to consider:

  • Do platforms ultimately serve their users’ interests or the more powerful interests applying pressure from the outside?
  • Is staying alive to “fight another day” ultimately of more use to platform users than taking a stand that might result in being permanently shut down? 
  • Is it wise to attempt to satisfy all stakeholders in content moderation issues? Should platforms choose a side (users v. outside complainants) or is it wiser to “play the middle” as much as possible?
  • Are there fungible advantages to deciding users are more important than outside entities who may have the power to dismantle services specializing in third-party content?

Resolution: The war between users and outside interests continues. As pressure mounts to moderate more and more content, users are often those who feel the squeeze first. The larger the platform, the higher the demands. But larger platforms are more capable of absorbing the costs of compliance. Smaller ecosystems need more protection but are often incapable of obtaining the funds needed to fight legal battles on the behalf of their users.

True balance is impossible to achieve, as this research shows. Unfortunately, it appears preemptive removal of content remains the most cost effective way of satisfying competing moderation demands, even if it ultimately results in some loss to platforms’ user bases.

Originally posted to the Trust & Safety Foundation website.

Filed Under: , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015)”

Subscribe: RSS Leave a comment

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Terrorists, drug traffickers, mafia members, and CORRUPT CORPORATE EXECUTIVES have one thing in common: most are conspirators subject to federal prosecution. Federal conspiracy laws rest on the belief that criminal schemes are equally or more reprehensible than are the substantive offenses to which they are devoted. The Supreme Court has explained that a "collective criminal agreement ”[a] partnership in crime ”presents a greater potential threat to the public than individual delicts. Concerted action both increases the likelihood that the criminal object will be successfully attained and decreases the probability that the individuals involved will depart from their path of criminality." Moreover, observed the Court, "[g]roup association for criminal purposes often, if not normally, makes possible the attainment of ends more complex than those which one criminal could accomplish. Nor is the danger of a conspiratorial group limited to the particular end toward which it has embarked. "Finally, "[c]ombination in crime makes more likely the commission of crimes unrelated to the original purpose for which the group was formed." In sum, "the danger which a conspiracy generates is not confined to the substantive offense which is the immediate aim of the enterprise."Congress and the courts have fashioned federal conspiracy law accordingly.

Christenson says:

Conspiracy SPAM

The above post about criminal conspiracies seems to be testing for AI, or maybe setting up a fraudulent copyright strike, as the tail end appears to be some reasonable legal opinions stripped of their citations.

Flagged as SPAM; off-topic. Not gonna be bothered if this post also gets flagged and hidden.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Conspiracy SPAM

The statute of limitations for most federal crimes is five years. 140 The five-year limitation applies to the general conspiracy statute, 18 U.S.C. 371, and to the false claims conspiracy statute, 18 U.S.C. 286. Section 371 requires proof of an overt act;141 Section 286 does not.142 For conspiracy offenses with an overt act requirement, the statute of limitations begins with completion of the last overt act in furtherance of the conspiracy.143 For conspiracy offenses with no such requirement, the statute of limitations for an individual conspirator begins when he effectively withdraws from the scheme, when the conspiracy accomplishes the last of its objectives, or when it is abandoned.144

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Conspiracy SPAM

The Supreme Court has pointed out that "[t]here is a preference in the federal system for joint trials of defendants who are indicted together. Joint trials play a vital role in the criminal justice system. They promote efficiency and serve the interests of justice by avoiding the scandal and inequity of inconsistent verdicts."150 In conspiracy cases, a "conspiracy charge combined with substantive counts arising out of that conspiracy is a proper basis for joinder under Rule 8(b)."151 Moreover, "the preference in a conspiracy trial is that persons charged together should be tried together."152 In fact, "it will be the rare case, if ever, where a district court should sever the trial of alleged co-conspirators."153 The Supreme Court has reminded the lower courts that "a district court should grant a severance under Rule 14 only if there is a serious risk that a joint trial would compromise a specific trial right of one of the defendants, or prevent the jury from making a reliable judgment about guilt or innocence."154 The Court noted that the risk may be more substantial in complex cases with multiple defendants, but that "less drastic measures, such as limiting instructions, often will suffice to cure any risk of prejudice."155 Subsequently lower federal appellate court opinions have emphasized the curative effect of appropriate jury instructions.156

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re: Conspiracy SPAM

In discussing whether the inducement of patent infringement
under 35 U.S.C. § 271(b) requires the accused inducer to possess knowledge
of the asserted patent, the Court answered, "Yes."24 But, in determining the
type of knowledge required, the Court declared that a defendant must be at
least willfully blind-as that term is understood in criminal law-to potential infringement, 25 casting aside prior Federal Circuit law on the issue.26
Though it may appear that Global-Tech is limited to inducement actions, the
Supreme Court’s approach to attaching criminal mens rea concepts to civil
law violations suggests that the lower courts may similarly import other
criminal mens rea standards into the infringement statute. 27

This comment has been flagged by the community. Click here to show it.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re:3 Conspiracy SPAM

Commil USA v. Cisco Systems

The question in this case is whether it “counts” as a defense if the defendant doesn’t think the patent is valid. The Court says that it does not. This means that a defendant who actually believes a patent is invalid can be liable for inducing infringement if it turns out that the patent was valid.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re:4 Conspiracy SPAM

From the comment section of the Techdirt/StreamScale article:

"The code and documentation are now hosted at and ARMv8 SIMD acceleration has been added recently. StreamScale may have bullied someone out of jerasure (shame on them) but there is no way they can slow down the momentum :-)"

That’s your friend Loic Dachary from Redhat posting on your forum that he continues to publish a library that has already been withdrawn from support and marked by the author with a warning about the same patent that was in your article.

This is the same Loic Dachary telling Fujitsu that the StreamScale patent can "safely be ignored".

Please remove the false and misleading article that Techdirt wrote about StreamScale and StreamScale patents immediately.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re:5 Conspiracy SPAM


These are third parties (in public) responding to your inducement regarding StreamScale and StreamScale patents. They appear to have acted on your advice.

Please remove the false and misleading article that Techdirt wrote about StreamScale and StreamScale patents immediately.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re:7 Re:

The lesson from Wordtech is that corporate officer and employees should beware. The corporate officer, and presumably outside patent counsel, may be personally liable for advice that they give and decisions that they make that lead the corporation or anyone else to infringe.

This comment has been flagged by the community. Click here to show it.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Re: Re: Re:9 Conspiracy SPAM

I did not threaten you. I pleaded with you. I prayed that you might see the wisdom of changing your ways and honoring legitimate third party patent rights. I had every hope that you might simply do the right thing, withdraw from your plan to deprive me of my patent rights, denounce the false statements made by you and others, and turn over a new leaf of legitimacy, even at this late date. There is no threat. There is a lawsuit.

This comment has been flagged by the community. Click here to show it.

This comment has been flagged by the community. Click here to show it.

TKnarr (profile) says:

And this is going to remain the case as long as platforms are liable for failing to remove content but not liable for wrongfully removing content, and copyright holders are entitled to damages for proven infringement but not liable for damages for wrongful claims of infringement.

One addition I’d love to make to the DMCA is that if a counter-notice is filed and the copyright holder fails to file an infringement suit within 90 days of being informed of the counter-notice, they are barred from ever bringing any further claims for the items involved (including future DMCA notices) and are automatically liable for statutory damages equal to the damages that would have resulted from their claimed infringement.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Older Stuff
15:43 Content Moderation Case Study: Facebook Struggles To Correctly Moderate The Word 'Hoe' (2021) (21)
15:32 Content Moderation Case Study: Linkedin Blocks Access To Journalist Profiles In China (2021) (1)
16:12 Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018) (11)
15:30 Content Moderation Case Study: Tumblr's Approach To Adult Content (2013) (5)
15:41 Content Moderation Case Study: Twitter's Self-Deleting Tweets Feature Creates New Moderation Problems (2)
15:47 Content Moderation Case Studies: Coca Cola Realizes Custom Bottle Labels Involve Moderation Issues (2021) (14)
15:28 Content Moderation Case Study: Bing Search Results Erases Images Of 'Tank Man' On Anniversary Of Tiananmen Square Crackdown (2021) (33)
15:32 Content Moderation Case Study: Twitter Removes 'Verified' Badge In Response To Policy Violations (2017) (8)
15:36 Content Moderation Case Study: Spam "Hacks" in Among Us (2020) (4)
15:37 Content Moderation Case Study: YouTube Deals With Disturbing Content Disguised As Videos For Kids (2017) (11)
15:48 Content Moderation Case Study: Twitter Temporarily Locks Account Of Indian Technology Minister For Copyright Violations (2021) (8)
15:45 Content Moderation Case Study: Spotify Comes Under Fire For Hosting Joe Rogan's Podcast (2020) (64)
15:48 Content Moderation Case Study: Twitter Experiences Problems Moderating Audio Tweets (2020) (6)
15:48 Content Moderation Case Study: Dealing With 'Cheap Fake' Modified Political Videos (2020) (9)
15:35 Content Moderation Case Study: Facebook Removes Image Of Two Men Kissing (2011) (13)
15:23 Content Moderation Case Study: Instagram Takes Down Instagram Account Of Book About Instagram (2020) (90)
15:49 Content Moderation Case Study: YouTube Relocates Video Accused Of Inflated Views (2014) (2)
15:34 Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015) (23)
16:03 Content Moderation Case Study: Roblox Tries To Deal With Adult Content On A Platform Used By Many Kids (2020) (0)
15:43 Content Moderation Case Study: Twitter Suspends Users Who Tweet The Word 'Memphis' (2021) (10)
15:35 Content Moderation Case Study: Time Warner Cable Doesn't Want Anyone To See Critical Parody (2013) (14)
15:38 Content Moderation Case Studies: Twitter Clarifies Hacked Material Policy After Hunter Biden Controversy (2020) (9)
15:42 Content Moderation Case Study: Kik Tries To Get Abuse Under Control (2017) (1)
15:31 Content Moderation Case Study: Newsletter Platform Substack Lets Users Make Most Of The Moderation Calls (2020) (8)
15:40 Content Moderation Case Study: Knitting Community Ravelry Bans All Talk Supporting President Trump (2019) (29)
15:50 Content Moderation Case Study: YouTube's New Policy On Nazi Content Results In Removal Of Historical And Education Videos (2019) (5)
15:36 Content Moderation Case Study: Google Removes Popular App That Removed Chinese Apps From Users' Phones (2020) (28)
15:42 Content Moderation Case Studies: How To Moderate World Leaders Justifying Violence (2020) (5)
15:47 Content Moderation Case Study: Apple Blocks WordPress Updates In Dispute Over Non-Existent In-app Purchase (2020) (18)
15:47 Content Moderation Case Study: Google Refuses To Honor Questionable Requests For Removal Of 'Defamatory' Content (2019) (25)
More arrow