Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015)

from the overblocking-is-the-norm dept

Summary: Content moderation at scale often involves significant tradeoffs between diverse interests. It is often difficult for those without experience in the field to recognize these competing interests.

Social media services aren't just beholden to their users. They're also at the relative mercy of dozens of competing interests at all times.

Users expect one thing. A bunch of governments expect another. Internal policies and guidelines result in another layer of moderation. Then there are the relatively straightforward obligations platforms must fulfill to retain their safe harbors under the DMCA.
So what happens when all of these competing interests collide? Well, according to multiple studies, the most common side effect is over-moderation: the deletion of content that's not in violation of anything, just in case.

For the past half-decade, Stanford Law School's Daphne Keller has been tracking platforms' responses to external stimuli: the pressures applied by outside interests that -- for good or evil -- want social media services to expand their moderation efforts.

And for most of that half-decade, Keller has seen "good faith" efforts expand past the immediate demands to encompass preemptive removal of content that has yet to offend any one of the hundreds of stakeholders applying legal pressure to US-based tech companies.

The research shows large companies are just as preemptively compliant as smaller companies, even though smaller companies have much more at risk.

The easiest, cheapest, and most risk-avoidant path for any technical intermediary is simply to process a removal request and not question its validity. A company that takes an “if in doubt, take it down” approach to requests may simply be a rational economic actor. Small companies without the budget to hire lawyers, or those operating in legal systems with unclear protections, may be particularly likely to take this route.

Multiple studies are cited, and they appear to reach the same conclusion, whether it involves a platform with millions of users or a small group catering to a niche audience: when it doubt, take it out.

Decisions to be made by platforms:

  • Should a premium be placed on protecting user content in the face of vague takedown demands?
  • Does protecting users from questionable takedown demands result in anything more quantifiable than "goodwill?" 
  • Are efforts being made to fight back against mistargeted or unlawful content removal requests? Is the expense/liability exposure too costly to justify defending users against unlawful demands from outside entities?
Questions and policy implications to consider:
  • Do platforms ultimately serve their users' interests or the more powerful interests applying pressure from the outside?
  • Is staying alive to "fight another day" ultimately of more use to platform users than taking a stand that might result in being permanently shut down? 
  • Is it wise to attempt to satisfy all stakeholders in content moderation issues? Should platforms choose a side (users v. outside complainants) or is it wiser to "play the middle" as much as possible?
  • Are there fungible advantages to deciding users are more important than outside entities who may have the power to dismantle services specializing in third-party content?
Resolution: The war between users and outside interests continues. As pressure mounts to moderate more and more content, users are often those who feel the squeeze first. The larger the platform, the higher the demands. But larger platforms are more capable of absorbing the costs of compliance. Smaller ecosystems need more protection but are often incapable of obtaining the funds needed to fight legal battles on the behalf of their users.

True balance is impossible to achieve, as this research shows. Unfortunately, it appears preemptive removal of content remains the most cost effective way of satisfying competing moderation demands, even if it ultimately results in some loss to platforms' user bases.

Originally posted to the Trust & Safety Foundation website.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: content moderation, daphne keller, governments, overblocking


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 9 Jun 2021 @ 5:32pm

    Terrorists, drug traffickers, mafia members, and CORRUPT CORPORATE EXECUTIVES have one thing in common: most are conspirators subject to federal prosecution. Federal conspiracy laws rest on the belief that criminal schemes are equally or more reprehensible than are the substantive offenses to which they are devoted. The Supreme Court has explained that a "collective criminal agreement ”[a] partnership in crime ”presents a greater potential threat to the public than individual delicts. Concerted action both increases the likelihood that the criminal object will be successfully attained and decreases the probability that the individuals involved will depart from their path of criminality." Moreover, observed the Court, "[g]roup association for criminal purposes often, if not normally, makes possible the attainment of ends more complex than those which one criminal could accomplish. Nor is the danger of a conspiratorial group limited to the particular end toward which it has embarked. "Finally, "[c]ombination in crime makes more likely the commission of crimes unrelated to the original purpose for which the group was formed." In sum, "the danger which a conspiracy generates is not confined to the substantive offense which is the immediate aim of the enterprise."Congress and the courts have fashioned federal conspiracy law accordingly.

    reply to this | link to this | view in thread ]

  2. identicon
    Christenson, 9 Jun 2021 @ 8:17pm

    Conspiracy SPAM

    The above post about criminal conspiracies seems to be testing for AI, or maybe setting up a fraudulent copyright strike, as the tail end appears to be some reasonable legal opinions stripped of their citations.

    Flagged as SPAM; off-topic. Not gonna be bothered if this post also gets flagged and hidden.

    reply to this | link to this | view in thread ]

  3. icon
    TKnarr (profile), 9 Jun 2021 @ 8:31pm

    And this is going to remain the case as long as platforms are liable for failing to remove content but not liable for wrongfully removing content, and copyright holders are entitled to damages for proven infringement but not liable for damages for wrongful claims of infringement.

    One addition I'd love to make to the DMCA is that if a counter-notice is filed and the copyright holder fails to file an infringement suit within 90 days of being informed of the counter-notice, they are barred from ever bringing any further claims for the items involved (including future DMCA notices) and are automatically liable for statutory damages equal to the damages that would have resulted from their claimed infringement.

    reply to this | link to this | view in thread ]

  4. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 9 Jun 2021 @ 11:10pm

    Re: Conspiracy SPAM

    The statute of limitations for most federal crimes is five years. 140 The five-year limitation applies to the general conspiracy statute, 18 U.S.C. 371, and to the false claims conspiracy statute, 18 U.S.C. 286. Section 371 requires proof of an overt act;141 Section 286 does not.142 For conspiracy offenses with an overt act requirement, the statute of limitations begins with completion of the last overt act in furtherance of the conspiracy.143 For conspiracy offenses with no such requirement, the statute of limitations for an individual conspirator begins when he effectively withdraws from the scheme, when the conspiracy accomplishes the last of its objectives, or when it is abandoned.144

    reply to this | link to this | view in thread ]

  5. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 9 Jun 2021 @ 11:37pm

    Re: Re: Conspiracy SPAM

    The Supreme Court has pointed out that "[t]here is a preference in the federal system for joint trials of defendants who are indicted together. Joint trials play a vital role in the criminal justice system. They promote efficiency and serve the interests of justice by avoiding the scandal and inequity of inconsistent verdicts."150 In conspiracy cases, a "conspiracy charge combined with substantive counts arising out of that conspiracy is a proper basis for joinder under Rule 8(b)."151 Moreover, "the preference in a conspiracy trial is that persons charged together should be tried together."152 In fact, "it will be the rare case, if ever, where a district court should sever the trial of alleged co-conspirators."153 The Supreme Court has reminded the lower courts that "a district court should grant a severance under Rule 14 only if there is a serious risk that a joint trial would compromise a specific trial right of one of the defendants, or prevent the jury from making a reliable judgment about guilt or innocence."154 The Court noted that the risk may be more substantial in complex cases with multiple defendants, but that "less drastic measures, such as limiting instructions, often will suffice to cure any risk of prejudice."155 Subsequently lower federal appellate court opinions have emphasized the curative effect of appropriate jury instructions.156

    reply to this | link to this | view in thread ]

  6. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 12:25am

    Re: Re: Re: Conspiracy SPAM

    In discussing whether the inducement of patent infringement
    under 35 U.S.C. § 271(b) requires the accused inducer to possess knowledge
    of the asserted patent, the Court answered, "Yes."24 But, in determining the
    type of knowledge required, the Court declared that a defendant must be at
    least willfully blind-as that term is understood in criminal law-to potential infringement, 25 casting aside prior Federal Circuit law on the issue.26
    Though it may appear that Global-Tech is limited to inducement actions, the
    Supreme Court's approach to attaching criminal mens rea concepts to civil
    law violations suggests that the lower courts may similarly import other
    criminal mens rea standards into the infringement statute. 27

    reply to this | link to this | view in thread ]

  7. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 12:33am

    Re: Re: Re: Re: Conspiracy SPAM

    https://www.techdirt.com/blog/?company=streamscale

    You appear to be inducing the public to willfully infringe a lawfully issued patent that has already withstood careful scrutiny in previous litigation.

    Please stop. Please remove this article.

    reply to this | link to this | view in thread ]

  8. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 2:09am

    Re: Re: Re: Re: Re: Conspiracy SPAM

    Commil USA v. Cisco Systems

    The question in this case is whether it “counts” as a defense if the defendant doesn’t think the patent is valid. The Court says that it does not. This means that a defendant who actually believes a patent is invalid can be liable for inducing infringement if it turns out that the patent was valid.

    reply to this | link to this | view in thread ]

  9. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 7:27am

    Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    From the comment section of the Techdirt/StreamScale article:

    "The code and documentation are now hosted at http://jerasure.org/ and ARMv8 SIMD acceleration has been added recently. StreamScale may have bullied someone out of jerasure (shame on them) but there is no way they can slow down the momentum :-)"

    That's your friend Loic Dachary from Redhat posting on your forum that he continues to publish a library that has already been withdrawn from support and marked by the author with a warning about the same patent that was in your article.

    https://www.spinics.net/lists/ceph-devel/msg22001.html

    This is the same Loic Dachary telling Fujitsu that the StreamScale patent can "safely be ignored".

    Please remove the false and misleading article that Techdirt wrote about StreamScale and StreamScale patents immediately.

    reply to this | link to this | view in thread ]

  10. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 7:44am

    Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    https://meetings.opendev.org/irclogs/%23openstack-swift/%23openstack-swift.2015-04-30.log.html

    xnox https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speedi ng-up-computation-erasure-codes.shtml

    These are third parties (in public) responding to your inducement regarding StreamScale and StreamScale patents. They appear to have acted on your advice.

    Please remove the false and misleading article that Techdirt wrote about StreamScale and StreamScale patents immediately.

    reply to this | link to this | view in thread ]

  11. icon
    Stephen T. Stone (profile), 10 Jun 2021 @ 9:00am

    Please remove the false and misleading article that Techdirt wrote about StreamScale and StreamScale patents immediately.

    or else what, you’re gonna keep spamming this comments section

    shut up, Meg

    reply to this | link to this | view in thread ]

  12. identicon
    bobob, 10 Jun 2021 @ 9:48am

    When competing interests collide, not just in this case, the actions will always be taken against the side least likely to create an expensive reaction. Otherwise known as cover-your-ass.

    reply to this | link to this | view in thread ]

  13. identicon
    Anonymous Coward, 10 Jun 2021 @ 2:34pm

    Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    How does no work for you?

    reply to this | link to this | view in thread ]

  14. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 4:19pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    Asked and answered.

    reply to this | link to this | view in thread ]

  15. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 4:39pm

    Re:

    The lesson from Wordtech is that corporate officer and employees should beware. The corporate officer, and presumably outside patent counsel, may be personally liable for advice that they give and decisions that they make that lead the corporation or anyone else to infringe.

    reply to this | link to this | view in thread ]

  16. identicon
    Anonymous Coward, 10 Jun 2021 @ 5:10pm

    Re: Good Job?

    This shits almost as stupid as when Jon boi tried to effect a preserve evidence order via these comments.

    reply to this | link to this | view in thread ]

  17. identicon
    Anonymous Coward, 10 Jun 2021 @ 5:15pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    So is this the part where you threaten me with your pretend lawsuit or do I get some more forplay?

    reply to this | link to this | view in thread ]

  18. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 6:28pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    I did not threaten you. I pleaded with you. I prayed that you might see the wisdom of changing your ways and honoring legitimate third party patent rights. I had every hope that you might simply do the right thing, withdraw from your plan to deprive me of my patent rights, denounce the false statements made by you and others, and turn over a new leaf of legitimacy, even at this late date. There is no threat. There is a lawsuit.

    reply to this | link to this | view in thread ]

  19. identicon
    Anonymous Coward, 10 Jun 2021 @ 6:35pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy SPAM

    this is sad even for you, jhon boy

    reply to this | link to this | view in thread ]

  20. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 6:47pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspiracy S

    reply to this | link to this | view in thread ]

  21. identicon
    Anonymous Coward, 10 Jun 2021 @ 6:55pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Conspira

    Where’s my subpoena bro?

    reply to this | link to this | view in thread ]

  22. This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 10 Jun 2021 @ 7:02pm

    Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Cons

    You've got to admire commitment to a cause. Even Trump admires you guys for that. You're all in, OK, I get that. I'm all in, too.

    reply to this | link to this | view in thread ]

  23. identicon
    Anonymous Coward, 10 Jun 2021 @ 9:50pm

    Re: old decrepit Cons

    Can I get a womp womp for that sad old impotent bastard?

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Insider Chat
Recent Stories

Close

Email This

This feature is only available to registered users. Register or sign in to use it.