EFF Highlights Stories Of Bad Content Moderation With New TOSsed Out Site

from the content-moderation-is-impossible dept

We've pointed out for many years that content moderation at scale isn't just hard, it's impossible to do well. At the scale of giant platforms, there needs to be some level of moderation or the platforms and users will get overwhelmed with spam or abuse. But at that scale, there will be a ton of mistakes -- both type I and type II errors (blocking content that shouldn't be blocked and failing to block content that probably should be blocked). Some -- frankly dishonest -- people have used a few examples of certain content moderation choices to falsely claim that there is "anti-conservative bias" in content moderation choices. We've pointed out time and time again why the evidence doesn't support this, though many people insist it's true (and I'll predict they'll say so again in the comments, but when asked for evidence, they will fail to present any).

That's not to say that the big platforms and their content moderation practices are done well. As we noted at the very beginning, that's an impossible request. And it's important to document the mistakes. First, it helps get those mistakes corrected. Second, while it will still be impossible for the platforms to moderate well, they can still get better and make fewer errors. Third, it can help people understand that errors are not because someone hates you or has animus towards a political group or political belief, but because they fuck up the moderation choices all the time. Fourth, it can actually help to find what actual patterns there are in these mistakes, rather than relying on moral panics. To that end, it's cool to see that the EFF has launched a new site, creatively dubbed TOSsed Out to help track stories of bad content moderation practices.

Just looking through the stories already there should show you that bad content moderation choices certainly aren't limited to "conservatives," but certainly do seem to end up impacting actually marginalized groups:

EFF is launching TOSsed Out with several examples of TOS enforcement gone wrong, and invites visitors to the site to submit more. In one example, a reverend couldn’t initially promote a Black Lives Matter-themed concert on Facebook, eventually discovering that using the words “Black Lives Matter” required additional review. Other examples include queer sex education videos being removed and automated filters on Tumblr flagging a law professor’s black and white drawings of design patents as adult content. Political speech is also impacted; one case highlights the removal of a parody account lampooning presidential candidate Beto O’Rourke.

“The current debates and complaints too often center on people with huge followings getting kicked off of social media because of their political ideologies. This threatens to miss the bigger problem. TOS enforcement by corporate gatekeepers far more often hits people without the resources and networks to fight back to regain their voice online,” said EFF Policy Analyst Katharine Trendacosta. “Platforms over-filter in response to pressure to weed out objectionable content, and a broad range of people at the margins are paying the price. With TOSsed Out, we seek to put pressure on those platforms to take a closer look at who is being actually hurt by their speech moderation rules, instead of just responding to the headline of the day.”

As the EFF notes, this is something of a reaction to the White House's intellectually dishonest campaign database building program to get people to report stories of social media bias against conservatives. Unlike the White House, which is focused on pulling some one-sided anecdotes it can misrepresent for political points, the TOSsed out effort is a real opportunity to track what kinds of mistakes happen in content moderation and how to deal with them.

Filed Under: content moderation, mistakes, tossed out, type 1 errors, type 2 errors
Companies: eff


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Stephen T. Stone (profile), 22 May 2019 @ 9:44am

    bad content moderation choices certainly aren't limited to "conservatives," but certainly do seem to end up impacting actually marginalized groups

    …which is always the way it goes. LGBT content is often one of the first targets of sex-related content moderation because of unfair cultural assumptions that such content is sexual in nature. A picture of a same sex couple kissing is far more likely to get dinged than a picture of an opposite sex couple kissing, even if both pictures are not even remotely sexual.

    reply to this | link to this | view in chronology ]

  • icon
    TheLizard (profile), 22 May 2019 @ 9:50am

    Fact-check: FALSE

    There is plenty of evidence, you simply have a list of excuses to make it look like incompetence instead of intent.

    Explain #LearnToCode in your next article and maybe it will be something less than unadulterated bullshit, which this one is.

    reply to this | link to this | view in chronology ]

    • icon
      TheLizard (profile), 22 May 2019 @ 9:55am

      Re: Fact-check: FALSE

      Let's clarify this a bit:

      It's not really "anti-conservative" bias, that's easy to explain away, especially when you can define "conservative" one way or another.

      It's really more "anti-establishment" bias. Bias against the narrative of the global corporate and media elite.

      That's especially clear on Twitter, when blue-checked journalists are given a pass on obviously TOS-violating behavior, but any far-left or far-right viewpoint, or even populist ones, will get banned outright when they have a popular message that goes against the advertiser- and mainstream-journalist-friendly narrative.

      reply to this | link to this | view in chronology ]

      • icon
        Mike Masnick (profile), 22 May 2019 @ 10:06am

        Re: Re: Fact-check: FALSE

        That's especially clear on Twitter, when blue-checked journalists are given a pass on obviously TOS-violating behavior, but any far-left or far-right viewpoint, or even populist ones, will get banned outright when they have a popular message that goes against the advertiser- and mainstream-journalist-friendly narrative.

        Two assertions are made here without any evidence. Can you back up either one and show that it is because of this bias, rather than -- as clearly stated in the article -- due to the vagaries and everyday errors that will come with content moderation?

        If not, well, you failed.

        reply to this | link to this | view in chronology ]

        • This comment has been flagged by the community. Click here to show it
          icon
          TheLizard (profile), 22 May 2019 @ 10:21am

          Re: Re: Re: Fact-check: FALSE

          LearnToCode

          reply to this | link to this | view in chronology ]

          • icon
            Mike Masnick (profile), 22 May 2019 @ 10:28am

            Re: Re: Re: Re: Fact-check: FALSE

            Right, so you failed. I asked for evidence, and you did not provide it.

            Congrats on proving my point.

            reply to this | link to this | view in chronology ]

            • icon
              TheLizard (profile), 22 May 2019 @ 10:34am

              Re: Re: Re: Re: Re: Fact-check: FALSE

              I DID, you can't be bothered to look into it.

              That's why all this banning WORKS. Because nobody SEES it, and people like you dismiss it because you can't be bother to do any research yourself.

              Here you go:

              https://www.oneangrygamer.net/2019/03/twitter-temporarily-banned-journalist-brad-glasgow-for-usi ng-learn-to-code-hashtag-during-appeal/80639/

              Next, you'll post an excuse about how that kind of thing isn't evidence.

              I'll post another example, and you'll do the same.

              After several, you'll go, "well it's TOS, they would do that to anybody," or "well they're going to make mistakes."

              You've made up your mind, social media reinforces your opinion, and on it goes until you don't really have any opinions of your own, they are given to you instead.

              reply to this | link to this | view in chronology ]

              • identicon
                Anonymous Coward, 22 May 2019 @ 10:45am

                Re: Re: Re: Re: Re: Re: Fact-check: FALSE

                Read the linked article. I don't conclude from that article alone that it's evidence of anti-conservative bias. I conclude that the hashtag was still on twitter's flag list, the tweet got flagged, and the account was either auto-banned with no actual review, or some dolt clicked the "ban" button without bothering to look at context.

                Could it be bias? Sure. But that one article alone doesn't convince me.

                Having previously worked for a hosting provider and seen content removals firsthand, I feel pretty comfortable stating there's a WHOLE lot more incompetence going around than bias.

                reply to this | link to this | view in chronology ]

                • icon
                  TheLizard (profile), 22 May 2019 @ 10:59am

                  Re: Re: Re: Re: Re: Re: Re: Fact-check: FALSE

                  You're ignoring the fact that they were that kind of ban-happy over the hashtag in the first place. It was to protect journalists (Twitter's protected class), who were happy telling laid-off factory workers and miners that THEY should learn to code, but couldn't take a joke when they were reminded of it after the online rags started mass lay-offs of journalists.

                  THAT is the bias.

                  reply to this | link to this | view in chronology ]

                  • icon
                    Stephen T. Stone (profile), 22 May 2019 @ 11:02am

                    That would prove a bias for journalists, not against conservatives.

                    reply to this | link to this | view in chronology ]

                    • icon
                      TheLizard (profile), 22 May 2019 @ 11:21am

                      Re:

                      Right, as I pointed out in my 2nd post, a lot of it just has to do with anti-establishment viewpoints.

                      But, if you're always protecting journalists this way, you will, almost by definition, have a bias against conservatives, as journalists as a group are generally very liberal.

                      Comprehensive meta-study on that:

                      https://www.mrc.org/special-reports/liberal-mediaevery-poll-shows-journalists-are-more-liberal -american-public-%E2%80%94-and

                      reply to this | link to this | view in chronology ]

                      • icon
                        Stephen T. Stone (profile), 22 May 2019 @ 11:32am

                        journalists as a group are generally very liberal

                        “Conservatives as a group are generally very homophobic.” I, too, can make sweeping generalizations about entire segments of the population based on mere anecdotal evidence (e.g., prominent conservative personalities and lawmakers voicing anti-queer beliefs and supporting efforts to marginalize queer people). You know what that makes me? An asshole.

                        reply to this | link to this | view in chronology ]

                      • identicon
                        Rocky, 22 May 2019 @ 11:39am

                        Re: Re:

                        Have you ever wondered why most journalists are liberal? Because being a conservative means you don't go out of your way to question the status quo which liberals and progressives do.

                        That means, as a conservative you have no right to complain that the media is liberal.

                        reply to this | link to this | view in chronology ]

              • identicon
                bob, 22 May 2019 @ 10:50am

                Re: Re: Re: Re: Re: Re: Fact-check: do it correctly

                At least this reply has something to look at for potential evidence. I only say potential because I haven't read your source material yet. So I can't definitively state that you have evidence.

                The problem before was you just said there's a problem with X thing. But you didn't give any details. Well there is a lot of problems with a lot of things so what's your point.

                That is not presenting evidence that's you wanting others to prove your point without you having to make an effort. You stated an idea with no backing. You must first show evidence and do some research, not someone else.

                However in your later reply you actually posted something that at least could be evidence. That is how it works.

                Now if what you presented is factually correct then you have a chance that someone will reply. If they just state their opinion refuting you then you can ignore the response because they lack evidence.

                But if they reply with their own valid evidence supporting their viewpoint. Now you can debate the merits of each viewpoint and weigh the evidence.

                After seeing the evidence hopefully the party that was wrong will acknowledge their error. Then instead of being a dick because their viewpoint was defeated each party can move on.

                reply to this | link to this | view in chronology ]

              • icon
                Stephen T. Stone (profile), 22 May 2019 @ 10:55am

                One anecdotal story is not evidence of a pattern of bias.

                reply to this | link to this | view in chronology ]

                • icon
                  TheLizard (profile), 22 May 2019 @ 11:16am

                  Re:

                  Right, that's true.

                  But there ARE a lot of anecdotes, that do draw a pattern. This was just one of the most obvious and egregious ones.

                  The Joe Rogan interview with Jack Dorsey, Vijaya Gadde, and Tim Pool is very instructive. Pool points out that their TOS itself is biased against conservatives, which is why one user was permanent-lifetime banned for posting "Men aren't women, though."

                  https://thefederalist.com/2018/11/25/twitter-permanently-bans-feminist-writing-men-are nt-women/

                  reply to this | link to this | view in chronology ]

                  • icon
                    Stephen T. Stone (profile), 22 May 2019 @ 11:20am

                    their TOS itself is biased against conservatives

                    If the expressed beliefs that violate the TOS are more closely associated with conservatives than other political affiliations, maybe consider that the TOS is biased against those beliefs in particular rather than conservatives in general.

                    reply to this | link to this | view in chronology ]

              • icon
                Mike Masnick (profile), 22 May 2019 @ 12:07pm

                Re: Re: Re: Re: Re: Re: Fact-check: FALSE

                I DID, you can't be bothered to look into it.

                I asked you for two things:

                Proof of your assertions and evidence that it wasn't because of the vagaries of content moderation always making mistakes.

                Your link provided neither thing.

                Thus you have failed to provide evidence.

                Care to try again?

                reply to this | link to this | view in chronology ]

        • This comment has been flagged by the community. Click here to show it
          identicon
          Alice Andalack, 22 May 2019 @ 10:41am

          You can't NOW prove Jews suffered in WW2.

          Holocaust Paradox: Long Lives for Those Who Survived...

          https://www.bloomberg.com/opinion/articles/2019-01-28/holocaust-health-paradox-survivors -lived-longer

          Goes against all expectation. Despite the alleged horrors, they're living longer than the contemporary 18 year old soldiers. And so many! Just how effective were the Krauts, anyway?

          POINT IS: It's impossible to prove anything to a biased corporatist. A corporatist which openly advocates corporations having government-conferred power to control even Constitutionally protected speech of course won't admit that there's an anti-conservative bias!

          You're trying to shift burden of proof by expanding "rights" of a mere legal fiction. But whatever_corporation, being a mere legal fiction which is let exist in first place on promise to serve The Public, has to prove its innocence, not The Public to prove every detail.

          reply to this | link to this | view in chronology ]

          • identicon
            bob, 22 May 2019 @ 11:01am

            Re: You can't NOW prove Jews suffered in WW2.

            Pulling a Godwin and denying the holocaust happened won't change or erase what the Nazis did.

            Just how effective were the Krauts, anyway?

            At least 6 million Jews specifically.
            Link

            Considering the cruel and abysmal conditions for concentration camps it's safer to believe that those that survived had a mixture of luck and good genes.

            Now go take your obviously trolling behaviour and depart you holocaust denying piece of crap.

            reply to this | link to this | view in chronology ]

          • identicon
            Anonymous Coward, 22 May 2019 @ 12:46pm

            Re:

            Hey you piece of shit. Speaking as someone who’s known a holocaust survivor. Go fuck yourself

            reply to this | link to this | view in chronology ]

      • icon
        Stephen T. Stone (profile), 22 May 2019 @ 10:10am

        “Anti-conservative bias” in moderation on social media platforms is a myth. To claim it as a fact, you must prove two things:

        1. Any punishment of a conservative user — and only a conservative user — on a given platform is done only because of their political beliefs.

        2. The platform has a distinct and provable pattern of politically motivated punishments that target conservatives/Republicans/right-wingers but leave liberals/Democrats/left-wingers alone.

        I wish you the best of luck; you will certainly need it.

        reply to this | link to this | view in chronology ]

    • icon
      Gary (profile), 22 May 2019 @ 9:56am

      Re: Fact-check: YUP

      There is plenty of evidence, you simply have a list of excuses to make it look like incompetence instead of intent.

      And as Mike predicted, someone jumps up to claim discrimination without any evidence. Brilliant!

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 May 2019 @ 10:09am

      Re: Fact-check: FALSE

      There are lots of reasons that something might be removed that have nothing to do with bias. I used to work for a company that hosted a website that blamed an earthquake in New Zealand on homosexuals. They got a deluge of complaints demanding the site be removed, but didn't act based on free speech grounds. It wasn't until a DMCA notice came in against one of the photos on the site that they did something, which was to completely terminate the account - not specifically for the copyright issue (though that was the excuse given to the customer), but because the company didn't want to deal with the complaints anymore.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 May 2019 @ 10:09am

      Re: Fact-check: FALSE

      there is "anti-conservative bias" in content moderation

      There is an anti-asshole bias in content moderation. If more conservatives than non are being moderated then perhaps conservatives have an asshole bias that you simply can't admit to.

      reply to this | link to this | view in chronology ]

      • icon
        Stephen T. Stone (profile), 22 May 2019 @ 10:12am

        Similarly: If “conservative opinions” are being targeted, it would help to examine which opinions are the targets.

        reply to this | link to this | view in chronology ]

        • icon
          Wendy Cockcroft (profile), 23 May 2019 @ 5:23am

          Re:

          ^This. And, as I commented before, "conservatives" have no desire to clean house and sweep out the idiots that make the rest of us look bad. If they kicked out the crazies and refused to let them use the label without being challenged for it, we'd see a difference. The people defending the indefensible need to have a word with themselves.

          reply to this | link to this | view in chronology ]

    • icon
      Coyne Tibbets (profile), 22 May 2019 @ 11:01pm

      Re: Fact-check: FALSE

      You mean this #LearnToCode? The tag that would be used to belittle and harass journalists?

      You know, just maybe, if conservatives could learn to write a three-word sentence without two of the words being belittling, harassing, derogatory, threatening, and/or racist, they would have less trouble on social media.

      reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    • icon
      Gary (profile), 22 May 2019 @ 10:40am

      Re: BIAS

      You keep repeating that but this isn't an example of Bias - it's an example of stupidity, and maybe pettiness. This doesn't prove an anti-conservative bent.

      reply to this | link to this | view in chronology ]

    • This comment has been flagged by the community. Click here to show it
      identicon
      Alice Andalack, 22 May 2019 @ 10:45am

      Re: BIAS -- @ "TheLizard" -- You're not going to win here...

      With mere proof.

      Techdirt is proof against proof.

      The site is for entertainment purposes only.

      I know, you see apparently people who can be reasoned with, but no one has ever changed Masnick's views. -- Took hundreds of thousands of dollars to force Masnick to even put up a link to fair treatment of Ayyadurai's views! And yet Masnick is back just now with another repeat!

      Anyhoo, for your own sanity, don't take this site seriously.

      reply to this | link to this | view in chronology ]

    • icon
      James Burkhardt (profile), 22 May 2019 @ 12:21pm

      Re: BIAS

      So, this article doesn't show an anti-conservative bias, a repeated point that has been made. This evidence shows exactly what Mike noted - issues with the vagaries of content moderation at scale. The hashtag was flagged after it became associated in part with harassment campaigns. Not because of a conservative association. This lead to either an automated temp ban or a worker enforcing an inflexible policy (both of which Mike and Techdirt have noted are issues when performing content moderation at scale), not because of the conservative viewpoints but because of the use of the banned hashtag.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 22 May 2019 @ 10:39am

    Spent some years volunteer moderating default subs on reddit.

    A volunteer moderator team will ever be overwhelmed with content to review. Burnout necessitates recruiting new volunteer mods. This provides in-roads for conflict of interest. Each new round of volunteer mods brought on included effort to add ambiguous rule-sets to the code of conduct then used to selectively remove perfectly reasonable comments or submissions.

    Considerable time spent weeding out bad actors on the mod team. Closed moderator chat conversation became plagued with conflict as bad actors worked to speed up burnout, harass, and try to remove mods that worked to operate with clearly defined rules and observe free speech. By the time I moved on I was convinced digital media figures routinely fight for control of moderator positions in order to promote their own content, and remove competitors from view.

    reply to this | link to this | view in chronology ]

    • identicon
      MathFox, 22 May 2019 @ 11:48am

      Also someone with moderation experience. On websites were very few comments were deleted, but you need to do some housekeeping to keep a forum decent. The basis that works:

      • Remove commercial spam
      • Remove comments with foul language
      • Welcome people with differing opinions
      • Feel free to reorganize if a discussion strays too much from the original topic
        Moderation is a tool to keep a forum enjoyable for the visitors... not something to propagate an agenda.

      reply to this | link to this | view in chronology ]

  • icon
    ECA (profile), 22 May 2019 @ 11:56am

    THERE IS BIAS

    For any site that has to deal with this and WANTS FACTS, rather then innuendo, lies, fabrication, distortion, and JUST STICKING WITH FACTS/DATA.

    There is bias.
    Proving Circular Logic or Facts based on a Creation of data that is unreliable..IS SUCH A PAIN..

    reply to this | link to this | view in chronology ]

    • icon
      James Burkhardt (profile), 22 May 2019 @ 1:42pm

      Re: THERE IS BIAS

      Showing Political bias in moderation requires that you show the partisan aspects of the content are the cause of moderation, in addition to showing that moderation happens significantly more often to individuals of a specific partisan affiliation.

      The issue with analyzing this is that most social media will provide a biased perspective on who is moderated, due to bias in selecting whom to follow. Those whose views are on the right likely does not follow many whose views are radical and left, and vice versa. That leads to me seeing more complaints about how social media is biased against progressives, seeing mostly the leftists who are unfairly punished and those radical leftists who may clearly fall afoul of TOS, even if I may agree with the views reflected in the posts being taken down. But I mostly am friends with moderate right individuals, so I see far fewer documented cases of moderation of conservatives, with only the ones that make the news cycle really coming up. (though i tend to see a lot of fake "facebook censored this" memes from both sides) This is why a general statistical analysis, rather then personal experience, is important to prove bias.

      You talk about wanting facts, but you provide none. Everytime I see 'proof' presented to me, I find the proof can be explained by well known issues relating to content moderation at scale that match my own experiences as a forum moderator back in the early 2000's, without any analysis of if this proof is in fact typical of a platform in general.

      reply to this | link to this | view in chronology ]

      • icon
        ECA (profile), 22 May 2019 @ 2:47pm

        Re: Re: THERE IS BIAS

        IMO..
        The only real problems with debating or publishing 1 person OPINION, tends to be the idea stuck in a person head..
        "Im right you are wrong". attitudes that Any/all sides may have.

        biased against progressives
        seeing mostly the leftists who are unfairly punished
        radical leftists who may clearly fall afoul of TOS
        moderate right individuals

        Love the wording you are using, But I tend to think of people as individuals first, then ask them where they got there Ideas/ideals.. And most cant answer Where they got those ideas.

        Old friend came back after about 5 years wondering the USA. He made a random statement, "OBAMA DID IT". I Asked him "WHAT did Obama Do?? Explain your comment". My friend, Stuttered, abit, so I asked him "Who told you that? since you dont know, what Obama did."

        I think there is more Variance in Both sides then we think, esp. with self opinions..I cant see a WHOLE group seeing 1 concept/idea/... in any situation. Dogma be damned, quit listening to what others SAY..Make up your own minds. OR throw your bible away and Just believe what they tell you.

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 22 May 2019 @ 12:43pm

    This is why I love the EFF.

    I know some people hate them and want to see them gone, but I'm glad to have them

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 May 2019 @ 6:21pm

      Re: This is why I love the EFF.

      There's generally very little reason to dislike the EFF.

      Which is why you notice that those who do generally happen to have heavily moneyed interests, work in the intellectual property field, and are pretty scummy.

      Redundant description, I know.

      reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.