Content Moderation At Scale Especially Doesn't Work When You Hide All The Rules

from the getting-twitchy dept

For quite a while now, we've pointed out that doing any serious content moderation on major internet sites is laughably difficult, if not impossible. Whether done in a purely automated format, or with real human oversight, everything ends up boiling down to just how much collateral damage are we all willing to accept when sites attempt to enforce moderation rules. Even when sites attempt to communicate the rules to the public in a somewhat transparent fashion, such as Facebook, it all inevitably goes to the kind of hell that includes nixing accounts for sharing what is purely art.

But when sites don't bother to tell their users what the rules are, even after exacting punishment for violation of those rules? Well, that's when you have a bunch of Twitch streamers wondering what the hell is going on.

Earlier this week, Twitch suspended a streamer named Quqco for wearing a cosplay of Street Fighter heroine Chun-Li on stream, deeming her outfit “sexually suggestive.” This took Quqco—and many others—by surprise, given that the outfit was not overtly risqué. But this was not an isolated incident. In the past few days, Twitch has been cracking down on so-called “sexually suggestive” content more aggressively than usual, and picking some questionable targets for its crusade.

The whole post is long and very detailed and should absolutely be read in full. The article lists numerous examples of Twitch streamers being handed out warnings and suspensions under any number of circumstances that range from cosplaying as Street Fighter characters, to wearing a sports bra and baggy shorts on a stream, to drawing clothed cartoon characters. All those interviewed for the post that were on the receiving end of these warnings or suspensions were completely flummoxed.

Especially when their own inquiries into their punishment were being responded to like this.

For example, Fareeha’s warning specifically accused her of wearing “underwear or lingerie,” when she was, in reality, wearing baggy gym clothes in a setting where you’d expect to see them. Twitch’s guidelines around what streamers can and cannot wear are vague and contextual; the sort of attire Fareeha was wearing might not have cut it if a streamer was broadcasting from their bedroom, but streamers regularly wear gym clothes in the gym. Some men even go shirtless. It’s not clear why Twitch singled out Fareeha.

Fareeha specifically believes she may be the victim of another aspect of the internet that makes content moderation at scale impossible: trolls. It turns out that there are dedicated Discord channels out there where people get off on doing mass-reports of Twitch streamers they don't like for any number of reasons. Obviously, those many numbers of reasons often fall into categories that include "they're women doing things we don't like" and "they're people that don't look like us doing things we don't like." The theory goes that whatever automation Twitch has built into its moderation system -- and there surely must be something of that sort of thing -- it's likely to be sensitive to mass reporting of ToS violations. This is being gamed by trolls to get the system to punish Twitch streamers who otherwise would never have been punished.

Is that what's happening? Nobody knows, mostly because Twitch is being frustratingly opaque on the matter.

Saruei, who was suspended for drawing “nude” characters, declined to speak to Kotaku out of concern that she could face further repercussions from Twitch. However, prior to her recent suspension, she spoke out against what she feels is “hypocrisy” on Twitch’s part. Suggestive poses apparently aren’t allowed in her drawings, she said, but it’s fine when some people do them IRL.

“These are suggestive poses, right?” she said of her own art while discussing her Twitch warning during a recent (now-unavailable) stream. “We agree with that, right? Why I can’t draw waifus like this when there is fucking Twitch girls that can do it?” She went on to express frustration about the lack of clarity that she, like others, has had to deal with. “I hope it won’t happen again, because I asked them ‘What is the problem with these drawings?’” she said. “Is it the clothing or the pose? What is against Twitch guidelines? I need to know.”

Those last two sentences are all you need to hear to recognize the massive problem that is Twitch attempting to do content moderation at a scale that is largely impossible while not being transparent as to how to stay on the right side of that moderation. There is entirely too much gray area, too much room for gaming the moderation, and too much stupid collateral damage to make any of this worth it.

Meanwhile, the Twitch streamers who make Twitch's product attractive are stuck trying to figure out what the rules are. Until they decide to go somewhere else, that is.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: content moderation, content moderation at scale, quqco, rules, transparency
Companies: twitch

Reader Comments

Subscribe: RSS

View by: Time | Thread

  1. icon
    Gary (profile), 20 Sep 2019 @ 9:25pm


    The thing going on with these is "Brigading." Where an small army (large minority?) of asshats all get together and submit (fake) complaints.

    The algorithm automatically trips since 10, 20, or 50 flags have been tripped.

    Blue Balls tries to do this at TD, but withought backing the effort fails mo matter how many posts they make.

    On Twitch, this takes down the targets and causes serious problems.

    But who is to blame - Twitch, or the "Brigade" of Trolls?

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Special Affiliate Offer

Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.