Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support

from the pain-in-the-[removed] dept

As you should know by now as readers here, content moderation at scale is impossible to do well. Examples for how and why this is so are extensive on these pages, but the crux of the matter is that scaling moderation for content across huge platforms and a variety of avenues in a way that everyone both agrees is right and that doesn't create false-positives is, well, self-evidently impossible. Not everyone agrees what should be moderated, for starters, nor does anyone trust these platforms to actually get it right. Meanwhile, some massive amount of the public does agree that these platforms should be doing something.

And that's how you get big platforms trying to automate content moderation in a way that makes everyone look incredibly dumb. Yet another example of this is the Google Play store banning a video player application over "sexual content and profanity", but just not for the reasons you're probably already imagining.

This time, the puritan robot overlords that run the Play Store briefly decided that listing support for common subtitle files is enough to get your app banned. The developer for Just (Video) Player posted their story to Hacker News, writing in the app's bug tracker, "After a tiny unrelated description update, Just Player got suspended from the Google Play Store for "Sexual Content and Profanity policy". Google finds issues with following: Full description (en_US): “* Subtitles: SRT, SSA, ASS, TTML, VTT.”"

Yes, just listing standard video player features like support for the "ASS" subtitle format was apparently enough to temporarily earn a suspension. The developer says they "immediately filed an appeal" and today, the app is back up with the ASS subtitle listing still in the description.

While I admit to appreciating that all of this happened so as to create the situation where a publication writes "with the ASS subtitle listing still in the description", this story is in every other way indicative as to why broad, automated moderation is impossible to do well. It also lays bare the fact that there is clearly zero human review or intervention in this process. Instead, the filter that looks for naughty words and content detects "ASS", can't consider the context for why that word or acronym might appear in the description, and then just outright banned the app. Sure, Google reinstated it... but all that shows is that the human intervention that occurs after the fact can and should occur before the ban.

And, while I'm typically loathe to throw a ton of props Apple's way, it's worth noting that Apple's app review process actually does what Google's doesn't.

Saying Google's app review bots "ban first and ask questions later" would actually be an improvement over the current situation, since the bots can't ask questions. The bots ban, send an automated email, and it's up to the developers to figure out why they were banned and jump through hoops to make the bots happy, often without being able to speak to a human.

Google and Apple both collect a percentage of app sales, which the companies categorize as a necessary tax that pays for the infrastructure of the store ecosystem. Apple uses this money partly to hire an army of human app reviewers, a system that Google Play developers often hold up as an example that Google should follow. Instead, Google only has this bot system—or at best, it has an extremely small team of manual reviewers—and developers frequently complain that they are at the mercy of an illogical bot, with no human to speak to even during an appeals process.

Which moves this beyond mere annoyance and into a competitive issues between the two major app stores in the mobile market. Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation... but I doubt it. The more these stories crop up, the more developers will simply decide Google isn't worth the trouble.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: ass, content moderation, google play store, keyword filters, sexual content, video player
Companies: google


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Rekrul, 28 Jan 2021 @ 8:45pm

    If only there were a way for users to get apps from some other source that didn't require the blessing of Apple or Google...

    reply to this | link to this | view in chronology ]

    • icon
      Thad (profile), 29 Jan 2021 @ 7:56am

      Re:

      Sideloading is a thing, and it's not difficult on Android.

      But it's difficult enough that I don't expect my grandparents to figure out how to do it.

      reply to this | link to this | view in chronology ]

      • icon
        PaulT (profile), 30 Jan 2021 @ 12:20am

        Re: Re:

        It's difficult enough now, because the only people even considering it are those who are tech savvy enough to want it and have needs beyond the store supplied with their phone. It would be possible to make it easier, there just has to be the incentive to cater to that market.

        Even so, chances are people like that aren't going to do it themselves anyway, they'll just ask whichever friend/family members they already have doing their free tech support for them to do it.

        reply to this | link to this | view in chronology ]

        • identicon
          Rekrul, 4 Feb 2021 @ 10:43am

          Re: Re: Re:

          Not only that, but the majority of people wouldn't know how to find the apps to sideload them in the first place. If it's not on the pre-configured store, they wouldn't know it exists.

          It's not just older folks either, I've encountered younger people who are mostly clueless about technology. To be clear, I do not consider myself an expert in any way, but I've seen people in their 20s and 30s, who grew up around this technology, but who don't know how to do simple things, like change the Desktop screen size on their computer or unpack a Rar file.

          reply to this | link to this | view in chronology ]

          • icon
            PaulT (profile), 4 Feb 2021 @ 9:46pm

            Re: Re: Re: Re:

            That's all true, but most people have a person in their lives who they turn to for help with such things. There's nothing that can be done for the truly and wilfully clueless, they will always need help. But, you also can't pretend sideloading doesn't exist just because some people can't do it on their own. If enough desire exists for apps outside of official channels, there will be the incentive to make this as easy and foolproof as possible, then the remaining fools will just have to ask the same people they ask to do everything else for them.

            reply to this | link to this | view in chronology ]

  • icon
    PaulT (profile), 29 Jan 2021 @ 12:04am

    "Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation... but I doubt it"

    It's doubtful because this has been a complaint against Google since day one - if you have a problem with one of their services, it's hard to get an actual human being for support. This should not be a surprise to anyone, least of all developers who have tied their income to Google's platform.

    reply to this | link to this | view in chronology ]

  • identicon
    Bobvious, 29 Jan 2021 @ 12:54am

    Thank heavens

    Compilers don't produce ASSembly language anymore.

    reply to this | link to this | view in chronology ]

  • icon
    Ehud Gavron (profile), 29 Jan 2021 @ 12:55am

    Tourettes

    I'm reminded of the South Park episode where Cartman has Tourettes Syndrome. HE gets to say what he likes, but if others say it, they get in trouble.

    For the record, my audio player uses the Portable Electronic Network Interactive Sound codec. I guess I'll just go delete the codebase and start again.

    E

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 29 Jan 2021 @ 3:36am

      Re: Tourettes

      Beware, puritans spot penises everywhere: https://jira.atlassian.com/browse/JRASERVER-65811

      Change the string "allopenissues" to not include the word "penis"

      God. fucking. shit.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 29 Jan 2021 @ 3:46am

        Re: Re: Tourettes

        decades later, and we're still dealing with the Penistone and Scunthorpe problem because some people get upset about some words.

        We're doomed.

        reply to this | link to this | view in chronology ]

        • icon
          PaulT (profile), 29 Jan 2021 @ 5:30am

          Re: Re: Re: Tourettes

          Nah, the Scunthorpe Problem is because it's difficult (or takes a lot of manual intervention) to automate filters in ways that allow obviously acceptable words to get through but not open up loopholes that make the moderation ineffective.

          This is about a human idiot who made that same mistake, but those have been around long before Scunthorpe existed, let alone the concept named after them. As long as the response is to correctly laugh at the idiots and not enforce the idiocy on to everyone else, we'll be fine.

          reply to this | link to this | view in chronology ]

  • icon
    That Anonymous Coward (profile), 29 Jan 2021 @ 3:52am

    But if your child can see an ass online they might grow up to be like Ted Cruz... now you're all for it aren't you?

    reply to this | link to this | view in chronology ]

    • icon
      Ehud Gavron (profile), 29 Jan 2021 @ 6:49am

      Re: Your child traumatized by seeing Ted Cruz

      In the Indy 500 an (R) after the driver's name means they're a rookie.

      In the US Congress an (R) after your name means you're an ass.

      E

      reply to this | link to this | view in chronology ]

  • identicon
    Glenn, 29 Jan 2021 @ 5:17am

    I wasn't raised to view "ass" as profanity, so I don't get why anyone does. It all seems more about thought police than anything else (as in, "wow, people are really stupid aren't they").

    reply to this | link to this | view in chronology ]

  • icon
    nasch (profile), 29 Jan 2021 @ 9:10am

    Advantages and disadvantages

    The nice thing about little or no human review is that your app can go live in minutes or hours, whereas it used to take days on the Apple App Store. Maybe they've improved things recently. It does have its downside though.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 31 Jan 2021 @ 10:15pm

      Re: Advantages and disadvantages

      I was just about to say that relying solely on human review causes delays in apps (and also app updates) going live. You're also at the mercy of how the reviewer interprets the App Store rules (and, as noted, human moderators are unable to agree on everything).

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Mar 2021 @ 9:47pm

    similar experience

    One of my apps (which collects no information) had an accurate privacy policy: "This app collects no information." But... Google banned it... because I did not have a valid privacy policy. I appealed it, and a person finally told me (after I jumped through their robotic hoops) that the problem was that my privacy policy did not mention "privacy." I changed it to "This app protects your privacy by collecting no information," and they accepted that.
    73

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.