Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support

from the pain-in-the-[removed] dept

As you should know by now as readers here, content moderation at scale is impossible to do well. Examples for how and why this is so are extensive on these pages, but the crux of the matter is that scaling moderation for content across huge platforms and a variety of avenues in a way that everyone both agrees is right and that doesn’t create false-positives is, well, self-evidently impossible. Not everyone agrees what should be moderated, for starters, nor does anyone trust these platforms to actually get it right. Meanwhile, some massive amount of the public does agree that these platforms should be doing something.

And that’s how you get big platforms trying to automate content moderation in a way that makes everyone look incredibly dumb. Yet another example of this is the Google Play store banning a video player application over “sexual content and profanity”, but just not for the reasons you’re probably already imagining.

This time, the puritan robot overlords that run the Play Store briefly decided that listing support for common subtitle files is enough to get your app banned. The developer for Just (Video) Player posted their story to Hacker News, writing in the app’s bug tracker, “After a tiny unrelated description update, Just Player got suspended from the Google Play Store for “Sexual Content and Profanity policy”. Google finds issues with following: Full description (en_US): “* Subtitles: SRT, SSA, ASS, TTML, VTT.””

Yes, just listing standard video player features like support for the “ASS” subtitle format was apparently enough to temporarily earn a suspension. The developer says they “immediately filed an appeal” and today, the app is back up with the ASS subtitle listing still in the description.

While I admit to appreciating that all of this happened so as to create the situation where a publication writes “with the ASS subtitle listing still in the description”, this story is in every other way indicative as to why broad, automated moderation is impossible to do well. It also lays bare the fact that there is clearly zero human review or intervention in this process. Instead, the filter that looks for naughty words and content detects “ASS”, can’t consider the context for why that word or acronym might appear in the description, and then just outright banned the app. Sure, Google reinstated it… but all that shows is that the human intervention that occurs after the fact can and should occur before the ban.

And, while I’m typically loathe to throw a ton of props Apple’s way, it’s worth noting that Apple’s app review process actually does what Google’s doesn’t.

Saying Google’s app review bots “ban first and ask questions later” would actually be an improvement over the current situation, since the bots can’t ask questions. The bots ban, send an automated email, and it’s up to the developers to figure out why they were banned and jump through hoops to make the bots happy, often without being able to speak to a human.

Google and Apple both collect a percentage of app sales, which the companies categorize as a necessary tax that pays for the infrastructure of the store ecosystem. Apple uses this money partly to hire an army of human app reviewers, a system that Google Play developers often hold up as an example that Google should follow. Instead, Google only has this bot system—or at best, it has an extremely small team of manual reviewers—and developers frequently complain that they are at the mercy of an illogical bot, with no human to speak to even during an appeals process.

Which moves this beyond mere annoyance and into a competitive issues between the two major app stores in the mobile market. Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation… but I doubt it. The more these stories crop up, the more developers will simply decide Google isn’t worth the trouble.

Filed Under: , , , , ,
Companies: google

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support”

Subscribe: RSS Leave a comment
PaulT (profile) says:

Re: Re: Re:

It’s difficult enough now, because the only people even considering it are those who are tech savvy enough to want it and have needs beyond the store supplied with their phone. It would be possible to make it easier, there just has to be the incentive to cater to that market.

Even so, chances are people like that aren’t going to do it themselves anyway, they’ll just ask whichever friend/family members they already have doing their free tech support for them to do it.

Rekrul says:

Re: Re: Re: Re:

Not only that, but the majority of people wouldn’t know how to find the apps to sideload them in the first place. If it’s not on the pre-configured store, they wouldn’t know it exists.

It’s not just older folks either, I’ve encountered younger people who are mostly clueless about technology. To be clear, I do not consider myself an expert in any way, but I’ve seen people in their 20s and 30s, who grew up around this technology, but who don’t know how to do simple things, like change the Desktop screen size on their computer or unpack a Rar file.

PaulT (profile) says:

Re: Re: Re:2 Re:

That’s all true, but most people have a person in their lives who they turn to for help with such things. There’s nothing that can be done for the truly and wilfully clueless, they will always need help. But, you also can’t pretend sideloading doesn’t exist just because some people can’t do it on their own. If enough desire exists for apps outside of official channels, there will be the incentive to make this as easy and foolproof as possible, then the remaining fools will just have to ask the same people they ask to do everything else for them.

PaulT (profile) says:

"Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation… but I doubt it"

It’s doubtful because this has been a complaint against Google since day one – if you have a problem with one of their services, it’s hard to get an actual human being for support. This should not be a surprise to anyone, least of all developers who have tied their income to Google’s platform.

This comment has been deemed funny by the community.
Ehud Gavron (profile) says:


I’m reminded of the South Park episode where Cartman has Tourettes Syndrome. HE gets to say what he likes, but if others say it, they get in trouble.

For the record, my audio player uses the Portable Electronic Network Interactive Sound codec. I guess I’ll just go delete the codebase and start again.


PaulT (profile) says:

Re: Re: Re: Tourettes

Nah, the Scunthorpe Problem is because it’s difficult (or takes a lot of manual intervention) to automate filters in ways that allow obviously acceptable words to get through but not open up loopholes that make the moderation ineffective.

This is about a human idiot who made that same mistake, but those have been around long before Scunthorpe existed, let alone the concept named after them. As long as the response is to correctly laugh at the idiots and not enforce the idiocy on to everyone else, we’ll be fine.

Anonymous Coward says:

similar experience

One of my apps (which collects no information) had an accurate privacy policy: "This app collects no information." But… Google banned it… because I did not have a valid privacy policy. I appealed it, and a person finally told me (after I jumped through their robotic hoops) that the problem was that my privacy policy did not mention "privacy." I changed it to "This app protects your privacy by collecting no information," and they accepted that.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...