Content Moderation Is Impossible: Facebook's Attempts To Block Mask Gouging Took Down DIY Face Mask Instructions

from the impossibility-theorem dept

Content moderation at scale is impossible to do well. The latest example? Facebook’s rules to takedown content deemed to be people trying to price gouge medical supplies like face masks resulted in tutorials on how to make your own face masks being taken down:

Facebook?s systems threatened to ban the organizers of hand-sewn masks from posting or commenting, they said, landing them in what is colloquially known as ?Facebook Jail.? They said it also threatened to delete the groups. The issue has affected do-it-yourself mask makers in states like Pennsylvania, Illinois and California, they said.

As the NY Times notes, Facebook, like most other social media sites, has been aggressively trying to block price gouging medical supplies:

At the top of its list were ads for masks, hand sanitizer and others looking to profit from the sale of safety equipment. Facebook banned advertising for such equipment last month, and has taken down nearly all posts related to the sale of masks across its Craigslist-like section, called Marketplace.

But as the company ramped up efforts to crack down on scammers and other miscreants, volunteer coordinators may have been caught in the crossfire.

?The automated systems we set up to prevent the sale of medical masks needed by health workers have inadvertently blocked some efforts to donate supplies,? Facebook said in a statement. ?We apologize for this error and are working to update our systems to avoid mistakes like this going forward. We don?t want to put obstacles in the way of people doing a good thing.?

This is not an attack on Facebook, but, once again, it’s important to recognize just how impossible it is to do these things well, especially at scale, and especially in the midst of a pandemic where things are changing daily. With the US only changing its recommendations on face masks last week, demand for any kind of face mask, including homemade ones, has sky-rocketed. And if you trying to build systems that are trained to look out for posts “advertising” things having to do with face masks — which was important in the first few weeks of the pandemic — they’re inevitably going to lead to false positives flagging those who are actually trying to help.

Filed Under: , , , , , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation Is Impossible: Facebook's Attempts To Block Mask Gouging Took Down DIY Face Mask Instructions”

Subscribe: RSS Leave a comment
Anonymous Coward says:

At the top of its list were ads for masks, hand sanitizer and others looking to profit from the sale of safety equipment.

Why would they ban that in general? Not all "profit from the sale of safety equipment" would be gouging (almost all safety equipment is made by for-profit companies). Many people desperately need money, and letting them sell it for what they paid would be better than letting it sit on their shelf forever.

Lawrence D’Oliveiro says:

Re: Re: Governments have a terrible history

I don’t know about you, but we over here live in a democracy, where the Government is answerable to the voters. We have a system called “checks and balances”, to try to ensure that those who wield the power on our behalf are actually answerable to us in some form.

It certainly beats allowing large, faceless, amoral megacorporations to run your society.

Annonymouse says:

Re: Re:

It is the age old problem of quick management decisions that are simplistic tend to have foreseen repercussions if anybody in the decision making loop actually bothered to foresee anything.

QA: Lets do a Pareto Analysis before committing
Management: That will take time and we need to be decisive. (Makes decisions)
Real World: (Shctuff hits fan)
Management: Why didn’t you warn us?
QA: (makes sure management signed paperwork)

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...