Content Moderation Is Impossible: Facebook's Attempts To Block Mask Gouging Took Down DIY Face Mask Instructions

from the impossibility-theorem dept

Content moderation at scale is impossible to do well. The latest example? Facebook's rules to takedown content deemed to be people trying to price gouge medical supplies like face masks resulted in tutorials on how to make your own face masks being taken down:

Facebook’s systems threatened to ban the organizers of hand-sewn masks from posting or commenting, they said, landing them in what is colloquially known as “Facebook Jail.” They said it also threatened to delete the groups. The issue has affected do-it-yourself mask makers in states like Pennsylvania, Illinois and California, they said.

As the NY Times notes, Facebook, like most other social media sites, has been aggressively trying to block price gouging medical supplies:

At the top of its list were ads for masks, hand sanitizer and others looking to profit from the sale of safety equipment. Facebook banned advertising for such equipment last month, and has taken down nearly all posts related to the sale of masks across its Craigslist-like section, called Marketplace.

But as the company ramped up efforts to crack down on scammers and other miscreants, volunteer coordinators may have been caught in the crossfire.

“The automated systems we set up to prevent the sale of medical masks needed by health workers have inadvertently blocked some efforts to donate supplies,” Facebook said in a statement. “We apologize for this error and are working to update our systems to avoid mistakes like this going forward. We don’t want to put obstacles in the way of people doing a good thing.”

This is not an attack on Facebook, but, once again, it's important to recognize just how impossible it is to do these things well, especially at scale, and especially in the midst of a pandemic where things are changing daily. With the US only changing its recommendations on face masks last week, demand for any kind of face mask, including homemade ones, has sky-rocketed. And if you trying to build systems that are trained to look out for posts "advertising" things having to do with face masks -- which was important in the first few weeks of the pandemic -- they're inevitably going to lead to false positives flagging those who are actually trying to help.

Filed Under: content moderation, content moderation at scale, covid-19, diy, facemasks, hoarding, masnick's impossibility theorem, price gouging


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Upstream (profile), 8 Apr 2020 @ 12:30pm

    No matter how many times you tell them, and no matter how much evidence you provide, it appears that some people still are heavily invested in denying reality.

    Hmm, does this comment go here, or under the article about hospitals firing people for talking about PPE shortages, or under the Jon Cusack article, or under the Navy article, or . . . .

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 8 Apr 2020 @ 2:19pm

    At the top of its list were ads for masks, hand sanitizer and others looking to profit from the sale of safety equipment.

    Why would they ban that in general? Not all "profit from the sale of safety equipment" would be gouging (almost all safety equipment is made by for-profit companies). Many people desperately need money, and letting them sell it for what they paid would be better than letting it sit on their shelf forever.

    reply to this | link to this | view in chronology ]

  • identicon
    Lawrence D’Oliveiro, 8 Apr 2020 @ 3:37pm

    They Have Too Much Power

    If they cannot be trusted to exercise that power responsibly, then it must be taken away from them.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 8 Apr 2020 @ 6:01pm

      Re: They Have Too Much Power

      And just who do you trust to exercise even greater power by setting the rules for social media sites? Governments have a terrible history when and where they gain control over speech.

      reply to this | link to this | view in chronology ]

      • identicon
        Lawrence D’Oliveiro, 8 Apr 2020 @ 6:44pm

        Re: Governments have a terrible history

        I don’t know about you, but we over here live in a democracy, where the Government is answerable to the voters. We have a system called “checks and balances”, to try to ensure that those who wield the power on our behalf are actually answerable to us in some form.

        It certainly beats allowing large, faceless, amoral megacorporations to run your society.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 9 Apr 2020 @ 3:29am

          Re: Re: Governments have a terrible history

          Assuming that you are in the US, you are in a country where regulatory authorities have been captured and do the bidding of industry, rather than protect the consumers, due to a failure of checks and balances.

          reply to this | link to this | view in chronology ]

        • identicon
          Annonymouse, 9 Apr 2020 @ 8:35am

          Re: Re: Governments have a terrible history

          Love thus post but you forgot the sarcasm tags.

          reply to this | link to this | view in chronology ]

    • icon
      That One Guy (profile), 8 Apr 2020 @ 8:45pm

      ... So let's give that power to someone with even more!

      The first amendment would like to have a word with you, and I'm still waiting for you to post what regulations you think should be applied that would solve this or any other similar moderation issue.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Apr 2020 @ 5:41am

      Re: They Have Too Much Power

      "If they cannot be trusted to exercise that power responsibly, then it must be taken away from them."

      How does this work, exactly?

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Apr 2020 @ 12:46am

    Hmm. Facebook does a crappy job of moderation. Therefore, its impossible to do a good job of moderation at scale. What could possibly be the flaw in that logic?

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Apr 2020 @ 5:42am

      Re:

      Perhaps the flaw is in your lack of data.

      reply to this | link to this | view in chronology ]

    • identicon
      Annonymouse, 9 Apr 2020 @ 8:42am

      Re:

      It is the age old problem of quick management decisions that are simplistic tend to have foreseen repercussions if anybody in the decision making loop actually bothered to foresee anything.

      QA: Lets do a Pareto Analysis before committing
      Management: That will take time and we need to be decisive. (Makes decisions)
      Real World: (Shctuff hits fan)
      Management: Why didn't you warn us?
      QA: (makes sure management signed paperwork)

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Apr 2020 @ 9:53am

    Pollution control at scale is like, really hard and stuff. We should disband the EPA to protect scrappy growth hacking oil extraction startups.

    reply to this | link to this | view in chronology ]

  • identicon
    Mike, 29 Apr 2020 @ 2:35pm

    So...if you make cloth masks and want to sell them cheap....you STILL CAN'T do that because Facebook has blanketed Marketplace with a ban on anything Face, Mask, or Covering related

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Special Affiliate Offer

Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.