HideTechdirt is off for the long weekend! We'll be back with our regular posts tomorrow.
HideTechdirt is off for the long weekend! We'll be back with our regular posts tomorrow.

Oversight Board Overturning Instagram Takedown Of Ayahuasca Post Demonstrates The Impossibility Of Content Moderation

from the this-is-not-easy dept

Congress has been holding lots of "but think of the children online!" hearings over the past couple of months, and one prominent topic that comes up over and over again is the fact that people can find "drug" information online. Fears about kids and drugs goes back decades, but politicians love it, because it always works. And, of course, the media loves to run these overhyped stories. A quick search finds dozens of stories like the following in just the last month or so:

There are many more such stories, but you get the idea. The Instagram stories, in particular, were targeted by the recent Senate hearing which was focused on allowing Senators to grandstand on how bad Instagram supposedly is for kids. Instagram responded to the report by pointing out that it's doing a ton of stuff to try to block such content:

"We removed 1.8 million pieces of content related to drug sales in the last quarter alone, and due to our improving detection technology, the prevalence of such content is about 0.05 percent of content viewed, or about 5 views per every 10,000. We'll continue to improve in this area in our ongoing efforts to keep Instagram safe, particularly for our youngest community members."

But, within a day of that report getting so much attention, you had the Oversight Board step in to demonstrate how much more difficult an issue this is than it sounds to grandstanding politicians or narrative-driving journalists. The Oversight Board overturned Instagram's decision to take down a post that was promoting ayahuasca as medicine. Instagram took down the post (which had just started trending) because it was promoting it as medical treatment:

In July 2021, an Instagram account for a spiritual school based in Brazil posted a picture of a dark brown liquid in a jar and two bottles, described as ayahuasca in the accompanying text in Portuguese. Ayahuasca is a plant-based brew with psychoactive properties that has deeply-rooted religious and ceremonial uses among Indigenous and other groups in some South American countries, and related communities elsewhere. Ayahuasca contains plants which are sources of dimethyltryptamine (DMT), a substance that is prohibited under Schedule I of the 1971 UN Convention on Psychotropic Substances and the law of many countries, though there are relevant exceptions under international law for substances containing DMT such as ayahuasca, and exceptions under some national laws, including in Brazil, for other uses such as religious and Indigenous use.

The text states that “AYAHUASCA IS FOR THOSE WHO HAVE THE COURAGE TO FACE THEMSELVES” and includes statements that ayahuasca is for those who want to “correct themselves,” “enlighten,” “overcome fear” and “break free.” It further states ayahuasca is a “remedy” and “can help you” if one has humility and respect. It ends with “Ayahuasca, Ayahuasca!/Gratitude, Queen of the Forest!”

The content was viewed over 15,500 times and no users reported it. The post was flagged for review by Meta’s automated systems because it had received around 4,000 views and was “trending.” Meta specified neither the image nor the text triggered automatic review. The post was subsequently reviewed by a human moderator and removed. Meta told the Board that it was removed for violating Facebook’s Community Standard on Regulated Goods, but later stated it removed the content for “violating the Instagram Community Guidelines, which include a link to the Facebook Community Standard on Regulated Goods.” Meta notified the user that the post went against Instagram’s Community Guidelines, stating “post removed for sale of illegal or regulated goods.” The messaging also noted that Meta removes “posts promoting the use of hard drugs.”

So, there are a few interesting tidbits there worth pointing out. Contrary to the fear-mongering and grandstanding, this notes that Instagram has a system to proactively flag this kind of content, and that some of the priority of such reviews is based on whether or not the content is getting a lot of attention and going viral (these are both things that many critics of Facebook/Instagram/Meta insist the company does not do). And, if you just listened to the Senators at hearings, or just read all those news articles above, you might not realize th.

But, the more important thing that this case showcases, is just how difficult it is to make these kinds of calls at scale. What seems obvious to some is not so obvious, and there's a lot of subjectivity involved. This can be seen in the way the two sides (the poster of the original content and the company) explained their thinking. The organization that made the original post about ayahuasca says that it was just an image of a regulated and legal ceremony:

The user stated in their appeal that they are certain the post does not violate Instagram’s Community Guidelines, as their page is informative and never encouraged or recommended the purchase or sale of any product prohibited by the Community Guidelines. They said that they took the photo at one of their ceremonies, which are regulated and legal. According to the user, the account aims to demystify the sacred ayahuasca drink. They said that there is a great lack of knowledge about ayahuasca. The user stated that it brings spiritual comfort to people and their ceremonies can improve societal wellbeing. They further state that they have posted the same content previously on their account and that post remains online.

Instagram, for its part, takes the more hardline "drugs bad, mmkay" line that Congress and the press seems to want them to take:

Meta stated that the content violated Facebook’s Community Standards because “the user described ayahuasca with a heart emoji, referred to it as ‘medicine,’ and stated that it ‘can help you.’” Following a question from the Board about whether the content was removed for violating Instagram’s Community Guidelines or Facebook’s Community Standards, Meta responded the content was removed “for violating the Instagram Community Guidelines, which include a link to the Facebook Community Standards on Regulated Goods.” Specifically, the user violated Instagram’s prohibition on content “buying or selling illegal or prescription drugs (even if legal in your region).” Meta also cited another line of the Community Guidelines which links to the Community Standard on Regulated Goods, which “clarifies that Facebook prohibits content that ‘[c]oordinates or promotes (by which we mean speaks positively about [...] non-medical drugs.’”

Referring to Meta’s values, the company stated that “Safety” displaced “Voice.” Meta noted that users are permitted to advocate for the legalization of non-medical drugs and to discuss the medical and scientific benefits of non-medical drugs, but that there is no religious or traditional use allowance. Meta argued that this rule strikes the correct balance between “Voice” and “Safety.”

Meta also stated that prohibiting this content follows human rights principles. It stated that it considered the right to freedom of expression under Article 19 ICCPR and the right to freedom of religion or belief under Article 18 ICCPR, and argued that its decision satisfied the conditions required to restrict these rights.

According to Meta, Facebook’s Community Standard is easily accessible and its non-public definition of non-medical drug as a “substance that causes ‘a marked change in consciousness’” is apparent. It further argued that the decision sought to protect public health. It stated that dimethyltryptamine (DMT), one of the active hallucinogenic substances in ayahuasca, poses significant safety risks, citing the 1971 UN Convention on Psychotropic Substances.

The final ruling by the Oversight Board overturn's Instagram's decision to take it down, citing that Instagram's own community guidelines were somewhat confusing and focused on the buying and selling of illegal drugs (though it notes that the rules are confusing, because they said "illegal or prescription drugs (even if legal in your region)" which is, as the Board notes "confusing and contradictory." There was also some level of conflict between Instagram's rules and Facebook's rules, and an unclear delineation of which one should apply where.

But where the impossibility of content moderation on topics like this really shines through is in the discussion about "voice" vs. "safety," which was one of the key points that Instagram made in defending its decision. The Board says it values things differently than the company:

The Board concludes that Meta’s decision to remove the content was not consistent with the company’s values. In this case, as in many, Meta's values of “Voice,” “Safety,” and “Dignity” point in different directions. Meta's decision to take down the post weighted “Safety” over “Voice.” The Board would balance the values differently, believing that the genuine but not particularly strong interests in “Safety” are outweighed in this context by the value of “Voice” and the importance of recognizing the “Dignity” of those engaging in traditional or religious uses where there is historic evidence of such use, including by Indigenous and religious communities. Scientific research indicates that the use of ayahuasca in a controlled context in traditional and religious ceremonies is not linked to a serious risk of harm. Meta cited the European Court of Human Rights case of Franklin-Beentjes and Ceflu-Luz da Floresta v. The Netherlands (Case No. 28167/07, European Court of Human Rights, May 6, 2014) to demonstrate the risks of ayahuasca use. Other courts, however, have reached different conclusions – for example, the United States Supreme Court, in considering the risk of harm from “the circumscribed, sacramental use of hoasca” by members of an ayahuasca based religion, found that the government had not put forward sufficient evidence of harm from religious use, which was its burden, to justify the prohibition in these circumstances. Meta’s rationale does not appear to have taken into account controlled uses of ayahuasca which aim to mitigate health risks. In light of scientific research, Meta’s rationale did not demonstrate the danger of this post to the value of “Safety” in a manner sufficient to displace “Voice” and “Dignity” to the extent to justify removal of the post.

There should be no way to look at that paragraph and not realize just how subjective all of this is. Every one of these involves judgment calls and people weigh different values in very different ways. Even people who work at the same company. Figuring out how to (1) write rules that take all this into account, (2) express those rules in AI moderation systems, (3) express those rules clearly to tens of thousands of human moderators across hundreds of languages and cultures, (4) and expect it to work at a scale of billions of users and millions of posts per day... is literally impossible. It cannot be done well.

That doesn't mean that everyone should give up, but there should at least be some level of humility in recognizing the level of this task, and highlighting individual decisions that we disagree with is not evidence that people or the company "don't care" about this issue, but rather evidence of just how impossible a task this truly is.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: ayahuasca, brazil, content moderation, drugs, health misinformation
Companies: facebook, instagram, meta, oversight board


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • icon
    Rico R. (profile), 30 Dec 2021 @ 11:08am

    What I want to know is how many of these senators would be outraged if Instagram removed a photo of someone's child having their first communion on the grounds that it "promotes" underage drinking? Couldn't it be interpreted (as flawed as it may be) that such a post would run afoul of the same policy they're asking Instagram to crack down on?

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Dec 2021 @ 11:16am

    But sir, EVERYTHING in the US is regulated.

    In light of Wickard v Filburn, I'm concerned that they will pull my photos of wheat fields with the legend "Made by Norwegian Bachelor Farmers. Brings out your innate powers of resignation."

    reply to this | link to this | view in chronology ]

  • icon
    Jeffrey Nonken (profile), 30 Dec 2021 @ 11:23am

    Nothing is impossible to the person who doesn't have to do it.

    reply to this | link to this | view in chronology ]

  • identicon
    Glenn, 30 Dec 2021 @ 12:33pm

    So-called "kids" wouldn't even want drugs if it weren't for their parents and other so-called "grown-ups" making their lives hell, now would they.

    reply to this | link to this | view in chronology ]

  • identicon
    Ninja, 30 Dec 2021 @ 12:35pm

    As far as I know ayahuasca is also widely used in religious rituals. Shamanism if memory serves. Even though it is generally illegal (as in forbidden by law) here generally speaking you are free to produce and use in religious rituals. So Instagram has basically two options here: ban because it is illegal or allow because it is religious stuff? Also, if it's totally fine in country X but not in country Y but the persons deciding whether to ban or not are from country Y, then what? And what if the ones evaluating are quite christian and decide to ban based on their religious beliefs? Can Instagram guarantee the neutrality of the moderation? So many difficult questions in this specific case but it's true for many other issues. I'd say that one way to help mitigate wrongful takedowns would be to offer better appealing tools and combine other methods of moderation considering aspects such as account activity, longevity, exposure and amount of users flagging before taking action and when determining the urgency of an appeal. Even then I'm fairly sure collateral damage would still happen. It is an herculean task after all. Just a few thoughts not sorted in any particular order, my apologies if it came out in a messy way. Happy new year folks!

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 30 Dec 2021 @ 12:49pm

    An addressed issue with moderating the big sites like FaceBook and Twitter is that they are not a single community with common standards, but many overlapping communities with differing standards. The same image may or may not be offensive to the same person depending on context. I.e a breast feeding woman is acceptable within a community of mothers not so much on a dating site.

    reply to this | link to this | view in chronology ]

  • icon
    ECA (profile), 30 Dec 2021 @ 2:03pm

    And the problem is?

    Finding something to do, and BLAME anyone/everyone for something that ISNT A PROBLEM. And will probably not be enforced. CANT be enforced.
    And its a backdoor to #230. Who to blame.

    Who here is old enough to remember TV commercials about alcohol and Cig's?? AND CONDOMS.

    reply to this | link to this | view in chronology ]

  • icon
    Wyrm (profile), 31 Dec 2021 @ 8:22am

    Relevant xkcd:
    https://xkcd.com/1425/

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 2 Jan 2022 @ 9:36pm

    It's worth noting that allowing communities to discuss drugs, and allowing information about drugs to be posted, serves a powerful harm reduction role. Advice can be disseminated concerning potential hazards to avoid, how to respond to drug-related emergencies, and much more. I was an addict for years, and information from the internet had helped people I knew to know what is in a given pill via imprint lookup, check safe starting dosages for unfamiliar users to avoid overdose, check dose equivalencies when using a different drug than usual, and much more.

    It's also worth noting that these sources actually disclosed the health risks involved in the relevant drugs far better than any health class or DARE scare session - specific warnings about ketamine's effects on the bladder, how to know which cough syrup formulations should never be used for DXM abuse because of the dangerous side effects from the high doses of other ingredients, methamphetamine's neurotoxicity relative to standard amphetamine and the actual impact of sleep deprivation, cocaine's cardiotoxicity and how it increases roughly forty-fold when consumed alongside alcohol by combining in the bloodstream to form cocaethylene, and how GABAergics like alcohol barbiturates and benzodiazepines create a dangerous withdrawal state and are prone to a "kindling" effect where each withdrawal experience can make the next more severe (sometimes to the point of seizure or death), among many, many more details.

    I wouldn't want my kids using hard drugs, but if they are gonna go anywhere near them (as some kids will) then I want them to have the ability to access relevant information about those drugs, however they will pay attention to it. If I hadn't known how to react in a crisis, I have friends who would likely be dead. Having access to information about drug use saves lives, and even when the stakes aren't that high, that information increases the quality of life of those using drugs by helping them learn to prevent as many forms of harm as possible. Even if all you care about is whether addicts get clean, they have to survive that long first.

    If potential addicts know the actual effects of the drugs they might use, not portrayed as a scare tactic but as an honest assessment of what is involved and how it is dealt with, they might well change their minds. But even if they do use, they will be healthier and less prone to serious harm if they have access to relevant information about drugs.

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Make this the First Word or Last Word. No thanks. (get credits or sign in to see balance)    
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.