Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Study: Profanity Filter Causes Problems At Paleontology Conference (October 2020)

from the f-that dept

Summary: With the COVID pandemic still in full force, the Society of Vertebrate Paleontology moved its annual meeting online. The event was due to run for an entire week, but early issues caused attendees and moderators to question the contents of the pre-packaged content filter provided by Convey Services, which operated the virtual meeting software.

The effort to ensure followup Q&A sessions would be free of profanity and other disruptiveness went awry when terms commonly used by paleontologists got caught in the software’s filter. While words like “pubic,” “bone,” or “hell” might be appropriately blocked elsewhere, the blocklisting of these words disrupted the conference the software was supposed to keep from being disrupted.

?I would hope that actual swears or slurs would be censored, since paleontology is not a field that’s immune to racist/sexist jerks,? noted Brigid Christison, a masters’ student in biology at Carleton University, in an email.

[…]

?Words like ?bone,? ?pubic,? and ?stream? are frankly ridiculous to ban in a field where we regularly find pubic bones in streams,? Christison said.

More problematically, attendees noted the software blocked “Wang” but left “Johnson” intact, despite both being sexual slang. This appeared to indicate some bias on the part of Convey’s blocklist creators — a bias that effectively erased a surname belonging to more than 90 million Chinese citizens.

Decisions to be made by Convey:

  • Are generic ban lists effective enough in most situations to justify less content-specific alterations during more specialized use?
  • Given the global reach of the internet, is it still acceptable to block common surnames that can also be deployed as sexual slang?
  • Does providing separate keyword sets for different uses reduce the profitability of software sales and licenses?

Questions and policy implications to consider:

  • Do word ban lists raise many issues in more general use or are problems noticed more often when the software is deployed by entities that deal with specialized subject matter?
  • Does allowing licensees to alter keywords as needed eliminate some of the problems caused by non-specific ban lists?
  • Is shifting the cost of moderation to customers a solid business decision?

Resolution: The Society was able to alter the ban list to better fit the subject matter once the problem was noticed. It continued to edit the keywords provided by Convey, reducing the probability of overblocking as the week rolled on. Because the alterations could be made on the client side, disruption was minimal, if inadvertently comical.

Originally posted to the Trust & Safety Foundation website.

Filed Under: , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation Case Study: Profanity Filter Causes Problems At Paleontology Conference (October 2020)”

Subscribe: RSS Leave a comment
6 Comments
Anonymous Coward says:

This idea that blocking words with certain meanings has been practiced by the Chinese authorities for a good while now.

Know what? They aren’t very successful at it either since language is very malleable and flexible.

Words and phrases that have no relation to the banned words pop up to replace the banned ones. Everyone knows what those phrases and words are meaning but they are not banned until the authorities catch on. Then new ones come out as substitutes.

It’s just a new wack-a-mole. Unending in trying to prevent people from expressing what they mean. It does far more damage than it does good results as demonstrated by the article. Collateral damage is unending and there always seems to be a work around.

Ben (profile) says:

Re: It makes you wonder

To take your comment seriously for a moment, any time you get a place where students and lecturers and researchers (of any discipline, let alone palaeontology) go to hang out away from home, there will be the inevitable consumption of alcohol and subsequent high jinks. Sometimes even in conference sessions.
As noted by Brigid Christison in the article, palaeontology is not immune from racism, sexism, ageism, or any other -ism that can be a source of insult, offence or downright bullying or assault. These need not even be associated with alcohol.
Therefore, it is not unreasonable for the organisers to be cautious about dealing with these kinds of things before the fact.
That they got it a bit wrong is sad, but a consequence of believing that the computer can be the arbiter without a lot of contextual assistance.

TaboToka (profile) says:

Um, wot?

“I would hope that actual swears or slurs would be censored, since paleontology is not a field that’s immune to racist/sexist jerks,” noted Brigid Christison, a masters’ student in biology at Carleton University

How about this:

1) All attendees are presented with a Code of Conduct (CoC) beforehand that they must agree to as a condition of participation.

2) Attendees can flag any posts/comments they feel are inappropriate.

3) A moderator group reviews the flags and determines if the content violates the CoC. If so, they then delete the comment and either warn the violator or ban them outright based upon the severity.

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...
Older Stuff
15:43 Content Moderation Case Study: Facebook Struggles To Correctly Moderate The Word 'Hoe' (2021) (21)
15:32 Content Moderation Case Study: Linkedin Blocks Access To Journalist Profiles In China (2021) (1)
16:12 Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018) (11)
15:30 Content Moderation Case Study: Tumblr's Approach To Adult Content (2013) (5)
15:41 Content Moderation Case Study: Twitter's Self-Deleting Tweets Feature Creates New Moderation Problems (2)
15:47 Content Moderation Case Studies: Coca Cola Realizes Custom Bottle Labels Involve Moderation Issues (2021) (14)
15:28 Content Moderation Case Study: Bing Search Results Erases Images Of 'Tank Man' On Anniversary Of Tiananmen Square Crackdown (2021) (33)
15:32 Content Moderation Case Study: Twitter Removes 'Verified' Badge In Response To Policy Violations (2017) (8)
15:36 Content Moderation Case Study: Spam "Hacks" in Among Us (2020) (4)
15:37 Content Moderation Case Study: YouTube Deals With Disturbing Content Disguised As Videos For Kids (2017) (11)
15:48 Content Moderation Case Study: Twitter Temporarily Locks Account Of Indian Technology Minister For Copyright Violations (2021) (8)
15:45 Content Moderation Case Study: Spotify Comes Under Fire For Hosting Joe Rogan's Podcast (2020) (64)
15:48 Content Moderation Case Study: Twitter Experiences Problems Moderating Audio Tweets (2020) (6)
15:48 Content Moderation Case Study: Dealing With 'Cheap Fake' Modified Political Videos (2020) (9)
15:35 Content Moderation Case Study: Facebook Removes Image Of Two Men Kissing (2011) (13)
15:23 Content Moderation Case Study: Instagram Takes Down Instagram Account Of Book About Instagram (2020) (90)
15:49 Content Moderation Case Study: YouTube Relocates Video Accused Of Inflated Views (2014) (2)
15:34 Content Moderation Case Study: Pretty Much Every Platform Overreacts To Content Removal Stimuli (2015) (23)
16:03 Content Moderation Case Study: Roblox Tries To Deal With Adult Content On A Platform Used By Many Kids (2020) (0)
15:43 Content Moderation Case Study: Twitter Suspends Users Who Tweet The Word 'Memphis' (2021) (10)
15:35 Content Moderation Case Study: Time Warner Cable Doesn't Want Anyone To See Critical Parody (2013) (14)
15:38 Content Moderation Case Studies: Twitter Clarifies Hacked Material Policy After Hunter Biden Controversy (2020) (9)
15:42 Content Moderation Case Study: Kik Tries To Get Abuse Under Control (2017) (1)
15:31 Content Moderation Case Study: Newsletter Platform Substack Lets Users Make Most Of The Moderation Calls (2020) (8)
15:40 Content Moderation Case Study: Knitting Community Ravelry Bans All Talk Supporting President Trump (2019) (29)
15:50 Content Moderation Case Study: YouTube's New Policy On Nazi Content Results In Removal Of Historical And Education Videos (2019) (5)
15:36 Content Moderation Case Study: Google Removes Popular App That Removed Chinese Apps From Users' Phones (2020) (28)
15:42 Content Moderation Case Studies: How To Moderate World Leaders Justifying Violence (2020) (5)
15:47 Content Moderation Case Study: Apple Blocks WordPress Updates In Dispute Over Non-Existent In-app Purchase (2020) (18)
15:47 Content Moderation Case Study: Google Refuses To Honor Questionable Requests For Removal Of 'Defamatory' Content (2019) (25)
More arrow