Moderator Mayhem: A Mobile Game To See How Well YOU Can Handle Content Moderation
from the moderating-the-mayhem dept
Play Moderator Mayhem in your browser »
Today, we’re excited to announce the launch of our newest game in partnership with Engine. Moderator Mayhem is a mobile, browser-based game that lets you see how good a job you would do as a front line content moderator for a growing technology company that hosts user-generated content (in the game, it’s a “review” website that lets you review anything, not just businesses).
So much of the discussion lately around content moderation and trust & safety doesn’t come from a place of any kind of actual experience with moderating content and understanding the competing pressures, both internal and external, towards allowing free speech, protecting user safety, and complying with various laws and other factors.
A friend of mine in the trust & safety world once suggested that these conversations would be a lot more useful if everyone had to spend a few days moderating an actual community, and could learn how content moderation is not about “suppressing viewpoints,” but almost always about understanding really complex scenarios in which you have to make decisions in a very limited period of time, with limited information, and where there may not be any “right” answer.
Enter: Moderator Mayhem. It’s a browser-based mobile game, and you will learn that you have to make your moderation decisions by swiping left (take down) or right (keep up), and try to align content with the policies of the company (a fictional review site called TrustHive). Of course, users of your site may not like your decisions. They might appeal the decisions, and you might realize you missed some important context (or not!). Your manager might disagree with your decisions, and might not think you’re suited for the job. Your CEO might have his own views on how your moderation is going. So might the media.
And, of course, you don’t have much time to make your decisions, as the stack of flagged content you’re expected to review will keep growing and growing. Often, it would be helpful for you to investigate the more detailed context, and you can do that within the game, but it takes precious time. Sometimes you’ll learn something useful… but sometimes you won’t.
Also, there’s often no “correct” answer, so the game won’t tell you if you got something “right” or “wrong” because often there is no right or wrong. Your manager might tell you they disagree with your decision, or might not. But at the end of each session you’ll get a general update on how your manager thinks you’re doing in accurately applying company policy, and you’ll get a sense of your job security. Mess up too often and you may be looking for a new job. Apply the policy well enough, and maybe you can get promoted.
Of course, your manager is not the only stakeholder. You’ll also find out what the public thinks of your platform. Are you supportive of free speech, or too oppressive in your moderation? Are you allowing too much harassment and therefore not considered safe? One thing about the public is that they’re not shy about letting you know how they feel.
This is the second game we (Copia Gaming and our partner Randy Lubin of Leveraged Play) have created with Engine, following last year’s Startup Trail, which allowed players to explore some of the policy dilemmas and tradeoffs startups have to face. Moderator Mayhem is quite a bit different: there’s no time for policy considerations when you’re on the front lines and your queue keeps growing… so can you manage the mayhem?
(Note: while this game is designed for mobile browsers, it will also work in desktop browsers, though it will obviously have the form factor of a more mobile device).
Play Moderator Mayhem in your browser »
Filed Under: content moderation, moderator mayhem
Comments on “Moderator Mayhem: A Mobile Game To See How Well YOU Can Handle Content Moderation”
This comment has been flagged by the community. Click here to show it.
I’ll only play this if it gives me the vindication of suppressing all of Stephen T. Stone’s and Koby’s comments.
Re:
That will be included in the “in-app-purchase”. (How did that, in my opinion ever get out of a basement.)
This comment has been flagged by the community. Click here to show it.
So Realistic
Why play the game? Do the real thing! Techdirt allows you to suppress comments, as long as your political alignment is “correct”. Just press that report button below, and you too can obtain those sweet, sweet virtue signal points. Try it right now!
Re:
I mean, if you didn’t have a hard-earned reputation as a troll and/or flaming nutjob, maybe you wouldn’t be flagged for trolling.
…Nah, it must be those evil [insert word here].
Re:
I can sadly read your hateful, treasonous comments.
Which easily disproves everything you just said.
And Mike isn’t the sort of person to sue you to stop, so…
Perhaps you might want to actually have some fucking self-awareness before you get sentenced for treason.
54% accuracy on one of the rounds… I’ll check the job markets for “human coin-flip”
Interesting game.
It was an interesting experiment to play, and well worth the development time I think, thanks for that work to all involved.
Too bad that many people who really need to understand the process and how policies and rules actually work will never go through a round or two to get a glimpse of how tricky it actually is.
And BTW, I did much better than expected first round, so maybe I have found my new calling and can retire from my paying job?
Nah.
But still:
Game Over – Promoted.
Total Reviews: 98
Safety: High
Speech: High
I was definitely too slow though, I thought for sure I was going to get fired for that. And likely would have gotten fired in a real life version…
85% on the “err on the side of caution, and assume the worst absent the coontext”..
It’s ALMOST like moderating an IRC channel…
This comment has been flagged by the community. Click here to show it.
I haven’t played it, but given that it’s a game, the scoring is based on the views of the designers, not of a real public. If the designers don’t believe that large generic speech do suppress viewpoints, neither the gameplay nor scoring will be reflective of that, even when viewpoint suppression does happen in the real world.
This may be more of an exercise in brainwashing players to the point of view of the designers rather than a way of teaching players what moderation is really like.
Re:
Your cover judging aside, I think that what matters most by far is whether the moderation scenarios presented to the game are accurate to the common scenarios that real moderators have to deal with. The outcome along with the scoring are secondary, considering that many moderation scenarios don’t have simple right-or-wrong answers.
This is partially the point. The in-game company has a specific set of policies and definitions, but other companies have different policies. Meanwhile, it’s impossible to know what the public really thinks about these issues without seeing them play out in real life. You won’t get the public’s view from a game this simple, and the most reliable way to find out would be to run a real social media service. Your next best bet would be to read internal reports about moderation from social media companies. (Or you can work as a moderator if you have the stomach for it.)
It’s brainwashing the same way reading an opinion piece is brainwashing. If you don’t believe the arguments you’re getting at face value, you read another source.
But more importantly, you can ignore the game’s accuracy scoring and still get a sense of whether you, following your own moderation standards and no one else’s, can avoid “suppressing viewpoints”, again according to your own standards. How well can you avoid second-guessing your standards as the clock runs down? That’s a question that carries over no matter which social media company you’d work at, because you bring your standards with you.
Re:
Very little of the content in the game even has anything to do with political viewpoints because, contrary to what you might believe, political speech is only a small slice of the content that moderators deal with.
Re:
Oh look, Hyman Rosen is commenting again.
Please note that the owner of the site (Mike) has explicitly told him to stop being a homophobic jackass. More than once.
This comment has been flagged by the community. Click here to show it.
Re: Re:
I think you mean “transphobic”. But it’s not surprising that you would get those terms confused, since woke gender ideologues have successfully hijacked the LGB movement by attaching the delusional T.
Re: Re: Re:
Whatever, Hyman.
Stop being a transphobic/homophobic/anti-humanity jackass. That’s all Mike wants you to fucking do.
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:2
Unfortunately for Mike, in the real world, men cannot become women, gods do not exist, and Black people commit crimes in numbers disproportionately large to their share of the population. Insisting that people affirm otherwise cannot change reality. Yelling at people for pointing out reality cannot change reality. The entire world agreeing to affirm lies cannot change reality. Reality does not care what you think about it. It simply sits there being itself. If you insist on believing lies, sooner or later you will step out of a window thinking you can fly, and will plunge to the ground and die. Or, as the case may be, mutilate children in the name of an ideology that is so obviously false only a deluded fool could believe it.
Re: Re: Re:3
Whoosh
Moderate a Forum of like minded people can be easy, Twitter FkNo
As I said, like minded people in a forum/community/invite is like a busy club at times.
Something like Twitter is like David Lynch meets Disney and your nosey grandmother and then starts looking like that LSD-25 trip you sorta recall when a Twitter post flags moderate. <– From lurking. There is no way I would take the job.
Would AOC come out and play? (Answer in 2 words and a giggle.)
"Moderation"
Nothing about the word “moderation” requires that you mark up or remove content. That is specific to how the social media platforms define that role.
Plenty of users “moderate” content when they respond to and challenge other misinformed users. In many cases they are better equipped to do that than the platforms.
I do this for real, I don’t need a game.
Might be interesting to explore it and see how the baked-in design assumptions compare to my own experiences, though.
I didn’t like the game. It was impossible to do well.
Re:
Congrats! That was the point!
Moderation IS hard to do well!
Re:
This needs more funny votes.
Interesting game
If all the moderated content could be trained into an LLM then GPT-4 could be used to automatically moderate content on social media. Would reduce the amount of burnout amongst moderators.
Re:
GPT, the platform that routinely provides references to books and articles that don’t exis? That GPT?
You’re employed by the company you’re moderating for? 1/5, sheer fantasy