Content Moderation Case Study: Roblox Moderators Combat In-Game Reenactments Of Mass Shootings (2021)
from the modern-moderation dept
Online game platform Roblox has gone from a niche offering to a cultural phenomenon over its 15 years of existence. Rivalling Minecraft in its ability to attract young users, Roblox is played by over half of American children and has a user base of 164 million active users.
Roblox also gives players access to a robust set of creation tools, allowing users to create and craft their own experiences, as well as enjoy those created by others.
A surge in users during the COVID-19 pandemic created problems Roblox’s automated moderation systems — as well as its human moderators — are still attempting to solve. Roblox employs 1,600 human moderators who not only handle content flowing through in-game chat features but content created and shared with other users utilizing Roblox’s creation tools.
Users embraced the creation tools, some in healthier ways than others. If it happened in the real world, someone will try to approximate it online. Users have used a kid-focused game to create virtual red light districts where players can gather to engage in simulated sex with other players — an activity that tends to avoid moderation by utilizing out-of-game chat platforms like Discord to provide direct links to this content.
Perhaps more disturbingly, players are recreating mass shootings — many of them containing a racial element — inside the game, and inviting players to step into the shoes of mass murderers. Anti-Defamation League researcher Daniel Kelley was easily able to find recreations of the Christchurch Mosque shooting that occurred in New Zealand in 2019.
While Roblox proactively polices the platform for “terrorist content,” the continual resurfacing of content like this remains a problem without an immediate solution. As Russell Brandom of The Verge points out, 40 million daily users generate more content than can be manually-reviewed by human moderators. And the use of a keyword blocklist would result in users being unable to discuss (or recreate) the New Zealand town.
- How does catering to a younger user base affect moderation efforts?
- What steps can be taken to limit access to or creation of content when users utilize communication channels the company cannot directly monitor?
- What measures can be put in place to limit unintentional interaction with potentially harmful content by younger users? What tools can be used to curate content to provide “safer” areas for younger users to explore and interact with?
- How should companies respond to users who wish to discuss or otherwise interact with each other with content that involves newsworthy, but violent, events?
- How much can a more robust reporting process ease the load on human and AI moderation?
- Can direct monitoring of users and their interactions create additional legal risks when most users are minors? How can companies whose user bases are mostly children address potential legal risks while still giving users freedom to create and communicate on the platform?
Roblox updated its Community Standards to let users know this sort of content was prohibited. It also said it would engage in “proactive detection” that would put human eyes on content related to terms like this, allowing geographic references but not depictions of the mosque shooting.
Originally posted to the Trust and Safety Foundation website.
Filed Under: children, content moderation, mass shootings
Comments on “Content Moderation Case Study: Roblox Moderators Combat In-Game Reenactments Of Mass Shootings (2021)”
Perhaps they should release an adult version of the system so that players interested in more mature topics don’t have to sneak in the kid version?
Or maybe just dont put those topics in a kids game, you fucking inbred retard.
Some of their moderation tasks are difficult, to say the least. However, these are compounded by other parts of their business model and behavior. Roblox is problematic all the way down.
Having watched the recent exposé on Roblox’s moderation and user protection policies, on top of how they run their market for cosmetics… yeah, it’s pretty clear that Roblox does not have their users’ best interests in mind.
Re: REALITY SUCKS.
But its reality.
Truth hurts? only if you love the lies and cant deal with REAL.
Now a few restrictions MIGHT be a nice thing. Like a 18+ section. But how are you going to monitor/sort that? How can you really tell who is who and the age? There was a suggestion awhile back, about listing AGE of those signed in and restricting Access by AGE.
Building constructions that mimic real life ISNT BAD. It could show how things could have been better, if/maybe/choices were made. Simulating things Isnt bad. It can show kids WHAT they could have/may have been able to do to SAVE themselves.
Really It has been played mostly in America I am fond of this game I play this game in my Spare.left time I ready dissertation with help of <a href="https://www.reportwritinghelp.com/“>types of report writing</a> service
Not related, but automatic moderation isn’t much better. There’s stories making the rounds of Reddit about how Nintendo’s automatic detection software for Super Smash Bros Ultimate online is issuing warnings and bans over normal gameplay because its algorithms are incorrectly detecting "unsporting" gameplay:
Users are complaining about receiving bans as a result:
"I got banned for getting a Blade Beam spamming Cloud. I caught the rhythm and just stood there parrying every beam.
I got the message that it’s unsporting behavior to just stand still and do nothing."
Что такое влог и блог В чём их отлич
Sensitive touch rasprekrasnoy girls will flow through your body, dipping in depth boundless the ocean pleasure. In the quiet slip, donating your skin kisses, prelestress envelops the warmth of one's body. You will be surprised at, which sea bliss today it is possible to feel fromnude massage in Midtown.
Systematically visiting the four hands massage for clients, you guarantee himself excellent sexual relaxation.
And while, french massage and not violates practically any prohibitions, for the reason it's not about sexual contact.
In school sensual massage women will hold erotic 4hands massage. Similar swedish massage, as in principle, and relaxation, influences on some area human body, this give a chance male gain strength.
<a href=https://sites.google.com/view/eroticoutcallmassage>Список более 40 качественных блогов которые принимают</a>
A lot of what is in here is very, very wrong indeed. Roblox only employs about 30 moderators total, not 1600, as I have personally confirmed. A lot of what’s reported or submitted for use on roblox is run through a third-party filter. Roblox staff never oversee anything, they just sit back and let the bots take care of things. This is why all these ‘mosque shooting’ and related terrorism-centered games are still up on the site; staff simply do not care.