Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018)
from the sticker-moderation dept
Summary: Snapchat debuted to immediate success a decade ago, drawing in millions of users with its playful take on instant messaging that combined photos and short videos with a large selection of filters and “stickers.” Stickers are graphics that can be applied to messages, allowing users to punch up their presentations (so to speak).
Snapchat’s innovations in the messaging space proved incredibly popular, moving Snapchat from upstart to major player in a few short years. It also created more headaches for moderators as sent messages soared past millions per day to billions.
Continuing its expansion of user options, Snapchat announced its integration with Giphy, a large online repository of GIFs, in February 2018. This gave users access to Giphy’s library of images to use as stickers in messages.
But the addition of thousands of images to billions of messages quickly resulted in an unforeseen problem. In early March of 2018, Snapchat users reported a search of the GIPHY image database for the word “crime” surfaced a racist sticker, as reported by Josh Constine for TechCrunch:
“We first reported Instagram was building a GIPHY integration back in January before it launched a week later, with Snapchat adding a similar feature in February. But it wasn’t long before things went wrong. First spotted by a user in the U.K. around March 8th, the GIF included a racial slur.” — Josh Constine, TechCrunch
Both platforms immediately pulled the plug on the integration while they sorted things out with GIPHY.
Company Considerations:
- What measures can be put in place to prevent moderation problems from moving from one platform to another during cross-platform integration?
- What steps should be taken prior to launch to integrate moderation efforts between platforms?
- What can “upline” content providers do to ensure content moving from their platforms to others meets the content standards of the “downline” platforms?
Issue Considerations:
- What procedures aid in facilitating cross-platform moderation?
- Which party should have final say on moderation efforts, the content provider or the content user?
Resolution: Instagram was the first to reinstate its connection with GIPHY, promising to use more moderators to examine incoming content from the image site:
“We’ve been in close contact with GIPHY throughout this process and we’re confident that they have put measures in place to ensure that Instagram users have a good experience” an Instagram spokesperson told TechCrunch.
GIPHY offered its own apology for the racist image, blaming the slipup on a bug in its filters. Here’s what GIPHY’s spokesperson told Gizmodo:
After investigation of the incident, this sticker was available due to a bug in our content moderation filters specifically affecting GIF stickers.
We have fixed the bug and have re-moderated all of the GIF stickers in our library.
The GIPHY staff is also further reviewing every GIF sticker by hand and should be finished shortly.
Snapchat was the last to reinstate its connection to GIPHY, stating it was working directly with the site to revamp both moderation systems to ensure offensive content would be prevented from being uploaded to GIPHY and/or making the leap to connected social media services.
Originally published to the Trust & Safety Foundation website.
Filed Under: 3rd parties, bigotry, integration, racisms, stickers
Companies: giphy, snapchat
Comments on “Content Moderation Case Studies: Snapchat Disables GIPHY Integration After Racist 'Sticker' Is Discovered (2018)”
We have fixed the bug and have re-moderated all of the GIF stickers in our library.
Now this is a content moderation case study that leaves me content!
Emojis … providing the appearance of creative expression, without allowing you to say anything the company hasn’t approved the idea of before you began.
Re: Re:
Emojis are even more limited, even those which are not part of unicode. You only get what is implemented.
GIFs and stickers run in the millions. If you can’t find one that says what you want it to say, use your own bloody words or choose your image to do so.
If you like being a dick, you might get moderated by the service which is someone else’s private property. They aren’t going to be told by you what is approved for expression, either.
Re: Re:
Emojis… having nothing to do with this article, are a universal standard not controlled by any one company, and while the exact implementation can be varied by the company you’re choosing to use are limited mainly by your imagination.
Well, it doesn't exactly come as a surprise.
Since there’s always that one dick or douche around who gets their rocks off by being an asshole no platform can really afford to allow too open personalization.
This is why we can’t have nice things.
In more cynical moments I’m inclined to the belief that we have, as a society, become too permissive when people whose only hobby is to undermine others are never sanctioned beyond, at most, a "You aren’t welcome here". It’s shit like in the OP, writ large, which makes me wonder if it’ll eventually turn out that China is on to something with their "social rating scores".
Because when you look at the sheer amount of incentivized assholery among those 70-90 million americans still subscribing to Qanon and Dear Leader it sure does make you wonder if humanity is able to handle such a high ideal as "freedom". It always seems to result mainly in large amounts of people clamoring for their "rights" to be shitty unto other people.
Surely there has to be something society can do to rid itself of or push out persistent douchebags from their midst without invoking government power? How come when angry assholes migrate in herds led by a single voice liberal society just meanders around without any form of plan?
Re: Well, it doesn't exactly come as a surprise.
I don’t know what kind of society you mean that would "push out" (to where?) people you don’t like, but it isn’t a liberal one. I find that thought deeply disturbing.
Re: Re: Well, it doesn't exactly come as a surprise.
"Push out" in this context is comparable to kicking out an odious asshole from your own apartment. If you somehow think that’s not liberal, congratulations, it isn’t, it’s just people taking action removing assholes from their presence. If you find the whole concept disturbing, you are free to associate with assholes all you want.
Re: Re: Well, it doesn't exactly come as a surprise.
"I don’t know what kind of society you mean that would "push out" (to where?) people you don’t like, but it isn’t a liberal one"
What’s worse – people who are violently opposed to "liberals" being told to go elsewhere among people who they do agree with, or them remaining and disrupting the communities everyone else wants to be part of?
The fact that a society composed of only people like them would be a violent cesspool of hate and loathing should not be a factor in the rest of us deciding if we want to welcome them into the doors of our own society.
Re: Re: Well, it doesn't exactly come as a surprise.
"I don’t know what kind of society you mean that would "push out" (to where?) people you don’t like, but it isn’t a liberal one."
Go read up on Karl Popper’s "Paradox of Tolerance". If you have a liberal society and allow those who aren’t liberal to be part of it, essentially your society just drank poison. Because intolerant people will always end up with disproportionate power simply because they have the incentive to beat down and kick out the other whereas liberals are far more inclined to, sadly, tolerate those who are intolerant.
I think one Niemöller is enough.
"I find that thought deeply disturbing."
And I find it disturbing that more people aren’t having those thoughts. The US was founded by people who would not tolerate oppression and authoritarians. To the point of mounting a bloody revolution.
Yet today when an alt-right horde marches up the street flying the banners of nazi germany and the old slaver states, most liberals just stay inside and shrug. They don’t go out, meet that march halfway, and block its path, showing those benighted assholes unambiguously that their hatred is a minority opinion for which contemporary society has no place.
Because you people failed to forcefully and persistently tell racists and bigots to shove it, today 30% of the US belongs to that camp. Hitler had to make do with 12% when he took power and the last days of Trump-in-office came close to being a rinse-repeat of history with the one saving grace probably being that unlike Hitler Trump was blatantly obvious in his contempt visavi the military.
If you want to have and keep a liberal society then you are going to have to fight for it. And that struggle never ends, because there will always be those who want to use discontent, envy and bigotry to propel themselves into high office.
Jeffersson and Washington knew this. You all have forgotten it.
I invite you – cordially – to tell us all how it can be "liberal" to encourage those who would dismantle liberalism to take a seat at the table? How can it be liberal to build a society which has a core belief in base values and then invite those who reject not only that society but every value it was founded on?
Normally I have to read some passage from the text I’m about to quote to republicans who, despite always hollering about it, appear to never have read it;
The opposite also follows. Those who are incapable of aligning with this very simple sentence are in the wrong country.
Those people whining about "censorship" because they’re not allowed to use the N-word in open forums? The very fact that they feel compelled to use that word means that they reject the most basic rules of the society they live in.
And if you stand to defend them or their "right" to be assholes in the mistaken assumption that this is the liberal thing to do…then you’ve abandoned liberal ideology because what you really are doing is to defend their privilege of being assholes to someone else.
Malcolm X was not a decent person but he did have a few things correct and he was dead right when he made the claim that he feared the white liberal more than the nakedly aggressive conservative. For in the end the american liberal always ends up defending the right of the conservative to undermine the worth and value of the other – in the sacred name of "personal opinion" and the mistaken belief that every opinion is worthy of protection.
It’s not, and both your founding fathers and any nation with a national charter ensconcing free speech knew this.
Oh, the horror, there’s a bazillion pictures and one of them has a naughty word on it. Shut! Down! Everything!
Re: Re:
Yeah, pretty much. If you allow people to post things via a 3rd party and the 3rd party’s promised filters aren’t working as required, it’s better to remove that feature than put up with the PR fallout from people blaming you directly.
Talking of overreactions, we can point to your comment, though – "everything" was not shut down, only the feature that used a 3rd party as a bonus feature whose promised filtering wasn’t working properly. People using Snapchat without also using giphy would not have noticed any change.