Content Moderation At Scale Is Impossible: Twitter Locks Accounts For Fact Checking The President

from the same-shit,-new-day dept

Another day in which we get to explain how content moderation is impossible to do well at scale. On Wednesday, Twitter (and Facebook) chose to lock the Trump campaign’s account after it aired a dangerous and misleading clip from Fox News’ “Fox & Friends” in which the President falsely claimed that children are “almost immune” from COVID-19.

People can debate whether it was appropriate or not for Twitter (and Facebook) to make those content moderation decisions, but it seems perfectly defensible. Claiming that kids are “almost immune” is insane and dangerous. However, where things get sketchy on the content moderation front is that Twitter also then ended up freezing the accounts of journalists and activists who fact checked that “Fox & Friends” nonsense:

Or in the case of Bobby Lewis from Media Matters, Twitter suspended his account for simply mocking part of the Fox & Friends clip, noting that when a host asked the President to “say something to heal the racial divisions in America” Trump couldn’t do it and could only brag about himself:

Now, tons of people are reasonably pointing out that this is ridiculous, and arguing that Twitter is “bad” at content moderation. But, again, this all comes just a few weeks (has it been a few weeks? time has no meaning) since Facebook, Twitter, and YouTube all received tremendous criticism from people for not being fast enough in pulling down another nonsense video — one that Breitbart livestreamed of “doctors” spewing utter nonsense about COVID-19 in front of the Supreme Court. Indeed, at least week’s Congressional anti-trust hearing, Rep. David Cicilline lit into Facebook for leaving that video up for five hours, allowing it to get 20 million views (meanwhile, multiple Republican representatives yelled at Zuckerberg for taking down the video).

So, if you have some politicians screaming about how any clip of disinformation about COVID-19 must be taken down, it’s no surprise that social media platforms are going to rush to take that content down — and the easiest way to do that is to take down any of the clips, even the clips that are people debunking, criticizing, or mocking the speech. Would it be nice if content moderation systems could figure out which one is which? Yes, absolutely it would. But doing so would mean taking extra time to understand context (which isn’t always so easy to understand), and in the process also allowing the videos that some say are dangerous by themselves to remain online.

In fact, if Twitter said to keep up the videos that are people fact checking or criticizing the videos, you create a new dilemma — in that those who want the dangerous nonsense to spread can, themselves, retweet the videos criticizing the content, and add their own commentary in support of the video. And then what should Twitter do?

Part of the issue here is that there are always these difficult trade-offs in making these decisions, and even if you think it’s an easy call, the reality is that it’s going to be more complex than you think.

Filed Under: , , , , , , , ,
Companies: twitter

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Content Moderation At Scale Is Impossible: Twitter Locks Accounts For Fact Checking The President”

Subscribe: RSS Leave a comment
49 Comments
Anonymous Coward says:

Fals positives are inevitable....

….and moreso the more effort being put into finding the genuine content being moderated.

I still remember the time google delisted my site which offered a fix for a common browser hijack which loads of my friends’ parents kept getting infected with. I got tired of manually wlaking them through fixing their systems every time, and so created it to streamline the process to just sending them the url. My site ran a simple script which unlocked the registry for editing, then patched out the malicious registry entries it put there preventing any antimalware software from running, then downloaded and ran such a piece of software to clean it out.

But my site got flagged as a malicious attack site and so was delisted. Because it had enough that was comparable to the very things it existed to fix. Because they were looking for those kinds of sites. Which is what we want them doing.

William Null says:

Re: Re: If it's impossible...

Fails how? You don’t end up blocking anything, so you don’t end up blocking fact checking or news reports/classical art. The best solution to bad speech is more speech. You used to believe that.

Also as a wise person said, when you tear out a man’s tongue, you are not proving him a liar, you’re only telling the world that you fear what he might say.

Stephen T. Stone (profile) says:

Re: Re: Re:

In an ideal world, your bit about more speech would be fine. But we don’t live in that fantasy world. When the bad speech can overwhelm all other speech by sheer volume alone, the best solution is “kick the bad speakers all the way out of the house”. Thinking that’s “wrong” or “unfair” because of your free speech absolutism isn’t a Twitter problem or a Facebook problem. That’s a “you” problem.

That One Guy (profile) says:

Re: Re: Re: If it's impossible...

I present to you the same challenge I have posed to others with similar ideas of just not moderating: Spend the next week on 4chan’s /b board, going through the entire list of posts at least once a day. You don’t have to open any of them but you’re not allowed to hide or skip by any, you must look at any images that are posted and read any text.

If, at the end of the week you still maintain that not blocking anything is the proper choice then people might be more willing to take the idea serious when you propose it.

William Null says:

Re: Re: Re:2 If it's impossible...

That’s the problem with people like you. I am strong believer of "if you don’t like it, you don’t have to watch/read/listen to it." Don’t try to limit what people want to say just because YOU don’t like it being discussed. If I don’t like some, say, YouTube video, I just click away and don’t worry about it. I suggest you trying out the same course of action.

That One Guy (profile) says:

Re: Re: Re:3 'If you don't like the yelling racist losers just ignore them.'

What a surprise, yet another person presented with the challenge of getting first-hand experience of what a site would look like if sites didn’t moderate like they want refuses to subject themselves to what they would foist on others.

If you don’t like the idea that platforms aimed at the general public aren’t going to be asshole friendly feel free to find or create your own platform for that kind of stuff, because hate to break it to you but most people aren’t going to want to use a platform infested with bigoted losers and/or those that like posting/discussing stuff designed to disgust/horrify people just because they think it’s funny, and telling them ‘just don’t look at/read it’ isn’t going to fly any more than telling someone eating at a fast-food place to ‘just ignore the wildly racist and sexists group two tables over loudly discussing how inferiour those other groups are’ would.

Anonymous Coward says:

Re: Re: Re: If it's impossible...

The best solution to bad speech is more speech.

What about removal of speech when it is posted in inappropriate forums or contexts? Only allowing counter speech allows the trolls,bigots, racists and political activists to hijack any forum on any topic, a problem familiar to Usenet users.

This comment has been deemed insightful by the community.
Anonymous Coward says:

Re: Re: Re:3 If it's impossible...

You do not want free speech, you want the ability to force your speech onto other people. Just because one platform will not allow your speech does not mean that your rights are being infringed, as other platforms and outlets exist. Other people rights are being violated when you insist on speaking where you are not welcome.

This comment has been flagged by the community. Click here to show it.

Anonymous Coward says:

Sooner or later we will all get cv19
Children will get it
Children dying from it is basically non existent
So why don’t you all get off of TDS
and live in the real world

Summary of Findings Reported on 7/30/20:
(Note: Data represent cumulative counts since states began reporting)

Cumulative Number of Child COVID-19 Cases*
338,982 total child COVID-19 cases reported, and children represented 8.8% (338,982/3,835,573) of all cases

Overall rate: 447 cases per 100,000 children in the population

Change in Child COVID-19 Cases, 7/16/20 – 7/30/20
97,078 new child cases reported from 7/16-7/30 (241,904 to 338,982), a 40% increase in child cases
Testing (8 states reported)
Children made up between 3%-11.3% of total state tests, and between 3.6%-18.4% of children tested were tested positive
Hospitalizations (20 states and NYC reported)

Children were 0.6%-3.7% of total reported hospitalizations, and between 0.6%-8.9% of all child COVID-19 cases resulted in hospitalization
Mortality (43 states and NYC reported)*
Children were 0%-0.8% of all COVID-19 deaths, and 20 states reported zero child deaths

In states reporting, 0%-0.3% of all child COVID-19 cases resulted in death

*Note: Data represent cumulative counts since states began reporting

This comment has been deemed insightful by the community.
Anonymous Coward says:

Re: Re:

"Children made up between 3%-11.3% of total state tests, and between 3.6%-18.4% of children tested were tested positive

Children were 0.6%-3.7% of total reported hospitalizations, and between 0.6%-8.9% of all child COVID-19 cases resulted in hospitalization"

So…not immune then and your cult leader is a liar, as usual, got it.

This comment has been deemed insightful by the community.
Stephen T. Stone (profile) says:

Re:

Sooner or later we will all get cv19

You can resign yourself to this fate all you want. I’ll be wearing a facemask and social distancing.

Children will get it

Yes, and their getting it might either kill them or fuck them up later on in life, which seems like a good enough reason to prevent children from getting it. (The fact that they could unknowingly spread it to people who have a greater chance of dying from COVID is another good reason.)

Children dying from it is basically non existent

That’s a funny way of saying “yes, children have died from COVID, but who gives a shit”.

why don’t you all get off of TDS

Pointing out that Donald Trump and his administration have either unwittingly botched or knowingly fucked up their response to a pandemic is not “derangement”. Claiming that the virus will “go away like things go away” during a spike in both case numbers and deaths, opening schools and turning them into petri dishes for the virus, caring more about the economy than about the health and well-being of the general populace because of a goddamned political campaign — all those things are deranged.

How many of the 160,000 COVID deaths could have been prevented if the pandemic response from the federal government hadn’t been a complete shitshow at the outset of the outbreak? More importantly: How many of those 160,000 deaths do you consider “acceptable losses”?

This comment has been deemed insightful by the community.
PaulT (profile) says:

Re: Re: Re:

"Yes, and their getting it might either kill them or fuck them up later on in life"

…and before they’re diagnosed they will spread it to family, friends and anyone else in the community around them who might not be lucky enough to develop mild symptoms.

That’s one problem with these people – they’re incapable of thinking beyond the individual.

"How many of the 160,000 COVID deaths could have been prevented if the pandemic response from the federal government hadn’t been a complete shitshow at the outset of the outbreak?"

There were studies that showed that most of the deaths in the first couple of months would have been prevented had action been taken just 2 weeks earlier. On the day that effective action should have been taken, despite numerous other countries having already locked down, Trump was still referring to it as a "democrat hoax". Oh, and don’t forget that he’d already fired the pandemic response team and sold off a bunch of the necessary PPE before the pandemic started.

This comment has been deemed insightful by the community.
PaulT (profile) says:

Re: Re:

"Sooner or later we will all get cv19"

I’ll take later, after the vaccines are available, the spread has been controlled and we have medical facilities that aren’t overrun by Karens who just realised they aren’t immune because they shouted at the people telling them to wear a mask, thanks.

"In states reporting, 0%-0.3% of all child COVID-19 cases resulted in death"

Cool. But, why do morons like you think that it’s only the health of the child themselves that’s at risk? Why do you morons think that death is the only outcome worth worrying about, when there’s mounting evidence of the disease causing other long term health problems?

Oh, that’s right, you’re a Trump fan, so the facts have been carefully skewed to hide the truth.

This comment has been deemed insightful by the community.
Scary Devil Monastery (profile) says:

Re: Re: Re:

"Oh, that’s right, you’re a Trump fan, so the facts have been carefully skewed to hide the truth."

Worse. by now the Trump death cult is a religious movement. They’ll go to their deaths – or, more likely, the death of family members – to call it all a hoax and close their eyes and ears to the evidence that Fearless Leader might be wrong.

If anyone had told me, ten years ago, that the world would close its borders against the US because a horde of plague rats were deliberately spreading a lethal pandemic I’d have called that person a liar.

Today I’m inclined to believe we may all want to retain those closed borders once the pandemic’s gone. A nation marching that predictably to the tune of the manifestly insane has to be held at arms length.

Stephen T. Stone (profile) says:

Re: Re: Re:

the Trump death cult is a religious movement

Kind of a chicken/egg situation, that: The religious group behind a significant part of Trump’s victory and ongoing support — Christian evangelicals — is itself a death cult, since they believe in the Rapture and want to see it happen no matter what. That Trump supporters — who don’t care if they die so long as everyone they hate dies too — have all but turned their support of him into a religious movement is…well, “discomforting” is the most charitable descriptor for that turn of events.

Scary Devil Monastery (profile) says:

Re: Re: Re:2 Re:

"Kind of a chicken/egg situation…"

Stephen, the sort of chicken imagery your argument invokes is a nightmare drawn by Giger. One laying facehugger pods to, at that.

"That Trump supporters — who don’t care if they die so long as everyone they hate dies too — have all but turned their support of him into a religious movement is…well, “discomforting” is the most charitable descriptor for that turn of events."

We’ve seen that type of personality cult before. And it’s equally upsetting each and every time.

Perhaps it’s ironic that Trump himself appears to be less the type of cloth a Hitler or Kim Jong-un is cut from and more resembles one of the faceless minions who so dearly wished they could get away with being the Big Bad.

This comment has been deemed insightful by the community.
Scary Devil Monastery (profile) says:

Re: Re:

"So why don’t you all get off of TDS
and live in the real world"

The real world, where covid is infectious? That real world where if enough people get sick in one go hospitals get overwhelmed and need to start bringing the dead out by freight hauler?

That real world where Covid has, in the US, already killed more people than the US lost soliders in World War 1?

That real world where you are currently facing the death toll of a 9/11 every two days?

That real world where the US, alone, now stands for 25% of the global death toll from Covid while having only 4% of the global population?

THAT real world?

There are over 150,000 dead americans who didn’t have to be dead, but WE are the ones suffering a derangement for your president being the man who was at the helm when every other nation managed to reduce their infection rate and death toll?

I guess that’s it. You people are officially a Death Cult now.

This comment has been deemed insightful by the community.
Anonymous Coward says:

Re: Re:

You know that for almost all of the outbreak most schools have been closed, right?

So they account for 8.8% of cases without being inside, close together, in large groups? With out engaging, en masse, in a highly risky activity?

20 states reported 0 child deaths – Holy shit, it’s like keeping them home from school has kept children safe!

Scary Devil Monastery (profile) says:

Re: Re: Re:

"20 states reported 0 child deaths – Holy shit, it’s like keeping them home from school has kept children safe!"

I’m starting to think at least some parts of the evangelicals supporting Trump so well are thinking the fact children have been kept safe is the bad part. A partial armageddon is no good at summoning The Rapture, after all.

This comment has been deemed insightful by the community.
Anonymous Coward says:

Re: Content Moderation At Scale

I guess you don’t come around here all that often (that or you don’t bother trying to understand what you read here). Obscene or lewd postings? Breast feeding mothers’ advocate posts are blocked. As are post of a lot of classic art. Violent speech? Post trying to expose tyranny, oppression and war crimes get suppressed. Content moderation is like everything difficult – it seems easy if you willfully ignore the difficult parts.

This comment has been deemed insightful by the community.
Stephen T. Stone (profile) says:

Re:

This sort of thinking reminds me of the Simpsons episode where Marge fought to take “Itchy & Scratchy” off the air, but relented when she couldn’t justify censoring that cartoon while supporting a museum displaying the famously naked statue of David.

You can’t pick and choose what counts as “obscene, lewd, or violent” without catching a lot of content that might be newsworthy/of interest to the public. That explosion in Beirut injured thousands, killed more than a hundred people, and damaged property to the point where the city damn near looks like a war zone. Do videos of that explosion count as “violent” content?

Anonymous Coward says:

Re: Content Moderation At Scale

You’d probably end up in the trap of "what’s an opinion" versus "what’s a fact". It’s not like we live in the days where knowledge comes from the # volume of World Encyclopaedia Britannica Books where you have a select few researchers hold the keys to compiling knowledge, where anyone who seeks "facts" can just reference that.

Scary Devil Monastery (profile) says:

Re: Content Moderation At Scale

"Content moderation at scale is a lot easier if you try to block obscene, lewd, or violent speech…"

As some have pointed out it’s not that easy. Consider that your criteria would have blocked about 95% of pulitzer-prize-winning news stories as well, including many of the ones which shook the world.

A picture of a naked child running away from napalm bombardment will trigger every filter in the book, but if it can not be seen and acted on it guarantees there will be more children running – or failing to run from – bombardment.

The most important news of all is the one which contains thoroughly unpleasant imagery and language which no one really wants to see.

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...