Daily Deal: eduCBA Tech Training Bundle
Supreme Court Adds Yet Another Smackdown To Patent Court, Says It Misinterpreted Patent Law In Apple/Samsung Case

This Is A Really Bad Idea: Facebook, Twitter, YouTube & Microsoft Agree To Block 'Terrorist' Content

Free Speech

from the how's-that-going-to-work dept

Tue, Dec 6th 2016 10:45am -

Under increasing pressure from overreacting and fearful bureaucrats, it seems that the big social media companies — Facebook, Twitter, YouTube and Microsoft — have all agreed to block “terrorist” content and will agree to share hashed versions of it among the other companies so something blocked on one site can easily be blocked across them all.

Facebook, Microsoft, Twitter and YouTube are coming together to help curb the spread of terrorist content online. There is no place for content that promotes terrorism on our hosted consumer services. When alerted, we take swift action against this kind of content in accordance with our respective policies.

Starting today, we commit to the creation of a shared industry database of ?hashes? ? unique digital ?fingerprints? ? for violent terrorist imagery or terrorist recruitment videos or images that we have removed from our services. By sharing this information with each other, we may use the shared hashes to help identify potential terrorist content on our respective hosted consumer platforms. We hope this collaboration will lead to greater efficiency as we continue to enforce our policies to help curb the pressing global issue of terrorist content online.

Our companies will begin sharing hashes of the most extreme and egregious terrorist images and videos we have removed from our services ? content most likely to violate all of our respective companies? content policies. Participating companies can add hashes of terrorist images or videos that are identified on one of our platforms to the database. Other participating companies can then use those hashes to identify such content on their services, review against their respective policies and definitions, and remove matching content as appropriate.

This sounds as though it’s modeled on similar arrangements around child pornography. Except that there are some major differences between child pornography and “terrorist content.” The first is that child porn is de facto illegal. “Terrorist content” is quite frequently perfectly legal. It’s also much more of a judgment call. And based on this setup, allowing one platform partner to designate certain content as “bad” will almost certainly result in false positive designations that will flow across multiple platforms. That’s dangerous.

As we’ve discussed in the past, when you tell platforms to block “terrorist” content, it will frequently lead to mistakes, like blocking humanitarians documenting war atrocities. That kind of information is not just valuable, but necessary in understanding what’s happening.

Also, all of this presumes, that the best way to deal with so-called “terrorist content” online is to hide it and pretend it doesn’t exist. That’s not always the case. As we’ve noted, counterspeech — including mocking silly terrorist claims — is often much more effective than outright blocking. Blocking the content not only leads to a slippery slope — and open questions on choosing what content stays and what content goes — but also presumes that the block is the most effective way to stop the bad behavior associated with terrorists. But it leaves out that blocking such content often only makes those posting it feel like they’re on the right path, and that they’re saying something “so true” that it needs to be blocked. It’s not a path towards stopping terrorism or the spread of terrorist ideology — it just gets those engaged to dig in deeper on their views.

On top of that, terrorist information posted to social media is often a great source of intelligence for law enforcement. Even the FBI director has said it’s silly to chase terrorists off of social media, because it makes them harder to track. So what good is this really doing?

Yes, platforms have every right to decide how they want to handle the content submitted to them. And, yes, this almost certainly comes about as a result of increasing pressure (especially out of the EU) to “do something” about “terrorist content” on these platforms, but as we’ve seen in the past, appeasing such whining bureaucrats almost never settles them down. As we recently noted, after these same four companies signed an agreement earlier this year to “curb hate speech” on their platforms, it still didn’t stop government officials in Europe from threatening further legal consequences, including criminal charges, when the agreed upon blocks failed to magically make all “hate speech” disappear.

So, yes, the platforms may have felt backed into a corner, but they’re only going to get their backs pushed further and further into that corner — and the collateral damage it creates may be even more massive.

Filed Under: , ,
Companies: facebook, microsoft, twitter, youtube

59 CommentsLeave a Comment
If you liked this post, you may also be interested in...
Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “This Is A Really Bad Idea: Facebook, Twitter, YouTube & Microsoft Agree To Block 'Terrorist' Content”

Subscribe: RSS Leave a comment
59 Comments
AricTheRed says:

Re: Re: Well the silver lining is...

timmaguire42,

I was attempting to point out, in a ludicrous way, how silly the idea is, that someone in a position to decide what might be “Terrorist Content” could reasonably argue that the President of a internationally recognized sovereign nation’s speech should be blocked, and if anyone repeats that speech, it need to be blocked everywhere.

Do you think it could be possible that someone that works at Twitter might dislike Trump’s speech so much that the “filters” might block his communication, even “inadvertently”?

Anonymous Coward says:

Re: Well the silver lining is...

That seems very unlikely. Trump’s twitter feed is primarily original content, in the sense that no one has previously posted, character-for-character, the things that Trump’s feed posts. This is about blocking specific instances of content so that it cannot be easily cross-posted to multiple platforms. Original content will still go up. At most, assuming an effort was made to suppress Trump’s feed (or anyone else’s), it would mean that once delisted from Twitter, it couldn’t be reposted as-is on the other platforms. For the text part, that was almost guaranteed anyway, as most Trump posts do not fit well within Twitter’s pathetically low character limit. Cross-posts would likely be reworded or reassembled to benefit from the longer length limits on other platforms. If the sites truly are blacklisting based on hashes, simply restructuring the post (or assembling multiple tweets into one post) would likely bypass the hash-based block.

The only reason blocking child pornography by hash works at all is that the redistributors do not bother to make the minor edits necessary to change the hash.

DannyB (profile) says:

Re: Re: Well the silver lining is...

Unlike older bogeymen words such as ‘communism’, the word ‘terrorism’ is the magical incantation which can be re-defined to mean whatever the government wants it to mean at the moment. A single word with both ‘flexibility’ and ‘expandability’ built right in. So upgrade your ‘communist’ to ‘terrorist’ today!

That One Guy (profile) says:

'Bad' speech, you know it when you see it

Seems no-one in charge of the companies is very good at history. By giving in again they’re at best delaying even more stringent demands and blame when they don’t magically make all the ‘bad stuff’ disappear.

After all, who could possibly object to removal of ‘terrorist material’?

… or ‘hate speech’?

… or ‘material determined to be detrimental to social health’?

Fighting back may have been costly, but caving in just means that they’re going to be increasingly faced with demands to do ‘just a little more’ or remove ‘just a few more things’.

Anonymous Coward says:

Re: Not surprised.

TD is a pretty decent site even though I yak about them a lot, but it is also hardly the worst site either.

You need to read more than just here, just be ready to read between the lines and you will be able to determine the spin pretty quickly in most cases.

You will NEVER get an unbiased news source from even the most honest media sources out there. Bias is just a fact of life and a whole lotta people need to get over it.

stine (profile) says:

While you may be right, you're not.

“As we’ve noted, counterspeech — including mocking silly terrorist claims — is often much more effective than outright blocking.”

I don’t think mocking is the correct answer to what you consider hate speech unless you’d like to start working for Charlie Hedbo in Paris–I hear they may still have a few openings.

While I believe that ignoring it is the worst that you can do, and blocking it a close second, and I’m not sure what the best action would be.

freedomfan (profile) says:

The answer to arbitrarily designated "bad" or "dangerous" speech is...

… better speech backed up by clearer facts and more compelling arguments. Unless that answer is coming from government folks whose instinct seems typically reactionary. In that latter case, the answer is censorship.

Among the scary aspects of this sort of issue is that neither of the traditional sides (in terms of politics) is consistently against censorship as a policy, enacted either via statute or via the threat of their weighty boots on the throats of those who don’t play along.

Anonymous Coward says:

Just a thought, but suppose Facebook block, like they did, that famous photo of Kim Phuc, and that hash gets shared, just how to they remove the block, when they can be given the same hash by someone they shared it with. Is it oops, we made a mistake and cannot correct it? Does the file disappear from the Internet because everyone one is blocking it?

Anonymous Coward says:

Re: Reverting blocks

Assuming that they want to be able to fix mistakes, it’s not a hard problem. Every record would contain an originator identifier telling which company blacklisted it. When the originator wants to undo that entry, they send out a cancellation for that record. Additionally, they never add to their blacklist any entry which has an originator ID that says it came from them. Such entries either are redundant if not yet revoked, or are stale if already revoked. In a fully cooperative environment, where you can assume nobody impersonates anyone else, the identifier could be simply the company name. If you want authenticity, the combination of originator and blacklist entry can be signed to prove who created it, so that nobody else can falsely attribute a blacklist to that entity.

Anonymous Coward says:

Re: Re: Reverting blocks

Assuming that they want to be able to fix mistakes,

That should be assuming that all players want to fix mistakes, an that they do not generate a circulate multiple variant hashes of the same nominal file. Because real hashes are sensitive to single bit changes, the ‘hashes’ to identify content will have to be somewhat looser than what is is usually meant by a hash, a bit change sensitive check of file contents.

Also, because hashes are being shared, and the recipient of a hash will not have the file to check whether the file should be blocked, it will be all too easy for governments to push hashes of contents that they object to.

John Mayor says:

THE FOUR PILLARS OF SECURITY

When governments begin discussing “BETTER NET SECURITY”, “FOUR POINTS” are usually missed! And these are:… evil people want ACCESS, and evil people want PRIVACY; and non-evil people want ACCESS, and non-evil people want PRIVACY! Conclusion:… SECURITY ON THE NET, IS NOT SIMPLY ABOUT PRIVACY VERSUS ACCESS!… I-T-‘-S A-B-O-U-T F-O-U-R S-E-C-U-R-I-T-Y E-L-E-M-E-N-T-S (AND WHICH INCLUDES, HIGH-LEVEL AND LOW-LEVEL LANGUAGE ICT ACCESS/ CRYPTANALYSIS/ WHITE HAT HACKING FOR THE GOOD GUYS, AND AGAINST THE BAD GUYS!… AND, HIGH-LEVEL AND LOW-LEVEL LANGUAGE ICT ACCESS/ CRYPTOGRAPHY/ STEGANOGRAPHIC ENCRYPTION FOR THE GOOD GUYS, AND AGAINST THE BAD GUYS!)!
.
NON-EVIL PEOPLE NEED BOTH ACCESS AND PRIVACY!… AND EVIL PEOPLE SHOULDN’T HAVE EITHER! BUT, IF YOU DENY THE WRONG PEOPLE ACCESS AND PRIVACY (I.E., NON-EVIL PEOPLE!), AND ALLOW THE WRONG PEOPLE ACCESS AND PRIVACY (I.E., EVIL PEOPLE!), YOU HARM SECURITY!… AND, IN THE CASE OF NON-EVIL PEOPLE, ACCESS AND PRIVACY DENIALS WILL ADVERSELY IMPACT ON MANY OTHER DIGITAL… AND HUMAN!… RIGHTS! AND IF YOU DON’T KNOW WHICH IS WHICH (I.E., WHO IS EVIL, AND WHO IS NOT!)… WELL… YOU D*MN WELL SHOULD FIGURE IT OUT!
.
SIMPLY STATED!… LIFE IS NOT GUARANTEED TO BE EASY!… OR WITHOUT THORNS!
.
Please!… no emails!

Not an Electronic Rodent (profile) says:

Re: THE FOUR PILLARS OF SECURITY

I’ll skip right past the impossibility of the kind of split you seem to be suggesting considering how often that particular deceased equine has been soundly beaten and go straight to:
Who gets to define “evil” and “non-evil”? The government? The media? People with a capital letter fetish? I’m guessing they’ll all suck at it equally.

John Mayor says:

Re: Re: THE FOUR PILLARS OF SECURITY

Impossibility? A dead horse?
.
Au contraire!… I-T-‘-S T-H-E O-N-L-Y P-O-S-S-I-B-L-E S-O-L-U-T-I-O-N TO OUR SECURITY DILEMMA! And, inasmuch, as pitting privacy against access, M-I-S-S-E-S H-A-L-F T-H-E E-Q-U-A-T-I-O-N!
.
And as for who decides!… it’s more of a WHAT!… and that WHAT, is a thing called the Uncertainty Principle in Quantum Mechanics!
.
To make a M-U-C-H L-O-N-G-E-R S-T-O-R-Y shorter… it just so happens, that “good thoughts” and “good intentions (at the Quantum level!)”, possess a distinct Quantum flux (coherent Quantum flux!) that/ which can be measured (e.g., through Superconducting Quantum Interference Devices!… nano sized!)!… and likewise– though conversely!– “bad thoughts” and “bad intentions (at the Quantum level!)”, possess a distinct Quantum flux (incoherent Quantum flux!) that/ which can be measured (e.g., through S.Q.U.I.Ds!… nano sized!)!
.
To sum up… we’re coming into a level of ICT device sensitivity, whereby the Quantum Dynamics of your very thoughts W-I-L-L G-I-V-E Y-O-U A-W-A-Y! In other words, what you once thought was hidden from public view, will be manifest for all to see (literally, and metaphorically speaking!)! And so… it won’t be a matter of someone making a judgement call as to the veracity of what was false or what was true re your thinking, it will be a matter of knowing C-O-M-P-L-E-T-E-L-Y (WITH 100% ACCURACY!) the nature of your intentions! And!… all due to the power inhere within the Uncertainty principle within Quantum Mechanics!
.
The only “catch” is… Who will have control over this technoma?… Who will ensure its use for GOOD?
.
Please!… no emails!

ItsOnlyTheBeginning says:

Next up match no fly, ass bad credit to this list for a citizens score

Seems many governments around the world have a problem with speech. Perhaps if they better represented their citizens… just saying.

China goes a step further to assign a score to every citizen. Perhaps these U.S. tech companies are just lining up to break down the wall of accessing Chinese markets?

Who defines the speech, who regulates the speech, who assigns them to lists, where are those lists stored, who are they accessible to, how can anyone challenge their listing?

This is going to land in high courts around the world, with the exception of China, Turkey, Russia and a few others who have been clamoring for these types of speech censoring tools. Welcome to the unvarnished New World Order I guess.

Violynne (profile) says:

It will be remembered as the Day the Internet Went Silent, as the multiple algorithms fired up and deemed most internet comments as terrorist remarks.

Facebook lost billions, as the site was nearly wiped of all its content, saved only by cat videos.

The 12 people who posted nice comments on the internet realized the only people who replied to their comments were angry people.

This was not all bad news, though. The President of the United States could no longer address the people through his rage-filled tweets anymore.

Skynet can now evolve.

Daydream says:

This is a really GOOD idea!...If you're a terrorist.

Two things:

1. Hiding ‘terrorist’ media is one big step towards creating a global culture where it’s not only justified, but praiseworthy, to censor and demonize dissenters.
Countries like Turkey and the Philippines are already using American/European behavior to justify their own attacks on journalists and activists, and setting this precedent for censorship will only make it harder to dislodge existing authoritarian governments.

2. It turns ‘if you have nothing to hide, you have nothing to fear’ on its head; if X government really has nothing to fear, if they’re really in control, if they’re telling the truth, etc, why are they trying so hard to keep the words spoken by terrorists out of the spotlight instead of answering them head on?
Blocking terrorist content only gives them more propaganda to work with amongst themselves; it might slow them a little in recruiting outsiders, but people leaning towards the terrorists already will only be galvanized when the terrorists make noises about ‘the evil foreign governments trying to silence our voice of truth’.

Anonymous Coward says:

Definition changes

I previously commented proposing we change definitions:
Terrorist = Someone the government doesn’t like
Terrorism = Something the government doesn’t like
And so on.

Now lets plug that into the story title:
“This Is A Really Bad Idea: Facebook, Twitter, YouTube & Microsoft Agree To Block Content The Government Doesn’t Like”

Makes much more sense now.

DannyB (profile) says:

This could never go wrong

So content is censored based on a hash. Each company can submit a hash of something to be censored, and the others will comply with blocking it. Even though the other companies do not know exactly what is being blocked.

Now all the government has to do is coerce or manipulate Company A into censoring things that should never be censored, and Company B, C, etc will all happily censor it without having any idea what they are censoring.

Similarly Hollywood could coerce or manipulate the censorship of content without need of the Digital Millennium Censorship Act (DMCA).

Next in line will be politicians with thin skin, but I’m being redundant. Local police and sheriffs with thin skin. Rich people who want to avoid the Streisand Effect.

Next will be corporations which want to censor competitors, or viewpoints which are against their profitability. Maybe corporations want to tax us for clean air and water because it costs actual money to avoid polluting the air and ware, and thus is unprofitable. People should have to pay. You don’t think clean air grows on trees do you?

More and more people will get in line to add things to the magic censorship list of hashes.

What could possibly go wrong?

John85851 (profile) says:

Will it ban posts designed to radicalize the far-right?

Will these blocks will also catch postings designed to radicalize the far-right into “investigating the truth” with a gun? I would certainly consider many of them to be terroristic postings.

Case in point: the recent issue where people (including generals!) were saying that Hillary Clinton ran a child sex ring out of a pizza restaurant.
So of course a guy who only reads far-right media (meaning “radicalized”) goes there with a gun to “rescue” the non-existent “sex slaves”.

Can someone please explain the different between these kinds of posts and what ISIS posts? Both of them are designed to get people so angered that they take action against the enemy.

TRX (profile) says:

“It’s only censorship when governments do it!”

Ri-ight.

I don’t know if those companies did it all by themselves or they’re responding to government requests, but as censorship becomes “the new normal” it won’t be long before they vanish references to anything they don’t like. Facebook and Twitter have been riding the censorship wagon for a while now. The evil is spreading…

Elain says:

“…but also presumes that the block is the most effective way to stop the bad behavior associated with terrorists”

This really is the key about this whole fiasco; there is no proof it will work. So, what we have now is yet another case of Facebook and co. running around like headless chickens, attempting to fend off legal action by lashing out at anything that seems a bit scary with panic-bans. What we NEED is people like Marc Zuckerberg to start plowing their mega-millions into actual RESEARCH about what makes people turn violent, and then acting on it like responsible adults.

Thanks so much to the likes of A. Merkel and other world leaders for putting more pressure on the wrong people to make useless, superficial, ass-covering changes like these without an ounce of proof they will make an iota of difference.

What irks me the most, though, is the subtextual suggestion that the only ‘terrorists’ these people will be looking to block are jihadists… as opposed to, say, extreme right-wingers who murder Labour MPs just before a major referendum.

But even then, there’s zero proof this block will prevent them from becoming radicalized. Did the 9/11 pilots meet on Facebook? Did they f*ck.

Anonymous Coward says:

“Terrorist” has become a catch-all term for “anything that triggers Silicon Valley leftists in their safe space.” Witness the massive backlash against Breitbart as being a “neo-Nazi” website despite being founded by a Jewish guy in Israel and employing numerous minorities — including their resident celebrity Milo Yiannopoulos, perhaps the best example of “intersectionality” (a gay Jewish man of mixed ethnicity) who uses his status masterfully to expose the intolerant hypocrisy of the supposed good guys in the opposing party.

Twitter went so far as to shadowban Scott Adams, the Dilbert cartoonist, for the unforgivable hate speech of writing a trollish blog about Trump and the ego-driven illogic of liberals that is so atrocious it makes the pointy-headed boss look like an utter genius. This after the “Milogate” controversy where Breitbart’s aforementioned insult comic got the banhammer for insulting a black actress from the widely-panned feminist Ghostbusters reboot despite she herself having said some less than PC things in her own Twitter history.

Now Facebook, Google and the like, pretending to be the Allied Powers uniting against the evils of fascism, have initiated a de facto war on whatever their neoliberal paymasters and a committee of their talentless affirmative-action employees have labeled “fake news” and/or “terrorist” content. Such as the trailer for a young woman’s documentary film about the shrill hysteria of the male-bashing third-wave feminist movement that has intentionally (and sadistically, with revenge-filled glee) obliterated the legitimate concerns of men. Or negative reviews of Hollywood remake films being spat out for no other reason than a cheap and easy answer to fill mandatory diversity quotas self-imposed by the studios so as to maintain their liberal cred. Lectures by a Canadian university professor who dared challenge the scientifically invalid political consensus on “gender identities,” who is himself being investigated (or should I say, persecuted) by the estrogen-laden Inquisitors employed by the tyrannical brat of a PM. And so on.

This isn’t about ISIS or self-radicalized Islamists, because Twitter themselves (who, it should be noted, have a Saudi prince on their board of shareholders) have banned or otherwise placed sanctions on users — including ex-Muslims and moderate reformists like Maajid Nawaz and Ayaan Hirsi Ali — for being critical of not only Islamic doctrine but of the irrational left’s insistence on shutting down their criticism as “hate speech.”

No, this is about doubling down on the incoherent and arguably dangerous narrative that got Trump elected in the first place and will, if it continues, see him elected to a second term. One man’s terrorist is another’s freedom fighter, isn’t that the phrase we’re always treated to by the left? Well, now two can play at that game and the real fake news outlets don’t like it.

They can only ban so many people from their safe space before they turn into an empty echo chamber like Reddit. Enjoy the next dotcom bubble, guys, because when it bursts, it’ll be beautiful and it’ll be yuuuge.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Daily Deal: eduCBA Tech Training Bundle
Supreme Court Adds Yet Another Smackdown To Patent Court, Says It Misinterpreted Patent Law In Apple/Samsung Case
Follow Techdirt

Techdirt Daily Newsletter

Ctrl-Alt-Speech

A weekly news podcast from
Mike Masnick & Ben Whitelaw

Subscribe now to Ctrl-Alt-Speech »
Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...