Further Thoughts On Cloudflare And Infrastructure Moderation

from the clarifying dept

I rather expected that not everyone would agree with the points I raised in my recent post on Cloudflare and my thinking regarding both Cloudflare’s statement of principles on content moderation at the infrastructure layer and, secondly, its decision to block Kiwi Farms. That turned out to be an accurate expectation. It has also resulted in some thoughtful critiques, most of which I think misrepresent what I actually believe, to the point that it’s obvious that I was not clear enough in the original post. Therefore, we get a follow-up post (how exciting).

First, a general comment on the areas where I think most of the misconception is coming in. My post was actually a lot less about Cloudflare, in particular, and more about the unfortunate state of the world that we’re in, where this even has to be a discussion at all. In some sense, to me, the situation with Cloudflare is more a symptom of a much larger problem — the failure of society to make sure that people aren’t falling behind/left behind, such that they feel the need to attack and lash out at others. And, no that’s not saying “oh look at the poor KF trolls and assholes,” but noting that, in general, many of the people who end up on a place like Kiwi Farms don’t get there from nowhere. They may first have some more legitimate grievances, and slowly and further buy into the culture leading them to embrace pure hatred. And the more that society can do to prevent that, the less hold grievance culture is likely to take.

And, partly, I fear that by focusing on “where is the leverage point” rather than “how the fuck did we get here,” we never get to have that important discussion.

Second, I wanted to highlight how — unlike many content moderation efforts these days — Cloudflare was putting out actual principles that clearly involved a lot of thought. I thought that process alone was valuable, and I wish more sites were so open about their content moderation philosophies.

Finally (and this is a point most people missed, because I basically held it until the closing thought), I was trying to make the point that, as useful as I thought it was that Cloudflare was laying out their philosophy, the initial end result, of leaving harassers to continue their harassing, was clearly the wrong result, and that something should have been done about that (and I was glad that Cloudflare eventually got there). At some point, a company has to realize that if it’s the one thing standing in the way of ending what seems to be people getting seriously harmed, it has to take action. Cloudflare did get there, just way too late. But I could still respect that the company was trying to put some clear principles behind its policies, even if I disagreed with the initial end result the company came to based on those principles.

Given that general clarification, there are a couple of buckets of ideas that I want to clarify.

On the 1st Amendment and Section 230:

A few people, starting with Mike Dunford, raised the argument that when I pointed some blame government for letting things get so bad that an infrastructure provider had to step in to protect others from violence, I should also call out the 1st Amendment and Section 230 because both, ostensibly, helped protect Kiwi Farms in getting to this point, and in some ways block the government from doing anything about the site.

I don’t think that’s correct at all, but I can see how I failed to explain why as clearly — and I can see how one could get to that argument from what I wrote. The larger argument is that the failure of government, comes much earlier in the process. The issue is that there are societal level problems that lead people to think that harassment, especially targeting more marginalized communities, is an acceptable and reasonable approach to life, and an outlet for whatever grievances they have. A year or so ago, I highlighted another case of clearly unacceptable harassment in which I tried to make the same point.

The failures I’m talking about are not the failure of the government to step in and kill Kiwi Farms itself — because that would raise serious 1st Amendment issues. It’s the failure to deal with societal issues that make people feel that Kiwi Farms is an appropriate outlet for their own twisted beliefs. I was focused on trying to look at the root causes of the disease, and not just how best to cure the symptoms.

This, reasonably, got people upset, because if you’re getting killed by a disease, you don’t want to fucking hear about how awful it is that society lets you suffer from the disease. You want the fucking cure. And I was waxing on about the former, not the latter. The anger over that makes sense, but it’s important to me that we don’t lose site of the larger issue.

In short, my argument is that, as a society, we should be doing more to deal with the demand side issue of the people who were eager to contribute to the harassment via Kiwi Farms, such that communities like Kiwi Farms are less likely to form or attract the kind of following they did.

Some people responded to this point asking “how,” and I think comments like this one are very good ideas, highlighting how we’ve built a society that leaves many people behind. And that, by itself, contributes to this world where taking out anger on others is deemed a reasonable outlet. Again, some people are always going to be awful, obviously. But, right now we’re seeing that grievance culture is attractive to many, and that’s harder to keep going if you don’t have as many grievances. It’s not a perfect solution, by any means, but I’m not saying that there is a perfect solution here. Indeed, part of the whole point is that it’s messy.

But I fear that just focusing on “who can fix this” doesn’t give us any chance to deal with “how we got here.”

Attacking the 1st Amendment and Section 230 doesn’t fix any of this, by the way. Again, without 230, many sites would actually be incentivized to do less because the only way they’ll be considered liable is if they were taking a more active approach. Take away Section 230 and a company like Cloudflare is less likely to kick off Kiwi Farms. Section 230 is what allows the company to feel free to do so without facing the risk of legal liability. As for the 1st Amendment, I think I’ve spent decades explaining why attacking the 1st Amendment will almost certainly lead to more harm than good, and how the powerful will undoubtedly use it to silence the less powerful and the marginalized.

On the issue of governments v. private companies making moderation decisions:

This is related to the section above, but I want to be pretty clear here. Some read my post and the point about the failure of governments to mean that I want the government to have come in and shut Kiwi Farms down. Anyone who reads Techdirt regularly knows that basically goes against everything I’ve ever written.

Again, to repeat what I said above, I don’t think that’s appropriate by itself, and, in most cases, doing so would actually raise serious 1st Amendment issues, and almost definitely lead to dangerous abuse. That said, if law enforcement turns up evidence that the operator of Kiwi Farms himself participated in actions that violated the law via the site, then I could see reasonable legal action being taken.

But, again, my larger point was to look at the systemic issues that got us to this point, not begging the government to correct for just this symptom of the larger problem. Rather I’m hoping that there’s a larger discussion to be had about preventing the demand side of the equation.

On the right of private companies to boot whoever they want:

A few people argued with me that, as a private company, Cloudflare is free to do as they wanted, as if that were some sort of gotcha. And, that struck me as at least slightly amusing, because it’s the argument that I’ve been making for years for edge providers. They have their own 1st Amendment association rights not to do business with someone they disagree with. And my original post actually supported this (though only for people who read to the end, which clearly was not all of the critics).

The point that the post tried to raise was not that Cloudflare didn’t have the right to do this — obviously I thought they did, because I noted that it was clearly the right decision — but that it raises more questions that deserve consideration when dealing with infrastructure rather than edge providers.

Again, this is why we literally ran an entire symposium last year on the difference in moderation questions at the infrastructure level. Because the questions are actually more challenging and not the same as they are for edge providers, no matter how much people insist they are.

As I pointed out to a few people, are you willing to insist that all infrastructure players should have that same right? And if so, does that mean that domain registrars, DNS providers, phone companies, and broadband providers should all be free to deny you service if they think you’re a bad person?

Again — I feel the need to repeat this here — I think Cloudflare had every right to do what it did, and I think that it was the right thing to do given all else. But I also think it’s important to explore what this means in the long run, and what people find appropriate.

And “shut down the sites I don’t like” is not a great set of principles to work off of. It’s certainly not scalable. Especially since the same rationale is going to be used on sites you like as well, which is one of the points that Cloudflare raised, and which it was mocked for. But that was part of what I appreciated about Cloudflare’s post: that it recognized that there will be ramifications of these decisions.

People pushed back and pointed out that either they don’t care, or that better Cloudflare face the ramifications than those suffering from at the hands of Kiwi Farms users. And that’s a valid point. But, again, it’s one that can also be flipped around. Because there will absolutely be times when the party you support is flipped in this equation, and a site you like is targeted.

Infrastructure, schminfrastructure

A lot of people went beyond just the above point, and basically used all of the arguments I’ve always used about edge providers, and argued that they all applied to infrastructure providers as well. But infrastructure providers actually are a different beast. Just as I supported net neutrality as a principle (though was nervous about the methods by which it was implemented by the government), it was about understanding that infrastructure providers are very different. Many people argued that there is no difference — but I wonder if they’re also against net neutrality on the same basis.

At some level, we have to recognize that different layers of the stack have different trade-offs when it comes to moderation, and we can’t treat them all exactly the same. You can argue that you disagree with where Cloudflare’s various products fall along that stack spectrum from edge to infrastructure. And you can argue for different types of remedies at the different levels of the stack.

But I have a hard time accepting that all providers are identical.

Just as many people were arguing that Kiwi Farms is obviously different than other sites, and therefore it obviously deserved to be taken down, I’d argue that anti-DDoS providers are obviously different than hosting providers.

But Cloudflare isn’t a common carrier

This issue came up a few times in response to the argument above. A phone company or a broadband provider is a common carrier (though, actually, in the US we’ve decided broadband providers are mostly not common carriers, but let’s leave that aside for now). But this is part of the argument right now. And there are lots of efforts underway to declare edge providers common carriers (an argument I believe makes no sense at all).

But, since these arguments are happening right now, we should be willing to address the fundamentals of what does make a common carrier and what does not. And when we do that, that may help inform proper approaches to dealing with some of these questions.

My hope was that Cloudflare’s set of principles was part of that discussion. Indeed, Cloudflare was more or less calling out that some of its lower level infrastructure services should be considered to be common carrier type services.

At the very least, we should be having a clearer discussion on what does constitute a common carrier, and what does not.

Your principles are nonsense if they believe that Cloudflare shouldn’t take down Kiwi Farms

Again, I need to repeat: I said in the post that I thought Cloudflare was right to take down Kiwi Farms, and that coming to the opposite result raised questions about where the principles took you. But, that was, in part, because of the failures of everything else that rested this decision on Cloudflare’s shoulders.

What I appreciated about what Cloudflare wrote — and which no one outside of a very small number of people who study content moderation questions seemed to appreciate — was that Cloudflare seems to be one of the only companies actually willing to step up and discuss these things and to put forth principles on how to answer some of these questions.

I said I disagreed with the initial outcome, and agreed with the later one. But either way, I thought that it was good that Cloudflare was actually willing to lay out its thought process. No one else does this, at least, not to the extent that Cloudflare was willing to do so.

And that’s what I thought was useful about the initial post.

I know that many cynical people have dismissed that post as ass covering or avoiding tough questions, but that’s simply not true. Cloudflare presented a pretty thoughtful approach on how to deal with the issues that have been thrust upon it. I think what the Kiwi Farms scenario showed was that those principles may make sense in a vacuum, but are much harder to deal with in real life.

And that’s a big lesson of content moderation: your policies and principles don’t seem to last long when confronted with the real world.

But that doesn’t mean we ignore the principles entirely and just resort to being arbitrary.

This was all nonsense and a smokescreen

This is an extremely cynical take, but many people have been saying it to me. They argue that Cloudflare has no principles, that the executives there are horrible, horrible people who have evil in their hearts and want to see the world burn, or something to that effect. I get this type of thinking, and we actually hear it a lot about executives at various companies for this or that slight.

And it’s almost never true. We can (and should) criticize bad decisions, but I also think it’s important to recognize incentive structures and why reasonable people intending to do the right thing get to outcomes we dislike. It’s rarely because they’re inherently bad. It’s often because of the unfortunate incentive structure associated with Wall Street and the demands for quarterly growth. Or it’s because of other unfortunate systemic issues that put tremendous pressure on companies to do the wrong thing. In many cases, it’s because people are in over their heads, don’t have all of the information, and are trying to calculate a variety of trade-offs and variables without perfect information.

But it’s not because of inherently evil people running these companies.

In the case of Cloudflare, I know that the company and its execs have been really trying to think this through, and to explore the various trade-offs and truly understand what is the right thing to do. And that’s, in part, because I’ve been involved in a few of those discussions with different people at the company who were asking the really tough questions, and actually trying to grapple with the consequences of different choices. And, if it’s gotten down to the level where such folks are even talking to me about it, it means they’re talking to lots of other people with much more expertise who have thought a lot more deeply about it than I have.

I find the quick jump from “I disagree with the decision” to “it’s because they’re evil and they want to support harassers” not just deeply, depressingly cynical, but also fundamentally wrong.

The timing of all of this

Here’s one point that was raised last night by Blake Reid: he points out that my argument regarding the nuance and the principled nature of this would be a lot stronger if Cloudflare didn’t release its big statement when it did. And… he’s right. Other people raised similar issues, regarding how the timing and framing of this made it clear that this was a cop out.

This would have had much more impact if Cloudflare had made this all clear a while back, and not at the very moment when it was getting pressured to shut down Kiwi Farms. Because, when it’s at that moment of public pressure, it becomes clear that the initial post was a defense of not acting rather than “here are our general operating principles.”

And a few people have gotten to the crux of this. The timing of the statement was a mistake. It certainly came across as tone deaf. While I appreciated (and still think others should appreciate) the public display of the philosophy, in some ways, this was like the same mistake I discussed up top. It was a “let’s look at the deeper root cause” while someone was on the table suffering.

Right thing to do. Wrong time to do it.

Anyway, as I’ve tried to make clear over many, many years, anything related to content moderation is crazy complicated and there are no easy answers. I appreciate people pushing and prodding me to think more carefully about all of this.

Filed Under: content moderation, infrastructure, security, society
Companies: cloudflare, kiwi farms