Earlier this week I posted two examples of people falsely being told that a post or a search was deemed connected to child sexual abuse material. Earlier this week, I thought I had spotted another example, when someone on Bluesky alerted me that they had searched for “techdirt” and the results included a line saying “We think that your search might be associated with child sexual abuse. Child sexual abuse or viewing sexual imagery of children can lead to imprisonment and….”
I tried the search myself, but didn’t see the same results… until I VPN’d my way into a Polish server, and did:
On Bluesky, others quickly chimed in, noting that they also saw it elsewhere in the EU, though not always. It seemed to depend on a few factors, including whether or not you were logged in.
The whole thing seemed pretty strange, and we started to investigate what was going on. At one point, a colleague noted it was weird that the warning would appear where it seems to be in those images. That space is normally for summaries of the page you’re looking at. If there was an alert or a warning, it seems like it would appear above the search entirely.
And then… I realized. That line was not a warning from Google. It was Google doing a terrible job of summarizing what’s on Techdirt. Because, remember how I started this story off talking about the article I posted on Tuesday about mistaken claims of searching for CSAM? In the middle of that post, I included the text of that warning… which was identical to the text in the “warning” on Google.
In other words, for who knows what reason, when Google tried to summarize Techdirt, it just pulled that quote out of one article on the page, and for some category of searches that’s what it showed as the summary of the whole page. I do not know why it would choose that one sentence, because what a perplexing sentence to choose! And one that might cause people to not click on Techdirt!
There has been lots of talk about how Google’s search quality has been on the decline, and this seems like another example of the kinds of reasons why. Still, it a weird way, it’s yet another example of why policing stuff online can be so tricky. An AI summarizer might grab the wrong sentence, and when placed in the wrong context, it could look very, very bad.
I’m not entirely sure that there’s anything that can be done in such situations. It’s not like we live in Australia, where internet laws are upside down. It’s just one of those weird things, where maybe the answer is just… to use other search engines, rather than relying on an increasingly unreliable Google.
Back in December, we wrote about Appin. We were not writing about the reports (of which there have been many) that the organization that started as a sort of cybersecurity training school, but morphed into a kind of “hack-for-hire” scheme was involved in all sorts of nefarious activity. Rather we wrote about their (ab)use of the Indian court system to order Reuters to remove a big, detailed, investigative report on the company.
The history of Appin, and reporting on its involvement in hacking schemes, goes back a over a decade. Reports of Appin trying to hide and suppress such stories is a bit shorter but are abundant. And Appin has, at times, been quite successful, especially in trying to remove the name of the guy regularly accused of being behind Appin, Rajat Khare. See this SwissInfo report on how Qatar “spied on the world of football,” which was forced to remove Khare’s name while leaving in Appin’s.
Or how about the Bureau of Investigative Journalism story published in 2022, Inside the Global Hack-for-Hire Industry. An earlier version of that report names Khare. In April of 2023, the article was updated, and all mentions of Khare disappeared. There are many more examples as well.
In a move that has press freedom campaigners troubled, Rajat Khare, co-founder of Appin, an India-based tech company, has used a variety of law firms in a number of different jurisdictions to threaten these U.S., British, Swiss, Indian, and French-language media organizations.
On Nov. 16, Reuters published a special investigation under the headline “How an Indian startup hacked the world,” detailing how Appin allegedly became a “hack for hire powerhouse that stole secrets from executives, politicians, military officials and wealthy elites around the globe”—a claim that Khare strongly denies. Khare retained the powerhouse “media assassin” firm Clare Locke LLP, which boasts on its website about “killing stories,” to send Reuters several legal threats over the past year about the story, according to two people familiar with the matter.
After the removal of the Reuters story, which at least involved an actual court order, others appeared to be bullied into submission as well. Perhaps most shockingly, Lawfare (who, of anyone, should understand how ridiculous this is) redacted their version of the story about Reuters pulling down its article, saying that they did so after receiving “a letter notifying us that the Reuters story summarized in this article had been taken down pursuant to court order in response to allegations that it is false and defamatory. The letter demanded that we retract this post as well.” And they did so, despite no legal basis:
Unsurprisingly, we also received similar demands. We received multiple emails claiming to represent “Association of Appin Training Centers” legal department, and claiming (falsely) that by quoting the Reuters article (which we did not even do) we were also liable for violating the court order. Similar demands were also sent to our CDN provider, our domain registrar, and the domain registry.
The only thing we quoted from Reuters was their announcement about the removal — not from the original article. The other parts we quoted were from SentinelOne, the security research firm that Reuters used to analyze the data. At the time we wrote the article, SentinelOne’s report remained online (it, too, has since been removed “in light of a pending court order … out of an abundance of caution”).
In the meantime, though, all these attempts to pull down and hide the content appears to be causing a bit of a Streisand Effect. Beyond the Daily Beast article calling out the campaign, the website Distributed Denial of Secrets decided to republish the Reuters piece as part of its new “Greenhouse” project, noting:
In response to the unacceptable censorship by Appin and the Indian courts, Distributed Denial of Secrets is launching a new initiative to combat censorship: the Greenhouse Project. The Greenhouse Project continues DDoSecrets’ mission of ensuring the free transmission of data in the public interest by making the ‘publisher of last resort’ concept proposed by George Buchanan in 2007 a reality. By ensuring the reporting and source files are preserved, the Greenhouse Project builds on previous efforts creating a “warming effect” to reverse the chilling effects of censorship.
In addition, the Freedom of the Press Foundation, Politico, and Columbia Journalism Review have all run stories on Appin’s attempt to silence reporters. And, of course, all of this just keeps bringing more and more attention to the underlying claims about Khare and Appin. If Khare disputes those claims he could respond to them and refute them directly. Instead, he appears to be continuing a campaign of legal threats and dubious legal filings to seek to scare off reporters.
A few weeks back, we found out that our friends at Muckrock, the operators of DocumentCloud, had also received similar threats regarding documents hosted on that site.
Earlier this week, EFF sent a letter to the Association of Appin Training Centers, on behalf of both us and Muckrock, pointing out that the arguments they made in their letters to both of us did not appear to match what was in the actual court filing, which (1) does not clearly establish that the articles were defamatory based on the full evidence and a complete defense by Reuters, and (2) very clearly only apply to Reuters and Google. Furthermore, the letter points out that we are protected by the First Amendment, and any move to enforce a foreign order that violates the First Amendment would be barred under the SPEECH Act.
This kind of censorial bullying may work on other publications, but Techdirt believes that (1) important stories, especially around surveillance and hacking, deserve to be read and (2) it’s vitally important to call it out publicly when operations like Appin seek to silence reporting, especially when it’s done through abusing the legal process to silence and intimidate journalists and news organizations.
We want to thank David Greene and Aaron Mackey at EFF for their help with this.
To the Association of Appin Training Centers:
We represent and write on behalf of Techdirt and MuckRock Foundation (which runs the DocumentCloud hosting services), each of which received correspondence from you making certain assertions about the legal significance of an interim court order in the matter of Vinay Pandey v. Raphael Satter & Ors. Please direct any future correspondence about this matter to us.
We are concerned with two issues you raise in your correspondence.
First, you refer to the Reuters article as containing defamatory materials as determined by the court. However, the court’s order by its very terms is an interim order, that the defendants’ evidence has not yet been considered, and that a final determination of the defamatory character of the article has not been made. The order itself states ‘this is only a prima-facie opinion and the defendants shall have sufficient opportunity to express their views through reply, contest in the main suit etc. and the final decision shall be taken subsequently.
Second, you assert that reporting by others of the disputed statements made in the Reuters article ‘which itself is a violation of an Indian Court Order, thereby making you also liable under Contempt of Courts Act, 1971.’ But, again by its plain terms, the court’s interim order applies only to Reuters and to Google. The order does not require any other person or entity to depublish their articles or other pertinent materials. And the order does not address its effect on those outside the jurisdiction of Indian courts. The order is in no way the global takedown order your correspondence represents it to be. Moreover, both Techdirt and MuckRock Foundation are U.S. entities. Thus, even if the court’s order could apply beyond the parties named within it, it will be unenforceable in U.S. courts to the extent it and Indian defamation law is inconsistent with the First Amendment to the U.S. Constitution and 47 U.S.C. § 230, pursuant to the SPEECH Act, 28 U.S.C. § 4102. Since the First Amendment would not permit an interim depublication order in a defamation case, the Pandey order is unenforceable.
If you disagree, please provide us with legal authority so we can assess those arguments. Unless we hear from you otherwise, we will assume that you concede that the order binds only Reuters and Google and that you will cease asserting otherwise to our clients or to anyone else. ———————————————————– David Greene Civil Liberties Director/Senior Staff Attorney Electronic Frontier Foundation
Welp, here we go again. Last month I wrote about how Techdirt had been deleted from both Bing and DuckDuckGo. Over on the discussion at HackerNews, DDG’s CEO and founder, Gabriel Weinberg, jumped in to the conversation to note that this wasn’t intentional (which we never suspected it was). The resulting conversation on HackerNews is actually pretty interesting, as it appears there was some level of misunderstanding among many users about how much DuckDuckGo relies on Bing for its underlying web search.
Either way, a few hours later DuckDuckGo added back… a single link(!) to Techdirt’s front page, which we mentioned in an update. The next day, I heard from a couple people who said they had reached out to people at Microsoft, and I was told that this sometimes happen, and that the Bing team will eventually fix it (though it might happen faster if something gets public attention). Either way, about a day after I had written about Techdirt being erased, we were back in both Bing and DuckDuckGo and I considered it a one-off bug that had been fixed.
But… it’s back. I happened to just check on Bing and saw that we’re gone again (though now there’s also a big obnoxious box trying to get me to chat):
But, this time it’s weird, because it says there are 2,030 results (should be a lot more!) and then says “some results have been removed,” but it shows no results at all. If you click on the “2” at the bottom, it just takes you right back to this exact same view.
As for DDG, it still displays the one single link to our homepage and nothing else:
While that may be better than nothing, it’s pretty close to nothing. We do still get a fair bit of traffic from people searching for particular stories and now neither Bing nor DDG will send people to those stories. I did some searches on our most popular articles, like the Elon speedrun and the “you’re wrong about 230” and… all the results send people to other sites talking about our article.
Which isn’t really great.
Meanwhile, Google returns 94,900 results which is much closer to our total number of pages.
I’d really like for there to be real competition for Google out there in the search market, but it shouldn’t require me having to nag a trillion dollar company in Redmond every few weeks to put me back into their index.
As some of you might have seen, this past weekend, the NY Times ran a very nice profile about me, written by Kashmir Hill. There’s not much to say about it, other than it was an interesting (if somewhat awkward-feeling) experience to be the subject of a story, rather than the journalist covering it. But Hill is an excellent reporter and spoke to a bunch of different folks for the profile (including one who kindly called me in a panic to alert me that the NY Times was “sniffing around” for what they feared was a hit piece).
“Whenever tech policy news breaks I always want to see what Mike’s take is going to be,” said Senator Ron Wyden, Democrat of Oregon, in a statement. Mark Zuckerberg, the head of Meta, has called him “insightful and reasonable.” The tech entrepreneur Anil Dash said he “shows up and ships every day” and has been “filing constantly for decades on a beat that is thankless.”
Anyway, I thought regular Techdirt readers might enjoy reading it. I’ll just note a few bits that I found amusing. First, I’m going to make use of this description any time anyone asks me what I do for a living, since I’ve always struggled with that question:
The best way to describe how he makes a living is as an intellectual gig worker, equal parts business owner, tech journalist, policy analyst, research fellow and game designer.
This description is also fun and would go on my business cards if I still had them:
“an outsider whom insiders read…”
Finally, Hill (gently) mocks Techdirt’s design:
What Mr. Masnick apparently hasn’t had time for is a redesign of his blog. A wall of text, heavy on hyperlinks, it has not evolved much since its founding.
Which… fair enough. Some day, perhaps, we’ll update the site to make it a bit more modern, though it would help if companies and policy makers just stopped doing so much stupid stuff for just a little while to let us catch our breath. (But also, every time we mention updating the site, we get comments from people afraid that we’re going to destroy it and take away the blog format — that’s unlikely to happen. We like the old school blog setup, and changes would be more about improving the site and modernizing it around the edges, rather than the entire concept — and have no fear: we won’t fill every page with unnecessary images or pivot to video or anything like that).
A few months ago, Jack Yan pointed out to me that if they did a search for Techdirt on DuckDuckGo, it showed only one single link which was (bizarrely) to a random story from like eight years ago. There were literally no other results for Techdirt. I replicated it, but was travelling, and by the time I went back to write about it a few days later, everything seemed back to normal (in the interim there were a few days where it just found a couple hundred Techdirt posts). Jack wrote a short blog post on his own site about it.
This morning, however, someone alerted me to the fact that DuckDuckGo currently shows zero results for Techdirt. Not even some random old article. Zero. None. Zilch.
Of course, DDG is powered by Bing, so I went to check Bing, and sure enough there’s nothing there:
Bing appears to have deleted all links to Techdirt. Though at least it tells you that “some results have been removed.” Though it doesn’t say why.
At no point did anyone at Bing let us know that we’ve been removed from the search index. And, of course, BIng has every right to kick us out of their index for whatever reason they want. But it does seem odd.
So, hey, if you happen to know anyone at DuckDuckGo or on the Bing team, maybe ask them why they booted Techdirt? Apparently, I’m not the only person this has happened to.
Anyway, in the meantime, I figured I’d ask Bing’s space aged AI chat bot if it could tell me what happened. And… it actually provided a decent answer, first pointing to Jack Yan’s blog post:
And then coming up with a very speculative list of reasons why we got the boot:
I love that first one. Microsoft, a company with a $2.5 trillion market cap, “may not have enough resources” to crawl and index Techdirt? Cool. And the last one is of course possible: that Microsoft encountered “some legal or political pressure to remove or censor Techdirt.com, which is known for its critical and investigative reporting on various topics, such as technology, law, policy, and business,” but it would be nice if someone would just, you know, let me know?
I’m guessing it’s just a bug, but given that many Techdirt readers (for understandable reasons) prefer DDG to Google, it would be kinda nice if they could actually use it to get Techdirt results.
Now, of course, if this were a Trumpist nonsense peddler website, I’m sure there would be blaring headlines on Fox News and in the NY Post, and a whole set of hearings chaired by Jim Jordan about “censorship.” And we’d be hearing about it for years. That’s not going to happen with me. I’m sure the reality is much more mundane. I am guessing it’s just a glitch somewhere in the system.
But it would nice if it got fixed.
Updates: First off, I should note that it was Augusto Hermann who notified me this morning, and he’s now written his own blog post about it with some interesting additional info.
Second, after this story got popular on HackerNews, DuckDuckGo’s CEO chimed in to say this obviously wasn’t intentional and he was working on it. Later in the day, if you did the same search on DDG, it at least returned our front page… and nothing else. At least that’s some progress?
Bing, as of this moment late in the evening, still says it’s got nothing to show.
The Supreme Court is currently deliberating whether or not algorithms deserve protections under Section 230. And I hear from lots of people that maybe Section 230 wasn’t meant to cover algorithmic policing and recommendations of content. But that’s utter nonsense.
The whole area of content moderation first came about as a response to the earliest versions of spam. And one thing that people learned quite quickly is that you can’t manually police for spam if your site has even the slightest level of popularity. It will get flooded.
This is why any site needs to have some sort of automation to deal with spam. Indeed, for years, Techdirt has actually been using a combination of multiple different tools and setups to fight spam comments, but I’d never really looked into the numbers until just recently (mostly on a whim because I found the setting where those stats are!), and it’s kinda stunning. First off, we get way more spam attempts than I had even realized.
In the last six months alone, Techdirt received over 1.3 million attempts to spam our comments. That’s compared to the slightly over 40,000 legitimate comments we received in the same period. Here’s a chart of the spam comments per month:
I have no idea why spam grew so rapidly in January and February before falling in March, but even with 125,000+ spam messages in March, it completely dwarfs the amount of legitimate comments we got. The only possible way to keep up is to use automated systems. And, to be clear, while some small percentage of spam does get through, and we have a few legit comments caught in the spam filter, I’d argue we do a pretty good job of catching most spam, and allowing through most comments.
But, the larger point: without the multiple algorithmic systems we use to catch spam, we’d never be able to manage that amount of spam. Hell, we couldn’t handle manually dealing with less than 1% of the spam we actually get attempted right now. It would overwhelm us.
So if the courts (or, horror of horrors, Congress) were to decide that “algorithms” no longer are protected under Section 230, it would destroy Techdirt. While the 1st Amendment would eventually protect us, the lack of 230 protections would make using a spam filter a liability that would open up the risk of having to fight a full legal battle just to prove our right to block spam comments.
As such, our choices would be to turn off the algorithms and let spam flow, shut down our comments entirely, or risk ruinous lawsuits for the “harm” of trying to stop spam with an automated filter.
Technological filters (i.e., algorithms) should obviously be protected by Section 230, because without them, we lose the ability to fight spam, and the amount of such content is truly overwhelming. And that’s just for us, a pretty small site. Imagine how larger sites are dealing with this stuff.
If you’re already a subscriber, you should have received an email with details on attending the event. You can also go straight to the event page and register using the address associated with your Insider Shop purchase or your Patreon account. If you’ve purchased a membership or ticket but you didn’t receive an email or are unable to register, please contact us and let us know.
Once you’re registered, you can head to the event lobby at any time and we’ll invite everyone into the room when the party begins.
The party is hosted on the conference platform Remo. A microphone and webcam are encouraged (but not required) so you can chat with other attendees. We look forward to seeing you all and celebrating 25 years of Techdirt!
As we continue our celebration of Techdirt’s 25th anniversary and gear up for our anniversary party this Friday, it’s time to launch some new gear in our Threadless store! First, it’s the return of something we offered for a limited time on our 20th anniversary five years ago: the Old Logo gear. Yes, once upon a time that was the official logo of Techdirt! The first logo in fact, used until it was retired in 1999. But let’s face it — it’s not the prettiest logo. And so we’re also launching our new Retro Logo gear, featuring the current logo re-imagined as pixel art, for another way to celebrate the 25 year legacy of Techdirt.
As you know, Techdirt recently marked its 25th anniversary, and we’re celebrating with a special online party this Friday, September 9th at 11am PT / 2pm ET, where you can mix and mingle with Techdirt staff and other fans, and get the inside story on the history of the blog from Mike Masnick. Attendance is free for anyone with an active Crystal Ball, Watercooler, orBehind The Curtain membership from our Insider Shop, or the equivalent levels via our Patreon. You can also show us some extra support by buying a one-time ticket for $100.
If you’re already a subscriber, you should have received an email today (at the address associated with your Insider Shop purchase or your Patreon account) with details on attending the event. Please check for the email and use the registration link it contains. If you didn’t receive an invite but believe you should have, please contact us and let us know.
This site wouldn’t be here today without the support of our community, and we’re excited to see you on Friday and express our appreciation in person while marking this milestone in Techdirt’s history!
Techdirt began in 1997 as a simple newsletter about the latest technology news. Now, over 75,000 posts and nearly 2-million comments later, we’re celebrating our 25th anniversary!
To mark the occasion, we’re holding a special online event on September 9th, 2022 at 11am PT / 2pm ET, where you can meet and mingle with Techdirt staff and other readers, and get the inside story on the blog’s history from founder Mike Masnick.
If you are an active member or have purchased a ticket, you will receive an email the week of the event with access details for our special conference platform, Remo. A microphone and webcam are encouraged, so you can chat with other attendees!