Techdirt Has Been Deleted From Bing And DuckDuckGo
from the yeeted-from-search dept
A few months ago, Jack Yan pointed out to me that if they did a search for Techdirt on DuckDuckGo, it showed only one single link which was (bizarrely) to a random story from like eight years ago. There were literally no other results for Techdirt. I replicated it, but was travelling, and by the time I went back to write about it a few days later, everything seemed back to normal (in the interim there were a few days where it just found a couple hundred Techdirt posts). Jack wrote a short blog post on his own site about it.
This morning, however, someone alerted me to the fact that DuckDuckGo currently shows zero results for Techdirt. Not even some random old article. Zero. None. Zilch.
Of course, DDG is powered by Bing, so I went to check Bing, and sure enough there’s nothing there:
Bing appears to have deleted all links to Techdirt. Though at least it tells you that “some results have been removed.” Though it doesn’t say why.
At no point did anyone at Bing let us know that we’ve been removed from the search index. And, of course, BIng has every right to kick us out of their index for whatever reason they want. But it does seem odd.
So, hey, if you happen to know anyone at DuckDuckGo or on the Bing team, maybe ask them why they booted Techdirt? Apparently, I’m not the only person this has happened to.
Anyway, in the meantime, I figured I’d ask Bing’s space aged AI chat bot if it could tell me what happened. And… it actually provided a decent answer, first pointing to Jack Yan’s blog post:
And then coming up with a very speculative list of reasons why we got the boot:
I love that first one. Microsoft, a company with a $2.5 trillion market cap, “may not have enough resources” to crawl and index Techdirt? Cool. And the last one is of course possible: that Microsoft encountered “some legal or political pressure to remove or censor Techdirt.com, which is known for its critical and investigative reporting on various topics, such as technology, law, policy, and business,” but it would be nice if someone would just, you know, let me know?
I’m guessing it’s just a bug, but given that many Techdirt readers (for understandable reasons) prefer DDG to Google, it would be kinda nice if they could actually use it to get Techdirt results.
Now, of course, if this were a Trumpist nonsense peddler website, I’m sure there would be blaring headlines on Fox News and in the NY Post, and a whole set of hearings chaired by Jim Jordan about “censorship.” And we’d be hearing about it for years. That’s not going to happen with me. I’m sure the reality is much more mundane. I am guessing it’s just a glitch somewhere in the system.
But it would nice if it got fixed.
Updates: First off, I should note that it was Augusto Hermann who notified me this morning, and he’s now written his own blog post about it with some interesting additional info.
Second, after this story got popular on HackerNews, DuckDuckGo’s CEO chimed in to say this obviously wasn’t intentional and he was working on it. Later in the day, if you did the same search on DDG, it at least returned our front page… and nothing else. At least that’s some progress?
Bing, as of this moment late in the evening, still says it’s got nothing to show.
Filed Under: content moderation, search, search results
Companies: duckduckgo, microsoft, techdirt
Comments on “Techdirt Has Been Deleted From Bing And DuckDuckGo”
Bing currently shows links to your Wikipedia, Twitter, and Patreon pages, but not the actual website
Re: Oh, same wiki
You know who runs wikipedia and twitter, right?
Re: Re:
Yes: the Illuminati, in conjunction with the Rand Corporation and the reverse vampires.
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:
I won’t write his name forward but I’ll write it backward: Soros.
Re: Re: Re:2
That anti-Semitic bullshit doesn’t fly here, dipshit.
Re: Re: Re:3
I didn’t think I needed the /s but these days I guess it’s hard to tell who is serious…
Re: Re: Re:4
Poe’s Law is a bitch—especially when you make a post that can be easily mistaken for a post made by a real anti-Semite.
Re: Re: Re:4
Poe’s Law was needed back then, and it’s needed now. If anything, there was never not a time when Poe’s Law was not needed.
Re: Re: Re:5
Oops, I should’ve said:
I should really count my negative words…
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:6
What we do is do what we’ve always done: blame Republicans and straight white males.
If we’re wrong, we respect the right to parody, as us demeaned and downtrodden minorities deserve the right to make fun of the privileged.
If we’re right, fuck those white jock Trumpers.
Either way white Trumpists get the blame. It’s a win-win.
Re: Re: Re:7
Every accusation, a confession.
This comment has been flagged by the community. Click here to show it.
Re: Re: Re:8
Exactly. Everything that Trumpers say is a confession. We must hold them to everything they say and invalidate them, just like they tried to invalidate us.
We cannot let up on this. Letting up on humiliating straight white males caused Roe v. Wade to be repealed. Never again.
Re: Re: Re:9
You can’t shut us down, Matty.
Your wife doesn’t deserve you.
Re: Re: Re:6
Or you could try not not counting your negative words.
Re: Re: Re:4
If you pretend to be a shitty person by saying shitty things. What’s the difference?
Re: Re: Re:5
Popehat’s rule of fucking goats.
Re: Re: Re:5
The difference is whether or not you get a Comedy Club timeslot.
Re: Re: Re:2
I’d like to remind everyone that The Protocols of the Elders of Zion is not a good foundation to build your damn conspiracy theories on. Or governments, for that matter.
And that the Republicans and their rich fucks have successfilly polluted the minds of many, many politicians.
https://www.theguardian.com/world/2023/jul/09/us-religious-right-lgbtq-global-culture-fronts
And it’s very likely that they’ve been at it since the 70s.
Re: Re: Re:
Curse those Reverse Vampires, barging into your house uninvited in the middle of the day and pumping you full of blood. A true menace.
Re: Re: Re:2
Reverse cowgirls, on the other hand …
Re: Re:
Wikipedia is owned by a nonprofit organization founded by Jimmy Wales.
Twitter is a private for-profit corporation owned by Elon Musk.
How are those even remotely connected?
Re: Re: Re:
They’re both on the internet?
Re:
Yes, this is what I found about a month ago trying to get to the Techdirt website through DDG. I ended up going via Wikipedia because DDG wouldn’t return a direct link. I thought it was very weird. I guess I’ll have to start considering an alternate search engine.
Re: Re:
There are such things as bookmarks, and they make you independent of the search engines for returning to a site. Also, add the search box to the tool bar, and you can easily try different search engines without retyping the query, including a direct search of Wikipedia.
This comment has been flagged by the community. Click here to show it.
Looks like a pattern
You must be doing something right, then.
It is a pattern now, going on for a few years. Sites with independent from MSM propaganda line which have large following and are influential are being targeted for shadow banning. That includes google/YT. By influential, I mean causing people to think independently on their own, and make their own informed decision on any given subject. Tiktokers are not influencers, since large number of click bites does not mean influence on informed decision at all.
Re:
…hallucinated nobody mentally competent, ever.
Re:
I mean, it’s a theory. Sure. But if you’re going to get anyone to follow you, you’re going to need – and need to SHOW – more evidence. Like, smoking gun level evidence.
Re:
I might have to inform Mike that his invitations to the EU and even his recwnt invitation to teach judges about 230 are simply falseflag ops then…
Might want more evidence, though.
LOL. The last hit (in the first page) for me is “Emails ‘Inventor Sues Techdirt”, as a fortune article.
This comment has been flagged by the community. Click here to show it.
but of coooourse
Well this site does suck.
And now everyone who took the COVID shot (at your behest) is passing more gas than they normally would, which is releasing more methane into the environment, and causing this ridiculous global warming.
Please send more aluminum foil Please!
Re:
Sigh, another shill for the big tinfoil industry…
Re: Re:
Yeah.. Please was even uttered ..twice!
Re:
I think you should remove the hose from your anus and mouth, the fart gas does not cause technicolour hallucinations.
Re:
Big aluminium would like to remind you that your continued usage is not how you use aluminium foil.
Good grief - good luck
All I can say is “good luck”. Maybe someone knows someone on the inside – that’s your best bet.
It could be a mark of success.
If you’ve done a good job, it might upset someone.
I always find Microsoft incredibly useless for getting answers out of.
I’m surprised they didn’t respond with: “Have you tried turning it off and back on again?”
Re:
That’s actually useful advice.
Re: Re:
No it isn’t. It’s incredibly patronizing.
When stuff breaks on Linux people want to know what distro you’re using, what version, what version of software, what you were doing, what time of day it was, whether the moon is gibbous/waxing/full…
Every single conceivable detail.
Microsoft cronies just love passing the buck.
Re: Re: Re:
Linux is rarely used by people unfamiliar with technology. Windows and Bing absolutely are. As such, a lot of people calling Microsoft’s tech support are not familiar with troubleshooting at all, so they start off with the simplest fix first before moving on to the rest.
Eventually, Microsoft tech support will ask for more details, but they have to start off with the simple stuff.
Re: Re: Re:2
I thought Windows help people only had two things to suggest, switch it off and on again, and if that failed, reinstall Windows.
Re: Re: Re:3
No, they actually suggest a lot of different solutions, depending on the specific problem.
Re: Re: Re:3
Nope. They do suggest a bunch of solutions depending on the problem. Their unhelpfulness is often exaggerated.
Re: Re: Re:2
You do need to be brought up to speed about Linux, technology, users, desktops, user-friendliness, etc. Sadly, TD isn’t the place to do it.
Re: Re: Re:3
I’m merely stating that, statistically, Linux users are a lot more tech-savvy than Windows users on average.
Re: Re: Re:
You’re calling support for help. You’re literally asking to be patronized.
Re: Re: Re:2
You’re literally not. You’re asking for help.
Re: Re:
Pretty sure just the first part would be beneficial when it comes to Bing.
Re: Re:
Not for search engine indexing issues.
Re: Re:
Even better advice, if it’s a Microsoft product, “Have you tried turning it off and leaving it off?”
Re: Re: Re:
That was the best offer the AI Chatbot made. “How to remove Bing as your browser’s default search engine.”
Thanks, Bing AI Chatbot!
Re: Re:
It is if you’re dealing with memory issues on your smartphone, PC or router.
Not so much things like search index issues and programming.
Re: Re: Re:
That’s extremely fair, as is all this pushback I’m receiving. It’s especially embarrassing for me, since I have a minor in Computer Science.
Deindexing
Maybe someone is submitting [bogus] deindexing requests to Bing, claiming DMCA or using a forged court order, and Bing is acquiescing without taking a good look.
For comparison, see Eugene Volokh’s articles about people trying to “vanish” his pages in Google.
Maybe it’s just me but I’ve noticed fewer and fewer sites showing up on DDG. Now it’s mainly Reddit and a few other big sites. I wonder if Bing is doing a big purge and starting to move to a site whitelist in a misguided effort to reduce hate speech or misinformation or succumb to the latest moral panic. I wish DDG would build its own index instead of relying on Bing.
Re: AI?
Microsoft said that they’d rely more on AI/LLM vis-à-vis Bing; maybe TechDirt getting removed from search results is a fruit thereof.
Have you tried Bing Webmaster tools?
Rather than just shouting into the void?
https://www.bing.com/webmasters/about
Re:
Who was shouting into the void?
Also, are you the void?
What's in a name?
“x.net could not be found.[…]”
Not enough shilling
Well, we all know how big of a shill Mike is. If he hasn’t been shilling enough for Bing, these things happen.
Some results have been removed
When I search bing with domain:techdirt.com I get a message that some results have been removed
The results link to a support article with details about why this might have occurred.
well .. i just added your rss feed to my feedreader instead… the good news is that because people are talking about this in my spheres, i found you so you gained a new follower … in spite of this bs.
You can
tell the quality of your success by the Quality of your enemies.
HOW big is MSN? as not part of MS. Doing a spider search is HUGE anymore. Wonder if MSN was ready for the internet to EXPAND, and EXPAND, and EXPLODE..
Anyone notice if MSN is more Adverts then Search?
yegg at DDG says he's looking into it
“(DuckDuckGo CEO/Founder) Just seeing this and we’re looking into this now. This is not intentional.”
ref: https://news.ycombinator.com/item?id=36898217#36898661
This thread: https://news.ycombinator.com/item?id=36898217
Not just Techdirt
Other alternative tech info/news/discussion web sites are affected. Just try searching for techwontsave.us and you end up getting lots of techwillsaveus junk.
I have found turning off Safe Search will get you some more results. I guess Microsoft and Bing think independent thinking is too dangerous
Re: Safe Search on bing
Tried that but no change.
Yet another reason to not use bing, or duckduckgo for that matter
DDG founder is looking into
Per his comment at Hacker News
This comment has been flagged by the community. Click here to show it.
because it was just a glitch when Twitter shut down the NY Post’s account days before a presidential election when they had real evidence of a scandal involving a presidential candidate and his family?
you need scare quotes around “censorship” when hinting at real concerns of people who you deem to be beneath you. I mean, why does it matter if some filthy “trumpers” are censored, right? In fact, it matters so little, that you are actively admitting here that you won’t even consider the idea that you might be being censored yourself.
Also, those scare quotes are especially rich considering we now know for a fact that the FBI was explicitly in contact with Twitter and Facebook to tell them what to take down. So the FBI was in contact with Twitter, and Twitter censored the oldest newspaper in the country’s factual story about candidate days before an election
and you think it’s WRONG to have hearings about that, so the American people can learn what happened? To the point where you won’t even consider that Microsoft may have done the same to you, because that might put you in the same category as your political opponents, in being anti-censorship or in having the audacity to notice that they’re being censored?
You sir, are a coward.
Re:
ProTip: Shiny side out
Re: Re:
Do you think he has enough? I mean, one little gap and they’ve got you…
Re:
The election was more than two weeks away when NY Post’s Twitter account was suspended, and it was reinstated two days later.
Moreover, it wasn’t “real evidence” when the account was suspended, it was believed to be hacked material.
Trumpers not being able to whine about something on a certain platform isn’t censorship, dingus.
That’s because, unlike you, Masnick knows what is or isn’t censorship.
Feel free to provide evidence of that. So far, there’s been none.
Twitter suspended an account for two days and stopped people from linking to a story because of an internal policy about hacked materials. Again, feel free to provide evidence that it was at the FBI’s behest.
Yes, because if politicians had demanded hearings because of the editorial decisions Fox News makes, there would have been an uproar. The hearings were ridiculous, from beginning to end.
And you, sir, are a liar.
Re:
That was shocking right?! They were completely silenced and couldn’t share their crap with anyone! No wait, don’t they have a website or something…?
You upset you couldn’t look at Hunter’s dick?
Re: Re:
That last part…. I can’t decide if it’s funny or insightful. Maybe it’s both, something that doesn’t happen very often.
After all dirt you published about them, why would they do anything for you? Why wouldn’t they want your website wiped completely from all their systems? Rhetorical questions, it is probably a glitch. Nevertheless, I find it ironic that you so much depend on the company you dislike.
Re:
Because that makes their product even more useless.
Re:
Except they don’t depend on it.
Re: Re:
Don’t depend so much, that a whole publication with five screenshots had to be created to highlight this independence.
I see search results
I see full details using DuckDuckGo as of 4pm EDT July 27th. I can’t post a screen shot, so I copied some of the text:
All
Images
Videos
News
Maps
All regions
Safe search: moderate
Any time
Techdirt is an American Internet blog that reports on technology’s legal challenges and related business and economic policy issues.
https://en.wikipedia.org › wiki › Techdirt
Techdirt – Wikipedia
Techdirt founder Mike Masnick in 2012. Techdirt is an American Internet blog that reports on technology’s legal challenges and related business and economic policy issues, in context of the digital revolution. It focuses on intellectual property, patent, information privacy and copyright reform in particular.
https://twitter.com › techdirt
techdirt (@techdirt) / Twitter
techdirt. @techdirt. ·. May 23. Hey folks, just letting people know that the end came last week. While they had said the API cutoff would come at the end of April, they actually cut us off last week. Techdirt can no longer automatically post to Twitter, because of the new API rules. You can find us elsewhere. Quote Tweet.
Or it could be a simple Parser Error
If I enter simply ” techdirt “, it comes right up.
Re:
They have fixed it, as it did not come up for when thisthe article was published.
Re:
I think it is common with deindexing that direct site name searches still work. It doesn’t advertise to people who haven’t decided yet which site to find the information on, and it makes the delisting less conspicuous.
When told to delist by law enforcement, even the direct search will go away.
Re:
Maybe they fixed it but for me “techdirt” works while “tech dirt” does not.
This comment has been flagged by the community. Click here to show it.
It was me, I asked them to.
No not really but I will note if you think pages should be delisted for “Misinformation” reason #3 would apply.
FWIW I have noticed Bing has had much better results than google the last few years. ¯_ (ツ)_/¯
Re:
https://mediabiasfactcheck.com/techdirt/
This comment has been flagged by the community. Click here to show it.
Found the article about techdirt being potentially censored to be interesting until the article spoke negatively about conservatives and conservative websites.
As a result of the negative comment about conservatives, it may be that the search engines are correct in their actions against techdirt.
Re:
You haven’t thought that all the way through have you.
This comment has been flagged by the community. Click here to show it.
To be serious, SEO often penalizes self-links
The algos are always evolving (and I’ve been out of that game for years), but for a while the algos were basing a lot on in-coming links, and then when the SEO people got hold of that they fought that in various ways, including penalizing self-links, and…..
…..90% of your citations and links are to your own articles. You’ll say “debunked” and link to your own article, which just means you said the same thing (often a lie) already. Besides being obnoxious and self-promoting, it may well be hurting your score.
Probably should get…someone who understands the modern tech ecosystem to fix that for ya.
Re:
A galaxy-brain like you must be smarter than everyone else.
Me on the other hand, I’m dumb in comparison. I use Mike’s links to his previous articles where the links to the original source material are cited.
Re: Re:
Matty has in fact claimed exactly that before.
Re: Re: Re:
With less than zero supporting evidence for his claim, as always.
Hundreds if not thousands of sites get ‘delisted’ every day, and I’d lay long odds that none of those sites are “alerted” or otherwise contacted about such action. The obvious question then becomes ‘why should TD be an exception to that fact of business life?’
More to the point, if one can correctly spell ‘techdirt.com’, then why isn’t one simply going to the site directly – why does one need a search engine to find what they’ve already typed into the address bar? Oh, you’re looking for a specific article on a specific topic, is that it? Then the proper syntax is site:www.techdirt.com, not ‘domain something something’. (That site:… portion can be placed anywhere on the address bar, it doesn’t matter where.)
And whereas most browsers, and DNS servers, will just automatically insert the ‘www.’ portion for a correct resolution, failing to enter it into the search query can often result in a lack of responsive entries. Just sayin’……
Re:
Doing that makes no difference with this current issue.
Re: Re:
Hey you, yeah you out there in left field….. It would probably help if you participated a little more often in these discussions. Who knows, you might actually ‘get’ both the gist and the back-story of what’s going on, both here and in general.
IOW, keep your eye on the ball. Sometimes it bounces around all crazy-like, but paying attention is the only way to catch the damned thing.
Re:
DDG’s CEO acknowledged there was an issue which is supposedly fixed now.
Just saying…
This comment has been flagged by the community. Click here to show it.
Maybe it’s karma for continuing to mock, slur, insult and demean the half of America whose political views you disagree with, MM?
And why would you expect Fox News or the NY Post to rush to your defense with outrage coverage over something that you yourself literally admit IN THIS VERY POST is probably:
??
But anyway, cry harder. Play the victim more.
Re:
Sir, this is a Wendy’s
Re:
“But anyway, cry harder. Play the victim more.”
Not possible to take that crown away from the Trump “Conservatives”.
Re:
If that were an actual thing then conservative media would’ve burned to the ground years ago. Who’s really playing the victim here?
Re: Re:
“Half of America” only according to electoral college votes (taking the “America = U.S.” myopia in stride). The popular presidential vote (which would better correspond to the “half of” claim) or the de-gerrymandered representative votes tell a different story. As for the senate… if you get the same number of senators for Wyoming and California (which has about 70 times the population), talking about “half of America” seems like quite the euphemism.
It’s a stretch to say “this is like the Founding Fathers wanted it” since Wyoming and California are not even among the original union states.
Re: Re: Re:
They’re a noisy minority. They know it.
Re: Re: Re:2
A noisy minority with far too much political power, yes.
Re: Re: Re:3
Eventually market forces will excise straight white men from the gene pool, and all will be right with the world.
Re:
Funny that this is the sort of petty snipe you have to count for a victory, John Smith.
It’s almost like the press release you gloated about was never going to happen.
Re:
Jhon pls
I’m surprised the cops haven’t arrested you for actual fraud
It just went back up on ddg for me, with a note saying it was an ‘official site’ (it was removed from the results a few hours earlier). My guess is that they were trying to implement something which accidentally took it down for some reason
Results look different to me
Using Mozilla on Linux (16:19 CDT), the first thing that appears on DDG (under the page menus) is a link to Techdirt with a description and logo. The information box appears to on the top right with a link to Techdirt, description, Wikipedia link and Techdirt social media links.
Below are a few not very helpful search results, but the are related to Techidrt.
So…maybe temporary or maybe something client-side related?
Re:
Search for “techdirt.com”:
– techdirt.com landing page link, wikipedia, and various references to techdirt by other sites.
Search for “site: techdirt.com”:
– Not. One. Link.
Search for “site: slashdot.org” (Just to be sure the above worked for some sites):
– Plenty of slashdot articles.
Seems pretty broken to me.
Re: Re:
site:[space]techdirt.com should return nada – you can’t insert a space after the colon.
At least that’s how it used to be. I suppose that inserting that space is now possible, I haven’t read the “Shortcuts” page for some time, and my fingers are pattern-trained.
From the YCombinator post, this person had the same problem. Months after contacting Bing (apparently fruitlessly), their site magically reappeared.
https://www.jessesquires.com/blog/2022/03/25/my-website-disappeared-from-bing-and-duckduckgo/
A comment there refers to a LinkedIn blog post which may be relevant to this more controversial site, Bing Exploit Allows Website Owners to Deindex Competitors (not having an account, I can’t read it).
Well this is interesting: the home page is back for me: site:techdirt.com now returns 1 result: https://www.techdirt.com.
That makes it look like this isn’t just some glitch; someone is intentionally messing with the filters.
I may just need to get https://www.searxng.org/ up and running sooner than I thought.
I thought DuckDuckGo was powered by more than just Bing? I’d reach out to them and see if you can make sure you’re indexed by some of their other sources.
Updated
I’ve added an update with some of the latest. DDG’s CEO has said he’s working on it, and so far DDG has returned a link to ONLY Techdirt’s front page, but searches for any article don’t show anything. It’s basically as if the front page is the only thing in their index.
Bing still shows nothing at all.
Re:
“site:techdirt.com” now shows “about 1,900 results”
Just ask around.
I wouldn’t worry too much. As I always say, you’d get a better answer just asking your friends than using Bing.
Techdirt meet Tucker Carlson
Working on Bing in the UK now.
Vegan search engine?
Isn’t DDG the new “vegan” search engine, only used by the illiterati to signal their heightened sense of propriety?
Just be more awesome!!
Bing results back now
10:49 pst, all results appear to be up with a direct link
DuckDuckGo
Would this have something to do with it?
https://twitter.com/shanhoward/status/1514785110133936135
Re:
Unless she’s admitting to filing some bullshit to get TD scrubbed from DDG, I’m taking their CEO’s word for it.
She seems… unreliable.
TechDirt Shows Up on DuckDuckGo
I use DuckDuckGo regularly and have never had a problem with TechDirt not showing up. I checked it just now about 2 p.m. CDST, July 29, 2023, and the site showed up with a link that took me right into the main site as well as with other second party references. I don’t know what other people’s issues are, but I don’t have any with DDG.
The others
I don’t use DDG (it doesn’t save search history) nor Google (search results are too mainstreamed to popularity over quality):
But as of today, Bing, my hand-set default, offers over 1million results for the search Techdirt and “over” 10,000 for a site search site:techdirt.com. More accurately 100+ pages of 100 results. Clicking page 100 brings up only td site results.
It was clearly a bug.