If It’s Impossible To Compete With Google, How Come New Search Engines Keep Launching?

from the the-internet-is-quite-the-dynamic-place dept

We’re talking a lot these days about competition and antitrust, and the narrative over the past few years is that four companies — Facebook, Apple, Amazon, and Google — have basically sewn up the entire internet market, and no new entrants can ever succeed. Of course, we keep seeing that argument challenged by reality. First off, for a while people were including Netflix in that list, but over the last few years, Netflix has been facing competition from all different directions and is now struggling. On the social media front, TikTok certainly showed that it’s possible for other entrants to become very big, very fast, even if Facebook wants to kill them. And, of course, basically every month now we hear about this or that new social network that is gaining ground, especially among younger generations who don’t trust Facebook.

But, on search, we’ve been told that there really can’t be a new entrant, since Google has such control over the market. Of course, Bing is out there, and DuckDuckGo has carved out a pretty healthy slice of the market.

Perhaps most interesting to me, however, is how I keep hearing about new entrants in the search market. Last fall, privacy-protecting browser Brave announced that it was launching its own search engine, for example. However, in the last few weeks I’ve heard about two other brand new search engines as well. First up, Russ Roberts interviewed former Google exec Sridhar Ramaswamy, who recently launched the new search engine Neeva, which appears to be a search engine with a freemium model that promises not just no tracking (a la DDG), but also no ads ever.

Last year, the company raised $40 million from two top VC firms, Sequoia and Greylock, which, again, goes against the narrative that VCs won’t invest in these spaces. In just four months since the site launched, it has half a million monthly active users. That’s pretty tiny, but it’s still a starting point.

Then, just about the same time I learned about Neeva, I learned about another new search engine, called Yep (I wonder how much that domain cost!). Yep was just launched a few weeks ago, after the big search engine optimization company Ahrefs spent an apparent $60 million building it.

With Yep, their attempted differentiator is (like so many others) no tracking of personal info, including search history, and then a weird “profit-sharing” model, in which they promise to share 90% of ad profits with content publishers. I’ll be honest: I don’t quite understand what that means or how it works. First off, it seems unlikely that they’ll be making any “profits” in the short run (and perhaps longer) so is this just a future promise?

And, second, how are they going to (1) keep track of which content providers they owe money to and how much, and (2) get hooked up with those content providers to give them the money. The company’s “hypothetical” is that they would fund a ton for Wikipedia:

“Let’s say that the biggest search engine in the world makes $100B a year. Now, imagine if they gave $90B to content creators and publishers.

Wikipedia would probably earn a few billion dollars a year from its content. They’d be able to stop asking for donations and start paying the people who polish their articles a decent salary.

There would be no more need for paywalls and affiliate links, so publishers who’ve had to resort to chasing traffic with clickbait articles and filling their pages with ads would be able to get back to doing investigative pieces and quality analysis. A citizen journalist uncovering corruption on the side of a full-time job could get compensated without having to spend time trying to monetize content.

Again, this is not clear at all. How are they tracking that? How do they prevent gaming the system? Hell, they’re an SEO firm, they know that everyone tries to game search engines to get an indirect benefit. When you switch it to cold, hard cash, I imagine it’ll get that much worse. Perhaps the people at the company think their experience with SEO will help them spot the gamers, but it’s quite a challenge.

So, yes, neither of these may succeed. Both seem to have some pretty big challenges ahead. But I’m just generally fascinated by the idea that, despite the narrative about how it’s so impossible to build a search engine that there are “Venture Capital Kill Zones” where no VC would invest — and that includes search.

Yet, just here, within a week, I found out about approximately $100 million being spent on building two separate competing search engines, both with at least some plans to differentiate themselves in the market.

The internet is incredibly dynamic. There may be policy options for increasing competition, but it’s hard to argue that some companies have so dominated the field that no one even dares attempt to build competitors any more. They seem to be happening all around us.

Filed Under: , , , , ,
Companies: google, neeva, yep

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “If It’s Impossible To Compete With Google, How Come New Search Engines Keep Launching?”

Subscribe: RSS Leave a comment
31 Comments
Tmink says:

Timing

Considering that the original Google search engine patent expired in 2018, it’s reasonable to think that at least some of these new entrants were waiting until after that to make their pitch to get funding and develop the infrastructure to launch. Before that, I seem to remember most engines like duckduckgo used either the Bing or Google databases to generate results.

Naughty Autie says:

Re:

Before that, I seem to remember most engines like DuckDuckGo used either the Bing or Google databases to generate results.

I’m pretty sure they didn’t. That’s all Scroogle did, and Google had it shut down in 2012 for ‘copyright infringement’. Alphabet, as it’s now called, alleged that Scroogle was copying its code, not merely scraping its results.

Lostinlodos (profile) says:

Re: Re: Re:2

relatively recent

within the few years, I think?
I believe the always had their own crawler but I could be wrong. I started using them as a second option in 2014 or 2015 when Skraper shut down. Mainly because they, DDG, also pulled from classical index sites. Known to newer generations as farms. There was a point where bookmarked searches changed results enough to be noticeable a few years back.

But I miss altavista, SearchLight, and and LH. So I’m not the average.

glenn says:

All of the talk about competition and antitrust really just boils down to one thing: “big tech” envy. (If they’re “big,” it’s really just because lots of people freely choose to use them, which leaves the ones not chosen to complain about the “big” guys getting in their way. But politicians just complain because they’re not as popular as “big tech.”)

Rekrul says:

Every time I hear about a new search engine, I try it out. What I’ve found is that almost all of them fall far short of Google’s search results.

In Yep’s case it completely ignored the first part of what I searched for, and just showed me matches for the second part. Yeah, that first part was completely optional, I don’t know why I even bothered to include it…

And of course, no matter what search engine I use, they will all happily direct me to scam sites. “You’re looking for Bloodsucking Nerd Bikers from Jim’s Butthole? We have it! Just create a free account. Your credit card will be charged a $15 fee just to verify that this content is authorized for your country, but we pinky-swear it will be immediately refunded.”

David says:

Re:

We have it! Just create a free account. Your credit card will be charged a $15 fee just to verify that this content is authorized for your country, but we pinky-swear it will be immediately refunded.

That brings back memories. Friend of mine won on a one-armed bandit first time he tried and quit, not wanting to spoil the record.

I had the same with some adult “web site verification” where the site failed to continue anywhere past getting my credit card data and I just forgot about it until I got the next (or later) credit card notice with something like $49.90 (probably just below some criminal fraud limit) missing from an obscure company. I protested the company since I never got as far in the process as receiving a hidden smallprint warning about starting some subscription (the web site just failed) and I protested the payment with my card vendor.

I got reimbursed by both. When asking back with credit card services, they told me “hey, if they don’t complain, why would you?”.

Like my friend, I quit while I was ahead. I think I might have ended up more of a unicorn than he.

Anonymous Coward says:

Re:

And of course, no matter what search engine I use, they will all happily direct me to scam sites.

A search engines job is to catalog the WEB, and not curate the web. The distinction is important, as curation will make mistakes. There are other resources to protect you from scam sites, available as browser extensions, and which give you the option of ignoring the warning, so reducing the harm of mistakes..

Anonymous Coward says:

Re: Re:

There are other resources to protect you from scam sites, available as browser extensions, and which give you the option of ignoring the warning, so reducing the harm of mistakes.

Opera Mini doesn’t have that option, which means I have to be extremely vigilant. The weird thing is, Bitdefender never warns me about all the viruses that some of the pop-ups on webpages tell me I have.

Rekrul says:

Re: Re:

There are other resources to protect you from scam sites, available as browser extensions, and which give you the option of ignoring the warning, so reducing the harm of mistakes..

The sites I’m thinking of don’t generate any warnings. They give the appearance of legitimate sites.

For example, I was looking for an epub/pdf copy of a particular book (yes, I’m a dirty pirate, but I don’t have a Kindle, and I’m not going to pay for a DRM-infested copy that requires special software to read). I kept seeing sites that claimed I would be able to read and download this book if I registered for a free account. When I put in a fake email address to test it, most wanted a credit card number.

There was one site that didn’t, so I created an account with a throw-away address. When I clicked the download link, I was told that no download was available (big surprise!). Then I clicked the button for reading it online. It gave me an online reader display with exactly one page; The cover image used to lure suckers into registering.

So I went to account settings and tried to delete the account, but it told me that the password was wrong. Even after successfully changing the password, it still said it was wrong. I sent them an email telling them to delete my account and they did, noting that since I had created a free account they wouldn’t charge me anything. WTF? That would be a good trick considering I never entered any payment information.

David says:

Cough cough

If It’s Impossible To Compete With Google, How Come New Search Engines Keep Launching?

Frankly, that reminds me of Mark Twain saying “I cannot understand people who say it is hard to quit smoking. I did it hundreds of times.” or something to that effect.

This is not about the difficulty of trying but of succeeding.

xyzzy (profile) says:

Not really search engines

So DuckDuckGo should be take off the list, or at least classified as a search integrator, they claim many sources, but the mainly use a bing and yahoo back end. Result, poor results.

Yep has a back end and no front end, try it, no images, very little added value, just a raw results browser for traditional text results, they have a ways to go.

Bing itself produces inferior results, for the sorts of technical searches I do. Bing is dying. Yahoo is garbage.

Naughty Autie says:

Re:

If you don’t like something, you’re not forced to use it. There will be people who like what you want to scrap, and want to scrap the little you like. The thing about the Internet is that there’s room enough for a few more search engines, and if they weren’t there, your sole favourite could be shut down or broken up to end its monopoly. You know, being an IF fan should make you better at critical thinking, not worse.

Lostinlodos (profile) says:

A quick look at Wikipedia

A glance at list of search engines shows just how many are used by enough people to not get knocked off the list.

Results will very by user and settings. But I have opted into tracking on my devices for advertising targeting. With that, I find Bing gives me the best mainstream results.
But I also use a few indexes. I’m old school and like the methodology of modifiers over common language lexicon. .

Every time a new one pops up I wait a month and then throw a list of basic strings at it. See how that compares to Bing.

There’s a variance out there for everyone to be happy.
I use a dedicated app ImgSr, for image searches, especially reverse image searching. Because it uses the ArchiveTeam list of file types for images and uses results from multiple sources.
I use Shazam and Tunz for audio identification.
Etc etc. there’s no lack of competition. Just lack of public advertising of the competition.

Davvi (user link) says:

I use Presearch . com, decentralized search engine run by average ppl at home / on VPS'

Presearch is a truly decentralized platform that runs on over 70k home computers and VPS servers of average people around the world , it combines the search results from multiple search engines and does its best to incorporate results that are censored by others for various reasons. (it’s not just an aggregator) .

It’s also using its vast network of user run servers to actually build its own crawler / indexing service and is slowly becoming fully self sufficient.

A real worldwide community run search engine, something that many thought impossible because tech to run queries decentralized would be too complicated, it took them 5 years but they finally did it 🙂 .

Chris says:

Google isn't Google anymore.

Google when it launched in 1998 was a lexical search engine, a search engine that returned only documents that contained your keywords.

Today, Google is a “semantic” engine, an incomprehensible black box that returns AI generated made for adsense spam that maximizes short term ad yield to google while ignoring your keywords and making the user angry and despondent that the internet is dying or devoid of content.

New entrants seek to provide what Google used to provide. If Google realizes in time that credibility and execution matters they might just keep hold of their monopoly. But they are failing and bleeding cash in every non-search market they enter now due to a lack of product leadership.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...