If It’s Impossible To Compete With Google, How Come New Search Engines Keep Launching?
from the the-internet-is-quite-the-dynamic-place dept
We’re talking a lot these days about competition and antitrust, and the narrative over the past few years is that four companies — Facebook, Apple, Amazon, and Google — have basically sewn up the entire internet market, and no new entrants can ever succeed. Of course, we keep seeing that argument challenged by reality. First off, for a while people were including Netflix in that list, but over the last few years, Netflix has been facing competition from all different directions and is now struggling. On the social media front, TikTok certainly showed that it’s possible for other entrants to become very big, very fast, even if Facebook wants to kill them. And, of course, basically every month now we hear about this or that new social network that is gaining ground, especially among younger generations who don’t trust Facebook.
But, on search, we’ve been told that there really can’t be a new entrant, since Google has such control over the market. Of course, Bing is out there, and DuckDuckGo has carved out a pretty healthy slice of the market.
Perhaps most interesting to me, however, is how I keep hearing about new entrants in the search market. Last fall, privacy-protecting browser Brave announced that it was launching its own search engine, for example. However, in the last few weeks I’ve heard about two other brand new search engines as well. First up, Russ Roberts interviewed former Google exec Sridhar Ramaswamy, who recently launched the new search engine Neeva, which appears to be a search engine with a freemium model that promises not just no tracking (a la DDG), but also no ads ever.

Last year, the company raised $40 million from two top VC firms, Sequoia and Greylock, which, again, goes against the narrative that VCs won’t invest in these spaces. In just four months since the site launched, it has half a million monthly active users. That’s pretty tiny, but it’s still a starting point.
Then, just about the same time I learned about Neeva, I learned about another new search engine, called Yep (I wonder how much that domain cost!). Yep was just launched a few weeks ago, after the big search engine optimization company Ahrefs spent an apparent $60 million building it.

With Yep, their attempted differentiator is (like so many others) no tracking of personal info, including search history, and then a weird “profit-sharing” model, in which they promise to share 90% of ad profits with content publishers. I’ll be honest: I don’t quite understand what that means or how it works. First off, it seems unlikely that they’ll be making any “profits” in the short run (and perhaps longer) so is this just a future promise?
And, second, how are they going to (1) keep track of which content providers they owe money to and how much, and (2) get hooked up with those content providers to give them the money. The company’s “hypothetical” is that they would fund a ton for Wikipedia:
“Let’s say that the biggest search engine in the world makes $100B a year. Now, imagine if they gave $90B to content creators and publishers.
Wikipedia would probably earn a few billion dollars a year from its content. They’d be able to stop asking for donations and start paying the people who polish their articles a decent salary.
There would be no more need for paywalls and affiliate links, so publishers who’ve had to resort to chasing traffic with clickbait articles and filling their pages with ads would be able to get back to doing investigative pieces and quality analysis. A citizen journalist uncovering corruption on the side of a full-time job could get compensated without having to spend time trying to monetize content.
Again, this is not clear at all. How are they tracking that? How do they prevent gaming the system? Hell, they’re an SEO firm, they know that everyone tries to game search engines to get an indirect benefit. When you switch it to cold, hard cash, I imagine it’ll get that much worse. Perhaps the people at the company think their experience with SEO will help them spot the gamers, but it’s quite a challenge.
So, yes, neither of these may succeed. Both seem to have some pretty big challenges ahead. But I’m just generally fascinated by the idea that, despite the narrative about how it’s so impossible to build a search engine that there are “Venture Capital Kill Zones” where no VC would invest — and that includes search.
Yet, just here, within a week, I found out about approximately $100 million being spent on building two separate competing search engines, both with at least some plans to differentiate themselves in the market.
The internet is incredibly dynamic. There may be policy options for increasing competition, but it’s hard to argue that some companies have so dominated the field that no one even dares attempt to build competitors any more. They seem to be happening all around us.
Filed Under: competition, investing, kill zones, search, search engines, vcs
Companies: google, neeva, yep
Comments on “If It’s Impossible To Compete With Google, How Come New Search Engines Keep Launching?”
Neeva
Been using Neeva since last fall. It’s good. I still occasionally use Google when the search results from Neeva aren’t quite what I was expecting but it doesn’t happen all that often.
Re:
It looks interesting, but the “no tracking” claim is weird. Apparently, only people with accounts can search, but how many people concerned about tracking would search the web while logged in? I sure as hell wouldn’t.
Re: RE: Neeva
Imma try Neeva for a week. So far, it’s listing sites Google wouldn’t dare. +1
Timing
Considering that the original Google search engine patent expired in 2018, it’s reasonable to think that at least some of these new entrants were waiting until after that to make their pitch to get funding and develop the infrastructure to launch. Before that, I seem to remember most engines like duckduckgo used either the Bing or Google databases to generate results.
Re:
Before that, I seem to remember most engines like DuckDuckGo used either the Bing or Google databases to generate results.
I’m pretty sure they didn’t. That’s all Scroogle did, and Google had it shut down in 2012 for ‘copyright infringement’. Alphabet, as it’s now called, alleged that Scroogle was copying its code, not merely scraping its results.
Re: Re:
DDG uses Bing. As well as a few others and it’s own crawler.
Re: Re: Re:
DuckDuckGo’s partnership with Bing is relatively recent. It probably used only its own crawler before that.
Re: Re: Re:2
Dammit, you got there before me.
Re: Re: Re:2
within the few years, I think?
I believe the always had their own crawler but I could be wrong. I started using them as a second option in 2014 or 2015 when Skraper shut down. Mainly because they, DDG, also pulled from classical index sites. Known to newer generations as farms. There was a point where bookmarked searches changed results enough to be noticeable a few years back.
But I miss altavista, SearchLight, and and LH. So I’m not the average.
Re: Re: Re: REee
I use DDG on my work computer and half the time I get nonsense links for stuff I have to search at work. It’s frustrating because I don’t want to use Google, but I always go back to them so I can do a proper search.
All of the talk about competition and antitrust really just boils down to one thing: “big tech” envy. (If they’re “big,” it’s really just because lots of people freely choose to use them, which leaves the ones not chosen to complain about the “big” guys getting in their way. But politicians just complain because they’re not as popular as “big tech.”)
Objective reality has always been anathema to ferrals.
Every time I hear about a new search engine, I try it out. What I’ve found is that almost all of them fall far short of Google’s search results.
In Yep’s case it completely ignored the first part of what I searched for, and just showed me matches for the second part. Yeah, that first part was completely optional, I don’t know why I even bothered to include it…
And of course, no matter what search engine I use, they will all happily direct me to scam sites. “You’re looking for Bloodsucking Nerd Bikers from Jim’s Butthole? We have it! Just create a free account. Your credit card will be charged a $15 fee just to verify that this content is authorized for your country, but we pinky-swear it will be immediately refunded.”
Re:
That brings back memories. Friend of mine won on a one-armed bandit first time he tried and quit, not wanting to spoil the record.
I had the same with some adult “web site verification” where the site failed to continue anywhere past getting my credit card data and I just forgot about it until I got the next (or later) credit card notice with something like $49.90 (probably just below some criminal fraud limit) missing from an obscure company. I protested the company since I never got as far in the process as receiving a hidden smallprint warning about starting some subscription (the web site just failed) and I protested the payment with my card vendor.
I got reimbursed by both. When asking back with credit card services, they told me “hey, if they don’t complain, why would you?”.
Like my friend, I quit while I was ahead. I think I might have ended up more of a unicorn than he.
Re:
A search engines job is to catalog the WEB, and not curate the web. The distinction is important, as curation will make mistakes. There are other resources to protect you from scam sites, available as browser extensions, and which give you the option of ignoring the warning, so reducing the harm of mistakes..
Re: Re:
There are other resources to protect you from scam sites, available as browser extensions, and which give you the option of ignoring the warning, so reducing the harm of mistakes.
Opera Mini doesn’t have that option, which means I have to be extremely vigilant. The weird thing is, Bitdefender never warns me about all the viruses that some of the pop-ups on webpages tell me I have.
Re: Re:
Unless you’re researching scams, it stands to reason that scam sites are not a good result to return, even if by other metrics they are the correct result.
Re: Re:
A search engine’s job is to catalog the web and curate the most relevant results for people using the search engine. A good search engine will curate results in a way that lessens the chance of hitting a scam site result. A bad search engine won’t.
Re: Re:
The sites I’m thinking of don’t generate any warnings. They give the appearance of legitimate sites.
For example, I was looking for an epub/pdf copy of a particular book (yes, I’m a dirty pirate, but I don’t have a Kindle, and I’m not going to pay for a DRM-infested copy that requires special software to read). I kept seeing sites that claimed I would be able to read and download this book if I registered for a free account. When I put in a fake email address to test it, most wanted a credit card number.
There was one site that didn’t, so I created an account with a throw-away address. When I clicked the download link, I was told that no download was available (big surprise!). Then I clicked the button for reading it online. It gave me an online reader display with exactly one page; The cover image used to lure suckers into registering.
So I went to account settings and tried to delete the account, but it told me that the password was wrong. Even after successfully changing the password, it still said it was wrong. I sent them an email telling them to delete my account and they did, noting that since I had created a free account they wouldn’t charge me anything. WTF? That would be a good trick considering I never entered any payment information.
Cough cough
Frankly, that reminds me of Mark Twain saying “I cannot understand people who say it is hard to quit smoking. I did it hundreds of times.” or something to that effect.
This is not about the difficulty of trying but of succeeding.
Re:
This is true, but much of the narrative is about “kill zones” and how no one is even trying. This indicates otherwise.
Not really search engines
So DuckDuckGo should be take off the list, or at least classified as a search integrator, they claim many sources, but the mainly use a bing and yahoo back end. Result, poor results.
Yep has a back end and no front end, try it, no images, very little added value, just a raw results browser for traditional text results, they have a ways to go.
Bing itself produces inferior results, for the sorts of technical searches I do. Bing is dying. Yahoo is garbage.
Re:
If you don’t like something, you’re not forced to use it. There will be people who like what you want to scrap, and want to scrap the little you like. The thing about the Internet is that there’s room enough for a few more search engines, and if they weren’t there, your sole favourite could be shut down or broken up to end its monopoly. You know, being an IF fan should make you better at critical thinking, not worse.
A quick look at Wikipedia
A glance at list of search engines shows just how many are used by enough people to not get knocked off the list.
Results will very by user and settings. But I have opted into tracking on my devices for advertising targeting. With that, I find Bing gives me the best mainstream results.
But I also use a few indexes. I’m old school and like the methodology of modifiers over common language lexicon. .
Every time a new one pops up I wait a month and then throw a list of basic strings at it. See how that compares to Bing.
There’s a variance out there for everyone to be happy.
I use a dedicated app ImgSr, for image searches, especially reverse image searching. Because it uses the ArchiveTeam list of file types for images and uses results from multiple sources.
I use Shazam and Tunz for audio identification.
Etc etc. there’s no lack of competition. Just lack of public advertising of the competition.
I use Presearch . com, decentralized search engine run by average ppl at home / on VPS'
Presearch is a truly decentralized platform that runs on over 70k home computers and VPS servers of average people around the world , it combines the search results from multiple search engines and does its best to incorporate results that are censored by others for various reasons. (it’s not just an aggregator) .
It’s also using its vast network of user run servers to actually build its own crawler / indexing service and is slowly becoming fully self sufficient.
A real worldwide community run search engine, something that many thought impossible because tech to run queries decentralized would be too complicated, it took them 5 years but they finally did it 🙂 .
Re: Decentralized?
Interesting, I’ll check it out. Thanks.
I suppose that any search engine that displays actual results and not a pile of ads, ads-disguised-as-results andresults-artificially-pumped-up-by-ad-money and then maybe 1 or 2 lousy results will get the crown sooner or later. Google is actually killing what first made it relevant.
No tracking?
When I tried Neeva, Privacy Badger reported “neeva.containers.piwik.pro” as a potential tracker.
Whether or not this is a real tracker, and what it may or may not be tracking, is well beyond my level of expertise in the area, but I found it amusing, at least.
Alt search
Has a different take on search.
Google isn't Google anymore.
Google when it launched in 1998 was a lexical search engine, a search engine that returned only documents that contained your keywords.
Today, Google is a “semantic” engine, an incomprehensible black box that returns AI generated made for adsense spam that maximizes short term ad yield to google while ignoring your keywords and making the user angry and despondent that the internet is dying or devoid of content.
New entrants seek to provide what Google used to provide. If Google realizes in time that credibility and execution matters they might just keep hold of their monopoly. But they are failing and bleeding cash in every non-search market they enter now due to a lack of product leadership.
Just tried a hit on neva
The search on neva fails to execute with no cookies and browser gives error as too many redirects, soo I consider that a complete fail on privacy, get lost neva..