"When someone has a really high ad click probability, they're very hard to beat, so it's not a really competitive auction," McAfee told The Reg. "So that they don't just win [every auction], we do squashing. This makes the auction more competitive.
"It's like handicapping. We handicap the people with the high click probability."
This, McAfee said, can increase Yahoo!'s revenues. "The bidders respond by bidding higher. The one who was destined to lose is now back in the race, so they bid higher trying to displace the number one, and the number one is trying to fend them off so they bid higher too.
"We can make the competition a bit more fierce using squashing, even on keywords where there's not much bidding."
While that may seem like a neat trick from an economics standpoint, it certainly seems like a pretty questionable business practice from an advertisers' standpoint. Having a company secretly manipulate the results of an auction to make participants pay more? That sounds like fraud. As Eric Goldman notes, this appears to be a lawsuit waiting to happen.
We recently highlighted a part of one of Paul Graham's recent essays that told the story of how Yahoo missed the paid search opportunity, by fearing that it would cannibalize all the revenue coming in from its "portal" business. As we noted, it was a great example of why big companies so rarely notice disruptive innovation, even when it's handed to them. Ali Partovi picked up on the same part of Graham's essay, and wrote a similar story about why Microsoft also failed to see the opportunity in paid search, despite the fact that Partovi and others were pushing for it, both from the outside, and then inside (after Microsoft bought his company, LinkExchange):
From 1997 to 2000, we visited Yahoo more than a dozen times to pitch the Keywords idea: pay-for-placement, keyword-targeted text ads on the side of search results. Despite repeated rejection, we pitched every member of Yahoo's executive team multiple times, each time finding new ways to present the concept and new data to support how profitable and huge the opportunity might be, all in vain....
In late 1998, Microsoft bought LinkExchange for $265 million, telling us they liked the "Keywords" vision. As Microsoft employees, we continued pitching the Keywords deal not only to Yahoo, but also to the up-and-coming Google. I wasn't surprised to find that these companies were wary of partnering with Microsoft. My greater surprise was the seemingly insurmountable resistance we faced within Microsoft itself.
After almost two years of fighting bureaucratic obstacles, we finally got the green light to launch "Keywords" as an MSN Search feature in 2000. It started growing rapidly, and the MSN Ad Sales division feared (correctly) that it would cannibalize banner ad revenue. They therefore decided (incorrectly) to shut down Keywords after a few months. If Yahoo's demise stemmed in part from being ambivalent about technology, perhaps Microsoft's error stemmed in part from being ambivalent about ad sales: we couldn't get the senior execs interested enough to intervene.
Both cases highlight the same basic point: the claim that big companies will automatically recognize a disruptive innovation and "copy it" is wishful thinking in many cases. Time and time and time again we see stories more like the ones above, where truly disruptive innovation isn't just ignored, it's actively blocked at big legacy companies who fear it cannibalizing an existing business, rather than recognizing the opportunity.
In the end, both Microsoft and Yahoo failed to jump into keyword advertising in any serious way until long after Google established it as a giant business. At that point, both tried to play catch-up, with Yahoo buying Overture and Microsoft rebuilding its product -- and as we've also seen over and over again, by waiting that long, it was too late. The two companies still haven't come anywhere close to catching up in market share, even if the technology is considered to be about equal at this point.
So the fear of some big company coming out and just "copying" you is generally overblown. If your idea is really disruptive, they probably won't recognize it, and by the time they do, you'll have a big head start, and their attempts to copy what you did will prove a lot more difficult than they expected.
Microsoft's "other" co-founder, Paul Allen, has had a long string of business failures since leaving Microsoft. It's actually quite impressive. One of his most high profile ventures, back in the 90's, was Interval Research, which was designed to be a pure research institute "done right." It was described as trying to replicate Xerox PARC, but that it would actually commercialize the amazing ideas. Of course, as we've pointed out for ages, ideas are only a small part of innovation. Actual execution is the really difficult part, and one thing Interval was never able to do was execute. After eight years and over $100 million of Allen's money, the operation was shut down back in 2000. That was about the last we'd heard of Interval... until now.
Because, while Interval was unable to actually execute, thanks to the wonders of the US Patent system, it was able to secure lots of patents, and now it looks like Paul Allen has gone full on patent troll. He's using those patents to sue Google (and, separately, YouTube), Apple, AOL, eBay, Facebook, Netflix, Yahoo, Office Depot, OfficeMax and Staples -- you know, the companies that actually did innovate and did execute -- for being successful where he failed. Of course, Paul Allen has been tangentially related to patent trolling operations in the past, so perhaps it was just a matter of time. Still, this is a pretty disgusting situation all around.
The WSJ article about the lawsuits doesn't mention the actual patents (why do so few reporters actually point you to the useful info?), but they're the following:
6,263,507: "Browser for use in navigating a body of information, with particular application to browsing information represented by audio data."
6,034,652 & 6,788,314 (really the same patent, involving continuations): "Attention manager for occupying the peripheral attention of a person in the vicinity of a display device"
6,757,682: "Alerting users to items of current interest"
According to a report by Todd Bishop at Techflash, Interval may just be getting started: "This is the most recent step in a long process," he said in an email, "but it is not necessarily the end of the process." What a sad, sad legacy Mr. Allen is leaving behind.
We've been following the EFF's patent busting efforts for its list of the 10 worst patents, and it looks like an East Texas jury may have at least partially invalidated one of the patents, 6,411,947, which describes a method for automatically routing emails. As the EFF noted, this patent appeared to cover "basic natural language processing techniques taught in introductory computer science courses."
Things took a more interesting turn when the guy holding the patent, Erich Spangenberg and his hoarding company, Polaris IP, decided to sue Google, Yahoo, Amazon, AOL, IAC and Borders for daring to automate email responses without first paying him. If Spangenberg/Polaris sound familiar, it's because he's become one of the more prolific patent hoarders out there lately, and a couple years ago had to pay out $4 million to Daimler, after he apparently used various shell companies to move some patents around and sue Daimler multiple times over the same patent, even though an earlier settlement had him promising not to assert that patent against the company again. Spangenberg also believes in suing first before contacting a company, and always suing in East Texas, because the juries there like to hand out giant awards.
Spangenberg's legal strategy in this particular lawsuit was also quite questionable, as he demanded that Google hand over information concerning its lobbying efforts on patent reform. What that had to do with whether or not Google infringed on this particular patent was never clearly explained.
Either way, Spangenberg's faith in East Texas juries may have been misplaced this time around:
The jury found three of the patent's claims invalid based on the public use bar, obviousness, and for lacking written description. The jury also found that neither Google nor Yahoo! infringed those claims. Finally, the jury found the entire patent invalid due to improper inventorship.
Separately, per Google's request, the USPTO has already been re-examining the patent. The scorecard on this list of patents is increasingly tilting in the EFF's favor, but it's a statement of how awful the patent system is to note how long this has taken. The EFF announced its patent busting project in 2004. And while the process is on-going to invalidate many of them, it's taking quite a long time -- all the while allowing patent holders to create frivolous lawsuits that waste money that could be spent on actual innovation.
Last year, we wrote about a court ruling in Argentina that found Google and Yahoo liable for defamation claims, after a celebrity was upset that searches on her name had results that pointed to pornographic websites. There had actually been a similar decision in Argentina the year before as well. It seems silly to blame search engines if people don't like the search results on their name, but that's what happened. Thankfully, however, in an appeal to the first case we linked to above, involving Virginia Da Cunha, the court found that the sites could only be held liable if they were made aware of the "illegal content," and then failed to remove it. In other words, the court is effectively using a notice-and-takedown safeharbor setup. There are still problems with that, but it's a hell of a lot better than automatically fining Google and Yahoo even if those companies had no idea about the fact someone was upset with the search results. Still, it's not great. As another article notes, without an official safe harbor, the only effective way to win cases like this is to have the money to go to court. Even Google and Yahoo are still fighting a bunch of similar lawsuits and will have to keep going through the process, until there's a real safe harbor in place.
Paul Graham has written one of his typically worth-reading essays about why Yahoo! went from the darling of the internet world in the mid- to late-90's to whatever it is today (an also-ran's also-ran). I don't have much to say on the main point of the essay, so if you're interested in that, go read it. However, what did catch my eye, was one little aside about trying to get Yahoo to buy Google soon after Google came on the scene:
I remember telling David Filo in late 1998 or early 1999 that Yahoo should buy Google, because I and most of the other programmers in the company were using it instead of Yahoo for search. He told me that it wasn't worth worrying about. Search was only 6% of our traffic, and we were growing at 10% a month. It wasn't worth doing better.
I didn't say "But search traffic is worth more than other traffic!" I said "Oh, ok." Because I didn't realize either how much search traffic was worth. I'm not sure even Larry and Sergey did then. If they had, Google presumably wouldn't have expended any effort on enterprise search.
Whenever we talk about innovation and things like patents, one common refrain is that no innovation would occur without patents because big companies would immediately copy the technology and destroy any up-and-comer. We've pointed out plenty of times that this simply isn't true. For a truly disruptive innovation, big companies often won't even notice you until you're way ahead of them -- at which point copying is fruitless. Hell, for nearly the past decade now, Yahoo's tried every which way to "copy" Google, and it got them nowhere in terms of actual market share (actually, it got them so little that they recently gave up and outsourced it all to Microsoft).
The problem is encapsulated in the little exchange between Graham and Filo above (and, I've actually heard nearly an identical anecdote from some folks at AOL who looked at buying Google in '98/'99 as well). If a company is big enough to be the "feared" competitor that people always worry about, it's because they're making a lot of money from something. When a disruptive innovation comes along, they usually don't care because they're blinded by the cash cow that they already have. In fact, the really disruptive innovations are scary to these big companies, because it they usually look like they'll undermine the cash cow. Elsewhere in the post, Graham notes that before Yahoo! bought his company in '98, he showed Jerry Yang a new offering he was working on that would optimize revenue on shopping search -- but he notes that Yang didn't care:
Jerry didn't seem to care. I was confused. I was showing him technology that extracted the maximum value from search traffic, and he didn't care? I couldn't tell whether I was explaining it badly, or he was just very poker faced.
I didn't realize the answer till later, after I went to work at Yahoo. It was neither of my guesses. The reason Yahoo didn't care about a technique that extracted the full value of traffic was that advertisers were already overpaying for it. If they merely extracted the actual value, they'd have made less.
Real innovations threatens cash cows, and one of the most difficult things for any company to do is undermine their own cash cows. So stop worrying about some big, successful company copying your idea. If it's really innovative, they probably won't even notice it... until it's too late.
These kinds of lawsuits are coming fast and furious again these days. Glyn Moody points us to the news that 36 companies have been sued for patent infringement in Marshall, Texas (of course) for supposedly violating a patent (6,018,761) on spam filtering. The companies sued represent a who's who of corporate America, including Apple, Google, HP, RIM, Citigroup, Capital One, Alcatel Lucent, AIG, AOL, JP Morgan Chase, McAfee, Symantec, Yahoo, IBM and many others.
The patent itself is rather simple. So simple, I can repeat the entire claims section right here (not the abstract, the actual claims). Also, note how many typos there are. You would think, in such a short patent, someone would have caught typos like "usinig," "processine" and "usefiul.":
What is claimed is:
A method of obtaining context information about a sender of an electronic message using a mail processing comprising the steps of:
scanning the message, usinig the mail processine program to determine if the message contains a reference in a header portion of the message to at least one feature of the sender's context, wherein the sender's context is information about the sender or the message that is usefiul to the recipient in understanding more about the context in which the sender sent the message;
if the message contains such reference, using the mail processing program and such reference to obtain [sender] the context information from a location external to the message;
if the message does not contain such reference, using the mail processing program and information present in the message to indirectly obtain the [sender] context information using external reference sources to find a reference to the [sender] context information.
The method of claim 1, wherein the reference to at least one feature is a reference to a location where context information is stored.
The method of claim 1, wherein the reference to at least one feature is a hint usable to retrieve a location where context information is stored.
How could someone possibly approve this as a patent? This is about as basic a filter as you can imagine. Someone should sue the USPTO for fraud on America for approving this patent.
In the meantime, the press release announcing the lawsuit is funny as well. The lawyers claim "the company's patent is one of the building blocks for all email communications. InNova's complaint alleges that the defendant companies have used InNova's invention without permission for years." Please, don't make everyone laugh. It is not one of the "building blocks for all email communications." It's a basic filter that any first year programmer could have written in no time flat.
Oh, but it gets better:
"Email as we know it would essentially stop working if it weren't for InNova's invention," says Mr. Banys... "More than 80 percent of email is spam, which is why companies use InNova's invention rather than forcing employees to wade through billions of useless emails. Unfortunately, the defendants appear to be profiting from this invention without any consideration for InNova's legal patent rights."
First of all, actual spam filtering is a hell of a lot more sophisticated than the methods in this patent, and the idea that email would stop working without this patent existing is pretty laughable. This is such a basic concept that it boggles the mind that anyone thought it was patentable.
While the initial link above refers to InNova as a "Texas company," as per usual, it appears to be such in name only. There is no information as to who's actually behind the company, but it seems likely they're not based in Texas. The only reason the company is "based in Texas" is to file a lawsuit in Marshall. At the very least, InNova does have a very simple website where it pretends that it actually does something and has a "portfolio" of patents. Except, if you dig deeper, you see it's just this single patent. But my favorite part of the webpage is this opening paragraph:
Ours is a world of technology, where companies are measured, by their customers, their competitors and the media, by the quality, utility and innovation of their products. Because no single company has a lock on innovation, InNova Patent Licensing LLC offers medium- and large enterprises creative solutions to the problem of staying relevant in today's business climate.
InNova has no customers, competitors or products. All it has is a painfully ridiculous patent. And it's trying to lock down innovation. What a joke.
Perhaps Pakistan should just turn off its internet connection? We've already had stories recently about how courts have ordered that both Facebook and YouTube be blocked -- but supposedly both bans had been lifted. Then came the news that there were attempts to put Facebook's founders on trial, potentially for a death sentence, because some users had posted stuff people in Pakistan didn't like. The latest, sent in by Ali, is that a Pakistani court has ordered a whole bunch of popular websites be blocked from access. You may have heard of a few of these: Google, Yahoo, Bing, MSN, YouTube, Amazon and Hotmail. Who are they missing? Ebay? Twitter? I'm sure those will be blocked soon enough...
Someone who prefers to remain anonymous sent over this story about how Associated Press stories hosted on Yahoo News appear to have tons of comments from old stories. It's not entirely clear what's happening, though I have my suspicions (explained further down), but it appears that when new stories are showing up on certain topics, Yahoo is simply copying over older comments from previous stories on similar or related topics. The comments look as if they're about the story posted -- and the only way you can tell they're not is if you notice the date:
I'd go from one Yahoo article to another and notice that regardless of the subject matter, the first user comment was always the same -- at least on AP articles covering the Israeli/Palestinian conflict. The comment that kept reappearing was posted by "Robert" and it was a one liner. "Hamas is now in control of the Gaza Strip after winning an election there against Abbas Palestinian Authority." That was it. Fair enough -- I've got no quarrel with the messenger or the message. But somehow that one comment generated an incredible 184 responses and, last I checked, readers had given it 3212 thumbs up and 2525 thumbs down.
I got a little curious about why Robert's one liner had generated so much controversy. I've written hundreds of articles and never got anywhere near that kind of attention. Frankly, I was full of envy. How did 'Robert' pull this off with one miserly line? Then I noticed the strangest thing: it was dated March 09, 2010. The comment was two months old and was the lead comment of 40,000 responses. That seemed a little high considering the fact that the AP article I was reading had only been posted for thirty minutes.
What were Yahoo and AP up to? The answer is simple; they were porting comments from one article to another and, in this particular case, they've been doing it for two months.
Oddly (and inexplicably) the author of that post, Ahmed Amr, does not link to Yahoo to show this, but it's not hard to find. Here's a story published on June 3rd, 2010 at 9:19pm. Yet, there's that same first comment, from March 9th, at 12:47am. And here's a story published on May 6th at 1:09 pm with the identical comments, also beginning with the March 9th comment. To let you see what they both look like before they change (and I'll explain in a second why I think they'll change) I've turned both of those pages into PDFs, which you can see below (you may have to either download or view at full screen and scroll to see the "comments" at the bottom):
I've also looked around and found really similar things on other stories. While Amr is suggesting there's something nefarious going on with the AP "manipulating" comments (and he specifically calls out the reporters from the AP who he believes are a part of this), I'm going to guess that this is more typical (embarrassing) incompetence on the part of Yahoo, rather than malice.
Take a look at the two links I put above to the Yahoo stories. The URLs (as found by a quick search for the comment string Amr mentioned in his post) are as follows:
Notice something similar? The last bit of the URL string is identical "/ml_israel_palestinian". The only difference is that the second URL, the story from May 6th, inserts two additional directories, with the top one being the date of publication. We already know that, due to a total disregard for the basic principles of the way the internet can and does work, that the AP limits its partners from hosting AP articles for very long. I believe on most sites you can host the articles for a month and then you need to take them down completely. With most sites, what happens is you get a 404 error or page not found (to this day, I can't figure out why they don't at least point you to a place where you can find the missing article). However, it appears that Yahoo decides to recycle the URLs in an attempt to make the URLs simple and understandable. So, any basic story about the Israeli Palestinian conflict might appear under that first URL. For all I know, by the time you're reading this, it's an entirely different story than the one that was published on June 3rd.
After the date of publication, breaking the basic principle of a link to a news story being a link to that news story alone, Yahoo moves the story to a new date-defined directory, and the original URL is freed up for the next story on that particular topic. If this seems stupid and confusing to users and destructive to the very idea of the "link economy" or valuing earned or passed links, you're right. But take that up with Yahoo and the Associated Press.
Of course, here's where the real level of tech incompetence comes in: It appears that Yahoo News' comment system doesn't understand that Yahoo does this. So, it associates the comments to that last bit of the URL string "/ml_israel_palestinian" and the same comments will appear every time that string is used as the final part of a URL string. It's bizarre that Yahoo would do this, but apparently, that's how Yahoo rolls.
Amr suggests that this is part of a planned bit of "corporate fraud" by Yahoo and the AP, perhaps to make it look like certain stories are getting a hell of a lot more comments than they are. He also suggests other conspiracy theories involving pro-Israeli operatives, saying that as far as he can tell, this only happens on AP stories concerning the Israeli/Palestinian crisis. I believe Amr didn't try very hard to find alternatives. On my very first attempt to find an example related to something entirely different, I found the identical behavior. I just picked a popular story that likely would have multiple stories over multiple days: the BP oil spill in the Gulf. Then I looked for an AP story hosted by Yahoo News... Bingo.
The first news story I found was published on June 3rd at 2:28 pm, but the first comment on the story? Why it's from May 1st at 2:06am. And the URL? The string ends with "us_gulf_oil_spill_947." You can find the identical comments on this story which was published May 21st, but ends with the string "us_gulf_oil_spill" suggesting that Yahoo's comment system also ignores numbers at the end of that final URL part in replicating its comments.
And here's another story about the White House's response to the oil spill. Published June 3rd at 11:57 pm. First comment? May 10, 2010 12:58 pm. URL string? "us_gulf_oil_spill_washington_9". And here's a story from May 17th with the identical comments at the end, with the closing URL string "us_gulf_oil_spill_washington_1." Yup, Yahoo seems to just match up comments with pretty simple URL hashes.
You can see all of that below as embedded PDFs:
So while it's easy and tempting to ascribe this to "manipulation" and suggest malice on the part of the AP or Yahoo or whoever else (Israeli operatives? Seriously?), it seems pretty clear that this is more due to technical incompetence on Yahoo's part, somewhat driven by the AP's ridiculous "delete this story after x days" licensing policies.
Update: The AP got in touch to make it entirely clear that this is entirely Yahoo's incompetence and not its own:
The Associated Press distributes news content to Yahoo! News, but the display of AP stories and the curating of comments are entirely up to Yahoo!
While undoubtedly true, in the comments we've heard from multiple people who work at news sites that license AP content, and they note that AP has a weird feed process, whereby it gives a simple slug like the ones used above, so that it can force update stories, often leading people to see stories totally change over the course of the day. This is clearly a Yahoo issue, but AP's policies don't help.
What else is she supposed to say? Of course she's going to slag the company that totally destroyed Yahoo at its own game, but the real question is why would anyone listen? Yahoo isn't exactly in the position of being able to say what Google should be doing, given that every single move that Yahoo has tried against Google has failed -- sometimes miserably. Bartz's comments get more amusing the further the article goes on:
"Google is going to have a problem because Google is only known for search..."
Well, I guess she should know. While she won't admit it, people still tend to think of Yahoo as search as well. And Yahoo spent so much time trying to diversify that if they don't think about Yahoo as search these days... they just think about Yahoo as being a confused jumble of sites that don't do much. In the meantime, despite her claims, it does appear that Google has expanded well beyond search. AdSense (which is about other content sites) makes them lots of money. Pretty much everyone I know uses Gmail, Google Calendar and Google Maps rather than the Yahoo equivalents. I'm trying to think of what other areas Yahoo has expanded into that have been successful for the company.
But the funniest statement of all:
"Google has to grow a company the size of Yahoo every year to be interesting."
I don't think that anyone judges Google based on how "interesting" they are. Is that the metric they use at Yahoo? Does that explain the $47 million she apparently made last year? Because she made Yahoo so interesting? Well, I guess it should be admitted that Yahoo is the company that is trying (and so far, failing) to patent "interestingness," so perhaps she's just urging Google to be interesting for the sake of a future patent fight? In the meantime, I would assume that, at Google, they judge the company based on how much money it makes -- and on that front, it appears to be cleaning Yahoo's clock on a pretty regular basis.
As for "growing a company the size of Yahoo every year," that becomes easier and easier as Yahoo gets smaller and smaller.
Christopher Best: Now what did I miss on DoJ and Department of Edu? Don't see what article (if any) you guys are referring to... silverscarcat: Well, since you guys can't seem to find it... http://legalinsurrection.com/2013/05/the-fire-the-government-has-mandated-speech-codes-on-all-campuses/ Mike Masnick: we have an article upcoming about that, which ssc appears to be undermining by revealing to all of you. ;) Rikuo: DEATH TO THE HERETIC /sarcmarc Christopher Best: Wow, just when I thought there was no way to make college campuses more intolerant... silverscarcat: Well, Mike, I'm glad that you had it in the upcoming posts, I called my senators and rep today about it right after I read it. Jay: Wow... Some Guy created a new form of acronym for the DMCA. Automated Rights Trampling System. I think I'll use ARTS from now on. silverscarcat: Well, this is so true. Someone needs to tell their Senators that they're fulfilling what Bin Laden said in 2001. "I tell you, freedom and human rights in America are doomed. The U.S. government will lead the American people in — and the West in general — into an unbearable hell and a choking life." - Osama Bin Laden, in sole post 9/11 interview dennis deems: http://arstechnica.com/security/2013/05/reporters-use-google-find-breach-get-branded-as-hackers/ Jay: @Dennis It seems that hacker has become the new witch hunt... silverscarcat: BURN THE WITCH! Man, AJ has lost his marbles. Imagine if the law was not only working against DRM but paywalls... bob and AJ would be foaming at the mouth. Rikuo: paywalls? I wonder - since the new Xbox requires anyone getting a used game to pay for access to it (a paywall) what will bob say about it? silverscarcat: I won't be getting one, I know that much. A lot of gamers aren't either. As long as Sony doesn't do the same thing, they'll be the clear favorites for this generation along with the Wii-U Jay: By the Gods is the Xbox One a horrendous DRM machine...