According To WSJ, Google Not Just A 'Thief' But A 'Digital Vampire'
from the oh-please dept
There’s an absolutely huge business out there of folks trying to get more traffic from Google, called Search Engine Optimization. It’s a big deal. Traffic to your website is the lifeblood of most internet business models, and so any way to get more traffic is a good thing. Except if you’re in the newspaper business for some reason. Lately we keep seeing odd stories of newspaper business folks complaining about the fact that Google sends them traffic. The latest? Dow Jones CEO Les Hinton, who called Google a “digital vampire” claiming that it’s “sucking the blood” out of the newspaper industry (found via Mathew Ingram). He then goes on to suggest that at least some of this is the newspapers’ own fault for giving “Google’s fangs a great place to bite.”
So, uh, Mr. Hinton, here’s a suggestion: there’s a little thing called robots.txt. You can block Google from indexing your websites. Then everyone’s happy, right? That stops the bloodflow right there.
Except, perhaps the real issues is that, as everyone in every other business seems to recognize, traffic is important, and it’s up to the website receiving that traffic to capitalize on it. So, either Hinton doesn’t know this, or he’s simply lying. Neither one makes Dow Jones look particularly smart.
Filed Under: journalism, les hinton, newspapers, search engines, traffic
Companies: dow jones, google
Comments on “According To WSJ, Google Not Just A 'Thief' But A 'Digital Vampire'”
Google doesn’t always respect robots.txt
Re: Re:
citation ?
Re: Re:
Um, Google doesn’t respect improperly written robots.txt files.
One error is all it takes.
Re: Re: Re:
Nope. Google will take links from external sites and try hard to reach those pages even if they are blocked by robots.txt. Google also pushed sitemaps, but then ignores them and uses links from external sites to guide itself to content.
Re: Re: Re: Re:
you are so right. Web developers should not be expected to do their job properly and know how to properly create robots.txt
Also it’s not the plumbers fault for not knowing how to soder copper pipes together.
I know it took me an entire 15 minutes to learn how to block all webcrawlers from my online game club’s website that had fancy ummm… “sitemaps” and “forums” I guess a professional working at the WSJ shouldn’t be expected to do the same amount of work.
Re: Re: Re: Re:
citation ?
You can spew all the crap you want, but who is going to believe it without anything to back it up ?
Re: Re: Re: Re:
Well, then you can get in the offensive-
1. Google uses “Googlebot” as the useragent – block it
2. Google’s IP range is well known – block it
there, I got 2 solutions in under a minute. But that wont stop you from whining… will it?
Re: Re: Re: Security?
There are abuot 1001 different ways the newspaper could protect its content from spiders.. say store it in SQL database, or link it in from a directory outside of the HTTP server with php, asp, or if they are really worried they can use a C++ DLL and use double password authentication like my online banking (Google doesn’t seem to index that)or a Crypto image verification that way only authenticated users can have access to the content. Then when the spider tries to reach that link there isn’t any content on those pages because the spider isn’t logged in. The problem is they are trying to secure plain html with .htaccess and simple cookie authentication. The problem is with the web designer. Google’s spiders are not the only thing out there looking for free content behind paywalls.
Newspapers feel like it is not their job to have to proactively defend their content by writing robots.txt files except every other industry is responsible for protecting its own data online. They are the ones Posting the content, it is their job to protect it.
Re: Re: Re:
Google doesn’t do the thing they’re supposed to do when the person on the other end doesn’t write the file correctly? HOW DARE THEY!?
Re: Back that up ...
A little proof would be nice. Most server software really does work the way it is expected to … Most of Google’s apps work as expected. So show me an instance of Google not respecting a properly written robots.txt
Re: Re: Back that up ...
A little proof would be nice.
Proof? What proof? We don’t need no stinkin’ proof!
what is killing them are their registration and pay walls. If another site has the same news and is able to make a biz model out of not obstructing the news they win and you lose. just part of biz.
Maybe its time for the WSJ to hang it up and go home.
Re: Re:
not really.
I wouldn’t read WSJ even without a pay wall. Lots of people feel that way. They do a bad job as far as newspapers go, they’re on the way out as far as relevance.
WSJ Pay Wall
Hmmm,
google is bad, yet content that typically is only available behind their pay wall is purposely exposed for free to google’s spider.
Turn that capability off and enjoy your private life behind the pay wall, easy as that.
Digital Vampire...
That’s awesome. Google needs to find a way to own that title. 🙂
Re: Digital Vampire...
I vant to suck your binary number systems!
Do you hear that?
It sounds like the newspaper industry digging their own grave!
Re: Do you hear that?
they must have a hole the size of New York by now…
The newspapers, of course, are not complaining about being found via Google search. They’re complaining about their stories being aggregated into Google’s news service, which is different from Google search. But there’s no way to opt-out of Google news without opting out of indexing altogether, which would be suicide thanks to Google’s near monopoly on web search.
Re: Re:
But there’s no way to opt-out of Google news without opting out of indexing altogether
Incorrect. Straight from Google News:
“If you don’t want your site to be included in Google News, please let us know and we’ll remove it from our index.
Keep in mind that the removal process normally takes a few days and that your articles already included in Google News will expire after 30 days.”
Says nothing about being removed from all indexes. Just the index for Google News.
Re: Re: Re:
Ha-ha
Re: Re:
You mean the news service that puts about the first 40 words of the article and then links to the source? Yes that is something to complain about…not
Or it is the fact it also links to about a dozen other sources with the same article/subject allowing people to pick who’s version they want to read….lot more likely
Yellow Pages
For a century the yellow pages in the phone book is what drove people to businesses. Search engines have severely damaged that model. Yellow pages are basically a print media, and the though occurred to me why the yellow pages industry hasn’t been whining like the newspaper industry.
I think the answer is that the yellow pages industry embraced the Internet fairly early. Now when the yp-type companies call you they promise to get your listing prominently on the web, and then mention that they publish it in the local phone book almost as an afterthought. Newspapers had a choice early on and decided to live in their fantasy world and ignored the Internet. How different would things be if the newspapers had followed the model of the yellow pages industry and realized they do not have a divine right to be protected from change?
Re: Yellow Pages
Except the yellow pages business model has always been about data mining… going on the internet just made their model bigger and more profitable.
I think part of the problem
…is that these old guys seem to think there are still only one or two choices for news (occasionally owned by the same company!).
We will still find the same news from another source. The only difference is that you will loose the page views you would have received if you were linked from Google.
Disclaimer -I’ve had one or two WSJ subs in my life, usually when they were required reading for college classes.
I have a paywall on my site, it’s a big page in the front that says “if you haven’t paid to read my content, please don’t click on the following link”. But people and search engine today have no manners!
Re: #20 - AC
And what does the following link say?
“Click to exit”
Oh great.
I suppose next we can expect to see a poorly-written teen romance movie about Google now?
WSJ is a News Corporation (Fox, Rupert Murdoch) publication at this point. Their business content may still be first rate, however since it’s behind a pay wall I don’t know.
From what I’ve seen of their editorial content, the WSJ has become an echo chamber of capitalist protectionism (now there’s an oxymoron). The editorials are against regulation when it would force them to act more ethically, not that I’m sure there is a business – client ethics model at this point. They are for regulation when it protects their business models, which means they do not have to innovate to compete.
It’s amazing to me that all these high paid executives cannot figure out new business models to remain profitable. I guess executives earning 300 times the compensation of an average employee is no guarantee of excellence like those executives say it is.
There are models tying together print and electronic media that have the opportunity to work. I’ve seen some starts in this direction. I’ve seen some music industry business models that might also be applicable.
I wonder when these executives will start earning their compensation. Maybe this is a question stockholders should ask.
You know… I suppose those who made stone tablets and chisels would have said the same about newspaper.. Silly new-fangled stuff – don’t even have to chisel to put words on it!
Xenu™ Awaits
WSJ should just use the methodology Scientology® Uses.
What is killing the newspaper industry
What is killing the newspaper and TV news industries is your biased reporting and spin of the news. Quit blaming others for what is YOUR own fault.
IT takes money to make money
That’s right, when most companies feel technologically outgunned they hire someone to solve those problems for them. This is why people who can operate a “server” should be paid adequately for their skills. So many years have elapsed and did they ever complain about yahoo! news? The newspaper industry should have become google (and wouldnt that be strange?).
Other Profit Centers
News should be free, simple. They (Dow Jones/Wall Street Journal) should start optimizing their content better then they already are. Mr. CEO should hit up their advertisers and increase the pricing. I mean, I don’t know much about this (its just a thought) but don’t major magazines survive off of advertising revenue? I highly doubt Maxim is relying solely on publishing sales to survive. Why do they have to double dip? Maybe Dow Jones should stop releasing content online, and then someone could focus on getting some scholarly content syndicated. Maybe allow some critical thinking to start circulating through people’s blood. For a company that has been around for over 100 years, and is the largest financial news publishers in the world, maybe they should stop their bitching. I doubt Google is the reason for their problems. Blame it on the economy if you want to be that ignorant. I mean people go as far as to blame their weight on it.
-Bitter Google Fan
Mike is search advertising competitive like you imply?
Mike
With all due respect, the crux of your strong defense of Google depends on whether the search advertising market is competitive or not. I analyzed Google’s slogan — competition is but “a click away” — and found it can’t withstand close scutiny of the facts or logic. It is untrue and deceptive. See: http://www.precursorblog.com/content/what-one-click-away
Are you officially claiming the search advertisiing business is fully competitive?
Scott Cleland Precursor LLC
Re: Mike is search advertising competitive like you imply?
—With all due respect, the crux of your strong defense of Google depends on whether the search advertising market is competitive or not. I analyzed Google’s slogan — competition is but “a click away” — and found it can’t withstand close scutiny of the facts or logic. It is untrue and deceptive. See: http://www.precursorblog.com/content/what-one-click-away
Are you officially claiming the search advertisiing business is fully competitive?—-
Ummmm…are you high? Or did you just not read the story you were posting on? I tried applying some close ‘scutiny’ to the facts and logic of your post, and still fail to see what the competitiveness of the search advertising market has to do with Mike saying that if the newspapers wanted NOT to be listed in Google News, they DON’T HAVE TO BE, but it would be/is completely asinine to remove the traffic they get from Google. I know I shouldn’t feed the trolls, but you just challenged him on something that was not said, and insinuated that Google’s slogan about competition (which once again has nothing to do with the article) was something Mike is defending.
I just had to know…are you just high, are you a (very weak) troll, or are you just another shill? I can’t quite tell, so I figured I’d give you another opportunity to obfuscate the topic.
Re: Mike is search advertising competitive like you imply?
With all due respect, the crux of your strong defense of Google depends on whether the search advertising market is competitive or not.
What?!? Dude, I know AT&T pays you a ton to bash Google at every turn, but please, at least keep this relevant.
We’re not talking about *search advertising* at all. We’re talking about organic search.
Everyone knows that when the WSJ was bought out it ceased to be the WSJ that we all knew. Now it’s got a stupid CEO running it, and quite obviously they’re on the rocks.
See how they do without google
Google should remove all reference to WSJ for one day from their search results. I am pretty sure WSJ would change their tune as their traffic plummeted.
Wow
You have to admit he does raise some good points!
RT
http://www.complete-privacy.tk
Google Stolen My Money!
Google Adsense stolen my money around $400. They suddenly banned my website. Stay away from Google, they are High Risk investing! You will never know when your blog get banned.