Why Is Google Punishing Sites That Publish Full RSS Feeds? [UPDATED]

from the not-good-at-all dept

Last year, we explained why full text RSS feeds make sense. You can read the whole thing, but the short version is that it makes it easier to read, and that means more people actually read the full stories and are willing to discuss them, share them and get others interested in reading as well. It just makes the reading experience that much better. We've always had full text RSS feeds, and we're not about to change that. However, it appears that Google may be punishing sites that have full text feeds. A concerned reader pointed us to the news that the magazine Mental Floss has reluctantly ditched its full text feeds because Google banned the site and told them the only way to get back in was to get rid of the full text feeds. Update: Matt Cutts from Google has responded in the comments and explained what happened. Turns out, despite the original post, it had nothing to do with full text RSS feeds, but the site was hacked. I'm glad that's been cleared up now (and thanks to the multiple Google employees who quickly responded to this post).

The "problem," according to Google, was that there were plenty of sites republishing Mental Floss's feeds, and Google's anti-spam algorithm supposedly uses that as an indication of spam. Of course, rather than figuring out which is the real site, it simply bans them all. This concerns me for a variety of reasons. The reason we publish a full text RSS feed is to make it easier for anyone to do what they want with our content -- even if it's republishing it. There are a bunch of sites that republish our RSS feed (some in the mistaken belief that such sites would get us upset at the "copyright infringement"). Those sites are harmless for the most part. Either they get no traffic at all, or they end up driving more traffic to us. That's great. But, it's a bit troublesome that Google might potentially disappear us from their entire index just because we publish a full text feed and someone else uses that feed exactly as they're supposed to.

I could understand if the deletion of Mental Floss from the index was simply a mistake, and upon being alerted to it, they restored the site. But the fact that Google's response was to tell Mental Floss to ditch the full text feeds is worrisome. What makes this even more ridiculous is that Feedburner, which is owned by Google, tells people that full text feeds are better. So, you have part of Google telling people to use full text feeds, and another part of Google punishing them for doing so.

19 Comments | Leave a Comment..

NYTimes Values Tracking Over Visitors; Keeps Freakonomics Feeds Crippled

from the partially-ineffective dept

It appears that despite the massive amount of reader backlash against the NY Times for switching the Freakonomics blog to partial RSS feeds rather than full feeds, the powers that be have decided to keep the feeds partially crippled. In the long explanation, Stephen Dubner references my own post about why full feeds actually increase page views, but basically says that it wasn't convincing enough for the money watchers at the Times. The reasoning behind the decision is pretty tortured, and Scott Karp does a fantastic job ripping it apart:

"Sure, you can't place tracking cookies in these people's browsers or serve behaviorally target ads. But HOW IS THAT BETTER THAN NOT REACHING THEM AT ALL??? The idea that publishers, under pressure from advertisers, can put the horses back in the barn and get people to consume content through channels that publishers fully control, just like in old offline monopoly media, is so reactionary that it really does amount to betting against the Internet. It's true that adoption of RSS is still relatively low, but when you take the case of the Freakonomics blog -- where MOST of the readers read it via RSS -- the idea that you could somehow change ALL of their behavior, i.e. force them to come to the New York Times, is just ludicrous. There's no other word for it. Really, what's the point of "partnering" with the Freakonomics blog only to alienate the vast majority of the readers? How is that creating value for advertisers? So you can show ads to the few angry, resentful readers who reluctantly come to the New York Times?"

The NY Times' reasoning reminds me of the ridiculous reasoning that many newspapers (including the NY Times!) gave for many years about why they had to put up registration walls to get to their content. They insisted that they needed much better data about their readers to give to advertisers -- not recognizing that in doing so they were getting a lot fewer readers and the data they were getting was often bogus anyway. It's this same mistaken belief that leads the NY Times to insist it needs partial feeds to give advertisers better tracking data -- even if it means fewer of the type of readers that the NY Times should specifically be aiming for. How could that possibly make sense?

12 Comments | Leave a Comment..

Why Full Text Feeds Actually Increase Page Views (The Freakonomics Explanation)

from the why-full-feeds-make-sense dept

Last week, the Freakonomics blog got some extra attention by moving the blog to the NY Times. Of course, the blog had been in support of the immensely popular Freakonomics book, but the blog has taken on a life of its own. What was interesting was how people reacted to the news. While there were a few congratulations thrown in, the vast majority of the comments on the blog when the news broke was to complain about the NY Times' decision to switch the RSS feed from full text to partial text, where anyone who wanted to read the whole thing would have to click through. This has kicked off yet another round in the debate with some thoughtful discussions about full vs. partial feeds. Techdirt, of course, offers full feeds and always has. This means that plenty of people who read this site absolutely never visit the site. We're fine with that for a variety of reasons (one of which being that our business model isn't dependent on page views or ad impressions).

However, in our experience, full text feeds actually does lead to more page views, though understanding why is a little more involved. Full text feeds makes the reading process much easier. It means it's that much more likely that someone reads the full piece and actually understands what's being said -- which makes it much, much, much more likely that they'll then forward it on to someone else, or blog about it themselves, or post it to Digg or Reddit or Slashdot or Fark or any other such thing -- and that generates more traffic and interest and page views from new readers, who we hope subscribe to the RSS feed and become regular readers as well. The whole idea is that by making it easier and easier for anyone to read and fully grasp our content, the more likely they are to spread it via word of mouth, and that tends to lead to much greater adoption than by limiting what we give to our readers and begging them to come to our site if they want to read more than a sentence or two. So, while many people claim that partial feeds are needed to increase page views where ads are hosted, our experience has shown that full text feeds actually do a great deal to increase actual page views on the site by encouraging more usage. It's the same thing that we've talked about in other areas of the content industry. Taking value away from users to try to force a specific action is almost always going to be less desirable than providing people what they want. So while Dubner and Levitt may have to argue with the NYTimes beancounters who will claim that partial feeds will increase revenue, they may want to use the lessons they learned from their own book to recognize that the opposite may be true. Full feeds can actually drive more traffic overall.

39 Comments | Leave a Comment..