Want to know how to take a bad situation and make it significantly worse? Check this out. We were just discussing how the website DailyKos was going to sue its former pollster after an investigation turned up fairly compelling evidence that the data it presented was either faked or manipulated. Pretty quickly, a few folks sent over the news that Nate Silver, at the always fascinating FiveThirtyEight, (though, frankly, I miss his Baseball Prospectus work...) had received a cease & desist from R2K's law firm: As Silver notes in his post, the letter, in part, credits statements made on the Kos website to Silver, which is just really bad lawyering. On top of that, it takes issue with some of Nate's Twitter messages, such as one where he clearly opines that R2K's results have "maginal quality." That's a clear opinion statement, and not defamatory at all. The whole thing is bizarre and clearly makes R2K look even worse -- especially considering the claims on Kos that it was given weeks to defend its research and chose not to for whatever reasons. But as soon as the reports come out demonstrating the likely problems with the research, R2K rushes out the cease-and-desist letter... and to a third party site that was just reporting on what folks on Kos had found? Yikes.

A bunch of folks have sent in the ridiculous claims from a study done by the Japanese Computer Entertainment Suppliers Association (CESA) suggesting that unauthorized copies of handheld games have cost the industry $41.5 billion over a five year period. Like similar bogus reports, these numbers have little, if any, connection to reality. I was considering skipping posting about it altogether, given that it's really just the same old thing, but Dark Helmet sent in a nice starter list on why this study is completely bogus:

1. Every infringing download is counted as a lost sale
2. CESA took the numbers for Japanese handheld game piracy and multiplied it by four to get the worldwide numbers, because CESA "believes" Japan represents 1/4 of the market
3. Deviations in piracy levels in different world regions were not taken into account
4. Pricing for games per unauthorized copies were ALL based on the initial release price, not taking into account pricing fluctuations of games over time

In addition, I'd point out that the study apparently picked which sites to cover somewhat at random and may not accurately portray (in any way) the number of downloads. Also, there's no attempt to look at whether or not those who get unauthorized copies of some games end up buying other games as well (i.e., do they still spend the same amount otherwise -- just on different offerings?). Basically, there's very little in this study to take seriously.

Like the swallows returning to Capistrano, every May is marked by the release of intentionally misleading and bogus stats from the Business Software Alliance (or, more accurately, the Bogus Stats Alliance) concerning software "piracy." As with every other year, the stats are compiled by IDC, despite the fact that even IDC has admitted in the past that the BSA is purposely misrepresenting their findings. You would think, at some point, that IDC would stop providing numbers that are blatantly misrepresented... but I guess if the money's green, IDC will give you the numbers you want.

We've been covering these bogus stat reports for many years, providing a detailed look at how misleading the stats are, and pointing out how many in the press simply parrot the numbers without question. Two years ago, a VP at the BSA (who's now working at the Justice Department, of course) was kind enough to call me to try to explain the BSA's numbers (along with a PR person and a representative from IDC). When I challenged them on the whole "one copy equals one sale thing" they insisted that their numbers showed such a claim was accurate.

Thankfully, in the past few years, more and more in the press have started to sound skeptical of the BSA's numbers -- but it's still a minority. Last year, the BSA did a neat trick in getting some publications to run stories about the numbers, while then saying don't pay attention to the numbers as a way of fending off anyone who criticizes how incredibly misleading the numbers are.

This year, you would think the press would be extra skeptical, given that just a few weeks ago, the GAO report pointed out that these stats are totally baseless (and yes, the BSA was one of the reports they criticized). But, looking through the press coverage, most seem to be just reporting the ridiculous claims such as "$50 billion" in "losses" due to file sharing. Lots of the reports focus on "local" findings -- with local publications just covering the claims in that local country (for example, coverage in Malaysia, China, the Persian Gulf, the UK, Korea, India, Canada, etc.). Of course, in the past, even those numbers have been called into question. Last year, after people took a more detailed look at how "piracy" stats were counted in Canada, it came out that the findings were based on pure guesses. No one in Canada was surveyed. They just made up the data.

So, really, you would think that the mainstream press would at least put up some semblance of skepticism in seeing these same bogus numbers released yet again, with no serious changes to the methodology. But, for the most part the reports just repeat the BSA's talking points. Looking through the press reports, it's tough to find coverage that expresses any skepticism at all. They just repeat the numbers -- the same numbers the US government just said were bunk -- as if they were pure fact. Just a sampling: the AFP, the BBC, ComputerWeekly, Computerworld, the UK Press Association, Network World, eWeek and many, many others.

Business Week gets credit for being one of the very, very few sources that at least mentions the GAO's findings, though it does so in one sentence at the very bottom of the article. The National Journal also mentions the GAO report -- though neither seemed to ask (or get any responses from the BSA) to this rather crucial point. ITWire, at the very least, points out that the study is basically made up, noting that:

"estimates of piracy rates are based mostly on inferences and the 'gut feeling' of the BSA's research organisation IDC;

But that's about all I could find. For the most part, the press -- the one's we're told are supposed to be asking all the "tough" questions, simply reposted the BSA's press release as fact. You would think that, given that this report has come out every year for the past seven years -- and the methodology has been debunked widely time and time again -- this year by the US government -- and that the report itself admits that many of the numbers are based on hunches and guesses, that the press would stop reporting them as fact. Wishful thinking, I guess.

So, the MPAA's latest target for crackdowns on camcording appears to be India. You may remember that it went through a series of questionable claims about camcording in Canada and the US, where if you looked at the math, none of it added up. Apparently, the same thing is happening in India and the reporters at the Associated Press don't feel the need to investigate bogus Hollywood numbers. In this AP report about the MPAA's new "crackdown" on camcording in India, the reporter says that camcording is to blame for 90% of "pirated" movies:

A year in the making, the coalition to fight film piracy in India will work with movie theaters to crack down on camcorder piracy -- the source of 90 percent of all pirated DVDs -- with police to tighten enforcement, with Internet service providers to fight Internet piracy and with politicians to create more effective laws.

With these big professional reporters, you might think they would try to fact check a claim like "90% of all "pirated" DVDs come from camcorded movies." They might have trouble doing that, because the actual research suggests something quite different. A study that we wrote about a few years ago found otherwise. Specifically, it found that "77% appear to have been leaked originally by industry insiders."

But, of course, we need to save the AP, because they do real fact checking, right?

The copyright industry lobbyists absolutely love to throw around the bogus and debunked stat that copyright contributes $1.52 trillion to the economy. That number is derived by taking any business that kinda sorta maybe touches copyright (including things like furniture and jewelry) and then assuming that all of the revenue they make is entirely due to copyright. Yes, that's ridiculous. But, if the copyright lobbyists are going to use such bogus methodology to push their agenda, it seems only fair for those on the other side to use the same methodology. Last week, we wrote about a biased editorial by two newspaper industry lawyers in the WSJ (who failed to note the conflicts of interest), claiming that Google violated copyright law, and attacked the concept of fair use.

In response, Ed Black, from the Computer & Communications Industry Association wrote a letter to the editor highlighting those lawyers factual mistakes as well as the importance of fair use throughout the industry (thanks to Yano for sending this in). Most of the (short) letter discusses all the wonderful things that fair use allows, and then has this wonderful line at the end:

Businesses dependent upon exceptions to copyright contribute $2.2 trillion to the U.S. economy. They are responsible for one in eight jobs, for a total payroll of $1.2 trillion in 2006. Fair use is serious business; it is the glue that holds the Internet and new technology together. It is worth protecting.

This is fantastic. Of course, the number is just as bogus as the $1.52 trillion used by copyright maximalists, but I think that if they're going to use their methodology to make such ridiculous claims, it's only fair to do the same for the contributions to the economy of exceptions to copyright, and as the CCIA clearly demonstrates, the businesses that rely on weaker copyright contribute significantly more to the economy than those that rely on copyright. Thus, by the copyright maximalists own logic (and numbers), shouldn't we be fighting to expand the exceptions to copyright law?

Statistics lauding the growth of the Internet in China have become so commonplace as to inspire yawns, despite breathless press reports of hundreds of millions of Chinese going online and signing up for the 'net. With the Chinese Government declaring that their internet population surpassed the US last year, it would seem that the real opportunity for expansion and growth online is not in the West, but somewhere behind the Great Firewall of China. Cue the ads for Chinese Web Hosting, Chinese Industry Liaisons, and the omnipresent legions of Chinese business agents.

Many Western technology companies have heeded that call, but have found themselves cast onto the rocks of Chinese shores including companies like Microsoft, Google, Cisco, eBay, and Yahoo! The massive markets just never seem to have materialized in the Orient for these giants, or when success has loomed on the horizon the murky Chinese bureaucracy has stepped in to snatch defeat from the jaws of victory. Partnerships have vaporized overnight, and (particularly in the case of Cisco) core Intellectual Property has been outright stolen, reverse-engineered, or redistributed. Perilous waters, indeed.

So it was with this skepticism that my friend Gersham viewed the latest piece of propaganda emerging from our friends in China that we have now reached the new height of 338 million Chinese Internet users; a 13 percent increase since the end of 2008, and just about exactly one quarter of the country's population. All of this, of course, seems to have been tabulated and distributed by the slightly inaccurately-acronymed Chinese Internet Network Information Centre (CNNIC) which, by its own admission,takes orders from the Ministry of Information Industry (MII) to conduct daily business. In fact, google Chinese Internet Traffic and you'd be hard-pressed to find data that did NOT originate from the CNNIC. Hmm. Call me a cynic.

It is likely difficult for most (any) of us to corroborate or even conceptualize these high numbers, but it seems suspicious nonetheless, particularly from a country whose median income is around $3400 and whose Per-Capita GDP is ranked 104th, right behind Armenia. In trying to substantiate this, once can point to Alexa's site rankings which currently reveal that 3 Chinese-language web sites rank in the Top 20: Search Engine Baidu (#9), IM chat and portal QQ (#14), and portal Sina.com.cn (#18). Sounds good, right? But look closely at the rankings. Baidu, an undisputed leader in Search for China, reaches 5.73% of the internet populace, whereas Google.DE (#13) reaches roughly 3% of global internet users while servicing German, Swiss and Austrian users exclusively. Combine the populations of these three countries and they don't even add up to 100 million people.

Gersham pointed me toward the Firefox Download Stats, where, as of this writing, Germans have made 4,948,666 downloads of various Firefox versions compared to only 672,972 for China. Again, Germany has a population of 82Million vs. 1.3Billion in China. As a control, Americans have downloaded Firefox 7,959,727 times as of this writing. Do the Chinese really just prefer Internet Explorer?

In January 2009, Comscore measured the Chinese internet audience at closer to 180 Million users, still an impressive 18% of the Internet population. This site quotes murky Nielsen Online data pegging Chinese Internet Users at roughly 300 Million. Beyond these hearsay reports, empirical measurements are difficult to come by.

So, let’s throw up our hands and try to reverse-engineer the data using published stats. According to June 2009 data from Comscore, Google has captured 65% or so of US Search Traffic. This made it the #1 web site in the world, with 157 Million US Visitors in June, according to Comscore. In the Chinese Market, Baidu has captured 73% of Chinese search, with Google in the Number Two spot. Yet Baidu.com barely moves the needle by comparison, according to compete.com, alexa.com, and others, hitting roughly 600,000 unique visitors per month globally. High-side estimates of the Internet's penetration in the US peg it at 72.5% of the populace, or about 220 million. This makes the data on Google's penetration vs the addressable market reasonably accurate (71% if you do the math). Following this logic, if Baidu in fact has 73% of China's purported 338 Million users, it should be ranking as the #1 web site by far, with 246 Million unique visitors per month. In fact if any of this data were true, then Chinese sites should occupy at least 4 of the Top Ten global web sites.

Whatever your opinion of Compete's and Alexa's relative methodologies, it's impossible to reconcile anything even close to the numbers coming from the Chinese Government. If that isn't good enough for you, let's turn to profits. While serving what was allegedly the world's largest internet audience, Baidu appears to be tracking to earn about $500 Million in revenue this year. Google's revenue appears to be tracking to about $23 Billion for 2009 with its pithy 157 Million unique visitors. Any way you slice it, if China's internet userbase is as large as Beijing says it is, and if Baidu's market share of that audience is what it's widely purported to be, then both the number of uniques reported by external traffic sites and the revenues reported by the public company that owns Baidu should be exponentially greater.

These stats seem to either indicate that Chinese do not use search very often, or that there just aren't too many of them heading out into the wilds of the Internet. Either way, statistics emanating exclusively from bureaucratic sources within Beijing, particularly those which seem to fly in the face of all other external metrics, are not to be believed. The thesis of this post is not to suggest that China is NOT a massive opportunity for online properties and other technology purveyors, it is simply an attempt to point out that, like in a lot of cases in dealing with the People's Republic of China, things are not what they may seem. Pay no attention to the man behind the curtain.

Cross-posted from IanBell.com.

Every day, we see greater and greater evidence that too stringent copyright laws are harming creative efforts rather than encouraging them. The evidence is all around us and growing. But, very few in the government recognize this. They don't actually look at the real research on these things. Instead, they accept as gospel the ridiculous debunked research that comes out of the industry lobbyists who benefit the most from such protectionism that limits real and meaningful competition. And no one calls them on it. Take, for example, this Internet News report on how lobbyists for the music, movie and software industries all got together with Commerce Secretary Gary Locke, and talked up a new and misleading study from the International Intellectual Property Association that talks up the importance of copyright.

Unfortunately, the report has many, many problems. It mistakenly assumes that anything covered by copyright (i.e., any sort of creative output) is created solely because of copyright. Plenty of content/software/etc. is not created because of copyright -- but this study assumes that it is and that it needs to be protected -- even though for much of it, that's unlikely to be true. That leads to a very distorted picture, because it makes you automatically think that granting more copyright is better, even if much of that output would have been done anyway -- and even if there would have been more output without copyright. It falsely assumes that the production is because of copyright, rather than just covered by copyright.

But, with such bogus stats -- and gullible gov't officials -- it's easy to push for more protectionism and stronger copyright laws as being necessary to "protect" these industries. The industry officials went on and on at the get together, falsely blaming "piracy" for the music industry's troubles -- ignoring that real studies (i.e., not lobbyist-backed ones) are telling quite a different story. The overall industry is actually thriving -- in part because of the better ability to distribute and promote content and software at greatly diminished costs (often free). Ignoring these benefits, and treating them all as harm is a huge mistake.

But, it's these lobbyists and their bogus numbers that have the ears of our elected officials. What a shame. No one was invited to suggest the countering viewpoint or to show why the industry's numbers were clearly false. No one was there to discuss how stronger copyright actually harms creative output, and to show how these very industries have repeatedly fought against and shut down innovations that could have helped the economy. No one was there to question why politicians would simply accept industry-financed studies. And, no reporters seemed to have asked these questions, either. It's just politics and business as usual.

It's pretty easy to find examples of bad statistics, whether they stem from poor data collection or interpretation. But a writer over at the BBC thinks he's fingered the real culprit: people who respond to surveys incorrectly. While on one level, his claim may have a little bit of truth to it, it really seems like an abdication of responsibility for the media, who all too often don't check out the bogus stats they cite, or swallow manipulated data without much question. Furthermore, when many groups use push polls not just to collect data, but to influence opinions, it's hard to blame the average respondent too much. In short, bad responses to survey questions could be a problem -- but that doesn't excuse the people collecting the data, and especially the media reporting it, from ignoring the issue. If they know the data's inaccurate, why do they keep reporting it?

Recent court rulings have made it clear that sports leagues have overreached (by a long shot) in trying to force online fantasy sports sites to license player info. The courts have pointed out that player names and stats are factual information, not subject to copyright. Now, this has resulted in many fantasy sports sites to skip renewing any licensing deals. The NFL Players Assocation, despite already having lost such a case, still went to Yahoo and threatened it with a lawsuit over this issue. It appears Yahoo decided to be proactive and sued for a declaratory judgment that its uses of player info was not in violation. It's an aggressive move by Yahoo -- but it also shows (reasonably) that the company believes that it's likely to win (and, perhaps, that it was worried about whatever district the NFL PA would have filed its own lawsuit in). Either way, it's yet another chance to remind sports leagues that they don't get to copyright factual information.

There are plenty of instances of misleading and otherwise bad stats being used by anti-piracy groups, like the recent BSA numbers from Canada that were basically made up. Now, a group from the UK is saying that piracy costs that country's economy tens of billions of pounds. It makes the same mistake as plenty of other studies before it: counting every instance of piracy, or perhaps even just the availability of copyrighted material on file-sharing networks, as a lost sale. It's fallacious to assume that every single person that downloads a piece of content, or simply has access to it for free, would pay for it if the free version wasn't available. Furthermore, any study like this that says an entire economy is being harmed by X amount of money because of piracy is pretty much bogus. This money that's supposedly being lost because of piracy isn't being lost by the economy, as undoubtedly it's being spent elsewhere. It's not being flushed down the toilet or turned into ether, it's just not ending up in content companies' bank accounts.