Easily digestible tech news... http://www.techdirt.com/ en-us http://www.techdirt.com/images/td-88x31.gifhttp://www.techdirt.com/ Wed, 7 Nov 2012 12:42:00 PST Mike Masnick http://www.techdirt.com/articles/20121107/07473420959/why-press-is-getting-wrong-message-out-nate-silver-walloped-pundits-story.shtml http://www.techdirt.com/articles/20121107/07473420959/why-press-is-getting-wrong-message-out-nate-silver-walloped-pundits-story.shtml required class in school and that a combination of statistics and economics (the two go hand in hand) literacy (or lack thereof) is a major problem today, leading to numerous bad policy decisions. Finally, I've never been a fan (at all) of political punditry that focuses on the "horse race" aspect of politics. So, given all that, it has certainly been fun to follow the secondary storyline from last night -- which is how Nate Silver and his statistical genius "crushed" the pundits in predicting the election -- to the point that every single major press "pundit" was flat out wrong, and it looked like Silver had a perfect crystal ball. And, given how much Silver was attacked for being a "stats guy," (or for being biased, rather than neutral) you can certainly understand why it's tempting to wish he'd do something like Whitney McNamara's mock blog post: In many ways, I agree that yesterday was the "moneyball moment" in politics, in which the prognosticators were shown to be faulty, while the number crunchers were shown to be accurate. Hell, it was a much stronger example than the Moneyball case in baseball, which never had a "victory" quite as clearly aligned with the numbers.

Of course, if you look at what's happened to baseball since "Moneyball" and the success of the first statistical analysis guys, it should be a reminder that statistical prognostication is still about the probabilities -- and not about true predictions. And this is where the "suddenly-in-awe" pundits are still getting confused. They seem to think that Silver or other statistical modelers suddenly have a magic crystal ball with which they can predict the future. But probabilities and predictions are different, and Silver himself would likely admit (and, actually, did admit) that when you're dealing in probabilities, you're still going to be completely wrong some percentage of the time (he can even tell you what percentage of the time!) Even if the probabilities show a 90% likelihood that a certain event will happen, it still means that one time out of 10, you're going to be wrong.

Unfortunately, our brains don't deal that well with probabilities. We don't think in probabilities. Because we're dealing with a (mostly) binary situation, we assume that as soon as the probabilities tilt in our favor, it means that a "win" is somehow assured, and mentally, the probabilities turn into a prediction. It's very, very difficult for our brains not to think that way.

So I'm thrilled to see statistical analysis "win" over the moronic pundit-class who thinks that "storylines" or "momentum" (or, um, the ultimate in believing in anecdotes over data, "my friends see more yard signs" for one candidate) are valid methods for prognosticating. But it seems that the press, by going on to insist that Silver and his ilk are the new magic prognosticators, are missing the point just as much as those who thought the election could be predicted by political pundits.

Statistics is a tool for highlighting the probabilities. I'm sure that Nate Silver clones are going to be appearing a lot more on TV during the next major election cycles -- and I think that's a step forward. But now it seems like some people are expecting Silver and other stats guys to be right every time. And that's going to lead to backlash, just as the "failure" of Moneyball-type analysis to always get it exactly right resulted in some backlash in baseball. There will be data analysis in future election cycles -- likely from Silver himself -- that is wrong. That's the nature of probabilities. It will happen. And, unfortunately, people will then suddenly go back to arguing the opposite: that the stats geeks were "wrong."

But, as they say in the stats world, these are small sample size issues. Believing that statistical analysis is a perfect tool for predictions based on a single election is almost (though not quite) as weak as some of the traditional political punditry methods for predictions.

Hopefully, as with baseball, after a few years, the whole idea that these are entirely separate worlds will melt away. In baseball, every team now uses detailed statistical analysis as a tool, and most seem to understand that it suggests probabilities that help them find underexploited opportunities. But no one relies on it as a crystal ball that predicts the absolute future. Hopefully we'll reach that same sort of equilibrium in political analysis as well.

Permalink | Comments | Email This Story
]]> small-sample-sizes http://www.techdirt.com/comment_rss.php?sid=20121107/07473420959 Mon, 27 Feb 2012 17:00:00 PST Michael Ho http://www.techdirt.com/articles/20100209/1104298096/dailydirt-build-it-they-will-come.shtml http://www.techdirt.com/articles/20100209/1104298096/dailydirt-build-it-they-will-come.shtml

Enjoy your fifteen minutes of fame, Ed Weiland -- for being a bit less surprised than most about Linsanity. Weiland wrote in 2010: "... Jeremy Lin is a good enough player to start in the NBA and possibly star." [url]

Nate Silver called himself a forecaster, explained how he looked at baseball stats, and created the PECOTA system for evaluating MLB players. Baseball Prospectus bought the PECOTA system in 2003 and publishes its forecasts for all kinds of baseball fans and fantasy baseball leagues. [url]

The annual MIT Sloan Sports Analytics Conference has picked its top ten finalists for its Research Paper of the Year. These papers discuss various stats like "15% of basketball rebounds hit the floor before being collected." [url]

To find some other online challenges and games, check out what StumbleUpon has found to play. [url]  

By the way, StumbleUpon can also recommend some good Techdirt articles, too.

Permalink | Comments | Email This Story
]]> urls-we-dig-up http://www.techdirt.com/comment_rss.php?sid=20100209/1104298096 Thu, 6 Aug 2009 11:34:00 PDT Mike Masnick http://www.techdirt.com/articles/20090806/0215525784.shtml http://www.techdirt.com/articles/20090806/0215525784.shtml not to agree with Microsoft patent boss Horacio Gutierrez on very much when it comes to patents or Microsoft's patent strategy over the past few years. But I have to admit I'm fascinated by his plan to take the lessons of the book Moneyball and try to apply them to patents. Apparently, he's got a team of folks in Redmond, trying to put together data and stats the help judge the value of a patent. I'd be surprised if anything really accurate comes out of it (there are just too many variables and wildcards), but it is an intriguing idea. I wonder if he'll patent it...

Permalink | Comments | Email This Story
]]> fascinating http://www.techdirt.com/comment_rss.php?sid=20090806/0215525784