You, Yes You, Are To Blame For Junk Stats
from the 9-out-of-10-people-disagree dept
It’s pretty easy to find examples of bad statistics, whether they stem from poor data collection or interpretation. But a writer over at the BBC thinks he’s fingered the real culprit: people who respond to surveys incorrectly. While on one level, his claim may have a little bit of truth to it, it really seems like an abdication of responsibility for the media, who all too often don’t check out the bogus stats they cite, or swallow manipulated data without much question. Furthermore, when many groups use push polls not just to collect data, but to influence opinions, it’s hard to blame the average respondent too much. In short, bad responses to survey questions could be a problem — but that doesn’t excuse the people collecting the data, and especially the media reporting it, from ignoring the issue. If they know the data’s inaccurate, why do they keep reporting it?
Filed Under: junk stats, stats
Comments on “You, Yes You, Are To Blame For Junk Stats”
examples of Bad
I’ve filed it along with examples of Bad Statistics because the issue is of chi-squared into a probability is not quite as easy as it may seem. Yes, we all know better now, but I don’t think its one or the other. all those weight loss ads are junk you are what you eat.plain and simple,Thanks
loving seem
adult
if you are collecting bad stats it the fault of the person collecting the stats. I notice a lot of times surveys are forced upon you when you don’t want to be filling one out. This is a perfect example of when you would get back answers that are way off base.
News flash: people get annoyed when you hassle them to do surveys they have no interest in. Of course they will answer with a load of BS. Of course if you just make it voluntary, only those with a vested interest will respond, and the results will be biased.
The obvious outcome of all this?…Media should stop concocting and reporting these stats as if they actually mean something.
Amazing spam
Re:”examples of bad” – that is the most intelligent bit of comment spam I’ve seen in a long time. Is that autogenerated, do you think?
I should point out that this guy has been writing a column on bad statistics on the BBC News website for months, and is also the author of a rather good book on bad stats called The Tiger that Isn’t. He’s not citing people responding incorrectly as *the* cause of bad stats, merely one of the many many factors covered in his regular column.
Just use another poll to analyse the first one
A couple of years ago a big Swedish newspaper ran an online poll on their web page with the following question: “Do you answer truthfully to web poll questions?”
Enough said.
Junk stats? Junk stats are bad, running with them like they are the word of god is worse.
http://latimesblogs.latimes.com/technology/2009/06/bittorrent-vuze-survey-purchasing-dvd.html
I think that is the best example of junk stats possible. Their survey group including almost 65% Vuse users, in a study published by, well, Vuse. Talk about misleading and meaningless junk stats. Yet Carlo, your boss ran with this story like it had come from on high, because it supports his (still unproven) concept that somehow people who download everything are still willing to pay for it, and more often than people who don’t.
Really, you should chat with Mike before you put these things up, because it would seem that the two of you just don’t agree.
Unemployment
One of the most commonly cited measure in economics is unemployment. But unemployment of 10% does mean that one out of ten person are without job, because of the concept of “active population” which is people with jobs plus people looking “actively” for a job. Each country typically defines 5 to 10 “actively looking” levels and publishes one as “official unemployment”. OECD takes all those and publishes a normalized, that is comparable between countries, unemployment level. However a little know fact …
From OECD normalized numbers for 2007 (latest available):
* men aged 25-54
employment divided by population
France: 88.3% (hence 11.7% jobless)
USA: 87.5% (hence 12.5% jobless)
normalized unemployment rate
France: 6.3%
USA: 3.7%
So with a slightly higher relative working population in this age/sex group France has … a 70% higher unemployment rate than the USA.
* women aged 25-54
E/P
FR 76.1
USA 72.5
unemployment
FR 7.7
USA 3.8
So here we have clearly more women working in France (3.6 percentage point) relative to their population but unemployment is 102% higher in France!
* conclusion
Employment/population is an objective measure. Unemployment is an extremely subjective measure and so it’s easy to get very wide discrepancies as shown above.
* question
Any paper studying this discrepancy in OECD normalized numbers? For male 25-54 there’s no real reason not to work, so what are all those USA men doing?
* second question
What credit do you place on “empirical validation” of economic theories (and papers) who use unemployment by country to select the right policy?
Surveys
Another problem with survey accuracy is the design of the surveys. I’ve taken several surveys online (in exchange for payment or entries in sweepstakes) and have been very frustrated when questions are asked but don’t have enough viable answers available (or a way to explain your answer).
There doesn’t seem to be enough middle ground between some of the answers, and sometimes there isn’t a viable answer avialable, or the question doesn’t apply to me at all, based on previous answers I’ve given or my current situation.
Carlo: You, Yes You, Are To Blame For Junk Posts
My fault
I wonder how much I have affected advertising for our local paper. Because I refuse to divulge my information by creating an account, it frequently (every time I clear my cookies) asks me a few questions before I go to an article. I can just see the ad department trying to figure my habits as a female born in 1901 and living in zip code 90210.
Guilty as charged!
Yep,
Thats me, giving out bad answers as usual.
If you ring me in the middle of my dinner (actually if you ring me at all and I haven’t requested it) of course your going to get a skewed results.
If your web site insists on a pop-up for a survey and it keeps poping up then yes, your going to get cr*p answers.
If you keep sending me e-mail to fill out a survey then you only have yourself to blame.
And one guarenteed lost customer.