You, Yes You, Are To Blame For Junk Stats

from the 9-out-of-10-people-disagree dept

It’s pretty easy to find examples of bad statistics, whether they stem from poor data collection or interpretation. But a writer over at the BBC thinks he’s fingered the real culprit: people who respond to surveys incorrectly. While on one level, his claim may have a little bit of truth to it, it really seems like an abdication of responsibility for the media, who all too often don’t check out the bogus stats they cite, or swallow manipulated data without much question. Furthermore, when many groups use push polls not just to collect data, but to influence opinions, it’s hard to blame the average respondent too much. In short, bad responses to survey questions could be a problem — but that doesn’t excuse the people collecting the data, and especially the media reporting it, from ignoring the issue. If they know the data’s inaccurate, why do they keep reporting it?

Filed Under: ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “You, Yes You, Are To Blame For Junk Stats”

Subscribe: RSS Leave a comment
Anonymous Coward says:

News flash: people get annoyed when you hassle them to do surveys they have no interest in. Of course they will answer with a load of BS. Of course if you just make it voluntary, only those with a vested interest will respond, and the results will be biased.
The obvious outcome of all this?…Media should stop concocting and reporting these stats as if they actually mean something.

Anonymous Coward says:

Junk stats? Junk stats are bad, running with them like they are the word of god is worse.

I think that is the best example of junk stats possible. Their survey group including almost 65% Vuse users, in a study published by, well, Vuse. Talk about misleading and meaningless junk stats. Yet Carlo, your boss ran with this story like it had come from on high, because it supports his (still unproven) concept that somehow people who download everything are still willing to pay for it, and more often than people who don’t.

Really, you should chat with Mike before you put these things up, because it would seem that the two of you just don’t agree.

Laurent GUERBY (profile) says:


One of the most commonly cited measure in economics is unemployment. But unemployment of 10% does mean that one out of ten person are without job, because of the concept of “active population” which is people with jobs plus people looking “actively” for a job. Each country typically defines 5 to 10 “actively looking” levels and publishes one as “official unemployment”. OECD takes all those and publishes a normalized, that is comparable between countries, unemployment level. However a little know fact …

From OECD normalized numbers for 2007 (latest available):

* men aged 25-54

employment divided by population
France: 88.3% (hence 11.7% jobless)
USA: 87.5% (hence 12.5% jobless)

normalized unemployment rate
France: 6.3%
USA: 3.7%

So with a slightly higher relative working population in this age/sex group France has … a 70% higher unemployment rate than the USA.

* women aged 25-54

FR 76.1
USA 72.5

FR 7.7
USA 3.8

So here we have clearly more women working in France (3.6 percentage point) relative to their population but unemployment is 102% higher in France!

* conclusion

Employment/population is an objective measure. Unemployment is an extremely subjective measure and so it’s easy to get very wide discrepancies as shown above.

* question

Any paper studying this discrepancy in OECD normalized numbers? For male 25-54 there’s no real reason not to work, so what are all those USA men doing?

* second question

What credit do you place on “empirical validation” of economic theories (and papers) who use unemployment by country to select the right policy?

EEJ (profile) says:


Another problem with survey accuracy is the design of the surveys. I’ve taken several surveys online (in exchange for payment or entries in sweepstakes) and have been very frustrated when questions are asked but don’t have enough viable answers available (or a way to explain your answer).

There doesn’t seem to be enough middle ground between some of the answers, and sometimes there isn’t a viable answer avialable, or the question doesn’t apply to me at all, based on previous answers I’ve given or my current situation.

Ben says:

My fault

I wonder how much I have affected advertising for our local paper. Because I refuse to divulge my information by creating an account, it frequently (every time I clear my cookies) asks me a few questions before I go to an article. I can just see the ad department trying to figure my habits as a female born in 1901 and living in zip code 90210.

porkster says:

Guilty as charged!

Thats me, giving out bad answers as usual.

If you ring me in the middle of my dinner (actually if you ring me at all and I haven’t requested it) of course your going to get a skewed results.

If your web site insists on a pop-up for a survey and it keeps poping up then yes, your going to get cr*p answers.

If you keep sending me e-mail to fill out a survey then you only have yourself to blame.

And one guarenteed lost customer.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...