Another Reason We Need Open Government Data: To Avoid Information Asymmetries

from the database-of-intentions dept

Can the future aggregate actions of people be predicted from relevant sets of data that describe them? That, of course, is what Isaac Asimov's invented mathematical discipline of psychohistory was supposed to do. Some Japanese researchers claim to have made some progress towards that goal:

These guys have used ideas from statistical mechanics to model the behaviour of humans influenced by word-of-mouth interactions and advertisements. In this paper, Ishii and co derive a bunch of equations that they use to model the number of people who'll turn up to see a movie or visit an art show.
Inspired by this work, Nicklas Lundblad has written an interesting speculative piece about what the rise of predictability through the analysis of huge data sets might mean for society and openness. He notes that one of the "theorems" of psychohistory is that for it to be effective the data sets and the predictions derived from them must be kept secret from the populations involved – the idea being that if they were able to analyze that same data themselves, they might change their actions and thus nullify the predictions.

He points out that this creates a tension between predictability and openness:

There is an assumption here that is worth highlighting. And that is that for a democracy to remain open it can not be predictable by only a few. That is a complex and perhaps provocative assumption that I think we should examine. I believe this to be true, but others will say that our democracy already is predictable, in some sections and instances, only to a few and that they build their power base on that information asymmetry, but that it is reasonably open still. Maybe. But I think that those asymmetries are not systematic to our democracy, but confined to those phenomena, like stock markets, where they are certain to be important, but where they also do not threat the nature of democracy as such.

In summary, if we share the data and allow everyone to use it, then predictability goes down.
That, in its turn, is an argument for openness. If data held by a government, say, is released freely, anyone can explore its implications and then be able to modify their actions based on them, and thus escape being a statistical part of the predictability that would otherwise be implied by remaining in ignorance. As Lundblad writes:
If there is a conclusion here it seems to be to explore the amazing value of data under the imperative of openness to the the full extent possible to ensure that our societies gain from this new, fantastic age of data innovation, discovery and exploration that we are entering into, but never compromise on that openness in the pursuit of macro-social predictability.

Follow me @glynmoody on Twitter or identi.ca, and on Google+


Reader Comments

Subscribe: RSS

View by: Time | Thread


  1. icon
    btrussell (profile), 1 Feb 2012 @ 4:58am

    Re: Re: Raw Data Won’t Be Enough

    Keep in mind no one could afford to teach/learn math for another 150 years or so.

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Techdirt Gear
Show Now: Takedown
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.