Will A Psychologist Beat The Computer Science Geeks To The Netflix Prize?

from the moving-on-up dept

One of the key things that we expected would be useful when we created the Techdirt Insight Community, was the fact that our customers would be getting very different perspectives from the members of the community, based on an extremely diverse set of backgrounds, knowledge and experience. The idea was that while it's definitely useful to get the insight of someone who's gone through something similar (such as how to best position a product in a new market), sometimes you could gain a lot from someone who took a totally different view on it. It appears that same thing is happening in other areas as well. Wired has a fascinating story about how the various computer science/math geeks have had trouble advancing beyond a certain point in the Netflix challenge (the $1 million prize for improving Netflix's customer suggestion system by 10%), out of nowhere a management consultant with a psychology degree has been zooming up the charts using an entirely different method than the other leaders. The guy believes that all of the others have gotten into a bit of "groupthink," all using the same basic method -- while he's trying to apply some of his psychology background to the question in a way that he believes will work better in the long term. He hasn't yet passed the top teams, but he's been getting close. No matter how well he ends up doing in the end, it highlights, once again, the value of sometimes having someone from a totally different field take a look at the problem you're trying to solve.


Reader Comments (rss)

(Flattened / Threaded)

  1.  
    identicon
    Joel Coehoorn, Mar 3rd, 2008 @ 8:56am

    language translation

    There was a similar story a year or two ago about a used-car salesman who was able to vastly improve the field of language-translation. Rather than more traditional methods, he created a system that used a large library of text in the target languages (IE, something harvested the internet) as a source of naturally worded phrases. He then needed only a much smaller library of direct translation from the source language as a starting point.

    His method would use the small library as a starting point, and then repeatedly sift through the large library for matching phrases, scoring each one. It would shift a word at a time and try again. In the end he had a result much faster that was a lot more accurate. IIRC, the big thing was that is was able to accurately translate things like idioms that were traditionally impossible for automated systems.

    The point is he had no computer science background, and was able to open an entirely new direction that had been ignored by previous researches.

     

    reply to this | link to this | view in thread ]

  2.  
    identicon
    Alimas, Mar 3rd, 2008 @ 10:23am

    Totally Could.

    But I wonder how his different approach is effecting the directions all the other teams are taking?

     

    reply to this | link to this | view in thread ]

  3.  
    identicon
    Le Blue Dude, Mar 3rd, 2008 @ 10:35am

    Re: Totally Could.

    Odds are high the more skilled groups are taking some of the good points of it. And there's nothing wrong with that at all either: It means even more innovation as they try to make the psychologist's way work even better.

     

    reply to this | link to this | view in thread ]

  4.  
    identicon
    Anonymous Coward, Mar 3rd, 2008 @ 11:49am

    Medicine?

    Might sound stupid but I've always wondered if medicinal research groups have ever tried roping in consultants from other disciplines to help them.

    Take AIDS research for example (for the sake of argument we'll say that there's one team in the world working on a cure). Would it be at all useful for them to, say, bring an engineer on board, explain the basics, and ask for any ideas from them?

    Like I said, sounds stupid, but maybe the engineer would stumble on an answer, or set in motion a train of thought in one of the researchers that wouldn't have happened before.

    Gotta be worth a try.

     

    reply to this | link to this | view in thread ]

  5.  
    icon
    Mike (profile), Mar 3rd, 2008 @ 12:03pm

    Re: Medicine?

    Take AIDS research for example (for the sake of argument we'll say that there's one team in the world working on a cure). Would it be at all useful for them to, say, bring an engineer on board, explain the basics, and ask for any ideas from them?


    Oddly enough, I've been reading about the research that went into trying to cure AIDS in the late 80s early 90s as part of my research into medical innovation -- and there actually was some of this. One of the firms in question decided to go in a very different direction after realizing that all of the competitors were taking a single path towards a cure...

    But, yes, I think there's tremendous value in getting a fresh pair of eyes with a different background to take on some of these types of challenges.

     

    reply to this | link to this | view in thread ]

  6.  
    identicon
    Anonymous Coward, Mar 3rd, 2008 @ 12:07pm

    read the wired article. The psychologist is a computer geek!

     

    reply to this | link to this | view in thread ]

  7.  
    identicon
    Anonymous Coward, Mar 3rd, 2008 @ 1:15pm

    Ah...Another example of the Medici effect

     

    reply to this | link to this | view in thread ]

  8.  
    identicon
    Iron Chef, Mar 3rd, 2008 @ 1:56pm

    Interdisciplinary efforts

    How does one supplement an already intense field? Simple, look outside of the usual discipline. For example, Google likes to hire interdisciplinary folks, especially those from a physics background.

    Why? In the past large companies did scientific research, and some of this can be applied to computer science. Also, many similar challenges have been seen in alternative disciplines. Physics majors are comfortable with working in a theory-based environment too. So throw the idea on the wall, and see if it sticks. Much can be learned from Einstein.

     

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>
Follow Techdirt
A word from our sponsors...
Essential Reading
Techdirt Reading List
Techdirt Insider Chat
A word from our sponsors...
Recent Stories
A word from our sponsors...

Close

Email This