Virtual Video Store Clerks Take On The Geeks For Netflix Prize

from the wisdom-of-the-movie-geeks dept

We've been fascinated with the Netflix Challenge for a while now. That's Netflix's offer of $1 million to whomever can improve on their system for recommending movies by 10%. While there were a lot of early success stories in making improvements, all of the attempts seemed to bog down, making much more gradual improvements, but not getting close enough to hit that 10% mark. Earlier this year, we wrote about the surprising run up the leaderboard by a (previously) anonymous individual who approached the problem from a very different perspective, that of a psychologist, rather than a coder, which apparently was quite helpful in getting good results through a very different method.

Now some other folks are trying something completely different, relying on more of a "crowdsourcing" system, combined with a gaming element. They've set up a virtual video store, called Video Store Clerk, and set it up as a game for movie buffs. The game players act as a video store clerk, and can see how particular users rated three movies, and are then asked to predict how they would rate a fourth movie, with points given to correct answers. The idea is that they'll be able to use these crowdsourced predictions to create an even better model than the purely algorithmic model being worked on by various teams.

This reminds me of the research work by Luis von Ahn to do things like tag images via the "ESP Game." von Ahn has had numerous successes in creating fun casual online games, where the output data is actually very useful for taking on some sort of problem that is quite difficult to do algorithmically (such as identifying what's really in an image). The real question, though, is if movie recommendations really work that way as well. Perhaps I need to be a bigger movie buff, but so far, I'm not particularly good at figuring out how others would rank a movie. And, unlike the ESP game, frankly, the Video Store Clerk game isn't all that fun as currently designed. After playing it once, I had no desire to try again. Still, I'm intrigued by the different approach, and wonder if a more advanced (and more fun) version might be much more effective.

Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 13 Jun 2008 @ 6:54pm

    $1 million will buy a lot of hours of OLPC children's time. Just sayin'.

    reply to this | link to this | view in chronology ]

  • icon
    PaulT (profile), 14 Jun 2008 @ 3:54am

    @AC: Dumb reasoning. Just sayin'

    Seriously though, that's like saying that the $100m it takes to make the new blockbuster would be better spent on tackling child poverty in the 3rd world. Yes it would, but the point is that it's not *spent*, it's invested in an attempt to make a profit.

    Netflix isn't simply giving away $1m. It's using that as an incentive to get people to improve their service for them, and therefore increase their profits.

    reply to this | link to this | view in chronology ]

  • identicon
    Bill M, 14 Jun 2008 @ 4:25am

    @PaulT

    You missed his point. AC's saying to wire up all those OLPC-using kids to do the crowdsourced ratings/predictions. He's not making a point about solving poverty. It was actually pretty funny.

    And ironically, your post shows why it's so hard to predict what movie someone will like. People like different movies for different reasons, often quirky ones. What makes one bad movie a cult classic and another simply a forgotten throwaway? Some people "get" the point of a flick, and others don't. How about mood and setting? Ever watch a movie one ime and think it was great but the second time you saw it, you couldn't see what was so good about it?

    It's a very challenging problem to solve.

    reply to this | link to this | view in chronology ]

    • identicon
      Chuck Norris' Enemy (deceased), 16 Jun 2008 @ 6:38am

      Re: @PaulT

      I get the joke but in reality, how many kids in 3rd world countries are movie buffs?...have even seen a movie? Sorry for the depressing thought.

      reply to this | link to this | view in chronology ]

  • identicon
    Crowdish, 14 Jun 2008 @ 6:56am

    If someone succeeds in this challenge, I am sure that the $1m will be better spent than if Netflix had brought in Accenture or KPMG and paid them a $1m to "consult" on building a better system.

    Kudos to Netflix management on thinking outside the box.

    reply to this | link to this | view in chronology ]

  • identicon
    Sumbuddy Stoopid, 14 Jun 2008 @ 2:15pm

    Another amazing comment

    Netflix is cool, I love the service, and Blockbuster hasn't quit spamming me since I went with Netflix instead of them... The "Movies You'll Love" section is cool, I rate movies I've seen all the time, and often pick movies for my cue from there. Would be amazing if they make it better (with a game?) it's already pretty cool...

    reply to this | link to this | view in chronology ]

  • identicon
    sarah, 14 Jun 2008 @ 6:17pm

    librarians...

    Funny, this is what librarians do most of their day- Read-alikes for movies and books. I do it everyday and if my results are on target or not... forget programmers coding.

    reply to this | link to this | view in chronology ]

  • identicon
    Joe Smith, 15 Jun 2008 @ 10:32am

    Netflix and patents

    I have two nagging doubts about the Netflix prize:
    1. The leaders seem to all be achieving their results by tuning a combination of methods which may mean that there is going to be little in the way of generally useful algorithm or insight coming out of the process.
    2. There are a lot of patents already issued on collaborative filtering and there is a real chance that the winner will find himself being harrassed by a "patent licensing firm".

    reply to this | link to this | view in chronology ]

    • identicon
      This Tech, 25 Jun 2008 @ 4:09pm

      Re: Netflix and patents

      Probably a good thing you can't actually Patent Code, only the software that compiles it..

      what if someone went and patented from being used in HTML without paying that person..?
      It's the same with algorithms, I don't pay anyone for writting 2+2 = 4
      And that could technically be the result of an algorithm like:
      dim a as int
      dim c as int
      c = a & a

      output=a & " + " & a & " = " & c;

      ^--Now i know thats overkill to make a point but you can tell, this can't be patented, it's not possible to restrict a programming language in that way.

      reply to this | link to this | view in chronology ]

  • identicon
    Gene Siskel, 15 Jun 2008 @ 12:50pm

    The game would be a lot more fun if you got points for a near miss, especially as movie ratings can be so subjective. 10 points for a correct selection, 5 for being one star off, and nothing for more than that. Then maybe they could get folks to play more than once.

    reply to this | link to this | view in chronology ]

  • identicon
    Bill M, 16 Jun 2008 @ 9:48am

    I don't know enough about the specs of the OLPC to say this affirmatively, but they could potentially be using them to watch movies and become film buffs.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 16 Jun 2008 @ 9:10pm

    Some suggestions for the game maker:

    Don't end the game. Let people move up when they make enough points. That way you get more ratings. You can still look at the data to see if they're really bad predictors, and just throw the data away, but keep the game engaging.

    Let people skip a rating. If they're not familiar with any or most of the movies, you just get bad data by forcing people to pick a rating to get to the next one.

    reply to this | link to this | view in chronology ]

  • identicon
    Crowded, 11 Sep 2008 @ 2:19pm

    result?

    Has Netflix gotten close to the desired end result via this crowdsourcing experiment?

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Techdirt Gear
Shop Now: Copying Is Not Theft
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.