Virtual Video Store Clerks Take On The Geeks For Netflix Prize

from the wisdom-of-the-movie-geeks dept

We’ve been fascinated with the Netflix Challenge for a while now. That’s Netflix’s offer of $1 million to whomever can improve on their system for recommending movies by 10%. While there were a lot of early success stories in making improvements, all of the attempts seemed to bog down, making much more gradual improvements, but not getting close enough to hit that 10% mark. Earlier this year, we wrote about the surprising run up the leaderboard by a (previously) anonymous individual who approached the problem from a very different perspective, that of a psychologist, rather than a coder, which apparently was quite helpful in getting good results through a very different method.

Now some other folks are trying something completely different, relying on more of a “crowdsourcing” system, combined with a gaming element. They’ve set up a virtual video store, called Video Store Clerk, and set it up as a game for movie buffs. The game players act as a video store clerk, and can see how particular users rated three movies, and are then asked to predict how they would rate a fourth movie, with points given to correct answers. The idea is that they’ll be able to use these crowdsourced predictions to create an even better model than the purely algorithmic model being worked on by various teams.

This reminds me of the research work by Luis von Ahn to do things like tag images via the “ESP Game.” von Ahn has had numerous successes in creating fun casual online games, where the output data is actually very useful for taking on some sort of problem that is quite difficult to do algorithmically (such as identifying what’s really in an image). The real question, though, is if movie recommendations really work that way as well. Perhaps I need to be a bigger movie buff, but so far, I’m not particularly good at figuring out how others would rank a movie. And, unlike the ESP game, frankly, the Video Store Clerk game isn’t all that fun as currently designed. After playing it once, I had no desire to try again. Still, I’m intrigued by the different approach, and wonder if a more advanced (and more fun) version might be much more effective.

Filed Under: , , , , ,
Companies: netflix

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Virtual Video Store Clerks Take On The Geeks For Netflix Prize”

Subscribe: RSS Leave a comment
PaulT (profile) says:

@AC: Dumb reasoning. Just sayin’

Seriously though, that’s like saying that the $100m it takes to make the new blockbuster would be better spent on tackling child poverty in the 3rd world. Yes it would, but the point is that it’s not *spent*, it’s invested in an attempt to make a profit.

Netflix isn’t simply giving away $1m. It’s using that as an incentive to get people to improve their service for them, and therefore increase their profits.

Bill M (profile) says:


You missed his point. AC’s saying to wire up all those OLPC-using kids to do the crowdsourced ratings/predictions. He’s not making a point about solving poverty. It was actually pretty funny.

And ironically, your post shows why it’s so hard to predict what movie someone will like. People like different movies for different reasons, often quirky ones. What makes one bad movie a cult classic and another simply a forgotten throwaway? Some people “get” the point of a flick, and others don’t. How about mood and setting? Ever watch a movie one ime and think it was great but the second time you saw it, you couldn’t see what was so good about it?

It’s a very challenging problem to solve.

Sumbuddy Stoopid says:

Another amazing comment

Netflix is cool, I love the service, and Blockbuster hasn’t quit spamming me since I went with Netflix instead of them… The “Movies You’ll Love” section is cool, I rate movies I’ve seen all the time, and often pick movies for my cue from there. Would be amazing if they make it better (with a game?) it’s already pretty cool…

Joe Smith says:

Netflix and patents

I have two nagging doubts about the Netflix prize:
1. The leaders seem to all be achieving their results by tuning a combination of methods which may mean that there is going to be little in the way of generally useful algorithm or insight coming out of the process.
2. There are a lot of patents already issued on collaborative filtering and there is a real chance that the winner will find himself being harrassed by a “patent licensing firm”.

This Tech says:

Re: Netflix and patents

Probably a good thing you can’t actually Patent Code, only the software that compiles it..

what if someone went and patented from being used in HTML without paying that person..?
It’s the same with algorithms, I don’t pay anyone for writting 2+2 = 4
And that could technically be the result of an algorithm like:
dim a as int
dim c as int
c = a & a

output=a & ” + ” & a & ” = ” & c;

^–Now i know thats overkill to make a point but you can tell, this can’t be patented, it’s not possible to restrict a programming language in that way.

Anonymous Coward says:

Some suggestions for the game maker:

Don’t end the game. Let people move up when they make enough points. That way you get more ratings. You can still look at the data to see if they’re really bad predictors, and just throw the data away, but keep the game engaging.

Let people skip a rating. If they’re not familiar with any or most of the movies, you just get bad data by forcing people to pick a rating to get to the next one.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...