Netflix $1 Million Award Shows The Value Of Collaboration... But Kicks Up New Privacy Questions

from the good...-and-bad dept

Back in July, we wrote about how the Netflix $1 million prize showed how much further research efforts could get by collaborating, rather than hoarding. Now that the official prize has been awarded, we're hearing even more about that point:

The blending of different statistical and machine-learning techniques "only works well if you combine models that approach the problem differently," said Chris Volinsky, a scientist at AT&T Research and a leader of the Bellkor team [which won]. "That's why collaboration has been so effective, because different people approach problems differently."

Indeed. There's plenty of research out there showing the leaps that are made in innovation when people with different approaches collaborate. Yet, with so much of a focus on "patents" representing "innovation," the opposite occurs. The patent system is all about hoarding information and making it harder to collaborate by putting tollbooths in the process. Many of the final "teams" involved a whole bunch of different approaches. Imagine if each one had a patent on their method. Think of how expensive that kind of innovation would be. Then, realize that there are plenty of technologies that face that exact problem today.

In the meantime, Paul Ohm is raising some serious questions about people's privacy on the new Netflix Prizes that are being announced. While Netflix claims that the data is anonymized, we've seen before that anonymous datasets are almost never anonymous, and in Netflix's case, the details are pretty bad:

Although I give Netflix a pass for its past privacy breach, I am astonished to learn from the New York Times that the company plans a second act:

The new contest is going to present the contestants with demographic and behavioral data, and they will be asked to model individuals' "taste profiles," the company said. The data set of more than 100 million entries will include information about renters' ages, gender, ZIP codes, genre ratings and previously chosen movies. Unlike the first challenge, the contest will have no specific accuracy target. Instead, $500,000 will be awarded to the team in the lead after six months, and $500,000 to the leader after 18 months.

Netflix should cancel this new, irresponsible contest, which it has dubbed Netflix Prize 2. Researchers have known for more than a decade that gender plus ZIP code plus birthdate uniquely identifies a significant percentage of Americans (87% according to Latanya Sweeney's famous study.) True, Netflix plans to release age not birthdate, but simple arithmetic shows that for many people in the country, gender plus ZIP code plus age will narrow their private movie preferences down to at most a few hundred people. Netflix needs to understand the concept of "information entropy": even if it is not revealing information tied to a single person, it is revealing information tied to so few that we should consider this a privacy breach.

Ohm also points out that this prize almost certainly violates the law:

Because of this, if it releases the data, Netflix might be breaking the law. The Video Privacy Protection Act (VPPA), 18 USC 2710 prohibits a "video tape service provider" (a broadly defined term) from revealing "personally identifiable information" about its customers. Aggrieved customers can sue providers under the VPPA and courts can order "not less than $2500" in damages for each violation. If somebody brings a class action lawsuit under this statute, Netflix might face millions of dollars in damages.

Additionally, the FTC might also decide to fine Netflix for violating its privacy policy as an unfair business practice.

It seems rather surprising that Netflix's lawyers did not consider this.

17 Comments | Leave a Comment..

What The Netflix Prize Tells Us About Innovation, Collaboration, Info Sharing And Game Theory

from the fascinating! dept

While there's lots of attention being paid to the fact that some team has won the Netflix Prize (it probably won't be announced who until September), there's an interesting side story that's worth noting -- which is how important collaboration was in breaking through. Plenty of studies have shown that innovation happens much faster when you have the free and open sharing of information (rather than having it locked up, say, by patents), as that mixture of different approaches and ideas allows for breakthroughs to come much faster (in fact, studies have shown that much of the success in Silicon Valley came from the free sharing of info across companies as people rapidly moved around).

And, in fact, that's exactly what happened with the Netflix Prize. The first "team" to break the 10% finish line, BellKor, was actually a merger of a few separate teams, allowing them to combine different pieces of different approaches to actually leap ahead. So, rather than trying to hoard the idea for themselves to claim the entire prize themselves, they realized it was better to team up to make the real breakthrough.

But, then a second interesting thing happened. Since the rules allowed another 30 days for other teams to offer up solutions that beat the first one, a bunch of other teams realized that it was in their best interest to team up as well, in order to leap-frog the original team. So they created the aptly named Ensemble -- and, again, the merger of various teams and different approaches allowed them to jump forward. It's not clear who actually had the best solution (both teams claim they did), but it's nice to see yet another clear example of the value of collaboration in innovation, against the standard myth of the lone inventor having a "flash of genius." It's also interesting to see the game theory aspect of the "loser teams" recognizing that they had to team up in order to catch up with the leader in the space.

34 Comments | Leave a Comment..

Did Someone Finally Win The Netflix Prize?

from the nice-work dept

For years, we've been fascinated by the Netflix Prize -- the $1 million offer to any team that can come up with a recommendation algorithm that is shown to be 10% better that Netflix's current recommendation engine. For years, different teams worked on the problem, and the early improvements were fast, but then progress seemed to stall out. Some different approaches were tested out which pushed the numbers up even further, but getting that last little bit has proven quite elusive... until now. Apparently, two of the leading teams combined efforts and have submitted an entry that breaks the 10% barrier for the first time. If the results are verified, then other teams have 30 days to submit an algorithm that performs even better. But, if they can't, then this team should win the $1 million... at which point they're supposed to tell the world how they did it. Seems like a much more innovation friendly approach than locking it up with a patent.

8 Comments | Leave a Comment..

The Napoleon Dynamite Problem Stymies Netflix Prize Competitors

from the love-it-or-hate-it dept

We've been covering the ongoing race to claim the $1 million Netflix Prize for a while now, highlighting some surprising and unique methods for attacking the problem. Every time we write about it, it appears that the lead teams have inched just slightly closer to that 10% improvement hurdle, but progress has certainly been slow. Clive Thompson's latest NY Times piece looks at the latest standings, noting that the issue now is "The Napoleon Dynamite problem."

Apparently, the algorithms cooked up by various teams seems to work great for your typical mainstream movies, but where it runs into trouble is when it hits on quirky films, like Napoleon Dynamite or Lost in Translation or I Heart Huckabees, where people tend to have a rather strong and immediate love or hate reaction to those films, with very little in-between. No one seems quite sure what leads to such a strong polar reaction, and no algorithm can yet figure out how people will react to such films, which is where all of the various algorithms seem to run into a dead end.

Some folks believe that's just the nature of taste. It really can't just be programmed like an algorithm, but takes into account a variety of other factors: including what your friends think of something, or even if you happened to go see that movie with certain friends. Basically, there are external factors that could play into taste, that isn't necessarily indicated in the fact that you may have liked some other set of quirky movies, and therefore you must love Napoleon Dynamite. In some ways, it makes you wonder if we're all putting too much emphasis on an algorithmic approach to the issue, and if other recommendation systems, including what specific friends think of a movie might be more effective. Of course, Netflix is hedging its bets. It's been pushing social networking "friend recommendation" features for a while as well.

21 Comments | Leave a Comment..

Will A Psychologist Beat The Computer Science Geeks To The Netflix Prize?

from the moving-on-up dept

One of the key things that we expected would be useful when we created the Techdirt Insight Community, was the fact that our customers would be getting very different perspectives from the members of the community, based on an extremely diverse set of backgrounds, knowledge and experience. The idea was that while it's definitely useful to get the insight of someone who's gone through something similar (such as how to best position a product in a new market), sometimes you could gain a lot from someone who took a totally different view on it. It appears that same thing is happening in other areas as well. Wired has a fascinating story about how the various computer science/math geeks have had trouble advancing beyond a certain point in the Netflix challenge (the $1 million prize for improving Netflix's customer suggestion system by 10%), out of nowhere a management consultant with a psychology degree has been zooming up the charts using an entirely different method than the other leaders. The guy believes that all of the others have gotten into a bit of "groupthink," all using the same basic method -- while he's trying to apply some of his psychology background to the question in a way that he believes will work better in the long term. He hasn't yet passed the top teams, but he's been getting close. No matter how well he ends up doing in the end, it highlights, once again, the value of sometimes having someone from a totally different field take a look at the problem you're trying to solve.

8 Comments | Leave a Comment..