Log in

No account? Create an account
Prize psychology - Arvind Narayanan's journal [entries|archive|friends|userinfo]

Prize psychology [May. 9th, 2007|03:01 pm]
Arvind Narayanan
[Tags|, ]
[Current Mood |sillysilly]

This post at the Carnival of Mathematics talks about "something-illion," or our inability to perceive large numbers. A billion, to most people, doesn't really sound very different from a trillion. Sometimes people even confuse 10 and 10,000, like Obama yesterday, but let's call that the exception :)

The fact that we can't appreciate how small a 1-in-a-million chance truly is is why lotteries exist. I realized yesterday that it's probably also why prizes like the Netflix prize are so effective. There are 20,000 registered teams, of whom a significant fraction probably secretly believe they have a shot at winning.

Even if people understand the insignificance their a-priori probability of winning, they overestimate their abilities in relation to others. In a survey of MIT undergrads, 95% believed they'd graduate in the top half of their class! (can't find a link, unfortunately.) The same effect has been observed in other domains.

Plus there's the free publicity for Netflix. Without the million dollars, this would be the most boring subject possible for a news article. I mean, can you imagine
Santa Clara, CA.
May 09, 2007

Netflix Inc. (NFLX) announced today that longtime engineer Warren K. Jennings has achieved a 7% improvement in the k-nearest-neighbors algorithm for collaborative filtering. The improvements came via a careful application of the Restricted Boltzmann Machines technique, which outperformed Singular Value Decomposition based methods. Netflix shares rallied strongly to gain 1.2 (5.3%) in response to the news to end the day at 22.6 dollars a share.
Didn't think so.

Which is not to say that there aren't sound reasons to participate in the Netflix prize. The data is hard to come by, so even without the money it's great for testing out algorithms. But surely, 20,000 teams and, at an average of 20 hours per person (half a work week) and 2 people per team, almost a million man-hours spent on it can only be explained by global cognitive malfunction.

[User Picture]From: ephermata
2007-05-10 06:51 am (UTC)
It isn't just Netflix. Most skill-based or idea-based contests are like this. The fine print usually says that the contest owner gets the rights to all entries, winning or not. So you end up with a huge disproportion between the amount of money the contest owner has to pay and the man-hours expended in total. Great deal if you're running the contest!

The flip side, of course, is that many of the contest entrants wouldn't be able to sell their work or time elsewhere. So it's not quite an apples to apples comparison. The owner also has to eat the cost
of evaluating each entry, although that can be spread out over time. In
the Netflix case, anyway, the cost to evaluate doesn't seem so bad.
(Reply) (Thread)
[User Picture]From: arvindn
2007-05-10 06:57 am (UTC)
Oh absolutely, I was using Netflix just as an example.

"The owner also has to eat the cost of evaluating each entry"

The beautiful thing, at least in this instance, is that data mining conferences are doing that job for them!
(Reply) (Parent) (Thread)