at the Carnival of Mathematics talks about "something-illion," or our inability to perceive large numbers. A billion, to most people, doesn't really sound very different from a trillion. Sometimes people even confuse 10 and 10,000
, like Obama yesterday, but let's call that the exception :)
The fact that we can't appreciate how small a 1-in-a-million chance truly is is why lotteries exist. I realized yesterday that it's probably also why prizes like the Netflix prize
are so effective. There are 20,000 registered teams, of whom a significant fraction probably secretly believe they have a shot at winning.
Even if people understand the insignificance their a-priori probability of winning, they overestimate their abilities in relation to others. In a survey of MIT undergrads, 95% believed they'd graduate in the top half of their class! (can't find a link, unfortunately.) The same effect has been observed in other domains.
Plus there's the free publicity for Netflix. Without the million dollars, this would be the most boring subject possible for a news article. I mean, can you imagine
Santa Clara, CA.
May 09, 2007
Netflix Inc. (NFLX) announced today that longtime engineer Warren K. Jennings has achieved a 7% improvement in the k-nearest-neighbors algorithm for collaborative filtering. The improvements came via a careful application of the Restricted Boltzmann Machines technique, which outperformed Singular Value Decomposition based methods. Netflix shares rallied strongly to gain 1.2 (5.3%) in response to the news to end the day at 22.6 dollars a share.
Didn't think so.
Which is not to say that there aren't sound reasons to participate in the Netflix prize. The data is hard to come by, so even without the money it's great for testing out algorithms. But surely, 20,000 teams and, at an average of 20 hours per person (half a work week) and 2 people per team, almost a million man-hours spent on it can only be explained by global cognitive malfunction.