Business & Economics: September 2009 Archives
I'm a week late to this story that hits right in my field of expertise! Apparently the Netflix prediction contest has been won!
The company’s challenge, begun in October 2006, was both geeky and formidable: come up with a recommendation software that could do a better job accurately predicting the movies customers would like than Netflix’s in-house software, Cinematch. To qualify for the prize, entries had to be at least 10 percent better than Cinematch.The winner, formally announced Monday morning, is a seven-person team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel. The multinational team calls itself BellKor’s Pragmatic Chaos. The group — a merger of teams — was the longtime frontrunner in the contest, and in late June it finally surpassed the 10 percent barrier. Under the rules of the contest, that set off a 30-day period in which other teams could try to beat them.
That, in turn, prompted a wave of mergers among competing teams, who joined forces at the last minute to try to top the leader. In late July, Netflix declared the contest over and said two teams had passed the 10-percent threshold, BellKor and the Ensemble, a global alliance with some 30 members. Netflix publicly said the finish was too close to call. But Netflix officials at the time privately informed BellKor it had won. Though further review of the algorithms by expert judges was needed, it certainly seemed BellKor was the winner, as it turned out to be.
When the contest was announced I wasn't sure that a 10% improvement was possible given the dataset. Apparently I was wrong! Still, it was a close thing... if they had made the requirement 11% would they have still found a winner?
I love these sorts of contests. They're great for the issuer and for the winner because the cost is almost entirely borne by the many losers.
Yet the sort of sophisticated teamwork deployed in the Netflix contest, it seems, is a tricky business. Over three years, thousands of teams from 186 countries made submissions. Yet only two could breach the 10-percent hurdle. “Having these big collaborations may be great for innovation, but it’s very, very difficult,” said Greg McAlpin, a software consultant and a leader of the Ensemble. “Out of thousands, you have only two that succeeded. The big lesson for me was that most of those collaborations don’t work.”
Tens of thousands of man-years spent for Netflix's benefit, for a total cost to the company of under $2 million (including administrative costs). Slate runs some numbers.
Imagine if Netflix had paid all these math whizzes the prevailing wage—say, $100,000 a year. The company would have had to shell out more than $3 million for just one year of the top performers' time, and that's assuming it could've sussed out who the top performers were going to be. Of course, many of the programmers worked far longer than a year, some of them setting aside their primary occupations in order to work on the Netflix problem full-time. As Netflix CEO Reed Hastings admitted to the New York Times, "You look at the cumulative hours and you're getting Ph.D.s for a dollar an hour."But even that number discounts the contest's true benefits to Netflix. Had the company simply put out a help-wanted ad for software engineers, it probably wouldn't have been able to recruit many of the geniuses it found through the competition. That's because most of them already have other jobs. BellKor's members work for, among others, AT&T and Yahoo, and many members of the Ensemble are employed by the data-consulting firm Opera Solutions. The participants also spanned the globe. Netflix got submissions from people in more than 100 countries, and the winning team's members worked in New Jersey, Montreal, Israel, and Austria.
That's the main reason why I didn't enter the competition. The only way to really win is to be the one issuing the challenge.
Socialism is collapsing in Europe but we should definitely be trying it here in America!
A specter is haunting Europe — the specter of Socialism’s slow collapse.Even in the midst of one of the greatest challenges to capitalism in 75 years, involving a breakdown of the financial system due to “irrational exuberance,” greed and the weakness of regulatory systems, European Socialist parties and their left-wing cousins have not found a compelling response, let alone taken advantage of the right’s failures.
German voters clobbered the Social Democratic Party on Sunday, giving it only 23 percent of the vote, its worst performance since World War II.
Voters also punished left-leaning candidates in the summer’s European Parliament elections and trounced French Socialists in 2007. Where the left holds power, as in Spain and Britain, it is under attack. Where it is out, as in France, Italy and now Germany, it is divided and listless.
Socialism just doesn't work.
Also, viewing the current financial difficulties as a "challenge to capitalism" is moronic and displays a fundamental misunderstanding of the benefits of capitalism and the costs of burdensome regulation and government meddling. Capitalism promises efficient allocation of resources over time, not an endless trouble-free creation of wealth. Bumps in the road are par for the course, not a challenge to the capitalistic model.
Interesting observations about a Chinese Wal-Mart in Beijing.
They can’t compete on price in China, of course. So my guess is that they are trying to compete on selection, convenience, and customer service (thus all the sampling). That you can return stuff was very clear.
Also, they sell live turtles... for food or pets? Unknown.
(HT: BM.)
The new Mississippi River Bridge is being delayed due to conflicts over whether the contracting process is too sexist but not racist enough.
The Missouri Department of Transportation will delay awarding a contract to build the new Mississippi River bridge until it determines whether minority contractors are being fully utilized in the St. Louis region, the department announced Tuesday.The delay comes after minority contractors raised concerns this summer that the state hasn't been involving them in projects to the same level as it has women contractors. MoDOT is studying whether women and minority-owned businesses are being used to their fullest potential in transportation work. If the study shows that they are not, the state plans to ask the federal government for permission to have separate hiring goals for both groups in the $640 million bridge project.
Normally "women- and minority-owned businesses" are lumped together for preferential treatment, but minorities are complaining that sexism and racism should be separated because sexism is crowding out racism.
As a member of the group that these "goals" are specifically designed to exclude -- a white male -- I can't help but be amused. Here's an idea: hire the company that does the best work for the best value!
