Science, Technology & Health: September 2009 Archives
I'm a week late to this story that hits right in my field of expertise! Apparently the Netflix prediction contest has been won!
The company’s challenge, begun in October 2006, was both geeky and formidable: come up with a recommendation software that could do a better job accurately predicting the movies customers would like than Netflix’s in-house software, Cinematch. To qualify for the prize, entries had to be at least 10 percent better than Cinematch.The winner, formally announced Monday morning, is a seven-person team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel. The multinational team calls itself BellKor’s Pragmatic Chaos. The group — a merger of teams — was the longtime frontrunner in the contest, and in late June it finally surpassed the 10 percent barrier. Under the rules of the contest, that set off a 30-day period in which other teams could try to beat them.
That, in turn, prompted a wave of mergers among competing teams, who joined forces at the last minute to try to top the leader. In late July, Netflix declared the contest over and said two teams had passed the 10-percent threshold, BellKor and the Ensemble, a global alliance with some 30 members. Netflix publicly said the finish was too close to call. But Netflix officials at the time privately informed BellKor it had won. Though further review of the algorithms by expert judges was needed, it certainly seemed BellKor was the winner, as it turned out to be.
When the contest was announced I wasn't sure that a 10% improvement was possible given the dataset. Apparently I was wrong! Still, it was a close thing... if they had made the requirement 11% would they have still found a winner?
I love these sorts of contests. They're great for the issuer and for the winner because the cost is almost entirely borne by the many losers.
Yet the sort of sophisticated teamwork deployed in the Netflix contest, it seems, is a tricky business. Over three years, thousands of teams from 186 countries made submissions. Yet only two could breach the 10-percent hurdle. “Having these big collaborations may be great for innovation, but it’s very, very difficult,” said Greg McAlpin, a software consultant and a leader of the Ensemble. “Out of thousands, you have only two that succeeded. The big lesson for me was that most of those collaborations don’t work.”
Tens of thousands of man-years spent for Netflix's benefit, for a total cost to the company of under $2 million (including administrative costs). Slate runs some numbers.
Imagine if Netflix had paid all these math whizzes the prevailing wage—say, $100,000 a year. The company would have had to shell out more than $3 million for just one year of the top performers' time, and that's assuming it could've sussed out who the top performers were going to be. Of course, many of the programmers worked far longer than a year, some of them setting aside their primary occupations in order to work on the Netflix problem full-time. As Netflix CEO Reed Hastings admitted to the New York Times, "You look at the cumulative hours and you're getting Ph.D.s for a dollar an hour."But even that number discounts the contest's true benefits to Netflix. Had the company simply put out a help-wanted ad for software engineers, it probably wouldn't have been able to recruit many of the geniuses it found through the competition. That's because most of them already have other jobs. BellKor's members work for, among others, AT&T and Yahoo, and many members of the Ensemble are employed by the data-consulting firm Opera Solutions. The participants also spanned the globe. Netflix got submissions from people in more than 100 countries, and the winning team's members worked in New Jersey, Montreal, Israel, and Austria.
That's the main reason why I didn't enter the competition. The only way to really win is to be the one issuing the challenge.
I think it needs a little more ergonomic design work, but Honda's U3-X looks like it could be useful in at least a limited set of circumstances. Coolest feature: an omnidirectional wheel.
The single wheel on the U3-X — U stands for "unicycle" and "universal" — is made up of many tiny motor-controlled wheels, packed inside the bigger wheel, allowing the device to swerve in any direction.
Scientists have created aluminum that is transparent to ultraviolet radiation.
In this week’s Nature Physics an international team, led by Oxford University scientists, report that a short pulse from the FLASH laser ‘knocked out’ a core electron from every aluminium atom in a sample without disrupting the metal’s crystalline structure. This turned the aluminium nearly invisible to extreme ultraviolet radiation.''What we have created is a completely new state of matter nobody has seen before,’ said Professor Justin Wark of Oxford University’s Department of Physics, one of the authors of the paper. ‘Transparent aluminium is just the start. The physical properties of the matter we are creating are relevant to the conditions inside large planets, and we also hope that by studying it we can gain a greater understanding of what is going on during the creation of 'miniature stars' created by high-power laser implosions, which may one day allow the power of nuclear fusion to be harnessed here on Earth.’
At the very least this aluminum will make awesome lenses for sunglasses!
(HT: RD.)
As usual I was thinking about cybernetic augmentations, trying to list the places on mammal bodies where non-skin parts protrude from skin. Here are the ones I could think of:
- fingernails and toenails
- teeth
- eyes (sorta)
- hair
- antlers
- horns
- hooves
Are there any I'm missing?
Not quite to space, but two MIT students have photographed the curvature of the earth for $150.
Justin Lee and Oliver Yeh, M.I.T. students, had a goal of flying a camera high enough to photograph the curvature of the earth, they named it Project Icarus. With out having a NASA size budget for a rocket, they opted for the more cost effective method of filling a weather balloon with helium and suspending a Styrofoam cooler underneath that held the camera. They also placed some instant hand warmers inside the cooler to try to keep the camera and its battery from freezing.The balloon was launched from Sturbridge, Mass., on September 2, 2009. The University of Wisconsin maintains a balloon trajectory Web site that they used to try to determine where it might land. A GPS-enabled prepaid cell phone was placed in the cooler to let them track its return to Earth and to locate it after landing, a fairly low-tech but creative and effective navigation system.
The camera and balloon made it to 93,000 feet, high enough to see the curvature of the Earth. So high, that the cooler took 40 minutes to return to earth. It is around this altitude that a balloon will pop, allowing the rig to fall back to earth.
So. Cool. I'm sure the chicks are lining up to get ahold of these guys now.
(HT: CR.)
It's a funny stunt, but not that significant that a pigeon has more bandwidth than the South African internet.
JOHANNESBURG (Reuters) - A South African information technology company on Wednesday proved it was faster for them to transmit data with a carrier pigeon than to send it using Telkom , the country's leading internet service provider.Internet speed and connectivity in Africa's largest economy are poor because of a bandwidth shortage. It is also expensive.
Local news agency SAPA reported the 11-month-old pigeon, Winston, took one hour and eight minutes to fly the 80 km (50 miles) from Unlimited IT's offices near Pietermaritzburg to the coastal city of Durban with a data card was strapped to his leg.
Including downloading, the transfer took two hours, six minutes and 57 seconds -- the time it took for only four percent of the data to be transferred using a Telkom line.
A truck full of optical discs would have higher bandwidth than just about any network... the advantages of the internet are latency and low marginal costs.
So the theory of evolution is upended again.
For generations, scientists have believed Africa was the cradle of mankind.Now a stunning archaeological discovery suggests our primitive ancestors left Africa to explore the world around 800,000 years earlier than was previously thought before returning to their home continent.
It was there - hundreds of thousands of years later - that they evolved into modern humans and embarked on a second mass migration, researchers say.
Archaeologists have unearthed six ancient skeletons dating back 1.8 million years in the hills of Georgia which threaten to overturn the theory of human evolution.
The Georgian bones - which include incredibly well preserved skulls and teeth - are the earliest humans ever found outside Africa.
I have no comment about the following, other than that people who know me know what I'm thinking.
The remains belong to a race of short early humans with small primitive brains who walked and ran like modern people.
(HT: RD.)
Scientists have identified the best observatory site on earth.
The search for the best observatory site in the world has lead to the discovery of what is thought to be the coldest, driest, calmest place on Earth — a place where no human is thought to have ever set foot.To search for the perfect site to take pictures of the heavens, a U.S.-Australian research team combined data from satellites, ground stations and climate models in a study to assess the many factors that affect astronomy — cloud cover, temperature, sky-brightness, water vapor, wind speeds and atmospheric turbulence.
The researchers pinpointed a site, known simply as Ridge A, that is 13,297 feet (4,053 meters) high up on the Antarctic Plateau on the continent at the bottom of the world.
Wouldn't the fact that it's so far south be a disadvantage? The site would see less of the heavens over time than would a site closer to the equator.
