Microsoft Research Connections Blog
Next at Microsoft
Social Media Collective
Windows on Theory
Posted by Rob Knies
Perhaps you’ve heard about Project Adam over the last few days. That work, which shows that large-scale, commodity distributed systems are able to train extra-large deep neural networks efficiently, has received its share of attention in the tech media this week after being featured during the 2014 Microsoft Research Faculty Summit in the event-opening keynote address by Microsoft executive Harry Shum.Or maybe you saw the story On Welsh Corgis, Computer Vision, and the Power of Deep Learning, which appeared on the Microsoft Research website. That one was based on a fascinating interview with project colleagues Trishul Chilimbi and Johnson Apacible—one not dissimilar to Channel 9’s engaging video discussion with Chilimbi as part of the Microsoft Research Luminaries series.It’s always instructive to conduct one of these interviews. One minor drawback, though: Good stuff inevitably ends up on the cutting-room floor. Sometimes, the interview just runs too long to include everything—or maybe a passage of it veers from the story arc developed before the writing begins.As an example of the latter, when I talked with Chilimbi and Apacible, once I had exhausted my list of prepared questions, I ended, as has become a habit, by asking them if they had anything they wanted to add before we were through. Apacible had a great response, commending Microsoft Research management for its trust and support in backing such a risky project. That quote made it into the published story.Chilimbi also had a great response, but when analyzing which quotes I wanted to use, his seemed to veer from the direction I saw the story taking. He wanted to talk about how Project Adam’s big-data, deep-learning approach, used to classify an ImageNet collection of 14 million images, could alter the future of programming. That wasn’t where I wanted the article to go, but it was captivating and thought-provoking, and in the interest of encouraging you to go watch the video, here’s what you didn’t read on Monday.“The one thing that’s interesting and fundamental to me is how [deep learning] changes how we think about computers and programming,” Chilimbi said. “Say I would program a system to do the ImageNet classification task. As a programmer, the way I might go about it would be, ‘OK, I’ll program something to recognize faces or eyes.’
“That’s traditionally how we write programs. People have written programs that sought to do image-classification tasks, and the accuracy of those programs are way below the automatically learned system that operates on this task.”
Then, as researchers are wont to do, Chilimbi pivoted from the specific to the general.
“What it’s saying,” Chilimbi said, “is this methodology of learning and providing vast amounts of data and computing to train a model is a way of synthesizing a system that’s more complicated than anything we can program today. That’s pretty interesting. It makes you think of the possibilities.”
ImageNet is such a challenging dataset, I have a feeling that deep learning approaches, such as convolutional neural nets will continue to dominate the leaderboards for the next few years. However, I do think there is
http://www.pyimagesearch.com/2014/06/09/get-deep-learning-bandwagon-get-perspective/">a lot of hype surrounding deep learning approaches -- when you have a hammer, everything looks like a nail.