Posted by Rob Knies

 Translator App logo

Eager for foreign travel but worried about foreign languages? Fear no more, thanks to the Translator App for Windows Phone, updated April 16 and featuring several contributions from Microsoft Research.

The app, powered by the same state-of-the-art technology used in Bing Translator, and available for free download on Windows Phone Marketplace, enables a new travel experience by offering a variety of machine-translation scenarios certain to please the nascent globetrotter. With the app, users can translate street signs, posters, train schedules, and menus simply by pointing and scanning with their phone's camera; download highly optimized, compressed language packs to get translation assistance while avoiding huge data-connection bills; use keyboard input to gain instant translations, some of them featuring spoken playback; and, in German, French, Italian, Spanish, and both U.S. and U.K. English, tap an icon, speak, then tap again for a translation.

The translation aspects of the app, notes Vikram Dendi, director of product strategy for Microsoft Research Redmond’s Machine Translation team, depend on two things: the Microsoft Translator web service, used by such entities as Bing, Office, Internet Explorer, and Facebook; and a highly compressed and optimized version of the translation engine that uses downloadable language packs.

“The new Translator app,” Dendi says, “takes advantage of Windows Phone’s unique features to deliver a great translation experience, enabling you to go to new places with confidence.”

The research included cross-lab collaboration between researchers from Microsoft Research’s Redmond and Microsoft Research Asia, including members of the Machine Translation and Natural Language Processing groups, as well as Speech groups based in both Redmond and Beijing.

Chris Wendt, principal group program manager for the Machine Translation group, cites three significant Microsoft Research contributions to the new version of the Translator App:

  • The machine-translation service, end to end.
  • The machine-translation language pack for download.
  • The adaptation and optimization of the machine-translation service for speech input and travel scenarios.

Alex Acero, a research manager for Microsoft Research Redmond and head of that lab’s Speech group, mentions a couple of technical contributions that merit particular attention.

Xiaodong He on the Speech team worked on machine-translation adaptation algorithms,” Acero says, “and worked with the Machine Translation team to integrate them into the main system.”

That system represented a collaboration with the Microsoft Tellme team, with Microsoft Research building customizations atop the TellMe speech engine.

In December, during the International Workshop on Spoken Language Translation, He’s Chinese-English machine-translation entry took first place in the workshop’s Evaluation Campaign. New technologies pioneered in that entry, including topic adaptation and end-to-end optimization of translation models, will be integrated into the next version of the Translator App.

“There’s also a compact machine-translation engine that runs in the phone,” Acero says. “That is also cool, because it requires no data connection, important when you’re traveling if you don’t want to get hit with a huge bill when you come back home. The server engine is huge, so compressing it to fit in a phone was not an easy feat.”

Chris Quirk and Anthony Aue of the Natural Language Processing team get the credit for that contribution.

Frank Seide, senior researcher and research manager for Microsoft Research Asia, was joined by Kit Thambiratnam, now senior development lead for the Beijing-based Search Technology Center, on the Translating Telephone project, one of the key steps toward the Translator App. Seide knows how difficult it is to incorporate speech-to-speech translation.

“A big challenge is that machine-translation systems are built for the written form of language,” he says, “while a speech-to-speech system is faced with spoken language. Spoken language not only lacks punctuation and is riddled with ‘ums,’ ‘ahs,’ and folks correcting themselves mid-sentence, but it also differs grammatically quite a bit.”

Conquering such obstacles did not go unnoticed.

“In building this app,” Dendi concludes, “engineers and researchers on our team solved tremendous technical challenges and dealt with the nuances of voice, cameras, data availability, and language complexity. Our focus was on delivering a scenario that is truly useful.”