Microsoft Kinect turns sign language into spoken language in real time
Microsoft has had its unsung products before, but the Kinect may prove its best worst product. By that I mean it’s a great piece of hardware that the company can’t seem to sell to anyone but gamers. There has been an SDK on the market for some time to make Windows-based Kinect apps, but we rarely hear of any great success stories, and that’s a shame.
The latest bit of genius comes from the folks at Microsoft Research, who created the Kinect Sign Language Translator to translate sign language into spoken language and vice versa in real time.
Kinect captures the gestures while machine learning and pattern recognition programming help interpret the meaning. The translation is bidirectional, so a deaf person can converse in sign language with a person who does not read sign but speaks vocally.
Now, for the down side, at least for most readers of this blog: It’s done in Chinese. The lead researcher involved is Guobin Wu, the program manager of the Kinect Sign Language Translator project, who said on the Kinect blog that Microsoft Research Asia had reached out to a professor at the Chinese Academy of Sciences who has been researching sign language recognition technology for 10 years.
And because of the complexity of the Chinese language, it currently takes five people to establish the recognition patterns for just one word. So far, only 300 Chinese sign language words have been added out of a total of 4,000 words.
Wu says there are more than 20 million people in China who are hard of hearing (which is remarkable for a country with that large of a population) and an estimated 360 million such people around the world.
Wu said the work has been demonstrated at the Microsoft Research Faculty Summit and the Microsoft company meeting this year. Since those demos, the project has received much attention from researchers and the deaf community, so he expects to start building prototypes for different languages.
Microsoft will release a new generation of Kinect with the Xbox One in November. So far, there has been very little action around the SDK for building Windows apps. If it’s not updated, that will be a bad sign for progress.
Best Microsoft MCTS Certification, Microsoft MCITP Training at certkingdom.com
