Skip to main content

Alexa can now have a conversation with members of the deaf community

amazon echo in the dark
Greg Mombert/Digital Trends

As convenient as voice assistants have proven themselves to be, these A.I.-powered tools have lagged in their inability to serve the deaf community. But one enterprising computer scientist has developed a solution that will allow those with hearing impediments to experience the highs and lows of Amazon Alexa, Google Assistant, and Siri along with everyone else. Abhishek Singh, who first rose to prominence when he built Super Mario Bros in augmented reality, has created a web application that uses a camera to read sign language, and then translates those words into spoken language for an Amazon Echo. When the Echo speaks its response, the whole process plays out in reverse, resulting in a typed reply for hearing impaired or deaf people to read.

This app goes beyond the pseudo-solution provided by the Amazon Echo Show, which added a screen to the smart home hub in order to allow members of the deaf community to interact with the smart home assistant. However, this still did not allow Alexa and its user to truly have a conversation – Singh’s offering, however, fixes this.

Recommended Videos

“The project was a thought experiment inspired by observing a trend among companies of pushing voice-based assistants as a way to create instant, seamless interactions,” he told Fast Company. “If these devices are to become a central way we interact with our homes or perform tasks, then some thought needs to be given to those who cannot hear or speak. Seamless design needs to be inclusive in nature.”

In building the system, Singh trained an A.I. system within machine learning platform Tensorflow, signing words through his webcam ad nauseam in order to “teach” the system sign language. He then added Google’s text-to-speech capabilities in order to translate the sign language into spoken words.

While Singh’s solution is an elegant one, the coder still hopes that Amazon will ultimately recognize sign language on its own. “That’s where I hope this heads. And if this project leads to a push in that direction in any small way, then mission accomplished,” he said. “In an ideal world, I would have built this on the Show directly, but the devices aren’t that hackable yet, [I] wasn’t able to find a way to do it.”

Lulu Chang
Former Digital Trends Contributor
Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…
Adding AI to Alexa is the brain transplant we’ve all been waiting for
The Amazon Echo Show 8 at the Devices Event 2023.

“Alexa, turn the kitchen to cooking mode," my wife innocently requests.

“What do you mean by cooking mode?” her digital tormentor needles.

Read more
At long last, Amazon brings AI features to Alexa
Amazon SVP of Devices and Services Dave Limp demonstrates the Let's Chat feature of Alexa, powered by AI.

Nearly a year after ChatGPT introduced the world to the uncannily human possibilities of generative AI, Amazon has unveiled new Alexa features powered by large language models (LLM). At the annual Amazon Devices Event hosted at its new Arlington, Virginia, headquarters, the company announced some major Alexa improvements that will attempt to make replies much more conversational and lifelike, with less waiting time between your interactions and more meaningful replies.

A new feature called Let's Chat mimics the ChatGPT experience by allowing you to have a fluid conversation with Alexa, asking questions about everything from the voice assistant's football team allegiance to recipes. You can even ask it to write emails for you. In the demo with Dave Limp, outgoing senior vice president of devices and services, Alexa sometimes stalled and needed a second prompt to answer questions, suggesting the feature may still need some polish.

Read more
Amazon tweaks in-garage delivery option, so now you might have to pay
amazon tweaks in garage delivery to make it less convenient

Amazon will soon start adding a small fee for Prime members using its In-Garage Delivery option unless the delivery is made on the customer’s designated Amazon Day.

In-Garage Delivery does what it says on the tin, with the delivery person placing the package in a more secure location that's out of view of porch pirates, while also protecting it from bad weather.

Read more