When former WillowTree designer Matt Kubota’s partner Ana was diagnosed with Guillain-Barré Syndrome, we knew we had to help.
The rare autoimmune disorder left Ana temporarily paralyzed and unable to speak. So Ana and her caregivers looked to augmented and alternative communication (AAC) devices. But as they discovered, these devices are expensive, bulky, and outdated. While Ana recovered from the disease, she could only painstakingly communicate by blinking as caregivers pointed to individual letters on a posterboard.
Matt knew he and the WillowTree team could create a better solution for AAC users like Ana: stroke and trauma survivors, people with MS, non-verbal individuals, intubated COVID-19 patients and many more.
That vision became the Vocable AAC app.
It started on a WillowTree-wide Innovation Day and eventually became Vocable — a free, mobile platform for users to communicate via head and eye movements, or text-to-speech.
approximate cost of hi-tech speech generating devices
adults experienced difficulty speaking in the past year
children between ages 3–17 are affected by voice, speech, language, or swallowing disorders
stroke survivors have problems speaking, reading, or writing
We believe shared languages lie at the root of every ecosystem. Language is what connects every JavaScript programmer, every caregiver and child, and every doctor and patient, creating something greater than ourselves: community.
And the power of the WillowTree community proved itself to Matt and the Vocable development team time and time again. In the product’s first iteration — “Eye Lasers” — users experienced significant shakiness when trying to control the cursors that tracked their eye movements.
Surprisingly (or perhaps not surprisingly to Star Wars and robotics fans everywhere), WillowTree senior software engineer Chris Stroud’s love for building motorized droids held the solution. Using the codebase from Chris’s robot “child” BB-8 — a droid whose head balances on a movable ball — and PIDA controller design, the team gave users a more stable, precise experience when manipulating keyboards of any size and on any device.
“We’ve received positive feedback from the nurses working with COVID-19 and ventilator patients, including some heartbreaking stories of patients using the app on iPads to communicate with family members for the last time. [WillowTree’s] donated technology makes it possible for people in very difficult situations to communicate with caregivers and family members in a way they would not have been able to otherwise.”
Vocable’s head-tracking custom API follows facial features and movements ranging from head nods to the most subtle eye twitches.
In doing so, users control a cursor that selects letters, custom phrases from personalized categories, and automated suggested responses. The app’s machine learning-powered, predictive pattern detection powers a deep semantic understanding of an individual user’s speech and offers a faster, smoother, and better experience for today’s AAC users.
Vocable is a good listener, too. Through Listening Mode, Vocable offers suggested responses to yes/no questions, either/or questions, and questions that can be answered with a number. Users or caregivers can quickly and easily activate the feature by saying, “Hey, Vocable.”
“It’s very customized, such as how we highlight buttons and the cursor’s focus as you navigate: others might put a rectangle around elements, but we animate and make them pleasant,” said Sashank Patel, WillowTree Engineering Director. “Part of our philosophy was that engaging with the app should be delightful.”
If users have to spend so much of their day interacting with the Vocable app, it’s gotta look good, too, right? Based on feedback from research partners like the assistive technology teams at Duke University and speech language pathologists, WillowTree’s design team identified ways to accomplish exactly that.
Guiding our design work was the principle that high contrast can be so much more than black and white. That’s why Vocable incorporates the strongest ADA compliance through elements like a compelling high-contrast palette and large typography, promising a consistent and aesthetically pleasing experience across devices.
Because what’s accessible should also be enjoyable.
“WillowTree believes voice is the future. Vocable is the most powerful voice product we could ever create... I’ve worked at WillowTree for over six years and this is by far one of the most impactful projects I’ve been on, by any means — and I worked in healthcare IT prior to WillowTree.”
Voice is powerful. It’s also deeply personal. We tapped into iOS native capabilities to offer text-to-voice generators that more closely match an individual’s voice.
Then, we took the local and made it global. With the help of the localization management platform Crowdin, Vocable became a multilingual product that could serve people speaking different languages around the world.
Vocable gives voice to WillowTree’s commitment to connect and represent all of us. And Vocable’s story isn’t over — not even close.
It’s got plenty more to say.
