Sound Decision Leads to Revolution in Hearing

In early 2017, Achin Bhowmik was faced with an existential dilemma: keep giving robots, drones and machines more human-like senses or start helping people live better lives. He meticulously pondered each option like a scientist. The opportunity to re-invent hearing aids and turn them into multi-purpose, internet-connected wellness devices was a chance of a lifetime. His decision could revolutionize how people overcome hearing loss, a population expected to reach 630 million around the world, according to the World Health Organization.

It was prompted out-of-the-blue by a call from a recruiter eager to hire Bhowmik away from Intel, where he ran the Perceptual Computing Group and spent decades leading technology breakthroughs in mobile computing and 3D computer vision technologies.

"When I heard the name of the company, I said, 'Okay, send me something. I'll read it and if I'm not interested, I'll help find somebody,’” Bhowmik said.

But the name of the company rang a bell. Starkey Hearing Technologies was not only a leading manufacturer of hearing aids, it was the private company Bhowmik featured in a research paper he wrote while studying for his PhD at Auburn in the late 1990s.

Bhowmik remembered being fascinated by the company philosophy.

"The founder of Starkey is Bill Austin, who seemed like an Andy Grove or Steve Jobs…that kind of character, innovative and audacious,” said Bhowmik, recalling what he first learned when studying the company. “For him it was never about business. It was never about making money. It was about helping people."

Austin wanted Bhowmik to use his knowledge of perceptual computing, which is the study of developing devices that mimic human perception, to assist in making intelligent hearing aids. Austin knew the time was right for making a smarter device that used sensing and computing technologies to improve hearing, monitor health, and help people better interact with the world and live healthier lives.

Bhowmik took the leap to join Starkey in 2017 as chief technology officer and executive vice president of engineering. The Eden Prairie, MN-based Starkey is the only hearing aid manufacturer headquartered in the U.S. and one of the largest in the world. The company employs 6,000 people and sold $800 million in hearing aids last year, has built hearing aids for four U.S. presidents, two popes, Nelson Mandela and Mother Teresa, according to Bloomberg BusinessWeek.

“Instead of a ‘We can’t do that…’ he said anything is possible,” said Dave Fabry, Starkey's Chief Innovation Officer, describing Bhowmik’s determination in a video interview. “Within a year of saying he’d bring to market the first hearing aid with embedded sensors, AI and ML, Bhowmik delivered.”

After joining Starkey, Bhowmik formed teams chartered with disrupting the hearing aid market with new technologies. His team developed hearing aids so small that they fit invisibly inside a patient's ear. But, the major breakthrough was finding a way to fit enough technology inside the device, so it can sense and track the wearer’s physical and cognitive health and connect to cloud computing-powered applications on the internet, including voice search, language translation and speech transcription.

Bringing AI to Connected Hearing Aids

One of his chief goals is to bring the capabilities of artificial intelligence and machine learning into what has remained a mostly insular hearing device industry. If done right, he strongly believes that smart hearing devices will be quite valuable for people both with and without hearing loss.

The first revolutionary product from Bhowmik's efforts is Livio AI. The tiny device has built-in Bluetooth so it can connect to iOS and Android devices. It has sensors that can detect if the wearer takes a fall, which triggers a text alert to a family member or designated contact. A finger tap on the device or a voice command connects the hearing aid to the internet through a nearby mobile device. Accessories like microphones and wireless remotes help users hear their television and people talking in noisy environments.

"We're used to raising the volumes" on TVs and radios when we can't hear, Bhowmik said. "How about raising the volume of the world? I am having a conversation with you and you're speaking quietly. I can go to my app to discreetly increase the sound. You can whisper, and I can hear you."

The Livio AI connects to Thrive, a smartphone app that allows the owner to personalize and control many aspects of the wearable device.

“The device can store personalized hearing profiles for varying audio environments. For instance, a ‘restaurant mode’ would direct the microphones on the device to just pick up sounds in front of the user instead of behind, facilitating conversation and filtering out background noise like clanking dishes and silverware. The goal is to augment our perceptual capability," Bhowmik said.

Early in the development, Bhowmik knew that a smart hearing aid would be truly remarkable if people who had perfect hearing would find the device useful in their daily lives. What if they replaced or inspired innovation in today’s wireless Bluetooth earbuds, he thought? This means the device could be used to talk on the phone, listen to podcasts or music, and replace touch screens with voice commands to control devices. The discreet device could be a personal assistant for interacting with the digital world.

"Bill Austin's advice was to make it organic and blend into your ear," Bhowmik said. Innovations in recent years have allowed hearing aids to shrink in size, but now was the time to integrate computer technologies that connect to digital aspects of our lives.

From Hearing Aid to Multifunctional Device

If the hearing aid became a multifunctional product, it could expand the size of the hearing aid market. Right now, that market is limited to units dispensed from a hearing professional. According to The Hearing Review, sales of those units (which often retail for $4,000 or more) topped 4 million in the United States in 2018. Given the shipments of Bluetooth headphones and earbuds reached over 128 million units in 2016, and is still growing, Bhowmik believes that the potential for smart, connected hearing devices to reach a massive global market is huge.

Part of their appeal will come from health monitoring features available through the device, including heart rate monitoring, steps taken daily, and cognitive activity. The ability to tap into language translation from more than 25 languages to provide real-time translations could also be a game-changer.

The built-in AI automatically checks the integrity of the receiver, the microphone, the circuit and the heart rate sensor. In the future, the hearing aid will feature a hydration sensor to alert the wearer they need to drink water. It will measure body temperature and track emotion downswings, sleep patterns and automatically fine tune settings to optimize for different environments.

“We will look back in 10 years’ time and say this was the moment when people’s attitudes about hearing loss and hearing aids started to change because we were empowering them with all of this technology,” said Dr. Dave Fabry, Starkey Hearing Technologies’ Chief Innovations Officer.

Will there be a time when these devices can communicate with the cloud directly?

"We’re working on it," said Bhowmik.

Joe Keller is contributing writer. He has written Fast Company, The New York Times and other publications. Follow him on Twitter @joelkeller.

Ken Kaplan contributed to this story. He is Editor in Chief for The Forecast by Nutanix. Find him on Twitter @kenekaplan.

© 2019 Nutanix, Inc. All rights reserved. For additional legal information, please go here.