Tag: AI audio device

Summary:University of Washington researchers have developed AI-powered “proactive hearing assistant” headphones that automatically isolate conversation partners in noisy environments by detecting turn-taking speech rhythms. Key Takeaways: The system uses two AI models to identify who is speaking and suppress all non-participant voices and background noise in real time. Initial testing showed users rated filtered audio more than twice as favorably as unfiltered baseline audio. The open-source technology could eventually be integrated into hearing aids, earbuds, and smart glasses to provide hands-free, intent-aware sound filtering. Holding a conversation in a crowded room often leads to the frustrating “cocktail party problem,” or the challenge of separating the voices of conversation partners from a hubbub. It’s a mentally taxing situation that can be exacerbated by hearing impairment. As a solution to this common conundrum, researchers at the University of Washington have developed smart headphones that proactively isolate all the wearer’s conversation partners in a noisy soundscape. The headphones are powered by an AI model that detects the cadence of a conversation and another model that mutes any voices which don’t follow that pattern, along with other unwanted background noises. The prototype uses off-the-shelf hardware and can identify conversation partners using just two to four seconds of audio. The system’s developers think the technology that recognizes turn-taking rhythms could one day help users of hearing aids, earbuds, and smart glasses to filter their soundscapes without the need to manually direct the AI’s “attention.” The Prototype System The team presented the technology Nov. 7 in Suzhou, China at the Conference on Empirical Methods in Natural Language Processing. The underlying code is open-source and available for download. “Existing approaches to identifying who the wearer is listening to predominantly involve electrodes implanted in the brain to track attention,” said senior author Shyam Gollakota, a UW professor in the Paul G. Allen School of Computer Science & Engineering. “Our insight is that when we’re conversing with a specific group of people, our speech naturally follows a turn-taking rhythm. And we can train AI to predict and track those rhythms using only audio, without the need for implanting electrodes.” The prototype system, dubbed “proactive hearing assistants,” activates when the person wearing the headphones begins speaking. From there, one AI model begins tracking conversation participants by performing a “who spoke when” analysis and looking for low overlap in exchanges. The system then forwards the result to a second model which isolates the participants and plays the cleaned up audio for the wearer. According to researchers, the system is fast enough to avoid confusing audio lag for the user, and can currently juggle one to four conversation partners in addition to the wearer’s audio. The team tested the headphones with 11 participants, who rated qualities like noise suppression and comprehension with and without the AI filtration. Overall, the group rated the filtered audio more than twice as favorably as the baseline. Refining the Technology Gollakota’s team has been experimenting with AI-powered hearing assistants for the past few years. They developed one smart headphone prototype that can pick a person’s audio out of a crowd when the wearer looks at them, and another that creates a “sound bubble” by muting all sounds within a set distance of the wearer. “Everything we’ve done previously requires the user to manually select a specific speaker or a distance within which to listen, which is not great for user experience,” said lead author Guilin Hu, a doctoral student in the Allen School. “What we’ve demonstrated is a technology that’s proactive — something that infers human intent noninvasively and automatically.” Plenty of work remains to refine the experience. The more dynamic a conversation gets, the more the system is likely to struggle, as participants talk over one another or speak in longer monologues. Participants entering and leaving a conversation present another hurdle, though Gollakota was surprised by how well the current prototype performed in these more complicated scenarios. The authors also note that the models were tested on English, Mandarin, and Japanese dialog, and that the rhythms of other languages might require further fine-tuning. Potential Hearing Aid Use The current prototype uses commercial over-the-ear headphones, microphones, and circuitry. Eventually, Gollakota expects to make the system small enough to run on a tiny chip within an earbud or a hearing aid. In concurrent work that appeared at MobiCom 2025, the authors demonstrated that it is possible to run AI models on tiny hearing aid devices. Co-authors include Malek Itani and Tuochao Chen, UW doctoral students in the Allen School. This research was funded by the Moore Inventor Fellows program. Featured image: The team combined off-the-shelf noise-canceling headphones with binaural microphones to create the prototype, pictured here. Photo: Hu et al./EMNLP

Tag: AI audio device

All

Turn-Taking AI Lets Smart Headphones Focus on the Voices That Matter

New AI-Powered Headphones ‘Cancel’ Only Unwanted Sounds

Mimi Partners with Circulr Sound

All

Sorry, No Posts Found

All

Sorry, No Posts Found

Sorry, No Posts Found

The Hearing Review

Our Parent Company

Key Resources

Helpful Links