Cornell Researchers Unveil New Smart Glasses Tech
Cornell builds new tech for smart glasses
Created on April 12|Last edited on April 12
Comment
In today's world, voice assistants have become an integral part of our daily lives, offering hands-free access to information, communication, and control over various devices. However, the main drawback to these voice assistants is that they require users to speak, limiting their utility in noisy environments and situations where vocalizing is socially inappropriate.
To address these challenges, a team of Cornell researchers has developed a groundbreaking solution: glasses equipped with acoustic sensing technology for silent speech recognition. Silent speech recognition interprets a person's speech movements without the need for them to vocalize any sounds, enabling communication in a discreet and quiet manner.
This innovative work by Cornell researchers introduces a minimally obtrusive and comfortable alternative to existing silent speech interface technologies, which often rely on cameras or intrusive sensors. The acoustic sensing glasses emit sound waves that interact with facial deformations caused by speech movements, allowing the system to accurately recognize and interpret the user's silent speech.
This exciting development has the potential to transform the way we interact with voice assistants, preserving privacy and enhancing user experience in a wide range of scenarios.
Acoustic Sensing
Acoustic sensing is a technique that utilizes sound waves to detect, analyze, and interpret movements and deformations in the surrounding environment. In the case of the glasses for silent speech recognition, acoustic sensing works by emitting sound waves from speakers placed on the glasses' frame and capturing the reflected and diffracted waves with microphones also placed on the frame.
When a person speaks, their articulators (such as the tongue, jaw, and lips) and connecting tissues move, causing deformations in the skin and muscles around the face. These movements modify the patterns of the sound waves emitted by the speakers on the glasses. As the sound waves travel through the facial region, they interact with the deformations caused by speech, resulting in changes to the sound waves' properties, such as amplitude, frequency, and phase.
The microphones on the glasses capture the modified sound waves, and the captured data is then processed by an algorithm (CNN) to analyze the patterns of the reflected and diffracted sound waves. The algorithm is trained to recognize these patterns and associate them with specific words or phrases.
By analyzing the differences in the sound wave patterns, the system can accurately recognize and interpret the user's silent speech. Acoustic sensing enables the glasses to continuously monitor and recognize silent speech without the need for invasive or obtrusive hardware, making it an ideal solution for a wide range of applications, such as communication, authentication, and facial expression tracking
The Development Process
The researchers conducted experiments to identify the optimal sensor setup, testing various sensor positions, orientations, and quantities. They started with the most unobtrusive setup, placing sensors on the legs of the glass frame, but found that the setup did not provide enough accuracy for silent speech recognition. They then moved the sensors to the front side, placing a speaker near the nose bridge and two microphones on either side of the frame. This setup achieved better performance, but still not enough to reach their design goals.
Final Design 
The team further optimized the setup by having the speaker and microphone on different sides of the frame, which significantly improved the performance. They then experimented with the microphone and speaker locations, ultimately choosing to use two speakers and two microphones on either side of the frame to capture more information and yield better performance.
The final design uses acoustic sensing and a comfortable, stable glass-frame form factor with optimized sensor placements to achieve continuous silent speech recognition.
As the demand for seamless, hands-free communication continues to grow, the integration of silent speech recognition into smart glasses offers a promising solution to overcome the limitations and privacy concerns associated with conventional voice assistants.
This technology not only expands the usability of voice assistants in various environments but also opens up new opportunities for interacting with other tools like ChatGPT!
The Paper:
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.