A Silent Speech Interface: Using Mouth-Based Sensors for Non-Vocal Communication
Abstract:
This white paper proposes a novel direction in assistive and ambient human-computer interaction: a silent speech interface using intraoral sensors. While recently reminded of this potential by technologies like Augmental's MouthPad, the concept itself originated over 15 years ago from personal experience living in Japan. The approach explores using real-time internal mouth movement data to interpret silently mouthed words, enabling users to communicate without vocalization — benefiting individuals with speech or lung impairments and those in environments where speaking aloud is socially discouraged.
1. Introduction:
Voice-based interfaces are becoming ubiquitous, yet they exclude users who cannot speak due to medical, situational, or cultural reasons. In Japan, for instance, speaking aloud on public transportation or in quiet public spaces is socially frowned upon. This paper introduces a silent speech interface that captures and interprets tongue, lip, and internal oral muscle movements to generate spoken or written output without audible speech.
2. Background:
Recent advances in intraoral computing, such as the MouthPad, have shown that the mouth can serve as a precise and fatigue-resistant input mechanism. Prior research into electromyographic (EMG) speech decoding and silent speech interfaces has laid foundational work, but most systems remain bulky or non-portable. A wearable, intraoral device could change that.
3. Technical Approach:
The proposed system would integrate: - Pressure-sensitive sensors on the palate and teeth - Inertial sensors (IMUs) to detect subtle jaw and tongue movement - Real-time sensor fusion to map gestures to phonemes - A trained model for lipreading-from-within: using internal gestures to decode intended speech
4. Potential Applications:
- **Assistive Communication**: For users with ALS, tracheostomies, or lung impairments
- **Private Interaction**: Silent texting, voice dictation, or interface control in public spaces
- **Covert Use Cases**: Military or security scenarios where soundless communication is needed
5. Feasibility and Constraints:
While hardware miniaturization and low-power Bluetooth protocols exist, challenges remain: - Dataset acquisition for silent mouthed speech (without audio cues) - Variability in oral anatomy - Latency and misinterpretation risks - Hygiene and long-term intraoral wear comfort
6. Social and Cultural Impact:
If successful, such a system could normalize silent, mouth-only interaction with machines — useful not just for people with disabilities but also for those seeking discreet or non-disruptive communication. The idea originated from personal experience living in Japan, where voice input feels socially intrusive in certain settings.
7. Next Steps:
- Prototype development using off-the-shelf palate sensor kits
- Dataset collection of silently mouthed phrases
- Model training using temporal gesture data
- Collaboration with linguists and speech therapists to map articulatory gestures to semantic output
Conclusion:
This paper proposes a silent speech interface as an extension of mouth-based assistive tech. By turning internal articulation into machine-readable commands, it offers a pathway to quiet, private, and inclusive communication. The potential spans accessibility, cultural sensitivity, and next-gen human-computer interaction paradigms.
Acknowledgments:
We acknowledge the work of Tomás Vega and the Augmental team for pioneering intraoral interaction, and thank the communities of silent speech decoding, wearable HCI, and neuro-linguistics for laying the groundwork for this vision.
Future Research Directions:
- Exploration of multilingual mouthing models
- Integration with AI-driven contextual inference engines
- Exploring sublingual EMG or photoplethysmography for enhanced articulation capture