Swype AI
Letting people with arthritis and motor disorders control a computer with hand motions and voice, when they otherwise couldn't.
The problem
For people with arthritis and other motor disorders, ordinary input devices can be painful or impossible to use. Swype AI is a system that lets them drive a computer through hand gestures and speech instead.
How it works
- A custom Fourier signal-analysis pipeline for voice recognition, paired with an NLP layer that detects keywords and maps them to commands.
- The same pipeline filters out accents and individual voice tendencies to recover clean raw text from speech, then passes the resulting commands to the computer to execute.
- A Kalman-filtered hand tracker with a landmark system reads gestures, so motion alone can move a cursor and trigger actions.
What I'm improving now
I'm refining the Kalman filter and building a better landmark system to detect gestures at a higher frame rate, the difference between a demo and something genuinely dependable day to day.
Recognition
- Runner-up at SXSW with a $2,500 cash prize.
- Best-AI-Hack and Best-Hack wins at Boost Hacks II and NOVA Hacks.