A11Y Tools

Earlier today, I got to thinking about devices for those with A11Y requirements.

Speech2Text4Keyboard Input

• Voice input;
• Listen to voice;
• Convert to text;
• Send text to system as if a keyboard;

ASL2Text4Keyboard Input

• Video input;
• Watches video for ASL;
• Convert to text;
• Send text to system as if a keyboard;

Text2ASL4Video Display

• Screen reader reads console input;
• That input is broken into words;
• "Known" words are displayed as a person talking, using ASL;
• "Unknown" words are displayed as a person spelling words out in ASL;
• "Query Mode" can be enabled, to add more "known" words;

Eyes2Keyboard Input

• Video Camera watches;
• Eye Movements are converted to Dasher images;
• Dasher sends text to system, as if a keyboard;

Touchscreen Reader/Presser

• Video Camera watches;
• OCR software watches video input;
• OCR software outputs text;
• Screen reader reads text;

• Voice input;
• Listen to voice;
• Convert to instructions;
• Camera guides the prong in moving to the selected item;
• Prong on device extends out to press the selected item;

Additions to the Hardware

• 10 Key Keypad;
• Joystick;
• Firing button;
• Trackball;
• Camera;

These would be used if the vocabulary could not differentiate between a keyboard value, and something to be written.

Or maybe just to be used as supplemental aids;

• Bluetooth;

Make it a Bluetooth enabled keyboard.

Existing Software

• Sphinx3: voice recognition;
• Dasher: predictive text input;
• Eyegazer: eye-tracking;
• Orca: screen reading;

Needed Software

• ASL2Text: This examines photographs for ASL signs, and outputs the corresponding word;
• Text2ASL: This displays a picture of a person making the appropriate ASL gesture;
• Prong Guidance Software: This is the hard one. I suspect that there is code from a robotics project that can be converted/modified/re-used, that will do this. Worst case scenario is to write the dumb pigeon scenario;

Platform

This is the type of one-off job, that Arduino's were designed for.