Getting Started
Lumen is an assistive communication application designed to help people communicate when speaking is difficult. Messages can be created using the communication board, keyboard input, or gesture-based controls and then spoken aloud using speech synthesis.
When gesture input is enabled, the blue highlight indicates the currently selected communication button. A Hello gesture activates the selected button, while a Nothing gesture advances the selector.
Lumenboard
The Lumenboard is Lumen's primary communication interface. It organizes frequently used phrases, controls, and navigation options into an accessible button grid.
In Basic mode, the board provides commonly used communication phrases along with access to Advanced mode, Saved Phrases, and Settings.
When gesture input is enabled, the selector moves through the available buttons and activates the selected option when the appropriate gesture is detected.
Letters & Spelling
Advanced mode provides access to letter groups for spelling custom messages.
Select a letter group such as ABCD, EFGH, or another available group to access individual letters. Selecting a letter adds it to the current message.
Use Back to return to the Advanced menu.
Preset Phrases
Basic mode includes ready-to-use communication phrases for common situations.
- Hello
- Please
- Help
- Emergency
- Bathroom
- Turn on TV
Activating a preset phrase sends it directly to speech output, allowing communication without manually spelling the message.
Message Area
The message area displays the text currently being composed. It is available in Advanced and Saved modes.
Use Space to add a space, Delete to remove the most recent character, and Delete All to clear the current message.
When available, the message field can also be edited directly using a keyboard.
Speech Output
Lumen uses browser-based speech synthesis to convert messages into spoken communication.
Activating a supported communication phrase can speak it immediately. For composed messages, use Say to read the message aloud.
Lumen prefers an English voice when one is available. The status area indicates when speech output is active.
Gesture Input
Lumen can use a Teachable Machine pose model to provide hands-free control of the communication interface.
- Nothing — advances the selector to the next available button.
- Hello — activates the currently selected button.
Gesture detection runs at a configurable interval. The default interval is five seconds and can be adjusted through Settings.
For reliable recognition, remain clearly visible to the camera and hold the desired gesture throughout the detection interval.
Default Gesture Model
Lumen's default gesture model is trained around a simple upper-body gesture: left arm raised and held at approximately a right angle (90°).
The model distinguishes between the Hello and Nothing classes based on this trained pose pattern, and is intended to recognize the user's left arm position clearly within the camera frame.
For the best recognition:
- Keep your left arm clearly visible.
- Hold the arm at approximately a 90° angle.
- Remain positioned within the camera's field of view.
- Maintain consistent lighting and positioning.
- Hold the gesture throughout the detection interval.
Camera Input
The camera provides the video input required for gesture recognition. Lumen requests camera permission when gesture functionality is initialized.
Recognition generally works best with good lighting, a clear view of the upper body, and minimal visual obstruction between the user and the camera.
Settings & Detection Speed
Settings allows you to adjust how frequently Lumen evaluates gesture input.
The detection interval can be configured from 1 to 10 seconds. A shorter interval makes the interface respond more frequently, while a longer interval provides additional time to prepare a gesture.
Saved Phrases
Saved Phrases allows frequently used custom messages to be stored for future communication.
In Advanced mode, compose a message and select Save. The saved phrase will appear on the Saved board for later use.
Saved phrases are stored locally on the device and remain available after the application is closed.
Customization
Lumen supports custom Teachable Machine pose models, allowing gesture recognition to be configured for different users and environments.
Enter a Teachable Machine pose model URL and select Load Model.
The model must contain exactly two classes named Hello and Nothing for Lumen's gesture controls to operate correctly.
Lumen remembers the most recently valid model URL locally on the device.
Troubleshooting
Camera isn't working
Confirm that camera permission has been granted. Close other applications that may be using the camera and reload Lumen if necessary.
Gesture isn't being detected
Improve the lighting around you, make sure your upper body is visible, and hold the gesture for the full detection interval. If using a custom model, verify that its classes are named Hello and Nothing.
Speech isn't playing
Check your device volume and make sure the browser tab is not muted. Some browsers require an initial user interaction before speech synthesis can begin.
Buttons aren't responding
Confirm that Lumen has finished loading and that the interface is receiving input. If the issue persists, reload the application and try again.
Privacy & Camera
Lumen requires camera access when gesture recognition is enabled. Camera video is processed within the browser for pose detection.
Saved phrases and the configured model URL are stored locally on the device.
Camera access is only required for features that use gesture recognition.
Installation & Launch
Lumen can be accessed directly through the Lumen website. To begin using gesture-based communication:
- Open the Lumen website.
- Allow camera access when prompted.
- Wait for the gesture recognition model to load.
- Position yourself within the camera view.
- Use the communication board or gesture controls.
You're ready
You now have the information needed to navigate Lumen, create messages, use gesture input, and customize the communication experience.