SALE-R

Design Philosophy

The SALE-R (SAY-lor) buttons are designed to offer directional control to the resident, allowing them to navigate the smart home interface using up, down, left, right, and select commands. These buttons serve both assistive and rehabilitative purposes. While they enable the user to control the entire smart home system through the Smart Motion Device (SMD), they also promote the use of residual limb mobility. Each time a button is physically pressed, the corresponding User Interface (UI) button in the bottom window of the Smart Mobile Device app lights up green.

Core Components

  • Optical Sensor: Each button is equipped with an optical sensor. The button mount includes a spring mechanism, and when the button is pressed, an internal flag interrupts the laser beam inside the optical switch. This interruption signals the sensor that the button has been activated, thereby registering a button press.
  • LED-circuit: Each button has an Light Emitting Diode (LED) circuit, comprising two LEDs, which light up red when the buttons are not pressed. Every time the buttons are pressed, the LEDs turn off.

Implementation

The primary circuitry of the SALE-R consists of a printed circuit board that detects button press signals, processes them, and sends them to the Smart Mobile Device using Bluetooth Low Energy (through the Pico W). The kick buttons were custom-designed and 3-D printed, with the physical descriptors shown in the figure above.

Engineering Challenges and Solutions

  • The main challenge the enabling wireless communication to the smart mobile device. While the SALE-R could connect to the HUB’s private wifi network, relaying the button press signals from the SALE-R to the HUB and back to the smart mobile device would induce delays. Hence, the way to solve this problem was to enable direct communication between the SALE-R and the Smart Mobile Device. This issue was solved using enabling communication over Bluetooth Low Energy.
  • Another challenge was designing a layout that would be ergonomically intuitive for the user. This was addressed by testing multiple configurations and ultimately selecting the one that offered the best combination of usability and compactness.

Performance

The SALE-R circuit can detect button presses almost instantly and transmit the results without delay. At present, the buttons are programmed such that the user has to hold the buttons for at least 0.2 seconds for it to register a button press. However, this value can be custom-programmed based on the needs of the user.

Voice Control

The home assistant ecosystem dedicated a year of development to implementing and integrating voice assistant functionality. We leverage this in our design to provide a voice assistant, which can control devices around the home. We set up the system to use all-local processing, allowing it to work offline and without exposing user data to easily-hacked third parties. While the system can support many different “assist satellites”, devices which provide microphone streams for processing by the hub, only the smart speaker was appropriately configured.

A wakeword engine, openwakeword, continuously monitors the audio stream for the configured word. When it is received, the engine begins recording and transmits the recording to OpenAI’s Whisper speech-to-text (STT) engine. Whisper transcribes the recording, and then Home Assistant’s internal engine interprets the meaning of the words. Then, a response is generated, which is sent to Piper, a text-to-speech (TTS) program. This generates an audio clip of the response being read, which is played on the speaker.