Voice Assistant

A voice assistant for my house that runs on my own server. A small speaker hears the wake word, my server works out what I meant, and it answers out loud.

What it does

• Controls thermostats, lamps, curtains, the bed's temperature, the mirror, timers, and alarms.
• Answers how I slept, gives a morning briefing, and reads the weather and calendar.
• “Flag that” marks a wrong action, and “undo” reverses it.
• Also works through Siri and a hold-to-talk button in my phone app.

How it works

• The device is a Home Assistant Voice Preview Edition. A custom firmware config removes the vendor's update pieces and adds my own wake-word models with an adjustable cutoff.
• Home Assistant itself is not in the loop. My server speaks the device's protocol directly.
• Speech to text is faster-whisper on an M2 Max. A short prompt listing the words the house uses fixed most mishearings: “set the lights to 100 percent brightness” comes back correct in about 400 ms.
• Speech out is Kokoro.
• To decide what to do, fixed rules go first. A small classifier trained on this house's own commands checks them and handles what they miss. Claude Haiku takes anything the classifier can't place.

Why a classifier replaced the local model

For a month a local language model (qwen3:8b) handled whatever the rules didn't. Over 304 real commands it took a median of 5.8 seconds, got 13 of 41 requests with a value wrong, claimed 7 actions that never ran, and acted on background noise 6 times, once turning the bed off. The classifier answers in about 33 ms.

Training a wake word

The first custom wake word was trained with microWakeWord: 20,000 synthetic clips across 5 spellings, 30,000 steps, on a rented GPU for $0.70. At its strictest cutoff it measured 0.375 false triggers per hour on a recorded dinner-party dataset, with 90.7% recall.

Those numbers are weaker than they look. 0.375 per hour is two events in 5.3 hours, and recall was measured on synthetic voices. In the room it misfired more than the stock wake word, so the stock word went back on.

The next wake word, “elparko”, is planned with sound-alikes like “El Paso” as weighted negatives and real recordings of the room. It replaces the stock word only if it measures at most 0.2 false triggers per hour and at least 95% recall.