Voice performance and deployment

Accessor now exposes repeatable local STT and TTS benchmarks. Results are file and software timings, not promises for a particular room, microphone, network or CPU. The development machine reports an Intel i7-1260P; the intended 2012 dual-core i7 must be measured separately. Limiting modern hardware to two threads does not emulate the older CPU, memory bandwidth, instructions or thermal limits. An SSD helps loading models, not their steady-state computation.

Changes

  • Piper is now the default spoken voice on new profiles: a pinned local neural engine with a persistent worker, one WAV per completed chunk, no key and no network. System, Kokoro, and Cartesia remain selectable. acc tts benchmark --provider piper measures it on your machine.
  • End-of-speech silence is configurable from 300–2000 ms, default 600 ms, previously 1000 ms. This removes 400 ms of intentional waiting for continuous speech; set it back to 1000–1200 ms if thoughtful pauses split sentences. The playback capture hold follows the selected cutoff instead of a fixed 1200 ms.
  • Local inference defaults to two CPU threads, spinning off. Thread count (1–8) and spinning are configurable and require restart. Spinning can shave latency while wasting idle CPU/battery; benchmark before enabling it.
  • Cartesia TTS plays incoming raw PCM from its HTTP streaming endpoint by default. It no longer has to receive a whole sentence’s audio before starting playback. The queue is bounded, has a 120 ms prebuffer, pauses without dropping samples during user capture, and feeds actual speaker samples to echo cancellation. Network starvation produces silence and rebuffering; incomplete audio errors are surfaced without automatically repeating already-spoken output. Set tts.streaming false for the previous completed-WAV, prefetched-sentence path. Streaming sentence requests are currently sequential, so longer replies can have gaps between chunks; the buffered option retains next-sentence prefetch.
  • Cancellation owns and stops in-flight synthesis/prefetch jobs. Spoken-audio caching stays in memory and is invalidated by locking.

The full model still decodes wake speech locally. Cartesia after-wake streaming overlaps upload with capture, but the current STT path also runs local recognition for controls and fallback. It does not eliminate local compute. Relevance filtering and the selected agent add further latency. /analytics separates the measured stages; /audio reports wake decoder timing and capture diagnostics.

Repeat on the Dell

Generate a non-private speech sample once, or copy the synthetic WAV used here:

acc tts test "Twenty nine. What is on my calendar tomorrow morning?" --provider cartesia --output sample.wav
acc stt benchmark sample.wav --runs 7 --engine canary
python scripts/benchmark_voice.py sample.wav --assets PATH_TO_ACCESSOR_ASSETS --output results/voice.json --runs 7
acc tts benchmark --provider piper --runs 4
acc tts benchmark --provider system --runs 4
acc tts benchmark --provider kokoro --runs 4
acc tts benchmark --provider cartesia --runs 4
acc tts benchmark --provider cartesia --runs 3 --playback

Only the Cartesia commands use the service/key and incur provider usage. The file benchmarks don’t open the microphone or contact an agent. --playback explicitly plays test audio; its playback-start flag is software timing, not a microphone measurement of audible onset. The STT comparison script isolates its settings, leaves the user’s configuration unchanged, and compares one/two/four threads plus spinning using clean, quiet, 10 dB synthetic noise and delayed-echo variants. Run benchmarks serially with other heavy work stopped. Reports include load, first decode, seven warm runs, median, nearest-rank P95, RTF, and exact recognized text. An RTF below 1 means computation faster than clip duration; it says nothing by itself about wake false positives or far-field accuracy.

The checked-in synthetic WAV can be used without a Cartesia account. Raw results and fixture metadata are in benchmarks/2026-09-21. The default Piper voice was measured separately on a 2012 dual-core i7-3520M (warm median 418.9 ms, RTF 0.14 for a 2.9-second reply); see benchmarks/2026-09-27. That is a machine of the intended class but not the exact Dell, so re-run there before relying on it.

Suggested starting setups

Setup Local recognition After-wake STT TTS Tradeoff
Default local (new profiles) Canary warm, 2 threads, no spin Local Piper No key, no network; fast local synthesis on any supported desktop CPU
Responsive cloud-assisted Canary warm, 2 threads, no spin Cartesia streaming Cartesia streaming Lowest cloud result wait; awake speech uploads before relevance is known
Conservative cloud upload Canary warm, 2 threads, no spin Cartesia finished clip Cartesia streaming Local word check before upload, additional transcription round trip
Entirely local voice Canary warm, 1–2 threads Local System Smallest footprint; OS voice quality varies
Local neural voice Canary Local Kokoro warm Better neural voice; competing CPU load, particularly on the old Dell

For the Dell, start with Piper (neutral, fast) or System TTS and Canary kept warm; use Cartesia streaming when network latency is acceptable and a key is available. Kokoro’s cold start and CPU demand can dominate response time. The current Whisper adapter spawns whisper-cli and reloads the model per utterance; a tiny model does not automatically mean a faster always-on experience. Lazy STT frees RAM when asleep but introduces wake-time loading. Parakeet’s larger model needs measurements before choosing it for a dual-core machine. None of these settings requires an OpenAI API key; agent login remains the harness’s own responsibility.

acc config set tts.provider piper
acc config set tts.piper-voice en_GB-alan-medium
acc config set stt.endpoint-ms 600
acc config set stt.threads 2
acc config set stt.spin false
acc config set stt.lazy false
acc config set tts.streaming true

Dedicated wake spotting

Accessor now ships an always-on keyword spotter (stt.wake-engine, default auto). The official sherpa-onnx 3M Chinese/English keyword model (sherpa-onnx-kws-zipformer-zh-en-3M-2025-12-20) is downloaded on first run and runs in-process beside the full Canary path. It accepts new keywords through a pronunciation list, so any numeric wake code works without training a model; the keyword phones are read from the model’s en.phone lexicon at startup. Official documentation lists a 4.4 MiB quantized encoder and 160/320 ms model chunk latency. Model and customization documentation.

Measured on the dual-core i7-3520M test machine (the intended class): the small model decoded a 3.11 s clip in ~0.16 s on one thread (RTF ~0.05) and detected “twenty nine” at 1.44 s, versus ~1.5 s of Canary inference after the utterance ended. On synthetic additive noise it stayed reliable to about 15 dB SNR and missed at 10 dB, where full Canary recognition still transcribed the phrase. Both detectors run together, so the union keeps Canary’s noisy-room recall and adds the spotter’s speed. A keyword hit only opens listening or interrupts playback; the request itself is still finalized by local recognition, so a false spot cannot submit a request. A hit that matches recent speaker text is rejected as echo, and hits while already awake and idle are ignored. Room false-accept and miss rates still need a corpus and physical testing before the spotter replaces any part of full recognition; stt.wake-engine canary is the conservative setting. The research scripts remain for benchmarking the model directly:

python3 scripts/test_wakeword.py --wav tests/fixtures/voice-benchmark.wav
python3 scripts/benchmark_keywords.py MODEL FILES --output results/kws.json

acc stt wake FILE... still runs the production matcher on recordings, and python3 scripts/test_room_wake.py still measures Canary recall across a room.

openWakeWord is another credible candidate: it supports ONNX on Windows and custom wake models, but “twenty nine” needs a trained and validated model. Review model licensing separately from its Apache-licensed code before distribution. Project documentation.

Neither candidate makes “29” an identity or password. A short common number can occur on TV or in ordinary conversation. Offer “hey twenty-nine” as a more distinct activation option and test similar numbers, accents and recordings of the assistant itself. The current password lock remains independent of wake technology.

Product work worth prioritizing next

  1. Microphone calibration and a room acceptance suite. Test 1/3/5 metres, facing/sideways/away, fan/TV/music, different voices, and TTS/alarm playback. Measure wake recall, false accepts per hour, interruptions, password success and median/P95 response. Synthetic noise tests cannot substitute for this. Consider a microphone near the user or a tested microphone array only after comparing the built-in mic; gain alone also amplifies noise.
  2. Validate the shipped wake spotter. A negative corpus and hours-long room runs are still needed to set the false-accept rate and choose the default threshold. Keep Canary as the recognition path while that data is gathered.
  3. Visible privacy state and recovery. Make locked/local-listening/cloud-active states obvious; retain hardware mic-off as the reliable capture switch. Add microphone unplug/reconnect recovery and a guided setup/level check.
  4. Sensitive-action policy. Voice passwords are replayable. A separate typed confirmation for sending/deleting mail, purchases and account changes is more useful than pretending wake codes identify household members.
  5. Packaging and soak tests. Signed installer/update rollback, suspend/resume, network-loss recovery, overnight CPU/RAM/power and bounded logging checks. Keep one user profile for now; accounts, permissions, voice identity and shared speakers need a deliberate design before mapping “29” and “30” to people.

Cartesia protocol references: streaming TTS bytes, manual streaming STT.