The Fully-Local Voice Stack

This is the real loop behind theLAB's ADI voice interface: say "hey Addie," talk, and get a spoken reply — with nothing leaving the tailnet. Four stages chain together, and three of them run on local hardware (only optional cloud models ever reach out). Wake detection and ASR live on apexlabs-voicebox, the LLM runs on your-llm-host, and TTS comes back from the same voicebox machine.

Every host below is reachable over a private Tailscale network, so the entire round-trip — audio in, transcript, generation, audio out — stays on machines we own.

# the ADI voice round-trip, all over the tailnet
 browser mic (16 kHz mono Int16 PCM, 80 ms frames)
        │  ws ── 1280-sample binary frames
        ▼
┌──────────────────────────────────────────────┐
│  apexlabs-voicebox                            │
│   [1] openWakeWord  ws://<host>:9293/ws        │  hey_addie.onnx
│        └─ score ≥ 0.5 → WAKE                   │
│   [2] capture → silence → Parakeet ASR :9292   │  {text}
│   [4] OmniVoice TTS :8881  (sha256 cache)      │  wav bytes
└───────────────┬──────────────────────────────┘
                │  transcript          ▲ reply text
                ▼                      │
        [3] Ollama  your-llm-host 100.64.87.104:11434

#01Wake word with openWakeWord

The browser captures mic audio as 16 kHz mono Int16 PCM and streams it as 1280-sample (80 ms) binary frames over a WebSocket to the wake server at ws://<host>:9293/ws on apexlabs-voicebox. Each socket gets its own stateful openWakeWord instance — state is per-connection, so two clients never bleed scores into each other. We run the hey_addie.onnx model; every frame is scored and compared against WAKE_THRESHOLD.

# one stateful oww per WebSocket connection
oww = Model(wakeword_models=["hey_addie.onnx"])
WAKE_THRESHOLD = 0.5

# for each 1280-sample (80 ms) Int16 frame off the socket:
score = oww.predict(frame)["hey_addie"]
if score >= WAKE_THRESHOLD:
    fire_wake()          # start capturing the utterance

#02Capture and transcribe with Parakeet

Once the wake fires, the server records the utterance until it hears quiet: SILENCE_FRAMES_NEEDED=12 frames of silence (≈ 960 ms) ends the capture. A COOLDOWN_FRAMES=25 window (≈ 2 s) then blocks a re-fire so one "hey Addie" can't trigger twice. The captured WAV goes to Parakeet (NVIDIA NeMo), exposed on an OpenAI-compatible endpoint.

files = {"file": ("utterance.wav", wav_bytes, "audio/wav")}
data  = {"response_format": "json"}
r = requests.post(
    "http://<host>:9292/v1/audio/transcriptions",
    files=files, data=data, timeout=60,
)
transcript = r.json()["text"]

#03Generate with Ollama

The transcript becomes a user turn sent to Ollama on the primary host your-llm-host over the tailnet. This is a standard /api/chat call — any ADI model can answer, including the distilled local models, with optional cloud models the only thing that ever leaves the network.

payload = {
    "model": "adi-qwen3.5-4b-glm5.2-general",
    "messages": [
        {"role": "system", "content": ADI_SYSTEM},
        {"role": "user",   "content": transcript},
    ],
    "stream": False,
}
r = requests.post("http://100.64.87.104:11434/api/chat",
                  json=payload, timeout=300)
reply = r.json()["message"]["content"]

Because the reply is what gets spoken, this is also where tool calls and structured outputs slot in — see Tool Calling & Structured Outputs.

#04Speak with OmniVoice TTS

The reply text goes to OmniVoice (the VibeVoice / Qwen3-TTS family) back on apexlabs-voicebox, also OpenAI-compatible. It returns raw WAV bytes the browser plays. Crucially, OmniVoice caches audio on disk keyed by a sha256 of the request, so any repeated phrase — "Sure, done." "I didn't catch that." — comes back instantly from cache instead of re-synthesizing.

payload = {
    "model":  "omnivoice",
    "voice":  "adi_ref",        # the ADI reference voice
    "input":  reply,
    "format": "wav",
}
r = requests.post("http://<host>:8881/v1/audio/speech",
                  json=payload, timeout=120)
audio_bytes = r.content         # sha256(request) cache → repeats are instant

#Putting it together

• Wake and ASR (:9293, :9292) and TTS (:8881) all live on apexlabs-voicebox — one box owns audio in and audio out.
• Only the LLM hop leaves the voicebox, going to your-llm-host at 100.64.87.104:11434 over Tailscale.
• Nothing but optional cloud models ever leaves local hardware — the mic audio, the transcript, and the synthesized voice never touch a third-party API.