Architecture breakdown · Sully.ai

Multilingual Voice AI & Runtime Diagnostic Harness

AI receptionist for healthcare: voice agents for scheduling and clinic operations. Outbound call reliability, LiveKit dispatch, UAE data residency and the MENA launch.

SYSTEM ARCHITECTURE // Sully.ai
Dubai, UAE · SEP 2025 — OCT 2026

Multilingual Voice AI & Runtime Diagnostic Harness

PROBLEM & CONSTRAINTS

Enterprise healthcare launches demand low-latency multilingual voice over live telephony and immediate root-cause visibility.

ARCHITECTURAL SOLUTION

Engineered end-to-end voice & harness reliability layers spanning Twilio/LiveKit ingress, ASR/LLM orchestration, deterministic evals, and trace telemetry.

PRODUCTION DATA & CONTROL FLOW:
[Telephony / SIP / Twilio] ──► [LiveKit Real-Time Media Router]
[LiveKit Stream] ──► [Speechmatics ASR] ──► [AI Agent Harness & Guardrails]
[Agent Harness] ──► [Eval & Runtime Trace Collector] ──► [Clinical EHR / Backend]
KEY PRODUCTION OUTCOMES:
  • ▸ Removed ~15 s of dead air at pickup on outbound calls: the agent was greeting the caller while the line was still ringing.
  • ▸ Made agent dispatch reliable on LiveKit: retry-and-verify, no duplicate agents in one call, no silent calls.
  • ▸ Designed UAE audio-residency guardrails so TTS and STT fallbacks stay within approved regional providers.
TRY IT // VOICE ACTIVITY DETECTION

At Sully the agent greeted callers while the line was still ringing, so they picked up to ~15 s of silence. The fix: wait until the person speaks, then greet. Here is that building block, running in your browser: Silero VAD v6, a 0.5 MB model that hears speech in 32 ms frames.

~15 s Sully, before the fix: dead air at pickup
— ms This model, on your device: time to notice your voice

Audio stays on your device; nothing is recorded or sent.

Measured from the 32 ms frame where your voice rose above the room to the model’s decision, plus the compute time. Your voice can start anywhere in that first frame, so it may read up to 32 ms high; the look-back stops at about a second. Mic and OS buffering before the page gets the audio are not counted. At Sully the win was not a faster model but the agent waiting for a voice instead of a ringing line.

← Sully.ai in the experience timeline

COPIED TO CLIPBOARD