Sub-300ms Voice AI Latency
Web Audio Worklets, Silero VAD & Indic Telephony

Why Latency > 500ms Destroys Conversational Credibility

Human conversational psychology has an unforgiving physical limit: between two humans conversing naturally, pause intervals between speaker transitions average 200 to 300 milliseconds. When a caller experiences a delay of 600ms to 1200ms (typical of generic API chaining setups: OpenAI Whisper HTTP → GPT-4 → ElevenLabs), the interaction feels disjointed, awkward, and clearly robotic.

Worse yet, without instant interruption handling (barge-in), the synthetic voice talks over the user when they clarify or correct their sentence, leading to frustrating conversational collisions.

Running 16kHz PCM DSP on Dedicated Audio Render Threads

Traditional browser implementations rely on the deprecated ScriptProcessorNode or MediaRecorder slicing chunks every 250ms. This causes severe audio clicks whenever the main JavaScript thread is occupied by React DOM recalculations or CSS animations.

We replaced this with a custom AudioWorkletProcessor running directly inside the Web Audio engine's real-time priority thread:

// pcm-recorder-worklet.js (Runs in Web Audio thread)
class PCMRecorderWorklet extends AudioWorkletProcessor {
  process(inputs, outputs, parameters) {
    const input = inputs[0];
    if (!input || !input[0]) return true;
    
    const float32Data = input[0];
    const int16Buffer = new Int16Array(float32Data.length);
    
    // Fast conversion: Float32 [-1.0, 1.0] to Int16 PCM
    for (let i = 0; i < float32Data.length; i++) {
      const s = Math.max(-1, Math.min(1, float32Data[i]));
      int16Buffer[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
    }
    
    // Post directly to Web Worker for VAD and network transmission
    this.port.postMessage(int16Buffer.buffer, [int16Buffer.buffer]);
    return true;
  }
}
registerProcessor('pcm-recorder-worklet', PCMRecorderWorklet);

Silero ONNX in Web Workers: 35ms Human Interruption Detection

Instead of waiting for cloud servers to detect when the user speaks, we run an optimized 2.4MB ONNX quantized version of Silero VAD directly in a browser Web Worker.