Overview

Your agent sends text to the router, all at once or streamed token by token from an LLM. The router splits it into sentences as it arrives and, for each one, checks your org’s cache. A sentence your org has synthesized before, with the same voice and settings, is replayed from the cache. Any other sentence goes to the provider (Sarvam, Raya or ElevenLabs) with the provider key you stored, streams back to you as the provider produces it, and is cached once the provider finishes it. You hear the sentences in order, as one reply.

Voice agents repeat themselves: greetings, confirmations, hold messages, the same answer to the same question. Each repeat served from the cache is a sentence your provider doesn’t synthesize or bill, and it starts playing sooner. The Usage page shows how much it saved.

• Each sentence reaches the provider verbatim, with your exact model, voice, language, speed, sample rate, format and options. The only change to the text is collapsing runs of whitespace.
• A cache hit replays a real provider take for that exact request: audio the provider made for the same sentence and settings.
• Audio is never spliced, re-encoded or altered. Every format is produced by the provider itself. A format the router can’t serve without re-encoding is refused rather than converted.
• A partial clip is never cached. A sentence is cached only once the provider has proven it complete. A take cut short by an error, a timeout or a dropped connection is never stored.

Providers don’t produce byte-identical audio even for two identical direct calls: each call is a new take. A cache hit is one of those real takes, replayed.

The router sends each sentence to the provider on its own, as the providers’ own LiveKit plugins do. Sentence boundaries come from the splitter each provider’s LiveKit plugin uses: LiveKit’s BlingFire tokenizer for ElevenLabs; for Sarvam and Raya, the router’s own splitter, built on Raya’s, that also ends sentences at the danda (।) and doesn’t split inside common abbreviations (Dr., a.m.), decimals or URLs.

When a sentence matches#

A sentence plays from the cache when all of these are the same as an earlier, completed one:

• your org (orgs never share audio, and every API key in an org shares its cache);
• the provider, model, voice, language, speed, sample rate, format and every option;
• the text, after whitespace is collapsed. Case and punctuation count, since they change how it’s spoken.

Defaults are filled in before matching, so leaving an option out and sending its default value share entries. The ways in share entries only when they send the provider the same request: the LiveKit plugin, /v1/stream and /v1/audio/speech do for the same settings, while the provider-API drop-ins cache the provider’s own request as your client sent it.

Some settings live at the provider and can change under the same name: a Sarvam pronunciation dictionary (dict_id), an ElevenLabs voice’s stored settings, an ElevenLabs dictionary named without a version. The router re-reads those about once a minute and includes them in the match, so after you change one, new sentences are synthesized afresh.

How long it is kept#

Your org’s retention, set in Settings: 365 days unless you change it, anywhere from 1 to 3,650 days, or 0 to stop caching.

• It counts from when the audio was cached. Playing it from the cache doesn’t extend it.
• Audio is deleted within about a day after its retention ends.
• A change applies to everything already cached: lowering it deletes older audio, raising it keeps audio longer.
• At 0, nothing new is cached and what was cached is deleted within about two days. Requests still work, each sentence synthesized by the provider.

What isn’t cached#

• Text. The cache stores audio under a one-way hash of the sentence and its settings. The sentence itself is never stored, and never logged.
• Anything incomplete. A sentence that failed, timed out or was cut off. A sentence the provider was already synthesizing when your agent was interrupted still finishes and is cached; sentences not yet sent to the provider are dropped.
• Sentences the provider can’t speak, which the router skips instead of failing the reply: for Sarvam, a sentence with no letter or digit (an emoji on its own); for Raya, punctuation alone.
• Sentences using a Sarvam pronunciation dictionary for the first time, until the router has read the dictionary (a moment after first use). They are synthesized and played, just not cached.
• Voice lists (GET /raya/v1/voices, GET /elevenlabs/v1/voices), passed through live.

• Our LiveKit pluginSwap your agent's TTS for tts_cache_router.TTS. Cache hits count as zero characters in LiveKit's metrics.
• Your provider's LiveKit pluginKeep livekit-plugins-sarvam, -elevenlabs or Raya's plugin and change its base URL and key.
• OpenAI-compatible HTTPPOST /v1/audio/speech, shaped like OpenAI's, so the OpenAI SDK works with a new base URL.
• WebSocket protocolStream text in and audio out from any language: the protocol the plugin speaks.
• Provider API drop-insSarvam's, Raya's and ElevenLabs' own APIs, mirrored: keep their SDKs and change the base URL.
• QuickstartSign up, store a key and hear your agent through the router in about ten minutes.

The router, this website and the LiveKit plugin are on GitHub. Its README shows how to run your own router with Docker Compose; point the plugin at it with base_url. The hosted router at api.ttscacherouter.dev runs the same code.