# Brisvox > Brisvox is an open-source Rust engine for real-time voice agents: phone and browser calls, interruptions, LLM tools, many calls per container. A caller talks, the agent listens (speech-to-text), thinks (an LLM with tools) and answers (text-to-speech), and the caller can interrupt at any moment. Licensed MIT or Apache-2.0. Self-hosted: there is no hosted cloud. Things to know before quoting this site: - An agent is one TOML or YAML file; the server is configured only by BRISVOX_* environment variables. API keys come from the environment variable a provider table names, never from the agent file. - Providers and transports are Cargo features (for example all-providers, webrtc, sip). - Capacity numbers come from runs with fake provider servers that have fixed latencies. They measure the engine and its provider clients, not the providers. - The live latency run was one evening on one Windows machine over the public internet, 100 turns per system. - Pipecat was measured at version 1.12.0 only. LiveKit Agents was not measured: nothing here says Brisvox is faster than LiveKit Agents. - Side-effect tools never run speculatively and never run twice for one call id. ## What Brisvox is Brisvox is an open-source engine, written in Rust, for real-time voice agents on phone calls and browser sessions. The caller talks, the agent transcribes, thinks with an LLM and its tools, and answers in a synthesised voice, and the caller can interrupt at any moment. One brisvox-server process runs hundreds of isolated calls per container: every call is an isolated async session, and a panic or a provider failure in one cannot touch another. Model inference runs on a dedicated pool, never on the async workers. - Repository: https://github.com/bilal07karadeniz/brisvox - Documentation (the book): https://bilal07karadeniz.github.io/brisvox/ - License: MIT or Apache-2.0, at your option - Latest release: v0.1.3, published 2026-10-31 (https://brisvox.com/changelog/) - Packages: crates.io brisvox and brisvox-server; PyPI brisvox (installs brisvox-agent); npm brisvox, @brisvox/agent, @brisvox/web; container image ghcr.io/bilal07karadeniz/brisvox-server ## How a call flows A call is one session: a chain of stages connected by bounded queues. Each call has its own chain. 1. AudioIn: Resamples the call to 16 kHz and forwards it at once. A copy goes to the VAD through a bounded, drop-oldest queue, so audio never waits on a model. 2. VadWorker: Silero voice-activity detection as jobs on the dedicated inference pool. Emits backdated speech start and stop events. 3. SttStage: Streams audio to the speech-to-text provider. At the VAD stop it sends Finalize immediately, in parallel with the end-of-turn model. 4. TurnDriver: The single authority over turns and epochs: end-of-turn decisions, barge-in judgement, the fast Clear towards the transport. 5. LlmRunner: One task per turn: streams the LLM, runs the tool loop, cuts the context to the text that was actually played. 6. SegmenterStage: Cuts the streaming text into speakable chunks. The first clause goes to the voice after one word. 7. TtsStage: Streams text to the text-to-speech provider and cuts audio into 20 ms frames at the call rate. 8. TransportOut: Paces audio onto the transport, tracks what was actually played and executes Clear. A speech-to-speech model (OpenAI Realtime, Gemini Live) can replace everything between AudioIn and TransportOut with one stage. ## Interruptions and turn detection Every user turn is a numbered epoch. Bot-side frames carry the epoch they were made for, and every stage drops frames from a stale epoch. When a caller barges in: a new epoch starts; a Clear goes straight to the transport (measured 0.022 ms at p95 from decision to applied, budget 1 ms, mock providers, Windows); the LLM round, the segmenter and the open TTS contexts are cancelled; and the LLM context is cut to the text that was actually played, using playback marks, playback positions and TTS word timestamps. The end of a turn is decided by a state machine with injected time. With the adaptive policy, when the VAD sees the caller fall silent, the STT Finalize and Smart Turn v3.2 (an on-device end-of-turn model) start at once; the turn ends when the model says the utterance is complete and the STT final has arrived, with a fallback silence deadline. Tools are marked read_only or side_effect. Read-only tools may run on a speculative LLM start and are cancelled on a barge-in. Side-effect tools never run speculatively, never run twice for one call id, and are not cancelled by an interruption. A later turn or another call is not covered by the never-twice rule: add your own idempotency key for rules such as one refund per order, ever (https://github.com/bilal07karadeniz/brisvox/blob/main/docs/TOOLS.md). ## Providers and transports | Stage | Cloud | On your own machine | |---|---|---| | Speech to text | Deepgram, AssemblyAI, Gladia | sherpa-onnx (CPU), any server speaking the OpenAI Realtime transcription protocol | | Language model | OpenAI, Anthropic, Gemini, Groq, Together, Fireworks, OpenRouter | Ollama, vLLM, any OpenAI-compatible endpoint | | Text to speech | Cartesia, ElevenLabs, Deepgram Aura | Kokoro-FastAPI, Chatterbox, any server of the OpenAI /v1/audio/speech shape | | Speech to speech | OpenAI Realtime, Gemini Live | not available | | On device | none needed | Silero VAD (compiled in), Smart Turn v3.2 end-of-turn detection (ONNX Runtime) | Transports: WebSocket (Generic audio protocol, built in); Twilio Media Streams (Built in, with transfer, DTMF and hang-up); Telnyx media streaming (Built in, same call control); WebRTC (Cargo feature webrtc, WHIP signaling, Opus); SIP trunk (Cargo feature sip, UDP, TCP or TLS with SRTP); RTVI (Pipecat's client SDKs connect unchanged). G.711 mu-law and A-law. No media server is needed. English and Turkish are supported. ## Benchmarks Source: https://brisvox.com/benchmarks/. Every number is in a file committed under bench/results/ in the repository (https://github.com/bilal07karadeniz/brisvox/blob/main/bench/results/README.md). ### Concurrent calls per container (fake providers) Cloud containers on AMD EPYC 9655P, Linux, 16 and 17 October 2026, release builds, one system at a time, 3 runs per cell on fresh containers. Every call went through each system's own provider client code to fake providers that speak the real Deepgram and OpenAI-compatible protocols over TLS with fixed latencies (STT final 150 ms after 300 ms of endpointing, LLM first token 350 ms, TTS first audio 150 ms), with the real Silero VAD and Smart Turn models on every call. The latency SLO is a p95 voice-to-voice of at most 1110 ms (the fake providers' own 860 ms plus 250 ms). LiveKit Agents was not measured. | Container and limit | Brisvox (median, runs) | Pipecat 1.12.0 (median, runs) | LiveKit Agents | |---|---|---|---| | 4 vCPU / 4 GB, within the latency slo | 250 (260, 250, 220) | 1 (1, 1, 0) | not measured | | 4 vCPU / 4 GB, no failed call, no lost turn | 430 (430, 480, 400) | 10 (10, 10, 10) | not measured | | 8 vCPU / 8 GB, within the latency slo | 470 (480, 460, 470) | 10 (10, 5, 10) | not measured | | 8 vCPU / 8 GB, no failed call, no lost turn | 780 (740, 780, 800) | 10 (10, 20, 10) | not measured | Server memory at the maximum (PSS summed over the container's processes): 4 vCPU / 4 GB: Brisvox 753 MB at 430 calls, Pipecat 2.4 GB at 10 calls; 8 vCPU / 8 GB: Brisvox 1300 MB at 780 calls, Pipecat 3.3 GB at 10 calls. Per call at the 4 vCPU / 4 GB maximum, idle baseline included: Brisvox 1.8 MB and 0.007 cores, Pipecat 239.9 MB and 0.137 cores. ### Voice-to-voice latency on real providers Deepgram nova-3 (STT), OpenRouter openai/gpt-6-luna with reasoning effort none pinned to one upstream (LLM), Deepgram Aura aura-2-thalia-en (TTS). 5 concurrent calls, 100 turns per system in strictly alternating blocks, windows development machine over the public internet, one evening (2026-10-14). | System | p50 | p90 | p95 | p99 | |---|---|---|---|---| | Brisvox | 1846 ms | 2209 ms | 2876 ms | 5199 ms | | Pipecat 1.12.0 | 1972 ms | 2980 ms | 4348 ms | 6141 ms | The median gap (126 ms) is within the LLM provider's own variance. Pipecat sent more than one LLM request in 56 of 100 turns, which is where its tail comes from; the cause was not investigated. The same nominal 100 ms of silence before the VAD stop is 96 ms in Pipecat and 128 ms in Brisvox in this setup. ### Engine overhead (mock providers, Windows) | Stage | p50 | p95 | Budget at p95 | |---|---|---|---| | End of turn to LLM request | 0.051 ms | 0.089 ms | 3 ms | | First TTS audio to transport | 0.035 ms | 0.064 ms | 2 ms | | Barge-in decision to Clear applied | 0.012 ms | 0.022 ms | 1 ms | ## Comparison: Brisvox, LiveKit Agents, Pipecat Source: https://brisvox.com/compare/. Measured rows come from this project's own harness. Documented rows are what each project's own documentation said on 2026-10-14; they are not benchmarks. No independent third-party density comparison of the three was found. ### Measured | Measured here | Brisvox | LiveKit Agents | Pipecat 1.12.0 | |---|---|---|---| | Concurrent calls within the latency SLO (p95 voice-to-voice at most 1110 ms), 4 vCPU / 4 GB. Median, runs in brackets | 250 (260, 250, 220) | not measured | 1 (1, 1, 0) | | Most concurrent calls with no failed call and no lost turn (any latency), 4 vCPU / 4 GB | 430 (430, 480, 400) | not measured | 10 (10, 10, 10) | | Server memory at that maximum, 4 vCPU / 4 GB (PSS summed over the container’s processes) | 753 MB at 430 calls | not measured | 2.4 GB at 10 calls | | Concurrent calls within the latency SLO, 8 vCPU / 8 GB | 470 (480, 460, 470) | not measured | 10 (10, 5, 10) | | Most concurrent calls with no failed call and no lost turn, 8 vCPU / 8 GB | 780 (740, 780, 800) | not measured | 10 (10, 20, 10) | | Server memory at that maximum, 8 vCPU / 8 GB | 1.3 GB at 780 calls | not measured | 3.3 GB at 10 calls | | Per call at the 4 vCPU / 4 GB maximum: memory and CPU, idle baseline included | 1.8 MB, 0.007 cores | not measured | 239.9 MB, 0.137 cores | | Voice-to-voice on real providers, English, p50 / p95 (5 concurrent calls, 100 turns, Deepgram Aura voice; Windows, public internet) | 1.85 s / 2.88 s | not measured | 1.97 s / 4.35 s | ### Documented (2026-10-14) | Topic | Brisvox | LiveKit Agents | Pipecat | |---|---|---|---| | Language | Rust engine; your agent in Python or TypeScript (SDKs on PyPI and npm) or in Rust | Python (livekit-agents 1.8.7); Node.js in a separate repository | Python (pipecat-ai 1.12.0) | | License | MIT or Apache-2.0 | Apache-2.0; the turn-detector text model and v1-mini use the LiveKit Model License | BSD-2-Clause | | Runtime model | One process, one isolated async session per call, models on a dedicated thread pool | One OS process per job, “each in its own process for isolation” | Documented as one session per process or container | | Needs a media server | No: WebSocket, Twilio and Telnyx built in; WebRTC and SIP are cargo features | Yes: a LiveKit server (Cloud or self-hosted), the agent joins a room | No for WebSocket and SmallWebRTC; the Daily and LiveKit transports use those vendors’ infrastructure | | Telephony | Twilio and Telnyx media streams, SIP trunk listener, DTMF, transfer | LiveKit SIP service: Twilio, Telnyx, Plivo and others, DTMF, cold and warm transfer | Twilio, Telnyx, Plivo, Exotel serializers; no SIP stack of its own found in the pages read (inferred) | | Turn detection | Smart Turn v3.2 (ONNX, on device) with an adaptive policy | Own turn-detector models: a text model (396 MB on disk, about 50 to 160 ms per turn on CPU) and the audio model v1-mini (size and cost not stated) | Smart Turn v3 (8 MB int8) | | Turkish | First class, with its own quality notes | Listed by both turn detectors | Listed by Smart Turn | | Interruptions | Turn epochs; the context keeps only the text actually played | Adaptive mode separates interruptions from backchannel and resumes after a false one | Cancels LLM and TTS, flushes audio, commits only the played text; a minimum-words turn-start strategy and a Krisp backchannel filter | | Tools | read_only and side_effect kinds, idempotency keys, never speculative; written in Rust, Python or TypeScript | Function tools, async tools, handoff, MCP (Python), provider-side tools; parallel calls not stated on the page read | Parallel calls by default, async tools, per-tool timeouts, cancel on interruption; MCP not checked | | Speech-to-speech | OpenAI Realtime, Gemini Live | Many realtime models | Several | | Overload handling | Admission control and graceful drain built in | Default load function (5 s average CPU, threshold 0.7) | None found in the docs; autoscaling guidance | | Hosted cloud | None, self-host only | Optional LiveKit Cloud | Optional Pipecat Cloud | | Vendor sizing guidance | None; see the measured rows | 4 cores / 8 GB for 10 to 25 jobs; a vendor test of 30 agents peaked at about 3.8 cores and 2.8 GB | Pipecat Cloud profile of 0.5 vCPU / 1 GB per session | Sources for the documented rows: https://pypi.org/project/livekit-agents/ , https://github.com/livekit/agents-js , https://github.com/livekit/agents , https://docs.livekit.io/agents/server/lifecycle/ , https://docs.livekit.io/agents/logic/turns/turn-detector/ , https://docs.livekit.io/deploy/custom/deployments/ , https://docs.livekit.io/telephony/ , https://docs.livekit.io/agents/logic/turns/ , https://docs.livekit.io/agents/logic/tools/ , https://docs.livekit.io/agents/models/realtime/ , https://docs.livekit.io/agents/server/options/ , https://pypi.org/project/pipecat-ai/ , https://github.com/pipecat-ai/pipecat , https://docs.pipecat.ai/pipecat/deployment/running-bots-in-production , https://docs.pipecat.ai/pipecat-cloud/enterprise/sizing , https://docs.pipecat.ai/pipecat/learn/transports , https://docs.pipecat.ai/api-reference/server/utilities/turn-detection/smart-turn-overview , https://github.com/pipecat-ai/smart-turn , https://docs.pipecat.ai/pipecat/fundamentals/interruptions , https://docs.pipecat.ai/pipecat/learn/function-calling. Notes with every URL: https://github.com/bilal07karadeniz/brisvox/blob/main/docs/research/frameworks-2026-10-14.md ## Quick start No API keys (everything on your own computer; needs Docker, 4 or more cores, 8 GB of memory; the first start downloads about 6.5 GB): git clone https://github.com/bilal07karadeniz/brisvox && cd brisvox docker compose -f deploy/local/compose.yaml up # then open http://localhost:8088/demo?force=ws Container image: docker run --rm ghcr.io/bilal07karadeniz/brisvox-server:latest --version From crates.io (Rust 1.99 or newer, a C compiler and cmake): cargo install brisvox-server --locked --features all-providers,call-control,smart-turn-ort,mcp,webrtc Prebuilt binaries for Linux (x86_64, aarch64; glibc 2.39 or newer), macOS (aarch64) and Windows (x86_64) are on https://github.com/bilal07karadeniz/brisvox/releases. SDKs: pip install brisvox (Python 3.11 or newer); npm i brisvox (Node 20 or newer; brisvox/web is the browser client); cargo add brisvox --features all-providers (Rust tools in your own binary). Validate an agent file offline before running it: brisvox-server check agent.toml --require-secrets ## Frequently asked questions ### What is Brisvox? Brisvox is an open-source real-time voice agent engine written in Rust. One brisvox-server process runs many isolated phone or browser calls: the caller talks, speech-to-text listens, an LLM with tools answers, text-to-speech speaks, and the caller can interrupt at any moment. The agent is one TOML or YAML file. Your own tools run in your application through the Python or TypeScript SDK, or in Rust inside the server binary. ### What license is Brisvox under, and what does it cost? Brisvox is dual-licensed under MIT or Apache-2.0, at your option. The software is free and there is no hosted cloud, so you pay only the STT, LLM, TTS and telephony providers you choose. Each call reports what providers bill by (LLM input, cached and output tokens, STT audio seconds, TTS characters) as a session.usage event and as Prometheus totals per provider and model. The speech test fixtures in the repository are under CC BY-SA 4.0 rather than the code license. ### Does Brisvox have a hosted cloud? No. Brisvox is self-hosted only: a prebuilt binary, a container image or a build from source, on your own machines. ### Is Brisvox production-ready? Brisvox is a 0.x project, first released on 15 October 2026. It follows Semantic Versioning, and its library API still changes between releases; the changelog lists each change. For running it, the server has health and readiness endpoints, Prometheus metrics, optional OpenTelemetry traces, admission control that refuses new calls instead of degrading running ones, graceful drain and per-stage provider failover. No external security audit is claimed. Measure it on your own calls and hardware before you rely on it. ### Who makes Brisvox? Brisvox is made by Bilal and developed in the open on GitHub at github.com/bilal07karadeniz/brisvox. Issues and pull requests are welcome there. ### How many concurrent calls can one server handle? In the cloud capacity runs of 16 and 17 October 2026 (Linux containers on AMD EPYC 9655P), one 4 vCPU / 4 GB container held 250 concurrent calls within a p95 voice-to-voice limit of 1110 ms, and 430 with no failed call or lost turn at any latency. An 8 vCPU / 8 GB container held 470 within the same limit and 780 at any latency (p95 about 3.7 s at 780). These runs used fake providers that speak the real Deepgram and OpenAI-compatible protocols over TLS with fixed latencies, and the real Silero VAD and Smart Turn models on every call. They measure the engine and its provider clients, not the providers. At the 4 vCPU maximum Brisvox used about 1.8 MB of memory and 0.007 cores per call. ### How low is the latency? On real providers (Deepgram nova-3, OpenRouter openai/gpt-6-luna pinned to one upstream, Deepgram Aura voice), with 5 concurrent calls and 100 turns per system, the median voice-to-voice time was 1.85 s for Brisvox and 1.97 s for Pipecat 1.12.0, and the p95 was 2.88 s against 4.35 s. The run was on one Windows machine over the public internet, and the median gap is within the LLM provider's own variance. The engine's own added time is 0.089 ms at p95 from the end of turn to the LLM request, against a budget of 3 ms, measured over mock providers. ### Has Brisvox been measured against LiveKit Agents? No. LiveKit Agents was not part of the capacity or latency runs, so this site makes no speed, latency or density claim against it. The LiveKit facts on the comparison pages come from LiveKit's own documentation. ### What does it cost to run a call? Brisvox does not set provider prices. Each call reports the units providers bill for: LLM tokens (input, cached, output), STT audio seconds and TTS characters, with duration, turns and latencies. They arrive as a session.usage event, in the transcript, in the log and as Prometheus totals per provider and model. Units, not money. ### How does Brisvox handle interruptions (barge-in)? Every user turn is a numbered epoch, and frames from a stale epoch are dropped at every stage, so audio from an answer the caller already cut off does not play. A Clear goes straight to the transport, and the LLM round, the segmenter and the open TTS contexts are cancelled. After the barge-in the LLM context holds only the text that was actually played, so the model does not believe it said words the caller never heard. ### What about a cough, a door or a "mm-hmm"? The interruption policy judges every candidate: a minimum number of words, backchannel lists per language such as "yeah" and "uh-huh", an echo guard that recognises the agent's own words coming back, and a level gate against a quieter second voice. In 0.1.4 (November 2026), pause-then-decide is on by default: at a barge-in candidate the agent pauses its audio and, within a decision window of 1500 ms by default, either interrupts or resumes the same response from the paused point. Any recognised word from the caller interrupts at once. ### How does Brisvox decide that the caller has finished speaking? When the Silero voice activity detector sees the caller fall silent, two things start at once: the speech-to-text Finalize and Smart Turn v3.2, an on-device end-of-turn model that runs on a dedicated inference pool. If the model says the utterance is complete, the turn ends as soon as the STT final arrives; if not, the turn waits, with a fallback silence deadline. A fixed-silence policy is also available. ### Can a tool with side effects run twice? Tools are marked read_only or side_effect. A side-effect tool never runs speculatively and never runs twice for one call id, and an identical repeat in the same turn is refused. A later turn or another call is not covered: for a rule such as one refund per order, ever, add your own idempotency key. The exact scope is in docs/TOOLS.md. ### Can an agent use MCP tools? Yes. An agent file can name MCP (Model Context Protocol) servers, as a program the server starts or a streamable HTTP endpoint. The tools on a server's allowlist are offered to the LLM with the same guarantees as the server's own tools. The cargo feature is mcp, and it is in the prebuilt binaries and the container image. ### Which languages does Brisvox support? Does it support Turkish? English and Turkish are supported. Turkish is a first-class language: the TTS stage rewrites written forms such as amounts, dates and times into the words a person says, and the interruption policy has Turkish backchannel and command lists. An agent can list several languages and follow the caller from one to the other in the middle of a call. ### Which providers does Brisvox support? Speech-to-text: Deepgram, AssemblyAI, Gladia, sherpa-onnx on your own machine, and servers that speak the OpenAI Realtime transcription protocol. Language models: OpenAI, Anthropic, Gemini and any OpenAI-compatible endpoint such as Groq, Together, Fireworks, vLLM, Ollama and OpenRouter. Text-to-speech: Cartesia, ElevenLabs, Deepgram Aura and local servers of the OpenAI /v1/audio/speech shape such as Kokoro-FastAPI and Chatterbox. Speech-to-speech: OpenAI Realtime and Gemini Live. ### Does Brisvox need a LiveKit or other media server? No. The generic WebSocket, Twilio Media Streams and Telnyx media streaming transports are built in, and WebRTC and a SIP trunk listener are optional cargo features. Callers connect to brisvox-server directly. ### Does Brisvox work with Twilio, Telnyx and SIP trunks? Yes. Twilio Media Streams and Telnyx media streaming are built in, and DTMF, call transfer and hang-up come from the call-control feature, which the prebuilt binaries include. A SIP trunk listener (UDP, TCP or TLS, with SRTP) is the separate cargo feature sip, which is not in the prebuilt binaries, so you build with it added. Phone audio is G.711 mu-law or A-law. ### How do I put a voice agent on a website? Use @brisvox/web, one dependency-free ES module for a talk button: WebRTC first, with an automatic WebSocket fallback on the same port when WebRTC media cannot get through, plus microphone, playback, barge-in and a clean hang-up. Build the server with the webrtc feature. ### Can I run a voice agent with no API keys? Yes. docker compose -f deploy/local/compose.yaml up runs speech recognition (sherpa-onnx), a language model (Ollama) and a voice (Kokoro) on your CPU, and you talk to it in the browser. It needs Docker, 4 or more cores and 8 GB of memory, and the first start downloads about 6.5 GB. Nothing leaves the machine. ### Can I self-host Brisvox with Docker? Yes. A multi-architecture image (linux/amd64 and linux/arm64) with the Smart Turn model inside is at ghcr.io/bilal07karadeniz/brisvox-server. Prebuilt binaries cover Linux x86_64 and aarch64 (glibc 2.39 or newer), macOS aarch64 and Windows x86_64, each release with a SHA256SUMS file. Listeners fail closed: off loopback the server refuses to start without authentication (a session secret or a static token). ### Which SDKs are available? Python: pip install brisvox (Python 3.11 or newer). TypeScript: npm i brisvox (Node 20 or newer), which includes the agent SDK and @brisvox/web, the browser client. The SDK runs your tools in your own process, and the server connects out to it over the agent protocol. Tools can also be written in Rust inside your own server binary with cargo add brisvox. ### Is Brisvox a Pipecat alternative? Can I keep my Pipecat clients? Yes, for phone and browser voice agents that you want to self-host as one process. Brisvox accepts RTVI, the protocol of Pipecat's client SDKs, so those clients can connect to a Brisvox server. The comparison page lists what Pipecat documents that Brisvox does not have. ## Releases ### v0.1.3 (2026-10-31) - Noise suppression of the caller's audio - Reload the agent file without a restart - Language switching mid-call - Cheaper turn detection is the default - Adapter API (library users) - Background speech and echo handling Full notes: https://github.com/bilal07karadeniz/brisvox/releases/tag/v0.1.3 ### v0.1.2 (2026-10-23) - Keyless local stack - MCP tools for agents - Grafana dashboard - Cheaper turn detection, opt-in - Benchmarks - TLS certificate checks Full notes: https://github.com/bilal07karadeniz/brisvox/releases/tag/v0.1.2 ### v0.1.1 (2026-10-15) - Usage and cost per call - Learned admission ceiling - Cautious start - A second agent per server Full notes: https://github.com/bilal07karadeniz/brisvox/releases/tag/v0.1.1 ### v0.1.0 (2026-10-15) - Engine - Server - Transports - Providers - On-device models - Conversation Full notes: https://github.com/bilal07karadeniz/brisvox/releases/tag/v0.1.0