Senior Product Engineer

Telnyx has launched LiveKit on Telnyx, a fully hosted platform for deploying voice AI agents with reduced costs and ultra-low latency. The platform allows developers to run existing LiveKit agents on Telnyx-owned infrastructure without code changes. By owning the entire infrastructure stack—carrier network, GPU clusters and telephony—Telnyx offers 50% lower speech-to-text and text-to-speech costs compared to LiveKit Cloud. The company is waiving session fees during the beta period, eliminating the current $0.01 per minute charge LiveKit customers typically pay. The platform achieves sub-200ms round-trip time by hosting speech models on colocated GPU infrastructure across 18 global points of presence. It includes enterprise telephony features and compliance standards including HIPAA, PCI and SOC 2. LiveKit on Telnyx is now available in beta.

Open-source wake word training in a single command. Wake words are the short spoken phrases, like "Hey Siri" or "Alexa", that activate a voice-enabled device or agent. They're the first step in any hands-free voice interaction, and getting them right matters: too sensitive and they fire constantly, too strict and users have to repeat themselves. Today LiveKit Incorporated is launching livekit-wakeword, an open-source wake word library built for simplicity and speed. Why LiveKit Incorporated built this. If you've tried training wake word models before, you know the pain: * Existing codebases are outdated, with broken dependencies everywhere. * Documentation is sparse or nonexistent, so training new models requires hours or even days of reverse-engineering. And even if you manage to train a model, you still end up with one that false-triggers constantly because you used the vanilla settings the authors provided. LiveKit Incorporated built livekit-wakeword to fix all of this. Now you can train your own wake word model from scratch, locally, with a single command. Use cases. Custom wake words unlock hands-free voice activation across a wide range of applications: * Voice agents: Give your AI agent a branded activation phrase ("Hey Jarvis," "OK Chef") instead of relying on a generic keyword. * Smart home assistant: Train a custom phrase for your home setup without depending on cloud services. * Robotics: Activate a robot with a spoken command in noisy warehouse or factory environments. * Kiosks & accessibility devices: Enable hands-free activation for retail, healthcare, or public-facing hardware. * In-car & embedded systems: Trigger voice control in vehicles or IoT devices running on constrained hardware. Performance. Even though its library is simple and fast, LiveKit Incorporated didn't sacrifice accuracy. Compared to openWakeWord, livekit-wakeword achieved dramatically better results across every metric: * 100x fewer false positives per hour * 60x lower detection error * 86% vs 69% recall | Metric | livekit-wakeword | openWakeWord | | False positives per hour (FPPH) | 0.08 | 8.50 | | Detection error tradeoff (AUT) | 0.0012 | 0.0720 | | Recall | 86% | 69% | FPPH measures how often the model incorrectly fires when no wake word was spoken - lower is better. AUT (area under the DET curve) captures the overall tradeoff between false positives and missed detections. See the full comparison for DET curves, test conditions, and detailed methodology. How it works. Under the hood, livekit-wakeword generates thousands of synthetic training samples using text-to-speech, then applies realistic audio augmentations (background noise, reverb, gain variation) to simulate real-world conditions. A lightweight convolutional-attention classifier trains on top of pre-computed audio embeddings, producing a small, fast model that generalizes well beyond its training data. Since its exported models use the same ONNX format and inference pipeline as openWakeWord, they're fully compatible. Your Home Assistant or legacy projects still work with zero changes. Part of the LiveKit ecosystem. livekit-wakeword is designed to work seamlessly with the LiveKit platform. Use a wake word to trigger a LiveKit Agent session. The wake word model runs locally on-device with minimal latency, and once activated, LiveKit handles the realtime audio streaming to your agent. Start building with livekit-wakeword. To train a new wake word model, install the library and run setup: 1 # install livekit-wakeword with training, evaluation, and export extras 2 pip install livekit-wakeword[train,eval,export] 3 4 # download required embedding models and datasets 5 livekit-wakeword setup Then create a config file for your wake word: 1 model_name: hey_robot 2 target_phrases: 3 - "hey robot" 4 5 n_samples: 10000 # synthetic training samples per class 6 model: 7 model_type: conv_attention # its new conv-attention classifier 8 model_size: small 9 steps: 50000 # training steps Check out the README for the full list of config options. Once your config is ready, you can train your model with a single command: 1 # generates synthetic data, augments, trains, and exports to ONNX 2 # your model will be saved to ./output/hey_robot/hey_robot.onnx 3 livekit-wakeword run configs/hey_robot.yaml That single command handles everything: synthetic data generation, augmentation, training, and ONNX export. You'll get a production-ready model file you can use right away. The exported model is a standard ONNX file, fully backward compatible with openWakeWord, so it drops into Home Assistant or any existing openWakeWord integration with zero changes. To run detection, just load the model and feed it audio: 1 from livekit.wakeword import WakeWordModel 2 3 # load your exported ONNX model 4 model = WakeWordModel(models=["hey_robot.onnx"]) 5 6 # feed 16kHz audio frames (int16 or float32) 7 scores = model.predict(audio_frame) 8 if scores["hey_robot"] > 0.5: 9 print("Wake word detected!") LiveKit Incorporated also provide a WakeWordListener that handles all the audio capture for you, so you can listen from the microphone without writing any audio code yourself: 1 from livekit.wakeword import WakeWordModel, WakeWordListener 2 3 model = WakeWordModel(models=["hey_robot.onnx"]) 4 5 # captures audio from the microphone and runs detection automatically 6 async with WakeWordListener(model, threshold=0.5) as listener: 7 while True: 8 detection = await listener.wait_for_detection 9 print(f"Detected {detection.name}!") For a complete example that uses wake word detection to spawn a LiveKit agent, check out hello-wakeword. Other runtimes. For production deployments, LiveKit Incorporated currently support Rust. More runtimes are on the roadmap. Future directions. On the hardware side, the current architecture already runs comfortably on single-board computers, but LiveKit Incorporated is taking it further. LiveKit Incorporated is building an end-to-end model that removes the need for a separate embedding model, making it small enough to run directly on ESP32 and other embedded microcontrollers. Want to get involved? Check out the repo and join its developer community to share what you're building.

Solving unwanted interruptions with Adaptive Interruption Handling. Knowing whose turn it is to speak remains one of the hardest problems in voice AI. You've probably already used its transformer-based End-of-Turn detection that tells the agent when the user is truly done speaking. But what about the other direction? When the agent is talking and the user wants to jump in? The naive solution of stopping the moment the user makes any sound sounds simple on paper. In reality it destroys the conversation. Most voice agents today rely on simple Voice Activity Detection (VAD) when speech is detected during the agent's turn. But VAD alone is not sufficient, because many sounds can trigger it: brief backchannels ("mm-hmm", "yeah"), user noises like sighs or coughs, or background sounds like typing, music or chatter. Treat every one of those as a full interruption and your agent becomes jittery and robotic. LiveKit Incorporated has been focused on solving this properly for a long time. Today LiveKit Incorporated is excited to announce Adaptive Interruption Handling is generally available in LiveKit Agents. How Adaptive Interruption Handling works. LiveKit Incorporated trained a brand-new audio-based interruption detection model specifically for this problem. When user speech is detected during the agent's turn, the model analyzes the user's audio stream within the first few hundred milliseconds of detected speech. It looks for distinctive acoustic characteristics of true interruptions, including: * Overall waveform shape * Strength and sharpness of speech onset * Duration of the signal * Prosodic features such as pitch and rhythm This allows it to quickly determine whether the user is beginning a new utterance or just making incidental sounds. To learn these patterns, the model was trained on examples of real interruptions, backchannels, and other sounds extracted from natural one-on-one conversations. It learned to discriminate between genuine attempts to interrupt and incidental speech or noise that would normally trigger a simple VAD. Architecturally, the system combines an audio encoder with a convolutional neural network (CNN) to extract and analyze acoustic patterns in the waveform. This design enables the model to identify the signatures of true interruptions while ignoring non-interruptive sounds, resulting in more natural and responsive conversations with the agent. The data challenge. In order to teach the model how to behave more human-like, LiveKit Incorporated needed to gather a diverse set of conversations that capture natural back-channeling and barge-ins. This kind of data is very sparse in human-agent conversations, because most voice agents today simply aren't able to handle it correctly. Instead, LiveKit Incorporated turned to human-to-human conversations. Its team went on a full data-gathering mission and collected hundreds of hours of real human speech across many different topics and language. The raw audio then went through a data enrichment pipeline, mixing in a variety of noises to simulate the real-world diversity of inputs LiveKit Incorporated expect to see. One particularly exciting outcome is that the model is multilingual and generalizes effectively to languages it has never seen before. Rather than simply memorizing patterns from the training data, it has learned the underlying conversational dynamics and can infer correctly in new scenarios. Benchmarks. LiveKit Incorporated evaluated the model on a held-out dataset and observed strong results in production: * 86% precision and 100% recall (at 500 ms overlap speech) * Rejects 51% of VAD-based barge-ins (false positives avoided) * Detects true barge-ins faster than VAD in 64% of cases * Completes inference in 30 ms or less * Median audio duration needed to trigger interruption: 216 ms * Consistent strong performance across noisy environments and multiple languages Using Adaptive Interruption Handling. The model is enabled by default in Python Agents v1.5.0+ and TypeScript Agents v1.2.0+. Every agent deployed to LiveKit Cloud gets it automatically with no extra models to deploy or manage. To fall back to classic VAD-based interruption detection, use the new turn_handling config: 1 session = AgentSession( 2 ... 3 turn_handling=TurnHandlingOptions( 4 interruption={ 5 "mode": "vad", 6 }, 7 ), 8 ) TypeScript 1 const session = new AgentSession({ 2 interruption: { 3 mode: "vad", 4 }, 5 }) When interruption.mode is not specified, it defaults to "adaptive" on LiveKit Cloud or in dev mode. This model is deployed directly in LiveKit Cloud data centers for optimal inference latency. LiveKit Incorporated has been building a family of models that significantly improve conversational flow. They're trained on proprietary data, tend to be larger, and are optimized for GPU inference, making them impractical to bundle into agent containers. Adaptive Interruption Handling is included at no extra cost for all agents deployed to LiveKit Cloud. For local development and testing, every plan includes 40,000 free inference requests per month. Try it today. Adaptive Interruption Handling is the missing piece that makes voice agents feel truly conversational instead of polite but robotic. The fastest way to try it is in the Agents Playground. Run an agent on your machine, speak over it, or backchannel and see the difference immediately. LiveKit Incorporated has also added a brand-new debugging panel with clear visualizations that show exactly when the model detects a barge-in versus a back-channel. It makes debugging and tuning extremely straightforward. Give it a try! LiveKit Incorporated'd love to hear what you build and any feedback you have.