Member of Technical Staff
RadixArk
Member of Technical Staff
RadixArk
Open-source AI infrastructure for large-scale inference
RadixArk is hiring a Member of Technical Staff — Performance in Palo Alto, CA — someone who can push LLM inference and training systems to the limit across real production workloads.
You’ll work on the performance-critical path of SGLang, Miles, and the RadixArk infrastructure stack: latency, throughput, GPU utilization, memory efficiency, scheduling, batching, kernel behavior, distributed execution, and cost-per-token. This is not a generic benchmarking role. You’ll be working on the systems that determine whether frontier-scale AI workloads are actually usable, affordable, and reliable in production.
Our customers care about real numbers: P99 latency, TTFT, tokens/sec/GPU, throughput under long-context workloads, cost-per-million tokens, RL rollout efficiency, and training-inference consistency. You’ll help us measure, debug, and improve these systems across NVIDIA, AMD, Google TPU, and cloud partner environments.
This role is for someone who loves performance debugging, understands that small systems details can create massive product impact, and wants to work at the frontier of AI infrastructure.
Analyze and improve performance across SGLang, Miles, and RadixArk production deployments
Benchmark LLM inference and training workloads across GPUs, TPUs, and cloud environments
Optimize latency, throughput, memory usage, batching, scheduling, routing, and GPU utilization
Investigate performance regressions in real customer environments
Work closely with kernel, runtime, distributed systems, and product engineers
Build internal tooling for profiling, tracing, benchmarking, and regression detection
Translate customer workload characteristics into concrete performance tuning strategies
Help define performance metrics that matter commercially, including cost-per-token and serving efficiency
Partner with customers and cloud partners on deep technical evaluations
Contribute performance insights back to open-source SGLang and Miles
Strong systems engineering background, especially in performance-critical software
Experience with GPU systems, distributed systems, inference serving, ML runtimes, or high-performance computing
Familiarity with profiling tools, performance debugging, tracing, and benchmark methodology
Comfort working with Python and C++
Experience with CUDA, Triton, Pallas, ROCm, XLA, or kernel-level optimization is a strong plus
Understanding of LLM inference concepts such as batching, KV cache, prefill/decode, speculative decoding, MoE, long context, and P99 latency
Ability to debug messy real-world performance issues across software, hardware, and infrastructure layers
Strong communication skills — you should be able to explain performance tradeoffs to both engineers and customers
Prior experience with production AI infrastructure, cloud GPU environments, or open-source ML systems is a plus
RadixArk is an infrastructure-first company built by engineers who've shipped production AI systems, created SGLang (30K+ GitHub stars, the fastest open LLM serving engine), and developed Miles (our large-scale RL framework). Founded by AI infrastructure veterans from xAI and NVIDIA, we're on a mission to democratize frontier-level AI infrastructure by building world-class open systems for inference and training. Our team has optimized kernels serving billions of tokens daily, designed distributed training systems coordinating 10,000+ GPUs, and contributed to infrastructure that powers leading AI companies and research labs.
Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.
RadixArk is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
RadixArk develops large-scale AI infrastructure for training and deploying models. Its core offering is the open-source SGLang engine that accelerates model inference, complemented by Miles, an open-source framework tailored for large-scale reinforcement learning and post-training tasks. The company provides managed infrastructure and production-grade tooling for model inference and post-training, with additional commercial products built on top of its open-source foundations. Revenue comes from managed hosting services and other paid offerings. Unlike many competitors, RadixArk pursues an open-core model to keep its core technology open and accessible to researchers, startups, and enterprises while monetizing via hosted services and premium tools. The goal is to democratize frontier AI infrastructure, reducing duplicated effort and enabling the AI community to train, deploy, and operate advanced models more efficiently and affordably.
Explore more opportunities at RadixArk