Cloud Inference Engineer
Luminal
Cloud Inference Engineer
Luminal
Serverless AI deployment with GPU kernels
CUDA + GPU inference optimization
vLLM, SGLang, or TensorRT-LLM experience
KV caching, paged attention, batching, token streaming, etc.
Distributed compute (with GPUs is a super plus)
No degree required
Luminal (YC S25) builds an AI compiler and serving stack that makes models 10x faster and production ready with one line.
Founding, on site in downtown SF. Ship low latency, high throughput model serving on Luminal Cloud.
Day to day responsibilities:
Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.
Conducting model performance reviews
Improve scheduler, batcher, autoscaling; profile latency, cost, utilization
Sometimes write kernels and, yes, occasional tasteful shitposting
San Francisco, CA, US · Full-time · On-site. Annual base salary: $150,000–$250,000 USD. Equity: 0.15%–0.75%. Visa: US citizen/visa only.
Luminal provides a serverless cloud platform for deploying AI models with greater speed and reliability. Its main offering is an open-source machine learning compiler that automatically translates PyTorch models into optimized GPU kernels (e.g., Flash Attention) without manual tuning, working directly with CUDA and Metal through Rust. The platform offers features like auto-scaling, auto-batching, and on-demand weight streaming to handle varying traffic and minimize idle GPU time, acting as a drop-in upgrade to PyTorch to simplify production deployment. Differences from competitors include an end-to-end compiler-driven path that generates high-performance GPU code without extra engineering and a focus on compute-lean, low-overhead deployment in a serverless framework. Luminal aims to make it easier and cheaper to move AI models from research to production by reducing the engineering burden and speeding up inference, with YC-backed funding and a San Francisco base.
Explore more opportunities at Luminal