Backend Software Engineer
Rockstar
Backend Software Engineer
Rockstar
AI-powered, human-led recruitment platform
Rockstar · San Francisco, United States · — · Posted 2026-11-04
Workplace: on_site
Rockstar is recruiting for a fast-growing AI infrastructure startup building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. The client’s mission is to deliver truly serverless intelligence to the AI world. Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.
About the Client
The client is building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. Their mission is to deliver truly serverless intelligence to the AI world.
Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.
The Role
The client is hiring a Software Engineer to help design, build, and scale the core systems that power large-scale routing, model training, and deployment.
The engineer will work on everything from distributed training pipelines, cloud-native infrastructure, and internal developer platforms that support fine-tuning, reinforcement learning, and inference at scale, stretching right up to the frontend. This role sits at the intersection of product engineering and ML systems—the engineer will collaborate closely with ML scientists while owning production-grade infrastructure operating at 99.99% uptime, alongside a team that has shipped hundreds of models at places like TikTok, MongoDB, and Neeva.
This is an ideal role for an early-career engineer who wants to work on real distributed systems, GPU workloads, and modern ML infrastructure, and actually see their work transform the AI capabilities of frontier application layer companies.
What You’ll Do
Build & Scale Core Infrastructure
Design and implement systems that support large-scale ML workloads, including fine-tuning and reinforcement learning.
Build distributed training, inference, and logging pipelines that are efficient, fault-tolerant, and observable.
Cloud & Systems Engineering
Work on cloud-native systems using containers and orchestration (e.g., Kubernetes).
Optimize systems for performance, reliability, and cost efficiency, especially for GPU-heavy workloads.
Implement monitoring, logging, and observability for long-running training jobs and production services.
Bring the Client’s Capabilities to Customers
Talk to customers. Join their calls, find what’s slowing them down, and ship the fix yourself.
Realize the client’s features. Develop the frontend and backend logic that enables customers to access the client’s capabilities.
Who You Are
Required
1–3 years of engineering experience, ideally working on production systems on the backend and infrastructure layer.
Strong fundamentals in distributed systems, networking, and backend architecture.
Comfortable writing code with and without AI and taking accountability for your code.
Excited to work on-site in San Francisco with a fast-moving early-stage team.
Powered by Workable
Rockstar provides pay-as-you-go AI-assisted recruitment for small to mid-market companies, combining an AI sourcing platform with human recruiters to speed hires and cut costs. It works with a 10-minute setup, AI sourcing and filtering of thousands of profiles, and human-driven campaign strategy, all with live pipeline visibility and personalized outreach. It differentiates itself by pairing AI with hands-on screening and strategy, offering contract-free, per-role pricing and a satisfaction guarantee. Its goal is to make hiring more affordable and scalable for growing businesses while giving access to top talent.
Explore more opportunities at Rockstar