SambaNova Systems

Enterprise AI platform with hardware stack

Principal Cloud Platform Engineer

Platform Engineering +1Confirmed live in the last 24 hours

Full-timeIn-person

About the Job

Requirements

A B.S. in Computer Science, Computer Engineering, or a related field is required.
At least 8 years of experience in Site Reliability Engineering, platform engineering, or distributed systems roles is required.
Experience supporting a large-scale, customer-facing service in a public cloud environment such as AWS, GCP, or Azure is required.
Strong programming skills in Python, Go, Rust, or Java, with recent hands-on coding, are required.
Experience designing and building a platform service, control plane, or scheduler used by multiple teams or customers is required.
Kubernetes experience, such as writing operators or controllers, running cluster upgrades, or configuring the scheduler, is required.
Experience with multi-tenant systems, including resource quotas, isolation, and fair sharing of capacity, is required.
Experience designing for failure, including retries, idempotency, and safe rollouts, is required.
A deep understanding of monitoring and observability principles and tools such as Prometheus, Grafana, ELK Stack, or Datadog is required.
Experience with Infrastructure as Code tools such as Terraform or CloudFormation is required.
Experience with CI/CD principles and tools such as Jenkins, GitHub Actions, or ArgoCD is required.
Strong Linux/Unix system administration fundamentals are required.

Responsibilities

Share ownership of the production inference service across regions, covering availability, latency, performance, change management, and capacity planning.
Design and build platform services, Kubernetes operators, and controllers that deploy, schedule, and scale model workloads.
Design scheduling for shared accelerator capacity, including quotas, priority, and fairness between customers.
Roll out upgrades without disrupting running workloads and define how the service recovers when a node or worker fails.
Design platform security, including tenant isolation, workload identity, and secrets management.
Stand up new regions with infrastructure as code in Terraform and Ansible, and build CI/CD pipelines for model and service deployments.
Build monitoring and alerting in Prometheus, Grafana, and Datadog, and define SLOs and SLIs that drive reliability work.
Find and eliminate performance bottlenecks, and design auto-scaling for variable inference loads.
Lead incident response and postmortems as part of a shared primary/secondary on-call rotation.
Forecast infrastructure needs and work with finance to manage cloud spend.

Desired Qualifications

Experience in a hybrid environment bridging cloud and on-premise/data center infrastructure is preferred.
Direct experience supporting ML/AI inference services in production is preferred.
Knowledge of model serving frameworks such as vLLM, SGLang, or Ray, including batching and KV cache management, is preferred.
Familiarity with GPU-accelerated computing and optimizing workloads for NVIDIA GPUs for mapping to RDUs is preferred.
Experience with batch or job scheduling systems such as Slurm, Volcano, or Kueue is preferred.
Experience with Kubernetes internals such as etcd, control plane high availability, or the scheduler is preferred.
Understanding of MLOps principles and practices is preferred.
Experience managing and tuning databases (SQL or NoSQL) and caching systems such as Redis or Memcached is preferred.

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide. At the core of SambaNova's technology is the RDU (Reconfigurable Dataflow Unit) — a chip built on a dataflow architecture rather than the traditional GPU model. Its decode performance is especially strong for agentic workloads like multi-turn agents, code generation, and long-running applications. RDUs are packaged into SambaRack, rack-scale hardware that lets customers deploy state-of-the-art models with better performance, greater energy efficiency, and faster time to value.

About the Team

The Cloud Platform team runs SambaNova’s production inference service. We handle capacity planning, deployment, monitoring, and incident response across regions in the United States, Asia, Europe, and Latin America.

About the Role

As a Principal Cloud Platform Engineer, you will design and build the software that deploys, schedules, and scales model workloads across our shared accelerator hardware, staying hands-on in the code while leading the technical design. You will set uptime and latency targets for our inference endpoints and share in the on-call rotation that keeps them running.

Responsibilities

Shared ownership of the production inference service across regions, covering availability, latency, performance, change management, and capacity planning
Designing and building platform services, Kubernetes operators, and controllers that deploy, schedule, and scale model workloads
Designing scheduling for shared accelerator capacity, including quotas, priority, and fairness between customers
Rolling out upgrades without disrupting running workloads, and defining how the service recovers when a node or worker fails
Designing platform security, including tenant isolation, workload identity, and secrets management
Standing up new regions with infrastructure as code in Terraform and Ansible, and building CI/CD pipelines for model and service deployments
Building monitoring and alerting in Prometheus, Grafana, and Datadog, and defining SLOs and SLIs that drive reliability work
Finding and eliminating performance bottlenecks, and designing auto-scaling for variable inference loads
Leading incident response and postmortems as part of a shared primary/secondary on-call rotation
Forecasting infrastructure needs and working with finance to manage cloud spend

Required Qualifications

B.S. in Computer Science, Computer Engineering, or related field
8+ years of experience in Site Reliability Engineering, platform engineering, or distributed systems roles
Experience supporting a large-scale, customer-facing service in a public cloud environment (AWS, GCP, Azure)
Strong programming skills in Python, Go, Rust, or Java, with recent hands-on coding
Experience designing and building a platform service, control plane, or scheduler used by multiple teams or customers
Kubernetes experience such as writing operators or controllers, running cluster upgrades, or configuring the scheduler
Experience with multi-tenant systems, including resource quotas, isolation, and fair sharing of capacity
Experience designing for failure, including retries, idempotency, and safe rollouts
Deep understanding of monitoring and observability principles and tools (e.g., Prometheus, Grafana, ELK Stack, Datadog)
Experience with Infrastructure as Code (e.g., Terraform, CloudFormation)
Experience with CI/CD principles and tools (e.g., Jenkins, GitHub Actions, ArgoCD)
Strong Linux/Unix system administration fundamentals

Preferred Qualifications

Experience in a hybrid environment bridging cloud and on-premise/data center infrastructure
Direct experience supporting ML/AI inference services in production
Knowledge of model serving frameworks like vLLM, SGLang, or Ray, including batching and KV cache management
Familiarity with GPU-accelerated computing and optimizing workloads for NVIDIA GPUs for purposes of mapping to RDUs
Experience with batch or job scheduling systems such as Slurm, Volcano, or Kueue
Experience with Kubernetes internals such as etcd, control plane high availability, or the scheduler
Understanding of MLOps principles and practices
Experience managing and tuning databases (SQL or NoSQL) and caching systems (Redis, Memcached)

Base Salary Range:

Base Pay Range

$210,000—$280,000 USD

Submission Guidelines
Please note that in order to be considered an applicant for any position at SambaNova Systems, you must submit an application form for each position for which you believe you are qualified.

EEO Policy
SambaNova Systems is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard basis of age (40 and over), color, disability, gender identity, genetic information, marital status, military or veteran status, national origin/ancestry, race, religion, creed, sex (including pregnancy, childbirth, breastfeeding), sexual orientation, and any other applicable status protected by federal, state, or local laws.

Benefits Summary for US-Based, Full-Time Employment Positions
SambaNova offers a competitive total rewards package, including the base salary, plus equity and benefits. We cover 95% premium coverage for employee medical insurance, and 77% premium coverage for dependents and offer a Health Savings Account (HSA) with employer contribution. We also offer Dental, Vision, Short/Long term Disability, Basic Life, Voluntary Life, and AD&D insurance plans in addition to Flexible Spending Account (FSA) options like Health Care, Limited Purpose, and Dependent Care. Our library of well-being benefits available to you and your dependents includes a full subscription to Headspace, Gympass+ membership with access to physical gyms, One Medical membership, counseling services with an Employee Assistance Program, and much more.

Base Pay Range 210000 - 280000 USD

About the Company

SambaNova Systems

Enterprise AI platform with hardware stack

Generating company summary.

Company size: 201-500
Company stage: Series F
Total funding: $2.5B
Headquarters: Palo Alto, CA
Founded: 2017

Recently Posted Jobs

Explore more opportunities at SambaNova Systems

See more jobs