D

Docker

Containerization platform for building apps

Staff Software Engineer - AI Gateway

AI Application EngineeringUpdated on Nov 13
Full-timeRemote in USA$170.3k - $275.6k

About the Job

Requirements

  • At least 8 years of backend engineering experience and a track record of owning production-grade systems at significant scale.
  • Deep specialization in distributed systems, cloud-native infrastructure, and API design.
  • Experience building secure, high-throughput services such as gateways, proxies, load balancers, or policy engines.
  • Fluency in Go and/or Rust.
  • A track record of identifying systemic gaps in architecture, process, or tooling and driving fixes across team boundaries.
  • A product mindset and experience shaping technical direction rather than only executing against it.
  • Ability to operate autonomously with wide latitude in deciding how to solve problems and whom to involve.
  • A bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Responsibilities

  • Set technical direction for core AI Gateway systems, including the model router, MCP gateway, and control plane, and ensure architectural decisions scale with usage.
  • Identify gaps in infrastructure, process, or tooling and drive solutions from idea through production.
  • Help define, deliver, and implement the Gateway's technical strategy in partnership with engineering leadership.
  • Own complex system problems including dynamic model selection, automatic failover, cost-based routing, and end-to-end policy enforcement.
  • Provide technical expertise in secure credential storage, session summarization, caching, and rate limiting.
  • Build infrastructure to evaluate, replay, and backtest AI agent and large language model behavior.
  • Mentor senior and mid-level engineers and raise the team's technical bar through code, design reviews, and example.
  • Partner with product, design, and leadership to shape the control-plane user interface and product direction.
  • Define a unified abstraction layer across model and tool providers including OpenAI, Anthropic, Google, and AWS Bedrock.
  • Architect secure, scalable identity and credential vaulting for tool and model access.
  • Build infrastructure for real-time and historical analytics of AI agent behavior.
  • Ensure policy enforcement and logging work end-to-end from prompt through tool to response.
  • Set the bar for developer experience through intuitive APIs and first-class observability.
  • Participate in the team's on-call rotation, respond to incidents, debug production issues, and improve system reliability.

Desired Qualifications

  • Prior experience with OpenAI, Anthropic, or similar large language model APIs.
  • Familiarity with retrieval-augmented generation architectures, vector databases, or agent frameworks such as LangChain, AutoGPT, or CrewAI.
  • Experience with policy engines such as Open Policy Agent, observability frameworks such as OpenTelemetry, or API gateways.
  • Understanding of OAuth 2.1, secret management, and cloud identity and access management systems.
  • Experience with Kubernetes, Docker, and microservices architecture.
  • A track record of setting technical standards adopted by other teams or engineers.

About the Company

Docker

Containerization platform for building apps

Docker provides a containerization platform that lets developers build, share, and run applications inside isolated containers. It packages an application and its dependencies into a container image that runs the same on any system that supports Docker. Its main tools include Docker Desktop for local development, Docker Hub for container images, and a command-line interface to manage containers. It uses a freemium pricing model (Personal, Pro, Team, Business) and differentiates itself with its large ecosystem and official image repository to speed up development and deployment.

Company size
1,001-5,000
Company stage
Series C
Total funding
$526M
Headquarters
Palo Alto, CA
Founded
2013

Recently Posted Jobs

Explore more opportunities at Docker

See more jobs