A

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Software Engineer - Staff Cloud Security Engineer

Security Engineering & ArchitecturePosted on Nov 13
Full-timeSan Francisco, CA, USANo salary listed

About the Job

Requirements

  • At least 6 years of security or infrastructure engineering experience, including at least 3 years focused on cloud security in production.
  • Deep, current, hands-on experience securing AWS, GCP, and Cloudflare, including work performed recently at keyboard level.
  • Prior ownership of cloud security, including making and defending decisions about IAM and secrets.
  • Strong experience with workload identity, OIDC, workload identity federation, IAM design, and secret managers.
  • Experience rotating production credentials, such as database or API credentials, without causing an outage.
  • Experience writing cloud detections, responding to alerts, and investigating cloud events using audit logs.
  • Experience with Terraform or a similar infrastructure-as-code tool and at least one automation language, such as Python, Go, or TypeScript.
  • Experience securing build pipelines and CI/CD, such as GitHub Actions and deployment credentials.
  • Hands-on use of AI tools for building, scripting, and investigation, with judgment about their appropriate use in security workflows.
  • Ability to assess risk and distinguish real exposures from theoretical ones.
  • Ability to improve existing systems and documentation while maintaining work progress.

Responsibilities

  • Design the cloud identity model for workloads, including least-privilege IAM, workload identity federation, and short-lived credentials.
  • Own machine secrets and credential rotation end to end, ensuring each credential has an owner, is stored in a central secret manager, and has a tested rotation path.
  • Set access rules for environments and services, including outbound connection limits, environment separation, and boundaries around agent runtimes.
  • Build detections and alerts for high-risk changes to cloud infrastructure and build systems, and document actionable response steps.
  • Integrate cloud security checks into infrastructure code and build pipelines.
  • Secure the machine credentials and permissions used by source control, build pipelines, and cloud deployments.
  • Assess technical access requested by vendors and managed platforms before onboarding.
  • Lead technical response to cloud infrastructure security events and coordinate recovery with affected teams.
  • Make cloud security posture visible to engineering and leadership and improve it continuously.
  • Participate in the security on-call rotation.

Desired Qualifications

  • Experience securing serverless platforms such as Vercel.
  • Experience securing infrastructure for AI agent runtimes or sandboxed code execution.
  • Kubernetes security, Postgres access controls, or data platform security such as Databricks.
  • Experience leading incident response in a cloud environment.
  • Compliance evidence work for cloud controls, including SOC 2 or ISO 27001.
  • Experience as an early security hire at a fast-growing startup.
  • Experience working with data protection requirements in logs and access controls.

About the Company

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Arena Intelligence provides a crowdsourced, open platform for evaluating large language models (LLMs) by running head-to-head comparisons of two anonymous models on user prompts. Users vote on the better response and, after voting, model identities are revealed; the results generate large-scale preference data used to compute Elo ratings published on a public leaderboard. Originating from LMSYS and UC Berkeley’s Sky Computing Lab as Chatbot Arena, the project evolved into a company and rebranded to Arena, with $100 million seed funding in 2024 to scale the platform. The underlying framework, FastChat, remains open source, including the user interface and serving backend. Arena differentiates itself by providing a transparent, real-world benchmarking method that aggregates community-driven judgments and offers analytics to AI labs and enterprises, aiming to guide model development with observable performance data.

Company size
51-200
Company stage
Series B
Total funding
$450M
Headquarters
San Francisco, CA
Founded
2025

Recently Posted Jobs

Explore more opportunities at Arena Intelligence

See more jobs