A

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Member of Product Staff

Product ManagementConfirmed live in the last 24 hours
Full-timeCalifornia, USANo salary listedIn-person

About the Job

Requirements

  • 5–10+ years of product management experience on consumer, developer, or enterprise products, ideally including 0-to-1 work at an early-stage company.
  • A track record of repeated, attributable impact across multiple projects and roles or companies, with quantifiable outcomes such as revenue, engagement, or efficiency and sustained impact after handoff.
  • Product judgment, taste, and autonomous ownership, including identifying and validating opportunities, building buy-in, shipping, and achieving results.
  • Experience making consequential project decisions, including ending important projects when it benefits the product.
  • Ability to communicate clearly and persuasively and build buy-in across engineering, research, design, and leadership.
  • Technical depth to work directly with engineers and researchers.
  • Conviction about AI evaluation and Arena's mission, with the ability to articulate the importance of the domain and potential product direction.
  • Comfort working with ambiguity and shifting scope in a startup environment.

Responsibilities

  • Own product areas end to end, from identifying opportunities and building conviction to shipping and driving measurable outcomes.
  • Ship, iterate, measure results, course-correct, and drive sustained impact against company goals.
  • Work with machine learning researchers to turn research into products, creating methods, models, and data for the people who need them.
  • Develop a deep understanding of users, including consumers, labs, enterprises, and researchers, and how they use Arena.
  • Monitor user feedback, usage data, and analytics to identify trends, prioritize initiatives, and refine shipped products.
  • Write product requirements documents, specifications, and narratives that align engineering, design, research, and leadership, creating clarity amid ambiguity.
  • Make high-stakes product decisions under uncertainty and be accountable for outcomes.
  • Raise the product quality bar, introduce practices others adopt, unblock teammates, and uphold craftsmanship.
  • For Enterprise & API: Partner directly with customers to integrate models, run evaluations, and deliver high-quality results; translate feedback into product and engineering initiatives; and own packaging and monetization so new products are instrumented, billable, and correctly represented in company systems.
  • For Evaluation: Partner with research to develop metrics, methodologies, and evaluation protocols; disentangle performance factors such as style, reasoning, and robustness; and protect the scientific integrity and transparency of the Arena leaderboard.
  • For Growth: Run experiments and partner with design and marketing engineering to turn insights about high-expectation users into a compelling product.

Desired Qualifications

  • Production experience with AI or large language model systems, including inference pipelines, evaluation workflows, model integration, or AI-powered product features.
  • Experience with developer- or enterprise-facing products and fluency in consumption- or usage-based revenue models.
  • Comfort working with real-world usage data and designing metrics beyond standard benchmarks, including experiments conducted with statistical rigor.
  • Experience growing a consumer or developer product at scale, with hands-on ownership of experimentation and analytics.
  • Experience measurably raising a team's bar by introducing practices, tools, or standards that others adopted.

About the Company

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Arena Intelligence provides a crowdsourced, open platform for evaluating large language models (LLMs) by running head-to-head comparisons of two anonymous models on user prompts. Users vote on the better response and, after voting, model identities are revealed; the results generate large-scale preference data used to compute Elo ratings published on a public leaderboard. Originating from LMSYS and UC Berkeley’s Sky Computing Lab as Chatbot Arena, the project evolved into a company and rebranded to Arena, with $100 million seed funding in 2024 to scale the platform. The underlying framework, FastChat, remains open source, including the user interface and serving backend. Arena differentiates itself by providing a transparent, real-world benchmarking method that aggregates community-driven judgments and offers analytics to AI labs and enterprises, aiming to guide model development with observable performance data.

Company size
51-200
Company stage
Series B
Total funding
$450M
Headquarters
San Francisco, CA
Founded
2025

Recently Posted Jobs

Explore more opportunities at Arena Intelligence

See more jobs