A

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Applied AI Engineer

AI Application EngineeringConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USA$150k - $250k /yrIn-person

About the Job

Requirements

  • At least 4 years of software engineering experience with strong fundamentals in data structures, algorithms, and system design.
  • Production experience with TypeScript and modern web frameworks.
  • Hands-on experience integrating large language model provider APIs, including streaming responses.
  • Comfort working across frontend and backend, and willingness to investigate problems wherever they arise.
  • Familiarity with cloud infrastructure and continuous integration and continuous delivery workflows.
  • Strong problem-solving ability and ability to operate amid ambiguity.
  • Clear, concise communication skills to explain complex technical ideas to engineers and non-technical stakeholders.
  • A customer-first mindset and desire to work closely with multiple stakeholders.

Responsibilities

  • Own 5–15 active model lab relationships as the day-to-day point of contact.
  • Partner with researchers and product teams to integrate models, run evaluations, and deliver high-quality results.
  • Drive consistent delivery across onboarding, data and product releases, co-launches, and resolving unexpected issues quickly.
  • Own a key process improvement or automation, such as the model onboarding flow, customer data insights, or delivery tooling.
  • Translate customer feedback and research direction into concrete product ideas and engineering work.
  • Work across the stack to ship pragmatic solutions that advance the platform for customers.
  • Expand strategic accounts through trust, execution, and technical judgment.

About the Company

Arena Intelligence

Crowdsourced LLM evaluation and comparison platform

Arena Intelligence provides a crowdsourced, open platform for evaluating large language models (LLMs) by running head-to-head comparisons of two anonymous models on user prompts. Users vote on the better response and, after voting, model identities are revealed; the results generate large-scale preference data used to compute Elo ratings published on a public leaderboard. Originating from LMSYS and UC Berkeley’s Sky Computing Lab as Chatbot Arena, the project evolved into a company and rebranded to Arena, with $100 million seed funding in 2024 to scale the platform. The underlying framework, FastChat, remains open source, including the user interface and serving backend. Arena differentiates itself by providing a transparent, real-world benchmarking method that aggregates community-driven judgments and offers analytics to AI labs and enterprises, aiming to guide model development with observable performance data.

Company size
51-200
Company stage
Series B
Total funding
$450M
Headquarters
San Francisco, CA
Founded
2025

Recently Posted Jobs

Explore more opportunities at Arena Intelligence

See more jobs