A

Artificial Analysis

Independent AI model and API benchmarks

Member of Technical Staff - Inference

Applied Machine LearningConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USA

About the Job

Requirements

  • At least 3 years of professional experience, including at least 2 years at or working closely with inference providers, neoclouds, or teams serving models in production.
  • Strong analytical and critical thinking skills.
  • Proficiency in Python and data analysis.
  • Hands-on familiarity with the model serving stack, such as vLLM, SGLang, or TensorRT-LLM, and inference APIs across providers.
  • Fluency in inference performance metrics and economics, including tokens per second, time to first token, throughput versus latency trade-offs, and cost per token.
  • Genuine, demonstrable interest and knowledge of Frontier AI, including informed opinions about the direction of AI.

Responsibilities

  • Own performance and price coverage of serverless API inference across the provider ecosystem, from frontier labs to specialist providers.
  • Drive new benchmarking dimensions, including cached pricing, endpoint accuracy, and agentic performance, while keeping measurement aligned with industry deployment practices.
  • Work directly with inference providers and neoclouds to benchmark their endpoints, resolve methodology questions, and shape how the market measures serving performance.
  • Produce industry analysis of inference performance and economics, including throughput, time to first token, and price-performance frontiers.
  • Shape the roadmap of the inference benchmarking platform together with engineers and the pillar lead.
  • Use cutting-edge AI tools in an AI-native workflow to increase leverage in AI benchmarking.
  • Manage relationships with teams at leading AI labs and major enterprises as a trusted, independent voice.
  • Evaluate major models across capabilities as they are released.

About the Company

Artificial Analysis

Independent AI model and API benchmarks

Independent benchmarking and analysis of AI models and API hosting providers, with benchmarks and leaderboards across intelligence, coding agents, image/speech/video, and inference metrics to help teams choose the right model and provider. It publishes data-driven reports like Intelligence Index releases, capability indexes, and trend analyses, packaging benchmarks as insights and data products for AI buyers. Unlike others, it does not host or sell models, focusing on unbiased evaluation and transparent metrics. The goal is to help AI buyers understand the landscape and select AI models and API providers that fit their use case.

Company size
51-200
Company stage
Seed
Total funding
$250K
Headquarters
Newark, DE
Founded
2024

Recently Posted Jobs

Explore more opportunities at Artificial Analysis

See more jobs