A

AfterQuery

LLM app development platform with observability

Research Program Manager

Program ManagementConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USAIn-person

About the Job

Requirements

  • Requires 1–3+ years in research operations, technical program management, or a similar role in a fast-moving research or engineering organization.
  • Requires a technical background in computer science, machine learning, statistics, or a related field, with working Python and SQL.
  • Requires experience evaluating a product, an agent, or a foundation model, including reading results tables and assessing whether a number is trustworthy.
  • Requires written and verbal communication skills sufficient to distill technical results into actionable summaries for partners, executives, and academic collaborators, and to write clear documents.
  • Requires a track record of running cross-functional programs end to end, including managing external partners, contractors, or collaborators.
  • Requires comfort with ambiguity, building processes from scratch, and using judgment to keep processes light so researchers stay effective.

Responsibilities

  • Serve as the point of contact for collaborations with academic and industry partners, including fellowships, grants, and joint research projects, and run each from scoping through delivery.
  • Coordinate benchmark releases end to end, from launch checklist through publication, and keep releases on cadence.
  • Maintain AfterQuery's leaderboards and benchmark sites, and verify every result before it goes to a community leaderboard or publication.
  • Track progress across all active research projects, anticipate resourcing gaps and technical bottlenecks, unblock projects that fall behind, and escalate with a clear recommendation when a timeline or scope needs to change.
  • Coordinate the expert pool for upcoming benchmarks by forecasting expert needs, managing capacity, and keeping contributor logistics running.
  • Write the program's external and internal communications, including launch briefs, partner updates, planning documents, and status memos that turn technical detail into clear, actionable summaries.
  • Run operations behind research programs, including public benchmark releases, collaborations with academic and industry partners, and the research roadmap.
  • Read results tables, run evaluations, sanity-check numbers before they go public, and turn loosely specified research needs into engineering and operational roadmaps.

Desired Qualifications

  • A graduate degree or research experience in a technical field.
  • Experience contributing to a benchmark, dataset, or evaluation project in any role.
  • Familiarity with public leaderboards or open research communities.
  • Exposure to programs with external partners, such as fellowships, grants, or academic collaborations.
  • Exposure to expert or annotator pools, or other human-data work.
  • Comfort with lightweight dashboards or building small tools with AI assistants.

About the Company

AfterQuery

LLM app development platform with observability

AfterQuery provides a data platform for developers to build, evaluate, and monitor LLM-powered applications. It connects to enterprise data sources like Notion, Slack, and databases and prepares data for use in Retrieval-Augmented Generation (RAG). It offers evaluation tools to test models before deployment and observability to monitor performance, costs, and issues such as hallucinations and prompt drift in production. Its end-to-end, data-centric workflow targets production-ready AI solutions for enterprises, backed by YC.

Company size
201-500
Company stage
Series A
Total funding
$30.1M
Headquarters
San Francisco, CA
Founded
2024

Recently Posted Jobs

Explore more opportunities at AfterQuery

See more jobs