H

HUD

AI agent evaluation platform and benchmarks

Senior Research Manager

AI ResearchConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • Experience leading technical research projects to completion, from an open question to evidence, a decision, and a working result.
  • Experience directly managing and mentoring researchers or research engineers while remaining engaged in technical work.
  • A strong understanding of machine learning and reinforcement learning, including how training objectives, data, and feedback shape model behavior.
  • Experience with agent training data, evaluations, benchmarks, synthetic data, or model evaluation infrastructure.
  • Sound experimental judgment to distinguish a useful training signal from a task or metric that only looks convincing.
  • Strong written communication and the ability to explain methods and findings to researchers, engineers, and external partners.

Responsibilities

  • Set the research direction for data quality, including how HUD measures how training data impacts agent behavior and performance after training, and what makes data valuable.
  • Lead synthetic data efforts to understand how various data types can be efficiently transformed into formats useful for training.
  • Lead research engineers from problem definition through experiments, implementation, and clear conclusions; coach them to strengthen technical judgment and execution.
  • Design and review experiments that connect model behavior and failure modes to data, environment, and reward design.
  • Develop methods for validating and improving training data at scale, including trajectory audits, adversarial checks, and feedback loops that help users improve their data.
  • Partner with research engineers, domain experts, and data vendors to turn research insights into better workflows, tools, and quality standards.
  • Communicate findings and tradeoffs clearly so the team can prioritize work and apply what it learns across research areas.

Desired Qualifications

  • Built scalable data quality systems or validation pipelines for model training.
  • Experience diagnosing reward hacking, grader errors, or other subtle agent failure modes.
  • Experience translating research findings into tools and processes used by others.
  • Early-stage startup experience and strong communication skills for collaboration across teams and time zones.

About the Company

HUD

AI agent evaluation platform and benchmarks

HUD provides an evaluation platform for AI agents that perform computer-use tasks. It offers an interface that connects to HUD evaluation environments, allowing users to run benchmarks across hundreds of environments and thousands of tasks. Users can integrate their agents using various adapters, and interact with the system through an asynchronous API for efficient experiments. The platform then collects telemetry and benchmarking results to inform agent improvements. HUD differentiates itself by concentrating on evaluating and improving knowledge-work agents across a wide range of tasks and environments, rather than just building models, and by enabling scalable, tool-enabled testing via adapters and an asynchronous workflow. The company’s goal is to help organizations and developers assess, compare, and enhance their AI agents so they perform better in real-world knowledge-work scenarios.

Company size
11-50
Company stage
Seed
Total funding
$130K
Headquarters
San Francisco, CA
Founded
2025

Recently Posted Jobs

Explore more opportunities at HUD

See more jobs