P

Prime Intellect

Open frontier-model stack for enterprise RL

Research Engineer - Data and Evaluation

AI ResearchUpdated on Nov 13
Full-time

About the Job

Requirements

  • Strong software engineering skills in Python and experience building data pipelines, evaluation systems, or machine-learning workflows used by others.
  • Hands-on experience working with language models or agents, with an understanding of how training data and evaluation choices affect model behavior.
  • Ability to inspect complex data and model outputs, identify failure modes, and turn an ambiguous quality question into a testable measurement.
  • A record of owning technical projects through experimentation, implementation, and validation while collaborating closely with researchers and engineers.
  • Hands-on experience building technical evaluations or agent environments, with the engineering depth to inspect model traces and debug harness and scoring code.

Responsibilities

  • Design and build challenging evaluations for coding and long-horizon agents.
  • Build executable environments in Verifiers, with realistic task setup and clear success criteria.
  • Develop and test scoring logic and verifiers, investigating reward hacks and misleading measures of progress.
  • Trace failures through model outputs and harness code to distinguish model limitations from problems in the environment or scoring.
  • Build and scale reinforcement-learning data pipelines that generate and curate challenging coding and agentic tasks.
  • Work with reinforcement-learning researchers to test whether changes to tasks and training data improve performance on held-out tasks.

Desired Qualifications

  • Experience with reinforcement-learning or post-training data, including task difficulty selection and analysis of rollout or reward distributions.
  • Contributions to Verifiers, Prime-RL, or comparable environment and evaluation frameworks.
  • Work on model-based grading, long-horizon coding agents, or diagnosing reward exploitation.
  • Relevant experience may include large-scale synthetic data generation, dataset curation, or distributed processing for model training.
  • Relevant experience may include evaluation methodology, executable coding tasks, agent environments, verifiers, or reward design.
  • Relevant experience may include experiment analysis, statistical testing, contamination detection, and reproducible research tooling.

About the Company

Prime Intellect

Open frontier-model stack for enterprise RL

Prime Intellect provides an open, end-to-end superintelligence stack for enterprises to train and continually improve their own AI agents. The platform includes scalable compute, large-scale reinforcement learning, environments, sandboxes, evaluations, and deployment tools, and it ships open frontier models to customers. It covers the full lifecycle from training to deployment plus post-training support, enabling agentic training at scale. Its differentiator is the open, end-to-end stack used across thousands of customers with strong backing from top investors, and its goal is to help organizations build and iteratively improve their AI agents using open frontier models.

Company size
51-200
Company stage
Series A
Total funding
$150.5M
Headquarters
San Francisco, CA
Founded
2024

Recently Posted Jobs

Explore more opportunities at Prime Intellect

See more jobs