T

Turing

AI-matched, vetted software engineering services

Research Engineering Manager - STEM

Engineering LeadershipPosted on Nov 10
Full-timeUnited StatesIn-person

About the Job

Requirements

  • At least 8 years of hands-on machine learning or research engineering experience, including at least 5 years in industry and at least 3 years directly managing research engineers, machine learning engineers, or researchers, including hiring and performance management.
  • Experience owning production synthetic or curated data pipelines whose outputs drove training decisions, and measuring their quality and downstream impact.
  • Experience designing and shipping large language model evaluations or benchmarks used for model decisions, with understanding of contamination, saturation, judge bias, inter-annotator agreement, variance, and confidence intervals.
  • Ability to plan ablations, choose appropriate baselines, and distinguish real effects from noise.
  • Strong Python skills and production experience with PyTorch or JAX, plus working knowledge of supervised fine-tuning, direct preference optimization, proximal policy optimization or group relative policy optimization, and reinforcement learning with verifiable rewards.
  • Track record of setting technical direction for a team rather than only executing an existing roadmap.
  • Experience with distributed data processing and large evaluation or training jobs, such as Ray, Spark, or Kubernetes, on AWS or Google Cloud Platform.
  • Ability to communicate clearly in writing to scientists and non-specialists and present results to partner-lab researchers.
  • Daily use of large language model coding tools such as Claude Code, Cursor, or Copilot, with critical review of their output.
  • A PhD or Master's degree in artificial intelligence, machine learning, computer science, statistics, or a closely related field; exceptional equivalent research experience is also considered.

Responsibilities

  • Build the research engineering team by defining needed profiles, hiring research engineers, and developing engineers selected for research judgment and engineering rigor.
  • Run the team's operating rhythm, including project selection, planning, experiment reviews, design documents, and clear ownership.
  • Coach engineers on experimental design, code quality, and presenting results; provide regular direct feedback and run fair performance cycles.
  • Define team standards for evaluation methodology, data quality, and reproducibility so every dataset and result can be regenerated from a commit, configuration, and seed.
  • Develop experiment plans with research scientists, including hypotheses, baselines, ablations, and results that would change the team's conclusions.
  • Set the research agenda with research scientists, contribute original hypotheses, and take research prototypes to production scale while preserving their scientific intent.
  • Own synthetic data pipelines for generation, verification, filtering, deduplication, and versioning, combining programmatic generators, large language model-based generation, automated verifiers, and targeted expert review.
  • Evaluate synthetic data by its effect on models through ablations and fine-tuning or reinforcement learning experiments that test which data attributes drive gains.
  • Build data-quality systems using gold sets, large language model judge calibration against expert labels, sampling audits, and agreement metrics.
  • Identify gaps in frontier model capabilities and design evaluations for capabilities not measured by existing benchmarks.
  • Own benchmark methodology, including task taxonomy, difficulty calibration, graders and verifiers, contamination controls, held-out splits, and confidence intervals.
  • Ship deterministic and reproducible evaluation harnesses, including agentic and sandboxed-execution evaluations, that internal teams and partner labs can trust.
  • Determine whether evaluation results are reliable and provide sign-off for partner labs and internal teams.
  • Publish benchmarks, datasets, and findings with research scientists at venues such as NeurIPS, ICML, and ICLR.
  • Write code weekly on high-leverage technical work, including verifier logic, evaluation statistics, and critical pipeline components; spend at least a third of the role on direct technical work.
  • Resolve the team's most difficult debugging and methodology questions, including judge disagreements with experts, evaluation metric changes, and whether observed gains are real or noise.
  • Serve as a technical counterpart to frontier AI lab researchers by scoping requests, setting acceptance criteria, and explaining results and trade-offs.
  • Partner with Strategic Project Leads, delivery, and engineering teams to bring research advances into client work and team platforms.

Desired Qualifications

  • Experience with post-training data, evaluations, or reward modeling at a frontier AI lab or leading industry research group.
  • Experience building or contributing to a public benchmark or dataset used by the community.
  • First-author papers at NeurIPS, ICML, ICLR, ACL, EMNLP, or similar venues.
  • Experience designing reinforcement learning environments and verifiers for agentic tasks such as coding agents, tool use, computer use, or long-horizon workflows.
  • Experience with data valuation or attribution to measure each data slice's contribution to a model.
  • Depth in a STEM domain such as mathematics, physics, chemistry, or biology.
  • Experience managing technical leads or managers, or running several concurrent programs.
  • Experience directing large expert networks through specifications, rubrics, and rapid feedback loops.

About the Company

Turing

AI-matched, vetted software engineering services

Turing uses artificial intelligence to connect businesses with a global pool of over 3 million vetted software engineers from 150 countries for custom application development and on-demand software engineering. Clients specify the required skills, and the AI matches them to top engineers, speeding up hiring and project delivery while reducing recruitment effort. Engineers who pass Turing’s vetting gain access to high-paying remote jobs with a strong chance of repeat opportunities, reflected in a 99% rematch rate. The company differentiates itself by combining a large, pre-vetted talent pool with AI-driven matching to deliver fast, scalable engineering teams for businesses and steady remote work for engineers. Its goal is to accelerate project timelines for clients and provide reliable, well-compensated remote opportunities for engineers.

Company size
5,001-10,000
Company stage
Series E
Total funding
$247M
Headquarters
Palo Alto, CA
Founded
2018

Recently Posted Jobs

Explore more opportunities at Turing

See more jobs