Applied Researcher
Raydar
Applied Researcher
Raydar
AI-powered recruiting agency for tech roles
Raydar · New York, United States · Full-time · Posted 2026-11-11
Salary: USD 160,000–200,000
Workplace: on_site
About the company
Our client is an AI company.
The role
Raydar is recruiting for this role on behalf of our client. Design and run evaluations that test how well AI models handle demanding technical work, then turn the findings into training signals and next steps. The job spans task construction, scoring, reinforcement learning environments and repeatable experiment pipelines, carried out with subject-matter specialists.
What you'll do
- Set up task suites that mirror genuine investigator needs, complete with documented methods, reference scores and data withheld from training.
- Turn domain data and specialist procedures into runnable tasks with defined inputs, permitted tools, target results and verification steps.
- Write scoring criteria and automated verifiers, and probe them for vagueness and loopholes a model could exploit.
- Mine error patterns for the training tasks most worth pursuing, and check whether gains hold on unseen problems.
- Stand up repeatable systems that generate tasks, run models, score outputs and summarize results, with logs of inputs, configurations and task revisions.
- Review what models produce and how they invoke tools, then classify each miss as a model shortfall or a flaw in the task, its framing or its scoring.
- Partner with domain specialists so their expertise shapes task construction and checking, and agree on what a successful answer looks like.
- Report on gains, open questions and the next experiment or engineering step to take.
What we're looking for
- Track record in building test suites, scoring methods or data sourcing for model evaluation.
- Hands-on background designing benchmarks and assessing AI systems, ideally from a firm focused on AI data or evaluation.
- Enough machine learning background to judge what a model can and cannot do.
- Comfort with Python and SQL.
Bonus points
- Past lab work in a life sciences, pharmaceutical or materials setting.
- Previous experience in a senior research, applied ML, science lead or project lead role.
- A degree or academic background in biology, biomedical or chemical engineering, physics or a related field, or equivalent biotech or pharma work experience.
- Familiarity with reward schemes that can be checked automatically, agents that call tools, or reinforcement learning across several steps.
- Past assessment of analytical or investigative work performed by models.
- Past creation of tooling that generates tasks automatically, audits quality or coordinates expert labeling.
Compensation and benefits
- Base salary: USD 160,000 to 200,000 per year
- Equity
- Health coverage
Location and work model
- New York, NY, United States
- On-site, 5 days in-office per week
- Full-time
Powered by Workable
Raydar is an AI-powered talent acquisition firm that serves tech-focused organizations on a contingency basis, charging client companies only after a successful hire. It uses proprietary AI to identify a large pool of candidates and conducts a short 15-minute AI-driven interview (recorded and transcribed) before human recruiters review materials, provide interview prep, and assist with offers and negotiations. What sets it apart is the blend of automated sourcing and screening with hands-on human oversight and tailored client support. Its goal is to help organizations hire more efficiently by speeding up sourcing while maintaining thorough evaluation and negotiation assistance.
Explore more opportunities at Raydar