Founding Engineer
Raydar
Founding Engineer
Raydar
AI-powered recruiting agency for tech roles
Raydar · San Francisco, United States · Full-time · Posted 2026-11-10
Salary: USD 150,000–300,000
Workplace: on_site
About the company
Our client is an artificial intelligence company.
The role
Raydar is recruiting for this role on behalf of our client. Design and build test environments, evaluation harnesses and labeled datasets that measure how AI systems behave across multi-step tasks. The work combines reviewing system logs, spotting recurring errors and turning them into reusable test material. The role suits an engineer who wants to own problems from diagnosis through delivery.
What you'll do
- Construct simulated software environments that mimic the tools and workflows automated systems use.
- Build grading logic and evaluation systems that hold up when systems look for shortcuts.
- Create benchmarks whose scores track real-world results.
- Review interaction logs to find and measure previously unrecorded failure patterns.
- Turn findings into reusable datasets and test suites for internal improvement and external partners.
- Support data and training agreements alongside the founders while refining the evaluation tooling.
What we're looking for
- One to seven years of relevant experience.
- Hands-on work with training data, post-training, benchmarks or evaluation for machine learning models.
- Judgment to tell which failures or data points are meaningful and valuable.
- A track record of building or contributing to benchmarks, datasets or environments used to train or test models.
- Skill in replicating common business software well enough that automated agents can interact with it reliably.
- Understanding of grader design, evaluation pitfalls and the ways systems game metrics.
- Prior work at a data vendor is relevant if it produced training material or model assessments, not expert labeling alone.
- A concrete example of building a benchmark, dataset, environment or evaluation system.
- Commercial awareness of what data and environments are worth.
- Ability to explain technical decisions and trade-offs in an interview.
Bonus points
- Found a real flaw in a widely used benchmark.
- Caught a model gaming a grader and fixed the grader.
- Built environments or benchmarks a leading AI lab used for training or evaluation.
- Startup founding experience or participation in a selective fellowship.
- High-level competitive achievement in mathematics, programming, debate or games.
- Understanding of how benchmarks and datasets support commercial model evaluation.
Compensation and benefits
- Base salary: USD 150,000 to 300,000
- Equity
- Health, dental and vision coverage
- Retirement plan
- Flexible time off
Location and work model
- San Francisco, CA, United States
- On-site (in office)
- Full-time
Powered by Workable
Raydar is an AI-powered talent acquisition firm that serves tech-focused organizations on a contingency basis, charging client companies only after a successful hire. It uses proprietary AI to identify a large pool of candidates and conducts a short 15-minute AI-driven interview (recorded and transcribed) before human recruiters review materials, provide interview prep, and assist with offers and negotiations. What sets it apart is the blend of automated sourcing and screening with hands-on human oversight and tailored client support. Its goal is to help organizations hire more efficiently by speeding up sourcing while maintaining thorough evaluation and negotiation assistance.
Explore more opportunities at Raydar